model_analysis/visualize_decision_trees_tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Visualization of CatBoost decision trees tutorial"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import catboost\n",
    "from catboost import CatBoostRegressor, CatBoostClassifier, Pool"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Load boston dataset from sklearn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_boston"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "boston = load_boston()\n",
    "y = boston['target']\n",
    "X = boston['data']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "pool = catboost.Pool(\n",
    "    data=X, \n",
    "    label=y\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create and fit CatBoost model with trees of depth 2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = CatBoostRegressor(depth=2, verbose=False, iterations=1).fit(X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Currently only symmetric trees can be visualized.\n",
    "\n",
    "Let's consider symmetric decision tree. In such tree only one feature is used to build all splits at each tree level. There are three types of splits: \"FloatFeature\", \"OneHotFeature\" and \"OnlineCtr\". Model without categorical features contains only \"FloatFeature\" splits."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In visualised tree each node represents one split. Since there are three types of splits there are three types of tree nodes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### FloatFeature\n",
    "Let's look at the first tree of our model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.38.0 (20140413.2041)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"394pt\" height=\"218pt\"\n",
       " viewBox=\"0.00 0.00 394.00 218.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 214)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-214 390,-214 390,4 -4,4\"/>\n",
       "<!-- 0 -->\n",
       "<g id=\"node1\" class=\"node\"><title>0</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"200\" cy=\"-192\" rx=\"77.1866\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"200\" y=\"-188.3\" font-family=\"Times,serif\" font-size=\"14.00\">11, value&gt;393.795</text>\n",
       "</g>\n",
       "<!-- 1 -->\n",
       "<g id=\"node2\" class=\"node\"><title>1</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"114\" cy=\"-105\" rx=\"64.189\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"114\" y=\"-101.3\" font-family=\"Times,serif\" font-size=\"14.00\">9, value&gt;279.5</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;1 -->\n",
       "<g id=\"edge1\" class=\"edge\"><title>0&#45;&gt;1</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M183.009,-174.207C170.087,-161.435 152.145,-143.701 137.794,-129.518\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"140.236,-127.009 130.663,-122.469 135.315,-131.988 140.236,-127.009\"/>\n",
       "<text text-anchor=\"middle\" x=\"174\" y=\"-144.8\" font-family=\"Times,serif\" font-size=\"14.00\">Yes</text>\n",
       "</g>\n",
       "<!-- 2 -->\n",
       "<g id=\"node3\" class=\"node\"><title>2</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"261\" cy=\"-105\" rx=\"64.189\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"261\" y=\"-101.3\" font-family=\"Times,serif\" font-size=\"14.00\">9, value&gt;279.5</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;2 -->\n",
       "<g id=\"edge2\" class=\"edge\"><title>0&#45;&gt;2</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M212.052,-174.207C220.908,-161.866 233.088,-144.894 243.082,-130.967\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"245.986,-132.923 248.973,-122.758 240.299,-128.842 245.986,-132.923\"/>\n",
       "<text text-anchor=\"middle\" x=\"243.5\" y=\"-144.8\" font-family=\"Times,serif\" font-size=\"14.00\">No</text>\n",
       "</g>\n",
       "<!-- 3 -->\n",
       "<g id=\"node4\" class=\"node\"><title>3</title>\n",
       "<polygon fill=\"none\" stroke=\"red\" points=\"80,-36 0,-36 0,-0 80,-0 80,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"40\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">val = 0.587</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;3 -->\n",
       "<g id=\"edge3\" class=\"edge\"><title>1&#45;&gt;3</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M99.38,-87.2067C88.6056,-74.8308 73.7756,-57.7963 61.6317,-43.8472\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"64.0489,-41.2934 54.8429,-36.0493 58.7694,-45.8897 64.0489,-41.2934\"/>\n",
       "<text text-anchor=\"middle\" x=\"93\" y=\"-57.8\" font-family=\"Times,serif\" font-size=\"14.00\">Yes</text>\n",
       "</g>\n",
       "<!-- 4 -->\n",
       "<g id=\"node5\" class=\"node\"><title>4</title>\n",
       "<polygon fill=\"none\" stroke=\"red\" points=\"178,-36 98,-36 98,-0 178,-0 178,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"138\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">val = 0.765</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;4 -->\n",
       "<g id=\"edge4\" class=\"edge\"><title>1&#45;&gt;4</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M118.857,-86.799C122.175,-75.0474 126.639,-59.2383 130.421,-45.8421\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"133.801,-46.7503 133.15,-36.1754 127.065,-44.8481 133.801,-46.7503\"/>\n",
       "<text text-anchor=\"middle\" x=\"136.5\" y=\"-57.8\" font-family=\"Times,serif\" font-size=\"14.00\">No</text>\n",
       "</g>\n",
       "<!-- 5 -->\n",
       "<g id=\"node6\" class=\"node\"><title>5</title>\n",
       "<polygon fill=\"none\" stroke=\"red\" points=\"288,-36 208,-36 208,-0 288,-0 288,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"248\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">val = 0.619</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;5 -->\n",
       "<g id=\"edge5\" class=\"edge\"><title>2&#45;&gt;5</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M258.369,-86.799C256.59,-75.1626 254.201,-59.5479 252.166,-46.2368\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"255.599,-45.5313 250.627,-36.1754 248.679,-46.5897 255.599,-45.5313\"/>\n",
       "<text text-anchor=\"middle\" x=\"267\" y=\"-57.8\" font-family=\"Times,serif\" font-size=\"14.00\">Yes</text>\n",
       "</g>\n",
       "<!-- 6 -->\n",
       "<g id=\"node7\" class=\"node\"><title>6</title>\n",
       "<polygon fill=\"none\" stroke=\"red\" points=\"386,-36 306,-36 306,-0 386,-0 386,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"346\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">val = 0.851</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;6 -->\n",
       "<g id=\"edge6\" class=\"edge\"><title>2&#45;&gt;6</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M277.793,-87.2067C290.285,-74.7151 307.523,-57.4775 321.543,-43.457\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"324.354,-45.5953 328.951,-36.0493 319.405,-40.6455 324.354,-45.5953\"/>\n",
       "<text text-anchor=\"middle\" x=\"317.5\" y=\"-57.8\" font-family=\"Times,serif\" font-size=\"14.00\">No</text>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7f69c96a4c90>"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.plot_tree(\n",
    "    tree_idx=0,\n",
    "    # pool=pool, \n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our model doesn't have categorical features, so there are only \"FloatFeature\" nodes in visualised tree.\n",
    "Node corresponding to \"FloatFeature\" split contains feature index and border value, which are used to split objects.\n",
    "\n",
    "In this example, the node of depth 0 shows that objects are splitted by their 0th feature with border value $393.795$. Analogously, nodes of depth 1 split objects by their 2nd feature with border value $279.5$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\"pool\" parameter is optional for models without one hot features. Features are labeled with their external indexes from pool or features names if pool is provided, otherwise internal indexes are used. For semicolon-separated pool with 2 features  \"f1;label;f2\" external feature indexes are 0 and 2, internal indexes are 0 and 1 respectively."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### OneHotFeature\n",
    "\n",
    "We will use `catboost.datasets.titanic` dataset, which contains categorical data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "from catboost.datasets import titanic\n",
    "titanic_df = titanic()\n",
    "\n",
    "X = titanic_df[0].drop('Survived',axis=1)\n",
    "y = titanic_df[0].Survived"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>Braund, Mr. Owen Harris</td>\n",
       "      <td>male</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>A/5 21171</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>Cumings, Mrs. John Bradley (Florence Briggs Th...</td>\n",
       "      <td>female</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17599</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>C85</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>Heikkinen, Miss. Laina</td>\n",
       "      <td>female</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>STON/O2. 3101282</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>Futrelle, Mrs. Jacques Heath (Lily May Peel)</td>\n",
       "      <td>female</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>113803</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>C123</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>Allen, Mr. William Henry</td>\n",
       "      <td>male</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>373450</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Pclass                                               Name  \\\n",
       "0            1       3                            Braund, Mr. Owen Harris   \n",
       "1            2       1  Cumings, Mrs. John Bradley (Florence Briggs Th...   \n",
       "2            3       3                             Heikkinen, Miss. Laina   \n",
       "3            4       1       Futrelle, Mrs. Jacques Heath (Lily May Peel)   \n",
       "4            5       3                           Allen, Mr. William Henry   \n",
       "\n",
       "      Sex   Age  SibSp  Parch            Ticket     Fare Cabin Embarked  \n",
       "0    male  22.0      1      0         A/5 21171   7.2500   NaN        S  \n",
       "1  female  38.0      1      0          PC 17599  71.2833   C85        C  \n",
       "2  female  26.0      0      0  STON/O2. 3101282   7.9250   NaN        S  \n",
       "3  female  35.0      1      0            113803  53.1000  C123        S  \n",
       "4    male  35.0      0      0            373450   8.0500   NaN        S  "
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Processing NaN values in categorical features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "is_cat = (X.dtypes != float)\n",
    "for feature, feat_is_cat in is_cat.to_dict().items():\n",
    "    if feat_is_cat:\n",
    "        X[feature].fillna(\"NAN\", inplace=True)\n",
    "\n",
    "cat_features_index = np.where(is_cat)[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "pool = Pool(X, y, cat_features=cat_features_index, feature_names=list(X.columns))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define and fit CatBoost model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = CatBoostClassifier(\n",
    "    max_depth=2, verbose=False, max_ctr_complexity=1, random_seed=42, iterations=2).fit(pool)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.38.0 (20140413.2041)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"408pt\" height=\"218pt\"\n",
       " viewBox=\"0.00 0.00 408.00 218.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 214)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-214 404,-214 404,4 -4,4\"/>\n",
       "<!-- 0 -->\n",
       "<g id=\"node1\" class=\"node\"><title>0</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"206\" cy=\"-192\" rx=\"149.473\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"206\" y=\"-188.3\" font-family=\"Times,serif\" font-size=\"14.00\">{Pclass} pr_num0 tb0 type4, border=9</text>\n",
       "</g>\n",
       "<!-- 1 -->\n",
       "<g id=\"node2\" class=\"node\"><title>1</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"104\" cy=\"-105\" rx=\"77.1866\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"104\" y=\"-101.3\" font-family=\"Times,serif\" font-size=\"14.00\">Sex, value=female</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;1 -->\n",
       "<g id=\"edge1\" class=\"edge\"><title>0&#45;&gt;1</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M185.359,-173.799C169.951,-160.959 148.73,-143.275 131.824,-129.187\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"133.839,-126.31 123.916,-122.597 129.358,-131.688 133.839,-126.31\"/>\n",
       "<text text-anchor=\"middle\" x=\"173\" y=\"-144.8\" font-family=\"Times,serif\" font-size=\"14.00\">Yes</text>\n",
       "</g>\n",
       "<!-- 2 -->\n",
       "<g id=\"node3\" class=\"node\"><title>2</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"277\" cy=\"-105\" rx=\"77.1866\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"277\" y=\"-101.3\" font-family=\"Times,serif\" font-size=\"14.00\">Sex, value=female</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;2 -->\n",
       "<g id=\"edge2\" class=\"edge\"><title>0&#45;&gt;2</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M220.368,-173.799C230.733,-161.39 244.877,-144.458 256.436,-130.618\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"259.171,-132.804 262.896,-122.885 253.799,-128.316 259.171,-132.804\"/>\n",
       "<text text-anchor=\"middle\" x=\"254.5\" y=\"-144.8\" font-family=\"Times,serif\" font-size=\"14.00\">No</text>\n",
       "</g>\n",
       "<!-- 3 -->\n",
       "<g id=\"node4\" class=\"node\"><title>3</title>\n",
       "<polygon fill=\"none\" stroke=\"red\" points=\"80,-36 0,-36 0,-0 80,-0 80,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"40\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">val = 0.000</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;3 -->\n",
       "<g id=\"edge3\" class=\"edge\"><title>1&#45;&gt;3</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M91.3557,-87.2067C82.1244,-74.9464 69.4508,-58.1143 59.0036,-44.2391\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"61.6483,-41.9328 52.8371,-36.0493 56.0561,-46.1433 61.6483,-41.9328\"/>\n",
       "<text text-anchor=\"middle\" x=\"87\" y=\"-57.8\" font-family=\"Times,serif\" font-size=\"14.00\">Yes</text>\n",
       "</g>\n",
       "<!-- 4 -->\n",
       "<g id=\"node5\" class=\"node\"><title>4</title>\n",
       "<polygon fill=\"none\" stroke=\"red\" points=\"182,-36 98,-36 98,-0 182,-0 182,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"140\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">val = &#45;0.705</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;4 -->\n",
       "<g id=\"edge4\" class=\"edge\"><title>1&#45;&gt;4</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M111.285,-86.799C116.311,-74.9322 123.089,-58.9279 128.798,-45.4488\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"132.049,-46.7486 132.726,-36.1754 125.603,-44.0186 132.049,-46.7486\"/>\n",
       "<text text-anchor=\"middle\" x=\"133.5\" y=\"-57.8\" font-family=\"Times,serif\" font-size=\"14.00\">No</text>\n",
       "</g>\n",
       "<!-- 5 -->\n",
       "<g id=\"node6\" class=\"node\"><title>5</title>\n",
       "<polygon fill=\"none\" stroke=\"red\" points=\"298,-36 218,-36 218,-0 298,-0 298,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"258\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">val = 0.835</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;5 -->\n",
       "<g id=\"edge5\" class=\"edge\"><title>2&#45;&gt;5</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M273.155,-86.799C270.554,-75.1626 267.064,-59.5479 264.088,-46.2368\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"267.436,-45.1711 261.839,-36.1754 260.605,-46.6981 267.436,-45.1711\"/>\n",
       "<text text-anchor=\"middle\" x=\"280\" y=\"-57.8\" font-family=\"Times,serif\" font-size=\"14.00\">Yes</text>\n",
       "</g>\n",
       "<!-- 6 -->\n",
       "<g id=\"node7\" class=\"node\"><title>6</title>\n",
       "<polygon fill=\"none\" stroke=\"red\" points=\"400,-36 316,-36 316,-0 400,-0 400,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"358\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">val = &#45;0.438</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;6 -->\n",
       "<g id=\"edge6\" class=\"edge\"><title>2&#45;&gt;6</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M293.003,-87.2067C304.907,-74.7151 321.333,-57.4775 334.694,-43.457\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"337.388,-45.7032 341.753,-36.0493 332.321,-40.8741 337.388,-45.7032\"/>\n",
       "<text text-anchor=\"middle\" x=\"331.5\" y=\"-57.8\" font-family=\"Times,serif\" font-size=\"14.00\">No</text>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7f69c97e6b10>"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.plot_tree(\n",
    "    tree_idx=0,\n",
    "    pool=pool # \"pool\" is required parameter for trees with one hot features\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first tree contains only one split made by \"OneHotFeature\" `Sex`. This split puts objects with `Sex=female` to the left and other objects to the right."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### OnlineCtr features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look at other trees, which contain \"OnlineCtr\" splits."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.38.0 (20140413.2041)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"408pt\" height=\"218pt\"\n",
       " viewBox=\"0.00 0.00 408.00 218.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 214)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-214 404,-214 404,4 -4,4\"/>\n",
       "<!-- 0 -->\n",
       "<g id=\"node1\" class=\"node\"><title>0</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"206\" cy=\"-192\" rx=\"149.473\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"206\" y=\"-188.3\" font-family=\"Times,serif\" font-size=\"14.00\">{Pclass} pr_num0 tb0 type0, border=6</text>\n",
       "</g>\n",
       "<!-- 1 -->\n",
       "<g id=\"node2\" class=\"node\"><title>1</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"104\" cy=\"-105\" rx=\"77.1866\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"104\" y=\"-101.3\" font-family=\"Times,serif\" font-size=\"14.00\">Sex, value=female</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;1 -->\n",
       "<g id=\"edge1\" class=\"edge\"><title>0&#45;&gt;1</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M185.359,-173.799C169.951,-160.959 148.73,-143.275 131.824,-129.187\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"133.839,-126.31 123.916,-122.597 129.358,-131.688 133.839,-126.31\"/>\n",
       "<text text-anchor=\"middle\" x=\"173\" y=\"-144.8\" font-family=\"Times,serif\" font-size=\"14.00\">Yes</text>\n",
       "</g>\n",
       "<!-- 2 -->\n",
       "<g id=\"node3\" class=\"node\"><title>2</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"277\" cy=\"-105\" rx=\"77.1866\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"277\" y=\"-101.3\" font-family=\"Times,serif\" font-size=\"14.00\">Sex, value=female</text>\n",
       "</g>\n",
       "<!-- 0&#45;&gt;2 -->\n",
       "<g id=\"edge2\" class=\"edge\"><title>0&#45;&gt;2</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M220.368,-173.799C230.733,-161.39 244.877,-144.458 256.436,-130.618\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"259.171,-132.804 262.896,-122.885 253.799,-128.316 259.171,-132.804\"/>\n",
       "<text text-anchor=\"middle\" x=\"254.5\" y=\"-144.8\" font-family=\"Times,serif\" font-size=\"14.00\">No</text>\n",
       "</g>\n",
       "<!-- 3 -->\n",
       "<g id=\"node4\" class=\"node\"><title>3</title>\n",
       "<polygon fill=\"none\" stroke=\"red\" points=\"80,-36 0,-36 0,-0 80,-0 80,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"40\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">val = 0.545</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;3 -->\n",
       "<g id=\"edge3\" class=\"edge\"><title>1&#45;&gt;3</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M91.3557,-87.2067C82.1244,-74.9464 69.4508,-58.1143 59.0036,-44.2391\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"61.6483,-41.9328 52.8371,-36.0493 56.0561,-46.1433 61.6483,-41.9328\"/>\n",
       "<text text-anchor=\"middle\" x=\"87\" y=\"-57.8\" font-family=\"Times,serif\" font-size=\"14.00\">Yes</text>\n",
       "</g>\n",
       "<!-- 4 -->\n",
       "<g id=\"node5\" class=\"node\"><title>4</title>\n",
       "<polygon fill=\"none\" stroke=\"red\" points=\"182,-36 98,-36 98,-0 182,-0 182,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"140\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">val = &#45;0.252</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;4 -->\n",
       "<g id=\"edge4\" class=\"edge\"><title>1&#45;&gt;4</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M111.285,-86.799C116.311,-74.9322 123.089,-58.9279 128.798,-45.4488\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"132.049,-46.7486 132.726,-36.1754 125.603,-44.0186 132.049,-46.7486\"/>\n",
       "<text text-anchor=\"middle\" x=\"133.5\" y=\"-57.8\" font-family=\"Times,serif\" font-size=\"14.00\">No</text>\n",
       "</g>\n",
       "<!-- 5 -->\n",
       "<g id=\"node6\" class=\"node\"><title>5</title>\n",
       "<polygon fill=\"none\" stroke=\"red\" points=\"298,-36 218,-36 218,-0 298,-0 298,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"258\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">val = 0.004</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;5 -->\n",
       "<g id=\"edge5\" class=\"edge\"><title>2&#45;&gt;5</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M273.155,-86.799C270.554,-75.1626 267.064,-59.5479 264.088,-46.2368\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"267.436,-45.1711 261.839,-36.1754 260.605,-46.6981 267.436,-45.1711\"/>\n",
       "<text text-anchor=\"middle\" x=\"280\" y=\"-57.8\" font-family=\"Times,serif\" font-size=\"14.00\">Yes</text>\n",
       "</g>\n",
       "<!-- 6 -->\n",
       "<g id=\"node7\" class=\"node\"><title>6</title>\n",
       "<polygon fill=\"none\" stroke=\"red\" points=\"400,-36 316,-36 316,-0 400,-0 400,-36\"/>\n",
       "<text text-anchor=\"middle\" x=\"358\" y=\"-14.3\" font-family=\"Times,serif\" font-size=\"14.00\">val = &#45;0.419</text>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;6 -->\n",
       "<g id=\"edge6\" class=\"edge\"><title>2&#45;&gt;6</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M293.003,-87.2067C304.907,-74.7151 321.333,-57.4775 334.694,-43.457\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"337.388,-45.7032 341.753,-36.0493 332.321,-40.8741 337.388,-45.7032\"/>\n",
       "<text text-anchor=\"middle\" x=\"331.5\" y=\"-57.8\" font-family=\"Times,serif\" font-size=\"14.00\">No</text>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7f69c97e6c50>"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.plot_tree(tree_idx=1, pool=pool)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The node of depth 0 corresponds to \"OnlineCtr\" split. This split is made by one feature `Pclass`."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}