recommendations/deep-recommender-ncf.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "serial-tiffany",
   "metadata": {},
   "source": [
    "# Deep Recommender: Neural Collaborative Filtering\n",
    "\n",
    "This is a prototype of a hybrid recommender that combines behavioral data (rating matrix) with content data (natural language descriptions of the items).  \n",
    "\n",
    "### Use Case\n",
    "Similar to the basic user/item-based collaborative filtering, we assume a fixed number of users and items and a collection of user-item pairs with known interaction scores (e.g. user ratings). We also assume additional unstructured data such as natural langage descriptions and images to be available for the items. Our goal is to estimate the scores for the user-item pairs where we do not have the data yet. \n",
    "\n",
    "### Prototype: Approach and Data\n",
    "We start with building a basic neural network that replicate the matrix factorization algorithm (classic collaborative filtering). We next extend this basic network to incorporate item features (embeddings) extracted using a pre-trained NLP model. We use `MovieLens 20M` dataset - see `datasets.md` for details.\n",
    "\n",
    "### Usage and Productization\n",
    "This prototype can be used to explore various model architectures and gains achieved by including user and/or item features. The feature embeddings can be computed using an external (frozen) network or jointly optimized with the main factorization network.   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "regular-choir",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Keras version 2.6.0\n",
      "Tensorflow version 2.6.0\n"
     ]
    }
   ],
   "source": [
    "#\n",
    "# Imports and settings\n",
    "#\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import warnings\n",
    "from tabulate import tabulate\n",
    "\n",
    "import tensorflow as tf\n",
    "from tensorflow.keras.models import Model\n",
    "from tensorflow.keras.layers import Input, Embedding, Dense, Dot, Flatten, Concatenate, Add\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "from sentence_transformers import SentenceTransformer\n",
    "\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "plt.rcParams.update({'pdf.fonttype': 'truetype'})\n",
    "warnings.simplefilter(\"ignore\")\n",
    "\n",
    "print(\"Keras version \" + tf.keras.__version__)\n",
    "print(\"Tensorflow version \" + tf.__version__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "persistent-travel",
   "metadata": {},
   "outputs": [],
   "source": [
    "#\n",
    "# Helper functions\n",
    "#\n",
    "def print_df(df, col_width = 10, rows = 10, max_cols = 10):\n",
    "    def short_srt(x):\n",
    "        return x if len(x) < col_width else x[:col_width-3] + \"...\"\n",
    "    df_short = df.head(rows).applymap(lambda x: short_srt(str(x)))\n",
    "    \n",
    "    if len(df_short.columns) > max_cols:\n",
    "        df_short = df_short.iloc[:, 0:max_cols-1]\n",
    "        df_short['...'] = '...'\n",
    "    \n",
    "    print(tabulate(df_short, headers='keys', tablefmt='psql'))\n",
    "    print(f'{len(df)} rows x {len(df.columns)} columns')\n",
    "    \n",
    "def plot_keras_training_hitory(history):\n",
    "    plt.plot(history.history['loss']);\n",
    "    plt.plot(history.history['val_loss']);\n",
    "    plt.xlabel('Training epoch')\n",
    "    plt.ylabel('Loss')\n",
    "    plt.legend(['train', 'validation'], loc='upper left')\n",
    "    plt.grid(True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "excessive-camera",
   "metadata": {},
   "source": [
    "# Load Data\n",
    "\n",
    "Load movie ratings and tags from the MovieLens 10M dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "particular-detroit",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+----+-----------+--------------------------------+--------------------------------+--------------------------------+\n",
      "|    |   item_id | title                          | genres                         | tag                            |\n",
      "|----+-----------+--------------------------------+--------------------------------+--------------------------------|\n",
      "|  0 |         0 | Toy Story (1995)               | Adventure|Animation|Childre... | watched computer animation ... |\n",
      "|  1 |         1 | Jumanji (1995)                 | Adventure|Children|Fantasy     | time travel adapted from:bo... |\n",
      "|  2 |         2 | Grumpier Old Men (1995)        | Comedy|Romance                 | old people that is actually... |\n",
      "|  4 |         3 | Father of the Bride Part II... | Comedy                         | diane keaton family sequel ... |\n",
      "|  5 |         4 | Heat (1995)                    | Action|Crime|Thriller          | realistic action al pacino ... |\n",
      "|  6 |         5 | Sabrina (1995)                 | Comedy|Romance                 | greg kinnear romance remade... |\n",
      "|  9 |         6 | GoldenEye (1995)               | Action|Adventure|Thriller      | 007 (series) assassin kille... |\n",
      "| 10 |         7 | American President, The (19... | Comedy|Drama|Romance           | president president us pres... |\n",
      "| 13 |         8 | Nixon (1995)                   | Drama                          | biopic character based on r... |\n",
      "| 14 |         9 | Cutthroat Island (1995)        | Action|Adventure|Romance       | pirates treasure action adv... |\n",
      "+----+-----------+--------------------------------+--------------------------------+--------------------------------+\n",
      "4744 rows x 4 columns\n",
      "+----+-----------+-----------+----------+-------------+\n",
      "|    |   user_id |   item_id |   rating | timestamp   |\n",
      "|----+-----------+-----------+----------+-------------|\n",
      "|  0 |         0 |         1 |        4 | 1112486...  |\n",
      "|  1 |         0 |        21 |        4 | 1112484...  |\n",
      "|  2 |         0 |        23 |        4 | 1112484...  |\n",
      "|  3 |         0 |        31 |        4 | 1112484...  |\n",
      "|  4 |         0 |        33 |        4 | 1112484...  |\n",
      "|  5 |         0 |        52 |        4 | 1094785...  |\n",
      "|  6 |         0 |        62 |        4 | 1094785...  |\n",
      "|  7 |         0 |        95 |        4 | 1112485...  |\n",
      "|  8 |         0 |       108 |        4 | 1112484...  |\n",
      "|  9 |         0 |       110 |        4 | 1112484...  |\n",
      "+----+-----------+-----------+----------+-------------+\n",
      "15324463 rows x 4 columns\n",
      "Max user id: 78407, max item id: 4743\n"
     ]
    }
   ],
   "source": [
    "#\n",
    "# Raw data\n",
    "# \n",
    "base_data_path = 'ml-20m'\n",
    "items = pd.read_csv(base_data_path + '/movies.csv', sep=',', skiprows=1, \n",
    "                     names=['item_id', 'title', 'genres'])\n",
    "ratings = pd.read_csv(base_data_path + '/ratings.csv', sep=',', skiprows=1,\n",
    "                      names=['user_id', 'item_id', 'rating', 'timestamp'])\n",
    "tags = pd.read_csv(base_data_path + '/tags.csv', sep=',', skiprows=1,\n",
    "                      names=['user_id', 'item_id', 'tag', 'timestamp'])\n",
    "\n",
    "#\n",
    "# Concatenate tags\n",
    "#\n",
    "tags_flat = tags[['item_id', 'tag']]\\\n",
    "    .groupby(by=['item_id'])['tag']\\\n",
    "    .apply(list)\\\n",
    "    .apply(lambda x: ' '.join(list(map(str, x))))\\\n",
    "    .apply(str.lower)\n",
    "items = items.join(tags_flat, on='item_id')\n",
    "\n",
    "#\n",
    "# Preprocessing\n",
    "#\n",
    "\n",
    "# Remove movies with short tag lines\n",
    "items = items[items['tag'].str.len() > 200]\n",
    "ratings = ratings[ratings['item_id'].isin(items['item_id'].unique())] \n",
    "# Remove users with less than 50 ratings\n",
    "ratings = ratings[ratings.groupby(by='user_id')['user_id'].transform('size') > 50] \n",
    "ratings['rating'] = ratings['rating'].round(0).astype(int)\n",
    "\n",
    "# Remap user and item IDs to sequental integers \n",
    "n_users, n_items = ratings['user_id'].nunique(), ratings['item_id'].nunique()\n",
    "item_id_lookup = { old_id:new_id for (old_id, new_id) in zip(items['item_id'].values, np.arange(n_items)) }\n",
    "items['item_id'] = items['item_id'].apply(lambda x: item_id_lookup[x])\n",
    "ratings['item_id'] = ratings['item_id'].apply(lambda x: item_id_lookup[x])\n",
    "\n",
    "user_id_lookup = { old_id:new_id for (old_id, new_id) in zip(ratings['user_id'].unique(), np.arange(n_users)) }\n",
    "ratings['user_id'] = ratings['user_id'].apply(lambda x: user_id_lookup[x])\n",
    "\n",
    "print_df(items, col_width = 30)\n",
    "print_df(ratings)\n",
    "\n",
    "#\n",
    "# Calculate statistics and limits\n",
    "#\n",
    "user_ids, item_ids = ratings['user_id'].unique(), ratings['item_id'].unique()\n",
    "max_user_id, max_item_id = max(ratings['user_id']), max(ratings['item_id'])\n",
    "print(f'Max user id: {max_user_id}, max item id: {max_item_id}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "seven-client",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjgAAAI4CAYAAABndZP2AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAABCoklEQVR4nO3de5hlVXnv++9PWsRNo4BoH0ITG5XoJu7YQgdxazyNJgjECIlEcRvFS+RkRxONkgS3ieJtRxM00UdjNhGOYIztXQlBsYPUZmvk1twvIi1ibMLlKAg2EhV9zx9zFL26qOquqq6qtWrW9/M866k5x7yN8a5Vq94aY15SVUiSJPXJg4ZdAUmSpLlmgiNJknrHBEeSJPWOCY4kSeodExxJktQ7JjiSJKl3pp3gJNkpyWVJzmrz+yW5MMnGJB9PsnMrf0ib39iWrxrYxxta+fVJnj1Qfngr25jkxDlsnyRJWoKWzWDd1wDXAQ9r8+8C/rqq1iX5O+AVwAfbzzur6nFJjm3rvSDJAcCxwC8CPwf8S5JfaPv6APBrwCbg4iRnVtW126rMXnvtVatWrZpB9bd2zz33sOuuu856+74xHlsYiy2MxdaMxxbGYmvGY4uFjsWGDRu+W1WPfMCCqtruC1gJnAs8EzgLCPBdYFlb/lTgnDZ9DvDUNr2srRfgDcAbBvZ5Ttvu/m1b+VbrTfU66KCDakecd955O7R93xiPLYzFFsZia8ZjC2OxNeOxxULHArikJskTptuD8zfAnwC7tflHAN+vqvva/CZgnza9D/Cdljzdl+Sutv4+wAUD+xzc5jsTyp8yWSWSHA8cD7BixQrGxsamWf0H2rx58w5t3zfGYwtjsYWx2Jrx2MJYbM14bDEqsdhugpPkOcDtVbUhydp5r9E2VNUpwCkAa9asqbVrZ1+dsbExdmT7vjEeWxiLLYzF1ozHFsZia8Zji1GJxXR6cJ4GPDfJkcAudOfgvBfYPcmy1ouzEri5rX8zsC+wKcky4OHA9wbKxw1uM1X5cCTTW8/neEmSNJK2exVVVb2hqlZW1Sq6k4S/XFUvAs4DjmmrHQd8vk2f2eZpy7/cxsjOBI5tV1ntB+wPXARcDOzfrsrauR3jzDlpnSRJWpJmchXVRH8KrEvyduAy4NRWfirwkSQbgTvoEhaq6poknwCuBe4DXlVVPwVI8mq6k453Ak6rqmt2oF6SJGmJm1GCU1VjwFibvhE4eJJ1/gP47Sm2fwfwjknKzwbOnkldJEmSpuKdjCVJUu+Y4EiSpN4xwZEkSb1jgiNJknrHBEeSJPWOCY4kSeodExxJktQ7JjiSJKl3THAkSVLvmOBIkqTeMcGRJEm9Y4IjSZJ6xwRHkiT1jgmOJEnqHRMcSZLUOyY4kiSpd0xwJElS75jgSJKk3jHBkSRJvWOCI0mSescER5Ik9Y4JjiRJ6h0THEmS1DsmOJIkqXdMcCRJUu+Y4EiSpN4xwZEkSb1jgiNJknrHBEeSJPWOCY4kSeodExxJktQ7JjiSJKl3THAkSVLvmOBIkqTeMcGRJEm9s90EJ8m+Sc5Lcm2Sa5K8ppXvmWR9khvazz1aeZK8L8nGJFcmOXBgX8e19W9IctxA+UFJrmrbvC9J5qOxkiRpaZhOD859wOur6gDgEOBVSQ4ATgTOrar9gXPbPMARwP7tdTzwQegSIuDNwFOAg4E3jydFbZ1XDmx3+I43TZIkLVXbTXCq6paqurRN/wC4DtgHOAo4va12OnB0mz4KOKM6FwC7J9kbeDawvqruqKo7gfXA4W3Zw6rqgqoq4IyBfUmSJM1Yupximisnq4DzgScC/1ZVu7fyAHdW1e5JzgLeWVVfacvOBf4UWAvsUlVvb+V/DtwLjLX1f7WV/wrwp1X1nEmOfzxdrxArVqw4aN26dTNvcbN582aWL18++cING6a3k4MOmvXxR80247HEGIstjMXWjMcWxmJrxmOLhY7FoYceuqGq1kwsXzbdHSRZDnwaeG1V3T14mkxVVZLpZ0qzVFWnAKcArFmzptauXTvrfY2NjTHl9oceOt0Kzfr4o2ab8VhijMUWxmJrxmMLY7E147HFqMRiWldRJXkwXXLz0ar6TCu+rQ0v0X7e3spvBvYd2HxlK9tW+cpJyiVJkmZlOldRBTgVuK6q3jOw6Exg/Eqo44DPD5S/pF1NdQhwV1XdApwDHJZkj3Zy8WHAOW3Z3UkOacd6ycC+JEmSZmw6Q1RPA14MXJXk8lb2P4B3Ap9I8grg28Dz27KzgSOBjcAPgZcBVNUdSd4GXNzWe2tV3dGmfx/4MPBQ4AvtJUmSNCvbTXDaycJT3ZfmWZOsX8CrptjXacBpk5RfQnfisiRJ0g7zTsaSJKl3THAkSVLvmOBIkqTeMcGRJEm9Y4IjSZJ6xwRHkiT1zrQf1aBJZKqr5wf06HEOkiQtFvbgSJKk3jHBkSRJvWOCI0mSescER5Ik9Y4JjiRJ6h0THEmS1DsmOJIkqXdMcCRJUu+Y4EiSpN4xwZEkSb1jgiNJknrHBEeSJPWOCY4kSeodExxJktQ7y4Zdgd5Ltr9O1fzXQ5KkJcQeHEmS1DsmOJIkqXdMcCRJUu+Y4EiSpN4xwZEkSb3jVVSjYDpXWoFXW0mSNE324EiSpN4xwZEkSb1jgiNJknrHc3AWE++KLEnStNiDI0mSescenL6xl0eSJHtwJElS/4xMgpPk8CTXJ9mY5MRh16fXkge+Nmx4YJkkSYvUSAxRJdkJ+ADwa8Am4OIkZ1bVtcOt2RI3l0mOw2KSpAU0EgkOcDCwsapuBEiyDjgKMMHpi1HsETLpkqTeGpUEZx/gOwPzm4CnTFwpyfHA8W12c5Lrd+CYewHf3YHt++WEE5ZePKZOupZeLKZmLLZmPLYwFlszHlssdCwePVnhqCQ401JVpwCnzMW+klxSVWvmYl99YDy2MBZbGIutGY8tjMXWjMcWoxKLUTnJ+GZg34H5la1MkiRpxkYlwbkY2D/Jfkl2Bo4FzhxynSRJ0iI1EkNUVXVfklcD5wA7AadV1TXzfNg5GerqEeOxhbHYwlhszXhsYSy2Zjy2GIlYpLySRJIk9cyoDFFJkiTNGRMcSZLUO0sywVmKj4VIclOSq5JcnuSSVrZnkvVJbmg/92jlSfK+Fp8rkxw43NrvuCSnJbk9ydUDZTNuf5Lj2vo3JDluGG3ZUVPE4qQkN7fPx+VJjhxY9oYWi+uTPHugfNH/HiXZN8l5Sa5Nck2S17TyJffZ2EYslupnY5ckFyW5osXjLa18vyQXtrZ9vF0YQ5KHtPmNbfmqgX1NGqfFYhux+HCSbw18Nla38tH4PamqJfWiO4n5m8BjgJ2BK4ADhl2vBWj3TcBeE8r+EjixTZ8IvKtNHwl8AQhwCHDhsOs/B+1/BnAgcPVs2w/sCdzYfu7RpvcYdtvmKBYnASdMsu4B7XfkIcB+7Xdnp778HgF7Awe26d2Ab7Q2L7nPxjZisVQ/GwGWt+kHAxe29/wTwLGt/O+A/96mfx/4uzZ9LPDxbcVp2O2bo1h8GDhmkvVH4vdkKfbg3P9YiKr6MTD+WIil6Cjg9DZ9OnD0QPkZ1bkA2D3J3kOo35ypqvOBOyYUz7T9zwbWV9UdVXUnsB44fN4rP8emiMVUjgLWVdWPqupbwEa636Fe/B5V1S1VdWmb/gFwHd2d1ZfcZ2MbsZhK3z8bVVWb2+yD26uAZwKfauUTPxvjn5lPAc9KEqaO06KxjVhMZSR+T5ZigjPZYyG29UvcFwV8KcmGdI+8AFhRVbe06VuBFW16qcRopu3ve1xe3bqTTxsfkmEJxaINKTyZ7r/TJf3ZmBALWKKfjSQ7JbkcuJ3uj/E3ge9X1X1tlcG23d/utvwu4BH0JB4TY1FV45+Nd7TPxl8neUgrG4nPxlJMcJaqp1fVgcARwKuSPGNwYXX9h0v2ngFLvf3AB4HHAquBW4B3D7U2CyzJcuDTwGur6u7BZUvtszFJLJbsZ6OqflpVq+nurn8w8ITh1mh4JsYiyROBN9DF5Jfphp3+dHg1fKClmOAsycdCVNXN7eftwGfpfllvGx96aj9vb6svlRjNtP29jUtV3da+wH4G/D1butB7H4skD6b7g/7RqvpMK16Sn43JYrGUPxvjqur7wHnAU+mGW8ZvkjvYtvvb3ZY/HPgePYvHQCwOb8OaVVU/Av5fRuyzsRQTnCX3WIgkuybZbXwaOAy4mq7d42exHwd8vk2fCbyknQl/CHDXQHd9n8y0/ecAhyXZo3XTH9bKFr0J51j9Jt3nA7pYHNuuENkP2B+4iJ78HrVzJE4Frquq9wwsWnKfjalisYQ/G49Msnubfijwa3TnJZ0HHNNWm/jZGP/MHAN8ufX+TRWnRWOKWHx94J+A0J2LNPjZGP7vyXydvTzKL7ozvL9BN576xmHXZwHa+xi6s/ivAK4ZbzPd+PC5wA3AvwB7tvIAH2jxuQpYM+w2zEEMPkbXvf4TunHfV8ym/cDL6U4S3Ai8bNjtmsNYfKS19Uq6L6e9B9Z/Y4vF9cARA+WL/vcIeDrd8NOVwOXtdeRS/GxsIxZL9bPxS8Blrd1XA29q5Y+hS1A2Ap8EHtLKd2nzG9vyx2wvTovltY1YfLl9Nq4G/oEtV1qNxO+Jj2qQJEm9sxSHqCRJUs+Z4EiSpN4xwZEkSb1jgiNJknrHBEeSJPWOCY4kSeodExxpCUvy0ySXJ7k6yT+N38xrG+uvTnLkwPxzk5w47xVdhJKcPVk8k5yU5IQhVElaUkxwpKXt3qpaXVVPpHvC+Ku2s/5qupu4AVBVZ1bVO+exfrPW7qI6r99x2zpGVR1Z3W3tJQ2BCY6kcV+jPdk3ycFJvpbksiT/muTx7bb7bwVe0Hp9XpDkpUne37b5cJL3tfVvTHJMK39Qkr9N8vUk61vPxviydya5tj2N+OSJFWq9HR9pdbkhySsHlv1xkovbtm9pZauSXJ/kDLq7q+47YX83JdmrTa9JMtam/+/Wpstbm3eb7TGmONYbk3wjyVeAx8/q3ZE0I8u2v4qkvkuyE/AsumcRAXwd+JWqui/JrwL/s6qel+RNdLddf3Xb7qUTdrU33S3/n0B3W/9PAb8FrAIOAB5F9zyf05I8gu7ZRk+oqtrG8NgvAYcAuwKXJfln4Il0z/Q5mO628GcmeQbwb638uKq6YAYhOAF4VVV9Nd3TtP8jyWFzcYwkB9E9j2k13XfupcCGGdRN0iyY4EhL20OTXE7Xc3MdsL6VPxw4Pcn+dM8nevA09/e56p46fW2SFa3s6cAnW/mtSc5r5XcB/wGcmuQs4Kwp9vn5qroXuLdte3Db52F0z8cBWE6XdPwb8O0ZJjcAXwXek+SjwGeqalNLcObiGL8CfLaqfgiQZNE9eFJajByikpa2e6tqNfBoul6K8XNw3gac187N+Q26BwlOx48GprOtFavqPrpk5VPAc4AvTrXqJPMB/qKdP7S6qh5XVeO9T/ds47D3seV77/42tfOIfhd4KPDVJE/YgWNIGgEmOJJovQt/CLw+yTK6Hpyb2+KXDqz6A2C3Ge7+q8Dz2rk4K4C1AG0o6OFVdTbwR8CTptj+qCS7tCGttcDFwDnAy9s+SLJPkkdNoy43AQe16eeNFyZ5bFVdVVXvavt/wg4cY6LzgaOTPLSd2/Mbs9iHpBlyiEoSAFV1WZIrgRcCf0k3RPVnwD8PrHYecGIb1vqLae7603Tn91wLfIfuHJS76BKlzyfZha635HVTbH9lO+5ewNuq6t+Bf0/yn4GvJQHYDPwO8NPt1OUtdENibwPGBspfm+RQ4GfANcAXqupHszzGVqrq0iQfB64AbqdLoCTNs1RN7P2VpLmVZHlVbW69MBcBT6uqW6ex3UnA5qp6wBVWkrQt9uBIWghntaukdqbrhdluciNJO8IeHEnaAUkuBB4yofjFVXXVMOojqWOCI0mSeserqCRJUu+Y4EiSpN4xwZEkSb1jgiNJknrHBEeSJPWOCY4kSeodExxJktQ7JjiSJKl3THAkSVLvmOBIkqTeMcGRJEm9Y4IjSZJ6xwRHkiT1jgmOJEnqHRMcSZLUOyY4kiSpd0xwJElS75jgSJKk3jHBkSRJvbNs2BWYrb322qtWrVo1L/u+55572HXXXedl33og472wjPfCM+YLy3gvrGHHe8OGDd+tqkdOLF+0Cc6qVau45JJL5mXfY2NjrF27dl72rQcy3gvLeC88Y76wjPfCGna8k3x7snKHqCRJUu+Y4EiSpN4xwZnMhg2QbP8lSZJGkgmOJEnqHRMcSZLUOyY4kiSpd0xwJElS75jgSJKk3jHBkSRJvWOCI0mSescER5Ik9Y4JjiRJ6p1ZJzhJ9k1yXpJrk1yT5DWtfM8k65Pc0H7u0cqT5H1JNia5MsmBA/s6rq1/Q5LjdrxZkiRpKduRHpz7gNdX1QHAIcCrkhwAnAicW1X7A+e2eYAjgP3b63jgg9AlRMCbgacABwNvHk+KJEmSZmPWCU5V3VJVl7bpHwDXAfsARwGnt9VOB45u00cBZ1TnAmD3JHsDzwbWV9UdVXUnsB44fLb1kiRJWjYXO0myCngycCGwoqpuaYtuBVa06X2A7wxstqmVTVU+2XGOp+v9YcWKFYyNjc1F9R9g88qVjJ188vZXnKfjLzWbN2+et/dSD2S8F54xX1jGe2GNarx3OMFJshz4NPDaqro7A0/ZrqpKUjt6jIH9nQKcArBmzZpau3btXO16K2PvfjdrTzhhOhWal+MvNWNjY8zXe6kHMt4Lz5gvLOO9sEY13jt0FVWSB9MlNx+tqs+04tva0BPt5+2t/GZg34HNV7ayqcolSZJmZUeuogpwKnBdVb1nYNGZwPiVUMcBnx8of0m7muoQ4K42lHUOcFiSPdrJxYe1MkmSpFnZkSGqpwEvBq5Kcnkr+x/AO4FPJHkF8G3g+W3Z2cCRwEbgh8DLAKrqjiRvAy5u6721qu7YgXpJkqQlbtYJTlV9BcgUi581yfoFvGqKfZ0GnDbbukiSJA3yTsaSJKl3THAkSVLvmOBIkqTeMcGRJEm9Y4IjSZJ6xwRHkiT1jgmOJEnqHRMcSZLUOyY4kiSpd0xwJElS75jgSJKk3jHBkSRJvWOCI0mSescER5Ik9Y4JjiRJ6h0THEmS1DsmOJIkqXdMcCRJUu+Y4EiSpN4xwZEkSb1jgiNJknrHBEeSJPWOCY4kSeodExxJktQ7JjiSJKl3THAkSVLvmOBIkqTe2aEEJ8lpSW5PcvVA2Z5J1ie5of3co5UnyfuSbExyZZIDB7Y5rq1/Q5LjdqROkiRJO9qD82Hg8AllJwLnVtX+wLltHuAIYP/2Oh74IHQJEfBm4CnAwcCbx5MiSZKk2dihBKeqzgfumFB8FHB6mz4dOHqg/IzqXADsnmRv4NnA+qq6o6ruBNbzwKRJkiRp2ubjHJwVVXVLm74VWNGm9wG+M7DeplY2VbkkSdKsLJvPnVdVJam52l+S4+mGt1ixYgVjY2NzteutbF65krGTT97+ivN0/KVm8+bN8/Ze6oGM98Iz5gvLeC+sUY33fCQ4tyXZu6puaUNQt7fym4F9B9Zb2cpuBtZOKB+bbMdVdQpwCsCaNWtq7dq1k622w8be/W7WnnDC9lesOcvdlrSxsTHm673UAxnvhWfMF5bxXlijGu/5GKI6Exi/Euo44PMD5S9pV1MdAtzVhrLOAQ5Lskc7ufiwViZJkjQrO9SDk+RjdL0veyXZRHc11DuBTyR5BfBt4Plt9bOBI4GNwA+BlwFU1R1J3gZc3NZ7a1VNPHFZkiRp2nYowamqF06x6FmTrFvAq6bYz2nAaTtSF0mSpHHeyViSJPWOCY4kSeodExxJktQ7JjiSJKl3THAkSVLvmOBIkqTemddHNfResv11vNuxJEkLzh4cSZLUOyY4kiSpd0xwJElS75jgSJKk3jHBkSRJvWOCI0mSescER5Ik9Y4JjiRJ6h0THEmS1DsmOJIkqXdMcCRJUu/4LKr55vOqJElacPbgSJKk3jHBkSRJveMQ1SiYzjAWOJQlSdI02YMjSZJ6xwRHkiT1jkNUi4lXZEmSNC324EiSpN6xB6dvpnvC8nTYGyRJWqRMcDQ1h8QkSYvUyAxRJTk8yfVJNiY5cdj10TQl03tJkrSARiLBSbIT8AHgCOAA4IVJDhhurTSntpX8bNhgoiRJmlMjkeAABwMbq+rGqvoxsA44ash10iiabiK0mF/zHYeZJJQmlpIWqdQInEOR5Bjg8Kr63Tb/YuApVfXqCesdDxzfZh8PXD9PVdoL+O487VsPZLwXlvFeeMZ8YRnvhTXseD+6qh45sXBRnWRcVacAp8z3cZJcUlVr5vs46hjvhWW8F54xX1jGe2GNarxHZYjqZmDfgfmVrUySJGnGRiXBuRjYP8l+SXYGjgXOHHKdJEnSIjUSQ1RVdV+SVwPnADsBp1XVNUOs0rwPg2krxnthGe+FZ8wXlvFeWCMZ75E4yViSJGkujcoQlSRJ0pwxwZEkSb1jgjOBj4yYvSSnJbk9ydUDZXsmWZ/khvZzj1aeJO9rcb4yyYED2xzX1r8hyXED5Qcluapt875k6d6BLsm+Sc5Lcm2Sa5K8ppUb73mSZJckFyW5osX8La18vyQXtjh9vF0oQZKHtPmNbfmqgX29oZVfn+TZA+V+/wxIslOSy5Kc1eaN9TxKclP7nb88ySWtbPF+p1SVr/aiO8H5m8BjgJ2BK4ADhl2vxfICngEcCFw9UPaXwIlt+kTgXW36SOALQIBDgAtb+Z7Aje3nHm16j7bsorZu2rZHDLvNQ4z13sCBbXo34Bt0jzkx3vMX8wDL2/SDgQtbfD4BHNvK/w74723694G/a9PHAh9v0we075aHAPu175yd/P6ZNOavA/4ROKvNG+v5jfdNwF4Tyhbtd4o9OFvzkRE7oKrOB+6YUHwUcHqbPh04eqD8jOpcAOyeZG/g2cD6qrqjqu4E1gOHt2UPq6oLqvtNOWNgX0tOVd1SVZe26R8A1wH7YLznTYvd5jb74PYq4JnAp1r5xJiPvxefAp7V/mM9ClhXVT+qqm8BG+m+e/z+GZBkJfDrwIfafDDWw7Bov1NMcLa2D/CdgflNrUyzt6KqbmnTtwIr2vRUsd5W+aZJype81h3/ZLoeBeM9j9qQyeXA7XRf3N8Evl9V97VVBuN0f2zb8ruARzDz92Kp+hvgT4CftflHYKznWwFfSrIh3aORYBF/p4zEfXC0NFRVJfG+BHMoyXLg08Brq+ruwSFt4z33quqnwOokuwOfBZ4w3Br1U5LnALdX1YYka4dcnaXk6VV1c5JHAeuTfH1w4WL7TrEHZ2s+MmLu3da6Jmk/b2/lU8V6W+UrJylfspI8mC65+WhVfaYVG+8FUFXfB84DnkrXNT/+z+JgnO6PbVv+cOB7zPy9WIqeBjw3yU10w0fPBN6LsZ5XVXVz+3k7XQJ/MIv4O8UEZ2s+MmLunQmMn0V/HPD5gfKXtDPxDwHuat2g5wCHJdmjna1/GHBOW3Z3kkPa2PpLBva15LQYnApcV1XvGVhkvOdJkke2nhuSPBT4Nbpzn84DjmmrTYz5+HtxDPDldu7BmcCx7cqf/YD96U6+9Punqao3VNXKqlpFF4cvV9WLMNbzJsmuSXYbn6b7LriaxfydMp9nMC/GF92Z4d+gG1t/47Drs5hewMeAW4Cf0I2vvoJuHPxc4AbgX4A927oBPtDifBWwZmA/L6c7GXAj8LKB8jV0v3DfBN5PuxP3UnwBT6cbL78SuLy9jjTe8xrzXwIuazG/GnhTK38M3R/NjcAngYe08l3a/Ma2/DED+3pji+v1DFxJ4vfPpHFfy5arqIz1/MX5MXRXk10BXDMek8X8neKjGiRJUu84RCVJknrHBEeSJPWOCY4kSeodExxJktQ7JjiSJKl3THAkSVLvmOBIS1iSnya5PMnVSf5p/EZ221h/dZIjB+afm+TEea/oHEnyoSQHtOn/Mc/HWpPkfVMsuynJXvN5fGmp8z440hKWZHNVLW/TpwPfqKp3bGP9l9Ld0OvVC1TFWWt3S01V/WyK5fe3faG1RxCsqarvDuP40lJgD46kcV+jPd03ycFJvpbksiT/muTx7Zb2bwVe0Hp9XpDkpUne37b5cJL3tfVvTHJMK39Qkr9N8vUk65OcPbDsnUmuTXJlkpMnVijJSUk+0upyQ5JXDiz74yQXt23f0spWJbk+yRl0d0zdd8L+xlrPyjuBh7Z2fLQt+50kF7Wy/5Vkp1a+OclfJbkmyb+02Iy1Nj53qmAmWZvkrDb9iCRfavv4EN1dYCXNIxMcSbQ/5s9iy/N4vg78SlU9GXgT8D+r6sdt+uNVtbqqPj7Jrvame4zEc4B3trLfAlYBBwAvpntAJUkeAfwm8ItV9UvA26eo3i/RPWzxqcCbkvxcksPonit0MLAaOCjJM9r6+wN/W1W/WFXfnmyHVXUicG9rx4uS/GfgBcDTqmo18FPgRW31XemebfSLwA9aPX+t1f2tU9R5ojcDX2n7+Czw89PcTtIsLdv+KpJ67KFJLqfrubkOWN/KHw6cnmR/umdePXia+/tcGxK6NsmKVvZ04JOt/NYk57Xyu4D/AE5tPR1nTbHPz1fVvcC9bduD2z4Po3s2FMByusTm34BvV9UF06zvuGcBBwEXdyNbPJQtT03+MfDFNn0V8KOq+kmSq+gSt+l4Bl2iR1X9c5I7Z1g/STNkD460tN3beiweTTds8qpW/jbgvKp6IvAbdA8znI4fDUxvcximqu6jS1Y+Rdfj88WpVp1kPsBftB6Y1VX1uKo6tS2/Z5p1HRTg9IH9Pb6qTmrLflJbTlb8Ga2NLWHzn0RpRJngSKKqfgj8IfD6JMvoenBubotfOrDqD4DdZrj7rwLPa+firKB7OjRJlgMPr6qzgT8CnjTF9kcl2aUNaa0FLgbOAV7e9kGSfZI8aob1+kmS8Z6pc4FjxveRZM8kj57h/rblfOC/tX0fAewxh/uWNAn/+5AEQFVdluRK4IXAX9INUf0Z8M8Dq50HnNiGtf5imrv+NN0Q0LXAd4BL6YandgM+n2QXuh6U102x/ZXtuHsBb6uqfwf+vZ0387U2pLQZ+B26c2em6xTgyiSXtvNw/gz4UpIHAT+h682a9ByeWXgL8LEk1wD/SjeUJmkeeZm4pHmXZHlVbW69MBfRncx76zS2OwnYXFUPuMJKkrbFHhxJC+GsdhPBnel6Ybab3EjSjrAHR5J2QJJnA++aUPytqvrNYdRHUscER5Ik9Y5XUUmSpN4xwZEkSb1jgiNJknrHBEeSJPWOCY4kSeodExxJktQ7JjiSJKl3THAkSVLvmOBIkqTeWdQJTpLTktye5Opprv/8JNcmuSbJP853/SRJ0nAs6kc1JHkGsBk4o6qeuJ119wc+ATyzqu5M8qiqun0h6ilJkhbWou7BqarzgTsGy5I8NskXk2xI8n+SPKEteiXwgaq6s21rciNJUk8t6gRnCqcAf1BVBwEnAH/byn8B+IUkX01yQZLDh1ZDSZI0r5YNuwJzKcly4L8Cn0wyXvyQ9nMZsD+wFlgJnJ/kv1TV9xe4mpIkaZ71KsGh65H6flWtnmTZJuDCqvoJ8K0k36BLeC5ewPpJkqQF0Kshqqq6my55+W2AdJ7UFn+OrveGJHvRDVndOIRqSpKkebaoE5wkHwO+Bjw+yaYkrwBeBLwiyRXANcBRbfVzgO8luRY4D/jjqvreMOotSZLm16K+TFySJGkyi7oHR5IkaTKL9iTjvfbaq1atWjXsatzvnnvuYddddx12NXrNGM8v4zv/jPH8Mr7zbxRjvGHDhu9W1SMnli/aBGfVqlVccsklw67G/cbGxli7du2wq9Frxnh+Gd/5Z4znl/Gdf6MY4yTfnqzcISpJktQ7JjiSJKl3THAkSaMnmflrw4bZbadeMsGRJEm9Y4IjSZJ6xwRHkiT1jgmOJEnqHRMcSZLUOyY4kiSpd0xwJElS75jgSJKk3jHBkSRJvWOCI0mSescER5Ik9Y4JjiRJ6h0THEmS1DsmOJIkqXdGJsFJsnuSTyX5epLrkjx12HWSJEmL07JhV2DAe4EvVtUxSXYG/tOwKyRJkhankUhwkjwceAbwUoCq+jHw42HWSZIkLV6pqmHXgSSrgVOAa4EnARuA11TVPRPWOx44HmDFihUHrVu3boFrOrXNmzezfPnyYVej14zx/DK+888Yz8CGDTPeZPPKlSzftGnmxzrooJlvs0SN4mf40EMP3VBVayaWj0qCswa4AHhaVV2Y5L3A3VX151Nts2bNmrrkkksWrI7bMzY2xtq1a4ddjV4zxvPL+M4/YzwDyYw3GTv5ZNaecMLMjzUCfwcXi1H8DCeZNMEZlZOMNwGbqurCNv8p4MAh1keSJC1iI5HgVNWtwHeSPL4VPYtuuEqSJGnGRuIk4+YPgI+2K6huBF425PpIkqRFamQSnKq6HHjAGJokSdJMjcQQlSRJ0lwywZEkSb1jgiNJknrHBEeSJPWOCY4kSeodExxJktQ7JjiSJKl3THAkSVLvmOBIkqTeMcGRJEm9Y4IjSZJ6xwRHkiT1jgmOJEnqnZFKcJLslOSyJGcNuy6SJGnxGqkEB3gNcN2wKyFJkha3kUlwkqwEfh340LDrIkmSFreRSXCAvwH+BPjZkOshSZIWuVTVsOtAkucAR1bV7ydZC5xQVc+ZZL3jgeMBVqxYcdC6desWtJ7bsnnzZpYvXz7savSaMZ5fxnf+GeMZ2LBhxptsXrmS5Zs2zfxYBx00822WqFH8DB966KEbqmrNxPJRSXD+AngxcB+wC/Aw4DNV9TtTbbNmzZq65JJLFqiG2zc2NsbatWuHXY1eM8bzy/jOP2M8A8mMNxk7+WTWnnDCzI81An8HF4tR/AwnmTTBGYkhqqp6Q1WtrKpVwLHAl7eV3EiSJG3LSCQ4kiRJc2nZsCswUVWNAWNDroYkSVrE7MGRJEm9Y4IjSZJ6xwRHkiT1jgmOJEnqHRMcSZLUOyY4kjQbycxfGzbMfBtJs2KCI0mSescER5Ik9Y4JjiRJ6h0THEmS1DsmOJIkqXdMcCRJUu+Y4EiSpN4xwZEkSb0zEglOkn2TnJfk2iTXJHnNsOskSVKv9fxmlcuGduSt3Qe8vqouTbIbsCHJ+qq6dtgVkyRJi89I9OBU1S1VdWmb/gFwHbDPcGslSZIWq1TVsOuwlSSrgPOBJ1bV3ROWHQ8cD7BixYqD1q1bt/AVnMLmzZtZvnz5sKvRa8Z4fhnfGdqwYcabbF65kuWbNs1so4MOmvFxemGh4gvGeAZG8TN86KGHbqiqNRPLRyrBSbIc+N/AO6rqM9tad82aNXXJJZcsTMWmYWxsjLVr1w67Gr1mjOeX8Z2hWZxbMHbyyaw94YSZbTRC39ELaqHiC8Z4BkbxM5xk0gRnJIaoAJI8GPg08NHtJTeStmOhTh70adeSRtRIJDhJApwKXFdV7xl2fSRJ0uI2EgkO8DTgxcAzk1zeXkcOu1KSJGlxGonLxKvqK4B93ZIkaU6MSg+OJEnSnDHBkSRJvWOCI0mSescER5Ik9Y4JjiRJ6h0THEmS1DsmOJIkqXdMcCRJUu+Y4EiSpN4xwZEkSb1jgiNJknrHBEeSJPWOCY6GI5n5a8OGmW8jSVqSTHAkSVLvjEyCk+TwJNcn2ZjkxGHXR5IkLV4jkeAk2Qn4AHAEcADwwiQHDLFCCzN84hCKJEnzYiQSHOBgYGNV3VhVPwbWAUcNuU6SJGmRSlUNuw4kOQY4vKp+t82/GHhKVb16wnrHA8e32ccD1y9oRbdtL+C7w65Ezxnj+WV8558xnl/Gd/6NYowfXVWPnFi4bBg1ma2qOgU4Zdj1mEySS6pqzbDr0WfGeH4Z3/lnjOeX8Z1/iynGozJEdTOw78D8ylYmSZI0Y6OS4FwM7J9kvyQ7A8cCZw65TpIkaZEaiSGqqrovyauBc4CdgNOq6pohV2umRnLorGeM8fwyvvPPGM8v4zv/Fk2MR+IkY0mSpLk0KkNUkiRJc8YER5Ik9Y4JzhzwMRPzK8lpSW5PcvWw69JHSfZNcl6Sa5Nck+Q1w65T3yTZJclFSa5oMX7LsOvUR0l2SnJZkrOGXZc+SnJTkquSXJ7kkmHXZ3s8B2cHtcdMfAP4NWAT3RVhL6yqa4dasR5J8gxgM3BGVT1x2PXpmyR7A3tX1aVJdgM2AEf7GZ47SQLsWlWbkzwY+Arwmqq6YMhV65UkrwPWAA+rqucMuz59k+QmYE1VjdqN/iZlD86O8zET86yqzgfuGHY9+qqqbqmqS9v0D4DrgH2GW6t+qc7mNvvg9vK/yzmUZCXw68CHhl0XjQYTnB23D/CdgflN+MdBi1SSVcCTgQuHXJXeacMnlwO3A+uryhjPrb8B/gT42ZDr0WcFfCnJhvbopJFmgiMJgCTLgU8Dr62qu4ddn76pqp9W1Wq6O7UfnMTh1jmS5DnA7VW1Ydh16bmnV9WBwBHAq9rpAyPLBGfH+ZgJLXrtvJBPAx+tqs8Muz59VlXfB84DDh9yVfrkacBz2zki64BnJvmH4Vapf6rq5vbzduCzdKdojCwTnB3nYya0qLUTYE8Frquq9wy7Pn2U5JFJdm/TD6W7KOHrQ61Uj1TVG6pqZVWtovsO/nJV/c6Qq9UrSXZtFyGQZFfgMGCkr2w1wdlBVXUfMP6YieuATyzCx0yMtCQfA74GPD7JpiSvGHadeuZpwIvp/uu9vL2OHHalemZv4LwkV9L9U7S+qryUWYvJCuArSa4ALgL+uaq+OOQ6bZOXiUuSpN6xB0eSJPWOCY4kSeodExxJktQ7JjiSJKl3THAkSVLvmOBIIyLJT9sl2lcn+afx+6ZsY/3Vg5dzJ3luH59mn+ToJAcMzL81ya8OqS43JdlrjvZ1dJI3bWP56rm8XD/JvyTZY672J406ExxpdNxbVavbE9PvAF61nfVXA/f/AayqM6vqnfNYv1lLZ8rvmyQ7bWPzo4H7E5yqelNV/cscVm9Y/gT4220sX83A+zsdSZZtY/FHgN+fyf6kxcwERxpNX6M9tDXJwUm+luSyJP+a5PHtrtlvBV7Qen1ekOSlSd7ftvlwkve19W9Mckwrf1CSv03y9STrk5w9sOydSa5NcmWSkydWKMlJST7S6nJDklcOLPvjJBe3bd/SylYluT7JGXR3PN13wv5uSvKuJJcCv53klW0fVyT5dJL/lOS/As8F/qq187GtbccM7OMtSS5NclWSJ7TyR7b2XZPkQ0m+PbHnJcnvJfmrgfnB+H2uPVDwmskeKtjadvXA/AlJTmrTj03yxbb9/xmv04TtfwH4UVV9t83/duu5uyLJ+VO8vw/4HAzU+8wkXwbOTbJ328d4b+CvtMOeCbxwYl2k3qoqX758jcAL2Nx+7gR8Eji8zT8MWNamfxX4dJt+KfD+ge3vnwc+3PbxILrej42t/Bjg7Fb+fwF3trJHANez5eafu09Sv5OAK4CHAnsB3wF+ju6W7acAafs9C3gGsIruyc6HTNHem4A/GZh/xMD024E/GGjLMQPL7p9v+xhf7/eBD7Xp9wNvaNOH0z0Fea8Jx3/keFza/BfoHiYIsGf7+VC65OwRA8fbq7Xt6oFtTwBOatPnAvu36afQPTZgYttfBrx7YP4qYJ/B2E/y/m7rc7BpoM6vB9448FnabWAfNwzG2ZevPr+21Z0paWE9NMnldD031wHrW/nDgdOT7E/3h/rB09zf56rqZ8C1SVa0sqcDn2zltyY5r5XfBfwHcGqSs+iSlMl8vqruBe5t2x7c9nkYcFlbZzmwP/BvwLer6oJt1PHjA9NPTPJ2YPe2j3Om2c7xh4NuAH6rTT8d+E2AqvpikjsnblRV/1/r3TqE7g//E4CvtsV/mOQ32/S+rT3f215F0j2R/b8Cn0wyXvyQSVbdG/j/Bua/Cnw4yScG2jPRtj4H66vqjjZ9MXBaugeofq6qLh9Y73a6pHS7bZEWO4eopNFxb1WtBh5N1xsyfg7O24Dzqjs35zeAXaa5vx8NTGfKtbj/mWoHA58CngNM9YyZic92qbbvv6ju/KHVVfW4qjq1Lb9nO3UcXP5h4NVV9V+AtzDzdv4UZvxP2zrg+cDzgM9WVSVZS9dD8tSqehJd4jaxLvex9ffn+PIHAd8fiMXqqvrPkxz33sF9VtXvAX9Gl0xtSPKISbbZ1ufg/jhW1fl0PWg30yVNL5lQz3sn2bfUOyY40oipqh8Cfwi8vp00+nC6P1bQDUeM+wGw2wx3/1Xgee1cnBXAWri/5+HhVXU28EfAk6bY/qgku7Q/wGvpegvOAV7e9kGSfZI8aob1gq4tt7SehxcNlM+2nc9v9TkMmOrqoc8CR9Gdm7KulT0cuLOqftjOnzlkku1uAx6V5BFJHkKXFFJVdwPfSvLb7dhJMlksrwMeNz6T5LFVdWFVvYmuZ2dfHtjuqT4HW0nyaOC2qvp74EPAgeN1oRuWvGmqbaU+McGRRlBVXQZcSfeH9y+Bv0hyGVv3UJwHHDB+Euo0d/1puvM1rgX+AbiUbnhqN+CsdE+7/grwuim2v7Id9wLgbVX171X1JeAfga8luYquF2imCQnAnwMX0iUnXx8oXwf8cTu59rHT3NdbgMPaicC/DdxKlzBsparupEs2Hl1VF7XiLwLLklwHvJOurRO3+wndScAX0Q0lDtb3RcAr0j11+Rq6BGqi84EnZ8s41l+1k6SvBv6V7lynie/vVJ+DidYCV7T1XgC8t5UfBFzQeuuk3vNp4tISk2R5VW1uvTAXAU+rqlunsd1JdCdCP+AKq1HTelV+WlX3JXkq8ME2/DcykrwX+KdaoEve2/HOrKpzF+J40rB5krG09JyV7iaCO9P1wmw3uVmEfh74RLp77/wYeOV21h+G/0l3ldVCudrkRkuJPTiSJKl3PAdHkiT1jgmOJEnqHRMcSZLUOyY4kiSpd0xwJElS75jgSJKk3jHBkSRJvWOCI0mSescER5Ik9Y4JjiRJ6h0THEmS1DsmOJIkqXdMcCRJUu+Y4EiSpN4xwZEkSb1jgiNJknrHBEeSJPWOCY4kSeodExxJktQ7JjiSJKl3THAkSVLvLBt2BWZrr732qlWrVs3Lvu+55x523XXXedn3qLGt/bNU2gm2ta9saz/NV1s3bNjw3ap65MTyRZvgrFq1iksuuWRe9j02NsbatWvnZd+jxrb2z1JpJ9jWvrKt/TRfbU3y7cnKHaKSJEm9Y4IjSZJ6xwRnMhs2QLL9lyRJGkkmOJIkqXdMcCRJUu+Y4EiSpN4xwZEkSb1jgiNJknrHBEeSJPWOCY4kSeqd7SY4SU5LcnuSqwfKTkpyc5LL2+vIgWVvSLIxyfVJnj1Qfngr25jkxIHy/ZJc2Mo/nmTnuWygJElaeqbTg/Nh4PBJyv+6qla319kASQ4AjgV+sW3zt0l2SrIT8AHgCOAA4IVtXYB3tX09DrgTeMWONEiSJGm7CU5VnQ/cMc39HQWsq6ofVdW3gI3Awe21sapurKofA+uAo5IEeCbwqbb96cDRM2uCJEnS1lJV218pWQWcVVVPbPMnAS8F7gYuAV5fVXcmeT9wQVX9Q1vvVOALbTeHV9XvtvIXA08BTmrrP66V7wt8Yfw4k9TjeOB4gBUrVhy0bt26mbd4GjbfdhvLN23a/ooHHTQvx19ImzdvZvny5cOuxoJYKm1dKu0E29pXtrWf5quthx566IaqWjOxfNks9/dB4G1AtZ/vBl4+++pNT1WdApwCsGbNmpqvR8yPvfvdrD3hhOlUaF6Ov5Dm6/H1o2iptHWptBNsa1/Z1n5a6LbOKsGpqtvGp5P8PXBWm70Z2Hdg1ZWtjCnKvwfsnmRZVd03YX1JkqRZmdVl4kn2Hpj9TWD8CqszgWOTPCTJfsD+wEXAxcD+7YqpnelORD6zuvGx84Bj2vbHAZ+fTZ0kSZLGbbcHJ8nHgLXAXkk2AW8G1iZZTTdEdRPw/wBU1TVJPgFcC9wHvKqqftr282rgHGAn4LSquqYd4k+BdUneDlwGnDpXjZMkSUvTdhOcqnrhJMVTJiFV9Q7gHZOUnw2cPUn5jXRXWUmSJM0J72QsSZJ6xwRHkiT1jgmOJEnqHRMcSZLUOyY4kiSpd0xwJElS75jgSJKk3jHBkSRJvWOCI0mSescER5Ik9Y4JjiRJ6h0THEmS1DsmOJIkqXdMcCRJUu+Y4EiSpN4xwZEkSb2z3QQnyWlJbk9y9UDZnknWJ7mh/dyjlSfJ+5JsTHJlkgMHtjmurX9DkuMGyg9KclXb5n1JMteNlCRJS8t0enA+DBw+oexE4Nyq2h84t80DHAHs317HAx+ELiEC3gw8BTgYePN4UtTWeeXAdhOPJUmSNCPbTXCq6nzgjgnFRwGnt+nTgaMHys+ozgXA7kn2Bp4NrK+qO6rqTmA9cHhb9rCquqCqCjhjYF+SJEmzsmyW262oqlva9K3Aija9D/CdgfU2tbJtlW+apHxSSY6n6xlixYoVjI2NzbL627Z55UrGTj55+yvO0/EX0ubNm+ctjqNmqbR1qbQTbGtf2dZ+Wui2zjbBuV9VVZKai8pM41inAKcArFmzptauXTsvxxl797tZe8IJ06nQvBx/IY2NjTFfcRw1S6WtS6WdYFv7yrb200K3dbZXUd3WhpdoP29v5TcD+w6st7KVbat85STlkiRJszbbBOdMYPxKqOOAzw+Uv6RdTXUIcFcbyjoHOCzJHu3k4sOAc9qyu5Mc0q6eesnAviRJkmZlu0NUST4GrAX2SrKJ7mqodwKfSPIK4NvA89vqZwNHAhuBHwIvA6iqO5K8Dbi4rffWqho/cfn36a7UeijwhfaSJEmate0mOFX1wikWPWuSdQt41RT7OQ04bZLyS4Anbq8ekiRJ0+WdjCVJUu+Y4EiSpN4xwZEkSb1jgiNJknrHBEeSJPWOCY4kSeodExxJktQ7JjiSJKl3THAkSVLvmOBIkqTeMcGRJEm9Y4IjSZJ6xwRHkiT1jgmOJEnqHRMcSZLUOyY4kiSpd3YowUlyU5Krklye5JJWtmeS9UluaD/3aOVJ8r4kG5NcmeTAgf0c19a/IclxO9YkSZK01M1FD86hVbW6qta0+ROBc6tqf+DcNg9wBLB/ex0PfBC6hAh4M/AU4GDgzeNJkSRJ0mzMxxDVUcDpbfp04OiB8jOqcwGwe5K9gWcD66vqjqq6E1gPHD4P9ZIkSUtEqmr2GyffAu4ECvhfVXVKku9X1e5teYA7q2r3JGcB76yqr7Rl5wJ/CqwFdqmqt7fyPwfuraqTJzne8XS9P6xYseKgdevWzbru27L5tttYvmnT9lc86KB5Of5C2rx5M8uXLx92NRbEUmnrUmkn2Na+sq39NF9tPfTQQzcMjCLdb9kO7vfpVXVzkkcB65N8fXBhVVWS2WdQE1TVKcApAGvWrKm1a9fO1a63Mvbud7P2hBOmU6F5Of5CGhsbY77iOGqWSluXSjvBtvaVbe2nhW7rDg1RVdXN7eftwGfpzqG5rQ090X7e3la/Gdh3YPOVrWyqckmSpFmZdYKTZNcku41PA4cBVwNnAuNXQh0HfL5Nnwm8pF1NdQhwV1XdApwDHJZkj3Zy8WGtTJIkaVZ2ZIhqBfDZ7jQblgH/WFVfTHIx8IkkrwC+DTy/rX82cCSwEfgh8DKAqrojyduAi9t6b62qO3agXpIkaYmbdYJTVTcCT5qk/HvAsyYpL+BVU+zrNOC02dZFkiRpkHcyliRJvWOCI0mSescER5Ik9Y4JjiRJ6h0THEmS1DsmOJIkqXdMcCRJUu+Y4EiSpN4xwZEkSb1jgiNJknrHBEeSJPWOCY4kSeodExxJktQ7s36auIBk++tUzX89JEnSVuzBkSRJvWOCI0mSemdkEpwkhye5PsnGJCcOuz5zJtn+S5IkzamRSHCS7AR8ADgCOAB4YZIDhlsrSZK0WI1EggMcDGysqhur6sfAOuCoIddp4Uynl2e+Xhs22LMkSeqdUbmKah/gOwPzm4CnTFwpyfHA8W12c5Lr56k+ewHfnad9j5YTTpi6rf1LcpbK+7pU2gm2ta9saz/NV1sfPVnhqCQ401JVpwCnzPdxklxSVWvm+zijwLb2z1JpJ9jWvrKt/bTQbR2VIaqbgX0H5le2MkmSpBkblQTnYmD/JPsl2Rk4FjhzyHWSJEmL1EgMUVXVfUleDZwD7AScVlXXDLFK8z4MNkJsa/8slXaCbe0r29pPC9rWlI8SkCRJPTMqQ1SSJElzxgRHkiT1jgnOgL49LiLJvknOS3JtkmuSvKaVn5Tk5iSXt9eRA9u8obX/+iTPHl7tZy7JTUmuam26pJXtmWR9khvazz1aeZK8r7X1yiQHDrf205fk8QPv3eVJ7k7y2r68r0lOS3J7kqsHymb8PiY5rq1/Q5LjhtGW7ZmirX+V5OutPZ9NsnsrX5Xk3oH39+8GtjmoffY3tniM1E2spmjnjD+vi+E7eoq2fnygnTclubyVL9r3FLb5N2Y0fl+ryld3HtJOwDeBxwA7A1cABwy7XjvYpr2BA9v0bsA36B6FcRJwwiTrH9Da/RBgvxaPnYbdjhm09yZgrwllfwmc2KZPBN7Vpo8EvgAEOAS4cNj1n2WbdwJupbvRVS/eV+AZwIHA1bN9H4E9gRvbzz3a9B7Dbts023oYsKxNv2ugrasG15uwn4ta+9PiccSw2zaNds7o87pYvqMna+uE5e8G3rTY39NWx6n+xozE76s9OFv07nERVXVLVV3apn8AXEd31+ipHAWsq6ofVdW3gI10cVnMjgJOb9OnA0cPlJ9RnQuA3ZPsPYT67ahnAd+sqm9vY51F9b5W1fnAHROKZ/o+PhtYX1V3VNWdwHrg8Hmv/AxN1taq+lJV3ddmL6C7L9iUWnsfVlUXVPfX4gy2xGckTPGeTmWqz+ui+I7eVltbL8zzgY9tax+L4T2Fbf6NGYnfVxOcLSZ7XMS2koFFJckq4MnAha3o1a2L8LTx7kMWfwwK+FKSDeke6wGwoqpuadO3Aiva9GJv67hj2frLso/vK8z8fexDmwFeTvcf77j9klyW5H8n+ZVWtg9d+8YtprbO5PPah/f0V4DbquqGgbJevKcT/saMxO+rCc4SkGQ58GngtVV1N/BB4LHAauAWui7TPnh6VR1I91T6VyV5xuDC9p9Qb+6LkO6mmM8FPtmK+vq+bqVv7+NUkrwRuA/4aCu6Bfj5qnoy8DrgH5M8bFj1mwNL4vM6wQvZ+h+SXrynk/yNud8wf19NcLbo5eMikjyY7oP30ar6DEBV3VZVP62qnwF/z5bhikUdg6q6uf28HfgsXbtuGx96aj9vb6sv6rY2RwCXVtVt0N/3tZnp+7io25zkpcBzgBe1PxC0IZvvtekNdOej/AJduwaHsRZFW2fxeV3s7+ky4LeAj4+X9eE9nexvDCPy+2qCs0XvHhfRxntPBa6rqvcMlA+ea/KbwPjZ/mcCxyZ5SJL9gP3pTnQbeUl2TbLb+DTdiZpX07Vp/Iz844DPt+kzgZe0s/oPAe4a6FJdLLb6b7CP7+uAmb6P5wCHJdmjDX0c1spGXpLDgT8BnltVPxwof2SSndr0Y+jexxtbe+9Ockj7nX8JW+IzsmbxeV3s39G/Cny9qu4felrs7+lUf2MYld/XHT1LuU8vujO8v0GXRb9x2PWZg/Y8na5r8Erg8vY6EvgIcFUrPxPYe2CbN7b2X88InrW/jbY+hu6qiiuAa8bfP+ARwLnADcC/AHu28gAfaG29Clgz7DbMsL27At8DHj5Q1ov3lS5puwX4Cd1Y/Ctm8z7Snb+ysb1eNux2zaCtG+nORxj/nf27tu7z2mf7cuBS4DcG9rOGLkH4JvB+2l3qR+U1RTtn/HldDN/Rk7W1lX8Y+L0J6y7a97TVcaq/MSPx++qjGiRJUu84RCVJknrHBEeSJPWOCY4kSeodExxJktQ7JjiSJKl3THAkzViSR2TLE5BvzdZPhd552PWbTJKfS/KpNr06A0+vltQ/XiYuaYckOQnYXFUnD7sug5Isqy0PrZy47KV09+B49cLWStJCsQdH0pxI8sokFye5Ismnk/ynVv7YJBckuSrJ25NsbuV7Jzm/9fpcPfCgwcF93pTkL9u2FyV5XCt/ZDvGxe31tFZ+UpKPJPkq3Y3kBve1qh1nZ+CtwAvasV/Q7oR9WjvGZUmOatu8NMnnkqxvdXl1kte1dS5Isue8BlXSrJngSJorn6mqX66qJwHX0d2tFuC9wHur6r+w9ROS/xtwTlWtBp5EdxfUydzVtn0/8DcD+/zrqvplurvBfmhg/QOAX62qF062s6r6MfAm4ONVtbqqPk5359wvV9XBwKHAX7VHfgA8ke4ZQr8MvAP4YXUPR/wa3S30JY2gZcOugKTeeGKStwO7A8vZ8iyZpwJHt+l/BMaHsi4GTmsP6/tcVV0+xX4/NvDzr9v0rwIHdI/CAeBh6Z5oDHBmVd07w7ofBjw3yQltfhfg59v0eVX1A+AHSe4C/qmVXwX80gyPI2mBmOBImisfBo6uqivaOS5rt7VyVZ2f5BnArwMfTvKeqjpjslUnmX4QcEhV/cfgii3huWcWdQ/wvKq6fsL+ngL8aKDoZwPzP8PvUGlkOUQlaa7sBtzSemReNFB+Ad0wEnRPgAYgyaOB26rq7+mGmA6cYr8vGPj5tTb9JeAPBva1eoZ1/UGr77hzgD9oT0cmyZNnuD9JI8YER9Jc+XPgQuCrwNcHyl8LvC7JlcDjgLta+VrgiiSX0SUv751iv3u0bV8D/FEr+0NgTZIrk1wL/N4M63oe3RDX5UleALwNeDBwZZJr2rykRczLxCXNq3Y11b1VVUmOBV5YVUdNc9ub6C7n/u581lFS/zh+LGm+HQS8vw3/fB94+XCrI2kpsAdHkiT1jufgSJKk3jHBkSRJvWOCI0mSescER5Ik9Y4JjiRJ6p3/H+HutiLYKyc+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x576 with 4 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#\n",
    "# Visualize the main statistics\n",
    "#\n",
    "fig, ax = plt.subplots(4, 1, figsize=(8, 8) )\n",
    "for i, c in enumerate(['user_id', 'item_id']):\n",
    "    sns.distplot(ratings.groupby(by=[c]).count().iloc[:, 0].values, kde=False, ax=ax[i], \n",
    "                 axlabel=f'Ratings per {c}', color='red', hist_kws=dict(alpha=1))\n",
    "    ax[i].grid(True)\n",
    "    \n",
    "rating_frequencies = ratings.groupby(by=['rating']).count().iloc[:, 0]\n",
    "ax[2].bar(rating_frequencies.index, rating_frequencies, color='red', width=.3)\n",
    "ax[2].grid(True)\n",
    "ax[2].set_xlabel('Ratings per rating value (stars)')\n",
    "plt.tight_layout()\n",
    "\n",
    "tag_frequencies = tags.groupby(by=['item_id']).count().iloc[:, 0].values\n",
    "sns.distplot(tag_frequencies, kde=False, ax=ax[3], \n",
    "                 axlabel=f'Tags per item', color='red', hist_kws=dict(alpha=1))\n",
    "ax[3].grid(True)\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "minus-pacific",
   "metadata": {},
   "outputs": [],
   "source": [
    "#\n",
    "# Fix the train-test split\n",
    "#\n",
    "n = ratings.shape[0]\n",
    "train_idx, test_idx, _, _ = train_test_split(np.arange(n), np.arange(n), test_size=0.20, random_state=42)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sophisticated-welding",
   "metadata": {},
   "source": [
    "# Model 1: Neural Collaborative Filtering (Factorization Network)\n",
    "\n",
    "In this section, we train and evaluate a basic network that replicates the standard matrix factorization algorithm."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "sexual-election",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/16\n",
      "47889/47889 [==============================] - 94s 2ms/step - loss: 0.8033 - root_mean_squared_error: 0.8962 - val_loss: 0.6982 - val_root_mean_squared_error: 0.8356\n",
      "Epoch 2/16\n",
      "47889/47889 [==============================] - 94s 2ms/step - loss: 0.6698 - root_mean_squared_error: 0.8184 - val_loss: 0.6734 - val_root_mean_squared_error: 0.8206\n",
      "Epoch 3/16\n",
      "47889/47889 [==============================] - 93s 2ms/step - loss: 0.6491 - root_mean_squared_error: 0.8056 - val_loss: 0.6664 - val_root_mean_squared_error: 0.8164\n",
      "Epoch 4/16\n",
      "47889/47889 [==============================] - 93s 2ms/step - loss: 0.6404 - root_mean_squared_error: 0.8002 - val_loss: 0.6814 - val_root_mean_squared_error: 0.8255\n",
      "Epoch 5/16\n",
      "47889/47889 [==============================] - 94s 2ms/step - loss: 0.6354 - root_mean_squared_error: 0.7971 - val_loss: 0.6820 - val_root_mean_squared_error: 0.8258\n",
      "Epoch 6/16\n",
      "47889/47889 [==============================] - 93s 2ms/step - loss: 0.6323 - root_mean_squared_error: 0.7952 - val_loss: 0.6614 - val_root_mean_squared_error: 0.8132\n",
      "Epoch 7/16\n",
      "47889/47889 [==============================] - 93s 2ms/step - loss: 0.6301 - root_mean_squared_error: 0.7938 - val_loss: 0.6724 - val_root_mean_squared_error: 0.8200\n",
      "Epoch 8/16\n",
      "47889/47889 [==============================] - 93s 2ms/step - loss: 0.6283 - root_mean_squared_error: 0.7926 - val_loss: 0.6613 - val_root_mean_squared_error: 0.8132\n",
      "Epoch 9/16\n",
      "47889/47889 [==============================] - 94s 2ms/step - loss: 0.6269 - root_mean_squared_error: 0.7918 - val_loss: 0.6922 - val_root_mean_squared_error: 0.8320\n",
      "Epoch 10/16\n",
      "47889/47889 [==============================] - 93s 2ms/step - loss: 0.6260 - root_mean_squared_error: 0.7912 - val_loss: 0.6601 - val_root_mean_squared_error: 0.8125\n",
      "Epoch 11/16\n",
      "47889/47889 [==============================] - 94s 2ms/step - loss: 0.6253 - root_mean_squared_error: 0.7907 - val_loss: 0.7001 - val_root_mean_squared_error: 0.8367\n",
      "Epoch 12/16\n",
      "47889/47889 [==============================] - 93s 2ms/step - loss: 0.6246 - root_mean_squared_error: 0.7903 - val_loss: 0.6673 - val_root_mean_squared_error: 0.8169\n",
      "Epoch 13/16\n",
      "47889/47889 [==============================] - 93s 2ms/step - loss: 0.6238 - root_mean_squared_error: 0.7898 - val_loss: 0.6614 - val_root_mean_squared_error: 0.8133\n",
      "Epoch 14/16\n",
      "47889/47889 [==============================] - 94s 2ms/step - loss: 0.6237 - root_mean_squared_error: 0.7897 - val_loss: 0.6603 - val_root_mean_squared_error: 0.8126\n",
      "Epoch 15/16\n",
      "47889/47889 [==============================] - 93s 2ms/step - loss: 0.6230 - root_mean_squared_error: 0.7893 - val_loss: 0.6613 - val_root_mean_squared_error: 0.8132\n",
      "Epoch 16/16\n",
      "47889/47889 [==============================] - 94s 2ms/step - loss: 0.6227 - root_mean_squared_error: 0.7891 - val_loss: 0.6677 - val_root_mean_squared_error: 0.8171\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEGCAYAAABy53LJAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAABBi0lEQVR4nO3deXhU5fXA8e+ZrGQhkATCnqCAyKJAEBdEgyva1n3BWqtWpbVFa7VW7GKp1VZ/tm51qytaUUppVVQUt8QdZRFZZd8Cyr4khOzn98d7E4aQZRJmMpPkfJ5nnpl773vvnAlkTu67iqpijDHGBMoX7gCMMca0LJY4jDHGNIolDmOMMY1iicMYY0yjWOIwxhjTKNHhDqA5pKena1ZWVpPO3bt3L4mJicENKMgiPcZIjw8iP8ZIjw8sxmCItPjmzp27TVU7HXRAVVv9Izs7W5sqNze3yec2l0iPMdLjU438GCM9PlWLMRgiLT5gjtbynWpVVcYYYxrFEocxxphGscRhjDGmUdpE43htysrKyM/Pp7i4uN5yKSkpLF26tJmiappIiDE+Pp4ePXoQExMT1jiMMaHXZhNHfn4+ycnJZGVlISJ1lisoKCA5ObkZI2u8cMeoqmzfvp38/Hx69+4dtjiMMc2jzVZVFRcXk5aWVm/SMIEREdLS0hq8ezPGtA5tNnEAljSCyH6WxrQdbTpxNGRnUSl7SmzaeWOM8WeJox67i8rYUxqaxLFr1y4ee+yxRp939tlns2vXruAHZIwxAQpp4hCRMSKyTERWisiEWo73EpFcEflKRBaIyNl+x273zlsmImcGes1giovxUV7pRkoGW12Jo7y8vN7zZsyYQYcOHYIejzHGBCpkvapEJAp4FDgdyAdmi8h0VV3iV+z3wFRVfVxEBgAzgCzv9VhgINANeE9E+nnnNHTNoImN8qFAWYUSGx3cOvwJEyawatUqhgwZQkxMDPHx8XTs2JFvvvmG5cuXc95557FhwwaKi4v55S9/ybhx4wDIyspizpw5FBYWctZZZ3HiiSfyySef0LNnT1577TXatWsX1DiNMaamUHbHHQGsVNXVACIyBTgX8P+SV6C99zoF2OS9PheYoqolwBoRWeldjwCu2Wh/en0xSzbtOWh/RaVSXFZBfEwUUb7GJY4B3drzxx8MrPP4Pffcw6JFi5g/fz55eXl873vfY9GiRdXdWZ999llSU1PZt28fxxxzDBdeeCFpaWkHXGPFihW8/PLL3H///VxzzTX897//5Uc/+lGj4jTGmMYKZeLoDmzw284Hjq1RZiLwjojcACQCp/mdO6vGud291w1dEwARGQeMA8jIyCAvL++A4ykpKRQUFABQVlpGRUXFQdeoqqCqqKzcvxGgstKy6uvXprCwkMrKSgoKCigqKiI7O5v09PTqc+677z7eeOMNADZs2MD8+fMZMWIEqkphYSGFhYVkZmZy+OGHU1FRwaBBg1i2bFm97xlqxcXFB/2cwX3W2vZHkkiPMdLjA4sxGCI9virhHgB4GTBJVf8uIscD/xKRQcG4sKo+CTwJMHz4cM3JyTng+NKlS6sHzd114ZC6rsGijbtJT46ja0pwq4CSkpLw+XwkJyeTkJBA+/btq+PJy8vj448/5osvviAhIYGcnByioqJITk5GREhKSgKgXbt2JCcnU1BQQEJCAoWFhWEdCBgfH8/QoUMP2p+Xl0fNn3+kifQYIz0+sBiDIdLjqxLKxLER6Om33cPb5+8aYAyAqn4uIvFAegPnNnTNoBERon1CaXll0K9d9YVfm927d9OxY0cSEhL45ptvmDVrVq3ljDEmHELZq2o20FdEeotILK6xe3qNMuuBUwFE5EggHtjqlRsrInEi0hvoC3wZ4DWDKtpHSBJHWloaI0eOZNCgQdx6660HHBszZgzl5eUceeSRTJgwgeOOOy7o72+MMU0VsjsOVS0XkfHATCAKeFZVF4vInbjFQaYDtwBPicivcK0IV3mLhywWkam4Ru9y4BeqWgFQ2zVD9RkAYnywt6wSVQ366OiXXnqp1v1xcXG89dZbtR5bu3YtAOnp6SxatKh6/69//eugxmaMMXUJaRuHqs7AdbH133eH3+slwMg6zr0buDuQa4ZStE+o0ErKK5WYKJtWwxhjbOR4A2K8n1AoqquMMaYlssTRgGhLHMYYcwBLHA2ovuOosMRhjDFgiaNBIkJslI8Su+MwxhjAEkdAYqN9VlVljDEeSxwBiITEUTVafNOmTVx00UW1lsnJyWHOnDn1XufBBx+kqKioetumaTfGNJYljgDERvsor6x0c1aFWbdu3Zg2bVqTz6+ZOGyadmNMY1niCEBclPsxBfOuY8KECTz66KPV2xMnTuSuu+7i1FNPZdiwYQwePJjXXnvtoPPWrl3LoEFuOq99+/YxduxYhg8fzvnnn8++ffuqy11//fUMHz6cgQMH8sc//hGAhx9+mE2bNjF69GhGjx4NuGnat23bBsD999/PoEGDGDRoEA8++GD1+x155JFcd911DBw4kDPOOOOA9zHGtD3hnuQwMrw1Ab5bWOuhdhXliC+Kw0oriInxgS/AXNtlMJx1T52HL730Um666SZ+8YtfADB16lRmzpzJjTfeSPv27dm2bRvHHXcc55xzTp0j1h9//HESEhKYM2cOa9asYdiwYdXH7r77blJTU6moqODUU09lwYIF3Hjjjdx///3k5uaSnp5+wLXmzp3Lc889xxdffIGqcuyxx3LyySfTsWPH6unbn3rqKS655BKbvt2YNs7uOAJQtRRHZRAXAhw6dChbtmxh06ZNfP3113Ts2JEuXbrw29/+lqOOOorTTjuNjRs3snnz5jqv8dFHH1V/gR911FEcddRR1cemTp3KsGHDGDp0KIsXL2bJkvqXLPnkk084//zzSUxMJCkpiQsuuICPP/4YgN69ezNkyBAAsrOzq6c9Mca0TXbHAfXeGewrKCA5OZn1m/bQPj6aHqkJQXvbiy++mGnTpvHdd99x6aWXMnnyZLZu3crcuXOJiYkhKyuL4uLiRl93zZo1/O1vf2P27Nl07NiRq666qknXqRIXF1f9OioqyqqqjGnj7I4jQHHRPkqCPAjw0ksvZcqUKUybNo2LL76Y3bt307lzZ2JiYsjNzWXdunX1nn/SSSdVT5S4aNEiFixYAMCePXtITEwkJSWFzZs3HzBhYl3TuY8aNYpXX32VoqIi9u7dyyuvvMKoUaOC+GmNMa2F3XEEKDbaR2FJeVCvOXDgQAoKCujevTtdu3bl8ssv5wc/+AGDBw9m+PDh9O/fv97zr7/+eq6++urqRvDs7GwAjj76aIYOHUr//v3p2bMnI0fun0dy3LhxjBkzhm7dupGbm1u9f9iwYVx11VWMGOFW6L322msZOnSoVUsZYw5iiSNAsdE+yooqqaxUfI1cf7w+Cxfub5RPT0/n888/r7VcYWEh4HpBVU2n3q5dO6ZMmUKBV53mb9KkSbVe54YbbuCGG26o3vZPDDfffDM333zzAeX93w9s+nZjjFVVBSzOm+3Q5qwyxrR1ljgCFOslDpuzyhjT1rXpxOEWGwxMbAgGAbYmjflZGmNatjabOOLj49m+fXvAX3hRPiHKJ5SWV4Q4spZHVdm+fTvx8fHhDsUY0wxC2jguImOAh3Drgz+tqvfUOP4AMNrbTAA6q2oHERkNPOBXtD8wVlVfFZFJwMnAbu/YVao6v7Gx9ejRg/z8fLZu3VpvueLi4uovxG17itnhE/YkxdV7TnPzjzFc4uPj6dGjR1hjMMY0j5AlDhGJAh4FTgfygdkiMt1bZxwAVf2VX/kbgKHe/lxgiLc/FVgJvON3+VtVtekz/QExMTH07t27wXJ5eXkMHToUgEdfmseC/F189JvRDZzVvPxjNMaYUAtlVdUIYKWqrlbVUmAKcG495S8DXq5l/0XAW6paVMuxZpWZlsDGXfsos55Vxpg2TELVqCkiFwFjVPVab/sK4FhVHV9L2UxgFtBDVStqHPsAuF9V3/C2JwHHAyXA+8AEVS2p5ZrjgHEAGRkZ2VOmTGnS5ygsLKxeC+Pj/DKeWVTKvaPakZEYOc1D/jFGokiPDyI/xkiPDyzGYIi0+EaPHj1XVYcfdEBVQ/LA3Sk87bd9BfBIHWVvA/5Ry/6uwFYgpsY+AeKA54E7GoolOztbmyo3N7f69axV2zTztjc0b9mWJl8vFPxjjESRHp9q5McY6fGpWozBEGnxAXO0lu/UUP7ZvBHo6bfdw9tXm7HUXk11CfCKqpZV7VDVb73PVAI8h6sSaxZZ6YkArNu+t7ne0hhjIk4oE8dsoK+I9BaRWFxymF6zkIj0BzoCtc21cVC7h4h09Z4FOA9YdPBpodE5OY74GB/rtoe9ucUYY8ImZL2qVLVcRMYDM3HdcZ9V1cUicifu9qcqiYwFpni3RdVEJAt3x/JhjUtPFpFOuOqq+cDPQvUZahIRMlMT7Y7DGNOmhXQch6rOAGbU2HdHje2JdZy7Fuhey/5Tghdh42WmJbBmmyUOY0zbFTldg1qIzLQE1u0oojKYywEaY0wLYomjkTLTEiktr+S7PU1fUc8YY1oySxyNlJnmlo61BnJjTFtliaORstKsS64xpm2zxNFIXVPiifYJ63bYHYcxpm2yxNFI0VE+eqYm2B2HMabNssTRBL1SE6yNwxjTZlniaIKsNJc4aoxZNMaYNsESRxNkpiVSWFLOjr2l4Q7FGGOanSWOJqjqkrvWqquMMW2QJY4myPS65K7fYQ3kxpi2xxJHE/RMbYcIrN1mdxzGmLbHEkcTxEVH0S2lnXXJNca0SZY4mqhXaoINAjTGtEmWOJooK93Gchhj2iZLHE3UKzWRHXtL2VNc1nBhY4xpRSxxNFGW1yV3vd11GGPaGEscTdTLplc3xrRRIU0cIjJGRJaJyEoRmVDL8QdEZL73WC4iu/yOVfgdm+63v7eIfOFd898iEhvKz1CXqrEca61nlTGmjQlZ4hCRKOBR4CxgAHCZiAzwL6Oqv1LVIao6BPgH8D+/w/uqjqnqOX777wUeUNU+wE7gmlB9hvokxUWTnhRnVVXGmDYnlHccI4CVqrpaVUuBKcC59ZS/DHi5vguKiACnANO8Xc8D5x16qE2TmZZgdxzGmDYnlImjO7DBbzvf23cQEckEegMf+O2OF5E5IjJLRM7z9qUBu1S1vKFrNofMtATW21gOY0wbEx3uADxjgWmqWuG3L1NVN4rIYcAHIrIQ2B3oBUVkHDAOICMjg7y8vCYFVlhYWOe5uqeUb3eX8c77ucRGSZOuHwz1xRgJIj0+iPwYIz0+sBiDIdLjq6aqIXkAxwMz/bZvB26vo+xXwAn1XGsScBEgwDYgurb3qOuRnZ2tTZWbm1vnsVe/ytfM297QZd/tafL1g6G+GCNBpMenGvkxRnp8qhZjMERafMAcreU7NZRVVbOBvl4vqFjcXcX0moVEpD/QEfjcb19HEYnzXqcDI4El3gfJ9ZIIwJXAayH8DPXqlWpdco0xbU/IEoe6dojxwExgKTBVVReLyJ0i4t9LaiwwxUsKVY4E5ojI17hEcY+qLvGO3QbcLCIrcW0ez4TqMzQky+uSa5MdGmPakpC2cajqDGBGjX131NieWMt5nwGD67jmalyPrbDrkBBDcny03XEYY9oUGzl+CESErLRE65JrjGlTLHEcol7WJdcY08ZY4jhEWWkJ5O/cR1lFZbhDMcaYZmGJ4xBlpiVSUals2rUv3KEYY0yzsMRxiDK9LrlrrYHcGNNGWOI4RFnprkvuemsgN8a0EZY4DlHn5DjiY3x2x2GMaTMscRwiESEzNdEGARpj2gxLHEHQKy3BBgEaY9oMSxxBkJWWwLodRVRWasOFjTGmhbPEEQS90hIpLa9kc0FxuEMxxpiQs8QRBFlpXpfcbVZdZYxp/SxxBEFmqtcld4c1kBtjWj9LHEHQrUM80T6xLrnGmDbBEkcQREf56JmawHpLHMaYNsASR5D0Sk2w6dWNMW2CJY4gyUpzdxwHLmRojDGtjyWOIOmVlkhBSTk79paGOxRjjAkpSxxBUt0l19o5jDGtXEgTh4iMEZFlIrJSRCbUcvwBEZnvPZaLyC5v/xAR+VxEFovIAhG51O+cSSKyxu+8IaH8DIHK9BKHdck1xrR20aG6sIhEAY8CpwP5wGwRma6qS6rKqOqv/MrfAAz1NouAH6vqChHpBswVkZmquss7fquqTgtV7E3Ro2MCIjYI0BjT+oXyjmMEsFJVV6tqKTAFOLee8pcBLwOo6nJVXeG93gRsATqFMNZDFh8TRdf28bb+uDGm1ZNQ9QISkYuAMap6rbd9BXCsqo6vpWwmMAvooaoVNY6NAJ4HBqpqpYhMAo4HSoD3gQmqWlLLNccB4wAyMjKyp0yZ0qTPUVhYSFJSUkBl7/1yH6UV8Ifj2zXpvZqqMTGGQ6THB5EfY6THBxZjMERafKNHj56rqsMPOqCqIXkAFwFP+21fATxSR9nbgH/Usr8rsAw4rsY+AeJwCeWOhmLJzs7WpsrNzQ247G3TvtZhd77T5PdqqsbEGA6RHp9q5McY6fGpWozBEGnxAXO0lu/UUFZVbQR6+m338PbVZixeNVUVEWkPvAn8TlVnVe1X1W+9z1QCPIerEosImWmJbN9bSkFxWbhDMcaYkAll4pgN9BWR3iISi0sO02sWEpH+QEfgc799scArwAtaoxFcRLp6zwKcBywK1QdorKouubaokzGmNQtZ4lDVcmA8MBNYCkxV1cUicqeInONXdCwwxbstqnIJcBJwVS3dbieLyEJgIZAO3BWqz9BYvSxxGGPagIC644pIIrBPXeN0P6A/8Jaq1lsno6ozgBk19t1RY3tiLee9CLxYxzVPCSTmcMhMc9Orr7OxHMaYVizQO46PgHgR6Q68g2vonhSqoFqqpLho0pNiWWdjOYwxrVigiUNUtQi4AHhMVS8GBoYurJYrMy3RZsk1xrRqAScOETkeuBzX0wkgKjQhtWyZqQk2CNAY06oFmjhuAm4HXvEauA8DckMWVQuWmZbIt7uLKS6raLiwMca0QAE1jqvqh8CHACLiA7ap6o2hDKylqprscMOOIvpmJIc5GmOMCb6A7jhE5CURae/1rloELBGRW0MbWsuUadOrG2NauUCrqgao6h7cgLu3gN64nlWmhuouudZAboxppQJNHDEiEoNLHNO98Ru2RmotOibEkBwfbYMAjTGtVqCJ45/AWiAR+MibzXZPqIJqyUSErLRE1lnPKmNMKxVQ4lDVh1W1u6qe7U0wuA4YHeLYWqxeaQlWVWWMabUCbRxPEZH7RWSO9/g77u7D1CIrLYGNO/dRVlEZ7lCMMSboAq2qehYowE0+eAmumuq5UAXV0mWmJlJeqWzatS/coRhjTNAFuub44ap6od/2n0RkfgjiaRX8u+RW9bIyxpjWItA7jn0icmLVhoiMBOzP6TpUJYv11s5hjGmFAr3j+BnwgoikeNs7gStDE1LL1zk5jvgYnw0CNMa0SoFOOfI1cLS3nCuqukdEbgIWhDC2FsvnE3qlJthYDmNMq9SoFQBVdY83ghzg5hDE02pkpiVal1xjTKt0KEvHStCiaIWqplevrLQB9saY1uVQEkeD34giMkZElonIShGZUMvxB/zWFF8uIrv8jl0pIiu8x5V++7NFZKF3zYdFJCITWGZ6IiXllWwuKA53KMYYE1T1tnGISAG1JwgB2jVwbhTwKHA6kA/MFpHpqrqkqoyq/sqv/A3AUO91KvBHYLj3/nO9c3cCjwPXAV/g1jMfg5t4MaJkprouueu2F9E1pd4flTHGtCj13nGoarKqtq/lkayqDTWsjwBWqupqVS0FpgDn1lP+MuBl7/WZwLuqusNLFu8CY0SkK9BeVWepqgIv4CZejDhZNkuuaUsW/ZfU7XPCHYVpJoF2x22K7sAGv+184NjaCnqTJvYGPqjn3O7eI7+W/bVdcxwwDiAjI4O8vLxGfwCAwsLCJp1bUalECXz81Tdk7F3dpPcOVFNjbC6RHh9EfoyRHF9M6W6Om/UzjpRoPk3uS1lsSsMnhUkk/xwh8uOrEsrE0RhjgWmqGrT1VlX1SeBJgOHDh2tOTk7jL7LyfZbO+ZIjv39nk2LoOTcXTUwhJ2dYk84PVF5eHk36fM0k0uODyI8xouPL/QtUluKjnJFlH8EZfw93RHWK6J8jkR9flUNpHG/IRqCn33YPb19txrK/mqq+czd6rwO55qFRhVmPc8Syx+Dbpg1XyUxLZK1VVZnWrHQvfPkkHPE9NnUbA3Oegy3fhDsqE2KhTByzgb4i0ltEYnHJYXrNQiLSH+gIfO63eyZwhoh0FJGOwBnATFX9FtgjIsd5val+DLwWkuhF4LzHKYtJhv9cBSUFjb5EZloC67cX4ZpjjGmF5v0L9u2EE29ibdZlEJsE7/w+3FGZEAtZ4lDVcmA8LgksBaaq6mIRuVNEzvErOhaYon7frqq6A/gzLvnMBu709gH8HHgaWAmsIpQ9qpI6sWTAr2HnGnj9JncX0giZaYkUlJSzY29paOIzJpwqyuDzR6DX8dBzBGWx7eHk38DKd2Hle+GOzoRQSNs4VHUGrsus/747amxPrOPcZ3HTudfcPwcYFLwo67e7w0DI+S3k3gW9R0H2VQGfW90ld0cRaUlxIYrQmDBZ/Ars3gBn/23/vhHXweynYebvoHcOREVKM6oJplBWVbUeo26Gw3Lgrdvgu0UBn5aVXjWWw9o5TCujCp8+BJ36Q98z9u+PjoMz/gxbv4F5k8IWngktSxyB8EXBBU9BfIrX3lEY0Gk9OiYggk12aFqfle/D5kUw8pfgq/E10v/7kHmi621VvDs88ZmQssQRqKTOcOHTsGMVvHlzQO0d8TFRdG0fb4nDHJoV75I952bYHZoOhE3y6YPQvjsMuujgYyJw5t1QtAM++tvBx02LZ4mjMXqfBCffBgv+DV+9GNApvdISrKrKNF3pXnj9lyQXroL3JoY7Gid/Lqz9GI77OUTH1l6m2xAY8kP44gnYsaZZwzOhZ4mjsU661SWQGbfClqUNFs9KS7Q7DtN0H90HezayLe0YWDgVNnwZ7ojc3UZ8CmQ3sJbbKX8AXwy8e0f95UyLY4mjsXxRcMHTEJcMU690fxHWIzMtke17SykoLmumAE2rsW0FfPYIHP1Dlh55CyR3dR00KivDF9P2VbD0dTjmWvc7UJ/2XeHEm2DpdFj3WbOEZ5qHJY6mSM6AC5+CbcvdnUc9MtP2z5JrTMBU4a3fQEwCnP4nKqLbwWkTYdM8WDAlfHF99jBExcKxPwus/PHjXVvI27eHN+GZoLLE0VSH5bhqq/mTYf5LdRazxGGaZOnrsOoDGP1b1zEDYPAl0H04vPenJs1kcMgKNsP8l13bRVVMDYlNgFP/CN/Od22DplWwxHEociZA1ih485Y65+fJrJpefYc1kJsAle51f6FnDHJVQlV8PjjrXij8Dj6+v/nj+uIJqCiFE25o3HmDL4Zuw+D9PzVYtWtaBksch6JqfEdMghvfUXrwXUVSXDTpSbGs22Z3HCZAH/8d9uTD2fcdPPK6x3A4aqyb6qM5eyuVFMDsZ2DAOZB2eOPO9flgzF+h4Fv47B+hic80K0sch6p9V7jgSTdS9q3a2zt6pSbYHYcJzPZV7sv1qLGQeULtZU77o9db6Q/NF9fcSVCy2w34a4pex8HA891o8z2bghqaaX6WOIKhz6luWpKvXoSvD67HtS65JiCqrrNFdDycXs8aMO27uf9vS1+H1R+GPq7yUvj8MVct2z276dc5bSJUlsP7TVvfxkQOSxzBkvNb6HUCvPEr2Lr8gEO90hL4dncxxWVBW6fKtEbfvAGr3ncN4skZ9Zc9fjx06OXaQirKQxvXwv9AwSYYedOhXadjlhs0+PXLsHFeMCIzYWJTVwZLVDRc9Aw8caJr77jufYhpB+xff3zDjiL6ZjTQ9z3cyopdXXR5CZQX1/5cUc+x6ueS6u2+++LgxBPqHmVsXPvY27dD54FwzHUNl4+JhzPugqk/hnnPwzHXhCauykrXBTdjkLuzPlSjbnE9EWf+Dq6e4aYnMS2OJY5gat8Nzn8SJl/oBmqd8zDg7jjAdcmN6MSxYTZMvcIljsaKinNVLNE1nqOi6b7pK3j1eteRoOaEeMb55H43RflVMwKfivzIc1z10Qd3waALoF3H4Me1YqZrv7vgqeB8yce3h9G/gzducgMDB5x76Nc0zc4SR7D1Pc3d0n/6oJuaZPBF1XccEb2M7Lx/uckbk7vCOY9AbKKXAGK951qSQnScSxhRsfUmhFUv3MDhi16A5C5u8jtzoO2rXKPxUZdC1sjAzxNxvZX+eRLk3Qtn3RP82D59CFJ6uYbtYBl6hVtu9t07oN8Y9//ItCiWOELhlN/D+lnw+i+h6xA6ph1Ocnw063dEYAN5RRnM/K37Re59Mlw8CRJSg/oWG3pewOGdElwX0uQujR8H0JqpurvTqLj6G8Tr0mUwDLsSZj8Fw6+GTkcEL7b1X8D6z2HMvRAVE7zrRkW7PyD+db4bG9LUnlombKzeIBSiYlx7R1QM/OcqpLyEzLQE1kZaz6q92+CF81zSOH48/Oh/QU8agDfN9l/dX63v/B4WTA3+e7RUy2a4pVZH3+6SalOc8nuISXRtJMFc3/7Th1z117ArgnfNKoefAn3PdNOu790W/OubkApp4hCRMSKyTERWisiEOspcIiJLRGSxiLzk7RstIvP9HsUicp53bJKIrPE7NiSUn6HJUnrA+f+EzQth5u1kpiVG1vTq334NT+ZA/mwX55l3h3aZT5/PvU/WKNfesfL90L1XS1FaBG9NgE5HwohxTb9OYjrk3OZ6ZK14JzixbV0Gy950ccUmBueaNZ3xZzeSPPcvobm+CZmQJQ4RiQIeBc4CBgCXiciAGmX6ArcDI1V1IHATgKrmquoQVR0CnAIUAf6/EbdWHVfV+aH6DIes35lwwo0w51nO0M/YuHMfZRURMNHbwmnwzJmglfCTt+Hosc3zvtFxMHayW27031dYl8xPHoDd6+F7fzv0qqBjroO0vq7asbz00GP77GGIbndoCa0hnY5wvcHmPhfQEgUmcoTyjmMEsFJVV6tqKTAFqNmF4jrgUVXdCaCqW2q5zkXAW6oaYfU8ATr1DugxgrPX/IXu+i2bdu0LXyyVFa5B8r/XuIV2xuVB92HNG0N8Clw+DRLSYPLFrmG4Ldqx2lUFDb4Ysk489OtFx8KZf4HtK13V46HYs8kNZB36I3c3E0o5t7vp2Wf+LrTvY4IqlImjO7DBbzvf2+evH9BPRD4VkVkiMqaW64wFXq6x724RWSAiD4hIZHfJiIqBi55FfNE8GvMw67fsDE8c+3a6L+pPH4LhP4EfTw98htNga98Vrvifu+N58UIorO3vhVburQnu/8bpfw7eNfudAX1Ohw/vhcKtTb/OrMdBK+CE8cGLrS4JqW5VzVXvw4p3Q/9+JihEg9mY5n9hkYuAMap6rbd9BXCsqo73K/MGUAZcAvQAPgIGq+ou73hXYAHQTVXL/PZ9B8QCTwKrVPWg7igiMg4YB5CRkZE9ZUrT1jAoLCwkKSmpSef6i9s0i+OX/5XZyadTNPSnqC94vVQaijFh73oGLfoL8cVbWdF3HN92OzNo7x2IuuJrv3sZR3/9e4oSejJ/yF1URCc0a1z+gvXvHIi0bV8yeNHdrDz8J+T3DGwcQ6DxJezNZ/icG/muy2ksP+LnjY4tqnwvx39+DdvThrN0wK8bdW5Tf4ZSWcYxs29AJYo5wx9GfVGNvkagmvPfuSkiLb7Ro0fPVdXhBx1Q1ZA8gOOBmX7btwO31yjzBHC13/b7wDF+278EnqznPXKANxqKJTs7W5sqNze3yef6q6io1Kf/cJnqH9ur3ttbdcZtqpu+Dsq1641xyeuqd3dT/b8+qus+D8r7NVa98S17W3ViR9UXzlMtK2m2mA6wZZl+/M705nmv0iLVBwapPnKsanlpwKc16v/hWxNU/5jStP9fH9/v/o9umt/oUw/pd2XJ6+59v3iy6dcIQLB+n0Ml0uID5mgt36mhrKqaDfQVkd4iEourcppeo8yr3pc/IpKOq7pa7Xf8MmpUU3l3HIiIAOcBi4IfevD5fMLUDtfxUMbdrk57zjPwz1Hw+IluArlDqVqoTWUl5N0D/74c0vu59oxexwX3PYKh35luhP2qD+C1XzTvKnG7N7rlfx89hmO/+BnMeda1A4XSJw/CrvXelOlBHBvh7+TfuCqgtyc0rntueYmrpjpsNHQ9OjSx1aX/91yPu9y/wL5dzfveptFCljhUtRwYD8wElgJTVXWxiNwpIud4xWYC20VkCZCL6y21HUBEsoCeQM3pPyeLyEJgIZAO3BWqzxBsPdOSebN4MFzyAtyyDM72etPMvB3u7w8vX+ZmPD3UXjElBW7qkLy/wtGXwdVvQUrN5qUIMvRHcMofYOFUeO+PoX+/8lLXo+mRY2D52zDqFvYmZrkJKp8+DTbODc377ljj3nfQhdB7VGjeA9zYi9G/g3WfwpLXAj/v6ylQuDk8A/JEXJfwfTvho/ua//1No4R05LiqzgBm1Nh3h99rBW72HjXPXcvBjemo6ilBD7SZZKUl8PGKrVRWKr6EVBhxnXtsWeqWn13wbzcgrF0qHHWJ+9LvenTj5gjavgqm/BC2rYAx97i1oVvCRHKjboGC71w30OQucPwvQvM+qz6AGb+B7SvgiO/BmL9Axyzm+0aRk7bV9e556lTIvsr1iAvmgMi3b3d/KJzRDH/rZF/l7qDe+YO7q/Mm3KxT1WSGXY92yyKHQ9ejYcjl8MU/XTfd1MPCE4dpkI0cb0ZHdm1PSXklz35aY+W2zke6wVC/WuK6qh52svulf/JkeHwkfPZIYD2PVr4HT412fzVe8T847vqWkTTAxXnWvW7ivpm/dWNNgmnXBjd25F/nux5Dl0+Dy15yU31Xvf9Rl8ANc1yynfc8/CMb5r0QnOqzZW/D8rdcD6L23Q79eg3xRbl5rHavd/9/GozvTdeVd+Qvw/t/5tQ/uLnP3r2j4bImbCxxNKPzhnbn7MFduOvNpUz/upZV0KKioe/pbr6oW5bB9/7ups9+53fw9/7w0lhX9VBecuB5qq6b7eSLIaWna88I11+Nh6JqKd7MkfDKz2BV7qFfs7zETWvx6AjX3fOU38P1n7ufc23iU9xkgT/9yLUNTb8Bnj3TjbRvqrJiePs2SD/CJfPm0vskOPIHbubd+lbdU3VtLx2z4Mgwz1ab3AVO/JWrsl37SXhjaQ12rG64TBNY4mhGUT7h/kuGMKJ3KrdMnc+nK+uZoychFY65Fq77AH7xpZsYcNNXbv2Fvx/hVorb9BWUFnHk0vvdX2hHngPXvLP/r+iWKCYexr4E6X3h3z86tC/sle/BY8fDB392cyON/xJOutW9R0O6DHZtQ+c97n75nsxxP/OmNNx++hDsXBvaBvG6nP5n1+D/3sS6y6z7DDbOcfOVhXLamUCdMB7a93CThH49xbV7mMDtWO3+WHrsBHh4aEgG2VriaGbxMVE8dcVweqcn8tN/zWXxpt0Nn9TpCDj9T/CrxXD5f12vl7nPuy+zv/Wj85aPXePyxZNCN69Qc2rXAX70X4jvAC9e5BqVG2PXephyuRtcCO5aYye7FfMaw+eDIT+EG+bC8Gtg9tPwyHCY/3LgvZV2rnV/8Q+8wFVBNrfU3u6LeMG/3Xortfn0IUhId50UIkFMOzjnITeX1ys/hfv6uCrG2c9AweZwRxeZdue7teqfzHHJ4oM/Q1wSnPV/IZm41BJHGKQkxPD8T0aQHB/NVc/NZkOg061HRbv1Pi5+Dn69DL7/ABw+moWDfw8n/brltGcEon03105TUQovXhBYd+WyYvjwPnhkhGsEP/UO+Pnn0Oe0Q4ulXQc3n9R1udAhE179GTx3Fmxe3PC5b98OEtU8DeJ1OfFmSOriqstqttdsXuIWazr2pw03oDenPqe5P5Su/cDdCe1c69aL+fsRbp61zx5x+9qywi3w5VPw7Bh4YKCbeVrV3WXetMjVPhz705As8GWJI0y6prTj+Z+MoKSsgiuf+5IdexvZBbddRzd1yKX/YkfawQM7W4VOR8APp7r6+ZcugZLCusuueBcePx5y73LtF7/40vXUCuYiQd2GwDXvwjn/cLPHPjHKJYbiPbWXX/6O6yWXc1t4u0PHJcFpE1034wX/PvDYZw9DTIKrFo00Ph/0yHZ32zfMc21TObdD2V7X7vfQ0e7f4MP/cz0TQzQLRkQp2uFqG54/x6uy/jUU73ZtdzfMg59+CCNvhA49QxqGJY4w6peRzDNXHUP+zn1c8/xs9pWGePBZS9TrWLjoOfh2vmvfqSg78PjOdfDyD2HyRe4v+ytegUv/FbpfHJ8Phv3YVV8N+7EbMPfIcFjwnwO/uMqK4a3fuAb2Y5uxQbwuR10K3bNdW0dVAt61ARb+xy0EFYp1WIJJBDIGuCT8s0/gxvnuLi6mHeTeDY8d5/4d3pvoEmRrSiLFe1xbz+RL4G994fUb3TLDo25xyfTnn7u2u7TDmy0kSxxhdkxWKg+PHcL8DbsY/9I8yiNh2vVI0/9s+P6DbiK818a7L4WyYveX5qMjYHWu+4v6+s9cI3hzSEiFHzwI173vqtX+dy08/wPY8o07/tnDsHONaxCPjm2emOrj87lxPYXfuTYX8CYz1NCNmQml1N6uw8g173g9EO93PQo/+wc8dYqrupnxG1jzMVSUhzvaxistgsWvuA4i9/VxbT1blsBxP4dxH7q7i1N+75JpGERAFwozZlBX7jx3EH94dRG/f3URf71gMNKa2iuCIftKNz4l9243q27+bPfFPOA8N+I4pUd44uqeDde+78Z9vPcneGIkZF8NX/3LxRZJ3aJ7jnB3Hp894rrpzp0Egy8KebVGyCV3cQMGj7nGVeUsn+m68857Hr78p5vC/4iz4YizSSzcDLsOc92uY5NdQg03VSgvhuI9pG37Ev77Inwzw1XJJWW4JYEHXgA9jomMeLHEETGuOC6TzbuLeSR3JRnt4/nV6f3CHVLkOelWKPjWDY5M7wdXvAqHjw53VG78yfCfuO7Q701063/HJLr1MSLNaRPdl+rz57ovpta23ndCKgy5zD1K97ou2Utfd+OfvvoXxwDMqSosEN/eJZH4FNeLr/p1CsS1P3C75iOuvRtMWlLg2hlKCqBkj7e9x3vtv13gV2aP3749UOnuigbD/pkjBl3gxjSFcLbgprLEEUFuOaMfm/cU89D7K+jcPo7Lj80Md0iRRcTN7zXwAuh5bGRUAflLTIdzH3F/+VaUR+b8YO27uV5WuXe5tTsyBoY7otCJTYQB57pHeQlsnMuiLz9kUJ+e7ou+tsfOtftfl9TR6aGxfDEuQcUlu2QT195Vq3X29vkd+3rDHo4+d3zzj/dpJEscEURE+MsFg9laWMIfXl1Ep6Q4zhjYJdxhRRZfVGgnCAyGbkPDHUH9ThgPu9ZGRqN9c4mOg8wT2LamFIbmBHZOZYV3Z1BHkineDb5o787EPzEke3ck3nZ0XMBd5XfuzYv4pAGWOCJOTJSPxy4fxmVPzuKGl7/ipeuOJTszwnu8mJYlph2c+2i4o4h8vijX7T0E4yBaushoaTEHSIiN5tmrjqFbh3b8ZNIcVm4pCHdIxhhTzRJHhEpLiuP5q0cQE+Xjymdns3lPcbhDMsYYwBJHROuVlsCkq49hV1EpVz77JXuKyxo+yRhjQswSR4Qb1D2FJ67IZuWWQsa9MIeSchtdbowJL0scLcCovp3428VHM2v1Dm6e+jWVla1oOgVjTItjvapaiPOGdmfznmL++tY3dE6O447vD7DR5caYsAjpHYeIjBGRZSKyUkQm1FHmEhFZIiKLReQlv/0VIjLfe0z3299bRL7wrvlvEYmwUWChM+6kw7h6ZBbPfbqWJz8KzcpexhjTkJDdcYhIFPAocDqQD8wWkemqusSvTF/gdmCkqu4Ukc5+l9inqkNqufS9wAOqOkVEngCuAR4P1eeIJCLCH743gC0FJe7Oo30c5w8N0xxNxpg2K5R3HCOAlaq6WlVLgSlAzQWNrwMeVdWdAKq6pb4LiqubOQWY5u16HjgvmEFHOp9PuP+Sozn+sDRu/c8CPloewAJHxhgTRKIhmrdeRC4Cxqjqtd72FcCxqjrer8yrwHJgJBAFTFTVt71j5cB8oBy4R1VfFZF0YJaq9vHK9ATeUtVBtbz/OGAcQEZGRvaUKVOa9DkKCwtJSkpq0rmhVFSm/PXLYrYWVXLhYcro3olE+yKzzSNSf4b+Ij3GSI8PLMZgiLT4Ro8ePVdVD1opLtyN49FAXyAH6AF8JCKDVXUXkKmqG0XkMOADEVkIBLBAt6OqTwJPAgwfPlxzcnKaFGBeXh5NPTfUso8t5prnZzN5xR5yt1Ry7ajDGDuiJwmx4f5nPVAk/wyrRHqMkR4fWIzBEOnxVQllVdVGwH+i/x7ePn/5wHRVLVPVNbi7j74AqrrRe14N5AFDge1ABxGJrueabUZG+3heH38iN2fH0SM1gTvfWMIJ93zAg+8tZ2djl6I1xpgAhTJxzAb6er2gYoGxwPQaZV7F3W3gVUP1A1aLSEcRifPbPxJYoq5eLRe4yDv/SuC1EH6GiCciHNUpmqk/PZ7/Xn88wzNTefC9FYy89wP+/MYSvt29L9whGmNamZDVaahquYiMB2bi2i+eVdXFInInMEdVp3vHzhCRJUAFcKuqbheRE4B/ikglLrnd49cb6zZgiojcBXwFPBOqz9DSZGem8vSVqSzfXMATeauY9NlaXvh8LecN6c5PTz6cPp0jp+7UGNNyhbQyXFVnADNq7LvD77UCN3sP/zKf4S2GVcs1V+N6bJk69MtI5v5Lh3DzGf14+uM1TJm9nmnz8jlzQBeuzzmco3t2CHeIxpgWLLJaUU1Q9eiYwMRzBnLDKX2Y9Nlanv9sLW8v/o4TDk/j5zl9GNknzUafG2MazeaqagPSkuK45Ywj+Oz2U/nd2UeyamshP3rmC8555FNmLPyWCpv7yhjTCJY42pCkuGiuO+kwPvrNaO65YDCFJeX8fPI8Tr//Q/49e73NvGuMCYgljjYoLjqKsSN68d7NJ/PY5cNIiIvitv8u5OT/y+Ppj1eze5+t+2GMqZu1cbRhUT7h7MFdOWtQFz5ZuY3H81Zx15tL+cuMpRzdswMn9klnZJ90hvbqQFx0VLjDNcZECEscBhFhVN9OjOrbiYX5u3l36WY+WbGVx/JW8Y8PVtIuJooRvVOrE0n/Lsn4InR6E2NM6FniMAcY3COFwT1SuPn0fuwpLuOL1Tv4dOU2Plm5jbtnLAUgLTGWE/qkc2KfNEb2SadHx4QwR22MaU6WOEyd2sfHcPqADE4fkAHAd7uL+XTltupE8vrXmwDISktgZJ90TuyTzvGHp9Ehoc0skWJMm2SJwwSsS0o8F2b34MLsHqgqK7cU8omXSF6bv4nJX6xHBAZ3T6lOJNmZHcMdtjEmyCxxmCYREfpmJNM3I5mrR/amrKKSBfm7+GTFdj5duY2nPlrN43mriIv2kZUMH+xeRN+MZPp1TqJfRjIdE+2uxJiWyhKHCYqYKB/ZmalkZ6byy9P6UlhSzuw1O/hk5TZyF67jf/M2UlhSXl0+PSmOfhkuifT1nvt1TiYlISaMn8IYEwhLHCYkkuKiGd2/M6P7d2ZU0hZOPvlkNu0uZvnmAlZsLmD55kJWbC5g6pwNFJXuH3jYOTnuwGSSkUTfjGTax1tCMSZSWOIwzUJE6N6hHd07tGP0EfuXlq+sVDbu2seKLS6ZuMRSyJQvN7CvbH9C6dI+/oBk0rNjAt07tqNrSjtio20cqzHNyRKHCSufT+iZmkDP1ARO6Z9Rvb+yUsnfuY/lmwtYvsUlk+WbC3hx1jpKyiury4lARnI83Tu6pNSjY7sDX3dIoF2sDV40JpgscZiI5PMJvdIS6JWWwGkD9ieUikpl48595O8sIn/XPjbu3MdG7/mrDTuZsfBbymtM2piWGFudTPYnlwS33bEdKe2sGsyYxrDEYVqUKL+EUpuKSmVLQXF1Qsnf6R4bd7m7l9xlWyguqzzgnOS4aJJjKshaMYtOyXF0To7znuMP2E5pF2PT0BuDJQ7TykT5hK4pru1jeC3HVZXte0sPuFPZuGsfi1ZtoKS8kq/W72JLQfFByQUgNspHp+Q40g9ILu65U1IcndvHV7+2dhfTmlniMG2KiJCeFEd6UtwBKyHm5W0lJ+cEwCWXwpJythSUsNV7bKl+LmZrQQkbdhQxb91Otu8trfV92sdH0zExlg7tYuiQEEvHBPfcISGGjt5z1f6q7aS4aLujMS1CSBOHiIwBHsKtOf60qt5TS5lLgImAAl+r6g9FZAjwONAetxb53ar6b6/8JOBkYLd3iatUdX4oP4dpW0SE5PgYkuNjOLxT/eu0l1VUsr2wtDqhVCWY7YUl7CwqY9e+MnYWlbJ6WyG79pZR4DeWpaZonxyQUFLaeYklMZYd35ayLnYtiXHRJMVFkRQXQ2JcFElx0SR6j6S4aKJs8knTDEKWOEQkCngUOB3IB2aLyHRVXeJXpi9wOzBSVXeKSFU/zSLgx6q6QkS6AXNFZKaq7vKO36qq00IVuzGBiony0SUlni4p8QGVL6uoZPe+MnYVlbKzqIyde0vZ5be9q6iUXUUu2eTvLGLhRre/tLySaSsWN3j9+BgfSV4S8U8oVQknMXb/vnaxUbSLiap+jj/gtc89e9sxUVb1ZvYL5R3HCGClqq4GEJEpwLnAEr8y1wGPqupOAFXd4j0vryqgqptEZAvQCdgVwniNCbmYKF91VVljvPN+LsOOPYG9JeUUFJezt6ScvaXlFJZUuNcl5RRWP1dUbxeWlLN5T/EB+/3HxwQq2ifViaQqqdRMNrt2FDNzxwLiot3++Bife472nr19cTFRxEf7Ha8q650XF+2zafsjnKiGZr1pEbkIGKOq13rbVwDHqup4vzKvAsuBkbjqrImq+naN64wAngcGqmqlV1V1PFACvA9MUNWSWt5/HDAOICMjI3vKlClN+hyFhYUkJdVfXRFukR5jpMcHkR9jMOOrVKW4HEorlNJKKKnwXldAaaX3XKGUVEBZJZRU7N/nyux/XeJdo7RCKSmvpFx91dctP7h/QcCifRAXBXFRQqz3HFfLc2yUEB/tv9/tO6isV6Zo716SkhLx4cYAAfhqPAuEra0p0v4fjh49eq6qHtTPJNyN49FAXyAH6AF8JCKDq6qkRKQr8C/gSlWt+m94O/AdEAs8CdwG3Fnzwqr6pHec4cOHa05OTpMCzMvLo6nnNpdIjzHS44PIjzHS44ODY6ysVIrLKyguq6S4rMJ7VHr7Kiip2n9AGb99pRUUlVZQVFbBvtIKikrLKSqtYFdpBfuK9m+XNCpDCa4mvIFS4kr6RNxrkeptX9W2uJ528TFRxFXfMfmq75oOuMtq6C4s2l3juwVfM6LPEKJ9PmKihOgoH9E+ITpKDtoXE+ULW5tWKBPHRqCn33YPb5+/fOALVS0D1ojIclwimS0i7YE3gd+p6qyqE1T1W+9liYg8B/w6VB/AGNN0Pp+QEBtNqJdnqahU9pW5RLKvKtmUumSz12/fvrIKVqxYyWGHH4YqVKq7+wKX5BS3rep61lUqKN6zt0+9slXnllVUVifDEr/Et6e4rPp1SXlldaIsrQggyX3xacCfXQRifD4vsbhkUjPJPHPlcDLTEpv2w61DKBPHbKCviPTGJYyxwA9rlHkVuAx4TkTSgX7AahGJBV4BXqjZCC4iXVX1W3H3kucBi0L4GYwxES7KJ9UdAhqSV76OnJMOb4aoaldRqZTUcxc2Z958BgwcTHllJWUVuv/Z73VFLfvKKyopr3SJrLxCKausrD4eHxP8KXdCljhUtVxExgMzce0Xz6rqYhG5E5ijqtO9Y2eIyBJct9tbVXW7iPwIOAlIE5GrvEtWdbudLCKdcHeS84GfheozGGNMMEU1cBdWuiGaHL8pdiJVSNs4VHUGMKPGvjv8Xitws/fwL/Mi8GId1zwl+JEaY4wJlHXONsYY0yiWOIwxxjSKJQ5jjDGNYonDGGNMo1jiMMYY0yiWOIwxxjSKJQ5jjDGNErJJDiOJiGwF1jXx9HRgWxDDCYVIjzHS44PIjzHS4wOLMRgiLb5MVe1Uc2ebSByHQkTm1DY7ZCSJ9BgjPT6I/BgjPT6wGIMh0uOrYlVVxhhjGsUShzHGmEaxxNGwJ8MdQAAiPcZIjw8iP8ZIjw8sxmCI9PgAa+MwxhjTSHbHYYwxplEscRhjjGkUSxz1EJExIrJMRFaKyIRwx+NPRHqKSK6ILBGRxSLyy3DHVBcRiRKRr0TkjXDHUpOIdBCRaSLyjYgsFZHjwx1TTSLyK+/feJGIvCwi8REQ07MiskVEFvntSxWRd0VkhffcMcLiu8/7d14gIq+ISIdwxefFc1CMfsduERH1VkaNOJY46iAiUcCjwFnAAOAyERkQ3qgOUA7coqoDgOOAX0RYfP5+CSwNdxB1eAh4W1X7A0cTYXGKSHfgRmC4qg7CraY5NrxRATAJGFNj3wTgfVXtC7zvbYfLJA6O711gkKoeBSwHbm/uoGqYxMExIiI9gTOA9c0dUKAscdRtBLBSVVeraikwBTg3zDFVU9VvVXWe97oA94XXPbxRHUxEegDfA54Odyw1iUgKboniZwBUtVRVd4U1qNpFA+1EJBpIADaFOR5U9SNgR43d5wLPe6+fB85rzpj81Rafqr6jquXe5iygR7MHdmA8tf0MAR4AfgNEbM8lSxx16w5s8NvOJwK/mAFEJAsYCnwR5lBq8yDul6AyzHHUpjewFXjOq0p7WkQSwx2UP1XdCPwN99fnt8BuVX0nvFHVKUNVv/VefwdE8uLZPwHeCncQNYnIucBGVf063LHUxxJHCyciScB/gZtUdU+44/EnIt8Htqjq3HDHUodoYBjwuKoOBfYS3uqVg3jtBOfiklw3IFFEfhTeqBqmrp9/RP7FLCK/w1X1Tg53LP5EJAH4LXBHuGNpiCWOum0Eevpt9/D2RQwRicEljcmq+r9wx1OLkcA5IrIWV9V3ioi8GN6QDpAP5Ktq1Z3aNFwiiSSnAWtUdauqlgH/A04Ic0x12SwiXQG85y1hjucgInIV8H3gco28QWyH4/5A+Nr7nekBzBORLmGNqhaWOOo2G+grIr1FJBbXIDk9zDFVExHB1c0vVdX7wx1PbVT1dlXtoapZuJ/fB6oaMX8tq+p3wAYROcLbdSqwJIwh1WY9cJyIJHj/5qcSYQ34fqYDV3qvrwReC2MsBxGRMbhq03NUtSjc8dSkqgtVtbOqZnm/M/nAMO//aUSxxFEHrxFtPDAT94s6VVUXhzeqA4wErsD9FT/fe5wd7qBaoBuAySKyABgC/CW84RzIuxuaBswDFuJ+Z8M+LYWIvAx8DhwhIvkicg1wD3C6iKzA3SndE2HxPQIkA+96vy9PhCu+emJsEWzKEWOMMY1idxzGGGMaxRKHMcaYRrHEYYwxplEscRhjjGkUSxzGGGMaxRKHaVNEJM2v+/J3IrLRbzu2gXOHi8jDAbzHZ8GLOHREZKKI/DrccZiWJzrcARjTnFR1O268BiIyEShU1b9VHReRaL+J8GqeOweYE8B7ROrIbmOCwu44TJsnIpNE5AkR+QL4PxEZISKfexMfflY1slxEcqrWFPH+Wn9WRPJEZLWI3Oh3vUK/8nl+631M9kZ/IyJne/vmisjDUstaJeLWMblPRGZ7a0j81O+6H4nIm+LWi3lCRHzesctEZKG4tTvu9bvWGBGZJyJfi8j7fm8zoLbPYEx97I7DGKcHcIKqVohIe2CUqpaLyGm40eQX1nJOf2A0bjTyMhF53JtPyt9QYCBuKvRPgZEiMgf4J3CSqq7xRhDX5hrcbLjHiEgc8KmIVM2MOwK3Tsw64G3gAq+K7F4gG9gJvCMi53nv+5Tf+6U28jMYcwBLHMY4/1HVCu91CvC8iPTFzfAaU8c5b6pqCVAiIltw04jn1yjzparmA4jIfCALKARWq+oar8zLwLharn8GcJSIXOQXV1+g1Lvuau+6LwMnAmVAnqpu9fZPxq03UgF8VPV+quq/BkQgn8GYA1jiMMbZ6/f6z0Cuqp4vbq2TvDrOKfF7XUHtv0+BlKmLADeo6swDdorkcPCU5U2dO+hQ4jNtlLVxGHOwFPZPoX9VCK6/DDjMS0oAl9ZRbiZwvTd9PiLST/YvNDXCm7nZ553/CfAlcLKIpItb+vgy4EPcancniUhv7zqpNd/ImMawvy6MOdj/4aqqfg+8GeyLq+o+Efk58LaI7MVN4V+bp3FVW/O8RvWt7F+OdTZuttc+QC7wiqpWisgEb1tw1VCvAYjIOOB/XqLZApwe7M9l2g6bHdeYMBCRJFUt9BLCo8AKVX0gwHNzgF+r6vdDGKIxdbKqKmPC4zqvsXwxrmrsn+ENx5jA2R2HMcaYRrE7DmOMMY1iicMYY0yjWOIwxhjTKJY4jDHGNIolDmOMMY3y/7CfjYN49fcVAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x_train, x_test = ratings[['user_id', 'item_id']].values[train_idx], ratings[['user_id', 'item_id']].values[test_idx]\n",
    "y_train, y_test = ratings['rating'].values[train_idx], ratings['rating'].values[test_idx]\n",
    "\n",
    "#\n",
    "# Model specification\n",
    "#\n",
    "embedding_dim = 8\n",
    "\n",
    "input_user = Input(shape=(1,))\n",
    "input_item = Input(shape=(1,))\n",
    "\n",
    "embedding_user = Embedding(input_dim=max_user_id + 1, output_dim=embedding_dim)(input_user)\n",
    "embedding_item = Embedding(input_dim=max_item_id + 1, output_dim=embedding_dim)(input_item)\n",
    "\n",
    "combined = Dot(axes=2)([embedding_user, embedding_item])\n",
    "\n",
    "score = Dense(1, activation='linear')(combined)\n",
    "\n",
    "model_sgd = Model(inputs=[input_user, input_item], outputs=score)\n",
    "\n",
    "#\n",
    "# Model training\n",
    "#\n",
    "opt = tf.keras.optimizers.Adam(learning_rate=0.01)\n",
    "model_sgd.compile(optimizer=opt, loss='mse', metrics=[tf.keras.metrics.RootMeanSquaredError()]) \n",
    "history_sgd = model_sgd.fit([x_train[:, 0], x_train[:, 1]], y_train, \n",
    "                    batch_size=256, epochs=16, verbose=1, \n",
    "                    validation_data=([x_test[:, 0], x_test[:, 1]], y_test)) \n",
    "\n",
    "plot_keras_training_hitory(history_sgd)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "skilled-restaurant",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmAAAAFzCAYAAACZwbV4AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAABd3UlEQVR4nO3dd3yV5f3/8dd1zsnejIS9kT2NEwfgaF242q+2Lhy1w1bbam1tq7bVah211dbZ9XOgaLVaFbeCuBUUCsgUZUOAJGQnZ1y/P06CARPIOPd9n5zzfj4eeXByzsl9f7gh57zPNY21FhERERFxj8/rAkRERESSjQKYiIiIiMsUwERERERcpgAmIiIi4jIFMBERERGXKYCJiIiIuCzgdQHt0aNHDzto0CCvy2i36upqsrKyvC4jqeiau0/X3H265u7TNXdfV77mCxcu3GGt7dnSY10qgA0aNIgFCxZ4XUa7zZs3j6lTp3pdRlLRNXefrrn7dM3dp2vuvq58zY0x61p7TF2QIiIiIi5TABMRERFxmQKYiIiIiMsUwERERERcpgAmIiIi4jIFMBERERGXKYCJiIiIuEwBTERERMRlngYwY8xPjDHLjDFLjTGPGWPSvaxHRERExA2eBTBjTF/gcqDYWjsW8ANne1WPiIiIiFu87oIMABnGmACQCWz2uB4RERERxxlrrXcnN+YK4PdALfCKtfacFp5zKXApQFFR0YGzZ892t8gYqKqqIjs72+sykoquuft0zd2na+4+XXP3deVrPm3atIXW2uKWHvMsgBljCoCngLOAcuDfwJPW2kda+5ni4mLbFTfj/s9//kNqaionn3yy16Ukja68eWtXpWvuPl1z9+mau68rX3NjTKsBzMsuyGOBz6212621QeA/wOEe1uOY7du3c+utt3pdhoiIiMQJLwPYeuBQY0ymMcYAxwDLPaxHRERExBWeBTBr7QfAk8DHwJLGWh7wqh4RERERtwS8PLm19nrgei9rEBEREXGb18tQiIiIiCQdBTARERERlymAiYiIiLhMAUxERETEZQpgIiIiIi5TABMRERFxmQKYiIiIiMsUwERERERcpgAmIiIi4jIFMBERERGXKYCJiIiIuEwBTERERMRlCmAiIiISt6qrq1m0aJHXZcScApiIiIjErY0bN3L55Zd7XUbMKYCJiIiIuEwBTERERMRlCmAiIiIiLlMAExEREXGZApiIiIiIyxTARERERFymACYiIiLiMgUwEREREZcpgImIiIi4TAFMRERExGUKYCIiIiIuUwATkZgoKyvjpZde8roMEZEuQQFMRGKipKSEm266yesyRES6BAUwEREREZcpgImIiLSDtdbrEiQBKICJiIi00caNG7nlllu8LkMSgAKYiIhIG1VXV/PCCy94XYYkAAUwEREREZcpgImIiIi4TAFMRERExGUKYCIiIiIuUwATERERcZkCmIiIiIjLFMBEREREXKYAJgmptraWlStXel2GiIhIixTAJCGtX7+eyy67zOsyREREWqQAJgmroaHB6xJERERapAAmIiIi4jJPA5gxJt8Y86QxZoUxZrkx5jAv6xERERFxQ8Dj898JvGSt/YYxJhXI9LgeEREREcd5FsCMMXnAUcBMAGttA6BBOyIiIpLwjLXWmxMbMxF4APgUmAAsBK6w1lbv9bxLgUsBioqKDpw9e7bLlXbejh072LlzJyNGjPC6lKSha+4+XXP3VVVVkZ2d7XUZSUX/z93Xla/5tGnTFlpri1t6zMsAVgy8D0yx1n5gjLkTqLDWXtvazxQXF9sFCxa4VmOs3H///cyaNYv58+d7XUrS0DV3n665+1566SVGjBjB4MGDvS4laej/ufu68jU3xrQawLwchL8R2Git/aDx+yeByR7WIyLSpaxbt47LL7/c6zJEpAM8C2DW2q3ABmNMU5viMUS7I0VEpI127drldQki0gFez4L8ETCrcQbkWuBCj+sRERERcZynAcxauwhosW9UREREJFFpJXwRERERlymAiYiISFzyaqUGNyiAiYiISFyqq6vzugTHKICJiIhIXKqsrNx9OxQKeVhJ7CmAiYiISFzauXPn7ttlZWUeVhJ7CmAiIiISl7Zt29bi7UTg9TpgIiIiIi164oknCAaDAGzatImxY8d6XFHsqAVMRERE4tLGjRspLyvDZwxr1qzxupyYUgATERGRuFRdXQ3GkJsaYNmyZV6XE1MKYA676667WLBggddliIiIdCnr168nGAxigIK0FJZ/+ikVFRVelxUzCmAOW7NmTcLN3BAREXHa3LlzATBAUUYa4UiEt956y9uiYkgBTEREROJKQ0MDTz/9NAFjMAZyUwNkp6bw5JNPJszq+ApgIiIiEldefvllSktLSQ9EY4oxhkHZ6Xz22We8//77HlcXGwpgIiIibZAoLS/xrrS0lHvvvZeC9FRSfF/GlN5Z6WSlpvCnO+6gtrbWwwpjQwFMRESkDRoaGrwuIeFZa7n99tupralmdEH2Ho/5jGFMfhZbt23j3nvv9ajC2FEAExERaYOamprdt9Ua5oxHHnmEt99+m2G5WWSnfHWt+IL0VAblZPDMM8/w4osvelBh7CiAiYiItEF5efnu21VVVd4VkqDmzZvH3//+d3pnpjEwJ6PV5w3Pz6Z7eiq33XYbixYtcq/AGFMAk4SjtddExAmlpaW7bzffJFo679133+W3v/0teakpjOmWizGm1ef6jGF891zSfYarr76apUuXulhp7CiAScLR2muSDMLhsNclJJ1HHnlk94e7RNsY2ktvvvkmv/71r8kO+JjcMxe/r/Xw1STV7+PAHrn4I2GuuvJKPvnkExcqjS0FMBGRLigRZoF1NRs3btz94W7jxo0eV9P1WWt56KGHuPbaa8n2+5jcI2+PWY/7kx7wc2CPXHzhED/9yU947rnnHKw29hTARES6oOYBTK1h7qipqcEYQ1pKBp999pnX5XRpNTU1/Pa3v9095qu4MI9Uf/sjSUbAz8GFeRSkBrjtttv405/+RDAYdKDi2FMAc5hmykgy0Lg7991///27r3nz2XniDGttNIBhyM8o4tNPP/W6pC5r3bp1XHrppcx94w2G52Uxrnsu/n2M+dqfFJ+PST3zGJiTwdNPP82PfvQjSkpKYlixMxTAHBaJRLwuIeko9LpP4+7ct3bt2t3XfNeuXR5Xk/hWrFhBOBzGGB89c/qzdu1ajQPrgFdeeYXvfOc7bNu8iQML8xmSl7XPAfdt5TOGkQU5TOiRy6qVK7j4oot47733YlCxcxTAHNY8gCkYuEOhV5JB824Wzchz3rx58wAwxkf/bgcA0cHj0jbV1dXc8LvfceONN5JBhEMK8+menhrz8/TKTOeQwnwidbX8/Oc/56677qK+vj7m54kFBTCHNQ8DWkXZHc2vucKYO3Sd3df89UQtMc4KhUK8/NLLpPrTMUBOeje6ZRXxwgsv6IN1G7z//vtccMEFvPb66wzLy6K4Zx4ZAb9j58tOCXBIUT4DsjN48sknueSSS1iyZIlj5+soBTCHNX9jqqur87CS5NF8QLKuuTsUwNxXX1+PAQyakee0d955h9KyUtJTv9waZ1CP8axdu5Zly5Z5WFl827lzJ7+5/nquvvpqasrLOKgwn6F5Wfhi0OW4P35jGNUth8k98yjZvInLLruM22+/ncrKSsfP3VYKYA5r/sakaePuaH7NtVq1O9TS6766ujoMUODzsW7dOq/LSVgNDQ3cf9/95GQUkOpP333/wO6jSEvJ4J577tEHkL1Ya3n++ec599xzmDdvHsPysjisKJ+CtBTXa+mZkcbhRfkMzMnguWef5dxzojXFQ8ulApjDmrfGKIC5IxQK7b4dT592EpmuubsikQh1tbUYoCgSYfXKlV6XlLAeffRRNm7ayKT+x+wxWDzFn8a4vkezdOlSXnjhBQ8rjC8bNmzg8ssv59ZbbyUtHOLwXgWutXq1JuDzMbIgh0N6FRCurea6667jl7/8Jdu3b/esJlAAc1zzT0bV1dUeVpI8mocBzQ5zR/Nr3ny/PHHG+vXriViLAfoAGzdvVvB1wLvvvsu//vUvBnQbRa+8wV95fHCPsRTmDuBPf/pz0i9LEQqFePTRR5l5wQV8unQpY7rlUNwzj6wWNtT2Sl5qCocU5nNAfhbvv/ce5517Ls8995xnrWEKYA5r3gKmAOaO5mFASyO4Q9fcXf/73/+A6Av4gL3uk9hYuXIlv7n+N+RnFlI86PgWn2OM4dAhp5Dmz+QXP/8FmzZtcrnK+PD555/z/e9/n/vuu4+CFD9TeuXTLzuj08tLLC+rpLIhxI6dpXy4rYzlZZ3/kOEzhsG5WRxelE+6DXPbbbdx5U9/ytatWzt97HbX4voZk4xaY9zXfHp+V1iMLxE0v+Y7duzwsJLk8M477+AnOgB/AJBmTNyvedSVvPfee/zoR5fjJ5Ujhp1BwN/6cgnpKZkcMewMamvq+d53vxeXs+2cEolEePzxx7n44ov54rM1jO+ey8QeuaT5YzPDsbIhRMhaGoJByuqDVDaE9v9DbZSZEqC4Zx6jCrJZtOgTzj//fF588UVXW8MUwBzWPICVlpZ6WEnyaGhowBjICBhNz3eJlkRwT2VlJQs++oisxu8DGIZZy/x58/Z4vZH2s9byxBNP8Itf/IIMfy7TR55DRrOZj63JzejO9JHfJhL08eMrfswrr7ziQrXeKikp4ac//Sl333033VL8HFZUQO+s9JgsquoWYwwDcjI5vKiADCLcfPPNXHfdda41liiAOSwYDIIxGF9AiyW6pKG+Hh/QMyPMli1bvC4nKdTV14EBX5aP9evXe11OQnvllVcIhkI0jwWTgPKKCt555x2vyuryNm/ezI9//BP++te/0id/KNNGnN2m8NUkJ70b00eeQ35GL2688Uau/fW1Cfuh+4MPPmDmzJn8b9EixnTLaWz16rpxIiPg56CeeRyQn8Vb8+dz4cyZriwv0nWvWBfRtAKvTc9J2vEBbrLWUt/YAtYzPcTmTVofyWnWWmprasFAODfMylWakeeUYDDI4489Rj9jaN4pNgzI9/mY9fDDcTG9visJBoM8+eSTzLxgJkuXLOPAgcdz+NDT9tnt2Jq0QAZHj/g/xvU7irfffofzzj2Pl156KWE2Sw+Hw/zjH//g6quvxtdQz6FFsRnrFQ9M49iwg4vyqanYxQ9/+EOeeuopR3+fFMAcZK1tXAjUEErLY+3nX3hdUsIrLy8nEolggF6ZETZv2aJuGYetW7cuOtvXgO1m2bB+AxUVFV6XlZCeffZZtpaUMH2vNwU/hqmRCCtWrdL2OG3U0NDAf//7X771rW9z1113kZ/em+NHz2Ro4YROBQqf8TGq9yEcN/p8Usnhpptu4vzzL+CVV17p0q9FkUiEW265hQcffJA+mWkcXJgfVzMcYyUvNYVDi/Lplhrgzjvv5IEHHnDsXApgDtq2bVs0PRtDJLOALZs3xe2eVIli7dq1ABgD/bLCBIMhtTw67MMPP4zeMGALLdZaFixY4G1RCWj79u387YEHGIJhWAuPTwR6GcNdf/6zFiDeh+rqap588knOOuts/vjHPxKu8XPk8DM5cviZZKXlxuw8TePCDhs6g4qdtdx4442ce+65zJkzp8u9D1hrufPOO3nppZcYmpfF2O65+H1dv9WrNSk+H5N65NI/O4NZs2bx8MMPO3IeBTAHLV26tPGWIZLVA2sty5cv97SmRNcUwHxY+mVHm/0/++wzL0tKeG+/8zY0TXrqBibN8Pbbb3taU6IJh8P84eabCdbVcSoWw1ff/PwYTrWWnaWl/PnPf1ZXZDPWWhYvXsxNN93Eaaedxl133YWvIZ2jD/gm00d+m975QxzpRjPG0L/bCI4ffQGHDzuV2l0hbrnlFk499TTuuOMOVqxY0SX+nebMmcPTTz/NoJxMhuZmel2OK4wxjCrIpndmGn/7298cmWWceO2HcSS6Lo8BYwjnFO2+b+LEiZ7WlciWLFlCwAcY6J8dJtVvWLp0KdOnT/e6tIS0bds2Fi9ajM1ofBPxQbhvmPlvzaempobMzOR4sXbavffey0cLFnAq0K2F8NWkH4apWF555RWGDRvG2Wef7V6RcWjbtm28+uqrPP/882zevJmUQBr9C0YwuMc4umf3ca0OYwz9Cg6gb/5wSirX8/n2JTz37PM888wzDBk8hJNOPoljjz2WgoIC12pqqx07dvDXv/6VbumpHJCflRDjvdrKGMPY7rlUhcq5/bbbePiRR2L6mqYA5qCFH3+M9TfufRVIh6xufPJJdL0RiT1rLYs++ZhMf3T3gYAPhucF+eTjjz2uLHE9//zz0U/wX26Rhx1oaVjbwCuvvMJpp53mWW2J4sEHH+SJJ57gUKB4H+GryVSgBLj3nnvIycnhpJNOcrjC+LJ9+3bmzZvHG2+8sXsmW8+c/hw8+AT6FRzQocH1sWKMoSh3IEW5A2kI1bG+dAVfbF/KX/7yF/7617uZNGki06dP56ijjiI/P9+zOpubM2cONTU1TO7dzfXwFYpEyMjI4KSTTmLOnDmEQu7vM+szhlH5WXxYsoO5c+fG9PdJAcwhmzZtYsP69di0nN33NeT2ZdGiRVRVVZGd3fbpzdI2q1evpnxXBb0zv2zSH10Q5N+frWXHjh306NHDw+oST01NDf9+8t/Yvhaavy52B7rBrEdncfLJJxMI6GWmI6y1/OMf/+Chhx5iAvD1Nv6cD8OZWBow3HLLLdTX13PGGWc4WarnduzYwZtvvskbr7/BkqXRhVALsgoZ2/dIBnQbQXZ6/LUspQbSGVY4kWGFE9lVs531pStZvXwlH398O3fccQeTJ0/eHcZyc2M3Nq29amtr8RnjyYD7YMRy8kkncfnllwPw/NP/cb0GYPffPTqpLnb0yuiQpjEwNvBl00C4YCDhLUv44IMPOOaYY7wqLWHNmzcPn4HsFEvTENfing38+7MM3nzzTc4880xP60s0s2bNoqa6hsihEXz/azac1EB4ZJht727jueee4/TTT/euyC6qpqaGm37/e+a/9RaTgVOJBqu2SsHwLSyPA3/+859Zu3Ytl19+Oamp3rX+xFpVVRXz58/n1Vdf5eOPP8ZaS35mT8b2PYJ+BSPIzegW0/N9sv4NymtKMPWWuVWzyc8sZNKA2AxtyMvsybjMnoztO4VdtdvZULqSlUtXsmDBAv74xzs45JCDOe6445gyZQrp6en7P2AMZWVlEbGW0roGuqW7+/8nxWeYM2cOEG2JS/Fo4P/OuugnzIyMjJgeVwHMIW/Onw9Z3cD35ZYMkexCTGom8+fPVwCLMWstc994ndEFQUJf7n9O3+wI/bItc994QwEshtauXcusR2cRGRiBlt7n+gBFcN/993HEEUfQs2dPt0vssr744gt+/atfsXHDBr4OHA4tDrrfnxQM38byGtHlK1avWsVvf/c7evXqFeuSXdPQ0MB7773Ha6++xrvvvUswGCQnvYBRvQ+lf7eR5GU418pdXlNCMFwPYdjesMGRcxhjyM8sJD+zkLF9j6CsZhsbSlewaOES3n33XdLT0jnyqCM57rjjKC4udqV1+YwzzuCll17kf1u2cEhhPhmB2Gwz1BYBn4/K2lqefPJJAArSUlw7d5OKhiCfllUxZsxojjvuuJge2/NZkMYYvzHmE2PM817XEislJSUsXbKEhoLBez5gfDTkD+Cdd9+ltrbWm+IS1NKlS9m0eQuH9frqGIHDiur435IlbN682YPKEk99fT03/v5GbMBiJ7Qyg8tAeHKYuvo6br755oRZiNJpL774It+5+GJKN23iAmAKpkPhq4kPw/EYzgbWrlzJRRde2CVXyw+FQjz//PN86+xvce211/LB+wsYVDCOY0ady9fHXszYvkc4Gr68YIyhW1YvJvSfyonjLmXqiLPok3sAb859i6uvvprzz7+AuXPnRtfgc1BWVhY33/wH/CmpfFiyi9I698dheWVbTT0LtleQm5/PDTfcSEpKbAOg5wEMuAJIqLUZ5s2bB0Co25CvPBbqPoSG+nptnBtjc+bMIT1gOKTwqy8OR/SuxxB9c5PO+8tf/sKa1WsIFYcgbR9PzIbwxDALFixwbB2dRFFVVcUNN9zAzTffTJ9QiB9EIgzpRPDa2xgM37eW3JoarrnmGu68884usRZVJBJh7ty5nH/+Bdx6661E6gIcOfxMTh7/PSYNPIbu2b2TYlaez/gozB1A8aCvccqE73PY0BlUltZy/fXX853vfIcPP/zQ0eUsBgwYwL333UePXr1YsH0XayuqiXSB5TM6KmwtK8uqWLRjF0OGDeO+++5zZAyxpwHMGNMPOAn4u5d1xNrrr7+BzeqBzcj7ymORnF6YtCzeeOMNDypLTDU1Nbzx+uscUlhHegst8t3TLeO6B3nxhecd/7SY6J5//nmeffZZIiMj0W7G/bCDLZGBEf71r3/x7rvvOl9gF7RkyRIuvOACXn/1VaYDM60lJ4bhq0l3DN+xlkOBp556iksvuSTu18h7/PHHuf7666ksrWXKsNOYPvIceucPwWfioe3AG35fgP7dRnDc6As4ePAJbNlQwlVXXcVrr73m6HkHDx7M3//+d44++mhWl1fzQUk5ZQnYGlZSW897W8v4orKGGTNmcPfddzvWbW+8XATOGPMkcDOQA1xlrT25hedcClwKUFRUdODs2bPdLbKdQqEQn332GTY1C5uSwevPPUmKz3DUSV+OPzIN1fhCdQwfPjwpPr05raKigi1bttAnK0yaHx767+tETCozZxy5+znVQUNJrY/+/ftrbaoOqq2tZcOGDdiAhb0m8b76n1dJ8aUw9bSpLf9wJfgiPgYOHJhQA8E7a8eOHZTu3IkfQwGW9lyZx159FZuSwrenTm33eeuAcmOIAD179ozL9acANmzYQENdkJyM7g5E0vb5z3OPs2nLl3vL9u3djzNOOcvDiqIsll21O8jOzqJ3796unLOqqopt27YRCoVI8flI9/vwOfBe9syLL7J567bd3/fpVcRpJ5wQ8/NAtNWrLhwhFImQmppKUVFRTN4rpk2bttBaW9zSY54NwjfGnAyUWGsXGmOmtvY8a+0DwAMAxcXFdmoHXmzc9NRTT0Vnh43/BjYjn/SqML2z/fx9cfXu5/h2bSJjxYvceOONHHXUUR5WmxiuuOJyNq9cxO2HlWEMZNZkU5PZh5Gr7939nIYw/PDtbkyZeizXXnuth9V2TVu3buWS71xCZaSS0PQQVO/5uC/ooyiliIe2PdTyAWog8EaAovwi/vbA38jL+2rrcDLZtGkTv/3Nb1ixciUTiXYDpLczYvixhAsLyZn1aLvPnwNkYHkGWAkcVFzML3/1K7p3797uYznpxBNPIj+lN4cPO9XrUmioMl/5ftsn3rfEWWt5Y/lrZPdI5eGHW/n9c0BdXR2zZs1i1qxZRCIRBmWnMyg3i0AMZyrWlpXt9X0pn73+QsyOD9AQjvBZRTUbqupIT0vjwosu4swzz4z5eK+WePm/ZwowwxjzBTAbmG6MecTDemJi/vy3ILMAm5Hf6nMiub0xKem89dZb7hWWoDZv3swnnyziiF617OsDWKofDi2sY/6b87RPXjvV1NTw81/8nMraSkJTQrSrmaZJJoQOC7Ft2zauve7aLr0pcWe9+eabXHzRRaxbtZr/A87EtDt8xUI2hnOAU4BFCxdy0cyZfBxnixZPmzaVjWWr2Fy+xutS4tba7f9jZ/Vmpk+f5up509PTufjii3lk1iyOPvpoPquo4Z2tZWyqqu0S2ytFrGVdZbTmDVV1nHTSSTw2ezZnn322K+ELPAxg1tprrLX9rLWDgLOBN6y153pVTyxUVVWx+H+Lacjrv+8nGh/B3H688+57mh3WSS+88AIGOKrP/gcUH92nnvqGIK+//rrzhSUIay0333wzn3/+OaFDQtGmk47qDuEDwyz6ZBF33313zGrsKiKRCPfccw/XXnst3erq+IGNMM7jjjWD4WAM37WWQEUlP/3JT3jsscfi5g308ssvZ/jw4Xz4+Qus3+ntvonBcD0ZGRl84xvfICMjI7okhYestawp+YRFG17noIMO8myHld69e/Ob3/6Wu+++m0HDhrG0tDI6Pqw+fseHba+t571t5awoq2LcxAn885//5Oqrr6Zbt9iuHbc/3refJpAPP/yQSDhMuGDAfp8bKhhAVWUFn376qQuVJaZQKMQLc55nXPcg3dP3/8I8JDdM/5wIz/73mbh5g4l3s2fP5s033yQyLgIxGIdqB1kiwyM89dRTvPrqq50/YBcRDAa58cYbmT17NgcDF1tLvuejmr5UhOG7NsIoa7n33nu5++6742LCSlpaGjfddBNDhg3m/bXP8e5n/6UuWL3/H3RAMFTPSY2rsp900kkEQ94FsKr6ct5c9QQfr3uNSZMncf311+P3u7c+V0vGjRvHfffdx69//WsC2bl8uK2cxTt2URfqeCNDTmqAgDGkpqRQkJZCTmrnRk1VB0Ms3F7Ox9t3kdejJzfffDN33PEnhg4d2qnjdlRcBDBr7byWBuB3Ne+++y4mJZ1IduF+nxvO6wfG1yXX5IkX8+bNY8fOUo7r17YXQmPguL61rF7zGYsXL3a4uq5v4cKF3Hfffdh+FntA7AKrHW+hJ/zhlj+wZk3idy2Fw2Guv/56XnvtNY4DTgYCnQxfL2DZApSUlfEPLC/Q+X+fNAz/BxwKPPHEE9xxxx1x8UGlqKiIe+65h+9+97tsq/ycl5f9i8+3L3G9tpRAGnPmzOGuu+6Krsoe2NcaLM6IRMKs3PoRryx7kMrgdn72s5/xxz/+0dOtiprz+Xwcf/zxPProo8ycOZOdwQjvbitjXWVNh/69RhXkkJMaoEf3bhxcVMCogo41wUesZc2uat7dWkYNfi677DIeevhhpkyZ4ulEuLgIYIkgFArxzrvv0dAYrPYrkEo4pxdvv60A1hHWWh6f/Ri9sywTegTb/HNH9G4gJxWeePxxB6vr+rZu3cq1110LuRApjhDTxhofhA8NEwqEuOaX11BRURHDg8efe++9l7fffpsTgaM6ubBqky1APVAfDPJF4/ex4MNwInAE0dXzn3jiiRgduXMCgQDnnHMO//znPxk+YigfffESc1c+RnlNiWs1pPjTqG1clb22tpYUv7sBbEflRl5b/jCLN8zjwOJJPPTQQ5xyyilxOZM+IyODiy66iIceeoiJkw9kRVkVH5SUU9ng/tjPsvog720r57Nd1UydNo1Zjz7KWWed5do4r31RAIuRZcuWUV1VSTh//92PTUL5A1i/fp1WaO+ApUuXsnLVar7Wv4b2TLpJ9cP0vrW88+47bNq0ybkCu7BgMMivfv0rquurCR0WAidep9IhdGiIku0l/O6G38VFS4sTXnvtNZ544gkOAQ6Loy7HfTEYjgPGAPfcc09cDcwfNGgQf/3rX7nmmmtoMJW8+unDfLL+DRo87A50Wl2wmg8/f5E3VjxGSibceOON3HrrrRQVFXld2n717duX22+/nd/85jeYjEw+KClnfQdbw9rLWstnu6r5qKScrPyC3XXE00xfBbAYeeedd8Dni3YttlHTWLGmjbul7Z544gmyUqItWu11bL96fESXDJGv+te//sXqVaujK913ZtD9/nSH8PgwH37wIf/5z38cPJE3tm7dyu233UZ/Y3Bm5SLn+DCcQXTx1ht/97u4aqX0+XyccMIJPProo5xyysmsKfmYl5b9g893LE2oIB+JhFm1dQEvLf0HG8pW8O1vf5tHZj3CUUcdFZetXq0xxjB9+nQefPAhDiwuZnlZFYt2VBCKOPdv1RCOsGD7LtbsquaYY47h/z34IAcffLBj5+soBbAYsNYy7835hHP6QKDtc/Rtei5kdmP+/PkOVpd4tm3bxltvvcW0PnWkd2DcaUGa5ZCiel6Y8zw1NTWxL7AL+/TTT6Pr+gyOQF/nz2eHWmwvyz333MPGjRv3/wNdyL333kuoro5vWIu/i7R+NZeK4Rs2QmlpKQ8++KDX5XxFbm4uV111FQ888ABDhg3io89f5I0Vj1JWvW3/Pxzntldu4NXlD7Fow1wmTBrPgw/+P773ve+RkZHhdWkdVlBQwK233spll13G9roGPt6xi6ADEz3qQmE+2r6LyrDlmmuu4dprryUrKyvm54mF/QYwY0ylMaZir68NxpinjTFf3ewwCX3++eds3bKZUMHAdv9sQ8FAlixZQnl5eewLS1AvvPACkUiE6W0cfN+SY/vVU1Nbpy2hmrHWcuedd0I6rW+yHWsmOsYsZEPce9+9+39+F7F69Wrmzp3L4dbSrQuGryZ9MUwEnn76aUpK3Btv1R4jRozgnnvu5pprriHsr+G15dFuSa+XieiIumANH659gbkrZpOW7ef3v/89t99+OwMGtH1oSzzz+XycddZZ/O53v6MiGGbB9l00hGMXwmobw1fI5+ePf/wjJ5xwQly3FralBezPwM+Ifh7uB1wFPEp08dR/OlZZF9LUhRjuQAALFwzEWqt98tooHA4z5/nnGNstRGFGx39xh+eF6Zdtee7Z/8awuq5t7ty5LF++nPCYsDPjvlqTAeERYd6a/1bCzE6dO3cuPuBwrwuJgSOJTjKK56ESTd2Ssx6dxYwZM6Ldkkv/yaay1V6X1ibWWtZuX8LLy/7BhvKVnHvuuTzyyMMceeSRcR0gOuroo4/mlltuoTZs+WTHLkIxaAmrD0dYuKMCk5LKnXfeycSJEztfqMPaEsBmWGvvt9ZWWmsrGrcG+pq19nEgPjcQc9k777yLze6JTW3/vlGRzO6YtCzee+89BypLPB9//DEl23cwtW9dp45jDEztU8vyFSv5/PPPY1Rd1xUKhbj/gfsx+QY7yP1xNHaExWQa7r3v3oQYx7Pok0/oawwZXbj1q0l3IN/n6xLhOCcnhyuvvJL77ruPPv178c6aZ3j/s+epD9V6XVqrahoqeXv1Uyz44iVGjDqAf/3rn1x66aWkp6d7XZqjDj74YH7b2BK2aEcFkU783ocikWiXJoZbb7uNUaNGxbBS57QlgNUYY/7PGONr/Po/onu5AjFYfKaLKy8vZ8WK5QT3t/p9a4yhIbcfH3z4IcFg25dTSFYvvfQSWSmGye1YeqI1h/VqwG/g5ZdfjkFlXdtLL73Els1bCI0JtXvJCbPIQDmU7SjDN88X/b69/BAeGebTZZ/y/vvvt//n48yu8nLyEiBIQnRWZF4kwq5du7wupc1GjRrF3/72ABdddBGbdq3ilWX/jy3la70u6yvW7fyUV5b9P0rrNnPFFVdw1113MWjQIK/Lcs0RRxzB1Vdfzc66BlaUdWyLOGstS3ZWUtkQ4oYbbmDcuHExrtI5bQlg5wDnASXAtsbb5xpjMoAfOlhbl/Dhhx9irSWc38EABoTzB1BXW8vSpUtjWFniqamp4a35b3JIYR2pMVj0OS/VMr57kJdfejGpt4Sy1vLY7Mei7dm92//zptxggoZgQxCz3WDKO9bqYwdHW8HiZe2pzkiEVry9dbW/UyAQYObMmTzwwAP06tuTt1Y/xeIN84hEOva7np9ZSIo/jdTUVHrm9Cc/c/8LbrcmFG7gw89f5IO1czhg5DD+9a9/ceaZZ+LzJd+8uBNPPJGzzz6bDVW1bKxqf0vl2ooaSmrrueyyyzjssMMcqNA5+/3XttautdaeYq3tYa3t2Xh7jbW21lobv4MCXLJgwYLo6vdZHV9bJJzbG4xhwYIFMaws8cyfP5+6+gaO6B27wbVTetezs7SMRYsWxeyYXc3ChQvZsH4DkWExXnC1vXwQHhpm4cKFXb5bOCc3Fyc7vepgj30JO9ch34bz+Xzk5Di5Jolzhg8fzgMPPMCMGTNYufUj5q58jJqGynYfZ9KA6eRnFtKzeyHTRp7NpAHTO1RPRe1OXl/+COt2LuP888/nrrvuol+/ti9flIi++93vMmnSJFbuqqa2HVsXVTQEWVtRw7HHHss3v/lNByt0RltmQfY0xvzSGPOAMeafTV9uFBfvrLV8+NFHBHN6t231+9YEUolkF/LBhx/GrrgE9Oqrr9AzMzqAPlYm9wiSETBJtS/h3p757zOYNIPt730Lhx1sMX7Ds88+63UpnVLUqxc7fD6sQ6M06mCPfQmdDGBBLGXWUljY8RYfr6WlpXHVVVfx29/+lppwOW+smEVZjfvLVZRUbOCNFY9iUsPccccdXHLJJQQCndvfMBH4/X5+/vOf4/cHWN7GrkhrLZ+WVZGTm8uPf/zjLjlZoS2p4b9AHvAaMKfZV9Jbt24dpTt3Es7r/IJJodw+rF61isrK9n8ySwZlZWUsXLCQwwprieXvWaofinvWMW/uXBoa2r+oa1dXWlrK22+/TXhgGLzdyzcqDcJ9wrz40ovU13e9ZQSaHH744eyKRHBqZbN02GNfQieHa68CGqxlypQpDp7FHdOmTeOee+4hMyeNeStms2WXey2t63Z+yvzV/6ZXn0Luu/8+DjzwQNfO3RX06dOHiy6+mO219eys2/9r8ZaaOnbVB7n88svjZi/M9mpLAMu01v7cWvuEtfappi/HK+sCFi5cCEA4t/MBLJzXF2vt7mPKnubPn0/EWg7rFfuQdGhRAzW1tXz00UcxP3a8e+2114iEI9jB3rd+NbGDLTXVNV16o/ojjjiCzIwMnFpiOR322JfQqQBmsbxlDD179GDChAkOncVdQ4cO5f7772fAoAG8s/o/rNv5qePnXLl1AR+sncP48eO499576dOnj+Pn7IrOOOMMevbsyepd+96uKGItn1XUcsDw4RxzzDEuVhhbbQlgzxtjTnS8ki5owYIFkJGHTe/82IhIViEmkKJxYK14443X6ZNt6ZcV+5WTx3QLkZVqknJR1pdefgnTzUA8fYAsBJNpeOWVV7yupMOys7P59jnnsAJY34Uniy8DNlnLJd/5TkJ1lfXs2ZO//vUvTJgwgQ/WzmHVNmc++Fpr+d/G+SzeMJejjz6a22+/vcuOpXNDWloa5513HrvqG9i1j427t9fWUxMMMfPCC7v0xIW2VH4F0RBW27gKfqUxJn42BvNIJBLhk0WLCGb3is0BfT6C2b345JNFsTleAqmqqmLx4v9xYI+6mHY/Ngn4YFL3Ot5/792kmg1ZVlbGmtVrCPeNs7+ziXZDfrTgoy69NMs3v/lNuhUU8LIxjo0Fc1IIy2s+H4MGDuT444/3upyYy8rK4tbbbuWoo45i0fo3WLk1ti3g1loWb5jHii0fMGPGDH7zm9+Qmtr2reqS1fHHH096ejob9jEjckNVHT179Ohysx731pZZkDnWWp+1NsNam9v4fTx9XvbEF198QU11NZGcGAUwIJLTiw0b1mtbor0sXLiQSCTChO6tfyJq8vDKDNZV+tm2o5wbF2Tz8Mq27Z02oXuQyqpqVq5c2dlyu4ymmZ+2MP7CgS20BBuCLF++3OtSOiwjI4OLL7mE9dbysdfFdMBbwM5IhO//4Af4/fEwQDD20tLS+M1vfsPUqVNZvGEeq7bGpgci2vL1Jqu2LeD000/nyiuvTNhrGGuZmZkcf/zxbKttaHGvyJpgiJ11DZwyY0aXv6atBjBjzMjGPye39OVeifFpyZIlAIRzimJ2zKZjaT2wPX300UdkBAzD8vYfwNZV+qkN+6hvCLKiPIV1lW37BR3XPYRpPFeyWLNmTfQVIN/rSlrQLfrHmjVrvK2jk0488UQmTZzIHGPY2oVawdZimQscc8wxHHrooV6X46hAIMB1113H0UcdzaINc9lc/lmnj/n5jiWs3PoRp512WpedoeelGTNmEI5E2Fz91fm9G6vr8Pl8nHTSSR5UFlv7agH7aeOff2zh63aH64p7q1atwqSkY9Na789PXfcevpqdlO3cTvqnz5O6bt/bDUUyewBd/00n1pYu+R/D8hoIONjVn51i6ZdjdwfrZLB582Z8Wb62DURwWzqYgGHz5s1eV9Ipfr+f666/npy8PGb7fNR0gRBWhuUJn49+ffvys5/9LCnCQyAQ4NfX/pqhQ4by0RcvUF3f8VX/y2q28cn61znwwAO54oorkuL6xdoBBxzAAcOHs7lmz5nQ1lq21DZw6KGH0rNnT4+qi51WX3qttZc23jzBWjut+ReQ9IPyv/hiHeH0PPY1KMlXvRMTDhJsaMBfuRVf9c59H9QfwGTksm7duhhX23VVV1fz+RfrGN6G1q/OGpbbwLKlS4jEYGPYrqC0tJRwapyN/2piwKQbysrKvK6k07p3786NN91Ehc/HYxhCcRzC6rE8agwmPZ2bb7mFzMz272/bVaWlpXHDjTfgD/j4YO3zRGz7XwdC4QY+WPs8+fl5XHfddV2+i8xLJ5x4IhX1QcLNZkOW1gepC4b4+te/7mFlsdOWz77vtvG+pPL5F58TTs+P+XGDaXl8tjb+9izzyrJly7DWuhLAhueFqK6p5YsvvnD8XPEgGAzGx9pfrfFHNwlPBGPHjuXnv/gFX2B5BuJyUH4Yy2wMJcbw2xtuYMCAAV6X5Lp+/fpx5VVXsqNqM8u3tH9P0kUb5lFZV8a1111LQUFB7AtMIocccggAociXvytl9UGMMRx88MFelRVT+xoD1ssYcyCQYYyZ1Gz811QgeT4WtaC6upqqykpseuznItj0XLZs2dLl9l1zypIlS/AZ2jT+q7NG5Id2nzMZGGOISQ4I7rktDrGauGhJqO6b448/nosvvpjFwKvEVwiLYPkvsAbLVVddxUEHHeR1SZ457rjjOPbYY/l083vsqtne5p/bVrGOtdsXc9ZZZzF5ctIPk+60vn37kpWZiQ/o0S06KLSiPkj//v0TpmV2Xy1gXyM61qsfcAdfjv/6KfBL50uLXyUlJQBEUrNjfuxIahb1dXVUVXVsZ/hEs3jRIgbkRMhwYQmiwowI+emwePFi508WB7KysvCFYzAALLjntjgxC2ChaI2J5Pzzz2fGjBm8BbxOfISwpvD1CXDRRRdx8skne12S56644gqys7P5eP1rbfowHI6E+WT9a/Tu1ZuLL77YhQoTnzGG0WNG4/P5OOKQQ7DWsisUZuzYsV6XFjOtvq1Zax8EHjTGnKmV7/fUFMBsWuzfHGxjqCspKUn6Bfvq6+tZtmwpx/VxZ4sgY2BkXgMfL1yAtTahWl9akpeXh6mPwd8xJbotDjT+mdb5QxKBSF2ky24x0hpjDD/96U8Jh8PMmTMHA0zHYtq5C3pvYAtASgq9g0F6d7CeCJZngY+BmTNnMnPmzA4eKbHk5eXx/e9/j1tvvZWNZSvp323kPp//WcknVNSW8qvf3EJaWix+AQRgzJixLPhoARZLTShMQyjM6NGjvS4rZtqyDthTxpiTjDFXG2Oua/pyo7h4tXNndDC9TYl9M2jTMZvOkcyWLl1KMBRmTDf3FuMc0y1IaVl5UkyE6NevH5HqCHS2dzdlz21xSIlBcTVABPr37x+Dg8UXn8/Hz372M0466STmAR3Zf+FEDL2BwoICLsZwYjsDHETD13PAQqItcxdeeGEHKklcJ5xwAgMHDOTTze/uc0B+KNzAim0fMmnS5C6/MGi8OeCAA7BA2EJlMPpCNWLECG+LiqH9BjBjzH3AWcCPAAN8ExjocF1xrbS0FACb0rZFPtvDpkaPmQizvzpryZIlGOCAfPcGYo8qSJ5xYAMHNv4ax+P+7417bSRiAIOvhrD5LndFWiwvAguA8847j4svvjjhW3zby+/3c/ElF7Ordiebyla3+ry125dQ11DNJZeo6zHWhgwZAkT3fqxqCGGMYdCgQd4WFUNtGQByuLX2fKDMWvtb4DDgAGfLim9lZWWYQAr4Y/FRf082EA1gTSEvma1atYpeWdaV8V9NCjMiZKQYVq9u/QU3UTS9kJmK+HvjbaopkV5s9+bz+bjqqqs45phjeBX40MUQ9gbwPvB///d/XHLJJQpfrTjyyCMpLCzis+2LWnzcWsvaHYsYNWo048aNc7e4JFBUVEQgECBio12QPbp3T6gu3rYEsKalaGuMMX2IDrHt6JCDhLBr1y4IxL71C4iGOp9f2xEBq1etZGC2u3sB+gwMzAqyetUqV8/rhX79+pGSkgJxmPVNqaFHzx4JPw7S7/fzq1/9isMOPZTngI9dCGFvYplHdJX+yy67TOFrH/x+PzNmnEJJxXqq6r7aK7GjahMVtaWcdtqpHlSX+Px+P4WFhUQs1IUj9OnTx+uSYqotAew5Y0w+cBvRsZpfAI86WFPc27VrF5GAQ5uqGoNJSaeiIun3O6e0rIwe6e4vitotPUxpaeKPwQsEAkyePBn/tjhbDCwCvu0+Djn4EK8rcUUgEOB3N9xA8YEH8gyw1MEQ9j6W14Bjjz02aVa576yvfe1rAKwvXfGVx9aXLic1NZWjjz7a7bKSRo8ePbBYgha69+jhdTkxtc8AZozxAa9ba8sbZ0IOBEZaa5N6EP6uXbsI+51rBrWBtKQPYPX19QSDITID7k/TzwrYpFkG5PDDD8dW2vhqBdsCtsFy+OGHe12Ja9LS0rjp5psZM2YMTxnDOgdC2KdYXgCOmDKFX/7yl1qlvY2KiooYO3Ysm8r3bBW31rKpfDWHH354wqxLFY/y8/OxQEM4Qn5+vtflxNQ+A5i1NgLc3ez7emttxzfJShCVVVXgd6gFDIj4Uqiurnbs+F1BXV205zvN734AS/NDbe1XN4FNRF/72tfIys7CtzxONoS04F/pp1fvXkk3oyw9PZ2b//AHevXpw6M+H6UxDGGbsDxpDCNHjuS6668nEHBxYGUCOOywwyirLiEnrYAe3QsBqKjdQV1DddL9P3Vbfn4+EQsN4XDC7S7Qllfd140xZxq1Ve9WXV2DdTiAVVUldwDz+aL/Nb1YpjIC+P1xEkgclpmZyTe/8U3MZgPx8NGqBNgJ53z7nKQMCXl5edx62234MjJ41Pioj8FvQBWWx3w+Crp35w+33EJ6enoMKk0uEydOBKBX3mCOOnwaACWVGwCYMGGCV2UlhYKCgt2L4SZVC1ij7wL/BuqNMRXGmEpjTFL3j9XW1jgyA3I3f2q0lS2JNb35hjzYFzsSIam6Z84880zS0tMwy73/jOVf7qdb926ccMIJXpfimX79+vG7G25ge+MiqZ1ZLT+C5QkMdX4/N/3hDwnXguCWkSNHkpaWRknl+t33lVSso6iwKOEGhseb5qEr6QKYtTbHWuuz1qZaa3Mbv0+s5anbIRKJUF9Xh3UwgFl/ILqgZRJLSYle31CkfaGgNmT22JewNtT+UBGMGFJSkqf1JS8vjzNOPwPfRp+3a4Jtj359+1vfJjXVuRbmrqC4uJgLL7qI/wGLOnGct4DPsVzxk59wwAFJvXpQp6SkpDBhwgS2V0UDmLWW7VUbKT6o2OPKEl/zDw1JF8BkT03ByMkAhj812sqWxAKBAD6fj4Z2toDVhMwe+xLWdCCANUQgLTVx1pppi7PPPpuUlBRPW8H8y/3k5ecxY8YMz2qIJ+eeey4Txo9njjGUdaAVbDOWN4Bp06ZF9+iUThkzZgy7akqx1lJdv4uGUB1jxozxuqyE13yCQ3Z27Pdf9pICWDvV1DQGI5+TLWAp1NfVEYl40P8WR9LTUmkIty8QZAYsc+bM4a677mLOnDkdmkVZHzakJdk4mYKCAk4/7XR8633ejAUrAbZFx35pjFKU3+/nl7/6Fb60NJ4xpl1dkWEs/zE+8gsKuPLKK7XcRAxEV2W3hG2IXbXbm90nTmr+epBorw0KYO3UNDvRyUH4Ta1ryd4NmZmZQW07A1hGwO6xL2FGBwJYXdiQlRX7jdbj3XnnnUdWVhb+xX53Zz9EILA4QGFRIaeffrqLJ45/vXv35geXXcZaa1nWjp/7ANhmI1z1s58l3IbmXmnauiscCVHZuChrom6VFU+aD0doGpqSKFoNYMaYbvv6crPIeLJ7eQinFmKF3UtcJMtaVK3Jyszq0BiuzqoNGbKyE3sF9pbk5eVxycWXwDaiXy4x6wy23PLDy36YUNuMxMrJJ5/M0CFDeNnnI9SGZFyDZa4xHHTQQUyZMsWFCpND797RDWAikTBV9eXk5OQm/E4N8aD5a0KijQ3dVwvYQqJ7tS4kOjx2FbC68fZC50uLT01dkNbhLkhQC1hufgEVDe430laEAuTl5bl+3ngwY8YMevTsgX9521vBbL7FplhSUlOwPS02vx3NZ5Houl/DDxiu1cRb4ff7+cFll1EeibTphfcdoB74wQ9+oK7HGEpLS6NbQTciNkxN/S769E7qHflc07zbMSPDoS0APdLqu5u1drC1dgjwGnCKtbaHtbY7cDLwilsFxptQKBS94WtDMAg37DEjj3BD205i/HueK0n17NmTsqC7sxGthbK66PYXySglJYVzzzkXdtDmVjA70UI+FPQoIDI1Ev2+jcw6g620zLxgpsLCPhQXFzNh/Hjm76cVrAbL+8YwbepUhg4d6mKFyaGwqJCIDVMbqqKwqNDrcpJC8wCWTC1gTQ611r7Q9I219kUgefYI2Us4HI7eMPu/dCbUsMeMPBNqawCLvhEpgPWktA4iLo5HqgoaGsLRcyerk08+mV69exFYHIiuSuuUIPiX+Rk9ejRHHHGEgyfq+owxnHf++VREIizex/M+BBqs5bzzz3ertKTSs2dPIjZCbUNVUr9GuKl5AEu0D2ltCWCbjTG/NsYMavz6FbDZ6cLiVdOKvG16biB1jxl5ts3jxqL/yZJ9FmS/fv0IhqG03r1fuq010V+JZB5cm5qayuU/uhxbYTGrnLv2ZpnB1lquuOKKhHthdcJBBx3EkEGD+KiVaxXBssDn4+CDDlLrl0N69OhBxIZpCNUlbSu52xKt1au5tgSwbwE9gaeB/zTe/paTRcWz3X3Q4Ta0TvlT95iR1+b9IyNBgKTf4LVp1tGWavdWpd9cEz3XgAEDXDtnPJoyZQpHHnkk/mV+Z5alKAHfah+nn346o0aNcuAEiccYw4knn8wmaylpoRvyc2BXJMKJWvPLMd26dUvYbXHiVSLvStKWlfBLrbVXAEdYaydba39srS3t7ImNMf2NMXONMZ8aY5YZY67o7DHd0BTATGNIcoIJB/c4V7JqCmCb3Axg1X5SAn569erl2jnjkTGGq666ipycHPwLY7wsRRgCCwP07tOb733vezE8cOKbPn06xpgWl6RYBqSnpWnmo4OaT85RAHOHry3jrbuo/f7NjDGHG2M+BZY3fj/BGHNPDM4dAq601o4GDgUuM8aMjsFxHbV7fai2jufqiMbB+sm4FlVz+fn55GRnutsCVu2jb9++SbkR9N4KCgq47AeXwU4wG2PXRWg+M9gqy1VXXpX0HzLaq0ePHowZPZrle3VDRrAs9/k49LDDtJSHg5qvqab11aSz2hIt/wR8DdgJYK1dDBzV2RNba7dYaz9uvF1JNOD17exxndb0qceE6hw7hwnW4/cHkj6AGWMYMGCQuy1gtakMHDTYtfPFu+OPP55BgwdFuyJjMSSxAfwr/BxYfCAHHXRQDA6YfA6fMoUt1hJudt9WoCoSUeuXw5pvhZNo2+KI+9rUtmet3bDXXeEWn9hBxphBwCSiCzjHtZycHIwxzgawUC05ubkamAyMHDWKz6tSCLswH6GywbCtGkaMGOH8yboIv9/PD77/A2ylxXzW+f+PZoWBBvjB938Qg+qS0yGHHAJA81UC1zT+qVDrrOYfipN9jK50ntnfrD5jzJPAHcBfgUOAK4Bia+3ZMSnAmGzgTeD31tr/tPD4pcClAEVFRQfOnj07FqftlDVr1hDypWBT9/0J6PXnnqRky6bd3xf27ssxp3xjv8c3dRWkBwyDBg3qbKldXkVFBVu2bKFvVpjUNjSEPfTf11m/efvu7wf06cn5px7TpnPVhgxba3wMGDBAXWN72bBhAzV1NZBL0yTdPbz6n1dJ8aUw9bSprR8kAlRAXm5e0o+x66zVq1fz2ssv409J4dtTp7ITiKSmMniwWm+dVF9fT2VlJTt37mTYsGEJPUA8nuzYsYOdO3d2yQ/H06ZNW2itLW7psbYMdPkecCfR7sFNRBdhjcnHV2NMCvAUMKul8AVgrX0AeACguLjYTp06NRan7pSHHnqYFdtrqR/59X0+L70qTPNfzy1VYf6+uHq/x89a+iyHTxjBzJkzO1doAtiyZQtnnXUW3x5ew4kD6/f7/MyabCCl2febGbn63jada9aqDF7dlMXzc+bo0+1eVq5cyXe+8x0ioyPYMV/90OYL+ihKKeKhbQ+1egyzwJCyIYXZj82mqKjIyXIT3pw5c9hRUkKvwkKyZs3iLz4fx5x4IhdeeKHXpSW0jRs3MmfOHGbNmsXLL7+sD2ouuf/++5k1axbz58/3upSYaksX5Ahr7TnW2iJrbaG19lyg0/PGTbR/7R/AcmvtHZ09npt69SoiEKpx7PimoZrCQq2yDNH91wYPGsjHO5xdC8Za+HhHGpMnT1b4asGIESM46uij8K/2R/e5aa8q8H3h47RTT1P4ioGxY8cSJDo5dSdQG4kwZswYj6tKfM0nOGiyg3RWWwLYX9p4X3tNAc4DphtjFjV+nRiD4zquV69emLrK6Lt2rIXqscG63Ru/Chxx5FGsKg9Q2eDcmLjN1T621RiOOPJIx87R1V1y8SUQokOLs5pPTXSbo3PPdaCy5DNy5EggGsA273WfOKf5oqCJvDyCuKPVLkhjzGFEtxzqaYz5abOHcoFOd3xba9+mxdEk8W/QoEHYcBDTUIVNy4npsX21ZbvPIVFHHnkkDz/8MJ/sSOGoPs4s/7Fge/SFVbPIWjdo0CCmTp3KvLfnETogBG1tAKgC33ofp3/zdLp37+5ojcmiaaX77gUFlJSU4Pf5kn7xYDck8qrs4r59RfhUIJtoSMtp9lUB7H8keQJrGujqqy2P+bGbjqkA9qURI0ZQ2LM7C0pS9v/kDlqwPY3Ro0Zpf7f9uOCCC7BBi1nT9s9OZoUhEAhw9tkxmbcjQPfu3clIT+ew4mJ2AH169yYlxbnfD4lSAJNYarUFzFr7JvCmMeb/WWvXuVhT3GsKR76aMsL5sd0z0FdbRmpamsbJNGOM4cijpvLsM/+hLlxN+j7aXwfmhFlX6SfiT2NwZg0Dc/a/YsrOOsPnFT6++61OL2+X8IYMGcKRRx7J2x+8HW0F2997fg341/k55dRTtHdeDBlj6Nu3LyGgzBgGqfXLFZr1KLHUlk7svxtj8pu+McYUGGNedq6k+JeTk0N+QTdMY3dhLPlqyhg0cJDGF+zlyCOPJBi2LNm573f880bUMjAnTFGPfH5dXMV5I2r3+XyAhY3dj0cdpQDWFueccw62wWLW7b8VzKw1YOGss85yobLkUlhURNgYdhmjSTsiXVBb3uV7WGvLm76x1pYBSf/bPnTIYAJ15TE/bqB+F0OGaC2fvY0fP56c7CxHuiEXbE9l4ID+9O8f29bMRDVq1CiGHzAc/9r97BEZAf/nfg477DD69OnjWn3JokePHoStpSYS0dg6kS6oLQEsYozZ3b5tjBlIbLfm7ZIGDRqEr648tjMhQw3Y+urdm1DLlwKBAEdPncaCHenUhWJ33J11huWlAaZNb9tirRLt/jr9tNOxuyyU7uOJm8HWWU477TS3Sksq+fn5u3eHKigo8LQWEWm/tgSwXwFvG2MeNsY8AswHrnG2rPjXv39/bCiICcZuPTBf3S4AzWZqxQknnEB9yPJBSewGwr69JQ0LfP3r+15UV/Y0ffp0UtNSMZ+33g3p+9xHt+7dtD2OQ/Ly8nbf1sbQIl3PfgOYtfYlYDLwODAbONBam9RjwIDd3VWmdlfMjmkaA5i6wlo2duxY+vfty5ub02NyvIiF+VvSmThxgrrI2ikzM5OpR0/Fv6mVTbrrwWwznPD1EzRw2SHaGFqka2s1gBljRjb+ORkYQHS9v83AgMb7ktrumZAxXIrCV1uGz++nb9++MTtmIjHGcPKMGawq97OhqvOTFJaVBthWY5gx49QYVJd8jjnmGGyDha1ffcxsig6+nz59uvuFJYnmG0M3vy0iXcO+3sWubPzzjy183e5wXXGvR48eZGXn4Kvd1yCY9vHVlDKg/wCt57MPJ5xwAikpAV7f2PltQF7fmEZ+Xq5mP3ZQcXEx6RnpmM1f7YY0mwy9evdi2LBhHlSWHJqHLm2fJdL1tBrArLXfafxzWgtfSf+x1hjD8GFD8dfELoCl1JUxfLjesPYlPz+fo446mvdKMgi21PXVRpUNhk92pPL1E07U4oodlJKSwiEHH4J/216zIUPg2+5jyuFTiG75Kk5oHrrUBSnS9eyrC/KMfX25WWS8GjVqFL6anRDu/LQ8U1+FravSfm5t8LWvfY3qBsviHR1vKXx/WyphGz2WdNwhhxyCrbHQfL3bnWDDloMPPtizupKBWsBEurZ9dUGe0vh1MfAP4JzGr78DFzlfWvybOHEiRML4qkpafDyS1R3rTyElNZVwTi8iWa2v1eOr3ALApEmTnCg1oRQXF1OQl8s7WzvecvXutjSGDB60e0896ZgJEyZEbwS/vM/sMBhjGD9+vDdFJYn09PQWb4tI17CvLsgLrbUXEt1sZLS19kxr7ZnAGPa/AUlSGD9+PMYY/I3haW8NAw8jktmdgu49qRt9Mg0DD2v1WP6KLWRlZTNkyBCnyk0YgUCAqdOPYdHO1A6tCbajzrC63M+xxx0f++KSTL9+/cjNy/1KABsydIgGhjuseejSzhkiXU9bfmv7W2ubJ4xtRGdFJr2srCyGH3AAgYrNnT5WauVWJk2aqBfSNpo+fTrBMHzcgW7ID7dFW86mTp0a46qSjzGGsWPGYkKNY70s+Mp9jB0z1tvCkoAm64h0bW15t3/dGPOyMWamMWYmMAd4zdmyuo6DiovxVW2HcHD/T26FqavE1lVw4IEHxrCyxDZu3Di6dyvg/Q50Q76/LZ0Dhg+jX79+DlSWfEaMGPHlGLBqsA02ep84SgFMpGtry0KsPwTuAyY0fj1grf2R04V1FZMnTwYbwV/ZwmJIbeRvbEFTAGs7n8/HMccex+LSVKqDbZ9pt7XGx9oKn7ofY2jw4GZ7l1a0cJ84IhAIeF2CiHRCW/u7PgbmWGt/ArxsjMlxsKYuZcyYMRhjWh2I3xa+qhKysrK1B2Q7HXvssYQj8GE7Nuh+r7HFTAuExk7TosRYMJXRMKz/yyIi+7bfAGaM+Q7wJHB/4119gWccrKlLyczMpF//Afiqd3T4GIGanYwaNVJrJrXTiBEj6Ne3D+9ta9uirNbCu9vSmThhPIWFhQ5Xlzx69+4dvWGBKsjOyda6VCIi+9GWFrDLgCk0di5Ya1cDevdqZszoUaTU7OzYD0dCmJpSjZnpAGMMxx53PMtLA5TV7z+8rqv0s6XaqPsxxtLS0nZ3h5lqo301RUTaoC0BrN5a29D0jTEmwJ7rXie94cOHYxtqMA017f5ZX2052AgHHHBA7AtLAscccwwW+Gjb/gfjf1CSgt/v4+ijj3a+sCSTmpoKFvy1fnr36u11OSIica8tAexNY8wvgQxjzHHAv4HnnC2ra2na787XgW2Jmn5Ge+Z1zMCBAxk4oD8Ltu87gFkLC7anM3HiRPLy8lyqLnk0bedkayxFRUUeVyMiEv/aEsB+DmwHlgDfBV4Afu1kUV3N7gDWgXFgvuodpKalqdumE446eiorygNUNrTeDbm5xseWasPRR091r7AkkpKSAhZsyNK9e+s7PoiISNQ+A5gxxg8st9b+zVr7TWvtNxpvqwuymZycHPr07dehmZCB6u2MGjUKv9/vQGXJYcqUKUQsLCltfVr+J9ujMyUPP/xwt8pKKkVFRRQUFADQrVs3j6sREYl/+wxg1towsNIYo5Xv92PihPGkVJdE+7raKhzEVO9g/LhxzhWWBEaOHEl+Xi6L9rEq/uKdqQwdMlizHx1y0kknUVxcDLA7iImISOva0gVZACwzxrxujHm26cvpwrqasWPHYoN1mNryNv+Mryoa2MYpgHWKz+fj4EMOZUlZOpEW8m9tCFaVBzjk0Nb34pTOaT6uLjc318NKRES6hrYspXyt41UkgMmTJwPRVe1DmW1rAfDv2oTf72f8+PFOlpYUDj74YF555RXWVX61K3d5WQphG32OOKP5ul9aA0xEZP9aDWDGmHTge8AwogPw/2GtDblVWFfTp08fevXuzcZdmwj1GtOmn0mp2MyYMWPIzMx0uLrE17SNU0vjwJbsDJCelsrYsdog2ilZWVkt3hYRkZbtqwvyQaCYaPg6AfijKxV1YYccfDApVVsgEtn/k4N1mOodHHTQQc4XlgS6d+/O4EEDWV761XFgy8vTGDd+/O6lEiT2MjIyWrwtIiIt21cAG22tPddaez/wDeBIl2rqsiZPnowNBdu0HIW/csvun5HYmDhpMqsqUvdYJbiiwbCxyjBpkq6zk9LT03ffTktr29ZQIiLJbF8BLNh0Q12PbTNp0iQgOg5sf/y7NpOWls6oUaOcLitpTJw4kfqQpS785XpgK8ujXZITJkzwqqyk0Dx0aU9TEZH929cg/AnGmIrG24boSvgVjbettVZTnfaSn5/PwEGD+Kxs636fm1K9jfHjx+3eQ086b8yY6Ni7upChKQ58titAwO/XVk8OU/euiEj7tNoCZq31W2tzG79yrLWBZrcVvloxZvRoUmp27Hs9sHAQasoYPXq0e4UlgcLCQnp070Zt6MsWmDUVAYYNG6puMYelpLS+BpuIiHxVW9YBk3YYOXJkdD2whqpWn+OrKQVrGTFihIuVJYeRo0bv7oK0FtZVpTBylIKu03w+vZSIiLSHXjVjrKmra18D8X3VO/d4rsTOsGHDaIgYsLC9zkdt0DJ06FCvyxIREdmDAliMDRo0CADfPlbE99WWkpmVRc+ePd0pKokMGTIEAAtsqIouytq0WbqIiEi8UACLsczMTAqLeuGrKWv1Of7acgYPHqzZYg7o378/ANYattX49rhPREQkXiiAOWDY0CEE6spbftBa/HXlDBk82NWakkWfPn0AiAAltT6yszK1N6GIiMQdBTAHDBkyBGrLIRL+ymMmWIMN1u3uKpPYysjIwO/3Y4GddT569erldUkiIiJfoQDmgMGDB4ONYOp2feWxpq5JDQx3TkpKACzsagjQvYfG2YmISPxRAHNAU+tWS+PATG0p0BjSxBGBQAoWKA/66datm9fliIiIfIUCmAMGDBiAz+/H1xi2mvPVlJFf0I28vDwPKksOKSnRAFZZb8nPz/e6HBERka9QAHNASkoK/fr1b7EFLFBbxrChGv/lpEAggLUQjKAAJiIiccnTAGaM+boxZqUxZo0x5hde1hJrw4YOIaW+fM87bQRfXbkG4DvM7/fvvq0ZkCIiEo88C2DGGD9wN3ACMBr4ljEmYfaMGTRoELa2Yo89IU19FTYc2r1Yqzij+QbnCmAiIhKPvGwBOxhYY61da61tAGYDp3pYT0wNHDgweiMS2n1f0+r4ux8TR6gFTERE4p2XAawvsKHZ9xsb70sIAwYMAMA0WwvMNC7OqgDmrOYBLDs728NKREREWmZssy4yV09szDeAr1trL2n8/jzgEGvtD/d63qXApQBFRUUHzp492/VaO8Jay6pVq3j19TdI8fs56qQzMfVVBGxQexM67O9//zvhcJipU6cydOjQPbokxTk7duxg586djBgxwutSkoauuft0zd3Xla/5tGnTFlpri1t6zMt3pk1A8036+jXetwdr7QPAAwDFxcV26tSprhQXC3ffcy/bdpZT1LM7f19cTcancxg/qCeXXHKJ16UltFmzZlFWVsasWbN47rnntOSHS+6//35mzZrF/PnzvS4laeiau0/X3H2Jes297IL8CBhujBlsjEkFzgae9bCemBs4oP8eY8D8DRUM0MbQjvP5vvxvnZGR4WElIiIiLfMsgFlrQ8APgZeB5cAT1tplXtXjhL59+2Js4xiwcBBbX7N7s2hxztChQykoKACia7KJiIjEG0/XAbPWvmCtPcBaO9Ra+3sva3FC3759dy9DYeorv7xPHPXDH/6Q4uJol7sxxuNqREREvkor4TuoV69e0RvW4quv2vM+cUxqaqrXJYiIiOyTApiDevbsufu2aagGoLCw0KtykoZmPYqISLxTAHPQl2HLYhqqMcbsHpskIiIiyUsBzEHNN4I2wVpy8/LVOiMiItIOffv25c9//rPXZcScApiDAoFAdEkEazGhOvLztR6ViIhIe2RnZzN58mSvy4g5BTCHNbV4mVA9+Xn53hYjIiIicUEBzGFNAcwfaSAvTxtDi4iIiAKY46IbQ1tMuEEbQ4uIiAigAOa4aAADQvUKYCIiIgIogDnO7/eDBRsKKoCJiIgIoADmuOjG0NHtiDIzM70tRkREROKCApjDdndBogAmIiIiUQpgDou2gEVlZGR4WImIiIjECwUwhymAiYiIyN4UwBzWPIClp6d7WImIiIjECwUwhymAiYiIyN4UwBymACYiIiJ7UwBzWPMAlpaW5mElIpJoMjIymDZtmtdliEgHBLwuINEpgImIUwYMGMC5557rdRki0gFqAXPYwIEDKSgoABTARCT2mn/IE5GuQ7+5DrvooosoLi4GIDU11eNqREREJB4ogDksJSWlxdsiIiKSvBTAHNY8dKmrQEREREABzHFq9RIREZG9KYA5LBDQRFNJDj6fj379+nldhohIl6AA5jAFMEkWAwcO5E9/+pPXZYiIdAkKYA7z+/1elyDiitTUVIqKirwuQ0SkS1AAc5gxxusSREREJM4ogImIiIi4TAFMRERExGUKYCIiIm3Ur18/7r77bq/LkASgACYiItJGWVlZjBs3zusyJAEogImIiIi4TAFMRERExGUKYCIiIiIuUwATERERcZkCmIiIiIjLFMBEREREXKYAJiIiIuIyBTARERERlymAiYiIiLhMAUxERETEZQpgIiIiIi7zJIAZY24zxqwwxvzPGPO0MSbfizpEREREvOBVC9irwFhr7XhgFXCNR3WIiIiIuM6TAGatfcVaG2r89n2gnxd1iIiIiHjBWGu9LcCY54DHrbWPtPL4pcClAEVFRQfOnj3bzfJiYseOHezcuZMRI0Z4XUrS0DV3X1VVFdnZ2V6XkVR0zd2na+6+rnzNp02bttBaW9zSY44FMGPMa0CvFh76lbX2v43P+RVQDJxh21BIcXGxXbBgQWwLdcH999/PrFmzmD9/vtelJA1dc/fNmzePqVOnel1GUtE1d5+uufu68jU3xrQawAJOndRae+y+HjfGzAROBo5pS/gSERERSRSOBbB9McZ8HbgaONpaW+NFDSIiIiJe8WoW5F+BHOBVY8wiY8x9HtUhIiIi4jpPWsCstcO8OK+IiIhIPNBK+CIiIiIuUwATERERcZkCmIiIiIjLFMBEREREXKYAJiIiIuIyBTARERERlymAiYiIiLhMAUxERETEZQpgIiIiIi5TABMRERFxmQKYiIiIiMsUwERERERcpgAmIiIi4jIFMBERERGXKYCJiIiIuEwBTERERMRlCmAiIiIiLlMAExEREXGZApiIiIiIyxTARERERFymACYiIiLiMgUwEREREZcpgImIiIi4TAFMRERExGUKYCIiIiIuUwATERERcZkCmIiIiIjLFMBEREREXKYAJiIiIuIyBTARERERlymAuaCoqIjrr7/e6zJEREQkTgS8LiAZ5OfnM3XqVK/LEBERkTihFjARERERlymAiYiIiLhMAUxERETEZQpgIiIiIi5TAJOElJqayoQJE7wuQ0REpEWaBSkJacCAAZx11llelyEiItIitYBJQvL7/WRmZnpdhoiISIsUwERERERcpgAmIiIi4jIFMBERERGXeRrAjDFXGmOsMaaHl3WIiIiIuMmzAGaM6Q8cD6z3qgYRERERL3jZAvYn4GrAeliDiIiIiOuMte7nH2PMqcB0a+0VxpgvgGJr7Y5WnnspcClAUVHRgbNnz3av0BipqqoiOzvb6zKSiq65+3TN3adr7j5dc/d15Ws+bdq0hdba4pYecyyAGWNeA3q18NCvgF8Cx1trd+0vgDVXXFxsFyxYENtCXTBv3jymTp3qdRlJRdfcfbrm7tM1d5+uufu68jU3xrQawBxbCd9ae2wrxYwDBgOLjTEA/YCPjTEHW2u3OlWPiIiISLxwfSsia+0SoLDp+/a0gImIiIgkAq0DJiIiIuIyzzfjttYO8roGERERETd5Mguyo4wx24F1XtfRAT0AdbG6S9fcfbrm7tM1d5+uufu68jUfaK3t2dIDXSqAdVXGmAWtzYIQZ+iau0/X3H265u7TNXdfol5zjQETERERcZkCmIiIiIjLFMDc8YDXBSQhXXP36Zq7T9fcfbrm7kvIa64xYCIiIiIuUwuYiIiIiMsUwGLIGPN1Y8xKY8waY8wvWng8zRjzeOPjHxhjBnlQZkIxxvzTGFNijFnayuPGGHNX4zX/nzFmsts1JhJjTH9jzFxjzKfGmGXGmCtaeI6ueQwZY9KNMR8aYxY3XvPftvAcvbY4wBjjN8Z8Yox5voXHdM1jzBjzhTFmiTFmkTHmKxs/J9priwJYjBhj/MDdwAnAaOBbxpjRez3tYqDMWjsM+BNwi7tVJqT/B3x9H4+fAAxv/LoUuNeFmhJZCLjSWjsaOBS4rIX/57rmsVUPTLfWTgAmAl83xhy613P02uKMK4DlrTyma+6Madbaia0sO5FQry0KYLFzMLDGWrvWWtsAzAZO3es5pwIPNt5+EjjGNO5ILh1jrZ0PlO7jKacCD9mo94F8Y0xvd6pLPNbaLdbajxtvVxJ9c+q719N0zWOo8TpWNX6b0vi19+BdvbbEmDGmH3AS8PdWnqJr7r6Eem1RAIudvsCGZt9v5KtvTLufY60NAbuA7q5Ul7za8u8iHdDY5TIJ+GCvh3TNY6yxK2wRUAK8aq1t9ZrrtSVm/gxcDURaeVzXPPYs8IoxZqEx5tIWHk+o1xYFMBFpN2NMNvAU8GNrbYXX9SQ6a23YWjsR6AccbIwZ63FJCc0YczJQYq1d6HUtSeYIa+1kol2NlxljjvK6ICcpgMXOJqB/s+/7Nd7X4nOMMQEgD9jpSnXJqy3/LtIOxpgUouFrlrX2Py08RdfcIdbacmAuXx33qNeW2JoCzDDGfEF0OMl0Y8wjez1H1zzGrLWbGv8sAZ4mOrSnuYR6bVEAi52PgOHGmMHGmFTgbODZvZ7zLHBB4+1vAG9YLcTmtGeB8xtnzxwK7LLWbvG6qK6qcYzLP4Dl1to7WnmarnkMGWN6GmPyG29nAMcBK/Z6ml5bYshae421tp+1dhDR1/I3rLXn7vU0XfMYMsZkGWNymm4DxwN7z25PqNeWgNcFJAprbcgY80PgZcAP/NNau8wY8ztggbX2WaJvXA8bY9YQHTh+tncVJwZjzGPAVKCHMWYjcD3RQcpYa+8DXgBOBNYANcCF3lSaMKYA5wFLGsckAfwSGAC65g7pDTzYONPaBzxhrX1ery3u0zV3VBHwdOM8hgDwqLX2JWPM9yAxX1u0Er6IiIiIy9QFKSIiIuIyBTARERERlymAiYiIiLhMAUxERETEZQpgIiIiIi5TABORhGCMCRtjFhljlhpjnmtaO2sfz59ojDmx2fczjDG/cLxQERG0DIWIJAhjTJW1Nrvx9oPAKmvt7/fx/JlAsbX2hy6VKCKymxZiFZFE9B4wHsAYczBwJ5AO1BJdvPFz4HdAhjHmCOBmIIPGQGaM+X9ABVAM9AKuttY+aYzxAX8FphPdFDhIdNHlJ138u4lIAlAXpIgklMYV44/hy63AVgBHWmsnAdcBN1lrGxpvP26tnWitfbyFQ/UGjgBOBv7QeN8ZwCBgNNEdAQ5z6u8hIolNLWAikigyGrdH6gssB15tvD+P6FY+wwFL41ZVbfCMtTYCfGqMKWq87wjg3433bzXGzI1Z9SKSVNQCJiKJotZaOxEYCBjgssb7bwDmWmvHAqcQ7Ypsi/pmt02sihQRAQUwEUkw1toa4HLgSmNMgGgL2KbGh2c2e2olkNPOw78DnGmM8TW2ik3tXLUikqwUwEQk4VhrPwH+B3wLuBW42RjzCXsOu5gLjG5cuuKsNh76KWAj8CnwCPAxsCtmhYtI0tAyFCIi7WCMybbWVhljugMfAlOstVu9rktEuhYNwhcRaZ/nGxd5TQVuUPgSkY5QC5iIiIiIyzQGTERERMRlCmAiIiIiLlMAExEREXGZApiIiIiIyxTARERERFymACYiIiLisv8PWdgNupNGJsQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "y_test_hat = model_sgd.predict([x_test[:, 0], x_test[:, 1]])[:, 0, 0]\n",
    "plot_data = pd.DataFrame(np.vstack([y_test, y_test_hat]).T, columns=['Rating', 'Predicted rating'])\n",
    "fig, ax = plt.subplots(1, figsize=(10, 6))\n",
    "sns.violinplot(data=plot_data, x = \"Rating\", y = \"Predicted rating\", ax=ax)\n",
    "plt.grid(True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "flexible-apache",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+-----+----------------------------------------------------+----------+\n",
      "|     | title                                              |   rating |\n",
      "|-----+----------------------------------------------------+----------|\n",
      "| 121 | Lord of the Rings: The Fellowship of the Ring, ... |        5 |\n",
      "| 155 | Freaks (1932)                                      |        5 |\n",
      "| 131 | Lord of the Rings: The Two Towers, The (2002)      |        5 |\n",
      "| 145 | Lord of the Rings: The Return of the King, The ... |        5 |\n",
      "|   0 | Jumanji (1995)                                     |        4 |\n",
      "| 106 | O Brother, Where Art Thou? (2000)                  |        4 |\n",
      "| 100 | Hook (1991)                                        |        4 |\n",
      "| 101 | Misery (1990)                                      |        4 |\n",
      "| 103 | Crouching Tiger, Hidden Dragon (Wo hu cang long... |        4 |\n",
      "| 104 | Dungeons & Dragons (2000)                          |        4 |\n",
      "+-----+----------------------------------------------------+----------+\n",
      "160 rows x 2 columns\n",
      "+----+----------------------------------------------+----------+\n",
      "|    | title                                        |   rating |\n",
      "|----+----------------------------------------------+----------|\n",
      "|  0 | Foodfight! (2012)                            |  5.33404 |\n",
      "|  1 | Cowspiracy: The Sustainability Secret (2014) |  4.8014  |\n",
      "|  2 | Killers from Space (1954)                    |  4.60296 |\n",
      "|  3 | Frank (2014)                                 |  4.59485 |\n",
      "|  4 | Tokyo Drifter (Tôkyô nagaremono) (1966)      |  4.53437 |\n",
      "|  5 | Seeing Other People (2004)                   |  4.52857 |\n",
      "|  6 | Hobbit, The (1977)                           |  4.5259  |\n",
      "|  7 | V/H/S/2 (2013)                               |  4.50467 |\n",
      "|  8 | East, The (2013)                             |  4.50435 |\n",
      "|  9 | Pillars of the Earth, The (2010)             |  4.50202 |\n",
      "+----+----------------------------------------------+----------+\n",
      "10 rows x 2 columns\n"
     ]
    }
   ],
   "source": [
    "def get_items(user_id):\n",
    "    u_items = ratings[ratings['user_id'] == user_id].merge(items, on='item_id', how='left')\n",
    "    return u_items[['title', 'rating']].sort_values(by=['rating'], ascending=False)\n",
    "\n",
    "def recommend_for_user(model, user_id, k):\n",
    "    x1, x2 = np.repeat(user_id, n_items), item_ids\n",
    "    ratings = model.predict([x1, x2]).ravel()\n",
    "    top_k_indexes = ratings.argsort()[-k:][::-1]\n",
    "    return x2[top_k_indexes], ratings[top_k_indexes]\n",
    "\n",
    "def show_recommendations(r_items, r_ratings):\n",
    "    r = pd.DataFrame(data=np.vstack([r_items, r_ratings]).T, columns=['item_id', 'rating'])\n",
    "    print_df(r.merge(items, on='item_id', how='left')[['title', 'rating']], col_width = 50)\n",
    "\n",
    "#\n",
    "# Generate recommendations for an example user\n",
    "#\n",
    "user_id = 0\n",
    "k_recomendations = 10\n",
    "print_df(get_items(user_id), col_width = 50, rows = 10)\n",
    "r_items, r_ratings = recommend_for_user(model_sgd, user_id, k_recomendations)\n",
    "show_recommendations(r_items, r_ratings)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "pending-treasure",
   "metadata": {},
   "source": [
    "# Model 2: Network with Item Features\n",
    "\n",
    "In this section, we build a simple extension of the basic neural collaborative filtering network to incroporate embeddigns produced based movie tags. \n",
    "\n",
    "We create item embeddigns by concatenating all movie tags into pseudo-sentence and encoding it using a pre-trained MiniLM model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "indonesian-bruce",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Item features shape: (15324463, 384)\n"
     ]
    }
   ],
   "source": [
    "text_model = SentenceTransformer('all-MiniLM-L6-v2')\n",
    "items['item_features'] = items['tag'].transform(text_model.encode)\n",
    "\n",
    "item_features_r = ratings.merge(items[['item_id', 'item_features']], on='item_id')['item_features']\n",
    "item_features_r = np.vstack(item_features_r)\n",
    "print(f'Item features shape: {item_features_r.shape}')\n",
    "feature_dim = item_features_r.shape[1]\n",
    "\n",
    "x1_train, x2_train, x3_train = ratings['user_id'].values[train_idx], ratings['item_id'].values[train_idx], item_features_r[train_idx]\n",
    "x1_test, x2_test, x3_test = ratings['user_id'].values[test_idx], ratings['item_id'].values[test_idx], item_features_r[test_idx]\n",
    "y_train, y_test = ratings['rating'].values[train_idx], ratings['rating'].values[test_idx]\n",
    "\n",
    "del item_features_r"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "breathing-colon",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/16\n",
      "47889/47889 [==============================] - 105s 2ms/step - loss: 0.9548 - root_mean_squared_error: 0.9771 - val_loss: 0.7311 - val_root_mean_squared_error: 0.8550\n",
      "Epoch 2/16\n",
      "47889/47889 [==============================] - 100s 2ms/step - loss: 0.6958 - root_mean_squared_error: 0.8342 - val_loss: 0.6848 - val_root_mean_squared_error: 0.8275\n",
      "Epoch 3/16\n",
      "47889/47889 [==============================] - 100s 2ms/step - loss: 0.6582 - root_mean_squared_error: 0.8113 - val_loss: 0.6700 - val_root_mean_squared_error: 0.8185\n",
      "Epoch 4/16\n",
      "47889/47889 [==============================] - 99s 2ms/step - loss: 0.6395 - root_mean_squared_error: 0.7997 - val_loss: 0.6619 - val_root_mean_squared_error: 0.8136\n",
      "Epoch 5/16\n",
      "47889/47889 [==============================] - 99s 2ms/step - loss: 0.6288 - root_mean_squared_error: 0.7930 - val_loss: 0.6579 - val_root_mean_squared_error: 0.8111\n",
      "Epoch 6/16\n",
      "47889/47889 [==============================] - 99s 2ms/step - loss: 0.6223 - root_mean_squared_error: 0.7889 - val_loss: 0.6554 - val_root_mean_squared_error: 0.8096\n",
      "Epoch 7/16\n",
      "47889/47889 [==============================] - 99s 2ms/step - loss: 0.6178 - root_mean_squared_error: 0.7860 - val_loss: 0.6547 - val_root_mean_squared_error: 0.8091\n",
      "Epoch 8/16\n",
      "47889/47889 [==============================] - 99s 2ms/step - loss: 0.6145 - root_mean_squared_error: 0.7839 - val_loss: 0.6532 - val_root_mean_squared_error: 0.8082\n",
      "Epoch 9/16\n",
      "47889/47889 [==============================] - 99s 2ms/step - loss: 0.6120 - root_mean_squared_error: 0.7823 - val_loss: 0.6536 - val_root_mean_squared_error: 0.8084\n",
      "Epoch 10/16\n",
      "47889/47889 [==============================] - 100s 2ms/step - loss: 0.6099 - root_mean_squared_error: 0.7810 - val_loss: 0.6520 - val_root_mean_squared_error: 0.8075\n",
      "Epoch 11/16\n",
      "47889/47889 [==============================] - 99s 2ms/step - loss: 0.6082 - root_mean_squared_error: 0.7799 - val_loss: 0.6517 - val_root_mean_squared_error: 0.8073\n",
      "Epoch 12/16\n",
      "47889/47889 [==============================] - 99s 2ms/step - loss: 0.6068 - root_mean_squared_error: 0.7789 - val_loss: 0.6511 - val_root_mean_squared_error: 0.8069\n",
      "Epoch 13/16\n",
      "47889/47889 [==============================] - 99s 2ms/step - loss: 0.6055 - root_mean_squared_error: 0.7782 - val_loss: 0.6518 - val_root_mean_squared_error: 0.8073\n",
      "Epoch 14/16\n",
      "47889/47889 [==============================] - 99s 2ms/step - loss: 0.6044 - root_mean_squared_error: 0.7775 - val_loss: 0.6513 - val_root_mean_squared_error: 0.8070\n",
      "Epoch 15/16\n",
      "47889/47889 [==============================] - 99s 2ms/step - loss: 0.6035 - root_mean_squared_error: 0.7768 - val_loss: 0.6508 - val_root_mean_squared_error: 0.8067\n",
      "Epoch 16/16\n",
      "47889/47889 [==============================] - 99s 2ms/step - loss: 0.6026 - root_mean_squared_error: 0.7763 - val_loss: 0.6510 - val_root_mean_squared_error: 0.8068\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAA1CklEQVR4nO3deXwddb34/9f7bNnbZmlT2pSk0EJ3usQWrEAqi8WFzUKLgHAFq34FRNSv6PUKP9TvxSsiesWlIhS9SMUi0CtFFknYl7YshW7QlSbQfUua/Zz374+Zk5ykJ2mSZnJOct7Px2MeZ+Yzn8/MO0173v3MZ+YzoqoYY4wx7fkSHYAxxpjkZAnCGGNMXJYgjDHGxGUJwhhjTFyWIIwxxsQVSHQAvaWgoEBLSkp63P7w4cNkZWX1XkC9LNnjg+SPMdnjA4uxNyR7fJBcMa5atWqPqg6Nu1NVB8QyY8YMPRbl5eXH1N5ryR6favLHmOzxqVqMvSHZ41NNrhiBldrB96pdYjLGGBOXJQhjjDFxWYIwxhgT14AZpI6nqamJyspK6uvrj1p38ODBrFu3rg+i6plkiS89PZ2ioiKCwWCiQzHGeGxAJ4jKykpycnIoKSlBRDqtW11dTU5OTh9F1n3JEJ+qsnfvXiorKxk9enRCYzHGeG9AX2Kqr68nPz//qMnBdI2IkJ+f36UemTGm/xvQCQKw5NDL7M/TmNQxoC8xdUVzOMLew434I0ryXmAyxpi+52kPQkTmisgGEdkoIjfH2V8sIv8SkdUiUiEiRTH7wiLylrss8y5G2Hmonvpmb96LceDAAX7zm990u92nP/1pDhw40PsBGWNMF3mWIETED9wNnAdMAC4TkQntqt0B/ElVpwC3Af8Zs69OVae6y/lexen3+Qj4fDRFvDl+Rwmiubm503bLly9nyJAh3gRljDFd4OUlppnARlXdDCAiS4ALgLUxdSYAN7nr5cCjHsbToVDAR1Mk7Mmxb775ZjZt2sTUqVMJBoOkp6eTm5vL+vXree+997jwwgvZvn079fX1fOMb32DhwoUAlJSUsHLlSmpqajjvvPOYNWsWK1asYOTIkTz22GNkZGR4Eq8xxkSJevTKURGZB8xV1Wvd7SuBWap6XUydvwCvqeovReRi4GGgQFX3ikgz8BbQDNyuqo/GOcdCYCFAYWHhjCVLlrTZP3jwYMaMGQPAT5/axPqdNXFjbYooEYU0f/cGYMcVZvPdc0/stM62bdu49NJLee2113jhhRe45JJLePXVV4lOLLhv3z7y8vKoq6ujrKyM5cuXk5+fz6RJk3juueeoqalh6tSplJeXM3XqVK666irOO+88FixY0K1Ye9PGjRs5ePDgEeU1NTVkZ2cnIKKuSfb4wGLsDckeHyRXjHPmzFmlqqXx9iV6kPrbwK9F5GrgeaAKiP5XvlhVq0TkBOBZEXlHVTfFNlbVRcAigNLSUi0rK2tz8HXr1rU8OxAMBfH7/XGDiBAh3BzpcH9HgqHgUZ9NyM7OxufzkZOTQ2ZmJjNnzmTy5Mkt+3/+85/zyCOPAFBVVcWOHTtantuI/gUaPXo0U6dOJScnh1mzZrFz586EPhORnp7OtGnTjiivqKig/e8gmSR7fGAx9oZkjw/6R4zgbYKoAkbFbBe5ZS1U9UPgYgARyQY+r6oH3H1V7udmEakApgFtEkR33PK5iR3u21/byPZ9tZxUmEN6sHtJortip/itqKjgmWee4ZVXXiEzM5OysrK4zxikpaW1rPv9furq6jyN0RhjwNu7mFYAY0VktIiEgAVAm7uRRKRARKIxfA+41y3PFZG0aB1gNm3HLnpVyO+E0Njc+yPVOTk5VFdXx9138OBBcnNzyczMZP369bz66qu9fn5jjOkpz3oQqtosItcBTwJ+4F5VXSMit+HMP74MKAP+U0QU5xLT193m44Hfi0gEJ4ndrqqeJYi0gJMgGjxIEPn5+cyePZtJkyaRkZFBYWFhy765c+fyu9/9jvHjx3PyySdz6qmn9vr5jTGmpzwdg1DV5cDydmU/jFlfCiyN0+5lYHL7cq/4fYJPoDHszb2uf/nLX+KWp6Wl8cQTT8Tdt3XrVgAKCgp49913W3oh3/72tz2J0Rhj2hvwU210hYgQ8Iknl5iMMaa/sgThCvqgsdmbZyGMMaY/sgThchKEEvHouRBjjOlvLEG4Aj5BUZo8Gocwxpj+xhKEK+j+Sdg4hDHGOCxBuAKWIIwxpg1LEC6/gE/Ek2chuiM6vcaHH37IvHnz4tYpKytj5cqVnR7nrrvuora2tmXbpg83xnSXJQiXiBAK+JKmBzFixAiWLj3iEZEua58gbPpwY0x3WYKIEfL7er0HcfPNN3P33Xe3bN966638+Mc/5qyzzmL69OlMnjyZxx577Ih2W7duZdKkSQDU1dVx9dVXM378eC666KI2czF97Wtfo7S0lIkTJ3LLLbcA8Ktf/YoPP/yQOXPmMGfOHMCZPnzPnj0A3HnnnUyaNIlJkyZx1113tZxv/PjxfPnLX2bixImce+65NueTMSku0bO59p0nboYd73S4OyPczHEITWFFQ36ELkz9PXwynHd7p1Xmz5/PjTfeyNe/7swi8tBDD/Hkk09yww03MGjQIPbs2cOpp57K+eef3+H7nn/729+SmZnJunXrWL16NdOnT2/Z95Of/IS8vDzC4TBnnXUWq1ev5oYbbuDOO++kvLycgoKCNsdatWoV9913H6+99hqqyqxZszjzzDPJzc3l/fff58EHH+QPf/gDl156KQ8//DBXXHHF0f8cjDEDkvUgYvgQUOjNRyGmTZvGrl27+PDDD3n77bfJzc1l+PDhfP/732fKlCmcffbZVFVVsXPnzg6P8fzzzzN//nwApkyZwpQpU1r2PfTQQ0yfPp1p06axZs0a1q7tfMqqF198kYsuuoisrCyys7O5+OKLeeGFF4DWacUBZsyY0TLdhzEmNaVOD+Io/9Ovq66GYDpb9hzmhIIsstODvXbqSy65hKVLl7Jjxw7mz5/PAw88wO7du1m1ahXBYJCSkpK403wfzZYtW7jjjjtYsWIFubm5XH311T06TpRNK26MiWU9iBjRWV17e9K++fPns2TJEpYuXcoll1zCwYMHGTZsGMFgkPLycrZt29Zp+zPOOIO//e1vALz77rusXr0agEOHDpGVlcXgwYPZuXNnm4n/Oppm/PTTT+fRRx+ltraWw4cP88gjj3D66af34k9rjBkoUqcH0QVBvw/x4FbXiRMnUl1dzciRIznuuOO4/PLL+dznPsfkyZMpLS1l3Lhxnbb/2te+xhVXXMH48eMZP348M2bMAOCUU05h2rRpjBs3jlGjRjF79uyWNgsXLmTu3LmMGDGC8vLylvLp06dz9dVXM3PmTACuvfZapk2bZpeTjDFHsAQRQ0QI+b251fWdd1oHyAsKCnjllVfi1qupcd6bXVJSwrvvvgtARkYGixcvjvua0cWLF8c9zvXXX8/111/fsh2bAG666SZuuummNvVjzwc2rbgxxi4xHSEU6P1bXY0xpj+yBNFOmvuwnNqsrsaYFDfgE0R3v+hDAR8RVZojliDiscRpTOoY0AkiPT2dvXv3dutLLeR372Syy0xHUFX27t1Lenp6okMxxvQBTwepRWQu8EvAD9yjqre3218M3AsMBfYBV6hqpbvvKuAHbtUfq+r93T1/UVERlZWV7N69+6h16+vrSU9PpzkcYeehBpr2BskMJc8YfjS+REtPT6eoqCjRYRhj+oBn34Ai4gfuBs4BKoEVIrJMVWMf9b0D+JOq3i8inwT+E7hSRPKAW4BSQIFVbtv93YkhGAwyevToLtWtqKhg2rRpNDZHuOA/nuC6OWO46dyTu3M6T0XjM8aYvuLlJaaZwEZV3ayqjcAS4IJ2dSYAz7rr5TH7PwU8rar73KTwNDDXw1hbhAI+RuZmsHVv7dErG2PMAOblNZSRwPaY7UpgVrs6bwMX41yGugjIEZH8DtqObH8CEVkILAQoLCykoqKix8HW1NS0tB8kDbyzdccxHa+3xcaXrJI9xmSPDyzG3pDs8UH/iBES/6Dct4Ffi8jVwPNAFRDuamNVXQQsAigtLdWysrIeB1JRUUG0/dP73+Efqz/iWI7X22LjS1bJHmOyxwcWY29I9vigf8QI3l5iqgJGxWwXuWUtVPVDVb1YVacB/+6WHehKWy+V5GdxsK6JA7WNfXVKY4xJOl4miBXAWBEZLSIhYAGwLLaCiBSISDSG7+Hc0QTwJHCuiOSKSC5wrlvWJ47PzwRgm41DGGNSmGcJQlWbgetwvtjXAQ+p6hoRuU1EznerlQEbROQ9oBD4idt2H/AjnCSzArjNLesTJflZAGzde7ivTmmMMUnH0zEIVV0OLG9X9sOY9aVA3Bcvq+q9tPYo+tTxedaDMMaYAf0kdU9lhPwMH5RuCcIYk9IsQXTg+PxMttklJmNMCrME0YGS/Ex7WM4Yk9IsQXSgOD+LPTUNHG5oTnQoxhiTEJYgOhC9k8nGIYwxqcoSRAeKW56FsHEIY0xqsgTRgejDcjYOYYxJVZYgOjAoPUh+VogP9lkPwhiTmixBdKI4P5Ote6wHYYxJTZYgOlGcn2VjEMaYlGUJohPF+Zl8dKie+qYuz0BujDEDhiWITpTkZ6EKlfvtMpMxJvVYguhE9FZXG4cwxqQiSxCdKLZpv40xKcwSRCdyM4PkpAfsaWpjTEqyBNEJEaEkP4tt+yxBGGNSjyWIoyi2ab+NMSnKEsRRFOdnUrm/jqZwJNGhGGNMn7IEcRTF+VmEI0rV/rpEh2KMMX3K0wQhInNFZIOIbBSRm+PsP15EykXkTRFZLSKfdstLRKRORN5yl995GWdnWqb9tnEIY0yKCXh1YBHxA3cD5wCVwAoRWaaqa2Oq/QB4SFV/KyITgOVAibtvk6pO9Sq+rmo77ffQxAZjjDF9yMsexExgo6puVtVGYAlwQbs6Cgxy1wcDH3oYT48My0kjPeizh+WMMSlHVNWbA4vMA+aq6rXu9pXALFW9LqbOccBTQC6QBZytqqtEpARYA7wHHAJ+oKovxDnHQmAhQGFh4YwlS5b0ON6amhqys7Pj7vvBi7UUZPi4cUZ6j49/rDqLL1kke4zJHh9YjL0h2eOD5Ipxzpw5q1S1NO5OVfVkAeYB98RsXwn8ul2dm4BvueunAWtxejVpQL5bPgPYDgzq7HwzZszQY1FeXt7hvoV/WqFn/bzimI5/rDqLL1kke4zJHp+qxdgbkj0+1eSKEVipHXyvenmJqQoYFbNd5JbFugZ4CEBVXwHSgQJVbVDVvW75KmATcJKHsXaqOD+LD/bWEo5409syxphk5GWCWAGMFZHRIhICFgDL2tX5ADgLQETG4ySI3SIy1B3kRkROAMYCmz2MtVPF+Zk0hiPsOFSfqBCMMabPeZYgVLUZuA54EliHc7fSGhG5TUTOd6t9C/iyiLwNPAhc7XZ5zgBWi8hbwFLgq6q6z6tYj6blVld7otoYk0I8u80VQFWX49y6Glv2w5j1tcDsOO0eBh72MrbuaL3VtZaPn5jgYIwxpo/Yk9RdcNzgDIJ+sWm/jTEpxRJEF/h9wqi8TLbZsxDGmBRiCaKLbNpvY0yqsQTRRdFpv9WjBwuNMSbZWILoouK8TGobw+yuaUh0KMYY0ycsQXRRcUH0Vle7zGSMSQ2WILqo9VkISxDGmNRgCaKLRg7JwO8Te1jOGJMyLEF0USjgY8SQdLZaD8IYkyIsQXRDSX6W9SCMMSnDEkQ3OLe6Wg/CGJMaLEF0Q0l+FgfrmjhQ25joUIwxxnOWILrh+Dxn0j4bhzDGpAJLEN1QUmDTfhtjUocliG6I9iBsHMIYkwosQXRDetDP8EHpNu23MSYlWILoJruTyRiTKixBdJPzLIQlCGPMwGcJopuKCzLZU9NATUNzokMxxhhPeZogRGSuiGwQkY0icnOc/ceLSLmIvCkiq0Xk0zH7vue22yAin/Iyzu4ozrM7mYwxqcGzBCEifuBu4DxgAnCZiExoV+0HwEOqOg1YAPzGbTvB3Z4IzAV+4x4v4Yrz7U4mY0xq8LIHMRPYqKqbVbURWAJc0K6OAoPc9cHAh+76BcASVW1Q1S3ARvd4CWcJwhiTKsSrV2iKyDxgrqpe625fCcxS1eti6hwHPAXkAlnA2aq6SkR+Dbyqqv/j1vsj8ISqLm13joXAQoDCwsIZS5Ys6XG8NTU1ZGdnd6nuDc8eZuqwAF+alNbj83VXd+JLlGSPMdnjA4uxNyR7fJBcMc6ZM2eVqpbG2xfo62DauQxYrKo/F5HTgD+LyKSuNlbVRcAigNLSUi0rK+txIBUVFXS1/Zi1L9EY8FFWdlqPz9dd3YkvUZI9xmSPDyzG3pDs8UH/iBG8vcRUBYyK2S5yy2JdAzwEoKqvAOlAQRfbJozd6mqMSQVeJogVwFgRGS0iIZxB52Xt6nwAnAUgIuNxEsRut94CEUkTkdHAWOB1D2PtluL8LD46WE99UzjRoRhjjGe6lCBEJEtEfO76SSJyvogEO2ujqs3AdcCTwDqcu5XWiMhtInK+W+1bwJdF5G3gQeBqdazB6VmsBf4JfF1Vk+bbuKTAGajevs96EcaYgaurYxDPA6eLSC7OoPIKYD5weWeNVHU5sLxd2Q9j1tcCszto+xPgJ12Mr0/FTvs9tjAnwdEYY4w3unqJSVS1FrgY+I2qXoLzjEJKKsm3h+WMMQNflxOEe5fR5cDjbllSPLiWCEMygwxKD9hAtTFmQOtqgrgR+B7wiDuOcAJQ7llUSU5EKCnIsmm/jTEDWpfGIFT1OeA5AHeweo+q3uBlYMnu+LxMVlceTHQYxhjjma7exfQXERkkIlnAu8BaEfmOt6Elt5L8LKoO1NEUjiQ6FGOM8URXLzFNUNVDwIXAE8Bo4EqvguoPivMzCUeUqv11iQ7FGGM80dUEEXSfe7gQWKaqTTgT7aWskgLnTiYbhzDGDFRdTRC/B7biTKj3vIgUA4e8Cqo/KM6zWV2NMQNbVwepfwX8KqZom4jM8Sak/mFoThoZQb8lCGPMgNXVQerBInKniKx0l5/j9CZSlohQnJ9pD8sZYwasrl5iuheoBi51l0PAfV4F1V8U52faGIQxZsDq6lxMJ6rq52O2/z8RecuDePqVkvwsytfvJhxR/D5JdDjGGNOrutqDqBORT0Q3RGQ2kPL3dxbnZ9EYjrDjUH2iQzHGmF7X1R7EV4E/ichgd3s/cJU3IfUfJdH3U+85zMghGQmOxhhjeleXehCq+raqngJMAaao6jTgk55G1g8cn9867bcxxgw03XqjnKoecp+oBrjJg3j6leMGZxDy++xOJmPMgHQsrxxN+VFZv08YlZdhz0IYYwakY0kQKT3VRlRJvk37bYwZmDodpBaRauInAgFsVBZnHOLlTXtRVURSvlNljBlAOk0QqnpML1wWkbnAL3HePnePqt7ebv8vgOiUHZnAMFUd4u4LA++4+z5Q1fOPJRavlORnUdcUZnd1A8MGpSc6HGOM6TVdvc2120TED9wNnANUAitEZJmqro3WUdVvxtS/HpgWc4g6VZ3qVXy9pTh6q+u+WksQxpgB5VjGII5mJrBRVTeraiOwBLigk/qXAQ96GI8nSvLdab/32DiEMWZgEVVvxppFZB4wV1WvdbevBGap6nVx6hYDrwJFqhp2y5qBt4Bm4HZVfTROu4XAQoDCwsIZS5Ys6XG8NTU1ZGdnd7tdc0RZ+HQtnxkd5PMnhXp8/qPpaXx9KdljTPb4wGLsDckeHyRXjHPmzFmlqqXx9nl2iambFgBLo8nBVayqVSJyAvCsiLyjqptiG6nqImARQGlpqZaVlfU4gIqKCnravmhlOeQMoaxs2tEr99CxxNdXkj3GZI8PLMbekOzxQf+IEby9xFQFjIrZLnLL4llAu8tLqlrlfm4GKmg7PpFUbNpvY8xA5GWCWAGMFZHRIhLCSQLL2lcSkXFALvBKTFmuiKS56wXAbGBt+7bJoiQ/iy17DuPV5TpjjEkEzxKEqjYD1wFPAuuAh1R1jYjcJiKxt6wuAJZo22/X8cBKEXkbKMcZg0jaBFGcn0l1fTMHapsSHYoxxvQaT8cgVHU5sLxd2Q/bbd8ap93LwGQvY+tNxe6dTNv21ZKb5d1AtTHG9CUvLzGljJZpv20cwhgzgFiC6AWj8jIRga17bNI+Y8zAYQmiF6QH/QwflG49CGPMgGIJAqD+4DEfojg/k237rAdhjBk4LEHs3QT/XUrhjmeP6TAl+VnWgzDGDCiWIIYUw7BxnPTeb+Gj1T0+zPH5meypaaS63m51NcYMDJYg/AH4/L00BXPgr1dA3f4eHSY6aZ+9Xc4YM1BYggDIHsqaid+FQx/C3xdCJNLtQ0Sn/f7AxiGMMQOEJQhX9aCT4byfwvtPwfM/63b76MNy9vpRY8xAYQkiVumX4JQvQMV/wvtPd6tpdlqAguwQ2+xZCGPMAGEJIpYIfPZOGD4JHr4W9m/tVvPi/CzrQRhjBgxLEO0FM+DSPwMKf70Smuq63LQ4P9PGIIwxA4YliHjyRsPF98CO1fD4t6CL03iX5Gfx0cF66pvCR69sjDFJzhJER046F868Gd56AFYt7lITu5PJGDOQWILozJnfhTHnwBP/FypXHbV6sT0LYYwZQCxBdMbng4sXQc5weOiLcHhPp9Vt2m9jzEBiCeJoMvNg/v9A7R5Y+iWIdDy+MCQzxOCMoN3JZIwZECxBdMVxp8Bn7oQtz8GzP+60anF+pl1iMsYMCJYgumra5TDj3+DFO2HdPzqsVpyfZQnCGDMgeJogRGSuiGwQkY0icnOc/b8Qkbfc5T0RORCz7yoRed9drvIyzi4776cwYjo8+jXYszFulZL8TCr319LY3P35nIwxJpl4liBExA/cDZwHTAAuE5EJsXVU9ZuqOlVVpwL/DfzdbZsH3ALMAmYCt4hIrlexdlkgDS79E/iDzsyvjUeONRTnZxFRqDrQ9QfsjDEmGXnZg5gJbFTVzaraCCwBLuik/mXAg+76p4CnVXWfqu4Hngbmehhr1w0ZBfPuhT0bYNkNRzxEF30WwgaqjTH9XcDDY48EtsdsV+L0CI4gIsXAaCD6Wrd4bUfGabcQWAhQWFhIRUVFj4OtqanpVvvjSy7nhHf/zPv1Q6gq+lxL+YEG59LSv159G/ko2ON4jjW+REj2GJM9PrAYe0Oyxwf9I0bwNkF0xwJgqap2a44KVV0ELAIoLS3VsrKyHgdQUVFBt9rrmfDX/Yx9bzFjT58HxadFY+L7Lz1JMG8EZWUTexzPMceXAMkeY7LHBxZjb0j2+KB/xAjeXmKqAkbFbBe5ZfEsoPXyUnfbJoYIXPgb55Wlf7sKqne4xcKJQ7N5dv0u9tY0JDhIY4zpOS8TxApgrIiMFpEQThJY1r6SiIwDcoFXYoqfBM4VkVx3cPpctyy5pA92HqJrqIa//RuEnfdR/+Az49l5qJ4r/vg6B2vtHdXGmP7JswShqs3AdThf7OuAh1R1jYjcJiLnx1RdACxRbR3tVdV9wI9wkswK4Da3LPkUToDz/xs+eBmevgWAWSfks+jKUjbtquGL971Odb0lCWNM/+PpGISqLgeWtyv7YbvtWztoey9wr2fB9abJ86ByJbx6NxTNgEmf54yThvKby6fz1f9ZxTWLV7L4Sx8jM5QsQz7GGHN09iR1bzn3RzDqVHjseti1DoCzJxRy14KprNy2j4V/WmXviTDG9CuWIHqLPwiX3g9p2c5DdPWHAPjslBH8bN4pvLhxD//ngTfsCWtjTL9hCaI35QyHSxbDvi1w76dg1f3QeJjPzyjiJxdN4tn1u7jxr2/SHLYkYYxJfpYgelvxx52eBMD/3gA/HwfLv8Plo2v5j89OYPk7O/jO0tVEIl17jakxxiSKjZp6YfznYNxnYftrsOKPzitLX1/ENcd/nFHTPsN1bzaRHvTx/y6ajIgkOlpjjInLehBeEYHjT4XP/wFuWgfn3AbVH3Luun/nrewbOf6N/+KXS59B1XoSxpjkZAmiL2QVwOxvwPVvwhUPkzHm43wl8Dg3vHsJm395Hrr+8U7fVGeMMYlgl5j6ks8HY85GxpwNBysp/8sdTNzxCLLkCzCoCGZcBdO/6Ax2G2NMglkPIkFkcBFlX/kFP5vwMF9pvJFK/0go/wn8YiI89EXYXHHEVOLGGNOXrAeRQD6fcPu86Xwj7OMTq2dy19nZXBh+Ct56ANY+BvljoPRLcMpliQ7VGJOCrAeRYAG/j7vmT+Xs8YXc+EwNf8v/qjOofdHvISMPnvw+3Dme8WvvcO6I2vGujVcYY/qE9SCSQNDv49dfmMaX/7SS7z68mrTgNM4/ZQGcsgB2vAMr/kju6r/D4y84DUI5zpxPRTNh1EwoKoWMxL+R1RgzsFiCSBLpQT+Lrizlqvte55t/fYu0gI9PTRwOwyfD5+7i5ewLKDulBLa/DpWvO89YvHAHqPtUdsHJTrIYNRNGzYL8sc6guDHG9JAliCSSEfJz79Uf44p7XuP6v7zJoi/OoOzkYc5OEcgb7SynzHfKGmqgapWbMFbA+n/Am3929qUPbu1hjJoJI2dAWk5ifjBjTL9kCSLJZKcFuP9LM/nCH17lK39exeJ/m8lpJ+bHr5yWDSec6Szg3PW0d6PTu9j+urNsfAZQEB8Mm+BekpoJRR+D3GJnkkFjjInDEkQSGpwR5M/XzGL+71/hmvtX8OdrZnWtoQgUjHWWaVc4ZXUHoGql08PY/hq8sxRWRl+zIZA9DAaNgEEj3WVEzKe7BNK8+DGNMUnOEkSSyssK8cC1s5i/6FWuvu91vjWth7+qjCEw5mxnAecOqN3roeoNOFgJh6rg0IdOz2PL89Bw6MhjZA2NkziK2iaRYEaPf1ZjTHKyBJHEhg1K54FrZ3HJ717h9tfrOJD1Htd8YjQ56cdwWcjnh8KJzhJP/SGo/qg1cRysal3fvw22vQz1B45sl5HHxyQTNhU5d1Rl5DrJqWU9Zjs9+jnYiccYk5QsQSS5EUMy+OtXTuUbi5/nrmfeZ/HLW/nqmSdy1WklZIQ8+HJNH+QsQ0/uuE7jYSdhRBPHoSo4WEXttvVkBYJwqBJ2vgt1+6GxppOTiZMk4iaTXAhlO2MkvqDz2bIecD9DMevuti/QQZsg/uY6pwdlScmYLvE0QYjIXOCXgB+4R1Vvj1PnUuBWQIG3VfULbnkYeMet9oGqnu9lrMmsKDeT66elkz9mGj9/egO3P7Gee17YwnVzTuSyWceTFujjL7xQVutYR4w1FRWUlZW1rdvcCPUHnWQRXeoPtN2ui9nev611v/bui5VOB3gR8KdBKBOC7tLhepZz6ayzdQQ07CSeNp+ROOWROPViyyOMrNwCb1bGnCcaU5bzGcp2ygJpzpiTMR7yLEGIiB+4GzgHqARWiMgyVV0bU2cs8D1gtqruF5FhMYeoU9WpXsXXH00uGszif5vJiq37uOPJDdz6v2v5wwtbuOGsMXx+ehEBfxI+9xAIQfZQZ+mOSASa6yHSBOFmCDe6600Qcbdb1pta93W0Hmlm04a1nHj8CGiqdZbG2pj1w05iqv7IWW+qhaY6Z52+mxNrLMDGLlQUX2vSiCaTloQSk1gCGU6PScRpI373013a7PN1UMfXZrtwx0ZYvctp6wvELO2345XFqaOR1t9VpNn5fbf5Xbuf0b8Lsfva/P6dfUXb34dX1zk/h8/fek6JntsXsx7d52u37XfqtWkXaC1rU+5v+3NF2w4AXvYgZgIbVXUzgIgsAS4A1sbU+TJwt6ruB1DVXR7GM2B8rCSPJQtP5aWNe/nZUxv47sPv8NuKTXzznJP43JQR+HwD4H+WPp/zBdeLttdVcGL7Hs7RqDqJKpos2icXiPmi9bf77H75Sy88x+yPTY05x2Hns7HmyLJoYovG1XjYqVezq7VOc32bHkqbpYeJbzzA+h417RNjADYlOgqJk3BaE8ppDQ2wMgSoOylnZ58cWR79/UXLRs6AL/2z938Kr15YIyLzgLmqeq27fSUwS1Wvi6nzKPAeMBvnMtStqvpPd18z8BbQDNyuqo/GOcdCYCFAYWHhjCVLlvQ43pqaGrKzs3vc3msdxaeqvLU7zN/fb2J7dYSibOGisSGmD/P3+dvq+uufYTLp0xhVgQjifsmIRhD3i6ezz9rD1WRnpCMadpdIzHpHZR1tNwM+Ij4/KgFU/C1LxBfdDsSUxW4H3G1/m3o1tbVkZ2a2OY/zc8aeN9JuX6TdvrYxt9YJd1C3o3O0/9md9abmZgLBICDuAiqt6yDuNi11VGLXpU3b+vShfDjy0z36azBnzpxVqlrawd8R9WQB5uGMO0S3rwR+3a7OP4BHgCAwGtgODHH3jXQ/TwC2Aid2dr4ZM2bosSgvLz+m9l47WnzhcET/9+0qnXNHuRZ/9x/62V+9oOXrd2okEumbALX//xkmA4vx2CV7fKrJFSOwUjv4XvXyQlkVMCpmu8gti1UJLFPVJlXdgtObGAugqlXu52agApjmYaz9ns8nfHbKCJ668Qx+Nm8K+2sbufq+FVz6+1d4bfPeRIdnjOmHvEwQK4CxIjJaRELAAmBZuzqPAmUAIlIAnARsFpFcEUmLKZ9N27EL04GA38clpaN49ltl/OjCSXywr5b5i17lyj++xlvbDyQ6PGNMP+JZglDVZuA64ElgHfCQqq4RkdtEJHrL6pPAXhFZC5QD31HVvTjjYCtF5G23/HaNufvJHF0o4OPKU4t57jtz+MFnxrPmw0NcePdLXHv/StZ9FOdpaWOMacfT5yBUdTmwvF3ZD2PWFbjJXWLrvAxM9jK2VJEe9HPt6SewYObx3PfiFha9sJnzfvkCn5lyHPNmFHHaCfmkB+3BMWPMkexJ6hSRnRbg+rPG8sXTSlj0wiYWv7SVx1d/RHrQxyfGFPDJcYV8ctwwhg9OT3SoxpgkYQkixQzODPKdT43j+k+O5bUt+3h23U7+tX4Xz6xzHkGZcNwgzho/jE+OG8YpRUMGxjMVxpgesQSRotKDfs48aShnnjSUW89X3t9Vw7Prd/Hsul3cXb6R/352I/lZIcpOHsZZ44dx+tiCY5sk0BjT71iCMIgIJxXmcFJhDl8980QO1Dby3Hu7eXb9Lp5Zt5OH36gk4BNmjs7jk+OGcdb4QkYXZCU6bGOMxyxBmCMMyQxxwdSRXDB1JM3hCG98cMDpXazfyY8fX8ePH1/H6IIsJ1mMG0ZpSR6hwMCYe8YY08oShOlUwO9j5ug8Zo7O4+bzxrF9Xy3lG3bxr3W7+PMr2/jji1vISQtw+kkF5DY1kbZpL+OPy2FIZijRoRtjjpElCNMto/Iy+eJpJXzxtBIONzTz0sY9lG/YxbPrd7HzUCMPrH8VgOGD0hl3XA7jhg9ivPt5wtAsgsk446wxJi5LEKbHstICnDtxOOdOHI6q8thT5eSNnsz6HYdY/1E163ZU89LGzTSFnQkhg35hzLAcxg/PaUke447LYWh2Wp9PLGiMOTpLEKZXiAhD0nyccdJQzjip9d0PTeEIm3cfZv2OQ6z7qJr1Ow7x8qa9/P3N1mm58rNCrQljeA7jjxvEmGHZ9gCfMQlmCcJ4Kuj3cfLwHE4ensMFU1vL9x9uZP2O6pbexvodh3jgtW3UNzlvkfP7hFG5GYzKy6QoN4Oi3ExGDsloWR+Wk2bPaBjjMUsQJiFys0KcdmI+p52Y31IWjijb9h52EsdHh9i0+zCVB+p4eu1O9tQ0tmkf8vsYMSSdotxoAsloWR+Zm8GwnHT8lkCMOSaWIEzS8PuEE4Zmc8LQbD49+bg2++oaw1QdqGX7/joq99dRtb+Oyv21VO6v45l1u9hT09CmftAvjIj2OIY4iePQjiZ87+1maE4aBdlp5GWFLIkY0wlLEKZfyAj5GTMshzHDcuLudxJIa9Ko3F/Xsv3shl3srnYSyB/eeb2ljU8gLytEQXZaS9IoyA65nzFlOSHys9IsmZiUYwnCDAhOAslmzLD4r+usbwrz6FPPccKEqeypaXCW6gZ21zSwu7qRPTUNbNlzmD01DS3jILFEIC8zNpk467lZIYZkBhmS4XwOzgg625khskJ9/9pXY3qTJQiTEtKDfoZnOQ/9dUZVOdwYZk+1k0R2Rz9rGluSyp6aBt74oJbd1Q3UNYU7PFbAJzFJI8SQjCCDY5JJNJEMiSaVjBA1jUpzOELAnhcxScAShDExRITstADZaQFKujDfVH1TmEN1TRyoa+JAbRP7axs5WNvEgbpGDtQ65dHtHYfqWb+jmgO1jRxu7Dix8OwTZAT9ZKcHyEkPkJMWICc9SHaas+2UB93y1u3stACDYrYzg36708scE0sQxhyD9KCf9KCfYYO69x6NxuYIB+uaOBhNJG5yeePd9QwvKqG6vomahmaq65upbmimur6JnYfqqa5vpqbBWY5GBLJDAbLSAmSm+ckKBcgM+clOC5CZFiAr5CczFCA7zd9mOyvN77SJrkePEfKTZnNupRRLEMYkQCjgY2iOM54Ra2jNJsrKxh61fTiiHG50EkhNvZNAqhui683UNDQ5yaW+mdrGZg43hqltaOZwQ5gdh+qpbQxzuKHZ+WxsRrVrcft9QsinZL/0DBlBP5khJ0G2rIda1zPc5JkZ8pMRilkPOtvRz/SA3020PtKDfkJ+n/V8koQlCGP6Ib9PGJQeZFAvvKMjElHqm8McbghT2+j0TtokkIZmZ3HX39+yjYLCYdQ1hqltDFPXFKa+KcyOQ03OemOY2qYwdY1hGpqPHPDvilDAR3rASRhpQV9LEklzy9KDPtJitwOt9aq2N7IluIW0gLM/Legj5G+tHwr4nPLofnc9Wm7JqZWnCUJE5gK/BPzAPap6e5w6lwK3Agq8rapfcMuvAn7gVvuxqt7vZazGpCqfT8gMOZeUIO2o9SsqdlBWNqVLxw5HlPomJ4nUNR75WdvoJJeG5gj1TWHqm8PUN0VoaA7T0BRpu6/J2Vfb2My+wxHq3ToNbpv6pjDNEbcrtGFtj/88gn45IqGE/O5nwEfQ7ySS2LKQ30fQ/UyLUxaKSUxBv48Nu5qR93a3nCvoLqGAEPL7CQaktczvI+gX/D7p87viPEsQIuIH7gbOASqBFSKyTFXXxtQZC3wPmK2q+0VkmFueB9wClOIkjlVu2/1exWuM6X1+n5CV5oxh9IXmcIRnyp9j1mmzaWh2kkdjc6RlvSG63hShMRyhwU1AsXXa1HfrNbp1GsPOvpqG5jZljc0Rmtx90bKjXrZ74/WjVGhLhDYJI5pA0gI+JowYxK+/ML3nf3Ad8PK3NhPYqKqbAURkCXABEJvavwzcHf3iV9VdbvmngKdVdZ/b9mlgLvCgh/EaY/q5gN9HekDIzUrs+0hUleaI0hSTXBpikslrr69g8tTpLfubws7SGFaa2m+HIy1ljWFtVz9CU1gZlZvhyc8h2tXRqe4eWGQeMFdVr3W3rwRmqep1MXUeBd4DZuNchrpVVf8pIt8G0lX1x269/wDqVPWOdudYCCwEKCwsnLFkyZIex1tTU0N2dvyHrJJBsscHyR9jsscHFmNvSPb4ILlinDNnzipVLY23L9GD1AFgLFAGFAHPi8jkrjZW1UXAIoDS0lItKyvrcSAVFRUcS3uvJXt8kPwxJnt8YDH2hmSPD/pHjABe3tRcBYyK2S5yy2JVAstUtUlVt+D0JsZ2sa0xxhgPeZkgVgBjRWS0iISABcCydnUexek9ICIFwEnAZuBJ4FwRyRWRXOBct8wYY0wf8ewSk6o2i8h1OF/sfuBeVV0jIrcBK1V1Ga2JYC0QBr6jqnsBRORHOEkG4LbogLUxxpi+4ekYhKouB5a3K/thzLoCN7lL+7b3Avd6GZ8xxpiO2cQqxhhj4rIEYYwxJi5LEMYYY+Ly7EG5viYiu4Ftx3CIAmBPL4XjhWSPD5I/xmSPDyzG3pDs8UFyxVisqkPj7RgwCeJYicjKjp4mTAbJHh8kf4zJHh9YjL0h2eOD/hEj2CUmY4wxHbAEYYwxJi5LEK0WJTqAo0j2+CD5Y0z2+MBi7A3JHh/0jxhtDMIYY0x81oMwxhgTlyUIY4wxcaV8ghCRuSKyQUQ2isjNiY6nPREZJSLlIrJWRNaIyDcSHVM8IuIXkTdF5B+JjiUeERkiIktFZL2IrBOR0xIdUywR+ab7+31XRB4UkfQkiOleEdklIu/GlOWJyNMi8r77mZuEMf7M/T2vFpFHRGRIAkOMG2PMvm+JiLqzWSedlE4QMe/NPg+YAFwmIhMSG9URmoFvqeoE4FTg60kYI8A3gHWJDqITvwT+qarjgFNIolhFZCRwA1CqqpNwZj9ekNioAFiM86rfWDcD/1LVscC/3O1EWsyRMT4NTFLVKTjvmPleXwfVzmKOjBERGYXzKoMP+jqgrkrpBEHMe7NVtRGIvjc7aajqR6r6hrtejfPFNjKxUbUlIkXAZ4B7Eh1LPCIyGDgD+COAqjaq6oGEBnWkAJAhIgEgE/gwwfGgqs8D7afZvwC4312/H7iwL2NqL16MqvqUqja7m6/ivHAsYTr4cwT4BfB/gaS9UyjVE8RIYHvMdiVJ9uUbS0RKgGnAawkOpb27cP6iRxIcR0dGA7uB+9zLYPeISFaig4pS1SrgDpz/SX4EHFTVpxIbVYcKVfUjd30HUJjIYLrgS8ATiQ6iPRG5AKhS1bcTHUtnUj1B9Bsikg08DNyoqocSHU+UiHwW2KWqqxIdSycCwHTgt6o6DThM4i+NtHCv41+Ak8hGAFkickViozo6930uSfu/XxH5d5xLtA8kOpZYIpIJfB/44dHqJlqqJ4h+8e5rEQniJIcHVPXviY6nndnA+SKyFecS3SdF5H8SG9IRKoFKVY32vJbiJIxkcTawRVV3q2oT8Hfg4wmOqSM7ReQ4APdzV4LjiUtErgY+C1yuyfew14k4/xl42/13UwS8ISLDExpVHKmeILry3uyEEhHBuXa+TlXvTHQ87anq91S1SFVLcP78nlXVpPrfr6ruALaLyMlu0VnA2gSG1N4HwKkikun+vs8iiQbR21kGXOWuXwU8lsBY4hKRuTiXPM9X1dpEx9Oeqr6jqsNUtcT9d1MJTHf/niaVlE4Q7kBW9L3Z64CHVHVNYqM6wmzgSpz/mb/lLp9OdFD90PXAAyKyGpgK/L/EhtPK7dksBd4A3sH5d5nwqRhE5EHgFeBkEakUkWuA24FzROR9nJ7P7UkY46+BHOBp99/L75Iwxn7BptowxhgTV0r3IIwxxnTMEoQxxpi4LEEYY4yJyxKEMcaYuCxBGGOMicsShBlwRCQ/5pbgHSJSFbMdOkrbUhH5VRfO8XLvRewdEblVRL6d6DhM/xRIdADG9DZV3YvzrAMicitQo6p3RPeLSCBmMrf2bVcCK7twjmR90tmYXmM9CJMSRGSxiPxORF4D/ktEZorIK+7kfS9Hn7IWkbLoOy3c/33fKyIVIrJZRG6IOV5NTP2KmHdNPOA+DY2IfNotWyUiv5I478oQ5z0aPxORFe77C74Sc9znReRxcd5X8jsR8bn7LhORd8R5d8RPY441V0TeEJG3ReRfMaeZEO9nMOZorAdhUkkR8HFVDYvIIOB0VW0WkbNxnqz+fJw244A5OE/mbhCR37rzJcWaBkzEmaL7JWC2iKwEfg+coapb3Kdp47kGZ/bWj4lIGvCSiERncp2J856SbcA/gYvdS1s/BWYA+4GnRORC97x/iDlfXjd/BmOOYAnCpJK/qWrYXR8M3C8iY3FmJA120OZxVW0AGkRkF8701pXt6ryuqpUAIvIWUALUAJtVdYtb50FgYZzjnwtMEZF5MXGNBRrd4252j/sg8AmgCahQ1d1u+QM477oIA89Hz6eqse8f6MrPYMwRLEGYVHI4Zv1HQLmqXiTOezYqOmjTELMeJv6/ma7U6YgA16vqk20KRco4cirtns6LcyzxmRRmYxAmVQ2mdWr3qz04/gbgBDf5AMzvoN6TwNfcKd0RkZOk9WVGM92Zhn1u+xeB14EzRaRAnFfmXgY8h/PmtDNEZLR7nLz2JzKmu+x/EiZV/RfOJaYfAI/39sFVtU5E/g/wTxE5jDO1fDz34FySesMd3N5N62s8V+DMTDoGKAceUdWIiNzsbgvO5aPHAERkIfB3N6HsAs7p7Z/LpBabzdUYj4hItqrWuF/8dwPvq+ovuti2DPi2qn7WwxCN6ZRdYjLGO192B63X4FzS+n1iwzGme6wHYYwxJi7rQRhjjInLEoQxxpi4LEEYY4yJyxKEMcaYuCxBGGOMiev/B3ibl/r09zLFAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def model_hybrid(embedding_dim):\n",
    "    input_user = Input(shape=(1,))\n",
    "    input_item = Input(shape=(1,))\n",
    "    input_item_e = Input(shape=(feature_dim,))\n",
    "    \n",
    "    embedding_user = Embedding(input_dim=max_user_id + 1, output_dim=embedding_dim)(input_user)\n",
    "    embedding_item = Embedding(input_dim=max_item_id + 1, output_dim=embedding_dim)(input_item)\n",
    "\n",
    "    #e1 = Flatten()(embedding_user)\n",
    "    #e2 = Flatten()(embedding_item)\n",
    "    e1 = embedding_user\n",
    "    e2 = embedding_item\n",
    "    \n",
    "    t = Dense(embedding_dim, activation='relu')(input_item_e)\n",
    "    e2 = Add()([e2, t])\n",
    "    #e2 = Dense(embedding_dim, activation='relu')(e2)\n",
    "\n",
    "    combined = Dot(axes=2)([e1, e2])\n",
    "    rating = Dense(1, activation='linear')(combined)\n",
    "\n",
    "    model = Model(inputs=[input_user, input_item, input_item_e], outputs=rating)\n",
    "    \n",
    "    return model\n",
    "\n",
    "#\n",
    "# Model training\n",
    "# \n",
    "model_hybrid_k = model_hybrid(embedding_dim = 8)\n",
    "model_hybrid_k.compile(optimizer='adam', loss='mse', metrics=[tf.keras.metrics.RootMeanSquaredError()]) \n",
    "history_hybrid = model_hybrid_k.fit([x1_train, x2_train, x3_train], y_train, \n",
    "                           batch_size=256, \n",
    "                           epochs=16, \n",
    "                           validation_data=([x1_test, x2_test, x3_test], y_test), \n",
    "                           verbose=1)\n",
    "plot_keras_training_hitory(history_hybrid)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "wired-cannon",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "+-----+----------------------------------------------------+----------+\n",
      "|     | title                                              |   rating |\n",
      "|-----+----------------------------------------------------+----------|\n",
      "| 121 | Lord of the Rings: The Fellowship of the Ring, ... |        5 |\n",
      "| 155 | Freaks (1932)                                      |        5 |\n",
      "| 131 | Lord of the Rings: The Two Towers, The (2002)      |        5 |\n",
      "| 145 | Lord of the Rings: The Return of the King, The ... |        5 |\n",
      "|   0 | Jumanji (1995)                                     |        4 |\n",
      "| 106 | O Brother, Where Art Thou? (2000)                  |        4 |\n",
      "| 100 | Hook (1991)                                        |        4 |\n",
      "| 101 | Misery (1990)                                      |        4 |\n",
      "| 103 | Crouching Tiger, Hidden Dragon (Wo hu cang long... |        4 |\n",
      "| 104 | Dungeons & Dragons (2000)                          |        4 |\n",
      "+-----+----------------------------------------------------+----------+\n",
      "160 rows x 2 columns\n",
      "+----+-------------------------------------+----------+\n",
      "|    | title                               |   rating |\n",
      "|----+-------------------------------------+----------|\n",
      "|  0 | Penguins of Madagascar (2014)       |  4.2071  |\n",
      "|  1 | Nico and Dani (Krámpack) (2000)     |  4.19753 |\n",
      "|  2 | North & South (2004)                |  4.19016 |\n",
      "|  3 | Wild (2014)                         |  4.18598 |\n",
      "|  4 | Sweeney Todd (2006)                 |  4.1834  |\n",
      "|  5 | Stunt Man, The (1980)               |  4.17744 |\n",
      "|  6 | Twilight Zone: The Movie (1983)     |  4.17293 |\n",
      "|  7 | In Harm's Way (1965)                |  4.16616 |\n",
      "|  8 | Bella (2006)                        |  4.16578 |\n",
      "|  9 | Ballad of Jack and Rose, The (2005) |  4.16335 |\n",
      "+----+-------------------------------------+----------+\n",
      "10 rows x 2 columns\n"
     ]
    }
   ],
   "source": [
    "def recommend_for_user_hybrid(model, user_id, k):\n",
    "    x1, x2, x3 = np.repeat(user_id, n_items), item_ids, np.vstack(items['item_features'].values)\n",
    "    ratings = model.predict([x1, x2, x3]).ravel()\n",
    "    top_k_indexes = ratings.argsort()[-k:][::-1]\n",
    "    return x2[top_k_indexes], ratings[top_k_indexes]\n",
    "\n",
    "#\n",
    "# Generate recommendations for an example user\n",
    "#\n",
    "user_id = 0\n",
    "k_recomendations = 10\n",
    "print_df(get_items(user_id), col_width = 50, rows = 10)\n",
    "r_items, r_ratings = recommend_for_user_hybrid(model_hybrid_k, user_id, k_recomendations)\n",
    "show_recommendations(r_items, r_ratings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "vanilla-honolulu",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}