notebooks/rnn_ner/deep_pavlov_ner.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Few-shot распознавание сущностей"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Для обучения ner-моделей обычно нужно много данных. Но есть способ сократить требование к объёму данных - использовать предобученные языковые модели вроде Берта. Из-за того, что языковая модель даёт хорошие эмбединги, можно завести ner всего с 10 предложений."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Такой подхол называется Few-shot learning. Подробнее с тем, как это работает, мы разберемся, когда будем проходить нейроныые сети. А пока научимся пользоваться готовыми штуками, чтобы решать практические задачи. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Берт и возможность тренировать few-shot ner модели есть в deep pavlov (http://docs.deeppavlov.ai/en/master/features/models/ner.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "colab_type": "code",
    "id": "B7CKfNAchJ4d",
    "outputId": "a653e5aa-5596-4248-bb0d-8a33d8000c7f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting deeppavlov\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/d7/9d/453d101981b293441be889ba91d6983abdeeb2b3abc070b47a4044f6a64b/deeppavlov-0.7.1-py3-none-any.whl (735kB)\n",
      "\u001b[K     |████████████████████████████████| 737kB 1.5MB/s eta 0:00:01\n",
      "\u001b[?25hCollecting fastapi==0.38.1\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/37/59/1a42dde38f1ae2a7a318947e54fabd1fc02ac423ecc91957c417065e7cc6/fastapi-0.38.1-py3-none-any.whl (160kB)\n",
      "\u001b[K     |████████████████████████████████| 163kB 5.6MB/s eta 0:00:01\n",
      "\u001b[?25hCollecting nltk==3.2.5\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/cc/87/76e691bbf1759ad6af5831649aae6a8d2fa184a1bcc71018ca6300399e5f/nltk-3.2.5.tar.gz (1.2MB)\n",
      "\u001b[K     |████████████████████████████████| 1.2MB 5.5MB/s eta 0:00:01\n",
      "\u001b[?25hCollecting pymorphy2-dicts-ru\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/7c/9b/358faaff410f65a4ad159275e897b5956dcb20576c5b8e764b971c1634d7/pymorphy2_dicts_ru-2.4.404381.4453942-py2.py3-none-any.whl (8.0MB)\n",
      "\u001b[K     |████████████████████████████████| 8.0MB 4.8MB/s eta 0:00:01\n",
      "\u001b[?25hCollecting uvicorn==0.9.0\n",
      "  Downloading https://files.pythonhosted.org/packages/5a/a9/2ab492155d9b76cf109c2370e201822ba3c7f4aed85f5a1b4d22907e7206/uvicorn-0.9.0.tar.gz\n",
      "Collecting numpy==1.16.4\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/87/2d/e4656149cbadd3a8a0369fcd1a9c7d61cc7b87b3903b85389c70c989a696/numpy-1.16.4-cp36-cp36m-manylinux1_x86_64.whl (17.3MB)\n",
      "\u001b[K     |████████████████████████████████| 17.3MB 4.6MB/s eta 0:00:01     |██████████████████▌             | 10.0MB 4.9MB/s eta 0:00:02\n",
      "\u001b[?25hCollecting overrides==1.9\n",
      "  Downloading https://files.pythonhosted.org/packages/de/55/3100c6d14c1ed177492fcf8f07c4a7d2d6c996c0a7fc6a9a0a41308e7eec/overrides-1.9.tar.gz\n",
      "Collecting fuzzywuzzy==0.17.0\n",
      "  Downloading https://files.pythonhosted.org/packages/d8/f1/5a267addb30ab7eaa1beab2b9323073815da4551076554ecc890a3595ec9/fuzzywuzzy-0.17.0-py2.py3-none-any.whl\n",
      "Collecting h5py==2.9.0\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/30/99/d7d4fbf2d02bb30fb76179911a250074b55b852d34e98dd452a9f394ac06/h5py-2.9.0-cp36-cp36m-manylinux1_x86_64.whl (2.8MB)\n",
      "\u001b[K     |████████████████████████████████| 2.8MB 5.1MB/s eta 0:00:01\n",
      "\u001b[?25hCollecting pandas==0.24.2\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/19/74/e50234bc82c553fecdbd566d8650801e3fe2d6d8c8d940638e3d8a7c5522/pandas-0.24.2-cp36-cp36m-manylinux1_x86_64.whl (10.1MB)\n",
      "\u001b[K     |████████████████████████████████| 10.1MB 1.1MB/s eta 0:00:01\n",
      "\u001b[?25hCollecting pyopenssl==19.0.0\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/01/c8/ceb170d81bd3941cbeb9940fc6cc2ef2ca4288d0ca8929ea4db5905d904d/pyOpenSSL-19.0.0-py2.py3-none-any.whl (53kB)\n",
      "\u001b[K     |████████████████████████████████| 61kB 3.8MB/s eta 0:00:011\n",
      "\u001b[?25hRequirement already satisfied: requests==2.22.0 in /usr/local/lib/python3.6/dist-packages (from deeppavlov) (2.22.0)\n",
      "Collecting rusenttokenize==0.0.5\n",
      "  Downloading https://files.pythonhosted.org/packages/25/4c/a2f00be5def774a3df2e5387145f1cb54e324607ec4a7e23f573645946e7/rusenttokenize-0.0.5-py3-none-any.whl\n",
      "Collecting keras==2.2.4\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/5e/10/aa32dad071ce52b5502266b5c659451cfd6ffcbf14e6c8c4f16c0ff5aaab/Keras-2.2.4-py2.py3-none-any.whl (312kB)\n",
      "\u001b[K     |████████████████████████████████| 317kB 5.1MB/s eta 0:00:01\n",
      "\u001b[?25hCollecting scikit-learn==0.21.2\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/85/04/49633f490f726da6e454fddc8e938bbb5bfed2001681118d3814c219b723/scikit_learn-0.21.2-cp36-cp36m-manylinux1_x86_64.whl (6.7MB)\n",
      "\u001b[K     |████████████████████████████████| 6.7MB 11.0MB/s eta 0:00:01\n",
      "\u001b[?25hCollecting tqdm==4.32.2\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/9f/3d/7a6b68b631d2ab54975f3a4863f3c4e9b26445353264ef01f465dc9b0208/tqdm-4.32.2-py2.py3-none-any.whl (50kB)\n",
      "\u001b[K     |████████████████████████████████| 51kB 2.1MB/s eta 0:00:011\n",
      "\u001b[?25hCollecting pymorphy2==0.8\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/a3/33/fff9675c68b5f6c63ec8c6e6ff57827dda28a1fa5b2c2d727dffff92dd47/pymorphy2-0.8-py2.py3-none-any.whl (46kB)\n",
      "\u001b[K     |████████████████████████████████| 51kB 3.5MB/s eta 0:00:011\n",
      "\u001b[?25hCollecting scipy==1.3.0\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/72/4c/5f81e7264b0a7a8bd570810f48cd346ba36faedbd2ba255c873ad556de76/scipy-1.3.0-cp36-cp36m-manylinux1_x86_64.whl (25.2MB)\n",
      "\u001b[K     |████████████████████████████████| 25.2MB 5.3MB/s eta 0:00:01     |███████████████████████████████▏| 24.5MB 5.3MB/s eta 0:00:01\n",
      "\u001b[?25hCollecting Cython==0.29.12\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/58/2f/b08ad77c639040baafc891621f0cfdb209e2266404ca13c3167970a6f6d6/Cython-0.29.12-cp36-cp36m-manylinux1_x86_64.whl (2.1MB)\n",
      "\u001b[K     |████████████████████████████████| 2.1MB 4.9MB/s eta 0:00:01\n",
      "\u001b[?25hCollecting pytelegrambotapi==3.6.6\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/25/5e/9711642455c4e17b1202d4f6403ede0fef37fc145038aee7193f3b24445e/pyTelegramBotAPI-3.6.6.tar.gz (49kB)\n",
      "\u001b[K     |████████████████████████████████| 51kB 3.2MB/s eta 0:00:011\n",
      "\u001b[?25hCollecting pydantic<=0.32.2,>=0.32.2\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/86/78/2edcc6e65ec020403ac6ba6deb54c1cf227c49232ce9d98a1ae7ebdfa3a1/pydantic-0.32.2-cp36-cp36m-manylinux1_x86_64.whl (5.1MB)\n",
      "\u001b[K     |████████████████████████████████| 5.1MB 4.2MB/s eta 0:00:01\n",
      "\u001b[?25hCollecting starlette<=0.12.8,>=0.11.1\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/97/4a/90a6a8685fcbdcf544a2293d0d0211ecba3cd7f309b831499d5c895383cb/starlette-0.12.8.tar.gz (45kB)\n",
      "\u001b[K     |████████████████████████████████| 51kB 3.3MB/s eta 0:00:011\n",
      "\u001b[?25hRequirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from nltk==3.2.5->deeppavlov) (1.13.0)\n",
      "Collecting click==7.*\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl (81kB)\n",
      "\u001b[K     |████████████████████████████████| 81kB 3.6MB/s eta 0:00:011\n",
      "\u001b[?25hCollecting h11==0.8.*\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/f9/f3/8e4cf5fa1a3d8bda942a0b1cf92f87815494216fd439f82eb99073141ba0/h11-0.8.1-py2.py3-none-any.whl (55kB)\n",
      "\u001b[K     |████████████████████████████████| 61kB 4.1MB/s eta 0:00:011\n",
      "\u001b[?25hCollecting websockets==8.*\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/bb/d9/856af84843912e2853b1b6e898ac8b802989fcf9ecf8e8445a1da263bf3b/websockets-8.1-cp36-cp36m-manylinux2010_x86_64.whl (78kB)\n",
      "\u001b[K     |████████████████████████████████| 81kB 2.8MB/s eta 0:00:01\n",
      "\u001b[?25hCollecting httptools==0.0.13\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/1b/03/215969db11abe8741e9c266a4cbe803a372bd86dd35fa0084c4df6d4bd00/httptools-0.0.13.tar.gz (104kB)\n",
      "\u001b[K     |████████████████████████████████| 112kB 5.4MB/s eta 0:00:01\n",
      "\u001b[?25hCollecting uvloop==0.*\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/41/48/586225bbb02d3bdca475b17e4be5ce5b3f09da2d6979f359916c1592a687/uvloop-0.14.0-cp36-cp36m-manylinux2010_x86_64.whl (3.9MB)\n",
      "\u001b[K     |████████████████████████████████| 3.9MB 5.7MB/s eta 0:00:01\n",
      "\u001b[?25hRequirement already satisfied: python-dateutil>=2.5.0 in /usr/local/lib/python3.6/dist-packages (from pandas==0.24.2->deeppavlov) (2.8.1)\n",
      "Collecting pytz>=2011k\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl (509kB)\n",
      "\u001b[K     |████████████████████████████████| 512kB 5.2MB/s eta 0:00:01\n",
      "\u001b[?25hCollecting cryptography>=2.3\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/ca/9a/7cece52c46546e214e10811b36b2da52ce1ea7fa203203a629b8dfadad53/cryptography-2.8-cp34-abi3-manylinux2010_x86_64.whl (2.3MB)\n",
      "\u001b[K     |████████████████████████████████| 2.3MB 5.6MB/s eta 0:00:01\n",
      "\u001b[?25hRequirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests==2.22.0->deeppavlov) (1.25.6)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests==2.22.0->deeppavlov) (2019.9.11)\n",
      "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests==2.22.0->deeppavlov) (3.0.4)\n",
      "Requirement already satisfied: idna<2.9,>=2.5 in /usr/lib/python3/dist-packages (from requests==2.22.0->deeppavlov) (2.6)\n",
      "Requirement already satisfied: keras-applications>=1.0.6 in /usr/local/lib/python3.6/dist-packages (from keras==2.2.4->deeppavlov) (1.0.8)\n",
      "Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from keras==2.2.4->deeppavlov) (1.1.0)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting pyyaml\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/8d/c9/e5be955a117a1ac548cdd31e37e8fd7b02ce987f9655f5c7563c656d5dcb/PyYAML-5.2.tar.gz (265kB)\n",
      "\u001b[K     |████████████████████████████████| 266kB 4.9MB/s eta 0:00:01\n",
      "\u001b[?25hCollecting joblib>=0.11\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/28/5c/cf6a2b65a321c4a209efcdf64c2689efae2cb62661f8f6f4bb28547cf1bf/joblib-0.14.1-py2.py3-none-any.whl (294kB)\n",
      "\u001b[K     |████████████████████████████████| 296kB 4.3MB/s eta 0:00:01\n",
      "\u001b[?25hCollecting pymorphy2-dicts<3.0,>=2.4\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/02/51/2465fd4f72328ab50877b54777764d928da8cb15b74e2680fc1bd8cb3173/pymorphy2_dicts-2.4.393442.3710985-py2.py3-none-any.whl (7.1MB)\n",
      "\u001b[K     |████████████████████████████████| 7.1MB 4.9MB/s eta 0:00:01\n",
      "\u001b[?25hCollecting dawg-python>=0.7\n",
      "  Downloading https://files.pythonhosted.org/packages/6a/84/ff1ce2071d4c650ec85745766c0047ccc3b5036f1d03559fd46bb38b5eeb/DAWG_Python-0.7.2-py2.py3-none-any.whl\n",
      "Collecting docopt>=0.6\n",
      "  Downloading https://files.pythonhosted.org/packages/a2/55/8f8cab2afd404cf578136ef2cc5dfb50baa1761b68c9da1fb1e4eed343c9/docopt-0.6.2.tar.gz\n",
      "Collecting dataclasses>=0.6; python_version < \"3.7\"\n",
      "  Downloading https://files.pythonhosted.org/packages/e1/d2/6f02df2616fd4016075f60157c7a0452b38d8f7938ae94343911e0fb0b09/dataclasses-0.7-py3-none-any.whl\n",
      "Collecting cffi!=1.11.3,>=1.8\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/49/72/0d42f94fe94afa8030350c26e9d787219f3f008ec9bf6b86c66532b29236/cffi-1.13.2-cp36-cp36m-manylinux1_x86_64.whl (397kB)\n",
      "\u001b[K     |████████████████████████████████| 399kB 5.1MB/s eta 0:00:01\n",
      "\u001b[?25hCollecting pycparser\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz (158kB)\n",
      "\u001b[K     |████████████████████████████████| 163kB 5.6MB/s eta 0:00:01\n",
      "\u001b[?25hBuilding wheels for collected packages: nltk, uvicorn, overrides, pytelegrambotapi, starlette, httptools, pyyaml, docopt, pycparser\n",
      "  Building wheel for nltk (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for nltk: filename=nltk-3.2.5-cp36-none-any.whl size=1392927 sha256=7508bbe2624b8a864d22247016e97f60d23b883b6032ef66d2ad30650c27663b\n",
      "  Stored in directory: /root/.cache/pip/wheels/da/8c/38/a8a36581975f8d03275c49960019f955e4d19fd14ae7e42d3d\n",
      "  Building wheel for uvicorn (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for uvicorn: filename=uvicorn-0.9.0-cp36-none-any.whl size=38247 sha256=1f44dc4e1be507021e3fa40b8093f38607dd380e9bfdcc5705b5aeea01775afc\n",
      "  Stored in directory: /root/.cache/pip/wheels/19/e5/d1/a50d405d3bb18fac538ef9606ed9b6cd5efb6e06b6de834507\n",
      "  Building wheel for overrides (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for overrides: filename=overrides-1.9-cp36-none-any.whl size=5046 sha256=8f77bdbdf41b538a81f1505dfdd72c2158ed62c9ecdc75e686dfcf1a935401e0\n",
      "  Stored in directory: /root/.cache/pip/wheels/8d/52/86/e5a83b1797e7d263b458d2334edd2704c78508b3eea9323718\n",
      "  Building wheel for pytelegrambotapi (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for pytelegrambotapi: filename=pyTelegramBotAPI-3.6.6-cp36-none-any.whl size=55074 sha256=fb0a441d12d3d2371ebdbadabd51d2f6e6605759593303f8f653be19d17f050d\n",
      "  Stored in directory: /root/.cache/pip/wheels/ae/69/d7/26f1fb04ac4d4c95bff643cea765a8e91c4348da25b4744e08\n",
      "  Building wheel for starlette (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for starlette: filename=starlette-0.12.8-cp36-none-any.whl size=59348 sha256=c2ae66ce30ebf8b6dfaba25e665b18a082e3a59ea5ccb71fa74d2405ca7ceb0b\n",
      "  Stored in directory: /root/.cache/pip/wheels/fd/a0/1d/17eb20c5742e3236799a7883e56325823d57fcd8ce2a0c9348\n",
      "  Building wheel for httptools (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for httptools: filename=httptools-0.0.13-cp36-cp36m-linux_x86_64.whl size=213777 sha256=54c637c352a57543044a0c7bd1783239f45224b64724851f30514c638585108b\n",
      "  Stored in directory: /root/.cache/pip/wheels/e8/3e/2e/013f99b42efc25cf3589730cf380738e46b1e5edaf2f78d525\n",
      "  Building wheel for pyyaml (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for pyyaml: filename=PyYAML-5.2-cp36-cp36m-linux_x86_64.whl size=45500 sha256=2e64e0829cfb7bc508232505be5504055b5d5c03bdd1500aeb82bcba7257753c\n",
      "  Stored in directory: /root/.cache/pip/wheels/54/b7/c7/2ada654ee54483c9329871665aaf4a6056c3ce36f29cf66e67\n",
      "  Building wheel for docopt (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=19852 sha256=e21094a8d0034d1e6a63886293986ca96679c5fffe3bfa72d6547298c878cb41\n",
      "  Stored in directory: /root/.cache/pip/wheels/9b/04/dd/7daf4150b6d9b12949298737de9431a324d4b797ffd63f526e\n",
      "  Building wheel for pycparser (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for pycparser: filename=pycparser-2.19-py2.py3-none-any.whl size=112040 sha256=8d3b6b5f254703234fa2c8c6354ab6fd9e3b660ccd92dde607994a14d57a7324\n",
      "  Stored in directory: /root/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511\n",
      "Successfully built nltk uvicorn overrides pytelegrambotapi starlette httptools pyyaml docopt pycparser\n",
      "Installing collected packages: dataclasses, pydantic, starlette, fastapi, nltk, pymorphy2-dicts-ru, click, h11, websockets, httptools, uvloop, uvicorn, numpy, overrides, fuzzywuzzy, h5py, pytz, pandas, pycparser, cffi, cryptography, pyopenssl, rusenttokenize, scipy, pyyaml, keras, joblib, scikit-learn, tqdm, pymorphy2-dicts, dawg-python, docopt, pymorphy2, Cython, pytelegrambotapi, deeppavlov\n",
      "  Found existing installation: numpy 1.17.3\n",
      "    Uninstalling numpy-1.17.3:\n",
      "      Successfully uninstalled numpy-1.17.3\n",
      "  Found existing installation: h5py 2.10.0\n",
      "    Uninstalling h5py-2.10.0:\n",
      "      Successfully uninstalled h5py-2.10.0\n",
      "  Found existing installation: cryptography 2.1.4\n",
      "    Uninstalling cryptography-2.1.4:\n",
      "      Successfully uninstalled cryptography-2.1.4\n",
      "Successfully installed Cython-0.29.12 cffi-1.13.2 click-7.0 cryptography-2.8 dataclasses-0.7 dawg-python-0.7.2 deeppavlov-0.7.1 docopt-0.6.2 fastapi-0.38.1 fuzzywuzzy-0.17.0 h11-0.8.1 h5py-2.9.0 httptools-0.0.13 joblib-0.14.1 keras-2.2.4 nltk-3.2.5 numpy-1.16.4 overrides-1.9 pandas-0.24.2 pycparser-2.19 pydantic-0.32.2 pymorphy2-0.8 pymorphy2-dicts-2.4.393442.3710985 pymorphy2-dicts-ru-2.4.404381.4453942 pyopenssl-19.0.0 pytelegrambotapi-3.6.6 pytz-2019.3 pyyaml-5.2 rusenttokenize-0.0.5 scikit-learn-0.21.2 scipy-1.3.0 starlette-0.12.8 tqdm-4.32.2 uvicorn-0.9.0 uvloop-0.14.0 websockets-8.1\n"
     ]
    }
   ],
   "source": [
    "!pip install deeppavlov"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting tensorflow==1.15.0\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/3f/98/5a99af92fb911d7a88a0005ad55005f35b4c1ba8d75fba02df726cd936e6/tensorflow-1.15.0-cp36-cp36m-manylinux2010_x86_64.whl (412.3MB)\n",
      "\u001b[K     |████████████████████████████████| 412.3MB 37kB/s  eta 0:00:01     |██████████▌                     | 135.1MB 4.5MB/s eta 0:01:02     |████████████                    | 155.3MB 5.2MB/s eta 0:00:50     |██████████████████████▏         | 285.2MB 4.7MB/s eta 0:00:27\n",
      "\u001b[?25hRequirement already satisfied: absl-py>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (0.8.1)\n",
      "Requirement already satisfied: astor>=0.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (0.8.0)\n",
      "Requirement already satisfied: termcolor>=1.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.1.0)\n",
      "Requirement already satisfied: grpcio>=1.8.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.25.0)\n",
      "Requirement already satisfied: opt-einsum>=2.3.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (3.1.0)\n",
      "Collecting tensorflow-estimator==1.15.1\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/de/62/2ee9cd74c9fa2fa450877847ba560b260f5d0fb70ee0595203082dafcc9d/tensorflow_estimator-1.15.1-py2.py3-none-any.whl (503kB)\n",
      "\u001b[K     |████████████████████████████████| 512kB 4.5MB/s eta 0:00:01\n",
      "\u001b[?25hRequirement already satisfied: wheel>=0.26 in /usr/lib/python3/dist-packages (from tensorflow==1.15.0) (0.30.0)\n",
      "Requirement already satisfied: keras-preprocessing>=1.0.5 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.1.0)\n",
      "Requirement already satisfied: keras-applications>=1.0.8 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.0.8)\n",
      "Requirement already satisfied: gast==0.2.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (0.2.2)\n",
      "Requirement already satisfied: protobuf>=3.6.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (3.10.0)\n",
      "Requirement already satisfied: wrapt>=1.11.1 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.11.2)\n",
      "Requirement already satisfied: numpy<2.0,>=1.16.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.16.4)\n",
      "Collecting tensorboard<1.16.0,>=1.15.0\n",
      "\u001b[?25l  Downloading https://files.pythonhosted.org/packages/1e/e9/d3d747a97f7188f48aa5eda486907f3b345cd409f0a0850468ba867db246/tensorboard-1.15.0-py3-none-any.whl (3.8MB)\n",
      "\u001b[K     |████████████████████████████████| 3.8MB 5.4MB/s eta 0:00:01     |████████████████████████████    | 3.3MB 5.4MB/s eta 0:00:01\n",
      "\u001b[?25hRequirement already satisfied: google-pasta>=0.1.6 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (0.1.8)\n",
      "Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow==1.15.0) (1.13.0)\n",
      "Requirement already satisfied: h5py in /usr/local/lib/python3.6/dist-packages (from keras-applications>=1.0.8->tensorflow==1.15.0) (2.9.0)\n",
      "Requirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from protobuf>=3.6.1->tensorflow==1.15.0) (41.6.0)\n",
      "Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow==1.15.0) (0.16.0)\n",
      "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard<1.16.0,>=1.15.0->tensorflow==1.15.0) (3.1.1)\n",
      "Installing collected packages: tensorflow-estimator, tensorboard, tensorflow\n",
      "Successfully installed tensorboard-1.15.0 tensorflow-1.15.0 tensorflow-estimator-1.15.1\n"
     ]
    }
   ],
   "source": [
    "!pip install tensorflow==1.15.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading package lists... Done\n",
      "Building dependency tree       \n",
      "Reading state information... Done\n",
      "The following additional packages will be installed:\n",
      "  git-man libcurl3-gnutls liberror-perl\n",
      "Suggested packages:\n",
      "  gettext-base git-daemon-run | git-daemon-sysvinit git-doc git-el git-email\n",
      "  git-gui gitk gitweb git-cvs git-mediawiki git-svn\n",
      "The following NEW packages will be installed:\n",
      "  git git-man libcurl3-gnutls liberror-perl\n",
      "0 upgraded, 4 newly installed, 0 to remove and 42 not upgraded.\n",
      "Need to get 4950 kB of archives.\n",
      "After this operation, 34.6 MB of additional disk space will be used.\n",
      "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libcurl3-gnutls amd64 7.58.0-2ubuntu3.8 [213 kB]\n",
      "Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 liberror-perl all 0.17025-1 [22.8 kB]\n",
      "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 git-man all 1:2.17.1-1ubuntu0.5 [803 kB]\n",
      "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 git amd64 1:2.17.1-1ubuntu0.5 [3912 kB]\n",
      "Fetched 4950 kB in 1s (3695 kB/s)\n",
      "debconf: delaying package configuration, since apt-utils is not installed\n",
      "Selecting previously unselected package libcurl3-gnutls:amd64.\n",
      "(Reading database ... 16326 files and directories currently installed.)\n",
      "Preparing to unpack .../libcurl3-gnutls_7.58.0-2ubuntu3.8_amd64.deb ...\n",
      "Unpacking libcurl3-gnutls:amd64 (7.58.0-2ubuntu3.8) ...\n",
      "Selecting previously unselected package liberror-perl.\n",
      "Preparing to unpack .../liberror-perl_0.17025-1_all.deb ...\n",
      "Unpacking liberror-perl (0.17025-1) ...\n",
      "Selecting previously unselected package git-man.\n",
      "Preparing to unpack .../git-man_1%3a2.17.1-1ubuntu0.5_all.deb ...\n",
      "Unpacking git-man (1:2.17.1-1ubuntu0.5) ...\n",
      "Selecting previously unselected package git.\n",
      "Preparing to unpack .../git_1%3a2.17.1-1ubuntu0.5_amd64.deb ...\n",
      "Unpacking git (1:2.17.1-1ubuntu0.5) ...\n",
      "Setting up git-man (1:2.17.1-1ubuntu0.5) ...\n",
      "Setting up liberror-perl (0.17025-1) ...\n",
      "Setting up libcurl3-gnutls:amd64 (7.58.0-2ubuntu3.8) ...\n",
      "Processing triggers for libc-bin (2.27-3ubuntu1) ...\n",
      "Setting up git (1:2.17.1-1ubuntu0.5) ...\n"
     ]
    }
   ],
   "source": [
    "!apt-get --yes install git"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 235
    },
    "colab_type": "code",
    "id": "EOtoNuwklpAG",
    "outputId": "671951d0-0875-40fa-ab37-eb1fad8904cd"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting git+https://github.com/deepmipt/bert.git@feat/multi_gpu\n",
      "  Cloning https://github.com/deepmipt/bert.git (to revision feat/multi_gpu) to /tmp/pip-req-build-zhp57c9m\n",
      "  Running command git clone -q https://github.com/deepmipt/bert.git /tmp/pip-req-build-zhp57c9m\n",
      "  Running command git checkout -b feat/multi_gpu --track origin/feat/multi_gpu\n",
      "  Switched to a new branch 'feat/multi_gpu'\n",
      "  Branch 'feat/multi_gpu' set up to track remote branch 'feat/multi_gpu' from 'origin'.\n",
      "Building wheels for collected packages: bert-dp\n",
      "  Building wheel for bert-dp (setup.py) ... \u001b[?25ldone\n",
      "\u001b[?25h  Created wheel for bert-dp: filename=bert_dp-1.0-cp36-none-any.whl size=20140 sha256=99ccc544941d70a8886a626f6ac5deb2aa51bc2add1739866b142d4b046b9bcb\n",
      "  Stored in directory: /tmp/pip-ephem-wheel-cache-c1c9539p/wheels/1e/41/94/886107eaf932532594886fd8bfc9cb9d4db632e94add49d326\n",
      "Successfully built bert-dp\n",
      "Installing collected packages: bert-dp\n",
      "Successfully installed bert-dp-1.0\n"
     ]
    }
   ],
   "source": [
    "!pip install git+https://github.com/deepmipt/bert.git@feat/multi_gpu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 75,
     "resources": {
      "http://localhost:8080/nbextensions/google.colab/files.js": {
       "data": "Ly8gQ29weXJpZ2h0IDIwMTcgR29vZ2xlIExMQwovLwovLyBMaWNlbnNlZCB1bmRlciB0aGUgQXBhY2hlIExpY2Vuc2UsIFZlcnNpb24gMi4wICh0aGUgIkxpY2Vuc2UiKTsKLy8geW91IG1heSBub3QgdXNlIHRoaXMgZmlsZSBleGNlcHQgaW4gY29tcGxpYW5jZSB3aXRoIHRoZSBMaWNlbnNlLgovLyBZb3UgbWF5IG9idGFpbiBhIGNvcHkgb2YgdGhlIExpY2Vuc2UgYXQKLy8KLy8gICAgICBodHRwOi8vd3d3LmFwYWNoZS5vcmcvbGljZW5zZXMvTElDRU5TRS0yLjAKLy8KLy8gVW5sZXNzIHJlcXVpcmVkIGJ5IGFwcGxpY2FibGUgbGF3IG9yIGFncmVlZCB0byBpbiB3cml0aW5nLCBzb2Z0d2FyZQovLyBkaXN0cmlidXRlZCB1bmRlciB0aGUgTGljZW5zZSBpcyBkaXN0cmlidXRlZCBvbiBhbiAiQVMgSVMiIEJBU0lTLAovLyBXSVRIT1VUIFdBUlJBTlRJRVMgT1IgQ09ORElUSU9OUyBPRiBBTlkgS0lORCwgZWl0aGVyIGV4cHJlc3Mgb3IgaW1wbGllZC4KLy8gU2VlIHRoZSBMaWNlbnNlIGZvciB0aGUgc3BlY2lmaWMgbGFuZ3VhZ2UgZ292ZXJuaW5nIHBlcm1pc3Npb25zIGFuZAovLyBsaW1pdGF0aW9ucyB1bmRlciB0aGUgTGljZW5zZS4KCi8qKgogKiBAZmlsZW92ZXJ2aWV3IEhlbHBlcnMgZm9yIGdvb2dsZS5jb2xhYiBQeXRob24gbW9kdWxlLgogKi8KKGZ1bmN0aW9uKHNjb3BlKSB7CmZ1bmN0aW9uIHNwYW4odGV4dCwgc3R5bGVBdHRyaWJ1dGVzID0ge30pIHsKICBjb25zdCBlbGVtZW50ID0gZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgnc3BhbicpOwogIGVsZW1lbnQudGV4dENvbnRlbnQgPSB0ZXh0OwogIGZvciAoY29uc3Qga2V5IG9mIE9iamVjdC5rZXlzKHN0eWxlQXR0cmlidXRlcykpIHsKICAgIGVsZW1lbnQuc3R5bGVba2V5XSA9IHN0eWxlQXR0cmlidXRlc1trZXldOwogIH0KICByZXR1cm4gZWxlbWVudDsKfQoKLy8gTWF4IG51bWJlciBvZiBieXRlcyB3aGljaCB3aWxsIGJlIHVwbG9hZGVkIGF0IGEgdGltZS4KY29uc3QgTUFYX1BBWUxPQURfU0laRSA9IDEwMCAqIDEwMjQ7Ci8vIE1heCBhbW91bnQgb2YgdGltZSB0byBibG9jayB3YWl0aW5nIGZvciB0aGUgdXNlci4KY29uc3QgRklMRV9DSEFOR0VfVElNRU9VVF9NUyA9IDMwICogMTAwMDsKCmZ1bmN0aW9uIF91cGxvYWRGaWxlcyhpbnB1dElkLCBvdXRwdXRJZCkgewogIGNvbnN0IHN0ZXBzID0gdXBsb2FkRmlsZXNTdGVwKGlucHV0SWQsIG91dHB1dElkKTsKICBjb25zdCBvdXRwdXRFbGVtZW50ID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQob3V0cHV0SWQpOwogIC8vIENhY2hlIHN0ZXBzIG9uIHRoZSBvdXRwdXRFbGVtZW50IHRvIG1ha2UgaXQgYXZhaWxhYmxlIGZvciB0aGUgbmV4dCBjYWxsCiAgLy8gdG8gdXBsb2FkRmlsZXNDb250aW51ZSBmcm9tIFB5dGhvbi4KICBvdXRwdXRFbGVtZW50LnN0ZXBzID0gc3RlcHM7CgogIHJldHVybiBfdXBsb2FkRmlsZXNDb250aW51ZShvdXRwdXRJZCk7Cn0KCi8vIFRoaXMgaXMgcm91Z2hseSBhbiBhc3luYyBnZW5lcmF0b3IgKG5vdCBzdXBwb3J0ZWQgaW4gdGhlIGJyb3dzZXIgeWV0KSwKLy8gd2hlcmUgdGhlcmUgYXJlIG11bHRpcGxlIGFzeW5jaHJvbm91cyBzdGVwcyBhbmQgdGhlIFB5dGhvbiBzaWRlIGlzIGdvaW5nCi8vIHRvIHBvbGwgZm9yIGNvbXBsZXRpb24gb2YgZWFjaCBzdGVwLgovLyBUaGlzIHVzZXMgYSBQcm9taXNlIHRvIGJsb2NrIHRoZSBweXRob24gc2lkZSBvbiBjb21wbGV0aW9uIG9mIGVhY2ggc3RlcCwKLy8gdGhlbiBwYXNzZXMgdGhlIHJlc3VsdCBvZiB0aGUgcHJldmlvdXMgc3RlcCBhcyB0aGUgaW5wdXQgdG8gdGhlIG5leHQgc3RlcC4KZnVuY3Rpb24gX3VwbG9hZEZpbGVzQ29udGludWUob3V0cHV0SWQpIHsKICBjb25zdCBvdXRwdXRFbGVtZW50ID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQob3V0cHV0SWQpOwogIGNvbnN0IHN0ZXBzID0gb3V0cHV0RWxlbWVudC5zdGVwczsKCiAgY29uc3QgbmV4dCA9IHN0ZXBzLm5leHQob3V0cHV0RWxlbWVudC5sYXN0UHJvbWlzZVZhbHVlKTsKICByZXR1cm4gUHJvbWlzZS5yZXNvbHZlKG5leHQudmFsdWUucHJvbWlzZSkudGhlbigodmFsdWUpID0+IHsKICAgIC8vIENhY2hlIHRoZSBsYXN0IHByb21pc2UgdmFsdWUgdG8gbWFrZSBpdCBhdmFpbGFibGUgdG8gdGhlIG5leHQKICAgIC8vIHN0ZXAgb2YgdGhlIGdlbmVyYXRvci4KICAgIG91dHB1dEVsZW1lbnQubGFzdFByb21pc2VWYWx1ZSA9IHZhbHVlOwogICAgcmV0dXJuIG5leHQudmFsdWUucmVzcG9uc2U7CiAgfSk7Cn0KCi8qKgogKiBHZW5lcmF0b3IgZnVuY3Rpb24gd2hpY2ggaXMgY2FsbGVkIGJldHdlZW4gZWFjaCBhc3luYyBzdGVwIG9mIHRoZSB1cGxvYWQKICogcHJvY2Vzcy4KICogQHBhcmFtIHtzdHJpbmd9IGlucHV0SWQgRWxlbWVudCBJRCBvZiB0aGUgaW5wdXQgZmlsZSBwaWNrZXIgZWxlbWVudC4KICogQHBhcmFtIHtzdHJpbmd9IG91dHB1dElkIEVsZW1lbnQgSUQgb2YgdGhlIG91dHB1dCBkaXNwbGF5LgogKiBAcmV0dXJuIHshSXRlcmFibGU8IU9iamVjdD59IEl0ZXJhYmxlIG9mIG5leHQgc3RlcHMuCiAqLwpmdW5jdGlvbiogdXBsb2FkRmlsZXNTdGVwKGlucHV0SWQsIG91dHB1dElkKSB7CiAgY29uc3QgaW5wdXRFbGVtZW50ID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoaW5wdXRJZCk7CiAgaW5wdXRFbGVtZW50LmRpc2FibGVkID0gZmFsc2U7CgogIGNvbnN0IG91dHB1dEVsZW1lbnQgPSBkb2N1bWVudC5nZXRFbGVtZW50QnlJZChvdXRwdXRJZCk7CiAgb3V0cHV0RWxlbWVudC5pbm5lckhUTUwgPSAnJzsKCiAgY29uc3QgcGlja2VkUHJvbWlzZSA9IG5ldyBQcm9taXNlKChyZXNvbHZlKSA9PiB7CiAgICBpbnB1dEVsZW1lbnQuYWRkRXZlbnRMaXN0ZW5lcignY2hhbmdlJywgKGUpID0+IHsKICAgICAgcmVzb2x2ZShlLnRhcmdldC5maWxlcyk7CiAgICB9KTsKICB9KTsKCiAgY29uc3QgY2FuY2VsID0gZG9jdW1lbnQuY3JlYXRlRWxlbWVudCgnYnV0dG9uJyk7CiAgaW5wdXRFbGVtZW50LnBhcmVudEVsZW1lbnQuYXBwZW5kQ2hpbGQoY2FuY2VsKTsKICBjYW5jZWwudGV4dENvbnRlbnQgPSAnQ2FuY2VsIHVwbG9hZCc7CiAgY29uc3QgY2FuY2VsUHJvbWlzZSA9IG5ldyBQcm9taXNlKChyZXNvbHZlKSA9PiB7CiAgICBjYW5jZWwub25jbGljayA9ICgpID0+IHsKICAgICAgcmVzb2x2ZShudWxsKTsKICAgIH07CiAgfSk7CgogIC8vIENhbmNlbCB1cGxvYWQgaWYgdXNlciBoYXNuJ3QgcGlja2VkIGFueXRoaW5nIGluIHRpbWVvdXQuCiAgY29uc3QgdGltZW91dFByb21pc2UgPSBuZXcgUHJvbWlzZSgocmVzb2x2ZSkgPT4gewogICAgc2V0VGltZW91dCgoKSA9PiB7CiAgICAgIHJlc29sdmUobnVsbCk7CiAgICB9LCBGSUxFX0NIQU5HRV9USU1FT1VUX01TKTsKICB9KTsKCiAgLy8gV2FpdCBmb3IgdGhlIHVzZXIgdG8gcGljayB0aGUgZmlsZXMuCiAgY29uc3QgZmlsZXMgPSB5aWVsZCB7CiAgICBwcm9taXNlOiBQcm9taXNlLnJhY2UoW3BpY2tlZFByb21pc2UsIHRpbWVvdXRQcm9taXNlLCBjYW5jZWxQcm9taXNlXSksCiAgICByZXNwb25zZTogewogICAgICBhY3Rpb246ICdzdGFydGluZycsCiAgICB9CiAgfTsKCiAgaWYgKCFmaWxlcykgewogICAgcmV0dXJuIHsKICAgICAgcmVzcG9uc2U6IHsKICAgICAgICBhY3Rpb246ICdjb21wbGV0ZScsCiAgICAgIH0KICAgIH07CiAgfQoKICBjYW5jZWwucmVtb3ZlKCk7CgogIC8vIERpc2FibGUgdGhlIGlucHV0IGVsZW1lbnQgc2luY2UgZnVydGhlciBwaWNrcyBhcmUgbm90IGFsbG93ZWQuCiAgaW5wdXRFbGVtZW50LmRpc2FibGVkID0gdHJ1ZTsKCiAgZm9yIChjb25zdCBmaWxlIG9mIGZpbGVzKSB7CiAgICBjb25zdCBsaSA9IGRvY3VtZW50LmNyZWF0ZUVsZW1lbnQoJ2xpJyk7CiAgICBsaS5hcHBlbmQoc3BhbihmaWxlLm5hbWUsIHtmb250V2VpZ2h0OiAnYm9sZCd9KSk7CiAgICBsaS5hcHBlbmQoc3BhbigKICAgICAgICBgKCR7ZmlsZS50eXBlIHx8ICduL2EnfSkgLSAke2ZpbGUuc2l6ZX0gYnl0ZXMsIGAgKwogICAgICAgIGBsYXN0IG1vZGlmaWVkOiAkewogICAgICAgICAgICBmaWxlLmxhc3RNb2RpZmllZERhdGUgPyBmaWxlLmxhc3RNb2RpZmllZERhdGUudG9Mb2NhbGVEYXRlU3RyaW5nKCkgOgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAnbi9hJ30gLSBgKSk7CiAgICBjb25zdCBwZXJjZW50ID0gc3BhbignMCUgZG9uZScpOwogICAgbGkuYXBwZW5kQ2hpbGQocGVyY2VudCk7CgogICAgb3V0cHV0RWxlbWVudC5hcHBlbmRDaGlsZChsaSk7CgogICAgY29uc3QgZmlsZURhdGFQcm9taXNlID0gbmV3IFByb21pc2UoKHJlc29sdmUpID0+IHsKICAgICAgY29uc3QgcmVhZGVyID0gbmV3IEZpbGVSZWFkZXIoKTsKICAgICAgcmVhZGVyLm9ubG9hZCA9IChlKSA9PiB7CiAgICAgICAgcmVzb2x2ZShlLnRhcmdldC5yZXN1bHQpOwogICAgICB9OwogICAgICByZWFkZXIucmVhZEFzQXJyYXlCdWZmZXIoZmlsZSk7CiAgICB9KTsKICAgIC8vIFdhaXQgZm9yIHRoZSBkYXRhIHRvIGJlIHJlYWR5LgogICAgbGV0IGZpbGVEYXRhID0geWllbGQgewogICAgICBwcm9taXNlOiBmaWxlRGF0YVByb21pc2UsCiAgICAgIHJlc3BvbnNlOiB7CiAgICAgICAgYWN0aW9uOiAnY29udGludWUnLAogICAgICB9CiAgICB9OwoKICAgIC8vIFVzZSBhIGNodW5rZWQgc2VuZGluZyB0byBhdm9pZCBtZXNzYWdlIHNpemUgbGltaXRzLiBTZWUgYi82MjExNTY2MC4KICAgIGxldCBwb3NpdGlvbiA9IDA7CiAgICB3aGlsZSAocG9zaXRpb24gPCBmaWxlRGF0YS5ieXRlTGVuZ3RoKSB7CiAgICAgIGNvbnN0IGxlbmd0aCA9IE1hdGgubWluKGZpbGVEYXRhLmJ5dGVMZW5ndGggLSBwb3NpdGlvbiwgTUFYX1BBWUxPQURfU0laRSk7CiAgICAgIGNvbnN0IGNodW5rID0gbmV3IFVpbnQ4QXJyYXkoZmlsZURhdGEsIHBvc2l0aW9uLCBsZW5ndGgpOwogICAgICBwb3NpdGlvbiArPSBsZW5ndGg7CgogICAgICBjb25zdCBiYXNlNjQgPSBidG9hKFN0cmluZy5mcm9tQ2hhckNvZGUuYXBwbHkobnVsbCwgY2h1bmspKTsKICAgICAgeWllbGQgewogICAgICAgIHJlc3BvbnNlOiB7CiAgICAgICAgICBhY3Rpb246ICdhcHBlbmQnLAogICAgICAgICAgZmlsZTogZmlsZS5uYW1lLAogICAgICAgICAgZGF0YTogYmFzZTY0LAogICAgICAgIH0sCiAgICAgIH07CiAgICAgIHBlcmNlbnQudGV4dENvbnRlbnQgPQogICAgICAgICAgYCR7TWF0aC5yb3VuZCgocG9zaXRpb24gLyBmaWxlRGF0YS5ieXRlTGVuZ3RoKSAqIDEwMCl9JSBkb25lYDsKICAgIH0KICB9CgogIC8vIEFsbCBkb25lLgogIHlpZWxkIHsKICAgIHJlc3BvbnNlOiB7CiAgICAgIGFjdGlvbjogJ2NvbXBsZXRlJywKICAgIH0KICB9Owp9CgpzY29wZS5nb29nbGUgPSBzY29wZS5nb29nbGUgfHwge307CnNjb3BlLmdvb2dsZS5jb2xhYiA9IHNjb3BlLmdvb2dsZS5jb2xhYiB8fCB7fTsKc2NvcGUuZ29vZ2xlLmNvbGFiLl9maWxlcyA9IHsKICBfdXBsb2FkRmlsZXMsCiAgX3VwbG9hZEZpbGVzQ29udGludWUsCn07Cn0pKHNlbGYpOwo=",
       "headers": [
        [
         "content-type",
         "application/javascript"
        ]
       ],
       "ok": true,
       "status": 200,
       "status_text": ""
      }
     }
    },
    "colab_type": "code",
    "id": "gFzJm3CXCy4Y",
    "outputId": "caf6dde8-9a9a-4adc-de83-485b75e2c937"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "     <input type=\"file\" id=\"files-e4e315bb-76eb-416d-ae6e-9b197babde45\" name=\"files[]\" multiple disabled />\n",
       "     <output id=\"result-e4e315bb-76eb-416d-ae6e-9b197babde45\">\n",
       "      Upload widget is only available when the cell has been executed in the\n",
       "      current browser session. Please rerun this cell to enable.\n",
       "      </output>\n",
       "      <script src=\"/nbextensions/google.colab/files.js\"></script> "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Saving valid.txt to valid.txt\n"
     ]
    }
   ],
   "source": [
    "# загрузка файлов для колаба\n",
    "# from google.colab import files\n",
    "# uploaded = files.upload()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Se1Jo3O8Dh7n"
   },
   "outputs": [],
   "source": [
    "# в колабе файлы не перезаписываются, создается новый с (1,2,3..). \n",
    "# Чтобы удалить старый файл перед загрузкой нового используйте rm file\n",
    "# rm .txt удаляет все txt файлы в текущей папке\n",
    "#!rm *.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "02ooybWVJeLz"
   },
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Возьмем тексты с авито по категории приставки. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "_Bf7wBP-Q-v0"
   },
   "outputs": [],
   "source": [
    "data = pd.read_csv('pristavki.csv', header=None, names=['text'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Чтобы научиться выделять названия разметим небольшое количество текстов. Для DeepPavlov разметку нужно поделить на три файла: train.txt, valid.txt, test.txt."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "test.txt  train.txt  valid.txt\r\n"
     ]
    }
   ],
   "source": [
    "!ls *.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Файлы для обучения, валидации и тестирования в формате IOB. B маркирует начало сущности, а I продолжение. O - обозначает несущность. PRS - это сокращение от PRISTAVKA. Можно было и полностью написать. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Продам O\r\n",
      "PS3 B-PRS\r\n",
      ", O\r\n",
      "2 O\r\n",
      "беспроводных O\r\n",
      "джойстика O\r\n",
      "в O\r\n",
      "комплекте O\r\n",
      "\r\n",
      "Продам O\r\n"
     ]
    }
   ],
   "source": [
    "!head train.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Попробуем теперь обучить разметчик сущностей поверх многоязычного берта. Знать, что это такое и как работает не обязательно. Обязательно указать путь к обучающий файлам."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "colab_type": "code",
    "id": "tsvydQwjbHo2",
    "outputId": "e4a23a23-40bd-4e8f-c08b-032c385a413e"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2019-12-19 12:08:47.177 INFO in 'deeppavlov.download'['download'] at line 117: Skipped http://files.deeppavlov.ai/deeppavlov_data/bert/multi_cased_L-12_H-768_A-12.zip download because of matching hashes\n",
      "2019-12-19 12:08:47.185 INFO in 'deeppavlov.core.trainers.fit_trainer'['fit_trainer'] at line 68: NNTrainer got additional init parameters ['pytest_max_batches', 'pytest_batch_size'] that will be ignored:\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/deeppavlov/core/trainers/nn_trainer.py:149: The name tf.summary.FileWriter is deprecated. Please use tf.compat.v1.summary.FileWriter instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
      "[nltk_data]   Unzipping tokenizers/punkt.zip.\n",
      "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
      "[nltk_data]   Unzipping corpora/stopwords.zip.\n",
      "[nltk_data] Downloading package perluniprops to /root/nltk_data...\n",
      "[nltk_data]   Unzipping misc/perluniprops.zip.\n",
      "[nltk_data] Downloading package nonbreaking_prefixes to\n",
      "[nltk_data]     /root/nltk_data...\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/bert_dp/tokenization.py:125: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data]   Unzipping corpora/nonbreaking_prefixes.zip.\n",
      "2019-12-19 12:08:55.232 INFO in 'deeppavlov.core.data.simple_vocab'['simple_vocab'] at line 101: [saving vocabulary to /root/tag.dict]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/deeppavlov/core/models/tf_model.py:37: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/deeppavlov/core/models/tf_model.py:222: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/deeppavlov/core/models/tf_model.py:222: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/deeppavlov/core/models/tf_model.py:193: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/deeppavlov/models/bert/bert_sequence_tagger.py:236: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/deeppavlov/models/bert/bert_sequence_tagger.py:314: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/bert_dp/modeling.py:178: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/bert_dp/modeling.py:418: The name tf.get_variable is deprecated. Please use tf.compat.v1.get_variable instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/bert_dp/modeling.py:499: The name tf.assert_less_equal is deprecated. Please use tf.compat.v1.assert_less_equal instead.\n",
      "\n",
      "WARNING:tensorflow:\n",
      "The TensorFlow contrib module will not be included in TensorFlow 2.0.\n",
      "For more information, please see:\n",
      "  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md\n",
      "  * https://github.com/tensorflow/addons\n",
      "  * https://github.com/tensorflow/io (for I/O related ops)\n",
      "If you depend on functionality not listed there, please file an issue.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/bert_dp/modeling.py:366: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/bert_dp/modeling.py:680: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use keras.layers.Dense instead.\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/layers/core.py:187: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use `layer.__call__` method instead.\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/bert_dp/modeling.py:283: The name tf.erf is deprecated. Please use tf.math.erf instead.\n",
      "\n",
      "WARNING:tensorflow:Variable *= will be deprecated. Use `var.assign(var * other)` if you want assignment to the variable value or `x = x * y` if you want a new python Tensor object.\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/deeppavlov/models/bert/bert_sequence_tagger.py:75: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/contrib/crf/python/ops/crf.py:213: dynamic_rnn (from tensorflow.python.ops.rnn) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use `keras.layers.RNN(cell)`, which is equivalent to this API\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/deeppavlov/core/models/tf_model.py:234: The name tf.train.AdadeltaOptimizer is deprecated. Please use tf.compat.v1.train.AdadeltaOptimizer instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/deeppavlov/core/models/tf_model.py:131: The name tf.get_collection is deprecated. Please use tf.compat.v1.get_collection instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/deeppavlov/core/models/tf_model.py:131: The name tf.GraphKeys is deprecated. Please use tf.compat.v1.GraphKeys instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/deeppavlov/core/models/tf_model.py:94: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/training/moving_averages.py:433: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts.\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/deeppavlov/models/bert/bert_sequence_tagger.py:671: The name tf.assign is deprecated. Please use tf.compat.v1.assign instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/deeppavlov/models/bert/bert_sequence_tagger.py:244: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.\n",
      "\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/deeppavlov/models/bert/bert_sequence_tagger.py:249: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use standard file APIs to check for files with this prefix.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2019-12-19 12:09:22.746 INFO in 'deeppavlov.models.bert.bert_sequence_tagger'['bert_sequence_tagger'] at line 251: [initializing model with Bert from /root/.deeppavlov/downloads/bert_models/multi_cased_L-12_H-768_A-12/bert_model.ckpt]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/deeppavlov/models/bert/bert_sequence_tagger.py:255: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.\n",
      "\n",
      "INFO:tensorflow:Restoring parameters from /root/.deeppavlov/downloads/bert_models/multi_cased_L-12_H-768_A-12/bert_model.ckpt\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2019-12-19 12:09:27.87 DEBUG in 'deeppavlov.metrics.fmeasure'['fmeasure'] at line 394: processed 50 tokens with 5 phrases; found: 16 phrases; correct: 0.\n",
      "\n",
      "precision:  0.00%; recall:  0.00%; FB1:  0.00\n",
      "\n",
      "\tPRS: precision:  0.00%; recall:  0.00%; F1:  0.00 16\n",
      "\n",
      "\n",
      "2019-12-19 12:09:27.88 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 198: Initial best ner_f1 of 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"valid\": {\"eval_examples_count\": 5, \"metrics\": {\"ner_f1\": 0, \"ner_token_f1\": 42.8571}, \"time_spent\": \"0:00:02\", \"epochs_done\": 0, \"batches_seen\": 0, \"train_examples_seen\": 0, \"impatience\": 0, \"patience_limit\": 100}}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2019-12-19 12:10:26.380 DEBUG in 'deeppavlov.metrics.fmeasure'['fmeasure'] at line 394: processed 145 tokens with 19 phrases; found: 17 phrases; correct: 0.\n",
      "\n",
      "precision:  88.24%; recall:  78.95%; FB1:  83.33\n",
      "\n",
      "\tPRS: precision:  88.24%; recall:  78.95%; F1:  83.33 17\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/deeppavlov/core/trainers/nn_trainer.py:249: The name tf.Summary is deprecated. Please use tf.compat.v1.Summary instead.\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2019-12-19 12:10:26.491 DEBUG in 'deeppavlov.metrics.fmeasure'['fmeasure'] at line 394: processed 50 tokens with 5 phrases; found: 4 phrases; correct: 0.\n",
      "\n",
      "precision:  75.00%; recall:  60.00%; FB1:  66.67\n",
      "\n",
      "\tPRS: precision:  75.00%; recall:  60.00%; F1:  66.67 4\n",
      "\n",
      "\n",
      "2019-12-19 12:10:26.493 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 206: Improved best ner_f1 of 66.6667\n",
      "2019-12-19 12:10:26.493 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 208: Saving model\n",
      "2019-12-19 12:10:26.566 INFO in 'deeppavlov.core.models.tf_model'['tf_model'] at line 75: [saving model to /root/model]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"train\": {\"eval_examples_count\": 16, \"metrics\": {\"ner_f1\": 83.3333, \"ner_token_f1\": 92.3077}, \"time_spent\": \"0:01:02\", \"epochs_done\": 9, \"batches_seen\": 20, \"train_examples_seen\": 300, \"head_learning_rate\": 0.009999999776482582, \"bert_learning_rate\": 1.9999999552965164e-05, \"loss\": 8.462892842292785}}\n",
      "{\"valid\": {\"eval_examples_count\": 5, \"metrics\": {\"ner_f1\": 66.6667, \"ner_token_f1\": 84.2105}, \"time_spent\": \"0:01:02\", \"epochs_done\": 9, \"batches_seen\": 20, \"train_examples_seen\": 300, \"impatience\": 0, \"patience_limit\": 100}}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2019-12-19 12:11:18.176 DEBUG in 'deeppavlov.metrics.fmeasure'['fmeasure'] at line 394: processed 164 tokens with 20 phrases; found: 20 phrases; correct: 0.\n",
      "\n",
      "precision:  100.00%; recall:  100.00%; FB1:  100.00\n",
      "\n",
      "\tPRS: precision:  100.00%; recall:  100.00%; F1:  100.00 20\n",
      "\n",
      "\n",
      "2019-12-19 12:11:18.283 DEBUG in 'deeppavlov.metrics.fmeasure'['fmeasure'] at line 394: processed 50 tokens with 5 phrases; found: 4 phrases; correct: 0.\n",
      "\n",
      "precision:  100.00%; recall:  80.00%; FB1:  88.89\n",
      "\n",
      "\tPRS: precision:  100.00%; recall:  80.00%; F1:  88.89 4\n",
      "\n",
      "\n",
      "2019-12-19 12:11:18.284 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 206: Improved best ner_f1 of 88.8889\n",
      "2019-12-19 12:11:18.285 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 208: Saving model\n",
      "2019-12-19 12:11:18.353 INFO in 'deeppavlov.core.models.tf_model'['tf_model'] at line 75: [saving model to /root/model]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"train\": {\"eval_examples_count\": 16, \"metrics\": {\"ner_f1\": 100.0, \"ner_token_f1\": 100.0}, \"time_spent\": \"0:01:53\", \"epochs_done\": 19, \"batches_seen\": 40, \"train_examples_seen\": 600, \"head_learning_rate\": 0.009999999776482582, \"bert_learning_rate\": 1.9999999552965164e-05, \"loss\": 0.5618863470852375}}\n",
      "{\"valid\": {\"eval_examples_count\": 5, \"metrics\": {\"ner_f1\": 88.8889, \"ner_token_f1\": 90.0}, \"time_spent\": \"0:01:54\", \"epochs_done\": 19, \"batches_seen\": 40, \"train_examples_seen\": 600, \"impatience\": 0, \"patience_limit\": 100}}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2019-12-19 12:12:10.312 DEBUG in 'deeppavlov.metrics.fmeasure'['fmeasure'] at line 394: processed 161 tokens with 20 phrases; found: 20 phrases; correct: 0.\n",
      "\n",
      "precision:  100.00%; recall:  100.00%; FB1:  100.00\n",
      "\n",
      "\tPRS: precision:  100.00%; recall:  100.00%; F1:  100.00 20\n",
      "\n",
      "\n",
      "2019-12-19 12:12:10.416 DEBUG in 'deeppavlov.metrics.fmeasure'['fmeasure'] at line 394: processed 50 tokens with 5 phrases; found: 4 phrases; correct: 0.\n",
      "\n",
      "precision:  50.00%; recall:  40.00%; FB1:  44.44\n",
      "\n",
      "\tPRS: precision:  50.00%; recall:  40.00%; F1:  44.44 4\n",
      "\n",
      "\n",
      "2019-12-19 12:12:10.417 INFO in 'deeppavlov.core.trainers.nn_trainer'['nn_trainer'] at line 211: Did not improve on the ner_f1 of 88.8889\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"train\": {\"eval_examples_count\": 16, \"metrics\": {\"ner_f1\": 100.0, \"ner_token_f1\": 100.0}, \"time_spent\": \"0:02:46\", \"epochs_done\": 29, \"batches_seen\": 60, \"train_examples_seen\": 900, \"head_learning_rate\": 0.009999999776482582, \"bert_learning_rate\": 1.9999999552965164e-05, \"loss\": 4.2094814639538525}}\n",
      "{\"valid\": {\"eval_examples_count\": 5, \"metrics\": {\"ner_f1\": 44.4444, \"ner_token_f1\": 77.7778}, \"time_spent\": \"0:02:46\", \"epochs_done\": 29, \"batches_seen\": 60, \"train_examples_seen\": 900, \"impatience\": 1, \"patience_limit\": 100}}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2019-12-19 12:12:12.192 INFO in 'deeppavlov.core.data.simple_vocab'['simple_vocab'] at line 115: [loading vocabulary from /root/tag.dict]\n",
      "2019-12-19 12:12:36.496 INFO in 'deeppavlov.core.models.tf_model'['tf_model'] at line 51: [loading model from /root/model]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from /root/model\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2019-12-19 12:12:40.976 DEBUG in 'deeppavlov.metrics.fmeasure'['fmeasure'] at line 394: processed 50 tokens with 5 phrases; found: 4 phrases; correct: 0.\n",
      "\n",
      "precision:  100.00%; recall:  80.00%; FB1:  88.89\n",
      "\n",
      "\tPRS: precision:  100.00%; recall:  80.00%; F1:  88.89 4\n",
      "\n",
      "\n",
      "2019-12-19 12:12:41.87 DEBUG in 'deeppavlov.metrics.fmeasure'['fmeasure'] at line 394: processed 51 tokens with 5 phrases; found: 5 phrases; correct: 0.\n",
      "\n",
      "precision:  80.00%; recall:  80.00%; FB1:  80.00\n",
      "\n",
      "\tPRS: precision:  80.00%; recall:  80.00%; F1:  80.00 5\n",
      "\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"valid\": {\"eval_examples_count\": 5, \"metrics\": {\"ner_f1\": 88.8889, \"ner_token_f1\": 90.0}, \"time_spent\": \"0:00:02\"}}\n",
      "{\"test\": {\"eval_examples_count\": 5, \"metrics\": {\"ner_f1\": 80.0, \"ner_token_f1\": 93.3333}, \"time_spent\": \"0:00:01\"}}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2019-12-19 12:12:41.840 INFO in 'deeppavlov.core.data.simple_vocab'['simple_vocab'] at line 115: [loading vocabulary from /root/tag.dict]\n",
      "2019-12-19 12:13:08.243 INFO in 'deeppavlov.core.models.tf_model'['tf_model'] at line 51: [loading model from /root/model]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from /root/model\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "from deeppavlov import configs, build_model, train_model\n",
    "\n",
    "with configs.ner.ner_ontonotes_bert_mult.open(encoding='utf8') as f:\n",
    "    ner_config = json.load(f)\n",
    "\n",
    "ner_config['dataset_reader']['data_path'] = './'  # directory with train.txt, valid.txt and test.txt files\n",
    "ner_config['metadata']['variables']['NER_PATH'] = './'\n",
    "ner_config['metadata']['download'] = [ner_config['metadata']['download'][-1]]  # do not download the pretrained ontonotes model\n",
    "\n",
    "ner_model = train_model(ner_config, download=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Обучается оно около 10 минут. Добавление примеров в обучение увеличит время обучения. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Посмотрим как это все размечается."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[['Playstation', '4'], ['Xbox', '360', 'продам'], ['Продам', 'ПС', '3']],\n",
       " [['B-PRS', 'I-PRS'], ['B-PRS', 'I-PRS', 'O'], ['O', 'B-PRS', 'I-PRS']]]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ner_model(['Playstation 4', 'Xbox 360 продам', 'Продам ПС 3'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "VKEWNLwhiK-H"
   },
   "outputs": [],
   "source": [
    "marked = []\n",
    "\n",
    "for text in data.text.values[:1000]:\n",
    "    # BERT имеет лимит на длину текста в 512 слов, возьмем даже еще меньше\n",
    "    if len(text.split()) > 100:\n",
    "        continue\n",
    "    pred = ner_model([text])\n",
    "    sent, tags = pred[0][0], pred[1][0]\n",
    "    \n",
    "    # достанем только тексты с сущностями\n",
    "    if len(set(tags[0])) > 1:\n",
    "        marked.append(list(zip(sent,tags)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "colab_type": "code",
    "id": "nbPmC0v1keh1",
    "outputId": "d6534ea0-136a-4cab-e09c-c5d1186627ab"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[('Xbox', 'B-PRS'),\n",
       "  ('в', 'O'),\n",
       "  ('отличном', 'O'),\n",
       "  ('состоянии', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('2', 'O'),\n",
       "  ('диска', 'O'),\n",
       "  ('с', 'O'),\n",
       "  ('играми', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Поддержка', 'O'),\n",
       "  ('носителей', 'O'),\n",
       "  (':', 'O'),\n",
       "  ('CD', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('DVD', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('флэшка', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Wi', 'B-PRS'),\n",
       "  ('-', 'I-PRS'),\n",
       "  ('Fi', 'I-PRS'),\n",
       "  ('(', 'O'),\n",
       "  ('802', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('11', 'O'),\n",
       "  (')', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Bluetooth', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Цвет', 'O'),\n",
       "  (':', 'O'),\n",
       "  ('черный', 'O')],\n",
       " [('Sony', 'B-PRS'),\n",
       "  ('PS3', 'B-PRS'),\n",
       "  ('в', 'O'),\n",
       "  ('отличном', 'O'),\n",
       "  ('состоянии', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('продаю', 'O'),\n",
       "  ('за', 'O'),\n",
       "  ('ненадобностью', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('память', 'O'),\n",
       "  ('12gb', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('встроенные', 'O'),\n",
       "  ('+', 'O'),\n",
       "  ('жесткий', 'O'),\n",
       "  ('диск', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('160gb', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('комплект', 'O'),\n",
       "  ('входит', 'O'),\n",
       "  ('два', 'O'),\n",
       "  ('джойстика', 'O'),\n",
       "  ('(', 'O'),\n",
       "  ('состояние', 'O'),\n",
       "  ('новых', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('залипают', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('тупят', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('играли', 'O'),\n",
       "  ('мало', 'O'),\n",
       "  (')', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('игра', 'O'),\n",
       "  ('FIFA', 'B-PRS'),\n",
       "  ('2014', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('кабель', 'O'),\n",
       "  ('hdmi', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('коробка', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Sega', 'B-PRS'),\n",
       "  ('игровая', 'O'),\n",
       "  ('приставка', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('можно', 'O'),\n",
       "  ('micro', 'B-PRS'),\n",
       "  ('SD', 'I-PRS'),\n",
       "  ('поставить', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('наличии', 'O'),\n",
       "  ('зарядка', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('приставка', 'O')],\n",
       " [('Battlefield', 'B-PRS'),\n",
       "  ('4', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('Granturismo', 'B-PRS'),\n",
       "  ('3', 'I-PRS'),\n",
       "  ('и', 'O'),\n",
       "  ('Звезды', 'O'),\n",
       "  ('Playstation', 'B-PRS'),\n",
       "  ('Битва', 'O'),\n",
       "  ('сильнейших', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('За', 'O'),\n",
       "  ('все', 'O'),\n",
       "  ('1500', 'O'),\n",
       "  ('руб', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Sony', 'B-PRS'),\n",
       "  ('Playstation', 'I-PRS'),\n",
       "  ('3', 'I-PRS'),\n",
       "  ('Slim', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('лицензионная', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('никогда', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('прошивалась', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Все', 'O'),\n",
       "  ('провода', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('комплекте', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('hdmi', 'O'),\n",
       "  ('кабель', 'O'),\n",
       "  ('для', 'O'),\n",
       "  ('большого', 'O'),\n",
       "  ('ТВ', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('тормозит', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('перегревается', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Купили', 'O'),\n",
       "  ('Пс4', 'B-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('эту', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('играем', 'O'),\n",
       "  ('почти', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Xbox', 'B-PRS'),\n",
       "  ('360', 'I-PRS'),\n",
       "  ('прошитый', 'O'),\n",
       "  ('LT', 'B-PRS'),\n",
       "  ('2', 'I-PRS'),\n",
       "  ('.', 'O'),\n",
       "  ('0', 'O'),\n",
       "  ('+', 'O'),\n",
       "  ('игр', 'O'),\n",
       "  ('к', 'O'),\n",
       "  ('нему', 'O'),\n",
       "  ('больше', 'O'),\n",
       "  ('50', 'O'),\n",
       "  ('шт', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Kaspersky', 'B-PRS'),\n",
       "  ('Anti', 'I-PRS'),\n",
       "  ('-', 'I-PRS'),\n",
       "  ('Virus', 'I-PRS'),\n",
       "  ('–', 'O'),\n",
       "  ('это', 'O'),\n",
       "  ('решение', 'O'),\n",
       "  ('для', 'O'),\n",
       "  ('базовой', 'O'),\n",
       "  ('защиты', 'O'),\n",
       "  ('компьютера', 'O'),\n",
       "  ('от', 'O'),\n",
       "  ('основных', 'O'),\n",
       "  ('видов', 'O'),\n",
       "  ('интернет', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('угроз', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('снижающее', 'O'),\n",
       "  ('его', 'O'),\n",
       "  ('производительность', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('Надежная', 'O'),\n",
       "  ('защита', 'O'),\n",
       "  ('от', 'O'),\n",
       "  ('вредоносных', 'O'),\n",
       "  ('программ', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('Высокая', 'O'),\n",
       "  ('скорость', 'O'),\n",
       "  ('работы', 'O'),\n",
       "  ('без', 'O'),\n",
       "  ('снижения', 'O'),\n",
       "  ('производительности', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('Удобное', 'O'),\n",
       "  ('управление', 'O'),\n",
       "  ('защитой', 'O'),\n",
       "  ('из', 'O'),\n",
       "  ('любой', 'O'),\n",
       "  ('точки', 'O')],\n",
       " [('Wi', 'B-PRS'),\n",
       "  ('-', 'I-PRS'),\n",
       "  ('Fi', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('2', 'O'),\n",
       "  ('камеры', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('Задняя', 'O'),\n",
       "  ('сенсорная', 'O'),\n",
       "  ('панель', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('Передний', 'O'),\n",
       "  ('сенсорный', 'O'),\n",
       "  ('экран', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('Зарядное', 'O'),\n",
       "  ('устройство', 'O'),\n",
       "  ('от', 'O'),\n",
       "  ('сети', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('USB', 'B-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('ПО', 'O'),\n",
       "  ('3', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('61', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Ps', 'B-PRS'),\n",
       "  ('move', 'O'),\n",
       "  ('с', 'O'),\n",
       "  ('камерой', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Продаю', 'O'),\n",
       "  ('из', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('за', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('надобности', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Работает', 'O'),\n",
       "  ('отлично', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Торга', 'O'),\n",
       "  ('нет', 'O')],\n",
       " [('Xbox', 'B-PRS'),\n",
       "  ('360', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('250', 'O'),\n",
       "  ('Гб', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('freeboot', 'O'),\n",
       "  ('(', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('надо', 'O'),\n",
       "  ('покупать', 'O'),\n",
       "  ('игры', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('скачал', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('установил', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('играй', 'O'),\n",
       "  (')', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('беспроводной', 'O'),\n",
       "  ('джойстик', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('сенсорная', 'O'),\n",
       "  ('панель', 'O')],\n",
       " [('Fifa', 'B-PRS'),\n",
       "  ('400', 'I-PRS'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Destiny', 'B-PRS'),\n",
       "  ('600', 'I-PRS')],\n",
       " [('Microsoft', 'B-PRS'),\n",
       "  ('Xbox', 'B-PRS'),\n",
       "  ('360', 'I-PRS'),\n",
       "  ('250', 'O'),\n",
       "  ('ГБ', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Игровая', 'O'),\n",
       "  ('приставка', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('поддержка', 'O'),\n",
       "  ('HD', 'B-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('разрешения', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('воспроизведение', 'O'),\n",
       "  ('видео', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Wi', 'B-PRS'),\n",
       "  ('-', 'I-PRS'),\n",
       "  ('Fi', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('LAN', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('комплекте', 'O'),\n",
       "  (':', 'O'),\n",
       "  ('сетевой', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('HDMI', 'B-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('джойстик', 'O'),\n",
       "  ('(', 'O'),\n",
       "  ('есть', 'O'),\n",
       "  ('повреждения', 'O'),\n",
       "  (')', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Цена', 'O'),\n",
       "  (':', 'O'),\n",
       "  ('6', 'O'),\n",
       "  ('500', 'O'),\n",
       "  ('руб', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('ул', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Проспект', 'O'),\n",
       "  ('Победы', 'O'),\n",
       "  ('182', 'O')],\n",
       " [('watch', 'B-PRS'),\n",
       "  ('dogs', 'I-PRS'),\n",
       "  ('2', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('2400', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('far', 'O'),\n",
       "  ('cry', 'O'),\n",
       "  ('primal', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('1000', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('при', 'O'),\n",
       "  ('покупке', 'O'),\n",
       "  ('двух', 'O'),\n",
       "  ('дисков', 'O'),\n",
       "  ('скидка', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('звоните', 'O')],\n",
       " [('Gran', 'B-PRS'),\n",
       "  ('Turismo', 'I-PRS'),\n",
       "  ('6', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('500', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Fight', 'B-PRS'),\n",
       "  ('Night', 'I-PRS'),\n",
       "  ('champion', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('500', 'O')],\n",
       " [('Ps3', 'B-PRS'),\n",
       "  ('в', 'O'),\n",
       "  ('хорошем', 'O'),\n",
       "  ('состоянии', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('2', 'O'),\n",
       "  ('джойстика', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('5', 'O'),\n",
       "  ('игр', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('диске', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('20', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('приставке', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Биспи', 'B-PRS'),\n",
       "  ('без', 'O'),\n",
       "  ('игр', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('без', 'O'),\n",
       "  ('флешки', 'O')],\n",
       " [('PS', 'B-PRS'),\n",
       "  ('4', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('500', 'O'),\n",
       "  ('гб', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('состояние', 'O'),\n",
       "  ('новой', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('комплекте', 'O'),\n",
       "  ('есть', 'O'),\n",
       "  ('игры', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('2', 'O'),\n",
       "  ('джостика', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Могу', 'O'),\n",
       "  ('продать', 'O'),\n",
       "  ('аккаунт', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('за', 'O'),\n",
       "  ('дорого', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('На', 'O'),\n",
       "  ('аккаунте', 'O'),\n",
       "  ('до', 'O'),\n",
       "  ('конца', 'O'),\n",
       "  ('года', 'O'),\n",
       "  ('ps', 'B-PRS'),\n",
       "  ('+', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('много', 'O'),\n",
       "  ('игр', 'O')],\n",
       " [('Sony', 'B-PRS'),\n",
       "  ('Playstaition', 'I-PRS'),\n",
       "  ('3', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('обем', 'O'),\n",
       "  ('памяти', 'O'),\n",
       "  (':', 'O'),\n",
       "  ('250гб', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('В', 'O'),\n",
       "  ('отличном', 'O'),\n",
       "  ('состоянии', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('провода', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('геймпад', 'O'),\n",
       "  ('2шт', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Не', 'O'),\n",
       "  ('прошивалась', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('имеет', 'O'),\n",
       "  ('доступ', 'O'),\n",
       "  ('к', 'O'),\n",
       "  ('интернету', 'O'),\n",
       "  ('!', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Лучшее', 'O'),\n",
       "  ('предложение', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('соотношении', 'O'),\n",
       "  ('цена', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('качество', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Игры', 'O'),\n",
       "  (':', 'O'),\n",
       "  ('Mortal', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('Dirt', 'B-PRS'),\n",
       "  ('3', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('FIFA', 'B-PRS'),\n",
       "  ('13', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('FIFA', 'B-PRS'),\n",
       "  ('16', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('Fifa', 'O'),\n",
       "  ('STREET', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('TENNIS', 'B-PRS'),\n",
       "  ('2', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('GTA', 'O'),\n",
       "  ('5', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('UFS', 'B-PRS'),\n",
       "  ('3', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('Gold', 'O'),\n",
       "  ('of', 'O'),\n",
       "  ('Wars', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('Grand', 'B-PRS'),\n",
       "  ('Turismo', 'I-PRS'),\n",
       "  ('6', 'I-PRS'),\n",
       "  ('.', 'O')],\n",
       " [('Анчартед', 'B-PRS'),\n",
       "  ('4', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('2000', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('анчартид', 'O'),\n",
       "  ('коллекция', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('1500', 'O')],\n",
       " [('NHL', 'B-PRS'),\n",
       "  ('15', 'I-PRS'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('FIFA', 'B-PRS'),\n",
       "  ('15', 'I-PRS')],\n",
       " [('Android', 'B-PRS'),\n",
       "  ('Совместимость', 'O'),\n",
       "  (':', 'O'),\n",
       "  ('\\xa0', 'O'),\n",
       "  ('поддержка', 'O'),\n",
       "  ('Android', 'B-PRS'),\n",
       "  ('3', 'I-PRS'),\n",
       "  ('.', 'O'),\n",
       "  ('2', 'O'),\n",
       "  ('или', 'O'),\n",
       "  ('выше', 'O'),\n",
       "  ('\\xa0', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Bluetooth', 'B-PRS'),\n",
       "  ('3', 'I-PRS'),\n",
       "  ('.', 'I-PRS'),\n",
       "  ('0', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('беспроводная', 'O'),\n",
       "  ('передача', 'O'),\n",
       "  ('до', 'O'),\n",
       "  ('10', 'O'),\n",
       "  ('м', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('\\xa0', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Удобные', 'O'),\n",
       "  ('клавиши', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('двойной', 'O'),\n",
       "  ('аналоговый', 'O'),\n",
       "  ('джойстик', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('точный', 'O'),\n",
       "  ('D', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('Pad', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('14', 'O'),\n",
       "  ('кнопок', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('\\xa0', 'O'),\n",
       "  ('Cпящий', 'O'),\n",
       "  ('режим', 'O'),\n",
       "  ('(', 'O'),\n",
       "  ('энергосбережения', 'O'),\n",
       "  (')', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Поддерживает', 'O'),\n",
       "  ('игры', 'O'),\n",
       "  ('\\xa0', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('только', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('платформе', 'O'),\n",
       "  ('\\xa0', 'O'),\n",
       "  ('Android', 'B-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('но', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('классические', 'O'),\n",
       "  ('аркадные', 'O'),\n",
       "  ('игры', 'O'),\n",
       "  ('эмулятор', 'O'),\n",
       "  ('PSP', 'B-PRS'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Встроенный', 'O'),\n",
       "  ('аккумулятор', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Размер', 'O'),\n",
       "  (':', 'O'),\n",
       "  ('\\xa0', 'O'),\n",
       "  ('14', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('50x9', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('60x2', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('50', 'O'),\n",
       "  ('cm', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('г', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Лениногорск', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('ул', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Ленинградская', 'O'),\n",
       "  ('25', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('стр', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('2', 'O')],\n",
       " [('NBA', 'B-PRS'),\n",
       "  ('2K15', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('800', 'O'),\n",
       "  ('р', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('(', 'O'),\n",
       "  ('состояние', 'O'),\n",
       "  ('диска', 'O'),\n",
       "  ('отличное', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('коробки', 'O'),\n",
       "  ('удов', 'O'),\n",
       "  ('.', 'O'),\n",
       "  (')', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Uncharted', 'O'),\n",
       "  ('4', 'O'),\n",
       "  ('A', 'O'),\n",
       "  (\"Thief's\", 'O'),\n",
       "  ('End', 'O'),\n",
       "  ('(', 'O'),\n",
       "  ('PS4', 'B-PRS'),\n",
       "  (')', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('English', 'O'),\n",
       "  ('version', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('1500', 'O'),\n",
       "  ('р', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('(', 'O'),\n",
       "  ('обратите', 'O'),\n",
       "  ('внимание', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('игре', 'O'),\n",
       "  ('нет', 'O'),\n",
       "  ('русского', 'O'),\n",
       "  ('языка', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('состояние', 'O'),\n",
       "  ('идеальное', 'O'),\n",
       "  (')', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Продажа', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('возможен', 'O'),\n",
       "  ('обмен', 'O'),\n",
       "  ('(', 'O'),\n",
       "  ('предложения', 'O'),\n",
       "  ('пишите', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('личку', 'O'),\n",
       "  (')', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('King', 'B-PRS'),\n",
       "  ('Kong', 'I-PRS'),\n",
       "  ('лицензионный', 'O'),\n",
       "  ('диск', 'O'),\n",
       "  ('x', 'B-PRS'),\n",
       "  ('-', 'I-PRS'),\n",
       "  ('box', 'I-PRS'),\n",
       "  ('360', 'I-PRS')],\n",
       " [('xbox', 'B-PRS'), ('one', 'I-PRS'), ('игры', 'O'), ('обмен', 'O')],\n",
       " [('Toefl', 'B-PRS'),\n",
       "  ('Test', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('Рекордный', 'O'),\n",
       "  ('Результат', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('50', 'O'),\n",
       "  ('рублей', 'O'),\n",
       "  (';', 'O'),\n",
       "  ('English', 'B-PRS'),\n",
       "  ('Platinum', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('50', 'O'),\n",
       "  ('рублей', 'O'),\n",
       "  (';', 'O'),\n",
       "  ('Англо', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('Русская', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('Русско', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('Английская', 'O'),\n",
       "  ('программа', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('тренажер', 'O'),\n",
       "  ('для', 'O'),\n",
       "  ('пополнения', 'O'),\n",
       "  ('лексического', 'O'),\n",
       "  ('запаса', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('50', 'O'),\n",
       "  ('рублей', 'O'),\n",
       "  (';', 'O'),\n",
       "  ('Учим', 'O'),\n",
       "  ('язык', 'O'),\n",
       "  ('English', 'B-PRS'),\n",
       "  ('-', 'I-PRS'),\n",
       "  ('4', 'I-PRS'),\n",
       "  ('CD', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('150', 'O'),\n",
       "  ('рублей', 'O'),\n",
       "  (';', 'O'),\n",
       "  ('English', 'O'),\n",
       "  ('(', 'O'),\n",
       "  ('Компьютерный', 'O'),\n",
       "  ('курс', 'O'),\n",
       "  ('+', 'O'),\n",
       "  ('аудиоразговорники', 'O'),\n",
       "  ('Doki', 'O'),\n",
       "  (')', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('уникальные', 'O'),\n",
       "  ('интерактивные', 'O'),\n",
       "  ('мультфильмы', 'O'),\n",
       "  ('для', 'O'),\n",
       "  ('изучения', 'O'),\n",
       "  ('английского', 'O'),\n",
       "  ('языка', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('2', 'O'),\n",
       "  ('диска', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('100', 'O'),\n",
       "  ('рублей', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('За', 'O'),\n",
       "  ('все', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('9', 'O'),\n",
       "  ('дисков', 'O'),\n",
       "  ('(', 'O'),\n",
       "  ('вместе', 'O'),\n",
       "  (')', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('250', 'O'),\n",
       "  ('рублей', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Сони', 'B-PRS'),\n",
       "  ('плейстешен', 'I-PRS'),\n",
       "  ('2', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('отличном', 'O'),\n",
       "  ('состоянии', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Doom', 'B-PRS'),\n",
       "  ('1500', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Call', 'B-PRS'),\n",
       "  ('of', 'I-PRS'),\n",
       "  ('Duty', 'I-PRS'),\n",
       "  ('1500', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Assassins', 'O'),\n",
       "  ('creed', 'O'),\n",
       "  ('черный', 'O'),\n",
       "  ('флаг', 'O'),\n",
       "  ('700', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('God', 'B-PRS'),\n",
       "  ('of', 'I-PRS'),\n",
       "  ('War', 'I-PRS'),\n",
       "  ('700', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Killzone', 'O'),\n",
       "  ('700', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('The', 'O'),\n",
       "  ('Crew', 'O'),\n",
       "  ('600', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Garden', 'B-PRS'),\n",
       "  ('Warfare', 'I-PRS'),\n",
       "  ('400', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Обмен', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('интересует', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Far', 'B-PRS'),\n",
       "  ('Cry', 'I-PRS'),\n",
       "  ('4', 'I-PRS'),\n",
       "  ('.', 'O'),\n",
       "  ('Специальное', 'O'),\n",
       "  ('издание', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Диск', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('отличном', 'O'),\n",
       "  ('состоянии', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Любые', 'O'),\n",
       "  ('проверки', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Могу', 'O'),\n",
       "  ('рассмотреть', 'O'),\n",
       "  ('обмен', 'O')],\n",
       " [('Батла', 'B-PRS'),\n",
       "  ('1000р', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('остальные', 'O'),\n",
       "  ('500р', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Watch', 'B-PRS'),\n",
       "  ('dogs', 'I-PRS'),\n",
       "  ('2', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('2700', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Far', 'O'),\n",
       "  ('cry', 'B-PRS'),\n",
       "  ('4', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('1400', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Homefront', 'O'),\n",
       "  ('the', 'O'),\n",
       "  ('revolution', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('2300', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Destiny', 'B-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('500', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Good', 'O'),\n",
       "  ('of', 'O'),\n",
       "  ('war', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('1500', 'O')],\n",
       " [('Fifa', 'B-PRS'),\n",
       "  ('08', 'I-PRS'),\n",
       "  ('лицензионный', 'O'),\n",
       "  ('диск', 'O'),\n",
       "  ('x', 'B-PRS'),\n",
       "  ('-', 'I-PRS'),\n",
       "  ('box', 'I-PRS'),\n",
       "  ('360', 'I-PRS')],\n",
       " [('Nyko', 'B-PRS'),\n",
       "  ('Zoom', 'I-PRS'),\n",
       "  ('уменьшает', 'O'),\n",
       "  ('пространство', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('необходимое', 'O'),\n",
       "  ('для', 'O'),\n",
       "  ('ваших', 'O'),\n",
       "  ('любимых', 'O'),\n",
       "  ('игр', 'O'),\n",
       "  ('снижает', 'O'),\n",
       "  ('диапазон', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('необходимый', 'O'),\n",
       "  ('для', 'O'),\n",
       "  ('игры', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('Kinect', 'B-PRS'),\n",
       "  ('на', 'O'),\n",
       "  ('40', 'O'),\n",
       "  ('%', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('что', 'O'),\n",
       "  ('позволяет', 'O'),\n",
       "  ('играть', 'O'),\n",
       "  ('ближе', 'O'),\n",
       "  ('к', 'O'),\n",
       "  ('телевизору', 'B-PRS'),\n",
       "  ('.', 'O'),\n",
       "  ('также', 'O'),\n",
       "  ('позволяет', 'O'),\n",
       "  ('двум', 'O'),\n",
       "  ('людям', 'O'),\n",
       "  ('играть', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('области', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('которая', 'O'),\n",
       "  ('раньше', 'O'),\n",
       "  ('подходила', 'O'),\n",
       "  ('только', 'O'),\n",
       "  ('для', 'O'),\n",
       "  ('одного', 'O'),\n",
       "  ('человека', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('что', 'O'),\n",
       "  ('делает', 'O'),\n",
       "  ('Nyko', 'B-PRS'),\n",
       "  ('просто', 'O'),\n",
       "  ('идеальным', 'O'),\n",
       "  ('дополнением', 'O'),\n",
       "  ('к', 'O'),\n",
       "  ('Kinect', 'B-PRS'),\n",
       "  ('.', 'O')],\n",
       " [('PS', 'B-PRS'),\n",
       "  ('Vita', 'I-PRS'),\n",
       "  ('в', 'O'),\n",
       "  ('отличном', 'O'),\n",
       "  ('состоянии', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('В', 'O'),\n",
       "  ('комплект', 'O'),\n",
       "  ('входит', 'O'),\n",
       "  ('карта', 'O'),\n",
       "  ('памяти', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('8gb', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('мягкий', 'O'),\n",
       "  ('чехол', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('противоударный', 'O'),\n",
       "  ('корпус', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('зарядное', 'O'),\n",
       "  ('устройство', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('три', 'O'),\n",
       "  ('картриджа', 'O'),\n",
       "  (':', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Borderland', 'B-PRS'),\n",
       "  ('2', 'I-PRS'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Ratchet', 'O'),\n",
       "  ('&', 'O'),\n",
       "  ('Clank', 'B-PRS'),\n",
       "  ('Trilogy', 'I-PRS'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Call', 'B-PRS'),\n",
       "  ('of', 'I-PRS'),\n",
       "  ('Duty', 'I-PRS'),\n",
       "  (':', 'O'),\n",
       "  ('Black', 'B-PRS'),\n",
       "  ('Ops', 'I-PRS'),\n",
       "  ('Declassified', 'O')],\n",
       " [('Sony', 'B-PRS'), ('PSP', 'I-PRS'), ('3008', 'I-PRS')],\n",
       " [('Dualshock', 'B-PRS'),\n",
       "  ('3', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('Для', 'O'),\n",
       "  ('ps3', 'B-PRS')],\n",
       " [('PlayStation', 'B-PRS'),\n",
       "  ('3CECH', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('3008B', 'O'),\n",
       "  ('320Gb', 'O'),\n",
       "  ('PS', 'B-PRS'),\n",
       "  ('Eye', 'I-PRS'),\n",
       "  ('+', 'O'),\n",
       "  ('PS', 'B-PRS'),\n",
       "  ('Move', 'I-PRS')],\n",
       " [('Dark', 'B-PRS'),\n",
       "  ('Soul', 'I-PRS'),\n",
       "  ('II', 'I-PRS'),\n",
       "  ('коллекционное', 'O'),\n",
       "  ('издание', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('1750', 'O'),\n",
       "  ('р', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Ледниковый', 'O'),\n",
       "  ('период', 'O'),\n",
       "  ('3', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('1000', 'O'),\n",
       "  ('р', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Мадагаскар', 'O'),\n",
       "  ('3', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('1000', 'O'),\n",
       "  ('р', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Праздник', 'O'),\n",
       "  ('спорта', 'O'),\n",
       "  ('2', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('600', 'O'),\n",
       "  ('р', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Dualshock', 'B-PRS'),\n",
       "  ('3', 'I-PRS'),\n",
       "  ('sixaxis', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('800', 'O'),\n",
       "  ('р', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Кружка', 'O'),\n",
       "  ('геймера', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('350', 'O')],\n",
       " [('Ben10', 'B-PRS'),\n",
       "  ('(', 'O'),\n",
       "  ('Ben', 'B-PRS'),\n",
       "  ('10', 'I-PRS'),\n",
       "  (')', 'O'),\n",
       "  ('Omniverse', 'B-PRS'),\n",
       "  ('2', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('лицензионный', 'O'),\n",
       "  ('диск', 'O'),\n",
       "  ('для', 'O'),\n",
       "  ('Playstation', 'B-PRS'),\n",
       "  ('3', 'I-PRS'),\n",
       "  ('(', 'O'),\n",
       "  ('PS3', 'B-PRS'),\n",
       "  (')', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('идеальное', 'O'),\n",
       "  ('состояние', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('английском', 'O'),\n",
       "  ('языке', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('для', 'O'),\n",
       "  ('1', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('2', 'O'),\n",
       "  ('игроков', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Стоимость', 'O'),\n",
       "  ('обмена', 'O'),\n",
       "  ('зависит', 'O'),\n",
       "  ('от', 'O'),\n",
       "  ('номинала', 'O'),\n",
       "  ('предлагаемой', 'O'),\n",
       "  ('игры', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Sega', 'B-PRS'),\n",
       "  ('mega', 'I-PRS'),\n",
       "  ('drive', 'I-PRS'),\n",
       "  ('2', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('сега', 'O'),\n",
       "  ('мега', 'O'),\n",
       "  ('драйв', 'O'),\n",
       "  ('2', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Два', 'O'),\n",
       "  ('джойстика', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('имеются', 'O'),\n",
       "  ('немного', 'O'),\n",
       "  ('дисков', 'O'),\n",
       "  ('игровых', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Полностью', 'O'),\n",
       "  ('рабочая', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Ребёнок', 'O'),\n",
       "  ('вырос', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('пс3', 'B-PRS'),\n",
       "  ('с', 'O'),\n",
       "  ('4', 'O'),\n",
       "  ('джестиками', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('играми', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('ней', 'O'),\n",
       "  ('уже', 'O'),\n",
       "  ('стоит', 'O'),\n",
       "  ('гта5', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('много', 'O'),\n",
       "  ('демо', 'O'),\n",
       "  ('версий', 'O'),\n",
       "  ('+', 'O'),\n",
       "  ('диски', 'O'),\n",
       "  ('с', 'O'),\n",
       "  ('играми', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('X', 'B-PRS'),\n",
       "  ('-', 'I-PRS'),\n",
       "  ('box', 'I-PRS'),\n",
       "  ('360', 'I-PRS'),\n",
       "  ('Slim', 'I-PRS'),\n",
       "  ('250', 'O'),\n",
       "  ('Gb', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('матовый', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Не', 'O'),\n",
       "  ('прошитый', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('состояние', 'O'),\n",
       "  ('4', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('5', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('В', 'O'),\n",
       "  ('комплекте', 'O'),\n",
       "  ('1', 'O'),\n",
       "  ('контроллер', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('3', 'O'),\n",
       "  ('игры', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('дисках', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Так', 'O'),\n",
       "  ('же', 'O'),\n",
       "  ('есть', 'O'),\n",
       "  ('ассортимент', 'O'),\n",
       "  ('игр', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('лицензии', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('300', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('400', 'O'),\n",
       "  ('рублей', 'O'),\n",
       "  ('за', 'O'),\n",
       "  ('штуку', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('подробности', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('личку', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Mafia', 'B-PRS'),\n",
       "  ('3', 'I-PRS'),\n",
       "  ('PS', 'B-PRS'),\n",
       "  ('4', 'I-PRS'),\n",
       "  ('лицензия', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('диск', 'O'),\n",
       "  ('из', 'O'),\n",
       "  ('бруталити', 'O'),\n",
       "  ('4', 'O'),\n",
       "  ('категория', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('брал', 'O'),\n",
       "  ('за', 'O'),\n",
       "  ('3000', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('обмен', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('интересен', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Gta', 'B-PRS'),\n",
       "  ('5', 'I-PRS'),\n",
       "  ('лицензия', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('подарок', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Watch', 'B-PRS'),\n",
       "  ('dogs', 'I-PRS'),\n",
       "  ('2', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('2000', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Watch', 'B-PRS'),\n",
       "  ('Dogs', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('1000', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Средиземье', 'O'),\n",
       "  (':', 'O'),\n",
       "  ('Тени', 'B-PRS'),\n",
       "  ('Мордора', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('1200', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Battlefield', 'B-PRS'),\n",
       "  ('4', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('900', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Destiny', 'B-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('800', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('NFS', 'B-PRS'),\n",
       "  ('Rivals', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('700', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Рассмотрю', 'O'),\n",
       "  ('любые', 'O'),\n",
       "  ('варианты', 'O'),\n",
       "  ('обмена', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('NHL', 'B-PRS'),\n",
       "  ('15', 'I-PRS'),\n",
       "  ('в', 'O'),\n",
       "  ('отличном', 'O'),\n",
       "  ('состоянии', 'O')],\n",
       " [('The', 'B-PRS'),\n",
       "  ('Ship', 'I-PRS'),\n",
       "  ('Остаться', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('живых', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('50', 'O'),\n",
       "  ('р', 'O'),\n",
       "  (';', 'O'),\n",
       "  ('Hospital', 'O'),\n",
       "  ('Tycoon', 'B-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('50р', 'O'),\n",
       "  (';', 'O'),\n",
       "  ('Diablo', 'B-PRS'),\n",
       "  ('Faster', 'I-PRS'),\n",
       "  ('Sun', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('50р', 'O'),\n",
       "  (';', 'O'),\n",
       "  ('Age', 'O'),\n",
       "  ('Empiers', 'O'),\n",
       "  ('СУПЕРКОЛЛЕКЦИЯ', 'O'),\n",
       "  ('(', 'O'),\n",
       "  ('5', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('1', 'O'),\n",
       "  (')', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('50р', 'O'),\n",
       "  (';', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Bloodrayme', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('продала', 'O'),\n",
       "  ('(', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('фото', 'O'),\n",
       "  ('осталась', 'O'),\n",
       "  (')', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Если', 'O'),\n",
       "  ('покупаете', 'O'),\n",
       "  ('все', 'O'),\n",
       "  ('4', 'O'),\n",
       "  ('диска', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('то', 'O'),\n",
       "  ('цена', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('150', 'O'),\n",
       "  ('р', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('+', 'O'),\n",
       "  ('подарок', 'O'),\n",
       "  ('Игромания', 'O'),\n",
       "  ('4', 'O'),\n",
       "  ('диска', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('PS4', 'B-PRS'),\n",
       "  ('+', 'I-PRS'),\n",
       "  ('2', 'O'),\n",
       "  ('джоя', 'O'),\n",
       "  ('+', 'O'),\n",
       "  ('Камера', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('пару', 'O'),\n",
       "  ('игр', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('выбор', 'O')],\n",
       " [('Плейстейшен', 'B-PRS'),\n",
       "  ('4', 'I-PRS'),\n",
       "  ('500', 'O'),\n",
       "  ('г', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Приставка', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('отличном', 'O'),\n",
       "  ('состоянии', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('черного', 'O'),\n",
       "  ('цвета', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Магазинный', 'O'),\n",
       "  ('комплект', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('один', 'O'),\n",
       "  ('геймпад', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('все', 'O'),\n",
       "  ('шнуры', 'O'),\n",
       "  ('необходимые', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('документы', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Использовалась', 'O'),\n",
       "  ('очень', 'O'),\n",
       "  ('редко', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('обклеена', 'O'),\n",
       "  ('виниловой', 'O'),\n",
       "  ('наклейкой', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('легко', 'O'),\n",
       "  ('снимается', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('следов', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('остается', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Интересует', 'O'),\n",
       "  ('обмен', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('Икс', 'B-PRS'),\n",
       "  ('Бокс', 'I-PRS'),\n",
       "  ('Ван', 'I-PRS'),\n",
       "  ('с', 'O'),\n",
       "  ('вашей', 'O'),\n",
       "  ('доплатой', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('зависимости', 'O'),\n",
       "  ('от', 'O'),\n",
       "  ('комплектации', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Xbox', 'B-PRS'),\n",
       "  ('one', 'I-PRS'),\n",
       "  ('без', 'O'),\n",
       "  ('коробки', 'O'),\n",
       "  ('+', 'O'),\n",
       "  ('второй', 'O'),\n",
       "  ('геймпад', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Три', 'O'),\n",
       "  ('игры', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Xbox', 'B-PRS'),\n",
       "  ('в', 'O'),\n",
       "  ('отличном', 'O'),\n",
       "  ('состоянии', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('меньше', 'O'),\n",
       "  ('года', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('использовании', 'O'),\n",
       "  ('был', 'O'),\n",
       "  ('мало', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('покупали', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('для', 'O'),\n",
       "  ('детей', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('+', 'O'),\n",
       "  ('игры', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('торг', 'O')],\n",
       " [('PS4', 'B-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('1208', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('500', 'O'),\n",
       "  ('Гб', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('Белая', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('идеальном', 'O'),\n",
       "  ('состоянии', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('коробка', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('документы', 'O')],\n",
       " [('PlayStation', 'B-PRS'),\n",
       "  ('Eye', 'I-PRS'),\n",
       "  ('камера', 'O'),\n",
       "  ('подходит', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('для', 'O'),\n",
       "  ('PS3', 'B-PRS'),\n",
       "  ('и', 'O'),\n",
       "  ('для', 'O'),\n",
       "  ('PS4', 'B-PRS'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Можно', 'O'),\n",
       "  ('использовать', 'O'),\n",
       "  ('как', 'O'),\n",
       "  ('веб', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('камеру', 'O'),\n",
       "  ('для', 'O'),\n",
       "  ('ПК', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('В', 'O'),\n",
       "  ('отличном', 'O'),\n",
       "  ('состоянии', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Могу', 'O'),\n",
       "  ('почтой', 'O'),\n",
       "  ('отправить', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('PS3', 'B-PRS'),\n",
       "  ('серая', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('2', 'O'),\n",
       "  ('джойстика', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('зарядка', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('джойстик', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('тюльпаны', 'O'),\n",
       "  ('к', 'O'),\n",
       "  ('телевизору', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('вилка', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Консоль', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('5', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('никогда', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('глючил', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('зависал', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('отлично', 'O'),\n",
       "  ('работает', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('+', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('подарок', 'O'),\n",
       "  ('6', 'O'),\n",
       "  ('игры', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('PS4', 'B-PRS'),\n",
       "  ('500gb', 'O'),\n",
       "  ('белый', 'O'),\n",
       "  ('ледник', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('новая', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('ни', 'O'),\n",
       "  ('разу', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('включалась', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('доставалась', 'O'),\n",
       "  ('из', 'O'),\n",
       "  ('коробки', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('2джойстика', 'O'),\n",
       "  ('+', 'O'),\n",
       "  ('фифа17', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('торг', 'O')],\n",
       " [('ps4', 'B-PRS'),\n",
       "  ('slim', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('500', 'O'),\n",
       "  ('Гб', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('б', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('у', 'O'),\n",
       "  ('2', 'O'),\n",
       "  ('месяца', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('использовалась', 'O'),\n",
       "  ('мало', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('жестком', 'O'),\n",
       "  ('диске', 'O'),\n",
       "  ('установлен', 'O'),\n",
       "  ('ведьмак', 'O'),\n",
       "  ('золотое', 'O'),\n",
       "  ('издание', 'O'),\n",
       "  ('шторм', 'O'),\n",
       "  ('4', 'I-PRS'),\n",
       "  ('и', 'O'),\n",
       "  ('пара', 'O'),\n",
       "  ('мелких', 'O'),\n",
       "  ('игр', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('обменяю', 'O'),\n",
       "  ('все', 'O'),\n",
       "  ('это', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('хороший', 'O'),\n",
       "  ('компьютер', 'O'),\n",
       "  ('или', 'O'),\n",
       "  ('что', 'O'),\n",
       "  ('нибудь', 'O'),\n",
       "  ('еще', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('ремонтах', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('была', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('никаких', 'O'),\n",
       "  ('косяков', 'O'),\n",
       "  ('нет', 'O')],\n",
       " [('Playstation', 'B-PRS'),\n",
       "  ('4', 'I-PRS'),\n",
       "  ('slim', 'I-PRS'),\n",
       "  ('новая', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('PS4', 'B-PRS'),\n",
       "  ('500gb', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Данная', 'O'),\n",
       "  ('консоль', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('500гб', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Модель', 'O'),\n",
       "  (':', 'O'),\n",
       "  ('CUH', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('2015A', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Абсолютно', 'O'),\n",
       "  ('новые', 'O'),\n",
       "  ('PS4', 'B-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('привезены', 'O'),\n",
       "  ('из', 'O'),\n",
       "  ('Америки', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Проверены', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('работают', 'O'),\n",
       "  ('отлично', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Приставка', 'O'),\n",
       "  ('замечательно', 'O'),\n",
       "  ('работает', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('России', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('воспроизводит', 'O'),\n",
       "  ('все', 'O'),\n",
       "  ('игры', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('русские', 'O'),\n",
       "  ('Blu', 'B-PRS'),\n",
       "  ('-', 'I-PRS'),\n",
       "  ('ray', 'I-PRS'),\n",
       "  ('диски', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('В', 'O'),\n",
       "  ('наличии', 'O'),\n",
       "  ('2е', 'O'),\n",
       "  ('штуки', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Assassins', 'B-PRS'), ('Creed', 'I-PRS'), ('Единство', 'O')],\n",
       " [('PSP', 'B-PRS'),\n",
       "  ('в', 'O'),\n",
       "  ('нормальном', 'O'),\n",
       "  ('состоянии', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('+', 'O'),\n",
       "  ('2', 'O'),\n",
       "  ('игры', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('разбит', 'O'),\n",
       "  ('дисплей', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('зарядка', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('комплекте', 'O')],\n",
       " [('PSP', 'B-PRS'),\n",
       "  ('в', 'O'),\n",
       "  ('хорошем', 'O'),\n",
       "  ('состояние', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Ни', 'O'),\n",
       "  ('разу', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('падала', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('утопленник', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('без', 'O'),\n",
       "  ('прошивки', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Игры', 'O'),\n",
       "  (':', 'O'),\n",
       "  ('gta', 'O'),\n",
       "  ('vice', 'O'),\n",
       "  ('city', 'O'),\n",
       "  ('stories', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('need', 'O'),\n",
       "  ('for', 'O'),\n",
       "  ('speed', 'O'),\n",
       "  ('undercover', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('тачки', 'O'),\n",
       "  ('2', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Чехол', 'O'),\n",
       "  ('BLACK', 'O'),\n",
       "  ('HORNS', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('На', 'O'),\n",
       "  ('psp', 'B-PRS'),\n",
       "  ('нет', 'O'),\n",
       "  ('wi', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('fi', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Б', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('У', 'O'),\n",
       "  ('мало', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Ps3', 'B-PRS'),\n",
       "  ('+', 'O'),\n",
       "  ('6', 'O'),\n",
       "  ('игр', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('джостик', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('hdmi', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('птиание', 'O'),\n",
       "  ('к', 'O'),\n",
       "  ('ps3', 'B-PRS'),\n",
       "  ('и', 'O'),\n",
       "  ('джостику', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('игры', 'O'),\n",
       "  (':', 'O'),\n",
       "  ('Assasins', 'O'),\n",
       "  ('creed', 'O'),\n",
       "  ('4', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('Red', 'B-PRS'),\n",
       "  ('faction', 'I-PRS'),\n",
       "  ('armagedon', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('resistance', 'O'),\n",
       "  ('3', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('За', 'O'),\n",
       "  ('гранью', 'O'),\n",
       "  ('2', 'O'),\n",
       "  ('души', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('split', 'B-PRS'),\n",
       "  ('second', 'I-PRS'),\n",
       "  ('velocity', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('sacred', 'O'),\n",
       "  ('3', 'O'),\n",
       "  ('гнев', 'O'),\n",
       "  ('малахима', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Торг', 'O'),\n",
       "  ('уместен', 'O'),\n",
       "  ('!', 'O')],\n",
       " [('Плейстешин', 'B-PRS'),\n",
       "  ('в', 'O'),\n",
       "  ('хорошем', 'O'),\n",
       "  ('состоянии', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('игры', 'O'),\n",
       "  ('фифа17демо', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('фифа15', 'B-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('гта5', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('1джойстик', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('500гб', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Все', 'O'),\n",
       "  ('отлично', 'O'),\n",
       "  ('работает', 'O'),\n",
       "  ('все', 'O'),\n",
       "  ('шнуры', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('месте', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Возможен', 'O'),\n",
       "  ('обмен', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('телефон', 'B-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('или', 'O'),\n",
       "  ('торг', 'O')],\n",
       " [('Fifa', 'B-PRS'),\n",
       "  ('2016', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('700', 'O'),\n",
       "  ('р', 'O'),\n",
       "  ('.', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('farcry', 'O'),\n",
       "  ('primal', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('1200', 'O'),\n",
       "  ('р', 'O'),\n",
       "  ('.', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('gear', 'O'),\n",
       "  ('of', 'O'),\n",
       "  ('war', 'B-PRS'),\n",
       "  ('4', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('1600', 'O'),\n",
       "  ('р', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Ps3', 'B-PRS'),\n",
       "  ('2', 'I-PRS'),\n",
       "  ('джойстика', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('2', 'O'),\n",
       "  ('игры', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('дисках', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Gta5', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('fifa16', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('6900р', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('с', 'O'),\n",
       "  ('коробкой', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('проводами', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Подключил', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('играй', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Из', 'O'),\n",
       "  ('минусов', 'O'),\n",
       "  ('нет', 'O'),\n",
       "  ('одной', 'O'),\n",
       "  ('пластиковой', 'O'),\n",
       "  ('накладки', 'O'),\n",
       "  ('с', 'O'),\n",
       "  ('верху', 'O'),\n",
       "  ('приставки', 'O'),\n",
       "  ('(', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('что', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('влияет', 'O'),\n",
       "  (')', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Полностью', 'O'),\n",
       "  ('исправна', 'O'),\n",
       "  ('без', 'O'),\n",
       "  ('глюков', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('прошивалась', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('гарантия', 'O'),\n",
       "  ('2', 'O'),\n",
       "  ('недели', 'O'),\n",
       "  ('Территориально', 'O'),\n",
       "  ('авроры', 'O'),\n",
       "  ('199', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Возможен', 'O'),\n",
       "  ('обмен', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('ноутбук', 'O'),\n",
       "  ('даже', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('нерабочем', 'O'),\n",
       "  ('состоянии', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Псп', 'B-PRS'),\n",
       "  ('рабочая', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('мелкая', 'O'),\n",
       "  ('царапинка', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('экране', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('целая', 'O'),\n",
       "  ('крышка', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('рабочий', 'O'),\n",
       "  ('привод', 'O'),\n",
       "  ('для', 'O'),\n",
       "  ('дисков', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('флешка', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('8гб', 'O'),\n",
       "  ('с', 'O'),\n",
       "  ('прошивкой', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('игрой', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('червяки', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('рассматриваю', 'O'),\n",
       "  ('обмен', 'O')],\n",
       " [('Mirrors', 'B-PRS'),\n",
       "  ('Edge', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('600', 'O'),\n",
       "  ('₽', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Plants', 'B-PRS'),\n",
       "  ('vs', 'I-PRS'),\n",
       "  ('Zombies', 'I-PRS'),\n",
       "  ('Garden', 'I-PRS'),\n",
       "  ('Warfare', 'I-PRS'),\n",
       "  ('2', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('600', 'O'),\n",
       "  ('₽', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Все', 'O'),\n",
       "  ('вместе', 'O'),\n",
       "  ('900', 'O'),\n",
       "  ('₽', 'O')],\n",
       " [('Diablo', 'B-PRS'),\n",
       "  ('3', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('1500', 'O'),\n",
       "  ('р', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('UFC', 'B-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('800', 'O'),\n",
       "  ('р', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Sony', 'B-PRS'),\n",
       "  ('PlayStation', 'I-PRS'),\n",
       "  ('3', 'I-PRS'),\n",
       "  ('SuperSlim', 'I-PRS'),\n",
       "  ('500', 'O'),\n",
       "  ('ГБ', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('состояние', 'O'),\n",
       "  ('–', 'O'),\n",
       "  ('отличное', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('В', 'O'),\n",
       "  ('комплекте', 'O'),\n",
       "  (':', 'O'),\n",
       "  ('диск', 'O'),\n",
       "  ('с', 'O'),\n",
       "  ('игрой', 'O'),\n",
       "  ('Rayman', 'B-PRS'),\n",
       "  ('Legends', 'I-PRS'),\n",
       "  ('+', 'O'),\n",
       "  ('Rayman', 'B-PRS'),\n",
       "  ('Origins', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('комплект', 'O'),\n",
       "  ('проводов', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('беспроводной', 'O'),\n",
       "  ('контроллер', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('СРОЧНО', 'O')],\n",
       " [('PS3', 'B-PRS'),\n",
       "  ('с', 'O'),\n",
       "  ('пятью', 'O'),\n",
       "  ('играми', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('комплекте', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Игры', 'O'),\n",
       "  (':', 'O'),\n",
       "  ('UNCHARTED', 'O'),\n",
       "  ('3', 'O'),\n",
       "  ('иллюзии', 'O'),\n",
       "  ('Дрейка', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('THE', 'O'),\n",
       "  ('LAST', 'O'),\n",
       "  ('OF', 'O'),\n",
       "  ('US', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('GRAN', 'O'),\n",
       "  ('TURISMO', 'O'),\n",
       "  ('5', 'O'),\n",
       "  (',', 'O'),\n",
       "  (\"ASSASSIN'S\", 'B-PRS'),\n",
       "  ('CREED', 'O'),\n",
       "  ('IV', 'I-PRS'),\n",
       "  ('Чёрный', 'O'),\n",
       "  ('Флаг', 'O'),\n",
       "  ('и', 'O'),\n",
       "  (\"ASSASSIN'S\", 'B-PRS'),\n",
       "  ('CREED', 'O'),\n",
       "  ('ИЗГОЙ', 'O'),\n",
       "  ('все', 'O'),\n",
       "  ('игры', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('РУССКОМ', 'O'),\n",
       "  ('ЯЗЫКЕ', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Приставку', 'O'),\n",
       "  ('брал', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('2015', 'O'),\n",
       "  ('году', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('феврале', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Xbox', 'B-PRS'),\n",
       "  ('360', 'I-PRS'),\n",
       "  ('250', 'O'),\n",
       "  ('Gb', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('шитый', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('В', 'O'),\n",
       "  ('комплекте', 'O'),\n",
       "  ('коробка', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('приставка', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('кабель', 'O'),\n",
       "  ('питания', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('RCA', 'O'),\n",
       "  ('кабель', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('беспроводной', 'O'),\n",
       "  ('геймпад', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('наушники', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('На', 'O'),\n",
       "  ('жестком', 'O'),\n",
       "  ('диске', 'O'),\n",
       "  ('установлен', 'O'),\n",
       "  ('Ведьмак', 'O'),\n",
       "  ('2', 'O'),\n",
       "  ('Убийцы', 'O'),\n",
       "  ('королей', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Также', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('дисках', 'O'),\n",
       "  ('есть', 'O'),\n",
       "  ('GTA', 'B-PRS'),\n",
       "  ('V', 'I-PRS'),\n",
       "  ('и', 'O'),\n",
       "  ('Forza', 'B-PRS'),\n",
       "  ('Motorsport', 'I-PRS'),\n",
       "  ('4', 'I-PRS'),\n",
       "  ('.', 'O')],\n",
       " [('Playstation', 'B-PRS'), ('3', 'I-PRS')],\n",
       " [('Psp', 'B-PRS'),\n",
       "  ('в', 'O'),\n",
       "  ('хорошем', 'O'),\n",
       "  ('состоянии', 'O'),\n",
       "  ('+', 'O'),\n",
       "  ('карта', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('16гигов', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Прошивка', 'O'),\n",
       "  ('присутствует', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('игор', 'O'),\n",
       "  ('много', 'O')],\n",
       " [('Call', 'B-PRS'),\n",
       "  ('of', 'I-PRS'),\n",
       "  ('Duty', 'I-PRS'),\n",
       "  ('infinite', 'O'),\n",
       "  ('warfare', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('2000р', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Sony', 'B-PRS'),\n",
       "  ('Custom', 'I-PRS'),\n",
       "  ('Faceplate', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('специальная', 'O'),\n",
       "  ('лицевая', 'O'),\n",
       "  ('панель', 'O'),\n",
       "  ('для', 'O'),\n",
       "  ('игровой', 'O'),\n",
       "  ('консоли', 'O'),\n",
       "  ('нового', 'O'),\n",
       "  ('поколения', 'O'),\n",
       "  ('Sony', 'B-PRS'),\n",
       "  ('Playstation', 'I-PRS'),\n",
       "  ('4', 'I-PRS'),\n",
       "  ('.', 'O'),\n",
       "  ('Измените', 'O'),\n",
       "  ('облик', 'O'),\n",
       "  ('вашей', 'O'),\n",
       "  ('PS4', 'B-PRS'),\n",
       "  ('при', 'O'),\n",
       "  ('помощи', 'O'),\n",
       "  ('этой', 'O'),\n",
       "  ('пользовательской', 'O'),\n",
       "  ('лицевой', 'O'),\n",
       "  ('панели', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('(', 'O'),\n",
       "  ('Новая', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('оригинальной', 'O'),\n",
       "  ('упаковке', 'O'),\n",
       "  (')', 'O')],\n",
       " [('Watch', 'B-PRS'),\n",
       "  ('Dogs', 'I-PRS'),\n",
       "  ('700', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Call', 'B-PRS'),\n",
       "  ('Of', 'I-PRS'),\n",
       "  ('Duty', 'I-PRS'),\n",
       "  (':', 'O'),\n",
       "  ('Black', 'B-PRS'),\n",
       "  ('Ops', 'I-PRS'),\n",
       "  ('600', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Battlefield', 'B-PRS'),\n",
       "  ('4', 'I-PRS'),\n",
       "  ('500', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Assasins', 'B-PRS'),\n",
       "  ('Creed', 'I-PRS'),\n",
       "  ('3', 'I-PRS'),\n",
       "  ('500', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Epic', 'B-PRS'),\n",
       "  ('Mickey', 'I-PRS'),\n",
       "  ('Две', 'O'),\n",
       "  ('легенды', 'O'),\n",
       "  ('500', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Avatar', 'B-PRS'),\n",
       "  ('400', 'I-PRS'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Диски', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('отличном', 'O'),\n",
       "  ('состоянии', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Зеркало', 'O'),\n",
       "  ('поверхности', 'O'),\n",
       "  ('без', 'O'),\n",
       "  ('изъянов', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Доставка', 'O'),\n",
       "  ('по', 'O'),\n",
       "  ('городу', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Отправлю', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('регионы', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Почта', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('тк', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Для', 'O'),\n",
       "  ('связи', 'O'),\n",
       "  ('смс', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('номер', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('сообщение', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('Авито', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Battlefield', 'B-PRS'),\n",
       "  ('-', 'I-PRS'),\n",
       "  ('1', 'I-PRS'),\n",
       "  ('2000', 'O'),\n",
       "  ('Assassin', 'B-PRS'),\n",
       "  ('Creed', 'I-PRS'),\n",
       "  ('Синдикат', 'O'),\n",
       "  ('1500', 'O'),\n",
       "  ('Uncharted', 'O'),\n",
       "  ('коллекция', 'O'),\n",
       "  ('1500', 'O'),\n",
       "  ('Все', 'O'),\n",
       "  ('диски', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('идеальном', 'O'),\n",
       "  ('состоянии', 'O')],\n",
       " [('Final', 'B-PRS'),\n",
       "  ('Fantasy', 'I-PRS'),\n",
       "  ('XIII', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('2', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('600р', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Street', 'B-PRS'),\n",
       "  ('Fighter', 'I-PRS'),\n",
       "  ('IV', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('600р', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Need', 'B-PRS'),\n",
       "  ('for', 'I-PRS'),\n",
       "  ('Speed', 'I-PRS'),\n",
       "  (':', 'O'),\n",
       "  ('Rivals', 'B-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('700р', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Возможна', 'O'),\n",
       "  ('покупка', 'O'),\n",
       "  ('дисков', 'O'),\n",
       "  ('по', 'O'),\n",
       "  ('отдельности', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Все', 'O'),\n",
       "  ('диски', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('идеальном', 'O'),\n",
       "  ('состоянии', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('лицензия', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Grand', 'B-PRS'),\n",
       "  ('Theft', 'B-PRS'),\n",
       "  ('Auto', 'I-PRS'),\n",
       "  ('IV', 'I-PRS'),\n",
       "  ('and', 'O'),\n",
       "  ('Episodes', 'O'),\n",
       "  ('From', 'O'),\n",
       "  ('Liberty', 'B-PRS'),\n",
       "  ('City', 'I-PRS'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('GTA', 'B-PRS'),\n",
       "  ('4', 'I-PRS'),\n",
       "  ('+', 'O'),\n",
       "  ('The', 'O'),\n",
       "  ('lost', 'O'),\n",
       "  ('and', 'O'),\n",
       "  ('Damned', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('The', 'O'),\n",
       "  ('Ballad', 'O'),\n",
       "  ('of', 'O'),\n",
       "  ('gay', 'B-PRS'),\n",
       "  ('Tony', 'I-PRS')],\n",
       " [('Minecraft', 'B-PRS'),\n",
       "  ('+', 'O'),\n",
       "  ('builders', 'B-PRS'),\n",
       "  ('pack', 'I-PRS'),\n",
       "  ('+', 'O'),\n",
       "  ('версия', 'O'),\n",
       "  ('игры', 'O'),\n",
       "  ('для', 'O'),\n",
       "  ('windows', 'B-PRS'),\n",
       "  ('10', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Коды', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('загрузку', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Nintendo', 'B-PRS'),\n",
       "  ('2ds', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('CFW', 'O'),\n",
       "  ('(', 'O'),\n",
       "  ('кастомная', 'O'),\n",
       "  ('прошивка', 'O'),\n",
       "  (')', 'O'),\n",
       "  ('установлена', 'O'),\n",
       "  ('во', 'O'),\n",
       "  ('внутреннюю', 'O'),\n",
       "  ('память', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('100', 'O'),\n",
       "  ('%', 'O'),\n",
       "  ('быстрая', 'O'),\n",
       "  ('загрузка', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('никаких', 'O'),\n",
       "  ('эмунандов', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Можно', 'O'),\n",
       "  ('играть', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('онлайн', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('устанавливать', 'O'),\n",
       "  ('обновления', 'O'),\n",
       "  ('без', 'O'),\n",
       "  ('риска', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('что', 'O'),\n",
       "  ('прошивка', 'O'),\n",
       "  ('пропадет', 'O'),\n",
       "  ('или', 'O'),\n",
       "  ('что', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('то', 'O'),\n",
       "  ('сломается', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Установлена', 'O'),\n",
       "  ('карта', 'O'),\n",
       "  ('памяти', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('16гб', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('несколько', 'O'),\n",
       "  ('игр', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('включая', 'O'),\n",
       "  ('последних', 'O'),\n",
       "  ('покемонов', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('онлайн', 'O'),\n",
       "  ('работает', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('В', 'O'),\n",
       "  ('комплекте', 'O'),\n",
       "  ('зарядка', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('коробка', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('документация', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Продаю', 'O'),\n",
       "  ('по', 'O'),\n",
       "  ('причине', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('что', 'O'),\n",
       "  ('есть', 'O'),\n",
       "  ('вторая', 'O'),\n",
       "  ('приставка', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Одна', 'O'),\n",
       "  ('лежит', 'O'),\n",
       "  ('без', 'O'),\n",
       "  ('дела', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Grand', 'B-PRS'),\n",
       "  ('turismo', 'I-PRS'),\n",
       "  ('6', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('600', 'O'),\n",
       "  ('р', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Killzone', 'B-PRS'),\n",
       "  ('3', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('600', 'O'),\n",
       "  ('р', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Heavy', 'B-PRS'),\n",
       "  ('rain', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('600', 'O'),\n",
       "  ('р', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Uncharted', 'O'),\n",
       "  ('3', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('900', 'O'),\n",
       "  ('р', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('За', 'O'),\n",
       "  ('гранью', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('600', 'O'),\n",
       "  ('р', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Одни', 'O'),\n",
       "  ('из', 'O'),\n",
       "  ('нас', 'O'),\n",
       "  ('-', 'O'),\n",
       "  ('900', 'O'),\n",
       "  ('р', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Gta', 'B-PRS'),\n",
       "  ('4', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('600', 'O'),\n",
       "  ('р', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Кто', 'O'),\n",
       "  ('возьмет', 'O'),\n",
       "  ('все', 'O'),\n",
       "  ('сразу', 'O'),\n",
       "  ('сделаю', 'O'),\n",
       "  ('хорошую', 'O'),\n",
       "  ('скидку', 'O')],\n",
       " [('Сонька', 'B-PRS'),\n",
       "  ('2', 'I-PRS'),\n",
       "  ('джойстики', 'O'),\n",
       "  ('2', 'O'),\n",
       "  ('кары', 'O'),\n",
       "  ('памяти', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('32', 'O'),\n",
       "  ('гига', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('8', 'O'),\n",
       "  ('гигов', 'O'),\n",
       "  ('плюс', 'O'),\n",
       "  ('около', 'O'),\n",
       "  ('100', 'O'),\n",
       "  ('дисков', 'O')],\n",
       " [('hitman', 'B-PRS'), ('contracts', 'O')],\n",
       " [('PS3', 'B-PRS'),\n",
       "  ('фифа', 'O'),\n",
       "  ('17', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('16', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('другие', 'O'),\n",
       "  ('игры', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Приставка', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('хорошем', 'O'),\n",
       "  ('состоянии', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Возможен', 'O'),\n",
       "  ('торг', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('В', 'O'),\n",
       "  ('комплект', 'O'),\n",
       "  ('входит', 'O'),\n",
       "  ('2', 'O'),\n",
       "  ('джойстика', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('фифа17', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('16', 'O'),\n",
       "  ('gran', 'O'),\n",
       "  ('turismo', 'O'),\n",
       "  ('5', 'O'),\n",
       "  ('(', 'O'),\n",
       "  ('гонки', 'O'),\n",
       "  (')', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('праздник', 'O'),\n",
       "  ('спорта', 'B-PRS'),\n",
       "  ('2', 'I-PRS'),\n",
       "  ('.', 'O'),\n",
       "  ('Камера', 'O'),\n",
       "  ('для', 'O'),\n",
       "  ('праздника', 'O'),\n",
       "  ('спорта', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('2', 'O'),\n",
       "  ('джойстика', 'O'),\n",
       "  ('для', 'O'),\n",
       "  ('праздника', 'O'),\n",
       "  ('спорта', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Причина', 'O'),\n",
       "  ('продажи', 'O'),\n",
       "  (':', 'O'),\n",
       "  ('появилась', 'O'),\n",
       "  ('другая', 'O'),\n",
       "  ('приставка', 'O'),\n",
       "  ('ps4', 'B-PRS'),\n",
       "  ('.', 'O')],\n",
       " [('PS4', 'B-PRS'),\n",
       "  ('500GB', 'O'),\n",
       "  ('rev', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('1108', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('GamePad', 'O'),\n",
       "  ('новый', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('PS', 'B-PRS'),\n",
       "  ('Camera', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('игры', 'O'),\n",
       "  ('BattleField', 'B-PRS'),\n",
       "  ('1', 'I-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('Dragon', 'B-PRS'),\n",
       "  ('Age', 'I-PRS'),\n",
       "  ('Inquisition', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('Evil', 'B-PRS'),\n",
       "  ('Within', 'I-PRS'),\n",
       "  ('на', 'O'),\n",
       "  ('профиле', 'O'),\n",
       "  ('бонусом', 'O')],\n",
       " [('Sony', 'B-PRS'),\n",
       "  ('Play', 'I-PRS'),\n",
       "  ('Station', 'I-PRS'),\n",
       "  ('3', 'I-PRS'),\n",
       "  ('Super', 'I-PRS'),\n",
       "  ('Slim', 'I-PRS'),\n",
       "  ('250gb', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('В', 'O'),\n",
       "  ('комплекте', 'O'),\n",
       "  ('1', 'O'),\n",
       "  ('Беспроводной', 'O'),\n",
       "  ('джойстик', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('2', 'O'),\n",
       "  ('Диска', 'O'),\n",
       "  (':', 'O'),\n",
       "  ('Red', 'B-PRS'),\n",
       "  ('Dead', 'I-PRS'),\n",
       "  ('Detemption', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('Watch', 'B-PRS'),\n",
       "  ('Dogs', 'I-PRS')],\n",
       " [('Playstation', 'B-PRS'),\n",
       "  ('3', 'I-PRS'),\n",
       "  ('на', 'O'),\n",
       "  ('300', 'O'),\n",
       "  ('gb', 'O'),\n",
       "  ('памяти', 'O'),\n",
       "  ('не', 'O'),\n",
       "  ('чипованная', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('рабочая', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('4', 'O'),\n",
       "  ('игры', 'O'),\n",
       "  (':', 'O'),\n",
       "  ('crysis', 'B-PRS'),\n",
       "  ('2', 'I-PRS'),\n",
       "  (';', 'O'),\n",
       "  ('gta', 'O'),\n",
       "  ('4', 'O'),\n",
       "  (';', 'O'),\n",
       "  ('gta', 'O'),\n",
       "  ('5', 'O'),\n",
       "  (';', 'O'),\n",
       "  ('farcry', 'O'),\n",
       "  ('3', 'I-PRS'),\n",
       "  ('.', 'O')],\n",
       " [('PSP', 'B-PRS'),\n",
       "  ('e1008', 'I-PRS'),\n",
       "  ('с', 'O'),\n",
       "  ('набором', 'O'),\n",
       "  ('всех', 'O'),\n",
       "  ('аксессуаров', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('немного', 'O'),\n",
       "  ('отломан', 'O'),\n",
       "  ('джлстик', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('но', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('он', 'O'),\n",
       "  ('полностью', 'O'),\n",
       "  ('рабочий', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Xbox', 'B-PRS'),\n",
       "  ('360', 'I-PRS'),\n",
       "  ('slim', 'I-PRS'),\n",
       "  ('250gb', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Состояние', 'O'),\n",
       "  ('хорошее', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('прошитая', 'O'),\n",
       "  ('lt', 'O'),\n",
       "  ('3', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('0', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('xbox', 'B-PRS'),\n",
       "  ('live', 'I-PRS'),\n",
       "  ('не', 'O'),\n",
       "  ('забанен', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('игры', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('комплекте', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Watch', 'B-PRS'),\n",
       "  ('Dogs', 'I-PRS'),\n",
       "  ('2', 'I-PRS'),\n",
       "  ('—', 'O'),\n",
       "  ('мультиплатформенная', 'O'),\n",
       "  ('компьютерная', 'O'),\n",
       "  ('игра', 'O'),\n",
       "  ('в', 'O'),\n",
       "  ('жанре', 'O'),\n",
       "  ('приключенческого', 'O'),\n",
       "  ('боевика', 'O'),\n",
       "  ('с', 'O'),\n",
       "  ('открытым', 'O'),\n",
       "  ('миром', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('вышедшая', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('платформах', 'O'),\n",
       "  ('PlayStation', 'B-PRS'),\n",
       "  ('4', 'I-PRS'),\n",
       "  ('и', 'O'),\n",
       "  ('Xbox', 'B-PRS'),\n",
       "  ('One', 'I-PRS'),\n",
       "  ('15', 'O'),\n",
       "  ('ноября', 'O'),\n",
       "  ('2016', 'O'),\n",
       "  ('года', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('На', 'O'),\n",
       "  ('PC', 'B-PRS'),\n",
       "  ('игра', 'O'),\n",
       "  ('вышла', 'O'),\n",
       "  ('29', 'O'),\n",
       "  ('ноября', 'O'),\n",
       "  ('2016', 'O'),\n",
       "  ('года', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Обмен', 'O'),\n",
       "  (',', 'O'),\n",
       "  ('пррдажа', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Мы', 'O'),\n",
       "  ('находимся', 'O'),\n",
       "  ('ТК', 'O'),\n",
       "  ('Прииск', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Большой', 'O'),\n",
       "  ('выбор', 'O'),\n",
       "  ('игр', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('ps4', 'B-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('ps3', 'B-PRS'),\n",
       "  (',', 'O'),\n",
       "  ('xbox360', 'B-PRS'),\n",
       "  ('.', 'O')],\n",
       " [('Watch', 'B-PRS'),\n",
       "  ('Dogs', 'I-PRS'),\n",
       "  ('2', 'I-PRS'),\n",
       "  ('Deluxe', 'I-PRS'),\n",
       "  ('Edition', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('1900', 'O'),\n",
       "  ('руб', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  (\"Assassin's\", 'B-PRS'),\n",
       "  ('Creed', 'I-PRS'),\n",
       "  ('Syndicate', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('1400', 'O'),\n",
       "  ('руб', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('UFC', 'B-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('800', 'O'),\n",
       "  ('руб', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Destiny', 'O'),\n",
       "  ('The', 'B-PRS'),\n",
       "  ('Taken', 'I-PRS'),\n",
       "  ('King', 'I-PRS'),\n",
       "  ('-', 'O'),\n",
       "  ('1400', 'O'),\n",
       "  ('руб', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Возможен', 'O'),\n",
       "  ('обмен', 'O'),\n",
       "  ('.', 'O')],\n",
       " [('Mortal', 'B-PRS'),\n",
       "  ('Kombat', 'I-PRS'),\n",
       "  ('X', 'I-PRS'),\n",
       "  ('(', 'O'),\n",
       "  ('Диск', 'O'),\n",
       "  ('для', 'O'),\n",
       "  ('Playstation', 'B-PRS'),\n",
       "  ('4', 'I-PRS'),\n",
       "  (')', 'O'),\n",
       "  ('/', 'O'),\n",
       "  ('\\n', 'O'),\n",
       "  ('Торга', 'O'),\n",
       "  ('и', 'O'),\n",
       "  ('обмена', 'O'),\n",
       "  ('нет', 'O')],\n",
       " [('Horizon', 'B-PRS'),\n",
       "  ('Zero', 'I-PRS'),\n",
       "  ('Dawn', 'I-PRS'),\n",
       "  ('PS4', 'B-PRS'),\n",
       "  ('полностью', 'O'),\n",
       "  ('на', 'O'),\n",
       "  ('русском', 'O'),\n",
       "  ('.', 'O'),\n",
       "  ('Только', 'O'),\n",
       "  ('продажа', 'O'),\n",
       "  ('.', 'O')]]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "marked"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Приставки конечно размечаются, но ещё и другие слова на английском начали захватываться. Можно попробовать добавить в данные примеры с ними, или сделать даже ещё пару типов сущностей, чтобы модель научилась их разделять."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "В этой же моделе есть несколько стандартных сущностей."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2019-12-19 12:25:41.729 INFO in 'deeppavlov.core.data.utils'['utils'] at line 80: Downloading from http://files.deeppavlov.ai/deeppavlov_data/ner_ontonotes_bert_mult_v1.tar.gz to /root/.deeppavlov/ner_ontonotes_bert_mult_v1.tar.gz\n",
      "100%|██████████| 1.32G/1.32G [07:07<00:00, 3.09MB/s]\n",
      "2019-12-19 12:32:48.739 INFO in 'deeppavlov.core.data.utils'['utils'] at line 237: Extracting /root/.deeppavlov/ner_ontonotes_bert_mult_v1.tar.gz archive into /root/.deeppavlov/models\n",
      "2019-12-19 12:33:02.850 INFO in 'deeppavlov.download'['download'] at line 117: Skipped http://files.deeppavlov.ai/deeppavlov_data/bert/multi_cased_L-12_H-768_A-12.zip download because of matching hashes\n",
      "2019-12-19 12:33:03.371 INFO in 'deeppavlov.core.data.simple_vocab'['simple_vocab'] at line 115: [loading vocabulary from /root/.deeppavlov/models/ner_ontonotes_bert_mult/tag.dict]\n",
      "2019-12-19 12:33:28.634 INFO in 'deeppavlov.core.models.tf_model'['tf_model'] at line 51: [loading model from /root/.deeppavlov/models/ner_ontonotes_bert_mult/model]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from /root/.deeppavlov/models/ner_ontonotes_bert_mult/model\n"
     ]
    }
   ],
   "source": [
    "ner_model = build_model(configs.ner.ner_ontonotes_bert_mult, download=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[['Передай', 'привет', 'Михаилу', 'Нефедову']],\n",
       " [['O', 'O', 'B-PERSON', 'I-PERSON']]]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ner_model(['Передай привет Михаилу Нефедову'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[['Поезжай', 'в', 'Москву']], [['O', 'O', 'B-GPE']]]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ner_model(['Поезжай в Москву'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[['Поезжай', 'в', 'Альпы']], [['O', 'O', 'B-LOC']]]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ner_model(['Поезжай в Альпы'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[['Я', 'говорю', 'на', 'русском', 'и', 'английском', 'языках']],\n",
       " [['O', 'O', 'O', 'B-LANGUAGE', 'O', 'B-LANGUAGE', 'O']]]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ner_model(['Я говорю на русском и английском языках'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[['Бутылка', 'воды', 'стоит', '50', 'рублей']],\n",
       " [['O', 'O', 'O', 'B-MONEY', 'I-MONEY']]]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ner_model(['Бутылка воды стоит 50 рублей'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Но он конечно не идеальный."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[[['ле', 'Биг', 'мак']], [['B-PERSON', 'I-PERSON', 'I-PERSON']]]"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ner_model(['ле Биг мак'])"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "name": "deep_pavlov.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}