4.0 MedCAT_Tutorial___MedCATtrainer - Supervised Learning Data Annotation and Training.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Task: Install / Setup MedCATtrainer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Only one person on each host needs to do this...\n",
    "!git clone https://github.com/CogStack/MedCATtrainer.git\n",
    "!cd MedCATTrainer\n",
    "!export MCTRAINER_PORT=8005   # Optionally set a port for the trainer, default is 8001\n",
    "!docker-compose up"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## MedCATtrainer\n",
    "- Admin UI\n",
    " - Create projects\n",
    " - Upload Models etc.\n",
    "- User UI Walkthrough\n",
    " - Navigate documents\n",
    " - Annotate / validate documents"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Annotate Documents for T047,T048 semantic types\n",
    "Use the dataset exported from the previous notebook (data/train_data.csv). You can download to your local machine from jupyter by using the explorer view (http://localhost:8000/notebooks/tmp/medcat-cogstack-workshop)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once you've annoated some documents\n",
    "\n",
    "## Export the Annotations Using the Trainer API\n",
    "- Explore what's available in the donwloaded dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [],
   "source": [
    "URL = \"http://localhost:8001\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Authorization': 'Token affd97a49175d8ddd7da99216a8ed251506acf9c'}"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "payload = {\"username\": \"admin\", \"password\": \"admin\"}\n",
    "headers = {\n",
    "    'Authorization': f'Token {json.loads(requests.post(\"http://localhost:8001/api/api-token-auth/\", json=payload).text)[\"token\"]}',\n",
    "}\n",
    "headers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('Project Name:Example Annotation Project - UMLS (Diseases / Symptoms / Findings)',\n",
       "  'id:1'),\n",
       " ('Project Name:Cogstack Training Project', 'id:2')]"
      ]
     },
     "execution_count": 110,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[(f\"Project Name:{p['name']}\",f\"id:{p['id']}\") for p in json.loads(requests.get(f'{URL}/api/project-annotate-entities/', headers=headers).text)['results']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Fill in your project ids here. e.g:\n",
    "project_ids_to_download = [str(1),str(2)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [],
   "source": [
    "with_text = True\n",
    "download_url = f'{URL}/api/download-annos/?project_ids={ \",\".join(project_ids_to_download)}'\n",
    "if with_text:\n",
    "    download_url += '&with_text=True'\n",
    "projects = json.loads(requests.get(download_url, headers=headers).text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [],
   "source": [
    "# write output to disk\n",
    "username = '## YOUR USERNAME ##'\n",
    "username = 'searlt'\n",
    "json.dump(projects, open(f'data/MedCATtrainer_export_{username}.json', 'w'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Re-train MedCAT NER+L\n",
    "- instantiate MedCAT\n",
    "- train supevised using your annotations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [],
   "source": [
    "cdb_path = 'data/cdb-medmen.dat'\n",
    "vocab_path = 'data/vocab.dat'\n",
    "cdb = CDB()\n",
    "cdb.load_dict(cdb_path)\n",
    "vocab = Vocab()\n",
    "vocab.load_dict(vocab_path)\n",
    "cat = CAT(cdb, vocab)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cat.train_supervised(data_path=f\"data/MedCATtrainer_export_{username}.json\", \n",
    "                     nepochs=1,\n",
    "                     lr=0.1,\n",
    "                     anneal=False, # Unless we are reseting the CDB or cui_count this is False\n",
    "                     print_stats=True, \n",
    "                     use_filters=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run the re-trained model on the wider dataset\n",
    "- Re-use the code from the 3.0 Notebook to extract dieseases for each doc\n",
    "- How do the reuslts differ?\n",
    "\n",
    "Things that still might be of interst (not covered here)\n",
    "- Cross Validation - train/test splits\n",
    "- Plotting learning curves"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bonus: Train MetaCAT Models\n",
    "- The meta annotation modes you defined (Temporality / Status)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [],
   "source": [
    "from medcat.meta_cat import MetaCAT\n",
    "from tokenizers import ByteLevelBPETokenizer\n",
    "from itertools import chain\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# download pre-trained tokeniser models\n",
    "# meta anno model: tokeniser models trained via the github MedCAT tutorials\n",
    "# https://colab.research.google.com/drive/1rxzBZCTDcqsIjRXZ3u4yRZFOkUCCuwyy\n",
    "!wget https://zkcl.s3-eu-west-1.amazonaws.com/embeddings.npy -P ./data/\n",
    "!wget https://zkcl.s3-eu-west-1.amazonaws.com/medmen-merges.txt -P ./data/\n",
    "!wget https://zkcl.s3-eu-west-1.amazonaws.com/medmen-vocab.json -P ./data/\n",
    "\n",
    "# Get MedCAT models\n",
    "!wget https://s3-eu-west-1.amazonaws.com/zkcl/vocab.dat -P ./data/\n",
    "!wget https://s3-eu-west-1.amazonaws.com/zkcl/cdb-medmen.dat -P ./data/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Tokenizer instantiation\n",
    "tokenizer = ByteLevelBPETokenizer(vocab_file='data/medmen-vocab.json', merges_file='data/medmen-merges.txt')\n",
    "embeddings = np.load(open('data/embeddings.npy', 'rb'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [],
   "source": [
    "metacat = MetaCAT(tokenizer=tokenizer, embeddings=embeddings, \n",
    "                  pad_id=len(embeddings) -1, save_dir='mc_status', device='cpu')\n",
    "\n",
    "proj = json.load(open(f'data/MedCATtrainer_export_{username}.json'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "set()"
      ]
     },
     "execution_count": 131,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Tasks we are training for...\n",
    "set(chain.from_iterable([a['meta_anns'].keys() for d in proj['projects'][0]['documents'] for a in d['annotations']]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pick one of the above tasks \n",
    "metacat.train(f'data/MedCATtrainer_export_{username}.json', 'Status', nepochs=5)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}