cityscapes_dataset.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exploring the Cityscapes dataset\n",
    "\n",
    "In this Jupyter Notebook we explore the Cityscapes dataset making use of the dataloader [cityscapesDataset](./datasets/cityscapes.py)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install cityscapesscripts;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports\n",
    "We make use of the [cityscapesscripts package](https://github.com/mcordts/cityscapesScripts) to download the files. It is necessary to provide a *username* and *password* to download the dataset. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import torch\n",
    "import zipfile\n",
    "import numpy as np\n",
    "\n",
    "from importlib import reload\n",
    "from os.path import join as pjoin\n",
    "from torchvision import transforms\n",
    "from cityscapesscripts.download import downloader \n",
    "from datasets.cityscapes import cityscapesDataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "local_path = \"./Cityscapes\"\n",
    "pckgs_names = [\"gtFine_trainvaltest.zip\",\"leftImg8bit_trainvaltest.zip\"]\n",
    "dir_names = [\"gtFine\", \"leftImg8bit\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download the dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to obtain the necessary credentials, visit the [Cityscapes dataset webpage](https://www.cityscapes-dataset.com/register/) and register. Once you have registered, execute the next cell to log in."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "session = downloader.login()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Download zip files\n",
    "for pckg_name in pckgs_names:\n",
    "    if not os.path.isfile(pckg_name):\n",
    "        session = downloader.login()\n",
    "        downloader.download_packages(session=session, package_names=[pckg_name], destination_path=\"./\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Extract zip files \n",
    "for i, dir_name in enumerate(dir_names):\n",
    "    if not os.path.isdir(pjoin(local_path, dir_name)):\n",
    "        with zipfile.ZipFile(pckgs_names[i], \"r\") as zip_file:\n",
    "            zip_file.extractall(local_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Processing 5000 annotation files\n",
      "Progress: 100.0 %    Annotations files processed\n",
      "Annotations files processed\n"
     ]
    }
   ],
   "source": [
    "training_data = cityscapesDataset(local_path, split=\"train\")\n",
    "validation_data = cityscapesDataset(local_path, split=\"val\")\n",
    "test_data = cityscapesDataset(local_path, split=\"test\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Explore the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset_sizes = {\n",
    "    'train': len(training_data),\n",
    "    'val': len(validation_data),\n",
    "    'test': len(test_data),\n",
    "    'total': len(training_data) + len(validation_data) + len(test_data)\n",
    "}\n",
    "\n",
    "dataset_sizes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Both the image and its label are resized by default from (2048, 1024) to (512, 256)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "print(\"Image size = \", training_data[0][0].size())\n",
    "print(\"Label size = \", training_data[0][1].size())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "_, label = training_data[5]\n",
    "img, _ = training_data[5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "transforms.ToPILImage()(img)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "training_data.decode_segmap(label.numpy(), plot=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Training label ids \n",
    "Labels **-1** and **255** shall be ignored."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "training_data.label_ids()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check that resizing the label mask has not created invalid label ids"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "valid_labels = set(training_data.label_ids())\n",
    "all_valid = True\n",
    "for i in range(50):\n",
    "    j = np.random.randint(0, len(training_data))\n",
    "    _, lbl = training_data[j]\n",
    "    if not set(np.unique(lbl.numpy())).issubset(valid_labels):\n",
    "        print(\"INVALID LABEL !! > \", np.unique(lbl.numpy()))\n",
    "        all_valid = False\n",
    "        \n",
    "if all_valid:\n",
    "    print(\"ALL VALID :D !!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Training label colours"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "training_data.label_colours()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Training label names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "training_data.label_names()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Mean and standard deviation of the training images"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The images are normalized with the mean and the standard deviation of the images that form the training dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "training_data.tf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "t_mean = []\n",
    "t_std0 = []\n",
    "\n",
    "for img, lbl in training_data:\n",
    "    # shape (3, weight, height)\n",
    "    numpy_image = img.numpy()\n",
    "    \n",
    "    # shape (3,)\n",
    "    img_mean = np.mean(numpy_image, axis=(1,2)) \n",
    "    img_std0 = np.std(numpy_image, axis=(1,2))\n",
    "    \n",
    "    t_mean.append(img_mean)\n",
    "    t_std0.append(img_std0)\n",
    "\n",
    "# shape (N, 3) -> (mean across 0th axis) -> shape (3,)\n",
    "t_mean = np.array(t_mean).mean(axis=0)\n",
    "t_std0 = np.array(t_std0).mean(axis=0)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Mean = \", t_mean)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "print(\"Std. deviation (df = N) = \", t_std0)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}