Pythonで視覚化①_r1.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "collapsed_sections": [
        "gxBzFXzgxEha"
      ]
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "##**Pythonで視覚化①：Occupancy_detection :binary**\n",
        "Pythonで視覚化① で使用するデータセットは、[UCI Machine Learning Repository](https://archive.ics.uci.edu/ml/index.php) で公開されている [Occupancy_detection] です。\n",
        "\n",
        "###**このデータセットについて**\n",
        "[[Occupancy_detection]データセット](http://archive.ics.uci.edu/ml/datasets/Occupancy+Detection+) は、1分間隔で記録「date(日時分)」された、ある部屋の「温度(℃)、相対湿度(％)、明るさ(Lux)、CO2(ppm)、湿度比(kgwater-vapor/kg-air)」と「その時点での人の存在Ocuupancy」のデータで構成されています。\n",
        "目的変数である「人の存在[Ocuupancy]」は yes/no の二値、データは「date(日時分)」に沿った時系列データとなっており、「人の存在によって部屋の状況はどのように変化するか」や「部屋の状況で人の存在が予測できるか」を意図したものとなっています。\n",
        "\n",
        "▼データセットのソースには、1分ごとに撮影したタイムスタンプ付きの写真で人の存在を確認とあります。\n",
        "\n",
        "###**手順**\n",
        "- 「1．インストール」の「▷」をクリックしてください。（インストールが実行されます）。\n",
        "- Select_Datasetのドロップダウンメニュー(dataset:)で[Occupancy_detection :binary]を選択してください。\n",
        "- Load dataset の「▷」をクリックしてください。（データセットが読み込まれます）。"
      ],
      "metadata": {
        "id": "qqY9wmsmSe10"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **1．インストール**"
      ],
      "metadata": {
        "id": "gxBzFXzgxEha"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install seaborn-analyzer"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rCOw3doQ1kMp",
        "outputId": "28d2005f-0e37-4a43-be32-3fcbd74d4a91"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Requirement already satisfied: seaborn-analyzer in /usr/local/lib/python3.7/dist-packages (0.2.13)\n",
            "Requirement already satisfied: seaborn>=0.11.0 in /usr/local/lib/python3.7/dist-packages (from seaborn-analyzer) (0.11.2)\n",
            "Requirement already satisfied: matplotlib>=3.1.3 in /usr/local/lib/python3.7/dist-packages (from seaborn-analyzer) (3.2.2)\n",
            "Requirement already satisfied: lightgbm>=3.3.2 in /usr/local/lib/python3.7/dist-packages (from seaborn-analyzer) (3.3.3)\n",
            "Requirement already satisfied: numpy>=1.20.3 in /usr/local/lib/python3.7/dist-packages (from seaborn-analyzer) (1.21.6)\n",
            "Requirement already satisfied: scipy>=1.6.3 in /usr/local/lib/python3.7/dist-packages (from seaborn-analyzer) (1.7.3)\n",
            "Requirement already satisfied: pandas>=1.2.4 in /usr/local/lib/python3.7/dist-packages (from seaborn-analyzer) (1.3.5)\n",
            "Requirement already satisfied: scikit-learn>=0.24.2 in /usr/local/lib/python3.7/dist-packages (from seaborn-analyzer) (1.0.2)\n",
            "Requirement already satisfied: wheel in /usr/local/lib/python3.7/dist-packages (from lightgbm>=3.3.2->seaborn-analyzer) (0.38.4)\n",
            "Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.1.3->seaborn-analyzer) (2.8.2)\n",
            "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.1.3->seaborn-analyzer) (0.11.0)\n",
            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.1.3->seaborn-analyzer) (1.4.4)\n",
            "Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=3.1.3->seaborn-analyzer) (3.0.9)\n",
            "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from kiwisolver>=1.0.1->matplotlib>=3.1.3->seaborn-analyzer) (4.1.1)\n",
            "Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=1.2.4->seaborn-analyzer) (2022.6)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib>=3.1.3->seaborn-analyzer) (1.15.0)\n",
            "Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.24.2->seaborn-analyzer) (1.2.0)\n",
            "Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.24.2->seaborn-analyzer) (3.1.0)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install pandas-bokeh"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "huIyUa6entHZ",
        "outputId": "39881522-872a-4889-e884-23cb84faca45"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Requirement already satisfied: pandas-bokeh in /usr/local/lib/python3.7/dist-packages (0.5.5)\n",
            "Requirement already satisfied: bokeh>=2.0 in /usr/local/lib/python3.7/dist-packages (from pandas-bokeh) (2.3.3)\n",
            "Requirement already satisfied: pandas>=0.22.0 in /usr/local/lib/python3.7/dist-packages (from pandas-bokeh) (1.3.5)\n",
            "Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from bokeh>=2.0->pandas-bokeh) (2.8.2)\n",
            "Requirement already satisfied: numpy>=1.11.3 in /usr/local/lib/python3.7/dist-packages (from bokeh>=2.0->pandas-bokeh) (1.21.6)\n",
            "Requirement already satisfied: Jinja2>=2.9 in /usr/local/lib/python3.7/dist-packages (from bokeh>=2.0->pandas-bokeh) (2.11.3)\n",
            "Requirement already satisfied: tornado>=5.1 in /usr/local/lib/python3.7/dist-packages (from bokeh>=2.0->pandas-bokeh) (6.0.4)\n",
            "Requirement already satisfied: typing-extensions>=3.7.4 in /usr/local/lib/python3.7/dist-packages (from bokeh>=2.0->pandas-bokeh) (4.1.1)\n",
            "Requirement already satisfied: packaging>=16.8 in /usr/local/lib/python3.7/dist-packages (from bokeh>=2.0->pandas-bokeh) (21.3)\n",
            "Requirement already satisfied: PyYAML>=3.10 in /usr/local/lib/python3.7/dist-packages (from bokeh>=2.0->pandas-bokeh) (6.0)\n",
            "Requirement already satisfied: pillow>=7.1.0 in /usr/local/lib/python3.7/dist-packages (from bokeh>=2.0->pandas-bokeh) (7.1.2)\n",
            "Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from Jinja2>=2.9->bokeh>=2.0->pandas-bokeh) (2.0.1)\n",
            "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=16.8->bokeh>=2.0->pandas-bokeh) (3.0.9)\n",
            "Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=0.22.0->pandas-bokeh) (2022.6)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->bokeh>=2.0->pandas-bokeh) (1.15.0)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install scikit-learn==1.1.3"
      ],
      "metadata": {
        "id": "IjqMJB--t1Bq"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **2．データセット読込み**"
      ],
      "metadata": {
        "id": "7avNQxplxTWh"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@title Select_Dataset { run: \"auto\" }\n",
        "#@markdown  **<font color= \"Crimson\">注意</font>：かならず 実行する前に 設定してください。**</font>\n",
        "\n",
        "dataset = 'Occupancy_detection :binary' #@param ['Boston_housing :regression', 'Diabetes :regression', 'Breast_cancer :binary','Titanic :binary', 'Titanic(seaborn) :binary', 'Iris :classification', 'Loan_prediction :binary','wine :classification', 'Occupancy_detection :binary', 'Upload']"
      ],
      "metadata": {
        "id": "kYtA-M52ZFGQ",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#@title Load dataset\n",
        "\n",
        "#ライブラリインポート\n",
        "#import numpy as np    #数値計算ライブラリ\n",
        "import pandas as pd   #データを効率的に扱うライブラリ\n",
        "import seaborn as sns #視覚化ライブラリ\n",
        "import warnings       #警告を表示させないライブラリ\n",
        "warnings.simplefilter('ignore')\n",
        "\n",
        "'''\n",
        "dataset（ドロップダウンメニュー）で選択したデータセットを読込み、データフレーム（df）に格納。\n",
        "目的変数は、データフレームの最終列とし、FEATURES、TARGET、X、yを指定した後、データフレーム\n",
        "に関する情報と先頭5列を表示。\n",
        "任意のcsvデータを読込む場合は、datasetで'Upload'を選択。\n",
        "\n",
        "'''\n",
        "\n",
        "#任意のcsvデータ読込み及びデータフレーム格納、\n",
        "if dataset =='Upload':\n",
        "  from google.colab import files\n",
        "  uploaded = files.upload()#Upload\n",
        "  target = list(uploaded.keys())[0]\n",
        "  df = pd.read_csv(target)\n",
        "\n",
        "#Diabetes データセットの読込み及びデータフレーム格納、\n",
        "elif dataset == \"Diabetes :regression\":\n",
        "  from sklearn.datasets import load_diabetes\n",
        "  diabetes = load_diabetes()\n",
        "  df = pd.DataFrame(diabetes.data, columns = diabetes.feature_names)\n",
        "  df['target'] = diabetes.target\n",
        "\n",
        "#Breast_cancer データセットの読込み及びデータフレーム格納、\n",
        "elif dataset == \"Breast_cancer :binary\":\n",
        "  from sklearn.datasets import load_breast_cancer\n",
        "  breast_cancer = load_breast_cancer()\n",
        "  df = pd.DataFrame(breast_cancer.data, columns = breast_cancer.feature_names)\n",
        "  #df['target'] = breast_cancer.target  #目的変数をカテゴリー数値とする時\n",
        "  df['target'] = breast_cancer.target_names[breast_cancer.target]\n",
        "\n",
        "#Titanic データセットの読込み及びデータフレーム格納、\n",
        "elif dataset == \"Titanic :binary\":\n",
        "  data_url = \"https://raw.githubusercontent.com/datasciencedojo/datasets/master/titanic.csv\"\n",
        "  df = pd.read_csv(data_url)\n",
        "  #目的変数 Survived をデータフレーム最終列に移動\n",
        "  X = df.drop(['Survived'], axis=1)\n",
        "  y = df['Survived'] \n",
        "  df = pd.concat([X, y], axis=1)    #X,yを結合し、dfに格納\n",
        "\n",
        "#Titanic(seaborn) データセットの読込み及びデータフレーム格納、\n",
        "elif dataset == \"Titanic(seaborn) :binary\":\n",
        "  df = sns.load_dataset('titanic')\n",
        "  #重複データをカットし、目的変数 alive をデータフレーム最終列に移動\n",
        "  X = df.drop(['survived','pclass','embarked','who','adult_male','alive'], axis=1)\n",
        "  y = df['alive']                   #目的変数データ\n",
        "  df = pd.concat([X, y], axis=1)    #X,yを結合し、dfに格納\n",
        "\n",
        "#iris データセットの読込み及びデータフレーム格納、\n",
        "elif dataset == \"Iris :classification\":\n",
        "  from sklearn.datasets import load_iris\n",
        "  iris = load_iris()\n",
        "  df = pd.DataFrame(iris.data, columns = iris.feature_names)\n",
        "  #df['target'] = iris.target  #目的変数をカテゴリー数値とする時\n",
        "  df['target'] = iris.target_names[iris.target]\n",
        "\n",
        "#wine データセットの読込み及びデータフレーム格納、\n",
        "elif dataset == \"wine :classification\":\n",
        "  from sklearn.datasets import load_wine\n",
        "  wine = load_wine()\n",
        "  df = pd.DataFrame(wine.data, columns = wine.feature_names)\n",
        "  #df['target'] = wine.target  #目的変数をカテゴリー数値とする時\n",
        "  df['target'] = wine.target_names[wine.target]\n",
        "\n",
        "#Loan_prediction データセットの読込み及びデータフレーム格納、 \n",
        "elif dataset == \"Loan_prediction :binary\":\n",
        "  data_url = \"https://github.com/shrikant-temburwar/Loan-Prediction-Dataset/raw/master/train.csv\"\n",
        "  df = pd.read_csv(data_url)\n",
        "\n",
        "#Occupancy_detection データセットの読込み及びデータフレーム格納、 \n",
        "elif dataset =='Occupancy_detection :binary':\n",
        "  data_url = 'https://raw.githubusercontent.com/hima2b4/Auto_Profiling/main/Occupancy-detection-datatest.csv'\n",
        "  df = pd.read_csv(data_url)\n",
        "  df['date'] = pd.to_datetime(df['date'])    #[date]のデータ型をdatetime型に変更\n",
        "\n",
        "#Boston データセットの読込み及びデータフレーム格納 \n",
        "else:\n",
        "  from sklearn.datasets import load_boston\n",
        "  boston = load_boston()\n",
        "  df = pd.DataFrame(boston.data, columns = boston.feature_names)\n",
        "  df['target'] = boston.target\n",
        "\n",
        "#FEATURES、TARGET、X、yを指定 \n",
        "FEATURES = df.columns[:-1]    #説明変数のデータ項目を指定\n",
        "TARGET = df.columns[-1]       #目的変数のデータ項目を指定\n",
        "X = df.loc[:, FEATURES]       #FEATURESのすべてのデータをXに格納\n",
        "y = df.loc[:, TARGET]         #TARGETのすべてのデータをyに格納\n",
        "\n",
        "#データフレーム表示\n",
        "df.info(verbose=True)         #データフレーム情報表示（verbose=Trueで表示数制限カット）\n",
        "df.head()                     #データフレーム先頭5行表示"
      ],
      "metadata": {
        "id": "rwv_xazpmwbC",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **Pairplot**"
      ],
      "metadata": {
        "id": "DotwnMt-4jyO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@title Pairplot_classification\n",
        "\n",
        "from seaborn_analyzer import CustomPairPlot\n",
        "\n",
        "cp = CustomPairPlot()\n",
        "cp.pairanalyzer(df, hue=TARGET)\n"
      ],
      "metadata": {
        "id": "wHdrIA653faq",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#@title **データ項目一覧**\n",
        "#@markdown **※データ項目一覧を表示します。以後のデータ項目の入力は、表示された項目をコピーアンドペーストすると確実です。**\n",
        "print('データ項目名：',df.columns.values)"
      ],
      "metadata": {
        "cellView": "form",
        "id": "u-Jcwn6pdCuM"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **3．視覚化**"
      ],
      "metadata": {
        "id": "XCbx_iaHgEiy"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **ヒストグラム**"
      ],
      "metadata": {
        "id": "5122AEazrJ4T"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@title　Histgram_pandas(matplotlib)\n",
        "\n",
        "Column_name =  'xxx'#@param {type:\"raw\"}\n",
        "bins_number_slider = 12 #@param {type:\"slider\", min:5, max:20, step:1}\n",
        "\n",
        "import math\n",
        "print('------------------------------------------------------------')\n",
        "print('階級数（スタージェスの公式）：',round(1 + math.log2(len(df))))\n",
        "print('------------------------------------------------------------')\n",
        "\n",
        "#df[Column_name].hist(bins=bins_number_slider);\n",
        "df.hist(Column_name, bins=bins_number_slider);\n",
        "#df[Column_name].plot.hist();"
      ],
      "metadata": {
        "id": "Glbab33-qe1G",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#@title　Histogram for each target variable_pandas(matplotlib)\n",
        "\n",
        "Column_name =  'xxx'#@param {type:\"raw\"}\n",
        "Category_column_name =  'xxx'#@param {type:\"raw\"}\n",
        "\n",
        "#df[Column_name].hist(by=df[Category_column_name],sharex=True, sharey=True);\n",
        "df.hist(Column_name,by=df[Category_column_name],sharex=True, sharey=True);"
      ],
      "metadata": {
        "id": "DIrkhVJdrGN9",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#@title Histogram for each target variable_plotnine\n",
        "Column_name =  'xxx'#@param {type:\"raw\"}\n",
        "Category_column_name =  'xxx'#@param {type:\"raw\"}\n",
        "\n",
        "from plotnine import *\n",
        "\n",
        "(ggplot(df, aes(x = Column_name, fill = Category_column_name))\n",
        " + geom_histogram()\n",
        " + facet_wrap(Category_column_name))"
      ],
      "metadata": {
        "id": "FxIRCFMv1loZ",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#@title Stratified histogram by category_plotnine\n",
        "\n",
        "Column_name =  'xxx'#@param {type:\"raw\"}\n",
        "Category_column_name =  'xxx'#@param {type:\"raw\"}\n",
        "bins_number_slider = 15 #@param {type:\"slider\", min:5, max:20, step:1}\n",
        "\n",
        "from plotnine import *\n",
        "\n",
        "(ggplot(df, aes(Column_name, fill = Category_column_name)) \n",
        " + geom_histogram(bins = bins_number_slider, position = \"identity\", alpha = 0.7)\n",
        ") "
      ],
      "metadata": {
        "id": "kRPbNKP51N4J",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **折れ線グラフ**"
      ],
      "metadata": {
        "id": "XDNzQwJbgA7k"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@title Line-plot_pandas(matplotlib)\n",
        "\n",
        "X_column_name =  'xxx'#@param {type:\"raw\"}\n",
        "\n",
        "#df.plot(x = X_column_name);\n",
        "df.plot(kind='line', x = X_column_name);"
      ],
      "metadata": {
        "id": "yJPQWyfPsJqV",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#@title Line-plot(subplot)_pandas(matplotlib)\n",
        "\n",
        "X_column_name =  'xxx'#@param {type:\"raw\"}\n",
        "\n",
        "df.plot(kind='line',x= X_column_name, subplots= True, \n",
        "        #figsize= (6,6),\n",
        "        #title='pandas subplots',\n",
        "        #grid=True,\n",
        "        #colormap='Accent',\n",
        "        #alpha=0.5\n",
        "        );"
      ],
      "metadata": {
        "id": "Dx3y6PJmlCT5",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#@title Line-plot with rangetool_pandas-bokeh\n",
        "\n",
        "X_column_name =  'xxx'#@param {type:\"raw\"}\n",
        "\n",
        "import pandas_bokeh\n",
        "pandas_bokeh.output_notebook()\n",
        "\n",
        "df.plot_bokeh(x = X_column_name,rangetool=True)"
      ],
      "metadata": {
        "id": "4vgnbFntk__G",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}