CS-562-Proj-EMF-output-V13-Noted-CorrectionAfterPresentation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <font color=red>Note</font>\n",
    "- Function\n",
    "    - compute count aggregation function for string attributes (to get how many records)\n",
    "        - So, this code works for aggregation function like <font color=blue> \"x_count_prod\"</font>\n",
    "    - compute avg, sum, count for numeric attributes \n",
    "- For indexing part, it is based on the [C] rather than [V]. \n",
    "    - <font color=red> when swap the \"prod\" and \"cust\", the [C] part should also indexable </font>\n",
    "    - Therefore the MF_map construction has been changed and fix the bug "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## sample 7_2 swap error explaination\n",
    "SELECT ATTRIBUTE(S):\n",
    "\n",
    "<font color=red>prod, cust, </font>x_sum_quant,  <font color=blue>x_count_prod,</font>  x_avg_quant+1, y_avg_quant/2 ,x_avg_quant/y_avg_quant\n",
    "\n",
    "NUMBER OF GROUPING VARIABLES(n):\n",
    "\n",
    "x,y\n",
    "\n",
    "GROUPING ATTRIBUTES(V):\n",
    "\n",
    "prod, cust\n",
    "\n",
    "F-VECT([F]):\n",
    "\n",
    "x_avg_quant, y_avg_quant, x_count_prod, x_sum_quant\n",
    "\n",
    "SELECT CONDITION-VECT([C]):\n",
    "\n",
    "<font color=red>x.cust=cust and x.prod=prod </font> <font color = green>(not consistent with S will make index error)</font>\n",
    "\n",
    "y.cust=cust and y.prod<>prod  \n",
    "\n",
    "HAVING_CONDITION(G):\n",
    "\n",
    "y_avg_quant<>0 and x_avg_quant>=0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## sample 7_2  This will give right result with swap [C] as well\n",
    "SELECT ATTRIBUTE(S):\n",
    "\n",
    "<font color = green>prod, cust,</font> x_sum_quant, x_count_prod,  x_avg_quant+1, y_avg_quant/2 ,x_avg_quant/y_avg_quant\n",
    "\n",
    "NUMBER OF GROUPING VARIABLES(n):\n",
    "\n",
    "x,y\n",
    "\n",
    "GROUPING ATTRIBUTES(V):\n",
    "\n",
    "prod, cust\n",
    "\n",
    "F-VECT([F]):\n",
    "\n",
    "x_avg_quant, y_avg_quant, x_count_prod, x_sum_quant\n",
    "\n",
    "SELECT CONDITION-VECT([C]):\n",
    "\n",
    "<font color = green> x.prod=prod and x.cust=cust</font>\n",
    "\n",
    "<font color = green> y.prod<>prod and y.cust=cust</font>\n",
    "\n",
    "HAVING_CONDITION(G):\n",
    "\n",
    "y_avg_quant<>0 and x_avg_quant>=0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color=red> Correction the code after presentation</font>\n",
    "- change the MF_MAP index building method \n",
    "- Add the reverse combination indexable as well\n",
    "      t += \" + \".join(['str(tmprec' + str([j]) + \")\" for j in list(idx_combination)[::-1]]) \n",
    "      t += \" + \".join(['str(tmprec' + str([j]) + \")\" for j in list(idx_combination)]) \n",
    "\n",
    "- Result: those two combination will be indexable as well\n",
    "      MFMap[str(tmprec['prod']) + str(tmprec['cust'])]=MF_idx\n",
    "      MFMap[str(tmprec['cust']) + str(tmprec['prod'])]=MF_idx\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## sample 1 My code can handle this. \n",
    "- the result in cols 3_avg_quant, 3_sum_quant are the same because <font color = red>  there is only one record for gv = 3</font>\n",
    "\n",
    "SELECT ATTRIBUTE(S):\n",
    "\n",
    "cust,    1_avg_quant, 2_avg_quant,   3_avg_quant, 3_sum_quant\n",
    "\n",
    "NUMBER OF GROUPING VARIABLES(n):\n",
    "\n",
    "1,2,3\n",
    "\n",
    "GROUPING ATTRIBUTES(V):\n",
    "\n",
    "cust\n",
    "\n",
    "F-VECT([F]):\n",
    "\n",
    "1_sum_quant,    1_avg_quant, 2_avg_quant, 3_sum_quant, 3_avg_quant, 3_sum_quant\n",
    "\n",
    "SELECT CONDITION-VECT([C]):\n",
    "\n",
    "1.cust=cust     and 1.state='NY' and 1.year=2005\n",
    "\n",
    "2.cust=cust and      2.state='CT' and 2.year=2005\n",
    "\n",
    "3.cust=cust and 3.state='NJ' and 3.year=2005\n",
    "\n",
    "HAVING_CONDITION(G):\n",
    "\n",
    "1_avg_quant > 2_avg_quant and 1_avg_quant >= 3_avg_quant"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-04-28T16:47:13.054478Z",
     "start_time": "2020-04-28T16:47:13.051477Z"
    }
   },
   "source": [
    "# main Function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:38.421446Z",
     "start_time": "2020-05-09T02:16:38.417023Z"
    },
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [],
   "source": [
    "from collections import defaultdict\n",
    "import psycopg2\n",
    "from emf2py import emf_to_py\n",
    "from utils import *\n",
    "import argparse\n",
    "import re\n",
    "import sys\n",
    "# 初始化数据库 默认读取参数\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set input file in there"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output_num = '7_2'\n",
    "\n",
    "input_file=\"input\"\n",
    "filename = \"output\"\n",
    "input_path = \"E:/OneDrive - stevens.edu/Stevens BIA/CS562/Proj/EMF2PY/Input_files/\"\n",
    "input_file = \"input%s.txt\"% output_num\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Connect DB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:38.851108Z",
     "start_time": "2020-05-09T02:16:38.778690Z"
    },
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "========================================\n",
      "In <<'sales'>> table\n",
      "========================================\n",
      "ATTRIBUTES     DTYPE          \n",
      "* cust         character varying\n",
      "* prod         character varying\n",
      "* day          integer        \n",
      "* month        integer        \n",
      "* year         integer        \n",
      "* state        character      \n",
      "* quant        integer        \n",
      "========================================\n"
     ]
    }
   ],
   "source": [
    "########################################\n",
    "# connect DB login  ==> get schema & log info\n",
    "########################################\n",
    "table = \"sales\"\n",
    "dbname = \"postgres\"\n",
    "host = \"localhost\"\n",
    "user = \"postgres\"\n",
    "password = \"00000000\"\n",
    "port = \"5388\"\n",
    "\n",
    "try:\n",
    "\tconnection = psycopg2.connect(user=user, password=password, host=host, port=port, database=dbname)\n",
    "\tquery = \"SELECT column_name, data_type FROM information_schema.columns WHERE table_name = '\" + table + \"';\"\n",
    "\tcursor = connection.cursor()\n",
    "\tcursor.execute(query)\n",
    "\tif cursor.rowcount == 0:\n",
    "\t\tprint(\"Connected, but no data in table: %s \" % table)\n",
    "\t\tsys.exit(0)\n",
    "\tschema = cursor.fetchall()\n",
    "except(Exception, psycopg2.Error) as error:\n",
    "\tprint(\"Error connecting to PostgreSQL database ==>\", error)\n",
    "\tsys.exit(0)\n",
    "\n",
    "print(\"=\" * 40)\n",
    "print(\"In <<'%s'>> table\" % format(table))\n",
    "print(\"=\" * 40)\n",
    "print(\"ATTRIBUTES\".ljust(14, \" \"), \"DTYPE\".ljust(15, \" \"))\n",
    "for i in schema:\n",
    "\tprint(\"*\", i[0].ljust(12, \" \"), i[1].ljust(15, \" \"))\n",
    "print(\"=\" * 40)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Get schema & log info\n",
    "- ready to pass to emf2py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:39.131890Z",
     "start_time": "2020-05-09T02:16:39.128889Z"
    },
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [],
   "source": [
    "PG_log_info = {'user':user, 'password':password,'host':host, 'port':port, 'database':dbname}\n",
    "initial_list=['user', 'password', 'host', 'port', 'database']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Utils - Parsing input file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:39.500021Z",
     "start_time": "2020-05-09T02:16:39.488028Z"
    },
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [],
   "source": [
    "########################################\n",
    "### Parsing for Input txt\n",
    "########################################\n",
    "import re\n",
    "\n",
    "def txt_parsing(txt):\n",
    "    # build the map for input text\n",
    "    key_place = {}\n",
    "    keyword = [\"(S)\", \"(n)\", \"(V)\", \"([F])\", \"([C])\", \"(G)\"]\n",
    "    for i in range(len(keyword)):\n",
    "        for j in range(i, len(txt)):\n",
    "            if txt[j].find(keyword[i]) > 0:\n",
    "                key_place[keyword[i]] = j\n",
    "    # {'(S)': 0, '(n)': 2, '(V)': 4, '([F])': 6, '([C])': 8, '(G)': 12}\n",
    "    # print(key_place)\n",
    "    \n",
    "    # find the distance with next key word\n",
    "    params = key_place\n",
    "    tmp = list(key_place.keys())  # ['(S)', '(n)', '(V)', '([F])', '([σ])', '(G)']\n",
    "    for i in range(len(key_place) - 1):\n",
    "        begin = key_place[tmp[i]] + 1  # begin with '(S)': 0+1\n",
    "        end = key_place[tmp[i + 1]]  # end with '(n)': 2\n",
    "        #     print(key_place[tmp[i]]+1,key_place[tmp[i+1]])\n",
    "        params[keyword[i]] = txt[begin:end]  # find the text from begin to end\n",
    "    # print(txt[key_place[tmp[i+1]]+1:]) # the last one\n",
    "    params[keyword[i + 1]] = txt[key_place[tmp[i + 1]] + 1:]\n",
    "    \n",
    "    # params\n",
    "    # '''\n",
    "    # {'(S)': ['cust,1_avg_quant, 2_avg_quant,3_avg_quant'],\n",
    "    #  '(n)': ['1,2,3'],\n",
    "    #  '(V)': ['cust'],\n",
    "    #  '([F])': ['1_sum_quant, 1_avg_quant, 2_avg_quant, 3_sum_quant, 3_avg_quant'],\n",
    "    #  '([C])': [\"1.cust=cust and 1.state='NY' and 1.year=2005\", \"2.cust=cust and 2.state='CT' and 2.year=2005\", \"3.cust=cust and 3.state='NJ' and 3.year=2005\"],\n",
    "    #  '(G)': ['1_avg_quant > 2_avg_quant and 1_avg_quant > 3_avg_quant']}'''\n",
    "    \n",
    "    # handle several lines\n",
    "    S = [i.strip() for i in params['(S)'][0].split(',')]\n",
    "    n = [i.strip() for i in params['(n)'][0].split(',')]\n",
    "    V = [i.strip() for i in params['(V)'][0].split(',')]\n",
    "    F = [i.strip() for i in params['([F])'][0].split(',')]\n",
    "    C = params[\"([C])\"]\n",
    "    G = [i.strip() for i in params['(G)'][0].split(',')]\n",
    "    return S, n, V, F, C, G\n",
    "\n",
    "\n",
    "with open(input_path+input_file, 'r', encoding='utf') as f:\n",
    "    txt = f.read().strip().split('\\n')\n",
    "    \n",
    "S,n,V,F,C,G = txt_parsing(txt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:39.700443Z",
     "start_time": "2020-05-09T02:16:39.696502Z"
    },
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['prod',\n",
       " 'cust',\n",
       " 'x_sum_quant',\n",
       " 'x_count_prod',\n",
       " 'x_avg_quant+1',\n",
       " 'y_avg_quant/2',\n",
       " 'x_avg_quant/y_avg_quant']"
      ]
     },
     "execution_count": 112,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "S"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:39.909468Z",
     "start_time": "2020-05-09T02:16:39.905406Z"
    },
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['x', 'y']"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:40.148906Z",
     "start_time": "2020-05-09T02:16:40.145572Z"
    },
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['prod',\n",
       " 'cust',\n",
       " 'x_sum_quant',\n",
       " 'x_count_prod',\n",
       " 'x_avg_quant+1',\n",
       " 'y_avg_quant/2',\n",
       " 'x_avg_quant/y_avg_quant']"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "S"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:40.364889Z",
     "start_time": "2020-05-09T02:16:40.360889Z"
    },
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['x', 'y']"
      ]
     },
     "execution_count": 115,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:40.596038Z",
     "start_time": "2020-05-09T02:16:40.593037Z"
    },
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['prod', 'cust']"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "V"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:41.096770Z",
     "start_time": "2020-05-09T02:16:41.093516Z"
    },
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['x_avg_quant', 'y_avg_quant', 'x_count_prod', 'x_sum_quant']"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "F"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:41.132106Z",
     "start_time": "2020-05-09T02:16:41.128105Z"
    },
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['x.prod=prod and x.cust=cust', 'y.prod<>prod and y.cust=cust']"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "C"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:41.168083Z",
     "start_time": "2020-05-09T02:16:41.164097Z"
    },
    "pycharm": {
     "is_executing": false
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['y_avg_quant<>0 and x_avg_quant>=0']"
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "G"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# EMF Processing "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:41.241276Z",
     "start_time": "2020-05-09T02:16:41.239275Z"
    },
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [],
   "source": [
    "from itertools import combinations_with_replacement as cmb"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-08T21:14:45.698113Z",
     "start_time": "2020-05-08T21:14:45.693112Z"
    }
   },
   "source": [
    "## initialization For variables\n",
    "- T_cols = ['cust', 'prod', 'day', 'month', 'year', 'state', 'quant']\n",
    "- V_cols = ['cust', 'prod']\n",
    "- MF_cols = V+F+(other agg function - avg extend to sum count) \n",
    "    - = ['cust', 'prod', 'x_count_quant', 'x_sum_quant', 'x_avg_quant', 'y_count_quant', 'y_sum_quant', 'y_avg_quant', 'x_count_prod', 'x_sum_quant', 'y_count_prod']\n",
    "    - <font color=red>For each gv and attrs related to agg</font>\n",
    "    \n",
    "- MF_dtypes = For retrieve type, Save all combination of (gv + agg + attributes). But dont need to add into MF_cols\n",
    "- C_cols = ['1.prod=prod and 1.month=month and 1.year=1997', '2.prod=prod and 2.year=1997'] # condition "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:41.340710Z",
     "start_time": "2020-05-09T02:16:41.329706Z"
    }
   },
   "outputs": [],
   "source": [
    "########################################\n",
    "### initialization\n",
    "########################################\n",
    "# T_cols = ['cust', 'prod', 'day', 'month', 'year', 'state', 'quant']\n",
    "T_cols = [i[0] for i in schema]\n",
    "\n",
    "## three parsing mapping\n",
    "sql_dtypes_maps = {\"character varying\": \"''\", \"character\": \"''\", \"integer\": 0}\n",
    "logic_map = {\">=\": \" >= \", \"<=\": \" <= \", \"=\": \" == \", \"<>\": ' != ', \">\": \" > \", \"<\": \" < \"}  # for insert space ans subtitute logic symbol in order. Only execute once in loop\n",
    "format_length_map = {\"''\": \"{:<12}\", 0: \"{:>15}\", 0.0: \"{:>15}\"}\n",
    "\n",
    "\n",
    "V_cols = [i.strip() for i in V]  # grouping variable = ['cust', 'prod']\n",
    "\n",
    "# MF_dtypes = {'cust': \"''\", 'prod': \"''\", 'day': 0, 'month': 0, 'year': 0, 'state': \"''\", 'quant': 0, '_1_avg_quant': 0.0, '_1_count_quant': 0, '_1_sum_quant': 0.0, '_1_max_quant': 0.0, '_1_min_quant': 0.0, '_2_avg_quant': 0.0, '_2_count_quant': 0, '_2_sum_quant': 0.0, '_2_max_quant': 0.0, '_2_min_quant': 0.0}\n",
    "# MF_cols = ['prod', 'month', '_1_avg_quant', '_1_count_quant', '_1_sum_quant', '_1_max_quant', '_1_min_quant', '_2_avg_quant', '_2_count_quant', '_2_sum_quant', '_2_max_quant', '_2_min_quant'... month etc ]\n",
    "MF_cols = V[:] # Grouping variables\n",
    "for agg_col in F[:]: # for agg function. build MF_Vector\n",
    "    tmp_mf = []\n",
    "    if \"avg\" in agg_col:\n",
    "        tmp_mf.append(agg_col.replace('avg', 'count'))\n",
    "        tmp_mf.append(agg_col.replace('avg', 'sum'))\n",
    "        tmp_mf.append(agg_col.replace('avg', 'avg'))\n",
    "    else:\n",
    "        tmp_mf = [agg_col]\n",
    "    MF_cols.extend(tmp_mf)\n",
    "    \n",
    "\n",
    "########## save dtype for retrieve type But dont need to add into MF_cols\n",
    "MF_dtypes = {i[0]: sql_dtypes_maps[i[1]] for i in schema}\n",
    "for i in schema:  # ('prod', 'character varying'),...]\n",
    "    if type(sql_dtypes_maps[i[1]]) == str:  # if it is string\n",
    "        for gv in n:\n",
    "            Agg_F = [\"{}_count_{}\".format(gv, i[0])]\n",
    "            #             MF_cols.extend(Agg_F)\n",
    "            MF_dtypes.update({i: j for i, j in zip(Agg_F, [0])})\n",
    "    else:  # int\n",
    "        for gv in n:\n",
    "            Agg_F = (\"{}_avg_{},{}_count_{},{}_sum_{},{}_max_{},{}_min_{}\".format(gv, i[0], gv, i[0], gv, i[0], gv, i[0],gv, i[0])).split(',')\n",
    "            #   MF_cols.extend(Agg_F) # There only focus on quant. if other than quant, there should be more\n",
    "            MF_dtypes.update({i: j for i, j in zip(Agg_F, [0.0, 0, 0.0, 0.0, 0.0])})\n",
    "########## \n",
    "\n",
    "\n",
    "C_cols = [row.strip().split(\"and\") for row in C if len(row) > 2]\n",
    "# ['1.prod=prod and 1.month=month and 1.year=1997', '2.prod=prod and 2.year=1997']\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:41.376195Z",
     "start_time": "2020-05-09T02:16:41.372194Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['prod', 'cust', 'x_count_quant', 'x_sum_quant', 'x_avg_quant', 'y_count_quant', 'y_sum_quant', 'y_avg_quant', 'x_count_prod', 'x_sum_quant']\n"
     ]
    }
   ],
   "source": [
    "print(MF_cols)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Connection_initialize "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:41.755316Z",
     "start_time": "2020-05-09T02:16:41.749315Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "try:\n",
      "    connection = psycopg2.connect(user='postgres', password='00000000', host='localhost', port='5388', database='postgres', )\n",
      "    query = \"SELECT * FROM sales\"\n",
      "    cursor = connection.cursor()\n",
      "    cursor.execute(query)\n",
      "    if cursor.rowcount == 0:\n",
      "        print(\"Connected, but no data!\")\n",
      "        sys.exit(0)\n",
      "                    \n",
      "    records = cursor.fetchall()\n",
      "except(Exception, psycopg2.Error) as error:\n",
      "    print(\"Error connecting to PostgreSQL database ==>\", error)\n",
      "    sys.exit(0)\n",
      "\n",
      "                    \n"
     ]
    }
   ],
   "source": [
    "########################################\n",
    "### Connection_initialize\n",
    "########################################\n",
    "import_text = 'import psycopg2\\nimport sys\\nfrom collections import defaultdict'\n",
    "initial_list = list(PG_log_info.keys())\n",
    "connection_text = '\\ntry:\\n    connection = psycopg2.connect('\n",
    "for i in initial_list:\n",
    "    connection_text += str(i) + \"='\" + str(PG_log_info[i]) + \"', \"\n",
    "connection_text += ''')\\n    query = \"SELECT * FROM sales\"\\n    cursor = connection.cursor()\\n    cursor.execute(query)\\n    if cursor.rowcount == 0:\\n        print(\"Connected, but no data!\")\\n        sys.exit(0)\n",
    "                    \\n    records = cursor.fetchall()\\nexcept(Exception, psycopg2.Error) as error:\\n    print(\"Error connecting to PostgreSQL database ==>\", error)\\n    sys.exit(0)\\n\n",
    "                    '''\n",
    "\n",
    "print(connection_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-08T21:18:16.286689Z",
     "start_time": "2020-05-08T21:18:16.284688Z"
    }
   },
   "source": [
    "## Initializing MF_Vector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:42.173194Z",
     "start_time": "2020-05-09T02:16:42.170193Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MFVector = defaultdict(lambda: {'prod':'', 'cust':'', 'x_count_quant':0, 'x_sum_quant':0.0, 'x_avg_quant':0.0, 'y_count_quant':0, 'y_sum_quant':0.0, 'y_avg_quant':0.0, 'x_count_prod':0, 'x_sum_quant':0.0})\n",
      "MF_idx=0\n",
      "MFMap = {}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "########################################\n",
    "# Initializing MF_Vector\n",
    "########################################\n",
    "MFVector_text = \"MFVector = defaultdict(lambda: {\"\n",
    "for col_name in MF_cols:\n",
    "    MFVector_text += \"'\" + col_name + \"'\" + ':' + str(MF_dtypes[col_name]) + ', '\n",
    "MFVector_text = MFVector_text[:-2] + '})' + '\\nMF_idx=0\\nMFMap = {}\\n'\n",
    "\n",
    "print(MFVector_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-08T21:18:25.406736Z",
     "start_time": "2020-05-08T21:18:25.404736Z"
    }
   },
   "source": [
    "## Initializing Map for MF_Structure"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:42.553237Z",
     "start_time": "2020-05-09T02:16:42.545236Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "for i in range(len(records)):\n",
      "\ttmprec = {x[0]:x[1] for x in zip(['cust', 'prod', 'day', 'month', 'year', 'state', 'quant'],records[i])}\n",
      "\tif (str(tmprec['prod']) + str(tmprec['cust'])) not in MFMap:\n",
      "\t\tMFVector[MF_idx]['prod']=tmprec['prod']\n",
      "\t\tMFVector[MF_idx]['cust']=tmprec['cust']\n",
      "\t\tMFMap[str(tmprec['prod'])]=MF_idx\n",
      "\t\tMFMap[str(tmprec['prod']) + str(tmprec['cust'])]=MF_idx\n",
      "\t\tMFMap[str(tmprec['cust']) + str(tmprec['prod'])]=MF_idx\n",
      "\t\tMFMap[str(tmprec['cust'])]=MF_idx\n",
      "\t\tMF_idx+=1 # new idx\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "########################################\n",
    "# Initializing Map for MF_Structure\n",
    "########################################\n",
    "scan_text = \"for i in range(len(records)):\\n\\ttmprec = {x[0]:x[1] for x in zip(\" + str(T_cols) + \",records[i])}\\n\"\n",
    "\n",
    "MFMap_initing = []\n",
    "MFMap_initing.append(\"\\tif (\" + \" + \".join(['str(tmprec' + str([i]) + \")\" for i in V_cols]) + \") not in MFMap:\\n\")\n",
    "MFMap_initing.append( \"\\t\\t\" + \"\\n\\t\\t\".join(['MFVector[MF_idx]' + str([i]) + '=tmprec' + str([i]) for i in V_cols]) + '\\n')\n",
    "\n",
    "for idx_combination in cmb(V_cols, len(V_cols)):  # ['cust', 'prod] ==>>> ['cust','cust],['prod','prod'], ['cust','prod']\n",
    "    t = '\\t\\tMFMap['\n",
    "    if len(set(idx_combination)) > 1:  # if they are not the same\n",
    "        t = '\\t\\tMFMap['\n",
    "        t += \" + \".join(['str(tmprec' + str([j]) + \")\" for j in list(idx_combination)])  # MFMap[tmprec['prod'] + tmprec['month']]=MF_idx\n",
    "        t += ']=MF_idx\\n'\n",
    "        \n",
    "        ######################################## Correction on May 8\n",
    "        t += '\\t\\tMFMap['\n",
    "        t += \" + \".join(['str(tmprec' + str([j]) + \")\" for j in list(idx_combination)[::-1]]) \n",
    "        t += ']=MF_idx\\n'\n",
    "        #########################################\n",
    "    else:  # if they are the same\n",
    "        t += 'str(tmprec' + str(list(set(idx_combination))) + \")\"  # MFMap[tmprec['prod']]=MF_idx\n",
    "        t += ']=MF_idx\\n'\n",
    "    MFMap_initing.append(t)\n",
    "\n",
    "MFMap_text = scan_text + \"\".join(MFMap_initing) + \"\\t\\tMF_idx+=1 # new idx\\n\\n\"  # MFMap_initial_text\n",
    "\n",
    "print(MFMap_text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:42.741093Z",
     "start_time": "2020-05-09T02:16:42.738105Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['cust', 'cust']"
      ]
     },
     "execution_count": 126,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(idx_combination)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color=red>Updating for MF_Structure  </font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:43.181323Z",
     "start_time": "2020-05-09T02:16:43.162317Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "for gv in ['x', 'y']:\n",
      "\tif gv == 'x':\n",
      "\t\tfor i in range(len(records)):\n",
      "\t\t\ttmprec = {x[0]:x[1] for x in zip(['cust', 'prod', 'day', 'month', 'year', 'state', 'quant'],records[i])}\n",
      "\t\t\tindex = MFMap[str(tmprec['prod'])+str(tmprec['cust'])]\n",
      "\t\t\tif ((tmprec['prod']==MFVector[index]['prod']) and (tmprec['cust']==MFVector[index]['cust'])):\n",
      "\t\t\t\tMFVector[index]['x_count_quant']+=1\n",
      "\t\t\t\tMFVector[index]['x_sum_quant']+=tmprec['quant']\n",
      "\t\t\t\tMFVector[index]['x_avg_quant']=MFVector[index]['x_sum_quant']/MFVector[index]['x_count_quant']\n",
      "\t\t\t\tMFVector[index]['x_count_prod']+=1\n",
      "\n",
      "\tif gv == 'y':\n",
      "\t\tfor i in range(len(records)):\n",
      "\t\t\ttmprec = {x[0]:x[1] for x in zip(['cust', 'prod', 'day', 'month', 'year', 'state', 'quant'],records[i])}\n",
      "\t\t\tindex = MFMap[str(tmprec['cust'])]\n",
      "\t\t\tif ((tmprec['prod']!=MFVector[index]['prod']) and (tmprec['cust']==MFVector[index]['cust'])):\n",
      "\t\t\t\tMFVector[index]['y_count_quant']+=1\n",
      "\t\t\t\tMFVector[index]['y_sum_quant']+=tmprec['quant']\n",
      "\t\t\t\tMFVector[index]['y_avg_quant']=MFVector[index]['y_sum_quant']/MFVector[index]['y_count_quant']\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "\n",
    "########################################\n",
    "## # Updating for MF_Structure  -  V1\n",
    "########################################\n",
    "MF_update = \"for gv in \" + str(n) + \":\\n\"\n",
    "scan_text_2 = \"\\t\\tfor i in range(len(records)):\\n\\t\\t\\ttmprec = {x[0]:x[1] for x in zip(\" + str(T_cols) + \",records[i])}\\n\"\n",
    "for one_gv in range(len(n)):\n",
    "    Map_index = '\\t\\t\\tindex = MFMap['\n",
    "    indexing_cols = []\n",
    "    Condition = '\\t\\t\\tif ('\n",
    "    conditioning_cols = []\n",
    "    Update = ''\n",
    "    updating_cols = []\n",
    "    for piece in C_cols[one_gv]:  # C_cols = [['1.prod=prod ', ' 1.month=month ', ' 1.year=1997'], ['2.prod=prod ', ' 2.year=1997']]\n",
    "\n",
    "        ############# indexing_cols  E.g. 1.cust=cust\n",
    "        #### output ####  index = MFMap[tmprec['prod']+tmprec['month']]\n",
    "        if ('=' in piece) and (\">=\" not in piece) and (\"<=\" not in piece):  # find the text after \"=\"\n",
    "            t = piece.split(\"=\")[-1].strip()  # find the indexing cols which exist \"=\" symbol. E.g. 1.cust=cust , the later 'cust'\n",
    "            # print(t)\n",
    "            if t in V_cols:  # intersect with MF_Vector combined index\n",
    "                # print(V_cols)\n",
    "                indexing_cols.append(\"str(tmprec['\" + t + \"'])\")  # This have to be string\n",
    "                t = ''\n",
    "\n",
    "        ############ Condition  # subsitute the condition symbol in the map\n",
    "        #### output #### if ((tmprec['prod']==MFVector[index]['prod']) and (tmprec['month']==MFVector[index]['month']) and (tmprec['year']==1997)):\n",
    "        for key in logic_map.keys():  # order in the logic map matter. if find the '=' first,  the\">=\" maybe include\n",
    "            if key in piece:  # subsitute, and break into two part, based on \" \" whitespace because the map contains the white space in both side\n",
    "                #                 x = piece.replace(key, logic_map[key]).strip().split(\" \")\n",
    "                x = [i.strip() for i in piece.replace(key, logic_map[key]).split(\" \") if len(i.strip()) >= 1]  # '2.prod=prod' ==> [2.prod, ==,  prod]\n",
    "                break  # only replace once\n",
    "\n",
    "        if x[2] in V_cols:  # the later one not exist in V_cols for MF_V indexing\n",
    "            conditioning_cols.append(\"(tmprec['\" + x[0][(x[0].find('.') + 1):] + \"']\" + x[1] + \"MFVector[index][\" + \"'\" + x[2] + \"'])\")  # x =  [2.prod, ==,  prod]\n",
    "        else:\n",
    "            conditioning_cols.append(\"(tmprec['\" + x[0][(x[0].find('.') + 1):] + \"']\" + x[1] + x[2] + \")\")  # E.g. x = [3.year, ==, 1997]\n",
    "\n",
    "    ############ Update loop for each gv\n",
    "    agg_cols = [col for col in MF_cols if (n[one_gv]) == col[0]]  # find the related gv cols in F. at the first begining one . e.g. gv = first of '2_avg_quant'\n",
    "    #     print(agg_cols)\n",
    "\n",
    "    ######## key code: for handle one gv for other attrs\n",
    "    ##### e.g. for [x_count_prod,  x_avg_quant, x_sum_quant]\n",
    "    ##### get gv_attrs_set = [prod, quant]\n",
    "    ########\n",
    "    gv_attrs_set = list(set([gv_attrs.split('_')[2] for gv_attrs in agg_cols]))  # get gv_attrs_set = [prod, quant]\n",
    "    for attrs in gv_attrs_set:  # for gv=x, for attrs in [prod, quant] ...\n",
    "        agg_attr_cols = [each for each in agg_cols if each.find(attrs) > 1]  # 找到该 gv 下面 只包含  prod 的所有 agg_cols =  x_count_prod\n",
    "\n",
    "        updating_col = []  # one gv, one attrs for it\n",
    "        for agg_col in agg_attr_cols:\n",
    "            y = agg_col.split('_')  # ['2', 'avg', 'quant'] # meet avg , break\n",
    "            # identify agg function\n",
    "            if 'avg' == y[-2]:\n",
    "                updating_col = []  # meet 'avg', append with sum and count\n",
    "                updating_col.append(\"\\t\\t\\t\\tMFVector[index]['\" + agg_col.replace('avg', 'count') + \"']+=1\\n\")\n",
    "                updating_col.append( \"\\t\\t\\t\\tMFVector[index]['\" + agg_col.replace('avg', 'sum') + \"']+=\" + \"tmprec['\" + y[-1] + \"']\\n\")\n",
    "                updating_col.append(\"\\t\\t\\t\\tMFVector[index]['\" + agg_col + \"']=\" + \"MFVector[index]['\" + agg_col.replace('avg','sum') + \"']\" \n",
    "                                    + '/' + \"MFVector[index]['\" + agg_col.replace('avg', 'count') + \"']\\n\")\n",
    "                break\n",
    "            elif ('sum' == y[-2]):\n",
    "                updating_col.append(\"\\t\\t\\t\\tMFVector[index]['\" + agg_col.replace('avg', 'sum') + \"']+=\" + \"tmprec['\" + y[-1] + \"']\\n\")\n",
    "            elif ('count' == y[-2]):\n",
    "                updating_col.append(\"\\t\\t\\t\\tMFVector[index]['\" + agg_col.replace('avg', 'count') + \"']+=1\\n\")\n",
    "            else:\n",
    "                print(\":< This poor codes cannot handle this aggregation function:\", y[-2])\n",
    "        #             print(updating_col) # one gv, one attrs for it\n",
    "        updating_cols.extend(updating_col)  # one gv, every attrs for it!!\n",
    "    #         print(updating_cols)  # one gv, every attrs for it\n",
    "\n",
    "    gv_judge = \"\\tif gv == \" + str(\"'\" + n[one_gv] + \"'\") + \":\\n\"\n",
    "    Map_index += \"+\".join(indexing_cols) + \"]\\n\"  # index = MFMap[str(tmprec['cust'])+str(tmprec['prod'])]\n",
    "    Condition += \" and \".join(\n",
    "        conditioning_cols) + \"):\\n\"  # if ((tmprec['cust']==MFVector[index]['cust']) and (tmprec['prod']==MFVector[index]['prod'])):\n",
    "    Update += \"\".join(updating_cols) + \"\\n\"  # MFVector[index]['2_count_quant']+=1\n",
    "    # print(Map_index,Condition, Update)\n",
    "    one_gv_scan_text = gv_judge + scan_text_2 + Map_index + Condition + Update\n",
    "\n",
    "    MF_update += one_gv_scan_text\n",
    "\n",
    "print(MF_update)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Having\n",
    "- two functions for handling piece in condition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:43.603374Z",
     "start_time": "2020-05-09T02:16:43.593035Z"
    }
   },
   "outputs": [],
   "source": [
    "########################################\n",
    "# process piece in Condition like\n",
    "# ['1_sum_quant/2',\"==\",\"3\"]  ==> [\"MFVector[index]['1_sum_quant']/2\", '==', '3']\n",
    "########################################\n",
    "def compute_str_process(x):\n",
    "\n",
    "    '''\n",
    "    process the computational symbol after logic processing\n",
    "\n",
    "    input1 = ['1_sum_quant/2',\"==\",\"3\"]\n",
    "    output1 = [\"MFVector[index]['1_sum_quant']/2\", '==', '3']\n",
    "\n",
    "    input2 = ['2_count_prod+2', '==', '1_count_prod+2']\n",
    "    output2 = [\"MFVector[index]['2_count_prod']+2\", '==', \"MFVector[index]['1_count_prod']+2\"]\n",
    "    '''\n",
    "    # test\n",
    "    out = []\n",
    "    for frac in x:  # three kinds\n",
    "        \n",
    "        # if gv in the frac and find and it contains compute symbol  [\\+\\-\\/\\*] or other col\n",
    "        if (\"_\" in frac) and len(re.findall('(\\w+)[\\+\\-\\/\\*]', frac)) >= 1:\n",
    "            col = re.findall('(\\w+)[\\+\\-\\/\\*]', frac)[0]  # extract the col\n",
    "            compute_symbol = re.findall('[\\+\\-\\/\\*]', frac)\n",
    "            \n",
    "            suffix = frac[len(col) + 1::]  # it contains compute symbol  [\\+\\-\\/\\*] or other col\n",
    "            if re.findall('[\\D]+', suffix):  # not pure number --> suffix is one col in MFVector\n",
    "                out.extend([\"MFVector[index]['\" + col + \"']\" + compute_symbol[0] + \"MFVector[index]['\" + suffix + \"']\"])\n",
    "            else:  # pure number\n",
    "                out.extend([\"MFVector[index]['\" + col + \"']\" + compute_symbol[0] + suffix])\n",
    "        elif (\"_\" in frac):  # if only the gv in the frac\n",
    "            out.extend([\"MFVector[index]['\" + frac + \"']\"])\n",
    "        else:  # not col in this frac. it means it is \"==\" or \"0\" or \">=\", etc\n",
    "            out.append(frac)\n",
    "    return \"\".join(out)\n",
    "\n",
    "\n",
    "########################################\n",
    "### Parsing for G\n",
    "########################################\n",
    "def G_parsing(G):\n",
    "    '''\n",
    "    input =  [\"2_count_prod+2= 1_count_prod+2 and 1_avg_quant<>0\"]  # G\n",
    "    output = \"MFVector[index]['2_count_prod']+2==MFVector[index]['1_count_prod']+2\n",
    "            and MFVector[index]['1_avg_quant']!=0\"\n",
    "\n",
    "    '''\n",
    "    Having_c = []\n",
    "    logic_map = {\">=\": \" >= \", \"<=\": \" <= \", \"=\": \" == \", \"<>\": ' != ', \">\": \" > \", \"<\": \" < \"}  # from left to right, it only execute once. target: insert space for separating\n",
    "    for one_condition in G[0].split('and'):  # piece = [\"2_count_prod+2= 1_count_prod+2\" and 1_avg_quant<>0]\n",
    "        piece = one_condition[:]\n",
    "        \n",
    "        # replace once for the logic map\n",
    "        for key in logic_map.keys():  # order in the logic map matter. if find the '=' first,  the\">=\" maybe include\n",
    "            if key in piece:  # subsitute, and break into two part, based on \" \" whitespace because the map contains the white space in both side\n",
    "                x = [i.strip() for i in piece.replace(key, logic_map[key]).split(\" \") if len(i.strip()) >= 1]\n",
    "                # print(x)   # x = ['2_count_prod', '==', '1_count_prod+2']\n",
    "                break  # only replace once\n",
    "        \n",
    "        # x = ['2_count_prod', '==', '1_count_prod+2']\n",
    "        # further replace for +-*/ symbol with re and add the prefix: MFVector[index] etc\n",
    "        out = compute_str_process(x)\n",
    "        Having_c.append(\"\".join(out))  # \"MFVector[index]['2_count_prod']==MFVector[index]['1_count_prod']+2\n",
    "    return \" and \".join(Having_c)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:43.821061Z",
     "start_time": "2020-05-09T02:16:43.817060Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "output_idx=0\n",
      "for index in range(len(MFVector)):\n",
      "\tif MFVector[index]['y_avg_quant']!=0 and MFVector[index]['x_avg_quant']>=0:\n",
      "\t\toutput_idx+=1\n",
      "\t\tprint(end='{:<6}|'.format(output_idx))\n",
      "\n"
     ]
    }
   ],
   "source": [
    "########################################\n",
    "# Having\n",
    "########################################\n",
    "Having_text = 'output_idx=0\\n'\n",
    "Having_text += \"for index in range(len(MFVector)):\\n\"\n",
    "Having_text += \"\\tif \" + G_parsing(G) + \":\\n\"\n",
    "Having_text += '\\t\\toutput_idx+=1\\n'  # for output index\n",
    "Having_text += \"\\t\\tprint(end='{:<6}|'.format(output_idx))\\n\"\n",
    "\n",
    "print(Having_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Selection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:44.225457Z",
     "start_time": "2020-05-09T02:16:44.216464Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\t\tprint('{:<15}|'.format(MFVector[index]['prod']), end='')\n",
      "\t\tprint('{:<15}|'.format(MFVector[index]['cust']), end='')\n",
      "\t\tprint('{:>15.2f}|'.format(MFVector[index]['x_sum_quant']), end='')\n",
      "\t\tprint('{:>15.0f}|'.format(MFVector[index]['x_count_prod']), end='')\n",
      "\t\tprint('{:>15.2f}|'.format(MFVector[index]['x_avg_quant']+1), end='')\n",
      "\t\tprint('{:>15.2f}|'.format(MFVector[index]['y_avg_quant']/2), end='')\n",
      "\t\tprint('{:>15.2f}|'.format(MFVector[index]['x_avg_quant']/MFVector[index]['y_avg_quant']), end='')\n",
      "\t\tprint('')\n"
     ]
    }
   ],
   "source": [
    "########################################\n",
    "# Output\n",
    "########################################\n",
    "### Table_cols_name_output\n",
    "Table_cols_name_output = \"print('='*100)\\n\"\n",
    "format_ = \"print('{:<6}|\"\n",
    "cols = \"'.format('index'\"\n",
    "\n",
    "for col in S:\n",
    "    try:  # detect exist or not in MF_dtypes\n",
    "        format_ += format_length_map[MF_dtypes[i]] + '|'\n",
    "        cols += \",'\" + col + \"'\"\n",
    "    except:  # if not , it is float, and the selection part need change\n",
    "        format_ += format_length_map[0] + '|'\n",
    "        cols += \",'\" + col + \"'\"\n",
    "Table_cols_name_output += format_ + cols + '))\\n' + \"print('='*100)\\n\"\n",
    "\n",
    "Selection_text = ''\n",
    "for col in S:\n",
    "    try:  # simple\n",
    "        if type(MF_dtypes[col]) == str:\n",
    "            Selection_text += \"\\t\\tprint('{:<15}|'.format(MFVector[index]['\" + col + \"']), end='')\\n\"\n",
    "        elif (type(MF_dtypes[col]) == int):\n",
    "            Selection_text += \"\\t\\tprint('{:>15.0f}|'.format(MFVector[index]['\" + col + \"']), end='')\\n\"\n",
    "        elif (type(MF_dtypes[col]) == float):\n",
    "            Selection_text += \"\\t\\tprint('{:>15.2f}|'.format(MFVector[index]['\" + col + \"']), end='')\\n\"\n",
    "    except:  # the combined output\n",
    "        Selection_text += \"\\t\\tprint('{:>15.2f}|'.format(\" + compute_str_process(\n",
    "            [col]) + \"), end='')\\n\"  # it can only process list\n",
    "\n",
    "Selection_text += \"\\t\\tprint('')\"\n",
    "\n",
    "print(Selection_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Combined the result and return"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:44.605753Z",
     "start_time": "2020-05-09T02:16:44.601751Z"
    }
   },
   "outputs": [],
   "source": [
    "res = '# Import and Connecting DB\\n'\n",
    "res += import_text + '\\n'\n",
    "res += connection_text + '\\n'\n",
    "res += '# Initializing MF_Vector\\n'\n",
    "res += MFVector_text + '\\n'\n",
    "res += '# Initializing Map for MF_Structure\\n'\n",
    "res += MFMap_text + '\\n'\n",
    "res += '# Updating for MF_Structure\\n'\n",
    "res += MF_update + '\\n'\n",
    "res += '# Outputing\\n'\n",
    "res += Table_cols_name_output\n",
    "res += '# Having\\n'\n",
    "res += Having_text + '\\n'\n",
    "res += Selection_text + '\\n'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:44.807552Z",
     "start_time": "2020-05-09T02:16:44.801539Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The file has been saved:  ./Output_files/out7_2.py\n"
     ]
    }
   ],
   "source": [
    "########################################\n",
    "# Save result\n",
    "########################################\n",
    "to_file = \"./Output_files/out%s.py\" % output_num\n",
    "with open(to_file, \"w\", encoding='utf-8') as f:\n",
    "    f.write(res)\n",
    "    print(\"The file has been saved: \", to_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preview of output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:45.222597Z",
     "start_time": "2020-05-09T02:16:45.218595Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# Import and Connecting DB\n",
      "import psycopg2\n",
      "import sys\n",
      "from collections import defaultdict\n",
      "\n",
      "try:\n",
      "    connection = psycopg2.connect(user='postgres', password='00000000', host='localhost', port='5388', database='postgres', )\n",
      "    query = \"SELECT * FROM sales\"\n",
      "    cursor = connection.cursor()\n",
      "    cursor.execute(query)\n",
      "    if cursor.rowcount == 0:\n",
      "        print(\"Connected, but no data!\")\n",
      "        sys.exit(0)\n",
      "                    \n",
      "    records = cursor.fetchall()\n",
      "except(Exception, psycopg2.Error) as error:\n",
      "    print(\"Error connecting to PostgreSQL database ==>\", error)\n",
      "    sys.exit(0)\n",
      "\n",
      "                    \n",
      "# Initializing MF_Vector\n",
      "MFVector = defaultdict(lambda: {'prod':'', 'cust':'', 'x_count_quant':0, 'x_sum_quant':0.0, 'x_avg_quant':0.0, 'y_count_quant':0, 'y_sum_quant':0.0, 'y_avg_quant':0.0, 'x_count_prod':0, 'x_sum_quant':0.0})\n",
      "MF_idx=0\n",
      "MFMap = {}\n",
      "\n",
      "# Initializing Map for MF_Structure\n",
      "for i in range(len(records)):\n",
      "\ttmprec = {x[0]:x[1] for x in zip(['cust', 'prod', 'day', 'month', 'year', 'state', 'quant'],records[i])}\n",
      "\tif (str(tmprec['prod']) + str(tmprec['cust'])) not in MFMap:\n",
      "\t\tMFVector[MF_idx]['prod']=tmprec['prod']\n",
      "\t\tMFVector[MF_idx]['cust']=tmprec['cust']\n",
      "\t\tMFMap[str(tmprec['prod'])]=MF_idx\n",
      "\t\tMFMap[str(tmprec['prod']) + str(tmprec['cust'])]=MF_idx\n",
      "\t\tMFMap[str(tmprec['cust']) + str(tmprec['prod'])]=MF_idx\n",
      "\t\tMFMap[str(tmprec['cust'])]=MF_idx\n",
      "\t\tMF_idx+=1 # new idx\n",
      "\n",
      "\n",
      "# Updating for MF_Structure\n",
      "for gv in ['x', 'y']:\n",
      "\tif gv == 'x':\n",
      "\t\tfor i in range(len(records)):\n",
      "\t\t\ttmprec = {x[0]:x[1] for x in zip(['cust', 'prod', 'day', 'month', 'year', 'state', 'quant'],records[i])}\n",
      "\t\t\tindex = MFMap[str(tmprec['prod'])+str(tmprec['cust'])]\n",
      "\t\t\tif ((tmprec['prod']==MFVector[index]['prod']) and (tmprec['cust']==MFVector[index]['cust'])):\n",
      "\t\t\t\tMFVector[index]['x_count_quant']+=1\n",
      "\t\t\t\tMFVector[index]['x_sum_quant']+=tmprec['quant']\n",
      "\t\t\t\tMFVector[index]['x_avg_quant']=MFVector[index]['x_sum_quant']/MFVector[index]['x_count_quant']\n",
      "\t\t\t\tMFVector[index]['x_count_prod']+=1\n",
      "\n",
      "\tif gv == 'y':\n",
      "\t\tfor i in range(len(records)):\n",
      "\t\t\ttmprec = {x[0]:x[1] for x in zip(['cust', 'prod', 'day', 'month', 'year', 'state', 'quant'],records[i])}\n",
      "\t\t\tindex = MFMap[str(tmprec['cust'])]\n",
      "\t\t\tif ((tmprec['prod']!=MFVector[index]['prod']) and (tmprec['cust']==MFVector[index]['cust'])):\n",
      "\t\t\t\tMFVector[index]['y_count_quant']+=1\n",
      "\t\t\t\tMFVector[index]['y_sum_quant']+=tmprec['quant']\n",
      "\t\t\t\tMFVector[index]['y_avg_quant']=MFVector[index]['y_sum_quant']/MFVector[index]['y_count_quant']\n",
      "\n",
      "\n",
      "# Outputing\n",
      "print('='*100)\n",
      "print('{:<6}|{:>15}|{:>15}|{:>15}|{:>15}|{:>15}|{:>15}|{:>15}|'.format('index','prod','cust','x_sum_quant','x_count_prod','x_avg_quant+1','y_avg_quant/2','x_avg_quant/y_avg_quant'))\n",
      "print('='*100)\n",
      "# Having\n",
      "output_idx=0\n",
      "for index in range(len(MFVector)):\n",
      "\tif MFVector[index]['y_avg_quant']!=0 and MFVector[index]['x_avg_quant']>=0:\n",
      "\t\toutput_idx+=1\n",
      "\t\tprint(end='{:<6}|'.format(output_idx))\n",
      "\n",
      "\t\tprint('{:<15}|'.format(MFVector[index]['prod']), end='')\n",
      "\t\tprint('{:<15}|'.format(MFVector[index]['cust']), end='')\n",
      "\t\tprint('{:>15.2f}|'.format(MFVector[index]['x_sum_quant']), end='')\n",
      "\t\tprint('{:>15.0f}|'.format(MFVector[index]['x_count_prod']), end='')\n",
      "\t\tprint('{:>15.2f}|'.format(MFVector[index]['x_avg_quant']+1), end='')\n",
      "\t\tprint('{:>15.2f}|'.format(MFVector[index]['y_avg_quant']/2), end='')\n",
      "\t\tprint('{:>15.2f}|'.format(MFVector[index]['x_avg_quant']/MFVector[index]['y_avg_quant']), end='')\n",
      "\t\tprint('')\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(res)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:16:45.422549Z",
     "start_time": "2020-05-09T02:16:45.419549Z"
    },
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [],
   "source": [
    "# !python ./Output_files/out9.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Py excute Result - <font color=red>Need copy back</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 137,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:20:32.069991Z",
     "start_time": "2020-05-09T02:20:32.046278Z"
    },
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [],
   "source": [
    "def test():\n",
    "    # Import and Connecting DB\n",
    "    import psycopg2\n",
    "    import sys\n",
    "    from collections import defaultdict\n",
    "\n",
    "    try:\n",
    "        connection = psycopg2.connect(user='postgres', password='00000000', host='localhost', port='5388', database='postgres', )\n",
    "        query = \"SELECT * FROM sales\"\n",
    "        cursor = connection.cursor()\n",
    "        cursor.execute(query)\n",
    "        if cursor.rowcount == 0:\n",
    "            print(\"Connected, but no data!\")\n",
    "            sys.exit(0)\n",
    "\n",
    "        records = cursor.fetchall()\n",
    "    except(Exception, psycopg2.Error) as error:\n",
    "        print(\"Error connecting to PostgreSQL database ==>\", error)\n",
    "        sys.exit(0)\n",
    "\n",
    "\n",
    "    # Initializing MF_Vector\n",
    "    MFVector = defaultdict(lambda: {'prod':'', 'cust':'', 'x_count_quant':0, 'x_sum_quant':0.0, 'x_avg_quant':0.0, 'y_count_quant':0, 'y_sum_quant':0.0, 'y_avg_quant':0.0, 'x_count_prod':0, 'x_sum_quant':0.0})\n",
    "    MF_idx=0\n",
    "    MFMap = {}\n",
    "\n",
    "    # Initializing Map for MF_Structure\n",
    "    for i in range(len(records)):\n",
    "        tmprec = {x[0]:x[1] for x in zip(['cust', 'prod', 'day', 'month', 'year', 'state', 'quant'],records[i])}\n",
    "        if (str(tmprec['prod']) + str(tmprec['cust'])) not in MFMap:\n",
    "            MFVector[MF_idx]['prod']=tmprec['prod']\n",
    "            MFVector[MF_idx]['cust']=tmprec['cust']\n",
    "            MFMap[str(tmprec['prod'])]=MF_idx\n",
    "            MFMap[str(tmprec['prod']) + str(tmprec['cust'])]=MF_idx\n",
    "            MFMap[str(tmprec['cust']) + str(tmprec['prod'])]=MF_idx\n",
    "            MFMap[str(tmprec['cust'])]=MF_idx\n",
    "            MF_idx+=1 # new idx\n",
    "\n",
    "\n",
    "    # Updating for MF_Structure\n",
    "    for gv in ['x', 'y']:\n",
    "        if gv == 'x':\n",
    "            for i in range(len(records)):\n",
    "                tmprec = {x[0]:x[1] for x in zip(['cust', 'prod', 'day', 'month', 'year', 'state', 'quant'],records[i])}\n",
    "                index = MFMap[str(tmprec['cust'])+str(tmprec['prod'])]\n",
    "                if ((tmprec['cust']==MFVector[index]['cust']) and (tmprec['prod']==MFVector[index]['prod']) and (tmprec['prod']==MFVector[index]['prod'])):\n",
    "                    MFVector[index]['x_count_prod']+=1\n",
    "                    MFVector[index]['x_count_quant']+=1\n",
    "                    MFVector[index]['x_sum_quant']+=tmprec['quant']\n",
    "                    MFVector[index]['x_avg_quant']=MFVector[index]['x_sum_quant']/MFVector[index]['x_count_quant']\n",
    "\n",
    "        if gv == 'y':\n",
    "            for i in range(len(records)):\n",
    "                tmprec = {x[0]:x[1] for x in zip(['cust', 'prod', 'day', 'month', 'year', 'state', 'quant'],records[i])}\n",
    "                index = MFMap[str(tmprec['cust'])]\n",
    "                if ((tmprec['cust']==MFVector[index]['cust']) and (tmprec['prod']!=MFVector[index]['prod'])):\n",
    "                    MFVector[index]['y_count_quant']+=1\n",
    "                    MFVector[index]['y_sum_quant']+=tmprec['quant']\n",
    "                    MFVector[index]['y_avg_quant']=MFVector[index]['y_sum_quant']/MFVector[index]['y_count_quant']\n",
    "\n",
    "\n",
    "    # Outputing\n",
    "    print('='*100)\n",
    "    print('{:<6}|{:>15}|{:>15}|{:>15}|{:>15}|{:>15}|{:>15}|{:>15}|'.format('index','prod','cust','x_sum_quant','x_count_prod','x_avg_quant+1','y_avg_quant/2','x_avg_quant/y_avg_quant'))\n",
    "    print('='*100)\n",
    "    # Having\n",
    "    output_idx=0\n",
    "    for index in range(len(MFVector)):\n",
    "        if MFVector[index]['y_avg_quant']!=0 and MFVector[index]['x_avg_quant']>=0:\n",
    "            output_idx+=1\n",
    "            print(end='{:<6}|'.format(output_idx))\n",
    "\n",
    "            print('{:<15}|'.format(MFVector[index]['prod']), end='')\n",
    "            print('{:<15}|'.format(MFVector[index]['cust']), end='')\n",
    "            print('{:>15.2f}|'.format(MFVector[index]['x_sum_quant']), end='')\n",
    "            print('{:>15.0f}|'.format(MFVector[index]['x_count_prod']), end='')\n",
    "            print('{:>15.2f}|'.format(MFVector[index]['x_avg_quant']+1), end='')\n",
    "            print('{:>15.2f}|'.format(MFVector[index]['y_avg_quant']/2), end='')\n",
    "            print('{:>15.2f}|'.format(MFVector[index]['x_avg_quant']/MFVector[index]['y_avg_quant']), end='')\n",
    "            print('')\n",
    "\n",
    "\n",
    "            \n",
    "    return MFVector\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:20:52.111683Z",
     "start_time": "2020-05-09T02:20:52.020085Z"
    },
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "====================================================================================================\n",
      "index |           prod|           cust|    x_sum_quant|   x_count_prod|  x_avg_quant+1|  y_avg_quant/2|x_avg_quant/y_avg_quant|\n",
      "====================================================================================================\n",
      "1     |Bread          |Helen          |       25032.00|              9|        2782.33|        1129.56|           1.23|\n",
      "2     |Butter         |Knuth          |        7545.00|              6|        1258.50|        1184.28|           0.53|\n",
      "3     |Coke           |Sam            |       11354.00|              5|        2271.80|        1195.95|           0.95|\n",
      "4     |Coke           |Emily          |       21681.00|              9|        2410.00|        1261.04|           0.96|\n",
      "5     |Bread          |Bloom          |       25343.00|             12|        2112.92|        1253.29|           0.84|\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>prod</th>\n",
       "      <th>cust</th>\n",
       "      <th>x_count_quant</th>\n",
       "      <th>x_sum_quant</th>\n",
       "      <th>x_avg_quant</th>\n",
       "      <th>y_count_quant</th>\n",
       "      <th>y_sum_quant</th>\n",
       "      <th>y_avg_quant</th>\n",
       "      <th>x_count_prod</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Pepsi</td>\n",
       "      <td>Bloom</td>\n",
       "      <td>8</td>\n",
       "      <td>15390</td>\n",
       "      <td>1923.75</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Bread</td>\n",
       "      <td>Knuth</td>\n",
       "      <td>12</td>\n",
       "      <td>32562</td>\n",
       "      <td>2713.5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Pepsi</td>\n",
       "      <td>Emily</td>\n",
       "      <td>7</td>\n",
       "      <td>22782</td>\n",
       "      <td>3254.57</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Fruits</td>\n",
       "      <td>Emily</td>\n",
       "      <td>13</td>\n",
       "      <td>35059</td>\n",
       "      <td>2696.85</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Milk</td>\n",
       "      <td>Helen</td>\n",
       "      <td>12</td>\n",
       "      <td>25239</td>\n",
       "      <td>2103.25</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Soap</td>\n",
       "      <td>Emily</td>\n",
       "      <td>8</td>\n",
       "      <td>19290</td>\n",
       "      <td>2411.25</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Eggs</td>\n",
       "      <td>Bloom</td>\n",
       "      <td>11</td>\n",
       "      <td>27553</td>\n",
       "      <td>2504.82</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Yogurt</td>\n",
       "      <td>Bloom</td>\n",
       "      <td>8</td>\n",
       "      <td>17670</td>\n",
       "      <td>2208.75</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Pepsi</td>\n",
       "      <td>Helen</td>\n",
       "      <td>13</td>\n",
       "      <td>34613</td>\n",
       "      <td>2662.54</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Bread</td>\n",
       "      <td>Emily</td>\n",
       "      <td>10</td>\n",
       "      <td>15343</td>\n",
       "      <td>1534.3</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>Cookies</td>\n",
       "      <td>Sam</td>\n",
       "      <td>7</td>\n",
       "      <td>12229</td>\n",
       "      <td>1747</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>Milk</td>\n",
       "      <td>Knuth</td>\n",
       "      <td>4</td>\n",
       "      <td>3696</td>\n",
       "      <td>924</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>Coke</td>\n",
       "      <td>Helen</td>\n",
       "      <td>15</td>\n",
       "      <td>39263</td>\n",
       "      <td>2617.53</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>Butter</td>\n",
       "      <td>Emily</td>\n",
       "      <td>9</td>\n",
       "      <td>20852</td>\n",
       "      <td>2316.89</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>Yogurt</td>\n",
       "      <td>Emily</td>\n",
       "      <td>7</td>\n",
       "      <td>16510</td>\n",
       "      <td>2358.57</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>Soap</td>\n",
       "      <td>Sam</td>\n",
       "      <td>11</td>\n",
       "      <td>24215</td>\n",
       "      <td>2201.36</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>Coke</td>\n",
       "      <td>Knuth</td>\n",
       "      <td>13</td>\n",
       "      <td>35449</td>\n",
       "      <td>2726.85</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>Milk</td>\n",
       "      <td>Sam</td>\n",
       "      <td>13</td>\n",
       "      <td>23589</td>\n",
       "      <td>1814.54</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>Yogurt</td>\n",
       "      <td>Helen</td>\n",
       "      <td>10</td>\n",
       "      <td>26550</td>\n",
       "      <td>2655</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>Pepsi</td>\n",
       "      <td>Knuth</td>\n",
       "      <td>13</td>\n",
       "      <td>34401</td>\n",
       "      <td>2646.23</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>Eggs</td>\n",
       "      <td>Knuth</td>\n",
       "      <td>10</td>\n",
       "      <td>25381</td>\n",
       "      <td>2538.1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>Yogurt</td>\n",
       "      <td>Knuth</td>\n",
       "      <td>8</td>\n",
       "      <td>21157</td>\n",
       "      <td>2644.62</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>Fruits</td>\n",
       "      <td>Helen</td>\n",
       "      <td>9</td>\n",
       "      <td>16561</td>\n",
       "      <td>1840.11</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>Cookies</td>\n",
       "      <td>Helen</td>\n",
       "      <td>5</td>\n",
       "      <td>8897</td>\n",
       "      <td>1779.4</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>Fruits</td>\n",
       "      <td>Knuth</td>\n",
       "      <td>14</td>\n",
       "      <td>25871</td>\n",
       "      <td>1847.93</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>Cookies</td>\n",
       "      <td>Emily</td>\n",
       "      <td>16</td>\n",
       "      <td>42765</td>\n",
       "      <td>2672.81</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>Coke</td>\n",
       "      <td>Bloom</td>\n",
       "      <td>9</td>\n",
       "      <td>21050</td>\n",
       "      <td>2338.89</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>Soap</td>\n",
       "      <td>Helen</td>\n",
       "      <td>9</td>\n",
       "      <td>22493</td>\n",
       "      <td>2499.22</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>Soap</td>\n",
       "      <td>Knuth</td>\n",
       "      <td>11</td>\n",
       "      <td>20416</td>\n",
       "      <td>1856</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>Cookies</td>\n",
       "      <td>Knuth</td>\n",
       "      <td>10</td>\n",
       "      <td>26081</td>\n",
       "      <td>2608.1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>Milk</td>\n",
       "      <td>Emily</td>\n",
       "      <td>8</td>\n",
       "      <td>21307</td>\n",
       "      <td>2663.38</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>31</th>\n",
       "      <td>Soap</td>\n",
       "      <td>Bloom</td>\n",
       "      <td>6</td>\n",
       "      <td>14264</td>\n",
       "      <td>2377.33</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>Eggs</td>\n",
       "      <td>Emily</td>\n",
       "      <td>12</td>\n",
       "      <td>33080</td>\n",
       "      <td>2756.67</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33</th>\n",
       "      <td>Milk</td>\n",
       "      <td>Bloom</td>\n",
       "      <td>13</td>\n",
       "      <td>35941</td>\n",
       "      <td>2764.69</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34</th>\n",
       "      <td>Yogurt</td>\n",
       "      <td>Sam</td>\n",
       "      <td>9</td>\n",
       "      <td>21319</td>\n",
       "      <td>2368.78</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>35</th>\n",
       "      <td>Fruits</td>\n",
       "      <td>Sam</td>\n",
       "      <td>10</td>\n",
       "      <td>33879</td>\n",
       "      <td>3387.9</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36</th>\n",
       "      <td>Butter</td>\n",
       "      <td>Sam</td>\n",
       "      <td>8</td>\n",
       "      <td>21612</td>\n",
       "      <td>2701.5</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37</th>\n",
       "      <td>Fruits</td>\n",
       "      <td>Bloom</td>\n",
       "      <td>17</td>\n",
       "      <td>42011</td>\n",
       "      <td>2471.24</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>17</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38</th>\n",
       "      <td>Eggs</td>\n",
       "      <td>Helen</td>\n",
       "      <td>8</td>\n",
       "      <td>17081</td>\n",
       "      <td>2135.12</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>39</th>\n",
       "      <td>Pepsi</td>\n",
       "      <td>Sam</td>\n",
       "      <td>13</td>\n",
       "      <td>38848</td>\n",
       "      <td>2988.31</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>Butter</td>\n",
       "      <td>Helen</td>\n",
       "      <td>22</td>\n",
       "      <td>41993</td>\n",
       "      <td>1908.77</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>22</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>41</th>\n",
       "      <td>Bread</td>\n",
       "      <td>Helen</td>\n",
       "      <td>9</td>\n",
       "      <td>25032</td>\n",
       "      <td>2781.33</td>\n",
       "      <td>103</td>\n",
       "      <td>232690</td>\n",
       "      <td>2259.13</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>42</th>\n",
       "      <td>Butter</td>\n",
       "      <td>Bloom</td>\n",
       "      <td>3</td>\n",
       "      <td>9051</td>\n",
       "      <td>3017</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>Bread</td>\n",
       "      <td>Sam</td>\n",
       "      <td>8</td>\n",
       "      <td>13427</td>\n",
       "      <td>1678.38</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>44</th>\n",
       "      <td>Eggs</td>\n",
       "      <td>Sam</td>\n",
       "      <td>8</td>\n",
       "      <td>18977</td>\n",
       "      <td>2372.12</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>45</th>\n",
       "      <td>Butter</td>\n",
       "      <td>Knuth</td>\n",
       "      <td>6</td>\n",
       "      <td>7545</td>\n",
       "      <td>1257.5</td>\n",
       "      <td>95</td>\n",
       "      <td>225014</td>\n",
       "      <td>2368.57</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>46</th>\n",
       "      <td>Cookies</td>\n",
       "      <td>Bloom</td>\n",
       "      <td>9</td>\n",
       "      <td>27622</td>\n",
       "      <td>3069.11</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>47</th>\n",
       "      <td>Coke</td>\n",
       "      <td>Sam</td>\n",
       "      <td>5</td>\n",
       "      <td>11354</td>\n",
       "      <td>2270.8</td>\n",
       "      <td>87</td>\n",
       "      <td>208095</td>\n",
       "      <td>2391.9</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48</th>\n",
       "      <td>Coke</td>\n",
       "      <td>Emily</td>\n",
       "      <td>9</td>\n",
       "      <td>21681</td>\n",
       "      <td>2409</td>\n",
       "      <td>90</td>\n",
       "      <td>226988</td>\n",
       "      <td>2522.09</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49</th>\n",
       "      <td>Bread</td>\n",
       "      <td>Bloom</td>\n",
       "      <td>12</td>\n",
       "      <td>25343</td>\n",
       "      <td>2111.92</td>\n",
       "      <td>84</td>\n",
       "      <td>210552</td>\n",
       "      <td>2506.57</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       prod   cust x_count_quant x_sum_quant x_avg_quant y_count_quant  \\\n",
       "0     Pepsi  Bloom             8       15390     1923.75             0   \n",
       "1     Bread  Knuth            12       32562      2713.5             0   \n",
       "2     Pepsi  Emily             7       22782     3254.57             0   \n",
       "3    Fruits  Emily            13       35059     2696.85             0   \n",
       "4      Milk  Helen            12       25239     2103.25             0   \n",
       "5      Soap  Emily             8       19290     2411.25             0   \n",
       "6      Eggs  Bloom            11       27553     2504.82             0   \n",
       "7    Yogurt  Bloom             8       17670     2208.75             0   \n",
       "8     Pepsi  Helen            13       34613     2662.54             0   \n",
       "9     Bread  Emily            10       15343      1534.3             0   \n",
       "10  Cookies    Sam             7       12229        1747             0   \n",
       "11     Milk  Knuth             4        3696         924             0   \n",
       "12     Coke  Helen            15       39263     2617.53             0   \n",
       "13   Butter  Emily             9       20852     2316.89             0   \n",
       "14   Yogurt  Emily             7       16510     2358.57             0   \n",
       "15     Soap    Sam            11       24215     2201.36             0   \n",
       "16     Coke  Knuth            13       35449     2726.85             0   \n",
       "17     Milk    Sam            13       23589     1814.54             0   \n",
       "18   Yogurt  Helen            10       26550        2655             0   \n",
       "19    Pepsi  Knuth            13       34401     2646.23             0   \n",
       "20     Eggs  Knuth            10       25381      2538.1             0   \n",
       "21   Yogurt  Knuth             8       21157     2644.62             0   \n",
       "22   Fruits  Helen             9       16561     1840.11             0   \n",
       "23  Cookies  Helen             5        8897      1779.4             0   \n",
       "24   Fruits  Knuth            14       25871     1847.93             0   \n",
       "25  Cookies  Emily            16       42765     2672.81             0   \n",
       "26     Coke  Bloom             9       21050     2338.89             0   \n",
       "27     Soap  Helen             9       22493     2499.22             0   \n",
       "28     Soap  Knuth            11       20416        1856             0   \n",
       "29  Cookies  Knuth            10       26081      2608.1             0   \n",
       "30     Milk  Emily             8       21307     2663.38             0   \n",
       "31     Soap  Bloom             6       14264     2377.33             0   \n",
       "32     Eggs  Emily            12       33080     2756.67             0   \n",
       "33     Milk  Bloom            13       35941     2764.69             0   \n",
       "34   Yogurt    Sam             9       21319     2368.78             0   \n",
       "35   Fruits    Sam            10       33879      3387.9             0   \n",
       "36   Butter    Sam             8       21612      2701.5             0   \n",
       "37   Fruits  Bloom            17       42011     2471.24             0   \n",
       "38     Eggs  Helen             8       17081     2135.12             0   \n",
       "39    Pepsi    Sam            13       38848     2988.31             0   \n",
       "40   Butter  Helen            22       41993     1908.77             0   \n",
       "41    Bread  Helen             9       25032     2781.33           103   \n",
       "42   Butter  Bloom             3        9051        3017             0   \n",
       "43    Bread    Sam             8       13427     1678.38             0   \n",
       "44     Eggs    Sam             8       18977     2372.12             0   \n",
       "45   Butter  Knuth             6        7545      1257.5            95   \n",
       "46  Cookies  Bloom             9       27622     3069.11             0   \n",
       "47     Coke    Sam             5       11354      2270.8            87   \n",
       "48     Coke  Emily             9       21681        2409            90   \n",
       "49    Bread  Bloom            12       25343     2111.92            84   \n",
       "\n",
       "   y_sum_quant y_avg_quant x_count_prod  \n",
       "0            0           0            8  \n",
       "1            0           0           12  \n",
       "2            0           0            7  \n",
       "3            0           0           13  \n",
       "4            0           0           12  \n",
       "5            0           0            8  \n",
       "6            0           0           11  \n",
       "7            0           0            8  \n",
       "8            0           0           13  \n",
       "9            0           0           10  \n",
       "10           0           0            7  \n",
       "11           0           0            4  \n",
       "12           0           0           15  \n",
       "13           0           0            9  \n",
       "14           0           0            7  \n",
       "15           0           0           11  \n",
       "16           0           0           13  \n",
       "17           0           0           13  \n",
       "18           0           0           10  \n",
       "19           0           0           13  \n",
       "20           0           0           10  \n",
       "21           0           0            8  \n",
       "22           0           0            9  \n",
       "23           0           0            5  \n",
       "24           0           0           14  \n",
       "25           0           0           16  \n",
       "26           0           0            9  \n",
       "27           0           0            9  \n",
       "28           0           0           11  \n",
       "29           0           0           10  \n",
       "30           0           0            8  \n",
       "31           0           0            6  \n",
       "32           0           0           12  \n",
       "33           0           0           13  \n",
       "34           0           0            9  \n",
       "35           0           0           10  \n",
       "36           0           0            8  \n",
       "37           0           0           17  \n",
       "38           0           0            8  \n",
       "39           0           0           13  \n",
       "40           0           0           22  \n",
       "41      232690     2259.13            9  \n",
       "42           0           0            3  \n",
       "43           0           0            8  \n",
       "44           0           0            8  \n",
       "45      225014     2368.57            6  \n",
       "46           0           0            9  \n",
       "47      208095      2391.9            5  \n",
       "48      226988     2522.09            9  \n",
       "49      210552     2506.57           12  "
      ]
     },
     "execution_count": 140,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "MFVector_result = pd.DataFrame(test()).T\n",
    "MFVector_result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-05-09T02:20:36.188508Z",
     "start_time": "2020-05-09T02:20:36.161760Z"
    },
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>prod</th>\n",
       "      <th>cust</th>\n",
       "      <th>x_count_quant</th>\n",
       "      <th>x_sum_quant</th>\n",
       "      <th>x_avg_quant</th>\n",
       "      <th>y_count_quant</th>\n",
       "      <th>y_sum_quant</th>\n",
       "      <th>y_avg_quant</th>\n",
       "      <th>x_count_prod</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>50</td>\n",
       "      <td>50</td>\n",
       "      <td>50</td>\n",
       "      <td>50.0</td>\n",
       "      <td>50.0</td>\n",
       "      <td>50</td>\n",
       "      <td>50.0</td>\n",
       "      <td>50.0</td>\n",
       "      <td>50</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>unique</th>\n",
       "      <td>10</td>\n",
       "      <td>5</td>\n",
       "      <td>16</td>\n",
       "      <td>50.0</td>\n",
       "      <td>50.0</td>\n",
       "      <td>6</td>\n",
       "      <td>6.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>16</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>top</th>\n",
       "      <td>Eggs</td>\n",
       "      <td>Sam</td>\n",
       "      <td>8</td>\n",
       "      <td>25343.0</td>\n",
       "      <td>1534.3</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>freq</th>\n",
       "      <td>5</td>\n",
       "      <td>10</td>\n",
       "      <td>9</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>45</td>\n",
       "      <td>45.0</td>\n",
       "      <td>45.0</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        prod cust  x_count_quant  x_sum_quant  x_avg_quant  y_count_quant  \\\n",
       "count     50   50             50         50.0         50.0             50   \n",
       "unique    10    5             16         50.0         50.0              6   \n",
       "top     Eggs  Sam              8      25343.0       1534.3              0   \n",
       "freq       5   10              9          1.0          1.0             45   \n",
       "\n",
       "        y_sum_quant  y_avg_quant  x_count_prod  \n",
       "count          50.0         50.0            50  \n",
       "unique          6.0          6.0            16  \n",
       "top             0.0          0.0             8  \n",
       "freq           45.0         45.0             9  "
      ]
     },
     "execution_count": 139,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "pycharm": {
     "is_executing": false
    }
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "165px"
   },
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}