datasets_notebook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Network of Painters: building a dataset from paintings datasets, then creating links\n",
    "\n",
    "The aim of this project is to create a dataset of painters from datasets such as WikiArt and Art500k, combining features, extending missing data of painters with web scraping through Google and Wiki API, and then creating links between painters based on similarity of style, geographical and social interaction.\n",
    "\n",
    "Note: One long-term goal would be to create a JSON file that contains all combined hierarchically. For example, a level in the structure could be art movement, inside it are artists with some base data like birthplace, year of birth and death and other geographical data, inside it are paintings with all contained data (even better would be including eras of painters in their substructure, and inside them the paintings). Then we could use this to create a network of art movements, artists, and paintings.\n",
    "\n",
    "NEXT STEPS:<br>\n",
    "-Add \"Places\" for Art500k datasets (+change datasets_notebook save.csv loads)<br>\n",
    "-Add aliases for painters in Art500k datasets<br>\n",
    "-Combine the datasets on authors<br>\n",
    "\n",
    "FURTHER STEPS: <br>\n",
    "-Define connections between painters<br>\n",
    "-Create a network of painters<br>\n",
    "-Analyze the network<br>\n",
    "\n",
    "<details><summary><u> Update 11.06:</u></summary>\n",
    "<p>\n",
    "I e-mailed an art researcher that Elisa suggested, Maximilian Schich, asking about datasets for our project. He said: \n",
    "\n",
    "-we do not have a record of social interactions between artists at the corpus scale. The closest thing is: co-exhibition networks, which you may already know from the work of Fraiberger et al. (incl. Laszlo Barabasi). (http://genetics.bwh.harvard.edu/courses/Biophysics205/Papers/All_papers/Fraiberger_2018.pdf page 2) The issue there is that the network is short, circa1985 to 2020.\n",
    "\n",
    "-Hyperlink networks (I guess WikiLinks, Pageranks and such), such as those found in Wikipedia are obviously beset with all kinds of issues, even though they do recapitulate the evolution of conventional style periods pretty well (cf. the work of Doron Goldfarb et al.. incl. myself). More locally speaking, it i a core topic in art history to shed light on the social network of artists and their patrons, but this does not lend itself to quantitative analysis. \n",
    "\n",
    "-I personally have done a visualization for Max Planck, based on the social network of 5500 individuals related to the Roman Baroque (https://zuccaro.schich.info/), which did reveal another issue, which is that for painters, art historians tend to research family relationships (more cliques), while for architects they focus on business relationships (more hubs). But here you got the inverse problem that there is not much information on the paintings\n",
    "\n",
    "-There is a question/issue he raised from this: \"Should we really assume social interaction influencing the styles of artists? Note that this may substantially underestimate the plasticity of the human brain/mind! It is like assuming that cellists only hang out with cellists, when we all know that grunge bands in Seatlle all did hang out together and missing a bassist. Meanwhile we do have evidence that artists such as Rubens did routinely hang out with different(!) artists, who could serve clients with different genres and if necessary styles. Bramante did build Gothic in Milan and Renaissance style in Rome at the same time. Rubens would call in Elsheimer to do miniatures, etc. And since the mid 19th century, all artists in the Western scene were essentially familiar, not only with the same corpus of classic artists and their works, but also with the contemporary production. Large art exhibitions in Paris literally drew millions of people each year in the mid 19th century (think Burning Man or SXSW today). So it is save to say that most artists of note were familiar with a great number of styles. Styles may bifurcate. for artists the opposite may be true (cf. run DMC meets Aerosmith => https://www.youtube.com/watch?v=4B_UYYPb-Gk). If I were you, I'd turn the question around, pointing into the opposite direction: **If two artists have similar style, can we find traces that they (eventually) knew each other**?\" He said influence is B.S. (literally) and there's 100 times more evidence for similarity than influence between two artworks, and suggested answering \"does style lead to social interaction?\"\n",
    "\n",
    "-\"Here is how this question can be attacked with the available data: The standard \"corpus\" for artists is their \"catalog raisonne\", i.e. the catalog of all their works, which does not exist for all artists and is typically a lot of work, sold in expensive books. We are a long way from a comprehensive dataset like this. Yet, for the purpose of a more limited project, you could use general conventional style similarity from the usual suspect databases (Wikiart, Art500k, etc.). As a proxy of social interaction, you could use the hyperlink and/or wikidata links connected to the same artists. Even though these two sources are limited, you could still compare the two graphs as in \"Wikipedia connection\" vs. \"visual similarity\".\n",
    "\n",
    "We have recently published a paper on general similarity using compression ensembles, using a subset of art500k/Wikiart, which is essentially 65k paintings with a reliably year as a data. We have also used the first 100 days of the hic et nunc NFT art platform (which coincidentally you get both social interaction and painting information). See \"Availability of data and materials\" in https://epjdatascience.springeropen.com/articles/10.1140/epjds/s13688-023-00397-3#Sec21 \"\n",
    "\n",
    "So this could be interesting to think about\n",
    "</p>\n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Complete dataset (Combination of WikiArt and Art500k)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data preparation is done in the GitHub repository [PainterPalette](https://github.com/me9hanics/PainterPalette). The final dataset is available as \"artists.csv\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>artist</th>\n",
       "      <th>Nationality</th>\n",
       "      <th>birth_place</th>\n",
       "      <th>birth_year</th>\n",
       "      <th>styles</th>\n",
       "      <th>styles_extended</th>\n",
       "      <th>StylesYears</th>\n",
       "      <th>StylesCount</th>\n",
       "      <th>PlacesCount</th>\n",
       "      <th>Contemporary</th>\n",
       "      <th>...</th>\n",
       "      <th>FirstYear</th>\n",
       "      <th>LastYear</th>\n",
       "      <th>Places</th>\n",
       "      <th>PlacesYears</th>\n",
       "      <th>PaintingSchool</th>\n",
       "      <th>Influencedby</th>\n",
       "      <th>Influencedon</th>\n",
       "      <th>Pupils</th>\n",
       "      <th>Teachers</th>\n",
       "      <th>FriendsandCoworkers</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Ad Reinhardt</td>\n",
       "      <td>American</td>\n",
       "      <td>Buffalo</td>\n",
       "      <td>1913.0</td>\n",
       "      <td>Abstract Art, Abstract Expressionism, Color Fi...</td>\n",
       "      <td>{Abstract Art:15},{Abstract Expressionism:5},{...</td>\n",
       "      <td>Expressionism:1944-1946,Abstract Art:1937-1941...</td>\n",
       "      <td>{Expressionism:7}, {Abstract Art:15}, {Color F...</td>\n",
       "      <td>{New York City:29},{NY:31},{US:32},{Buffalo:2}...</td>\n",
       "      <td>No</td>\n",
       "      <td>...</td>\n",
       "      <td>1937.0</td>\n",
       "      <td>1966.0</td>\n",
       "      <td>US, NY, Canberra, Fort Worth, Buffalo, Austral...</td>\n",
       "      <td>New York City:1938-1966,NY:1938-1966,US:1938-1...</td>\n",
       "      <td>New York School,American Abstract Artists,Iras...</td>\n",
       "      <td>Piet Mondrian,Kazimir Malevich,Josef Albers,</td>\n",
       "      <td>Donald Judd,Barnett Newman,Mark Rothko,Frank S...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Jackson Pollock,</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Adnan Coker</td>\n",
       "      <td>Turkish</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Abstract Art, Abstract Expressionism</td>\n",
       "      <td>{Abstract Art:25},{Abstract Expressionism:3}</td>\n",
       "      <td>Abstract Art:1992-2008,Abstract Expressionism:...</td>\n",
       "      <td>{Abstract Art:25}, {Abstract Expressionism:3}</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Yes</td>\n",
       "      <td>...</td>\n",
       "      <td>1968.0</td>\n",
       "      <td>2008.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Akkitham Narayanan</td>\n",
       "      <td>Indian</td>\n",
       "      <td>Kerala</td>\n",
       "      <td>1939.0</td>\n",
       "      <td>Abstract Art</td>\n",
       "      <td>{Abstract Art:17}</td>\n",
       "      <td>Abstract Art:1974-1974</td>\n",
       "      <td>{Abstract Art:17}</td>\n",
       "      <td>NaN</td>\n",
       "      <td>No</td>\n",
       "      <td>...</td>\n",
       "      <td>1974.0</td>\n",
       "      <td>1974.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Alberto Magnelli</td>\n",
       "      <td>Italian,French</td>\n",
       "      <td>Florence</td>\n",
       "      <td>1888.0</td>\n",
       "      <td>Abstract Art, Art Nouveau (Modern), Cubism, Ex...</td>\n",
       "      <td>{Abstract Art:19},{Art Nouveau (Modern):2},{Cu...</td>\n",
       "      <td>Abstract Art:1916-1971,Cubism:1914-1935,Metaph...</td>\n",
       "      <td>{Abstract Art:21}, {Cubism:10}, {Metaphysical ...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>No</td>\n",
       "      <td>...</td>\n",
       "      <td>1909.0</td>\n",
       "      <td>1971.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Abstraction-Création</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Alekos Kontopoulos</td>\n",
       "      <td>Greek</td>\n",
       "      <td>Lamia</td>\n",
       "      <td>1904.0</td>\n",
       "      <td>Abstract Art, Cubism, Expressionism, Post-Impr...</td>\n",
       "      <td>{Abstract Art:26},{Cubism:5},{Expressionism:10...</td>\n",
       "      <td>Post-Impressionism:1932-1955,Expressionism:193...</td>\n",
       "      <td>{Post-Impressionism:8}, {Expressionism:11}, {R...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>No</td>\n",
       "      <td>...</td>\n",
       "      <td>1931.0</td>\n",
       "      <td>1974.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2661</th>\n",
       "      <td>Gohar Fermanyan</td>\n",
       "      <td>Armenian</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Post-Impressionism</td>\n",
       "      <td>{Post-Impressionism:3}</td>\n",
       "      <td>NaN</td>\n",
       "      <td>{Post-Impressionism:3}</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2662</th>\n",
       "      <td>JAROSLAV KELUC</td>\n",
       "      <td>Czech</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Impressionism</td>\n",
       "      <td>{Impressionism:33}</td>\n",
       "      <td>Impressionism:1949-1979</td>\n",
       "      <td>{Impressionism:33}</td>\n",
       "      <td>NaN</td>\n",
       "      <td>No</td>\n",
       "      <td>...</td>\n",
       "      <td>1949.0</td>\n",
       "      <td>1979.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2663</th>\n",
       "      <td>Ding Yi</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Suixi County, Anhui</td>\n",
       "      <td>150.0</td>\n",
       "      <td>Maximalism</td>\n",
       "      <td>{Maximalism:29}</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>1989.0</td>\n",
       "      <td>1991.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2664</th>\n",
       "      <td>Phase 2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Street art</td>\n",
       "      <td>{Street art:13}</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>{New York:1},{United States:1}</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>New York, United States</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2665</th>\n",
       "      <td>TRACY 168</td>\n",
       "      <td>American</td>\n",
       "      <td>New York City</td>\n",
       "      <td>1958.0</td>\n",
       "      <td>Street art</td>\n",
       "      <td>{Street art:11}</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>{New York:1}</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>1987.0</td>\n",
       "      <td>1987.0</td>\n",
       "      <td>New York</td>\n",
       "      <td>New York:1987-1987</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2666 rows × 24 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                  artist     Nationality          birth_place  birth_year  \\\n",
       "0           Ad Reinhardt        American              Buffalo      1913.0   \n",
       "1            Adnan Coker         Turkish                  NaN         NaN   \n",
       "2     Akkitham Narayanan          Indian               Kerala      1939.0   \n",
       "3       Alberto Magnelli  Italian,French             Florence      1888.0   \n",
       "4     Alekos Kontopoulos           Greek                Lamia      1904.0   \n",
       "...                  ...             ...                  ...         ...   \n",
       "2661     Gohar Fermanyan        Armenian                  NaN         NaN   \n",
       "2662      JAROSLAV KELUC           Czech                  NaN         NaN   \n",
       "2663             Ding Yi             NaN  Suixi County, Anhui       150.0   \n",
       "2664             Phase 2             NaN                  NaN         NaN   \n",
       "2665           TRACY 168        American        New York City      1958.0   \n",
       "\n",
       "                                                 styles  \\\n",
       "0     Abstract Art, Abstract Expressionism, Color Fi...   \n",
       "1                  Abstract Art, Abstract Expressionism   \n",
       "2                                          Abstract Art   \n",
       "3     Abstract Art, Art Nouveau (Modern), Cubism, Ex...   \n",
       "4     Abstract Art, Cubism, Expressionism, Post-Impr...   \n",
       "...                                                 ...   \n",
       "2661                                 Post-Impressionism   \n",
       "2662                                      Impressionism   \n",
       "2663                                         Maximalism   \n",
       "2664                                         Street art   \n",
       "2665                                         Street art   \n",
       "\n",
       "                                        styles_extended  \\\n",
       "0     {Abstract Art:15},{Abstract Expressionism:5},{...   \n",
       "1          {Abstract Art:25},{Abstract Expressionism:3}   \n",
       "2                                     {Abstract Art:17}   \n",
       "3     {Abstract Art:19},{Art Nouveau (Modern):2},{Cu...   \n",
       "4     {Abstract Art:26},{Cubism:5},{Expressionism:10...   \n",
       "...                                                 ...   \n",
       "2661                             {Post-Impressionism:3}   \n",
       "2662                                 {Impressionism:33}   \n",
       "2663                                    {Maximalism:29}   \n",
       "2664                                    {Street art:13}   \n",
       "2665                                    {Street art:11}   \n",
       "\n",
       "                                            StylesYears  \\\n",
       "0     Expressionism:1944-1946,Abstract Art:1937-1941...   \n",
       "1     Abstract Art:1992-2008,Abstract Expressionism:...   \n",
       "2                                Abstract Art:1974-1974   \n",
       "3     Abstract Art:1916-1971,Cubism:1914-1935,Metaph...   \n",
       "4     Post-Impressionism:1932-1955,Expressionism:193...   \n",
       "...                                                 ...   \n",
       "2661                                                NaN   \n",
       "2662                            Impressionism:1949-1979   \n",
       "2663                                                NaN   \n",
       "2664                                                NaN   \n",
       "2665                                                NaN   \n",
       "\n",
       "                                            StylesCount  \\\n",
       "0     {Expressionism:7}, {Abstract Art:15}, {Color F...   \n",
       "1         {Abstract Art:25}, {Abstract Expressionism:3}   \n",
       "2                                     {Abstract Art:17}   \n",
       "3     {Abstract Art:21}, {Cubism:10}, {Metaphysical ...   \n",
       "4     {Post-Impressionism:8}, {Expressionism:11}, {R...   \n",
       "...                                                 ...   \n",
       "2661                             {Post-Impressionism:3}   \n",
       "2662                                 {Impressionism:33}   \n",
       "2663                                                NaN   \n",
       "2664                                                NaN   \n",
       "2665                                                NaN   \n",
       "\n",
       "                                            PlacesCount Contemporary  ...  \\\n",
       "0     {New York City:29},{NY:31},{US:32},{Buffalo:2}...           No  ...   \n",
       "1                                                   NaN          Yes  ...   \n",
       "2                                                   NaN           No  ...   \n",
       "3                                                   NaN           No  ...   \n",
       "4                                                   NaN           No  ...   \n",
       "...                                                 ...          ...  ...   \n",
       "2661                                                NaN          NaN  ...   \n",
       "2662                                                NaN           No  ...   \n",
       "2663                                                NaN          NaN  ...   \n",
       "2664                     {New York:1},{United States:1}          NaN  ...   \n",
       "2665                                       {New York:1}          NaN  ...   \n",
       "\n",
       "     FirstYear LastYear                                             Places  \\\n",
       "0       1937.0   1966.0  US, NY, Canberra, Fort Worth, Buffalo, Austral...   \n",
       "1       1968.0   2008.0                                                NaN   \n",
       "2       1974.0   1974.0                                                NaN   \n",
       "3       1909.0   1971.0                                                NaN   \n",
       "4       1931.0   1974.0                                                NaN   \n",
       "...        ...      ...                                                ...   \n",
       "2661       NaN      NaN                                                NaN   \n",
       "2662    1949.0   1979.0                                                NaN   \n",
       "2663    1989.0   1991.0                                                NaN   \n",
       "2664       NaN      NaN                            New York, United States   \n",
       "2665    1987.0   1987.0                                           New York   \n",
       "\n",
       "                                            PlacesYears  \\\n",
       "0     New York City:1938-1966,NY:1938-1966,US:1938-1...   \n",
       "1                                                   NaN   \n",
       "2                                                   NaN   \n",
       "3                                                   NaN   \n",
       "4                                                   NaN   \n",
       "...                                                 ...   \n",
       "2661                                                NaN   \n",
       "2662                                                NaN   \n",
       "2663                                                NaN   \n",
       "2664                                                NaN   \n",
       "2665                                 New York:1987-1987   \n",
       "\n",
       "                                         PaintingSchool  \\\n",
       "0     New York School,American Abstract Artists,Iras...   \n",
       "1                                                   NaN   \n",
       "2                                                   NaN   \n",
       "3                                  Abstraction-Création   \n",
       "4                                                   NaN   \n",
       "...                                                 ...   \n",
       "2661                                                NaN   \n",
       "2662                                                NaN   \n",
       "2663                                                NaN   \n",
       "2664                                                NaN   \n",
       "2665                                                NaN   \n",
       "\n",
       "                                      Influencedby  \\\n",
       "0     Piet Mondrian,Kazimir Malevich,Josef Albers,   \n",
       "1                                              NaN   \n",
       "2                                              NaN   \n",
       "3                                              NaN   \n",
       "4                                              NaN   \n",
       "...                                            ...   \n",
       "2661                                           NaN   \n",
       "2662                                           NaN   \n",
       "2663                                           NaN   \n",
       "2664                                           NaN   \n",
       "2665                                           NaN   \n",
       "\n",
       "                                           Influencedon Pupils Teachers  \\\n",
       "0     Donald Judd,Barnett Newman,Mark Rothko,Frank S...    NaN      NaN   \n",
       "1                                                   NaN    NaN      NaN   \n",
       "2                                                   NaN    NaN      NaN   \n",
       "3                                                   NaN    NaN      NaN   \n",
       "4                                                   NaN    NaN      NaN   \n",
       "...                                                 ...    ...      ...   \n",
       "2661                                                NaN    NaN      NaN   \n",
       "2662                                                NaN    NaN      NaN   \n",
       "2663                                                NaN    NaN      NaN   \n",
       "2664                                                NaN    NaN      NaN   \n",
       "2665                                                NaN    NaN      NaN   \n",
       "\n",
       "     FriendsandCoworkers  \n",
       "0       Jackson Pollock,  \n",
       "1                    NaN  \n",
       "2                    NaN  \n",
       "3                    NaN  \n",
       "4                    NaN  \n",
       "...                  ...  \n",
       "2661                 NaN  \n",
       "2662                 NaN  \n",
       "2663                 NaN  \n",
       "2664                 NaN  \n",
       "2665                 NaN  \n",
       "\n",
       "[2666 rows x 24 columns]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "artists = pd.read_csv('https://raw.githubusercontent.com/me9hanics/PainterPalette/main/datasets/artists.csv')\n",
    "artists"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Other: WikiArt data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load the cleaned paintings data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Length: 175313\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>artist</th>\n",
       "      <th>style</th>\n",
       "      <th>genre</th>\n",
       "      <th>movement</th>\n",
       "      <th>tags</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Andrei Rublev</td>\n",
       "      <td>Moscow school of icon painting</td>\n",
       "      <td>religious painting</td>\n",
       "      <td>Byzantine Art</td>\n",
       "      <td>['Christianity', 'saints-and-apostles', 'angel...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Andrei Rublev</td>\n",
       "      <td>Moscow school of icon painting</td>\n",
       "      <td>religious painting</td>\n",
       "      <td>Byzantine Art</td>\n",
       "      <td>['Christianity', 'Old-Testament', 'Daniel', 'p...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Andrei Rublev</td>\n",
       "      <td>Moscow school of icon painting</td>\n",
       "      <td>miniature</td>\n",
       "      <td>Byzantine Art</td>\n",
       "      <td>['Christianity', 'saints-and-apostles', 'Khitr...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Andrei Rublev</td>\n",
       "      <td>Moscow school of icon painting</td>\n",
       "      <td>religious painting</td>\n",
       "      <td>Byzantine Art</td>\n",
       "      <td>['Christianity', 'saints-and-apostles', 'St.-L...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Andrei Rublev</td>\n",
       "      <td>Moscow school of icon painting</td>\n",
       "      <td>miniature</td>\n",
       "      <td>Byzantine Art</td>\n",
       "      <td>['Christianity', 'arts-and-crafts', 'saints-an...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          artist                           style               genre  \\\n",
       "0  Andrei Rublev  Moscow school of icon painting  religious painting   \n",
       "1  Andrei Rublev  Moscow school of icon painting  religious painting   \n",
       "2  Andrei Rublev  Moscow school of icon painting           miniature   \n",
       "3  Andrei Rublev  Moscow school of icon painting  religious painting   \n",
       "4  Andrei Rublev  Moscow school of icon painting           miniature   \n",
       "\n",
       "        movement                                               tags  \n",
       "0  Byzantine Art  ['Christianity', 'saints-and-apostles', 'angel...  \n",
       "1  Byzantine Art  ['Christianity', 'Old-Testament', 'Daniel', 'p...  \n",
       "2  Byzantine Art  ['Christianity', 'saints-and-apostles', 'Khitr...  \n",
       "3  Byzantine Art  ['Christianity', 'saints-and-apostles', 'St.-L...  \n",
       "4  Byzantine Art  ['Christianity', 'arts-and-crafts', 'saints-an...  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wa_paintings = pd.read_csv('https://raw.githubusercontent.com/me9hanics/PainterPalette/main/datasets/wikiart_paintings_refined.csv') \n",
    "print(\"Length:\", len(wa_paintings))\n",
    "wa_paintings.head() #Consider dropping style: \"Unknown\" "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load the grouped data: artists grouped by style"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Length: 7646 \n",
      " Number of groups with only 1 count: 1115\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>style</th>\n",
       "      <th>artist</th>\n",
       "      <th>movement</th>\n",
       "      <th>count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2963</th>\n",
       "      <td>Impressionism</td>\n",
       "      <td>Claude Monet</td>\n",
       "      <td>Impressionism</td>\n",
       "      <td>1341</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5468</th>\n",
       "      <td>Realism</td>\n",
       "      <td>Claude Monet</td>\n",
       "      <td>Impressionism</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7041</th>\n",
       "      <td>Unknown</td>\n",
       "      <td>Claude Monet</td>\n",
       "      <td>Impressionism</td>\n",
       "      <td>12</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>462</th>\n",
       "      <td>Academicism</td>\n",
       "      <td>Claude Monet</td>\n",
       "      <td>Impressionism</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3339</th>\n",
       "      <td>Japonism</td>\n",
       "      <td>Claude Monet</td>\n",
       "      <td>Impressionism</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              style        artist       movement  count\n",
       "2963  Impressionism  Claude Monet  Impressionism   1341\n",
       "5468        Realism  Claude Monet  Impressionism     12\n",
       "7041        Unknown  Claude Monet  Impressionism     12\n",
       "462     Academicism  Claude Monet  Impressionism      1\n",
       "3339       Japonism  Claude Monet  Impressionism      1"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wa_grouped = pd.read_csv('https://raw.githubusercontent.com/me9hanics/PainterPalette/main/datasets/wikiart_artists_styles_grouped.csv')\n",
    "print(\"Length:\", len(wa_grouped), \"\\n\", \"Number of groups with only 1 count:\", len(wa_grouped[wa_grouped['count']==min(wa_grouped['count'])]))\n",
    "wa_grouped[wa_grouped['artist'].str.contains(\"Monet\")].sort_values(by=['count'], ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Birthplaces, birth years"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>artist</th>\n",
       "      <th>styles</th>\n",
       "      <th>movement</th>\n",
       "      <th>styles_extended</th>\n",
       "      <th>pictures_count</th>\n",
       "      <th>birth_place</th>\n",
       "      <th>birth_year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Ad Reinhardt</td>\n",
       "      <td>Abstract Art, Abstract Expressionism, Color Fi...</td>\n",
       "      <td>Abstract Expressionism</td>\n",
       "      <td>{Abstract Art:15},{Abstract Expressionism:5},{...</td>\n",
       "      <td>52</td>\n",
       "      <td>Buffalo</td>\n",
       "      <td>1913.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Adnan Coker</td>\n",
       "      <td>Abstract Art, Abstract Expressionism</td>\n",
       "      <td>Abstract Art</td>\n",
       "      <td>{Abstract Art:25},{Abstract Expressionism:3}</td>\n",
       "      <td>28</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Akkitham Narayanan</td>\n",
       "      <td>Abstract Art</td>\n",
       "      <td>Abstract Art</td>\n",
       "      <td>{Abstract Art:17}</td>\n",
       "      <td>17</td>\n",
       "      <td>Kerala</td>\n",
       "      <td>1939.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Alberto Magnelli</td>\n",
       "      <td>Abstract Art, Art Nouveau (Modern), Cubism, Ex...</td>\n",
       "      <td>Abstract Art</td>\n",
       "      <td>{Abstract Art:19},{Art Nouveau (Modern):2},{Cu...</td>\n",
       "      <td>35</td>\n",
       "      <td>Florence</td>\n",
       "      <td>1888.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Alekos Kontopoulos</td>\n",
       "      <td>Abstract Art, Cubism, Expressionism, Post-Impr...</td>\n",
       "      <td>Social Realism</td>\n",
       "      <td>{Abstract Art:26},{Cubism:5},{Expressionism:10...</td>\n",
       "      <td>79</td>\n",
       "      <td>Lamia</td>\n",
       "      <td>1904.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3198</th>\n",
       "      <td>Serhij Schyschko</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>Academic Art</td>\n",
       "      <td>{Unknown:9}</td>\n",
       "      <td>9</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3199</th>\n",
       "      <td>Vudon Baklytsky</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>Soviet Nonconformist Art</td>\n",
       "      <td>{Unknown:46}</td>\n",
       "      <td>46</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3200</th>\n",
       "      <td>Wolfgang Tillmans</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>Contemporary</td>\n",
       "      <td>{Unknown:9}</td>\n",
       "      <td>9</td>\n",
       "      <td>Remscheid</td>\n",
       "      <td>1968.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3201</th>\n",
       "      <td>Wu Daozi</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>Tang Dynasty (618–907)</td>\n",
       "      <td>{Unknown:8}</td>\n",
       "      <td>8</td>\n",
       "      <td>Chang'an</td>\n",
       "      <td>680.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3202</th>\n",
       "      <td>Olowe of Ise</td>\n",
       "      <td>Yoruba</td>\n",
       "      <td>Native Art</td>\n",
       "      <td>{Yoruba:13}</td>\n",
       "      <td>13</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3203 rows × 7 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                  artist                                             styles  \\\n",
       "0           Ad Reinhardt  Abstract Art, Abstract Expressionism, Color Fi...   \n",
       "1            Adnan Coker               Abstract Art, Abstract Expressionism   \n",
       "2     Akkitham Narayanan                                       Abstract Art   \n",
       "3       Alberto Magnelli  Abstract Art, Art Nouveau (Modern), Cubism, Ex...   \n",
       "4     Alekos Kontopoulos  Abstract Art, Cubism, Expressionism, Post-Impr...   \n",
       "...                  ...                                                ...   \n",
       "3198    Serhij Schyschko                                            Unknown   \n",
       "3199     Vudon Baklytsky                                            Unknown   \n",
       "3200   Wolfgang Tillmans                                            Unknown   \n",
       "3201            Wu Daozi                                            Unknown   \n",
       "3202        Olowe of Ise                                             Yoruba   \n",
       "\n",
       "                      movement  \\\n",
       "0       Abstract Expressionism   \n",
       "1                 Abstract Art   \n",
       "2                 Abstract Art   \n",
       "3                 Abstract Art   \n",
       "4               Social Realism   \n",
       "...                        ...   \n",
       "3198              Academic Art   \n",
       "3199  Soviet Nonconformist Art   \n",
       "3200              Contemporary   \n",
       "3201    Tang Dynasty (618–907)   \n",
       "3202                Native Art   \n",
       "\n",
       "                                        styles_extended  pictures_count  \\\n",
       "0     {Abstract Art:15},{Abstract Expressionism:5},{...              52   \n",
       "1          {Abstract Art:25},{Abstract Expressionism:3}              28   \n",
       "2                                     {Abstract Art:17}              17   \n",
       "3     {Abstract Art:19},{Art Nouveau (Modern):2},{Cu...              35   \n",
       "4     {Abstract Art:26},{Cubism:5},{Expressionism:10...              79   \n",
       "...                                                 ...             ...   \n",
       "3198                                        {Unknown:9}               9   \n",
       "3199                                       {Unknown:46}              46   \n",
       "3200                                        {Unknown:9}               9   \n",
       "3201                                        {Unknown:8}               8   \n",
       "3202                                        {Yoruba:13}              13   \n",
       "\n",
       "     birth_place  birth_year  \n",
       "0        Buffalo      1913.0  \n",
       "1            NaN         NaN  \n",
       "2         Kerala      1939.0  \n",
       "3       Florence      1888.0  \n",
       "4          Lamia      1904.0  \n",
       "...          ...         ...  \n",
       "3198         NaN         NaN  \n",
       "3199         NaN         NaN  \n",
       "3200   Remscheid      1968.0  \n",
       "3201    Chang'an       680.0  \n",
       "3202         NaN         NaN  \n",
       "\n",
       "[3203 rows x 7 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "artists_A = pd.read_csv('https://raw.githubusercontent.com/me9hanics/PainterPalette/main/datasets/wikiart_artists.csv')\n",
    "artists_A"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Art500K"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First dataset (from official website)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>author_name</th>\n",
       "      <th>Genre</th>\n",
       "      <th>Style</th>\n",
       "      <th>Nationality</th>\n",
       "      <th>PaintingSchool</th>\n",
       "      <th>ArtMovement</th>\n",
       "      <th>Date</th>\n",
       "      <th>Influencedby</th>\n",
       "      <th>Influencedon</th>\n",
       "      <th>Tag</th>\n",
       "      <th>Pupils</th>\n",
       "      <th>Location</th>\n",
       "      <th>Teachers</th>\n",
       "      <th>FriendsandCoworkers</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Gustave Courbet</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Auguste Rodin</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Frida Kahlo</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Banksy</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>in a settlement in Palestine in the middle east</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>El Greco</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>ca. 1610-1614</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>El Greco</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Diego Rivera</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>Claude Monet</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>Francisco Goya</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>Francisco Goya</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       author_name Genre Style Nationality PaintingSchool ArtMovement  \\\n",
       "0  Gustave Courbet   NaN   NaN         NaN            NaN         NaN   \n",
       "1    Auguste Rodin   NaN   NaN         NaN            NaN         NaN   \n",
       "2      Frida Kahlo   NaN   NaN         NaN            NaN         NaN   \n",
       "3           Banksy   NaN   NaN         NaN            NaN         NaN   \n",
       "4         El Greco   NaN   NaN         NaN            NaN         NaN   \n",
       "5         El Greco   NaN   NaN         NaN            NaN         NaN   \n",
       "6     Diego Rivera   NaN   NaN         NaN            NaN         NaN   \n",
       "7     Claude Monet   NaN   NaN         NaN            NaN         NaN   \n",
       "8   Francisco Goya   NaN   NaN         NaN            NaN         NaN   \n",
       "9   Francisco Goya   NaN   NaN         NaN            NaN         NaN   \n",
       "\n",
       "            Date Influencedby Influencedon  Tag Pupils  \\\n",
       "0            NaN          NaN          NaN  NaN    NaN   \n",
       "1            NaN          NaN          NaN  NaN    NaN   \n",
       "2            NaN          NaN          NaN  NaN    NaN   \n",
       "3            NaN          NaN          NaN  NaN    NaN   \n",
       "4  ca. 1610-1614          NaN          NaN  NaN    NaN   \n",
       "5            NaN          NaN          NaN  NaN    NaN   \n",
       "6            NaN          NaN          NaN  NaN    NaN   \n",
       "7            NaN          NaN          NaN  NaN    NaN   \n",
       "8            NaN          NaN          NaN  NaN    NaN   \n",
       "9            NaN          NaN          NaN  NaN    NaN   \n",
       "\n",
       "                                          Location Teachers  \\\n",
       "0                                              NaN      NaN   \n",
       "1                                              NaN      NaN   \n",
       "2                                              NaN      NaN   \n",
       "3  in a settlement in Palestine in the middle east      NaN   \n",
       "4                                              NaN      NaN   \n",
       "5                                              NaN      NaN   \n",
       "6                                              NaN      NaN   \n",
       "7                                              NaN      NaN   \n",
       "8                                              NaN      NaN   \n",
       "9                                              NaN      NaN   \n",
       "\n",
       "  FriendsandCoworkers  \n",
       "0                 NaN  \n",
       "1                 NaN  \n",
       "2                 NaN  \n",
       "3                 NaN  \n",
       "4                 NaN  \n",
       "5                 NaN  \n",
       "6                 NaN  \n",
       "7                 NaN  \n",
       "8                 NaN  \n",
       "9                 NaN  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "art500k = pd.read_csv('https://raw.githubusercontent.com/me9hanics/PainterPalette/main/datasets/art500k_paintings_cleaned.csv')\n",
    "(art500k[0:10])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>artist</th>\n",
       "      <th>Nationality</th>\n",
       "      <th>PaintingSchool</th>\n",
       "      <th>ArtMovement</th>\n",
       "      <th>Influencedby</th>\n",
       "      <th>Influencedon</th>\n",
       "      <th>Pupils</th>\n",
       "      <th>Teachers</th>\n",
       "      <th>FriendsandCoworkers</th>\n",
       "      <th>FirstYear</th>\n",
       "      <th>LastYear</th>\n",
       "      <th>Places</th>\n",
       "      <th>PlacesYears</th>\n",
       "      <th>StylesYears</th>\n",
       "      <th>StylesCount</th>\n",
       "      <th>PlacesCount</th>\n",
       "      <th>Contemporary</th>\n",
       "      <th>Type</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Gustave Courbet</td>\n",
       "      <td>French</td>\n",
       "      <td>NaN</td>\n",
       "      <td>{Realism:272}</td>\n",
       "      <td>Rembrandt,Caravaggio,Diego Velazquez,Peter Pau...</td>\n",
       "      <td>Edouard Manet,Claude Monet,Pierre-Auguste Reno...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1830.0</td>\n",
       "      <td>1877.0</td>\n",
       "      <td>London, Montpellier, Moscow, CA, UK, Norway, D...</td>\n",
       "      <td>France:1841-1876,Switzerland:1844-1874,Lille:1...</td>\n",
       "      <td>Realism:1835-1877,Romanticism:1830-1849</td>\n",
       "      <td>{Realism:257}, {Romanticism:13}</td>\n",
       "      <td>{France:88},{Switzerland:7},{Lille:8},{Paris:4...</td>\n",
       "      <td>No</td>\n",
       "      <td>Painting/Sculpture</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Auguste Rodin</td>\n",
       "      <td>French</td>\n",
       "      <td>NaN</td>\n",
       "      <td>{Modern art:3},{Impressionism:91}</td>\n",
       "      <td>Michelangelo,Donatello,</td>\n",
       "      <td>Georgia O'Keeffe,Man Ray,Aristide Maillol,Olex...</td>\n",
       "      <td>Constantin Brancusi,</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1865.0</td>\n",
       "      <td>1985.0</td>\n",
       "      <td>London, CA, UK, Switzerland, Lisbon, US, Germa...</td>\n",
       "      <td>France:1865-1889,Paris:1865-1898,CA:1891-1891,...</td>\n",
       "      <td>Impressionism:1865-1905</td>\n",
       "      <td>{Impressionism:90}</td>\n",
       "      <td>{France:52},{Paris:15},{Brussels:2},{Belgium:1...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Painting/Sculpture</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Frida Kahlo</td>\n",
       "      <td>Mexican</td>\n",
       "      <td>NaN</td>\n",
       "      <td>{Naïve Art (Primitivism),Surrealism:99}</td>\n",
       "      <td>Amedeo Modigliani,Diego Rivera,Jose Clemente O...</td>\n",
       "      <td>Judy Chicago,Georgia O'Keeffe,Feminist Art,</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1922.0</td>\n",
       "      <td>1954.0</td>\n",
       "      <td>CA, LA, New York, US, New Orleans, Washington ...</td>\n",
       "      <td>Mexico:1927-1954,San Francisco:1931-1933,Mexic...</td>\n",
       "      <td>Naïve Art (Primitivism):1922-1954,Surrealism:1...</td>\n",
       "      <td>{Naïve Art (Primitivism):99}, {Surrealism:15}</td>\n",
       "      <td>{Mexico:50},{San Francisco:6},{New York:4},{Me...</td>\n",
       "      <td>No</td>\n",
       "      <td>Painting/Sculpture</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Banksy</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2011.0</td>\n",
       "      <td>2011.0</td>\n",
       "      <td>Los Angeles, London, UK, Palestine, California...</td>\n",
       "      <td>London:2011-2011,UK:2011-2011</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>{Palestine:1},{Los Angeles:3},{California:3},{...</td>\n",
       "      <td>Yes</td>\n",
       "      <td>Painting/Sculpture</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>El Greco</td>\n",
       "      <td>Spanish,Greek</td>\n",
       "      <td>Cretan School</td>\n",
       "      <td>{Spanish Renaissance:1},{Renaissance:2},{Manne...</td>\n",
       "      <td>Byzantine Art,</td>\n",
       "      <td>Expressionism,Cubism,Eugene Delacroix,Edouard ...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Titian,</td>\n",
       "      <td>Giulio Clovio,</td>\n",
       "      <td>1568.0</td>\n",
       "      <td>1614.0</td>\n",
       "      <td>Seville, London, Illescas, Romania, Moscow, Gr...</td>\n",
       "      <td>Spain:1577-1599,London:1600-1600,UK:1600-1600,...</td>\n",
       "      <td>Mannerism (Late Renaissance):1568-1600</td>\n",
       "      <td>{Renaissance:2}, {XVI CenturySpanish Painting:...</td>\n",
       "      <td>{Spain:75},{Boston:1},{MA:1},{US:27},{Museo de...</td>\n",
       "      <td>No</td>\n",
       "      <td>Painting/Sculpture</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>Diego Rivera</td>\n",
       "      <td>Mexican</td>\n",
       "      <td>Mexican Mural Renaissance,La Ruche</td>\n",
       "      <td>{Social Realism,Muralism:146}</td>\n",
       "      <td>Marc Chagall,Robert Delaunay,</td>\n",
       "      <td>Frida Kahlo,Pedro Coronel,Vlady,</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Amedeo Modigliani,Saturnino Herran,Roberto Mon...</td>\n",
       "      <td>1904.0</td>\n",
       "      <td>1956.0</td>\n",
       "      <td>Moscow, CA, Acapulco, New York, Spain, Northam...</td>\n",
       "      <td>Acapulco:1956-1956,Mexico:1905-1956,Guerrero:1...</td>\n",
       "      <td>Cubism:1912-1916,Muralism:1922-1956,Art Deco:1...</td>\n",
       "      <td>{Post-impressionism:1}, {Cubism:19}, {Mexican ...</td>\n",
       "      <td>{France:1},{Paris:1},{Moscow:1},{Acapulco:2},{...</td>\n",
       "      <td>No</td>\n",
       "      <td>Painting/Sculpture</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Claude Monet</td>\n",
       "      <td>French</td>\n",
       "      <td>NaN</td>\n",
       "      <td>{Modern art:3},{Impressionism:1340}</td>\n",
       "      <td>Gustave Courbet,Charles-Francois Daubigny,John...</td>\n",
       "      <td>Childe Hassam,Robert Delaunay,Wassily Kandinsk...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Eugene Boudin,Charles Gleyre,</td>\n",
       "      <td>Alfred Sisley,Pierre-Auguste Renoir,Camille Pi...</td>\n",
       "      <td>1858.0</td>\n",
       "      <td>1926.0</td>\n",
       "      <td>London, Main, Moscow, Rotterdam, Giverny, CA, ...</td>\n",
       "      <td>France:1861-1924,London:1869-1889,UK:1869-1908...</td>\n",
       "      <td>Impressionist:1879-1904,Impressionism:1864-192...</td>\n",
       "      <td>{Nineteenth-Century European PaintingImpressio...</td>\n",
       "      <td>{France:79},{Giverny:1},{London:6},{UK:15},{Bo...</td>\n",
       "      <td>No</td>\n",
       "      <td>Painting/Sculpture</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            artist    Nationality                      PaintingSchool  \\\n",
       "0  Gustave Courbet         French                                 NaN   \n",
       "1    Auguste Rodin         French                                 NaN   \n",
       "2      Frida Kahlo        Mexican                                 NaN   \n",
       "3           Banksy            NaN                                 NaN   \n",
       "4         El Greco  Spanish,Greek                       Cretan School   \n",
       "5     Diego Rivera        Mexican  Mexican Mural Renaissance,La Ruche   \n",
       "6     Claude Monet         French                                 NaN   \n",
       "\n",
       "                                         ArtMovement  \\\n",
       "0                                      {Realism:272}   \n",
       "1                  {Modern art:3},{Impressionism:91}   \n",
       "2            {Naïve Art (Primitivism),Surrealism:99}   \n",
       "3                                                NaN   \n",
       "4  {Spanish Renaissance:1},{Renaissance:2},{Manne...   \n",
       "5                      {Social Realism,Muralism:146}   \n",
       "6                {Modern art:3},{Impressionism:1340}   \n",
       "\n",
       "                                        Influencedby  \\\n",
       "0  Rembrandt,Caravaggio,Diego Velazquez,Peter Pau...   \n",
       "1                            Michelangelo,Donatello,   \n",
       "2  Amedeo Modigliani,Diego Rivera,Jose Clemente O...   \n",
       "3                                                NaN   \n",
       "4                                     Byzantine Art,   \n",
       "5                      Marc Chagall,Robert Delaunay,   \n",
       "6  Gustave Courbet,Charles-Francois Daubigny,John...   \n",
       "\n",
       "                                        Influencedon                Pupils  \\\n",
       "0  Edouard Manet,Claude Monet,Pierre-Auguste Reno...                   NaN   \n",
       "1  Georgia O'Keeffe,Man Ray,Aristide Maillol,Olex...  Constantin Brancusi,   \n",
       "2        Judy Chicago,Georgia O'Keeffe,Feminist Art,                   NaN   \n",
       "3                                                NaN                   NaN   \n",
       "4  Expressionism,Cubism,Eugene Delacroix,Edouard ...                   NaN   \n",
       "5                   Frida Kahlo,Pedro Coronel,Vlady,                   NaN   \n",
       "6  Childe Hassam,Robert Delaunay,Wassily Kandinsk...                   NaN   \n",
       "\n",
       "                        Teachers  \\\n",
       "0                            NaN   \n",
       "1                            NaN   \n",
       "2                            NaN   \n",
       "3                            NaN   \n",
       "4                        Titian,   \n",
       "5                            NaN   \n",
       "6  Eugene Boudin,Charles Gleyre,   \n",
       "\n",
       "                                 FriendsandCoworkers  FirstYear  LastYear  \\\n",
       "0                                                NaN     1830.0    1877.0   \n",
       "1                                                NaN     1865.0    1985.0   \n",
       "2                                                NaN     1922.0    1954.0   \n",
       "3                                                NaN     2011.0    2011.0   \n",
       "4                                     Giulio Clovio,     1568.0    1614.0   \n",
       "5  Amedeo Modigliani,Saturnino Herran,Roberto Mon...     1904.0    1956.0   \n",
       "6  Alfred Sisley,Pierre-Auguste Renoir,Camille Pi...     1858.0    1926.0   \n",
       "\n",
       "                                              Places  \\\n",
       "0  London, Montpellier, Moscow, CA, UK, Norway, D...   \n",
       "1  London, CA, UK, Switzerland, Lisbon, US, Germa...   \n",
       "2  CA, LA, New York, US, New Orleans, Washington ...   \n",
       "3  Los Angeles, London, UK, Palestine, California...   \n",
       "4  Seville, London, Illescas, Romania, Moscow, Gr...   \n",
       "5  Moscow, CA, Acapulco, New York, Spain, Northam...   \n",
       "6  London, Main, Moscow, Rotterdam, Giverny, CA, ...   \n",
       "\n",
       "                                         PlacesYears  \\\n",
       "0  France:1841-1876,Switzerland:1844-1874,Lille:1...   \n",
       "1  France:1865-1889,Paris:1865-1898,CA:1891-1891,...   \n",
       "2  Mexico:1927-1954,San Francisco:1931-1933,Mexic...   \n",
       "3                      London:2011-2011,UK:2011-2011   \n",
       "4  Spain:1577-1599,London:1600-1600,UK:1600-1600,...   \n",
       "5  Acapulco:1956-1956,Mexico:1905-1956,Guerrero:1...   \n",
       "6  France:1861-1924,London:1869-1889,UK:1869-1908...   \n",
       "\n",
       "                                         StylesYears  \\\n",
       "0            Realism:1835-1877,Romanticism:1830-1849   \n",
       "1                            Impressionism:1865-1905   \n",
       "2  Naïve Art (Primitivism):1922-1954,Surrealism:1...   \n",
       "3                                                NaN   \n",
       "4             Mannerism (Late Renaissance):1568-1600   \n",
       "5  Cubism:1912-1916,Muralism:1922-1956,Art Deco:1...   \n",
       "6  Impressionist:1879-1904,Impressionism:1864-192...   \n",
       "\n",
       "                                         StylesCount  \\\n",
       "0                    {Realism:257}, {Romanticism:13}   \n",
       "1                                 {Impressionism:90}   \n",
       "2      {Naïve Art (Primitivism):99}, {Surrealism:15}   \n",
       "3                                                NaN   \n",
       "4  {Renaissance:2}, {XVI CenturySpanish Painting:...   \n",
       "5  {Post-impressionism:1}, {Cubism:19}, {Mexican ...   \n",
       "6  {Nineteenth-Century European PaintingImpressio...   \n",
       "\n",
       "                                         PlacesCount Contemporary  \\\n",
       "0  {France:88},{Switzerland:7},{Lille:8},{Paris:4...           No   \n",
       "1  {France:52},{Paris:15},{Brussels:2},{Belgium:1...          NaN   \n",
       "2  {Mexico:50},{San Francisco:6},{New York:4},{Me...           No   \n",
       "3  {Palestine:1},{Los Angeles:3},{California:3},{...          Yes   \n",
       "4  {Spain:75},{Boston:1},{MA:1},{US:27},{Museo de...           No   \n",
       "5  {France:1},{Paris:1},{Moscow:1},{Acapulco:2},{...           No   \n",
       "6  {France:79},{Giverny:1},{London:6},{UK:15},{Bo...           No   \n",
       "\n",
       "                 Type  \n",
       "0  Painting/Sculpture  \n",
       "1  Painting/Sculpture  \n",
       "2  Painting/Sculpture  \n",
       "3  Painting/Sculpture  \n",
       "4  Painting/Sculpture  \n",
       "5  Painting/Sculpture  \n",
       "6  Painting/Sculpture  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "art500k_artists = pd.read_csv('https://raw.githubusercontent.com/me9hanics/PainterPalette/main/datasets/art500k_artists.csv')\n",
    "art500k_artists[0:7]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating networks\n",
    "\n",
    "This is found in the networks folder, mostly in the networks.ipynb notebook.<br>\n",
    "Some work can be found in the *scipy.ipynb* notebook.<br>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}