1_Optional_Cover_Page_Generation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Synthetic Cover Page Generation\n",
    "\n",
    "This notebook is the first step in our pipeline. The aim was to use thousands of cover pages with a variety of fonts and font sizes and train a neural network to automatically recognize/extract characters from a new cover page. Due to the unfeasibility of physically taking that many photos of books, we decided to synthetically generate these pages ourselves.\n",
    "\n",
    "A brief outline of the process implemented in this notebook:\n",
    "\n",
    "1. Generate a list of possible urls hosted by the University of Rome's Italian Library website.\n",
    "2. Scrape the online library for all valid urls and get the page content.\n",
    "3. Use Beautiful Soup to extract and save content from the 'Author' and 'Title' tags as their respective lists.\n",
    "4. Save a list of common fonts (available for non-commercial use from Google Fonts library).\n",
    "5. Use a combination of (m font_styles, n font_sizes) for each (title, author) pair to generate mn cover pages in .pdf format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from bs4 import BeautifulSoup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Generate possible urls hosted by the University of Rome's Italian Library website."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "main5 = 'http://www.bibliotecaitaliana.it/indice/visualizza_scheda/bibit00000'\n",
    "main4 = 'http://www.bibliotecaitaliana.it/indice/visualizza_scheda/bibit0000'\n",
    "main3 = 'http://www.bibliotecaitaliana.it/indice/visualizza_scheda/bibit000'\n",
    "main2 = 'http://www.bibliotecaitaliana.it/indice/visualizza_scheda/bibit00'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "list1 = list(range(0,2000))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(str(list1[100]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "links = []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in list1:\n",
    "    if (len(str(i))) == 1:\n",
    "        links.append(str(main5+str(i)))\n",
    "    elif (len(str(i))) == 2:\n",
    "        links.append(str(main4+str(i)))\n",
    "    elif (len(str(i))) == 3:\n",
    "        links.append(str(main3+str(i)))\n",
    "    elif (len(str(i))) == 4:\n",
    "        links.append(str(main2+str(i)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "all_requests = []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tqdm import tqdm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Scrape the online library for all valid urls and get the page content."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### WARNING: This process will take around half an hour. The (authors,titles) data obtained by scraping are already present in the Data folder as pickled files. For help with pickle, please refer to the [documentation.](https://docs.python.org/3/library/pickle.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 2000/2000 [34:34<00:00,  1.01it/s]\n"
     ]
    }
   ],
   "source": [
    "for link in tqdm(links):\n",
    "    all_requests.append(requests.get(link))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "all_requests2 = all_requests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2000"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(all_requests2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "valid_pages = []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "for r in all_requests2:\n",
    "    if r.status_code == 200:\n",
    "        valid_pages.append(r)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Use Beautiful Soup to extract and save content from the 'Author' and 'Title' tags as their respective lists."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "soups = []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "for page in valid_pages:\n",
    "    soup = BeautifulSoup(page.content,'html.parser')\n",
    "    soups.append(soup)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "authors = []\n",
    "titles = []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "for soup in soups:\n",
    "    text_only = [text for text in soup.stripped_strings]\n",
    "    try:\n",
    "        author_index = text_only.index('Autore:')\n",
    "    except:\n",
    "        pass\n",
    "    authors.append(text_only[author_index+1])\n",
    "    try:\n",
    "        title_index = text_only.index('Titolo:')\n",
    "    except:\n",
    "        pass\n",
    "    titles.append(text_only[title_index+1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1629"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(authors)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1629"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(titles)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Store the output for future use\n",
    "\n",
    "The following two commands help to store the outputs to a pickled file. This can be used for future steps instead of scraping the website multiple times."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pickle"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('./Data/authors.pkl', 'wb') as f:\n",
    "    pickle.dump(authors, f)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('./Data/titles.pkl', 'wb') as f:\n",
    "    pickle.dump(titles, f)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use the fpdf library to generate images. More information can be found at their [website.](http://www.fpdf.org/)\n",
    "\n",
    "tqdm is a library that helps keep track of loop progress. More information about tqdm can be found [here](https://tqdm.github.io/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "from fpdf import FPDF\n",
    "from tqdm import tqdm\n",
    "import os\n",
    "import random"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Save a list of common fonts (available for non-commercial use from Google Fonts library).\n",
    "\n",
    "The fonts were manually downloaded using the [SkyFonts](http://skyfonts.com/) software. Only fonts free to use for academic purposes were downloaded."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "fonts = []"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "for root, dirs, files in os.walk(\"./Data/fonts\"):  \n",
    "    for filename in files:\n",
    "        fonts.append(filename[:-4])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "482\n"
     ]
    }
   ],
   "source": [
    "print(len(fonts))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "fname1 = './Data/fonts/'\n",
    "fname2 = '.ttf'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Use a combination of (m font_styles, n font_sizes) for each (title, author) pair to generate mn cover pages in .pdf format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_individual_cover(font, size, title, author):\n",
    "    f_path = fname1+font+fname2\n",
    "    t_height = size/3\n",
    "    output_file = './Data/sample_pdfs/'+str(title.replace(\" \",\"_\"))+'_'+str(author.replace(\" \",\"_\"))+'_'+str(font.replace(\" \",\"_\"))+\"_\"+str(size)+'.pdf'\n",
    "    \n",
    "    this_pdf = FPDF()\n",
    "    try:\n",
    "        this_pdf.add_font(family=font,style='',fname=f_path,uni=True)\n",
    "        this_pdf.add_page()\n",
    "        this_pdf.set_font(font,'',size)\n",
    "        this_pdf.write(h=t_height,txt=title)\n",
    "        this_pdf.multi_cell(w=10,h=t_height,txt=\"\\n\\n\")\n",
    "        this_pdf.write(h=t_height,txt=author)\n",
    "        this_pdf.output(output_file, 'F')\n",
    "        this_pdf.close()\n",
    "    except:\n",
    "        pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "font_sizes_range = list(range(33,96))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_cover_combos(book_title, book_author):\n",
    "    for f in random.choices(fonts,k=17):\n",
    "        sizes = random.choices(font_sizes_range,k=8)\n",
    "        for size in sizes:\n",
    "            create_individual_cover(f,size,book_title,book_author)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### WARNING : This process will take around half an hour for all the 1628 (author,title) combinations. A randomly sampled subset of 100 pdfs is present in the Data/sample_pdfs folder while their corresponding jpegs are present in the Data/sample_jpegs folder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|██████████| 1628/1628 [26:39<00:00,  1.09it/s]\n"
     ]
    }
   ],
   "source": [
    "for (author,title) in tqdm(set(zip(authors,titles))):\n",
    "    create_cover_combos(title, author)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For a user to try the pipeline we built and perform character extraction on an image, following this pdf_generation pipeline is not necessary since all the data obtained is already shared in the Data folder of this repo. But if a user wishes to generate more pdfs then this would come in handy.\n",
    "\n",
    "As we progressed, the direction of our project changed and we used a OpenCV pipeline instead of training a neural network. The major reason being that in order to perform supervised learning on this dataset, we needed a training dataset that was pre-labelled which in this scenario meant having the bounding box coordinates for each character in each cover image. Labelling an image for object detection is easier in cases where the object is singular (for e.g. detect a cat, face, animal etc.) But labelling an image with an average of 40 characters proved unfeasible due to time constraints.\n",
    "\n",
    "But towards the end of this project, we came to the conclusion that this pipeline would be extremely useful in building on the work completed by us so far. \n",
    "\n",
    "For instance, we can use a Neural Network on this data set of images, use the labelling of known output (the characters present), put them through the character extraction and prediction pipeline we built and in this way train it to learn the shapes of different characters in the same image thereby enabling it to predict characters in a new image."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}