Copy_of_PS1_Reviews.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Copy of PS1-Reviews.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/rajatha94/DBMSProject/blob/master/Copy_of_PS1_Reviews.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jc2R3x6QSIFi",
        "colab_type": "text"
      },
      "source": [
        "In this experiment, you will explore the accuracy of sentiment classificaiton using different feature representations of text documents.\n",
        "\n",
        "First, you will implement `createBasicFeatures`, which creates a sparse matrix representation of a collection of documents. For this exercise, you should have a feature for each word containing at least one alphabetic character. You may use the `numpy` and `sklearn` packages to help with implementing a sparse matrix.\n",
        "\n",
        "Then, you will implement `createFancyFeatures`, which can specify at any other features you choose to help improve performance on the classification task.\n",
        "\n",
        "The two code blocks at the end train and evaluate two models—logistic regression with L1 and L2 regularization—using your featurization functions. Besides held-out classification accuracy with 10-fold cross-validation, you will also see the features in each class given high weights by the model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AdVS67_HNRmW",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import json\n",
        "import requests\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.model_selection import cross_validate,LeaveOneOut,KFold\n",
        "import numpy as np\n",
        "from sklearn.feature_extraction.text import CountVectorizer\n",
        "from sklearn.feature_extraction.text import TfidfVectorizer"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PzjMY8fYQbB6",
        "colab_type": "code",
        "cellView": "both",
        "colab": {}
      },
      "source": [
        "# read in the movie review corpus\n",
        "def readReviews():\n",
        "  raw = requests.get(\"https://raw.githubusercontent.com/mutherr/CS6120-PS1-data/master/cornell_reviews.json\").text.strip()\n",
        "  corpus = [json.loads(line) for line in raw.split(\"\\n\")]\n",
        "\n",
        "  return corpus"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Qvd3I95FT01D",
        "colab_type": "text"
      },
      "source": [
        "This is where you will implement two functions to featurize the data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "039fPQcF7OkN",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#NB: The current contents are for testing only\n",
        "#This function should return: \n",
        "#  -a sparse numpy matrix of document features\n",
        "#  -a list of the correct class for each document\n",
        "#  -a list of the vocabulary used by the features, such that the ith term of the\n",
        "#    list is the word whose counts appear in the ith column of the matrix. \n",
        "\n",
        "# This function should create a feature representation using all tokens that\n",
        "# contain an alphabetic character.\n",
        "def createBasicFeatures(corpus):\n",
        "  #Your code here\n",
        "  corpusText = [row[\"text\"] for row in corpus]\n",
        "  classes = [row[\"class\"] for row in corpus]\n",
        "  # vectorizer = TfidfVectorizer(sublinear_tf=True, max_df=0.5, stop_words='english')\n",
        "  # X_train = vectorizer.fit_transform(corpusText)\n",
        "  vectorizer = CountVectorizer()\n",
        "  texts = vectorizer.fit_transform(corpusText)\n",
        "  vocab = vectorizer.get_feature_names()\n",
        "  return texts,classes,vocab\n",
        "\n",
        "# This function can add other features you want that help classification\n",
        "# accuracy, such as bigrams, word prefixes and suffixes, etc.\n",
        "def createFancyFeatures(corpus):\n",
        "  #Your code here\n",
        "  corpusText = [row[\"text\"] for row in corpus]\n",
        "  classes = [row[\"class\"] for row in corpus]\n",
        "  bigram_vectorizer = CountVectorizer(ngram_range=(1, 2), token_pattern=r'\\b\\w+\\b', min_df=1)\n",
        "  # X_train = vectorizer.fit_transform(corpusText)\n",
        "  # vectorizer = CountVectorizer()\n",
        "  texts = bigram_vectorizer.fit_transform(corpusText)\n",
        "  vocab = bigram_vectorizer.get_feature_names()\n",
        "  return texts,classes,vocab"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KfTBqBltXe7Y",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#given a numpy matrix representation of the features for the training set, the \n",
        "# vector of true classes for each example, and the vocabulary as described \n",
        "# above, this computes the accuracy of the model using leave one out cross \n",
        "# validation and reports the most indicative features for each class\n",
        "\n",
        "def evaluateModel(X,y,vocab,penalty=\"l1\"):\n",
        "  #create and fit the model\n",
        "  model = LogisticRegression(penalty=penalty,solver=\"liblinear\")\n",
        "  results = cross_validate(model,X,y,cv=KFold(n_splits=10, shuffle=True, random_state=1))\n",
        "  \n",
        "  #determine the average accuracy\n",
        "  scores = results[\"test_score\"]\n",
        "  avg_score = sum(scores)/len(scores)\n",
        "  \n",
        "  #determine the most informative features\n",
        "  # this requires us to fit the model to everything, because we need a\n",
        "  # single model to draw coefficients from, rather than 26\n",
        "  model.fit(X,y)\n",
        "  class0_weight_sorted = model.coef_[0, :].argsort()\n",
        "  class1_weight_sorted = (-model.coef_[0, :]).argsort()\n",
        "\n",
        "  termsToTake = 20\n",
        "  class0_indicators = [vocab[i] for i in class0_weight_sorted[:termsToTake]]\n",
        "  class1_indicators = [vocab[i] for i in class1_weight_sorted[:termsToTake]]\n",
        "\n",
        "  if model.classes_[0] == \"pos\":\n",
        "    return avg_score,class0_indicators,class1_indicators\n",
        "  else:\n",
        "    return avg_score,class1_indicators,class0_indicators\n",
        "\n",
        "def runEvaluation(X,y,vocab):\n",
        "  print(\"----------L1 Norm-----------\")\n",
        "  avg_score,pos_indicators,neg_indicators = evaluateModel(X,y,vocab,\"l1\")\n",
        "  print(\"The model's average accuracy is %f\"%avg_score)\n",
        "  print(\"The most informative terms for pos are: %s\"%pos_indicators)\n",
        "  print(\"The most informative terms for neg are: %s\"%neg_indicators)\n",
        "  #this call will fit a model with L2 normalization\n",
        "  print(\"----------L2 Norm-----------\")\n",
        "  avg_score,pos_indicators,neg_indicators = evaluateModel(X,y,vocab,\"l2\")\n",
        "  print(\"The model's average accuracy is %f\"%avg_score)\n",
        "  print(\"The most informative terms for pos are: %s\"%pos_indicators)\n",
        "  print(\"The most informative terms for neg are: %s\"%neg_indicators)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VWWq5VgmECKV",
        "colab_type": "code",
        "outputId": "a211fd8a-c0c5-4bf4-b5ab-b0c4881e0e44",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        }
      },
      "source": [
        "corpus = readReviews()\n",
        "print(corpus[0])"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "{'class': 'neg', 'text': 'plot : two teen couples go to a church party , drink and then drive . they get into an accident . one of the guys dies , but his girlfriend continues to see him in her life , and has nightmares . what\\'s the deal ? watch the movie and \" sorta \" find out . . . critique : a mind-fuck movie for the teen generation that touches on a very cool idea , but presents it in a very bad package . which is what makes this review an even harder one to write , since i generally applaud films which attempt to break the mold , mess with your head and such ( lost highway & memento ) , but there are good and bad ways of making all types of films , and these folks just didn\\'t snag this one correctly . they seem to have taken this pretty neat concept , but executed it terribly . so what are the problems with the movie ? well , its main problem is that it\\'s simply too jumbled . it starts off \" normal \" but then downshifts into this \" fantasy \" world in which you , as an audience member , have no idea what\\'s going on . there are dreams , there are characters coming back from the dead , there are others who look like the dead , there are strange apparitions , there are disappearances , there are a looooot of chase scenes , there are tons of weird things that happen , and most of it is simply not explained . now i personally don\\'t mind trying to unravel a film every now and then , but when all it does is give me the same clue over and over again , i get kind of fed up after a while , which is this film\\'s biggest problem . it\\'s obviously got this big secret to hide , but it seems to want to hide it completely until its final five minutes . and do they make things entertaining , thrilling or even engaging , in the meantime ? not really . the sad part is that the arrow and i both dig on flicks like this , so we actually figured most of it out by the half-way point , so all of the strangeness after that did start to make a little bit of sense , but it still didn\\'t the make the film all that more entertaining . i guess the bottom line with movies like this is that you should always make sure that the audience is \" into it \" even before they are given the secret password to enter your world of understanding . i mean , showing melissa sagemiller running away from visions for about 20 minutes throughout the movie is just plain lazy ! ! okay , we get it . . . there are people chasing her and we don\\'t know who they are . do we really need to see it over and over again ? how about giving us different scenes offering further insight into all of the strangeness going down in the movie ? apparently , the studio took this film away from its director and chopped it up themselves , and it shows . there might\\'ve been a pretty decent teen mind-fuck movie in here somewhere , but i guess \" the suits \" decided that turning it into a music video with little edge , would make more sense . the actors are pretty good for the most part , although wes bentley just seemed to be playing the exact same character that he did in american beauty , only in a new neighborhood . but my biggest kudos go out to sagemiller , who holds her own throughout the entire film , and actually has you feeling her character\\'s unraveling . overall , the film doesn\\'t stick because it doesn\\'t entertain , it\\'s confusing , it rarely excites and it feels pretty redundant for most of its runtime , despite a pretty cool ending and explanation to all of the craziness that came before it . oh , and by the way , this is not a horror or teen slasher flick . . . it\\'s just packaged to look that way because someone is apparently assuming that the genre is still hot with the kids . it also wrapped production two years ago and has been sitting on the shelves ever since . whatever . . . skip it ! where\\'s joblo coming from ? a nightmare of elm street 3 ( 7/10 ) - blair witch 2 ( 7/10 ) - the crow ( 9/10 ) - the crow : salvation ( 4/10 ) - lost highway ( 10/10 ) - memento ( 10/10 ) - the others ( 9/10 ) - stir of echoes ( 8/10 ) '}\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "72OUet02TjBo",
        "colab_type": "text"
      },
      "source": [
        "Run the following to train and evaluate two models using basic features:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4IpJ7PKjvc8I",
        "colab_type": "code",
        "outputId": "6fdfe81f-3d36-47e3-a670-a2a28a797879",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 173
        }
      },
      "source": [
        "X,y,vocab = createBasicFeatures(corpus)\n",
        "runEvaluation(X, y, vocab)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "----------L1 Norm-----------\n",
            "The model's average accuracy is 0.828500\n",
            "The most informative terms for pos are: ['flaws', 'memorable', 'terrific', 'edge', 'masterpiece', 'excellent', 'perfectly', 'sherri', 'gas', 'using', 'enjoyable', 'overall', 'fun', 'follows', 'different', 'quite', 'solid', 'allows', 'fantastic', 'side']\n",
            "The most informative terms for neg are: ['waste', 'mess', 'lame', 'worst', 'headed', 'ridiculous', 'unfortunately', 'awful', 'cheap', 'write', 'boring', 'tedious', 'superior', 'jesse', 'terrible', 'bad', 'poor', 'iii', 'designed', 'looks']\n",
            "----------L2 Norm-----------\n",
            "The model's average accuracy is 0.833000\n",
            "The most informative terms for pos are: ['fun', 'great', 'back', 'quite', 'well', 'seen', 'excellent', 'perfectly', 'memorable', 'overall', 'job', 'yet', 'american', 'terrific', 'pulp', 'bit', 'true', 'performances', 'husband', 'masterpiece']\n",
            "The most informative terms for neg are: ['bad', 'unfortunately', 'worst', 'waste', 'nothing', 'script', 'awful', 'boring', 'only', 'plot', 'poor', 'reason', 'looks', 'mess', 'supposed', 'lame', 'women', 'any', 'should', 'anyway']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4sjxLL2PTrJi",
        "colab_type": "text"
      },
      "source": [
        "Run the following to train and evaluate two models using extended features:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1iHudrPb5NPY",
        "colab_type": "code",
        "outputId": "0f3751dd-b5ed-457a-9c7a-cb84c18b2c99",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 173
        }
      },
      "source": [
        "X,y,vocab = createFancyFeatures(corpus)\n",
        "runEvaluation(X, y, vocab)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "----------L1 Norm-----------\n",
            "The model's average accuracy is 0.825500\n",
            "The most informative terms for pos are: ['even if', 'flaws', 'memorable', 'due to', 'masterpiece', 'overall', 'follows', 'her husband', 'terrific', 'by a', 'the more', 'as much', 'works', 'seen', 'great', 'gas', 'filmmaking', 'they were', 'loved', 'images']\n",
            "The most informative terms for neg are: ['ridiculous', 'waste', 'unfortunately', 'mess', 'awful', 'boring', 'worst', 'cheap', 'should have', 'lame', 'headed', 'bad', 'poor', 'write', 'maybe', '0', 'designed', 'tedious', 'terrible', 'metro']\n",
            "----------L2 Norm-----------\n",
            "The model's average accuracy is 0.853500\n",
            "The most informative terms for pos are: ['great', 'well', 'fun', 'seen', 'very', 'back', 'quite', 'also', 'many', 'yet', 'people', 'american', 'life', 'see', 'excellent', 'while', 'job', 'the two', 'by a', 'perfectly']\n",
            "The most informative terms for neg are: ['bad', 'only', 'unfortunately', 'worst', 'plot', 'any', 'nothing', 'boring', 'waste', 'script', 'poor', 'looks', 'here', 'director', 'have', 'to be', 'on', 'as a', 'supposed', 'awful']\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UzyJF853MgLL",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        ""
      ],
      "execution_count": 0,
      "outputs": []
    }
  ]
}