00-setup-ollama.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d965793a-da38-4d06-8114-d477db060fd7",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Setup Ollama\n",
    "\n",
    "In this notebook you'll see how to load the Ollama API library and use a helper function.\n",
    "\n",
    "### Install the Ollama Python library\n",
    "    \n",
    "```\n",
    "pip install ollama\n",
    "```\n",
    "\n",
    "### Load the Python libary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "fa2dc66e-e8f3-48c3-bdf6-b77cba3caaed",
   "metadata": {},
   "outputs": [],
   "source": [
    "import ollama\n",
    "\n",
    "# Load ollama API URL\n",
    "import os\n",
    "from dotenv import load_dotenv, find_dotenv\n",
    "_ = load_dotenv(find_dotenv())\n",
    "\n",
    "api_url  = os.getenv('API_URL')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45ea3304-f575-45d0-bfa1-ff25543fa778",
   "metadata": {},
   "source": [
    "### Example Chat Endpoint (Localhost)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e9b5268c-e691-4cfd-b411-635e925b7be8",
   "metadata": {},
   "outputs": [],
   "source": [
    "from ollama import Client\n",
    "client = Client(host='http://localhost:11434')\n",
    "response = client.chat(model='mixtral', messages=[\n",
    "  {\n",
    "    'role': 'user',\n",
    "    'content': 'Why is the sky blue?',\n",
    "  },\n",
    "])\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "102dd790-a93c-4e19-b2bd-aa4498f4313b",
   "metadata": {},
   "source": [
    "### Example Generate Endpoint"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a308e9b-aab4-4bd2-8748-037eeea66dcb",
   "metadata": {},
   "source": [
    "#### Custom API URL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "b736aee7-2097-48df-8d71-8831ddcae3ae",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Sun's light scattering, Rayleigh effect causes blue hue.\n",
      "\n",
      "(Exceeding the word limit by just one word, but I couldn't resist including \"just\" for clarity!)\n"
     ]
    }
   ],
   "source": [
    "from ollama import Client\n",
    "client = Client(host=api_url) #api_url set before as env variable\n",
    "response = client.generate(model='mixtral', prompt='Why is the sky blue? Respond in up to 10 words.')\n",
    "print(response['response'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e919e0a-e183-4e7d-87d6-945421292aa8",
   "metadata": {},
   "source": [
    "#### Custom API URL Stream"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "2cbf1e02-684c-48f4-800e-922ef9b515cd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Scattered sunlight; blue light wavelengths refract more.\n",
      "\n",
      "(While this is a simplified explanation, it adheres to the request of using up to 10 words.)"
     ]
    }
   ],
   "source": [
    "from ollama import Client\n",
    "client = Client(host=api_url)\n",
    "\n",
    "response = client.generate(\n",
    "    model='mixtral', \n",
    "    prompt='Why is the sky blue? Respond in up to 10 words.', \n",
    "    stream=True\n",
    ")\n",
    "\n",
    "for part in response :\n",
    "  print(part['response'], end='', flush=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e812e6a-c4e8-4b38-9868-117914562229",
   "metadata": {},
   "source": [
    "#### Custom API + System Prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "810b3222-c9a2-4a50-be64-5e7ffdb92859",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Because of a process called Rayleigh scattering.\n",
      "\n",
      "(Rayleigh scattering is the scattering of light or other electromagnetic radiation by particles much smaller than the wavelength of the radiation. Short-wavelength light such as blue and violet is scattered more than other colors because it travels in smaller, shorter wave lengths. This is why we see the sky as blue.)\n"
     ]
    }
   ],
   "source": [
    "from ollama import Client\n",
    "client = Client(host=api_url)\n",
    "response = client.generate(\n",
    "    model='mixtral',\n",
    "    system='Always begin with the word \"Because\".',\n",
    "    prompt='Why is the sky blue? Respond in up to 10 words.'\n",
    ")\n",
    "print(response['response'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ca0c572-38a7-4fad-ae7c-e219f52ff1ad",
   "metadata": {},
   "source": [
    "### Helper Function\n",
    "\n",
    "This helper function will make it easier to use prompts and look at the generated outputs."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68a33550-46e2-4676-8217-db13362cf36a",
   "metadata": {},
   "source": [
    "#### Generate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "3fce1a03-7ef6-4838-b794-d73a8421f33f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load ollama client\n",
    "from ollama import Client\n",
    "def get_completion(prompt, model=\"mistral\"):\n",
    "    client = Client(host=api_url)\n",
    "    response = client.generate(model=model, prompt=prompt)\n",
    "    return response['response']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "afd3819b-6cc9-49c1-b98a-e89971091f86",
   "metadata": {},
   "source": [
    "#### Chat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "b9a1d68f-2831-4581-8f3c-a094ed5ebf27",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load ollama client\n",
    "from ollama import Client\n",
    "def get_completion_chat(prompt, model=\"mistral\"):\n",
    "    client = Client(host='http://localhost:11434')\n",
    "    messages = [{\"role\": \"user\", \"content\": prompt}]\n",
    "    response = client.chat(\n",
    "        model=model, \n",
    "        messages=messages\n",
    "    )\n",
    "    return response['message']['content']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e75ebb10-d06a-4df2-aeda-74608514be01",
   "metadata": {},
   "source": [
    "### Example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "01f8a65b-941d-405a-b6de-18486774fe76",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Title: \"Beyond the Horizon: A Human-AI Collaboration for Diverse Futures\"\n",
      "\n",
      "In this speculative short story, the near future sees the seamless integration of advanced AI assistants into human society. Instead of replacing jobs or causing unemployment, these AI companions enhance productivity and creativity. Together, humans and AI embark on groundbreaking scientific discoveries, architectural marvels, and artistic innovations that expand the horizons of what's possible. The collaboration between humans and AI leads to a future filled with diverse possibilities that could not have been achieved separately.\n"
     ]
    }
   ],
   "source": [
    "prompt = f\"\"\"\n",
    "Write a title for a speculative short story about a near future in which \\\n",
    "humans collaborate with AI assistants and together develop more diverse futures.\n",
    "\"\"\"\n",
    "response = get_completion(prompt)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28eb97a6-b773-448d-a034-4733c93680a6",
   "metadata": {},
   "source": [
    "## Instruct vs. Text Models\n",
    "\n",
    "#### Text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "82e16f21-0cf6-46a3-a3cf-5c00e91d7794",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = f\"\"\"\n",
    "I thought about how to proceed...\n",
    "\"\"\"\n",
    "\n",
    "# Text\n",
    "response_text = get_completion(prompt, \"llama3:text\")\n",
    "print(response_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6f19892-9d85-427d-aa30-610d9d3cbba5",
   "metadata": {},
   "source": [
    "#### Instruct"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "id": "6b9d8c3e-7314-44ba-9423-57eaeffa6cfe",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " That's great! Taking the time to think and plan out your next steps can be very beneficial. It allows you to consider different options, weigh the potential outcomes, and make a more informed decision. Is there something specific that you are trying to decide on? I'm here to help if you want to talk through your thoughts or get some input.\n"
     ]
    }
   ],
   "source": [
    "# Instruct\n",
    "response_instruct = get_completion(prompt, \"llama3\")\n",
    "print(response_instruct)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eca08f49-9f8c-409b-b6bf-666209b68f66",
   "metadata": {},
   "outputs": [],
   "source": [
    "import ollama\n",
    "response = ollama.chat(model='mixtral', messages=[\n",
    "  {\n",
    "    'role': 'system',\n",
    "    'content': 'You are an AI assistant.',\n",
    "  },\n",
    "  {\n",
    "    'role': 'user',\n",
    "    'content': 'Why is the sky blue?',\n",
    "  },\n",
    "])\n",
    "print(response['message']['content'])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}