group_7_ass_1_2024_task1 request to merge with main branch

.gitignore

@@ -30,3 +30,9 @@
 *.exe
 *.out
 *.app
+
+*.ipynb_checkpoints
+*__pycache__
+*/Constants.py
+*/env
+*chroma/

Lecture_10/LLM_basics.ipynb

@@ -0,0 +1,285 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Basics of LLM applications\n",
+    "\n",
+    "## Basic Components\n",
+    "\n",
+    "The most basic components of any LLM application are the following:\n",
+    "\n",
+    "- **Input**\n",
+    "\n",
+    "As with any other program, we desire for any input by the user to be converted to a useful output through a series of predefined steps.\n",
+    "\n",
+    "- **Setup functions**\n",
+    "\n",
+    "The input must undergo a series of transformations in order to be used by the API\n",
+    "\n",
+    "- **System Prompt**\n",
+    "\n",
+    "This is a natural language description of the task the LLM will assist with, as well as instructions and specifications on how this task is to be achieved.\n",
+    "\n",
+    "- **API**\n",
+    "\n",
+    "This is the step that actually generates the response(s), which will then be displayed by the app through a frontend environment\n",
+    "\n",
+    "## Assembling the components\n",
+    "\n",
+    "These basic components can be assembled in many different ways depending on the desired outcome. One example of a simple architecture would be the following:\n",
+    "\n",
+    "![simple_design](_assets/simple_design.png)\n",
+    "\n",
+    "This example takes the initial API response as input for more rounds of generation, an approach that can be used both for response refinement and for the addition of \"memory\" to the model. This is just one very simple permutation of possible architectures that can be designed to meet the needs of the application.\n",
+    "\n",
+    "## Creating a simple Comand Line Interface (CLI) app\n",
+    "\n",
+    "We will now use the previously defined architecture to create an app that will answer a question, and then stores this answer in a chat history or \"memory\" to be used as context for further queries. A CLI app is one in which the frontend is the command line. In our case, we are using a Jupyter Notebook, which renders command line outputs in a more interactive manner.\n",
+    "\n",
+    "To start off, you will need the `openai` and `tiktoken` package. Make sure to first install them if you haven't yet, and then you can import them.\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Requirement already satisfied: openai in c:\\users\\alexander\\anaconda3\\envs\\tensorflow\\lib\\site-packages (0.27.7)\n",
+      "Requirement already satisfied: tiktoken in c:\\users\\alexander\\anaconda3\\envs\\tensorflow\\lib\\site-packages (0.4.0)\n",
+      "Requirement already satisfied: aiohttp in c:\\users\\alexander\\anaconda3\\envs\\tensorflow\\lib\\site-packages (from openai) (3.8.5)\n",
+      "Requirement already satisfied: requests>=2.20 in c:\\users\\alexander\\anaconda3\\envs\\tensorflow\\lib\\site-packages (from openai) (2.31.0)\n",
+      "Requirement already satisfied: tqdm in c:\\users\\alexander\\anaconda3\\envs\\tensorflow\\lib\\site-packages (from openai) (4.62.3)\n",
+      "Requirement already satisfied: regex>=2022.1.18 in c:\\users\\alexander\\anaconda3\\envs\\tensorflow\\lib\\site-packages (from tiktoken) (2023.5.5)\n",
+      "Requirement already satisfied: urllib3<3,>=1.21.1 in c:\\users\\alexander\\anaconda3\\envs\\tensorflow\\lib\\site-packages (from requests>=2.20->openai) (1.26.18)\n",
+      "Requirement already satisfied: certifi>=2017.4.17 in c:\\users\\alexander\\anaconda3\\envs\\tensorflow\\lib\\site-packages (from requests>=2.20->openai) (2023.11.17)\n",
+      "Requirement already satisfied: idna<4,>=2.5 in c:\\users\\alexander\\anaconda3\\envs\\tensorflow\\lib\\site-packages (from requests>=2.20->openai) (3.4)\n",
+      "Requirement already satisfied: charset-normalizer<4,>=2 in c:\\users\\alexander\\anaconda3\\envs\\tensorflow\\lib\\site-packages (from requests>=2.20->openai) (2.0.10)\n",
+      "Requirement already satisfied: multidict<7.0,>=4.5 in c:\\users\\alexander\\anaconda3\\envs\\tensorflow\\lib\\site-packages (from aiohttp->openai) (6.0.4)\n",
+      "Requirement already satisfied: attrs>=17.3.0 in c:\\users\\alexander\\anaconda3\\envs\\tensorflow\\lib\\site-packages (from aiohttp->openai) (21.4.0)\n",
+      "Requirement already satisfied: aiosignal>=1.1.2 in c:\\users\\alexander\\anaconda3\\envs\\tensorflow\\lib\\site-packages (from aiohttp->openai) (1.3.1)\n",
+      "Requirement already satisfied: frozenlist>=1.1.1 in c:\\users\\alexander\\anaconda3\\envs\\tensorflow\\lib\\site-packages (from aiohttp->openai) (1.4.0)\n",
+      "Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in c:\\users\\alexander\\anaconda3\\envs\\tensorflow\\lib\\site-packages (from aiohttp->openai) (4.0.2)\n",
+      "Requirement already satisfied: yarl<2.0,>=1.0 in c:\\users\\alexander\\anaconda3\\envs\\tensorflow\\lib\\site-packages (from aiohttp->openai) (1.9.2)\n",
+      "Requirement already satisfied: colorama in c:\\users\\alexander\\anaconda3\\envs\\tensorflow\\lib\\site-packages (from tqdm->openai) (0.4.4)\n"
+     ]
+    }
+   ],
+   "source": [
+    "!pip install openai tiktoken"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [
+    {
+     "ename": "ImportError",
+     "evalue": "cannot import name 'OpenAI' from 'openai' (C:\\Users\\Alexander\\anaconda3\\envs\\tensorflow\\lib\\site-packages\\openai\\__init__.py)",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[1;31mImportError\u001b[0m                               Traceback (most recent call last)",
+      "\u001b[1;32mC:\\Users\\ALEXAN~1\\AppData\\Local\\Temp/ipykernel_29228/2710875855.py\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[1;32mfrom\u001b[0m \u001b[0mopenai\u001b[0m \u001b[1;32mimport\u001b[0m \u001b[0mOpenAI\u001b[0m \u001b[1;31m# the tiktoken package is imported by openai while functioning, so no need to explicitly import it\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
+      "\u001b[1;31mImportError\u001b[0m: cannot import name 'OpenAI' from 'openai' (C:\\Users\\Alexander\\anaconda3\\envs\\tensorflow\\lib\\site-packages\\openai\\__init__.py)"
+     ]
+    }
+   ],
+   "source": [
+    "from openai import OpenAI # the tiktoken package is imported by openai while functioning, so no need to explicitly import it"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Acquiring and setting up an OpenAI API key\n",
+    "\n",
+    "To use the OpenAI API, you must have an account and generate a key. Enter the [OpenAI website](https://platform.openai.com/account/api-keys) and follow the steps.\n",
+    "\n",
+    "After generating a key, you will have to add as an environment variable for the API to work. The easiest way to do this is to create a Python file called `Constants.py` that contains the key as a string, and then importing this string into the current kernel. After these steps, you can now set the api key for openai to function."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from Constants import OPENAI_API_KEY\n",
+    "\n",
+    "client = OpenAI(api_key = OPENAI_API_KEY)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Handle user input\n",
+    "\n",
+    "Here, we take the user input and convert it into something usable for the API. In a CLI, we would use the built-in `input` function to get the user's input as a string, but for the sake of using a Jupyter Notebook, we will explicitly change the input string instead.\n",
+    "\n",
+    "We will be using the Chat Completion endpoint of the API, which takes a list of messages, each represented by a dictionary with two entries. The `\"role\"` entry of the dictionary specifies the role of the message's sender, which can be `\"system\"`, `\"user\"`, and `\"assistant\"`; the `\"content\"` entry contains the body of the message. Therefore, we must create this list with the necessary information.\n",
+    "\n",
+    "First we create a system prompt and add it in a dictionary to the list of messages"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "system_prompt = \"You will answer the user's question to the best of your abilities\"\n",
+    "messages = [{\"role\": \"system\", \"content\": system_prompt}]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Then we get the user input and add it to the list"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "user_input = \"what is an elephant?\"\n",
+    "messages.append({\"role\": \"user\", \"content\": user_input})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now we have the prompt and the query in the necessary format to be used with the API. It is also necessary to specify the model to be used with the `model` argument. In this case we will use the `\"gpt-3.5-turbo-1106\"` model."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "ChatCompletion(id='chatcmpl-8fxwYk12Z7vjVxvJUnQiTeiE9OE5b', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='An elephant is a large mammal known for its long trunk, tusks, and large ears. There are two main species of elephants: the African elephant and the Asian elephant. They are highly intelligent and social animals, known for their complex communication, strong family bonds, and remarkable memory. Elephants are herbivores and can be found in various habitats, including savannas, forests, and grasslands.', role='assistant', function_call=None, tool_calls=None))], created=1705012562, model='gpt-3.5-turbo-1106', object='chat.completion', system_fingerprint='fp_cbe4fa03fe', usage=CompletionUsage(completion_tokens=83, prompt_tokens=29, total_tokens=112))"
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "model = \"gpt-3.5-turbo-1106\"\n",
+    "response = client.chat.completions.create(model = model, messages = messages)\n",
+    "response"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "This returns a `ChatCompletion` class object with a lot of information on the response that has been returned. We can retrieve the response as a string by selecting a choice from the `choices` attribute of this class, which is a list of `Choice` class objects. Each of these objects has a `message` attribute which is a `ChatCompletionMessage` class object that contains the message and some information about it. The message string is stored in its `content` attribute."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "'An elephant is a large mammal known for its long trunk, tusks, and large ears. There are two main species of elephants: the African elephant and the Asian elephant. They are highly intelligent and social animals, known for their complex communication, strong family bonds, and remarkable memory. Elephants are herbivores and can be found in various habitats, including savannas, forests, and grasslands.'"
+      ]
+     },
+     "execution_count": 6,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "response.choices[0].message.content"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "We can now add this message to the list of messages, such that we can build a chat history for the model to use."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "messages.append({\"role\": \"assistant\", \"content\": response.choices[0].message.content})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Now we can reference previous responses and the assistant will be able to understand the context"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "\"In some cases, elephants have been known to attack humans, but these incidents are typically a result of feeling threatened or provoked. Elephants are generally peaceful animals and will avoid confrontation if possible. However, conflicts can arise in situations where humans encroach on their natural habitat, or if an elephant feels their young are threatened. It's important to respect their space and behavior to avoid such confrontations. Conservation efforts and responsible wildlife management aim to protect both elephants and humans from these types of interactions.\""
+      ]
+     },
+     "execution_count": 8,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "user_input = \"do they ever attack humans?\"\n",
+    "messages.append({\"role\": \"user\", \"content\": user_input})\n",
+    "response = client.chat.completions.create(model = model, messages = messages)\n",
+    "response.choices[0].message.content"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.8.12"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}