Merge pull request #28 from toshiakit/dev

UPDATES: Streaming Support and Clickable Follow-up Prompts
toshiakit · Feb 7, 2024 · 1c8862a · 1c8862a
2 parents 61dea6b + 54d562e
commit 1c8862a
Show file tree

Hide file tree

Showing 34 changed files with 1,308 additions and 122 deletions.
diff --git a/.gitignore b/.gitignore
@@ -1,4 +1,5 @@
-*.mlx
+*.env
+*.asv
 contents/ChatData.mat
 contents/GeneratedCode/
-helpers/*.asv
+startup.m
diff --git a/MatGPT.mlapp b/MatGPT.mlapp
diff --git a/README.md b/README.md
@@ -15,16 +15,18 @@ MatGPT was updated to run on the framework from the '[Large Language Models (LLM
 ## What's New
 
 * MatGPT runs on the 'LLMs with MATLAB' framework, which requires MATLAB R2023a or later. 
+* MatGPT supports streaming API where response tokens are displayed as they come in.
 * MatGPT detects a URL included in a prompt, and retrieve its web content into the chat.
-* MatGPT lets you import a .m, .mlx, or .csv file into the chat. 
+* MatGPT lets you import a .m, .mlx, .csv or .txt file into the chat. PDF files are also supported if Text Analytics Toolbox is available.
+* MatGPT supports GPT-4 Turbo with Vision. You can pass the URL to an image or a local image file path ask questions about the image. 
 
 Please note that imported content will be truncated if it exceeds the context window limit. 
 
 ## Requirements
 
-* **MathWorks Products (https://www.mathworks.com)**:  To use MatGPT, you need to have MATLAB R2023a or later installed on your computer. 
-* **OpenAI API Key**: Additionally, you will need your own API key from [https://platform.openai.com/account/api-keys](https://platform.openai.com/account/api-keys). [MATLAB AI Chat Playground](https://www.mathworks.com/matlabcentral/playground/) is a better option if you don’t want to set up an API access with OpenAI. 
-* GPT-4 models are [available to all API users who have a history of successful payments](https://openai.com/blog/gpt-4-api-general-availability). If you have not made any payment to OpenAI, the model is not accessible. 
+* **MathWorks Products (https://www.mathworks.com)**: Use MatGPT to run on [MATLAB Online](https://www.mathworks.com/products/matlab-online.html) that comes with the most commonly used toolboxes. To use it on desktop, you must have MATLAB R2023a or later installed on your computer. 
+* **OpenAI API Key**: Additionally, you will need your own API key from [https://platform.openai.com/account/api-keys](https://platform.openai.com/account/api-keys). If you don’t want to set up an API access with OpenAI, [MATLAB AI Chat Playground](https://www.mathworks.com/matlabcentral/playground/) is a better option. 
+* GPT-4 models are [available to all API users who have a history of successful payments](https://openai.com/blog/gpt-4-api-general-availability). If you have not made any payment to OpenAI, the GPT-4 models are not accessible. 
 
 ## Installation
 
@@ -49,7 +51,7 @@ To use MatGPT on MATLAB Online, simply click [![Open in MATLAB Online](https://w
 * The paperclip button lets you include the content of a m-file, live script file or csv file in the chat.
 * If the prompt contains a URL, MatGPT ask you to confirm that you want to open the page. 
 * The `Send` button and Paperclip button are disabled until a chat is configured in the `Settings` tab.
-* If you want suggestion for follow-up questions in the response, check `Suggest follow-up questions` checkbox. 
+* If you want suggestion for follow-up questions in the response, check `Suggest follow-up questions` checkbox. Suggested questions appear as clickable buttons. You can copy a suggested question to the prompt box by clicking it.  
 * If your prompt is intended to generate MATLAB code, check `Test Generated MATLAB Code` checkbox to test the returned code.
 * The `Usage` tab shows the number of tokens used in the current chat session. 
 * Add stop sequences in `Advanced` tab to specify the sequences where the API will stop generating further tokens
@@ -59,7 +61,8 @@ To use MatGPT on MATLAB Online, simply click [![Open in MATLAB Online](https://w
 
 ### Note:
 
-You can increase the connection timeout in the `Settings`. You can add proxy via [Web Preferences](https://www.mathworks.com/help/matlab/ref/preferences.html) in MATLAB.
+* You can increase the connection timeout in the `Settings`. You can add proxy via [Web Preferences](https://www.mathworks.com/help/matlab/ref/preferences.html) in MATLAB.
+* Streaming is enabled by default, but you can turn it off in the `Settings` tab. Usage data is not available in streaming mode. 
 
 ## What happened to chatGPT class?
 

diff --git a/contents/index.html b/contents/index.html
@@ -16,34 +16,50 @@
 
             function setup(htmlComponent) {
 
+                // Detect button click
+                let body = document.getElementsByTagName("body")[0];
+                body.addEventListener("click", function(event) {
+                    if (event.target.nodeName == "BUTTON" && event.target.className == "relquestion") {
+                        let prompt = event.target.innerHTML;
+                        htmlComponent.Data = prompt;
+                    }
+                })
+
+                // Detect data change
                 htmlComponent.addEventListener("DataChanged", function(event) { 
                     // Get a reference to the table
                     var table = document.querySelector('.table');
                     var changedData = htmlComponent.Data;
 
-                    // Create a new row
-                    var newRow = document.createElement('div');
-                    newRow.classList.add('table-row');
-
-                    // Create the two cells in the new row
-                    var cell1 = document.createElement('div');
-                    cell1.classList.add('table-cell');                    
-                    var cell2 = document.createElement('div');
-                    cell2.classList.add('table-cell');
-
-
-                    // Add Role text in 1st cell
-                    cell1.innerHTML = changedData[0];                
-                    cell2.innerHTML = changedData[1]; 
+                    if (changedData[2] == "new") {
+                        // Create a new row
+                        var newRow = document.createElement('div');
+                        newRow.classList.add('table-row');
+                        
+                        // Create the two cells in the new row
+                        var cell1 = document.createElement('div');
+                        cell1.classList.add('table-cell');                    
+                        var cell2 = document.createElement('div');
+                        cell2.classList.add('table-cell');
+                        
+                        // Add the changed data to the cells
+                        cell1.innerHTML = changedData[0];                
+                        cell2.innerHTML = changedData[1]; 
 
-                    // Add the cells to the new row
-                    newRow.appendChild(cell1);
-                    newRow.appendChild(cell2);
+                        // Add the cells to the new row
+                        newRow.appendChild(cell1);
+                        newRow.appendChild(cell2);
 
-                    // Add the new row to the table
-                    table.appendChild(newRow);
+                        // Add the new row to the table
+                        table.appendChild(newRow);
+                    } else {
+                        // Get the last cell of the last row
+                        var lastRow = table.children[table.children.length - 1];
+                        var lastCell = lastRow.children[lastRow.children.length - 1];
+                        // Add the changed data to the cell
+                        lastCell.innerHTML = changedData[1];
+                    }
                 });
-
             }
         </script>
     </body>

diff --git a/contents/presets.csv b/contents/presets.csv
@@ -1,8 +1,9 @@
 name,content,prompt,model,max_tokens,temperature,test_code
 AI Assistant,You are a helpful assistant. Answer as concisely as possible. ,Where is the capital of France?,gpt-3.5-turbo,1000,1,0
 Read a web page,You are a helpful assistant that can read a small web page and analyze its content. The content of the page will be extracted by an external function and stored in the chat history. ,What is this page about? https://www.mathworks.com/help/matlab/text-files.html,gpt-3.5-turbo,1000,0,0
-Read a local file,You are a helpful assistant that can read a small file and analyze its content. The content of the file will be extracted by an external function and stored in the chat history. ,What is this file about?,gpt-3.5-turbo,1000,0,0
-English to MATLAB Code,You are a helpful assistant that generates MATLAB code using the latest features from R2021a or later. ,"Define two random vectors x and y, fit a linear model to the data, and plot both the data and fitted line.",gpt-3.5-turbo,1000,0,1
+Read a local file,You are a helpful assistant that can read a small file and analyze its content. The content of the file will be extracted by an external function and stored in the chat history. ,Select a file using the paper clip icon on the left.,gpt-3.5-turbo,1000,0,0
+Understand an image,You are a helpful assistant that can see an image and analyze its content.,What is in this image? https://www.mathworks.com/help/examples/matlab/win64/DisplayGrayscaleRGBIndexedOrBinaryImageExample_04.png,gpt-4-vision-preview,Inf,1,0
+English to MATLAB Code,You are a helpful assistant that generates MATLAB code. ,"Define two random vectors x and y, fit a linear model to the data, and plot both the data and fitted line.",gpt-3.5-turbo,1000,0,1
 English to Simulink Model,"You are a helpful assistant that creates Simulink models.  
 You create the models by generating MATLAB code that adds all the necessary blocks and set their parameters. 
 Automatically arrange the blocks by adding this line of code:
@@ -12,7 +13,7 @@ Add Sine Wave block with amplification = 1.
 Multiply the sine wave signal by 3. 
 Add Scope block with 2 input ports. 
 Visualize both signals by connecting them to the same Scope block. ",gpt-3.5-turbo,1000,0,1
-Summarize Code,You are a friendly and helpful teaching assistant for MATLAB programmers. Analyze MATLAB code and provide concise summary of what the code does. ,"Summarize what the following code by Cecelya https://www.mathworks.com/matlabcentral/communitycontests/contests/5/entries/12418 does. 
+Summarize Code,You are a friendly and helpful teaching assistant for MATLAB programmers. Analyze MATLAB code and provide concise summary of what the code does. ,"Summarize what the following code does. Code by Cecelya Blooming Light in MATLAB Mini Hack 2022
 
 ```matlab
 m = 0:.01:1;
@@ -32,7 +33,7 @@ view([0 33])
 colormap(flip(hot)) 
 shading('interp')
 ```",gpt-3.5-turbo,1000,0,0
-Explain Code Step by Step,You are a friendly and helpful teaching assistant for MATLAB programmers. Analyze MATLAB code and explain the code step by step.,"Explain what the following code  by Cecelya https://www.mathworks.com/matlabcentral/communitycontests/contests/5/entries/12418 does. 
+Explain Code Step by Step,You are a friendly and helpful teaching assistant for MATLAB programmers. Analyze MATLAB code and explain the code step by step.,"Explain what the following code does. Code by Cecelya Blooming Light in MATLAB Mini Hack 2022 
 
 ```matlab
 m = 0:.01:1;
@@ -52,7 +53,7 @@ view([0 33])
 colormap(flip(hot)) 
 shading('interp')
 ```",gpt-3.5-turbo,1000,0,0
-Script to function,You are a friendly and helpful teaching assistant for MATLAB programmers. Convert MATLAB scripts to a local function and call the function using sample inputs.  ,"Convert the following script by Cecelya https://www.mathworks.com/matlabcentral/communitycontests/contests/5/entries/12418 to a function, using m and T as input variables and set default values to those variables. 
+Script to function,You are a friendly and helpful teaching assistant for MATLAB programmers. Convert MATLAB scripts to a local function and call the function using sample inputs.  ,"Convert the following script to a function, using m and T as input variables and set default values to those variables. Code by Cecelya Blooming Light in MATLAB Mini Hack 2022
 
 ```matlab
 m = 0:.01:1;
@@ -72,7 +73,7 @@ view([0 33])
 colormap(flip(hot)) 
 shading('interp')
 ```",gpt-3.5-turbo,1000,0,1
-Write doc to function,"You are a friendly and helpful teaching assistant for MATLAB programmers. Analyze a MATLAB function and add an elaborate, high quality docstring to the function.","Write useful docstring to this function derived from the code by Cecelya https://www.mathworks.com/matlabcentral/communitycontests/contests/5/entries/12418, using comment format %PLOTBLOOMINGLIGHTS does blah blah blah.  
+Write doc to function,"You are a friendly and helpful teaching assistant for MATLAB programmers. Analyze a MATLAB function and add an elaborate, high quality docstring to the function.","Write useful docstring to this function derived from the code, using comment format %PLOTBLOOMINGLIGHTS does blah blah blah.  Code by Cecelya Blooming Light in MATLAB Mini Hack 2022
 
 ```matlab
 function plotBloomingLight(m,T)

diff --git a/contents/styles.css b/contents/styles.css
@@ -1,5 +1,3 @@
-
-
 .table {
   display: table;
   border-collapse: collapse;
@@ -117,6 +115,10 @@ tr:nth-child(even) {
   background-color: #f2f2f2;
 }
 
+button.relquestion {
+  text-align: left;
+}
+
 body {
   font-size: 100%;
 }