initial flush commit

.gitignore

@@ -0,0 +1,57 @@
+# Prerequisites
+*.d
+
+# Object files
+*.o
+*.ko
+*.obj
+*.elf
+
+# Linker output
+*.ilk
+*.map
+*.exp
+
+# Precompiled Headers
+*.gch
+*.pch
+
+# Libraries
+*.lib
+*.a
+*.la
+*.lo
+
+# Shared objects (inc. Windows DLLs)
+*.dll
+*.so
+*.so.*
+*.dylib
+
+# Executables
+#*.exe
+*.out
+*.app
+*.i*86
+*.x86_64
+*.hex
+
+# Debug files
+*.dSYM/
+*.su
+*.idb
+*.pdb
+
+# Kernel Module Compile Results
+*.mod*
+*.cmd
+.tmp_versions/
+modules.order
+Module.symvers
+Mkfile.old
+dkms.conf
+*.lk1
+*.mk
+*.mk1
+*.sym
+.vscode/

README.md

@@ -0,0 +1,137 @@
+# doschgpt
+
+A proof-of-concept ChatGPT client for DOS. 
+
+<img src="images\doschgpt-front-combined.jpg" width="1000">
+
+Photos of the client running on [my 1984 IBM 5155 Portable PC](https://github.com/yeokm1/retro-configs/tree/master/desktops/ibm-5155) with a 4.77Mhz Intel 8088 CPU with MS-DOS 6.22.
+
+As there are no native HTTPS for DOS, a [HTTP-to-HTTPS proxy](https://github.com/yeokm1/http-to-https-proxy) like this I've written running on a modern machine is needed.
+
+This program is heavily based on sample code in the DOS networking [MTCP library](http://brutmanlabs.org/mTCP/). The program also requires a [DOS Packet Driver](https://en.wikipedia.org/wiki/PC/TCP_Packet_Driver) to be loaded and MTCP to be set for the machine/VM.
+
+**This program was written in a short time as a toy project. It has not been vigorously tested thus is NOT meant for production use.**
+
+## Using the application
+
+Application binary can be found in the `releases` directory or Github Releases section but do the following first.
+
+1. OpenAI requires an API key to use its APIs. Follow the [instructions on their website](https://platform.openai.com/account/api-keys) to obtain this key before proceeding.
+
+2. Downland and start up [http-to-https-proxy](https://github.com/yeokm1/http-to-https-proxy/releases)
+
+3. The application requires a config file named `doschgpt.ini`. Modify the configuration file to suit your needs in this order. A sample file can be found with the binary.
+
+* API key: Place your key without qoutes (API key in this sample file has been revoked)
+* Model: Language model to use, can use `gpt-3.5-turbo`
+* Request Temperature: How random the completion will be. More [details](https://platform.openai.com/docs/guides/chat/instructing-chat-models)
+* Proxy hostname: Hostname IP of the proxy
+* Proxy port: Proxy Port
+* Outgoing start port: Start of a range of randomly selected outgoing port
+* Outgoing end port: End of a range of randomly selected outgoing port
+* Socket connect timeout (ms): How long to wait when attempting to connect to proxy
+* Socket response timeout (ms): How long to wait for OpenAI's servers to reply
+
+4. Ensure that your DOS environment has loaded the following
+
+* Packet Driver
+* MTCP Config Environment variable
+* MTCP Config file configured by DHCP
+
+5. Just launch `doschgpt.exe` in your machine and fire away. Press the ESC key to leave. Your may use the following arguments for debug use
+
+* `-dri`: Print the outgoing port, number of prompt and completion tokens used after each request
+* `-drr`: Display the raw server return headers and json reply
+
+<img src="images\doschgpt-5155-front-start.jpg" width="500">
+
+Parsed options will be displayed.
+
+## Compilation
+
+To compile this application, you have to use Open Watcom 2.0 beta which you can download from [here](https://github.com/open-watcom/open-watcom-v2/releases/tag/2023-03-04-Build). Open Watcom 2.0 for 64-bit Windows which was released on 2023-03-04 02:28:15 is used. The v1.9 version seems to create binaries with issues on some platforms.
+
+During installation, Open Watcom may prompt to install the environment variables. I chose to not use that to avoid having that variables being permanent. Instead I use a batch file to set the variables whenever I need to compile.
+
+The program is compiled via a Makefile that is referenced from MTCP.
+
+```bash
+# Open cmd.exe
+cd doschgpt-code
+
+# If using Open Watcom v2.0 beta installed to C:\WATCOM2
+20setenv.bat
+
+# If using Open Watcom v1.9 installed to C:\WATCOM
+19setenv.bat
+
+# To compile
+wmake
+
+#  Only if using Open Watcom 1.9. To patch the Open Watcom runtime to support Compaq Portable. Not needed for Open Watcom 2.0 beta.
+PTACH.exe doschgpt.exe doschgpt.map -ml
+
+# To clean
+wmake clean
+```
+
+This application compiles against the [MTCP library](http://brutmanlabs.org/mTCP/). I have unzipped the latest version [mTCP-src_2022-07-01.zip](http://www.brutman.com/mTCP/download/mTCP-src_2022-07-01.zip) at the time of development to the `mtcpsrc` directory. When Brutman updates this library again in future, simply replace the contents of the `mtcpsrc` directory with the new library.
+
+`PTACH.exe` is a Win NT program compiled from MTCP sources.
+
+## Development
+
+I use Visual Studio Code text editor to code for ease of use. For ease of testing, I used a virtual machine to run the application as 16-bit DOS applications and the MTCP network stack cannot run on modern Windows.
+
+<img src="images\doschgpt-vbox.png" width="600">
+
+More details of my setup can be found [here](https://github.com/yeokm1/retro-configs/tree/master/vms/vbox-dos622).
+
+To easily transfer the binary, I used Python to host my build directory as a temporary webserver. Then use the MTCP tool `htget` to fetch the binary.
+
+```bash
+# On modern machine with binary
+python3 -m http.server 8000
+
+# Run on DOS machine/VM
+htget -o doschgpt.exe http://X.X.X.X:8000/doschgpt.exe
+```
+
+### Mock proxy
+
+[OpenAI implements rate limits on their API](https://platform.openai.com/docs/guides/rate-limits/overview) hence we should minimise calling their API repeatedly.
+
+To avoid calling the OpenAI's servers during testing, we can mock the server's using this `mockprox.go`Go program that will replay the contents of `reply.txt` whenever the API call is received. 
+
+<img src="images\doschgpt-mock-proxy.png" width="600">
+
+```bash
+cp correct.txt reply.txt
+go build mockprox.go
+mockprox.exe
+```
+
+## ChatGPT APIs
+
+Only one ChatGPT API is used which is the chat completion.
+
+```bash
+# Test API directly
+curl https://api.openai.com/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer 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" -d '{ "model": "gpt-3.5-turbo", "messages": [{"role": "user", "content": "What is MS-DOS?"}], "temperature": 0.7 }'
+
+# Call through the https proxy for testing
+curl --proxy "http://192.168.1.144:8080" https://api.openai.com/v1/chat/completions -H "Content-Type: application/json" -H "Authorization: Bearer sk-EhmTsEsKyH4qHZL2mr3hT3BlbkFJd6AcfdBrujJsBBGPRcQ" -d '{ "model": "gpt-3.5-turbo", "messages": [{"role": "user", "content": "What is MS-DOS?"}], "temperature": 0.7 }'
+```
+
+# Changelog
+
+* v0.2 (30 Mar 2023):
+* * Compiled with Open Watcom 2.0 Beta (2023-03-04 build) that solves the issue of app not starting on some PCs.
+* * Show date and time of compilation
+* * Will parse and print quotes that were escaped from the JSON reply
+* * Reduce size of user text entry buffer from 10240 to 2048 characters to reduce memory usage.
+* * Use the same buffer for send and receive on the socket to further cut down memory usage.
+* * API Body buffer dropped to 4096 bytes
+* * Print only one decimal point for temperature at start
+* v0.1 (26 Mar 2023): 
+* * Initial release

api-data/correct.txt

@@ -0,0 +1 @@
+{"id":"chatcmpl-XXXXX","object":"chat.completion","created":1679326062,"model":"gpt-3.5-turbo-0301","usage":{"prompt_tokens":13,"completion_tokens":114,"total_tokens":127},"choices":[{"message":{"role":"assistant","content":"\n\nMS-DOS (Microsoft Disk Operating System) is a command-line operating system developed by Microsoft Corporation. It was first released in 1981 and was widely used in personal computers until the mid-1990s. MS-DOS is known for its command-line interface, which requires users to type in specific commands to execute tasks. It was also the predecessor to modern graphical user interface (GUI) operating systems, such as Windows. MS-DOS is no longer in common use today but is still used in some embedded systems and as a boot disk for troubleshooting purposes."},"finish_reason":"stop","index":0}]}

api-data/incorrect.txt

@@ -0,0 +1,8 @@
+{
+    "error": {
+        "message": "Incorrect API key provided: sk-EhmTs**************************************RcQh. You can find your API key at https://platform.openai.com/account/api-keys.",
+        "type": "invalid_request_error",
+        "param": null,
+        "code": "invalid_api_key"
+    }
+}

api-data/large.txt

@@ -0,0 +1 @@
+{"id":"chatcmpl-XXXXX","object":"chat.completion","created":1679326062,"model":"gpt-3.5-turbo-0301","usage":{"prompt_tokens":13,"completion_tokens":114,"total_tokens":127},"choices":[{"message":{"role":"assistant","content":"MS-DOS (Microsoft Disk Operating System) is a command-line operating system developed by Microsoft Corporation. It was first released in 1981 and was widely used in personal computers until the mid-1990s. MS-DOS is known for its command-line interface, which requires users to type in specific commands to execute tasks. It was also the predecessor to modern graphical user interface (GUI) operating systems, such as Windows. MS-DOS is no longer in common use today but is still used in some embedded systems and as a boot disk for troubleshooting purposes.0123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900\n\n123456789001234567890\n\n012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890"},"finish_reason":"stop","index":0}]}

api-data/mockprox.go

@@ -0,0 +1,105 @@
+package main
+
+import (
+	"bufio"
+	"fmt"
+	"log"
+	"net/http"
+	"os"
+	"strconv"
+)
+
+var httpListenPort = 80
+var replyText []byte
+
+func handler(responseToRequest http.ResponseWriter, incomingRequest *http.Request) {
+
+	host := incomingRequest.Host
+	url := incomingRequest.URL
+	log.Printf("Received request for host %s and url %s", host, url)
+	log.Printf("%s", url.Path)
+
+	var reply []byte = []byte("Unknown request")
+
+	switch incomingRequest.Method {
+	case "POST":
+		switch url.Path {
+		case "/v1/chat/completions":
+			reply = replyText
+		}
+
+	}
+
+	// Prepare the requesting socket for writing. Access raw socket by hijacking
+	// Reference: https://stackoverflow.com/questions/29531993/accessing-the-underlying-socket-of-a-net-http-response
+
+	hj, ok := responseToRequest.(http.Hijacker)
+	if !ok {
+		http.Error(responseToRequest, "webserver doesn't support hijacking", http.StatusInternalServerError)
+		return
+	}
+	returnConn, _, err := hj.Hijack()
+
+	if err != nil {
+		http.Error(responseToRequest, err.Error(), http.StatusInternalServerError)
+		return
+	}
+
+	defer returnConn.Close()
+
+	fmt.Println(string(reply))
+
+	returnConn.Write(reply)
+
+	log.Println("End of handler")
+
+}
+
+func main() {
+
+	argsWithoutProg := os.Args[1:]
+
+	if len(argsWithoutProg) >= 3 {
+		parsedHTTPPort, err := strconv.ParseInt(argsWithoutProg[0], 10, 32)
+
+		if err != nil {
+			log.Printf("Cannot parse argument %s", argsWithoutProg[0])
+			return
+		}
+
+		httpListenPort = int(parsedHTTPPort)
+	}
+
+
+	replyText, _ = readFile("reply.txt")
+
+	log.Printf("Starting Mock Server listening to %d", httpListenPort)
+
+	http.HandleFunc("/", handler)
+
+	if err := http.ListenAndServe(":"+strconv.Itoa(httpListenPort), nil); err != nil {
+		log.Fatal(err)
+	}
+}
+
+func readFile(filename string) ([]byte, error) {
+	file, err := os.Open(filename)
+
+	if err != nil {
+		return nil, err
+	}
+	defer file.Close()
+
+	stats, statsErr := file.Stat()
+	if statsErr != nil {
+		return nil, statsErr
+	}
+
+	var size int64 = stats.Size()
+	bytes := make([]byte, size)
+
+	bufr := bufio.NewReader(file)
+	_, err = bufr.Read(bytes)
+
+	return bytes, err
+}

api-data/reply.txt

@@ -0,0 +1 @@
+{"id":"chatcmpl-XXXXX","object":"chat.completion","created":1679326062,"model":"gpt-3.5-turbo-0301","usage":{"prompt_tokens":13,"completion_tokens":114,"total_tokens":127},"choices":[{"message":{"role":"assistant","content":"MS-DOS (Microsoft Disk Operating System) is a command-line operating system developed by Microsoft Corporation. It was first released in 1981 and was widely used in personal computers until the mid-1990s. MS-DOS is known for its command-line interface, which requires users to type in specific commands to execute tasks. It was also the predecessor to modern graphical user interface (GUI) operating systems, such as Windows. MS-DOS is no longer in common use today but is still used in some embedded systems and as a boot disk for troubleshooting purposes.0123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900\n\n123456789001234567890\n\n012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890012001234567890012345678900123456789001234567890012345678900123456789001234567890012345678900123456789001234567890"},"finish_reason":"stop","index":0}]}

doschgpt-code/19setenv.bat

@@ -0,0 +1,13 @@
+@echo off
+echo Open Watcom Build Environment
+SET PATH=C:\WATCOM\BINW;%PATH%
+SET PATH=C:\WATCOM\BINNT;%PATH%
+SET INCLUDE=C:\WATCOM\H\NT;%INCLUDE%
+SET INCLUDE=C:\WATCOM\H\NT;%INCLUDE%
+SET INCLUDE=%INCLUDE%;C:\WATCOM\H\NT\DIRECTX
+SET INCLUDE=%INCLUDE%;C:\WATCOM\H\NT\DDK
+SET INCLUDE=C:\WATCOM\H;%INCLUDE%
+SET WATCOM=C:\WATCOM
+SET EDPATH=C:\WATCOM\EDDAT
+SET WHTMLHELP=C:\WATCOM\BINNT\HELP
+SET WIPFC=C:\WATCOM\WIPFC

doschgpt-code/20setenv.bat

@@ -0,0 +1,8 @@
+@echo off
+echo Open Watcom Build Environment
+PATH C:\WATCOM2\BINNT64;C:\WATCOM2\BINNT;%PATH%
+SET INCLUDE=C:\WATCOM2\H;C:\WATCOM2\H\NT;C:\WATCOM2\H\NT\DIRECTX;C:\WATCOM2\H\NT\DDK;%INCLUDE%
+SET WATCOM=C:\WATCOM2
+SET EDPATH=C:\WATCOM2\EDDAT
+SET WHTMLHELP=C:\WATCOM2\BINNT\HELP
+SET WIPFC=C:\WATCOM2\WIPFC