Windows API server doesn't bind to correct interface - but I can make it do it.

[DrJaymz]

API not contactable outside of machine. If there are multiple interfaces or Hyper-V virtual switch then cannot get it to listen on the correct interface. Do not seem to be able to set this on the CLI or get it to listen on all interfaces. I would expect by default to bind to 0.0.0.0 - so we don't need to worry about it.

Steps:

Localhost OK:
curl -X POST http://localhost:4891/v1/chat/completions -H "Content-Type: application/json" -d '{"model": "Llama-3-8B-Instruct", "messages": [{"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What is the capital of France?"}]}'
{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"content":"The capital of France is Paris.","role":"assistant"},"references":null}],"created":1739447845,"id":"placeholder","model":"Llama 3 8B Instruct","object":"chat.completion","usage":{"completion_tokens":7,"prompt_tokens":30,"total_tokens":37}} ✔

Actual IP No good:
curl -X POST http://192.168.0.100:4891/v1/chat/completions -H "Content-Type: application/json" -d '{"model": "Llama-3-8B-Instruct", "messages": [{"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What is the capital of France?"}]}'

Using Windows 11 - Workaround manually connect up:
$ netsh interface portproxy add v4tov4 listenport=4891 listenaddress=0.0.0.0 connectport=4891 connectaddress=127.0.0.1

Now:
curl -X POST http://192.168.0.100:4891/v1/chat/completions -H "Content-Type: application/json" -d '{"model": "Llama-3-8B-Instruct", "messages": [{"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What is the capital of France?"}]}'
{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"content":"The capital of France is Paris!","role":"assistant"},"references":null}],"created":1739448277,"id":"placeholder","model":"Llama 3 8B Instruct","object":"chat.completion","usage":{"completion_tokens":7,"prompt_tokens":30,"total_tokens":37}}

There is probably something obvious but I checked the documentation and didn't find the answer or a similar question.