No fetch implementation found

[codehruv]

Hi,

Thanks for building this great project.

When I deployed the changes using the CJS implementation to my machine, it surfaced an error

Error: No fetch implementation found.

After doing some reading, it seems like I have to install the node-fetch library and import that into the openai-streams library fetch call somehow.

Is there a clean way for doing this?

[ctjlewis]

You have to install node-fetch and pass it in as { fetch: nodeFetch } if your runtime does not have fetch available. Newer versions of Node have it ready out of the box.

The library originally shimmed this for you, but us having node-fetch in the bundle made it toxic for Edge environments so you have to pass it in.

It's tested against this though, so if you just pass in node-fetch you'll be OK. See tests (mockFetch is just node-fetch):

openai-streams/test/streams.test.ts

Lines 250 to 256 in f98e6b1

	const stream = await OpenAI("chat", {
	model: "gpt-3.5-turbo",
	messages: [
	{ role: "system", content: "You are a helpful assistant." },
	{ role: "user", content: "This is a test message, say hello!" },
	],
	}, { fetch: mockFetch });

[codehruv]

Thanks for the quick response.

That fixed the issue and it resurfaced another issue.

ReferenceError: ReadableStream is not defined

I couldn't find this in the tests file you shared. Is this a common error mode as well?

[ctjlewis]

You have to use openai-streams/node if you're on Node, this way the library will use NodeJS.Readable instead of WHATWG ReadableStream, which is for Edge runtime.

[codehruv]

I'm currently using openai-streams/node itself. Is there a workaround to somehow get the ReadableStream into the code via another package (similar to the fetch solution)?

Thanks for the idea on upgrading node. You are right. I will make that upgrade soon.

[ctjlewis]

Sorry, that was actually my mistake, I edited my response - ReadableStream is available in Edge runtime/browser, but not Node.js, so openai-streams/node is needed. I guess Node does support it on v16+ though, so if you upgraded to 16+ it would go away.

That being said, if you're getting that error from openai-streams/node, that would be a bug.

[ctjlewis]

Can you post a repro using Replit if you have time? Also what version of Node are you on (presumably 14)?

[codehruv]

Machine info

Node version: v16.16.0
OS: Amazon Linux

Code

const { OpenAI } = require("openai-streams/node");
const fetch = require('cross-fetch');

// inside a function
const stream = await OpenAI(
    "completions", 
    {
        prompt: requestBody.prompt,
        model: requestBody.model,
        max_tokens: requestBody.max_tokens,
        temperature: requestBody.temperature,
        top_p: requestBody.top_p,
        frequency_penalty: requestBody.frequency_penalty,
        presence_penalty: requestBody.presence_penalty,
        stop: requestBody.stop
    },
    {
        apiKey: openai_key,
        fetch: fetch
    });
    res.set({
    'Content-Type': 'application/json; charset=utf-8',
    'Transfer-Encoding': 'chunked'
    });
    
    // we need to track what was generated
    const chunks = [];
    var chunkText = "";
    
    for await (const chunk of stream) {
    if (!chunk) continue;
    chunkText = Buffer.from(chunk).toString("utf-8");
    chunks.push(chunkText);
    res.write(chunkText);
    }

Error Message

ReferenceError: ReadableStream is not defined
at N (pwd/node_modules/openai-streams/dist/lib/streaming/streams.js:5:796)
at V (pwd/node_modules/openai-streams/dist/lib/streaming/streams.js:5:1538)
at D (pwd/node_modules/openai-streams/dist/lib/openai/edge.js:1:1692)
at runMicrotasks ()
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async u (pwd/node_modules/openai-streams/dist/lib/openai/node.js:1:675)

[codehruv]

Even after switching from cross-fetch to node-fetch the error persists

[codehruv]

Upgraded my node version to 17. There's some issue with moving to version 21 on my machine right now.

[ctjlewis]

Upgraded my node version to 17. There's some issue with moving to version 21 on my machine right now.

Issue resolved? We'll get you up and running and add tests if needed

[ctjlewis]

If you're on 16, you can probably just use it normally again. See if ReadableStream is available:

$ node
Welcome to Node.js v18.8.0.
Type ".help" for more information.
> console.log(ReadableStream)
[class ReadableStream]

If you see it, just try the library how it comes out of the box, without custom { fetch } or openai-streams/node etc:

$ node
Welcome to Node.js v18.8.0.
Type ".help" for more information.
> await import('openai-streams')
[Module: null prototype] {
  ChatParser: [AsyncGeneratorFunction: O],
  ChatStream: [Function: v],
  EventStream: [Function: c],
  LogprobsParser: [AsyncGeneratorFunction: l],
  ...

[codehruv]

$ node
Welcome to Node.js v17.0.0.
Type ".help" for more information.
> console.log(ReadableStream)
Uncaught ReferenceError: ReadableStream is not defined
>

Looks like I don't have it on v17.0.0. I will try updating to 18.8.0 to see if that solves the issue.

[codehruv]

The OS on my machine doesn't allow upgrading to a higher version of node at the moment.

Is there a way to resolve this issue without upgrading my OS?

[ctjlewis]

I'll try to dig into this later. Sorry it's been such a pain for you to use. You're on a pretty modern version of Node and this is meant to work for your use case.

It's weird openai-streams/node doesn't work, we use it in pretty restrictive CJS runtimes (e.g. VS Code extension) successfully. I'll investigate it.