Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 BUG: calling fetch inside fetch handler throws internal error #7787

Open
hasangenc0 opened this issue Jan 15, 2025 · 17 comments
Open

🐛 BUG: calling fetch inside fetch handler throws internal error #7787

hasangenc0 opened this issue Jan 15, 2025 · 17 comments
Labels
bug Something that isn't working needs reproduction Needs reproduction from OP

Comments

@hasangenc0
Copy link

hasangenc0 commented Jan 15, 2025

Which Cloudflare product(s) does this pertain to?

Workers Runtime

What versions are you using?

3.103.0 [wrangler] 23.1.0 [node]

What operating system and version are you using?

Apple M1 Max MacOS Sonoma 14.7.2 (23H311)

Please provide a link to a minimal reproduction

https://github.com/hasangenc0/cf-worker-bug-repro

Describe the Bug

Just copy paste the example code in this URL.
Also repro here.

Run worker, then make a request.
You will see Error: internal error

Image

I'm wondering is that a outdated documentation or it's a real bug. But I'm sure that calling fetch inside fetch definitely causes this issue.

Please provide any relevant error logs

No response

@hasangenc0 hasangenc0 added the bug Something that isn't working label Jan 15, 2025
@github-project-automation github-project-automation bot moved this to Untriaged in workers-sdk Jan 15, 2025
@hasangenc0 hasangenc0 changed the title 🐛 BUG: calling fetch inside fetch export throws internal error 🐛 BUG: calling fetch inside fetch handler throws internal error Jan 15, 2025
@vicb
Copy link
Contributor

vicb commented Jan 16, 2025

This seems to work

3.60.3 [wrangler]

Maybe try to update wrangler

@hasangenc0
Copy link
Author

Maybe try to update wrangler

Thanks @vicb! Updated to 3.103.0 but still not working, I wonder fetch call in CF workers work different in local? Could anybody try locally?

@vicb
Copy link
Contributor

vicb commented Jan 16, 2025

export default {
	async fetch(request, env, ctx): Promise<Response> {
		const someHost = 'https://jsonplaceholder.typicode.com';
		const url1 = someHost + '/todos/1';
		const url2 = someHost + '/todos/2';

		const responses = await Promise.all([fetch(url1), fetch(url2)]);
		const results = await Promise.all(responses.map((r) => r.json()));

		const options = {
			headers: { 'content-type': 'application/json;charset=UTF-8' },
		};
		return new Response(JSON.stringify(results), options);
	},
};

works for me.

Can you try to clear your cache (or incognito), try/cache and log the exception details?

@hasangenc0
Copy link
Author

This is the exception trace @vicb:

Error: internal error
    at [object Object]
    at async Object.fetch (file:///Users/src/cf-worker-bug-repro/src/index.ts:8:21)
    at async jsonError (file:///Users/src/cf-worker-bug-repro/node_modules/.pnpm/wrangler@3.103.0_@cloudflare+workers-types@4.20250109.0/node_modules/wrangler/templates/middleware/middleware-miniflare3-json-error.ts:22:10)
    at async drainBody (file:///Users/src/cf-worker-bug-repro/node_modules/.pnpm/wrangler@3.103.0_@cloudflare+workers-types@4.20250109.0/node_modules/wrangler/templates/middleware/middleware-ensure-req-body-drained.ts:5:10)%

I'm using postman so no cookies or cache

Image

@hasangenc0
Copy link
Author

Also Is it possible that this won't work due to firewalls?

@vicb
Copy link
Contributor

vicb commented Jan 16, 2025

I think you should use GET instead, not sure if this can be the root cause

@hasangenc0
Copy link
Author

I actually use GET not sure why it says POST in the error

export default {
	async fetch(request) {
		// someHost is set up to return JSON responses
		console.log(request.method);
		const someHost = 'https://jsonplaceholder.typicode.com';
		const url1 = someHost + '/todos/1';
		const url2 = someHost + '/todos/2';

		const responses = await Promise.all([fetch(url1), fetch(url2)]);
		const results = await Promise.all(responses.map((r) => r.json()));

		const options = {
			headers: { 'content-type': 'application/json;charset=UTF-8' },
		};
		return new Response(JSON.stringify(results), options);
	},
} satisfies ExportedHandler;
Image

@hasangenc0
Copy link
Author

hasangenc0 commented Jan 17, 2025

More trace, fetch call is throwing the below error

"Error: internal error\n    at async Promise.all (index 1)\n    at async getData (file:///Users/src/cf-worker-bug-
repro/.wrangler/tmp/dev-w6652a/index.js:8:21)\n    at async Object.fetch (file:///Users/src/cf-worker-bug-
repro/.wrangler/tmp/dev-w6652a/index.js:22:23)\n    at async jsonError (file:///Users/src/cf-worker-bug-
repro/.wrangler/tmp/dev-w6652a/index.js:67:12)\n    at async drainBody (file:///Users/src/cf-worker-bug-
repro/.wrangler/tmp/dev-w6652a/index.js:38:12)"

@vicb
Copy link
Contributor

vicb commented Jan 17, 2025

You should try to log response.ok and response.statusText
Aloso maybe try response.text() instead of .json

@hasangenc0
Copy link
Author

Error is happening in fetch call tho, it's not even reaching .json

Here's a screen recording:

cf-error.mov

@hasangenc0
Copy link
Author

Is there a way to set up another http client other than fetch?

@penalosa penalosa added the needs reproduction Needs reproduction from OP label Jan 17, 2025
@penalosa
Copy link
Contributor

@hasangenc0 I've cloned your repro and everything seems to work for me. To clarify, your reproduction talks about a GET request, but the screenshot of Postman you've included has a body and sends a POST request. Can you reproduce this with just the example reproduction you provided?

@jldec
Copy link

jldec commented Jan 17, 2025

@hasangenc0 I played with https://github.com/hasangenc0/cf-worker-bug-repro and found that I could reproduce the behavior you reported by setting someHost to a bogus endpoint e.g. https://jsonplaceholder.typicode.coms (with an extra s)

This suggests that the problem may be caused by your local network not being able to resolve the endpoint in the code, or some other network failure which would cause fetch() to throw. You might be able to repro this with curl https://jsonplaceholder.typicode.com/todos/1.

The issue in wrangler seems to be that the stack trace for the fetch error is not captured properly, so the error message is misleading.

This becomes more obvious if you catch and return the error instead of letting the worker fail.
E.g.

export default {
	async fetch(request) {
		// someHost is set up to return JSON responses
		const someHost = 'https://jsonplaceholder.typicode.coms';
		const url1 = someHost + '/todos/1';
		const url2 = someHost + '/todos/2';

		let results;
		try {
			const resp1 = await fetch(url1);
			const resp2 = await fetch(url2);
			const json1 = await resp1.json();
			const json2 = await resp2.json();
			results = JSON.stringify([json1, json2], null, 2);
		} catch (e: any) {
			results = `${e.message}\n${e.stack}`;
		}
		return new Response(results);
	},
} satisfies ExportedHandler;

returns

internal error
Error: internal error
    at async Object.fetch (file:///Users/jldec/cloudflare/issue-7787/.wrangler/tmp/dev-PofTSg/index.js:12:21)
    at async jsonError (file:///Users/jldec/cloudflare/issue-7787/.wrangler/tmp/dev-PofTSg/index.js:55:12)
    at async drainBody (file:///Users/jldec/cloudflare/issue-7787/.wrangler/tmp/dev-PofTSg/index.js:28:12)

@hasangenc0
Copy link
Author

hasangenc0 commented Jan 17, 2025

Your reproduction talks about a GET request, but the screenshot of Postman you've included has a body and sends a POST request.

@penalosa I'm actually sending GET request and check the screenshot in the description and here, don't know why the error page shows POST.

To clarify everything I recorded the e2e experience :)

I send GET with no request body, using https://github.com/hasangenc0/cf-worker-bug-repro. (updated to @jldec's example)

Reproduction video

@hasangenc0
Copy link
Author

You might be able to repro this with curl https://jsonplaceholder.typicode.com/todos/1.

@jldec that's exactly what I can't figure out, curl just works, so I don't think there's an issue with the local network?

cf-cf.mov

@hasangenc0
Copy link
Author

Btw without the extra (s) is the same for me, see below:

Screen.Recording.2025-01-17.at.22.56.02.mov

Is there any way that I can debug the fetch call cause the error message is not very clear?

internal error
Error: internal error
    at async Object.fetch (file:///Users/hgenc/opensource/cf-worker-bug-repro/.wrangler/tmp/dev-PIjzxh/index.js:12:21)
    at async jsonError (file:///Users/hgenc/opensource/cf-worker-bug-repro/.wrangler/tmp/dev-PIjzxh/index.js:55:12)
    at async drainBody (file:///Users/hgenc/opensource/cf-worker-bug-repro/.wrangler/tmp/dev-PIjzxh/index.js:28:12)

@hasangenc0
Copy link
Author

Oh wow, replacing https to http worked! Does anyone know why?

export default {
	async fetch(request) {
		// someHost is set up to return JSON responses
		const someHost = 'http://jsonplaceholder.typicode.com';
		const url1 = someHost + '/todos/1';
		const url2 = someHost + '/todos/2';

		let results;
		try {
			const resp1 = await fetch(url1);
			const resp2 = await fetch(url2);
			const json1 = await resp1.json();
			const json2 = await resp2.json();
			results = JSON.stringify([json1, json2], null, 2);
		} catch (e: any) {
			results = `${e.message}\n${e.stack}`;
		}
		return new Response(results);
	},
} satisfies ExportedHandler;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something that isn't working needs reproduction Needs reproduction from OP
Projects
Status: Untriaged
Development

No branches or pull requests

4 participants