Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

StrawberryGraphQLError in Custom Extension causes 500 error response instead of formatted error #3204

Closed
staubina opened this issue Nov 6, 2023 · 12 comments · Fixed by #3217
Assignees
Labels
bug Something isn't working

Comments

@staubina
Copy link

staubina commented Nov 6, 2023

Description

I am attempting to add a Custom Extension to catch queries that are too large. I then want the extension to raise StrawberryGraphQLError and return a properly formatted error message.

The issue is that when I raise StrawberryGraphQLError an internal error response is sent to the requester.

MAX_QUERY_LENGTH = 8192 # 8KB

class MaxQueryLengthExtension(SchemaExtension):
    async def on_operation(self):
        if len(self.execution_context.query) > MAX_QUERY_LENGTH:
            raise StrawberryGraphQLError(message="Query too large")
        yield

Stack Trace

2023-11-06 11:26:31 Exception in ASGI application
2023-11-06 11:26:31 Traceback (most recent call last):
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/uvicorn/protocols/http/httptools_impl.py", line 426, in run_asgi
2023-11-06 11:26:31     result = await app(  # type: ignore[func-returns-value]
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
2023-11-06 11:26:31     return await self.app(scope, receive, send)
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/fastapi/applications.py", line 292, in __call__
2023-11-06 11:26:31     await super().__call__(scope, receive, send)
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/starlette/applications.py", line 122, in __call__
2023-11-06 11:26:31     await self.middleware_stack(scope, receive, send)
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/ddtrace/contrib/asgi/middleware.py", line 275, in __call__
2023-11-06 11:26:31     reraise(exc_type, exc_val, exc_tb)
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/six.py", line 719, in reraise
2023-11-06 11:26:31     raise value
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/ddtrace/contrib/asgi/middleware.py", line 270, in __call__
2023-11-06 11:26:31     return await self.app(scope, receive, wrapped_send)
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 184, in __call__
2023-11-06 11:26:31     raise exc
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 162, in __call__
2023-11-06 11:26:31     await self.app(scope, receive, _send)
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/starlette/middleware/cors.py", line 83, in __call__
2023-11-06 11:26:31     await self.app(scope, receive, send)
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
2023-11-06 11:26:31     raise exc
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
2023-11-06 11:26:31     await self.app(scope, receive, sender)
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
2023-11-06 11:26:31     raise e
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
2023-11-06 11:26:31     await self.app(scope, receive, send)
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 718, in __call__
2023-11-06 11:26:31     await route.handle(scope, receive, send)
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 276, in handle
2023-11-06 11:26:31     await self.app(scope, receive, send)
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 66, in app
2023-11-06 11:26:31     response = await func(request)
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/fastapi/routing.py", line 273, in app
2023-11-06 11:26:31     raw_response = await run_endpoint_function(
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/fastapi/routing.py", line 190, in run_endpoint_function
2023-11-06 11:26:31     return await dependant.call(**values)
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/strawberry/fastapi/router.py", line 227, in handle_http_post
2023-11-06 11:26:31     return await self.run(
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/strawberry/http/async_base_view.py", line 176, in run
2023-11-06 11:26:31     result = await self.execute_operation(
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/strawberry/http/async_base_view.py", line 115, in execute_operation
2023-11-06 11:26:31     return await self.schema.execute(
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/strawberry/schema/schema.py", line 256, in execute
2023-11-06 11:26:31     result = await execute(
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/strawberry/schema/execute.py", line 86, in execute
2023-11-06 11:26:31     async with extensions_runner.operation():
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/strawberry/extensions/context.py", line 191, in __aenter__
2023-11-06 11:26:31     await self.run_hooks_async()
2023-11-06 11:26:31   File "/usr/local/lib/python3.9/site-packages/strawberry/extensions/context.py", line 175, in run_hooks_async
2023-11-06 11:26:31     await hook.initialized_hook.__anext__()  # type: ignore[union-attr]
2023-11-06 11:26:31   File "/src/src/routers/graphql.py", line 78, in on_operation
2023-11-06 11:26:31     raise StrawberryGraphQLError(message="Query too large")
2023-11-06 11:26:31 strawberry.exceptions.StrawberryGraphQLError: Query too large

Describe the Bug

System Information

  • Operating system: Mac with Docker containers
  • Strawberry version (if applicable): strawberry-graphql[fastapi] == 0.209.2

Upvote & Fund

  • We're using Polar.sh so you can upvote and help fund this issue.
  • We receive the funding once the issue is completed & confirmed by you.
  • Thank you in advance for helping prioritize & fund our backlog.
Fund with Polar
@staubina staubina added the bug Something isn't working label Nov 6, 2023
@DoctorJohn
Copy link
Member

DoctorJohn commented Nov 10, 2023

Hey @staubina, with our current implementation this seems to be expected.

You might want to try to add your validation error to the execution context, similar to how our built-in ValidationCache and MaskErrors extensions do it. Based on your code this would roughtly look like this:

MAX_QUERY_LENGTH = 8192 # 8KB

class MaxQueryLengthExtension(SchemaExtension):
    async def on_operation(self):
        execution_context = self.execution_context

        if len(self.execution_context.query) > MAX_QUERY_LENGTH:
            error = StrawberryGraphQLError(message="Query too large")
            execution_context.errors = [error]
        yield

EDIT: as pointed out by Patrick, this won't work either since further execution is not stopped by simply adding an error to the execution context.

@patrick91
Copy link
Member

@jkimbo @DoctorJohn do you remember why we didn't send these errors to the response object?

/cc @erikwrede do you have any opinion on this?

Context: #1324

@erikwrede
Copy link
Member

erikwrede commented Nov 10, 2023

@patrick91 for continuity this should be handled the same way we handle field extension errors or normal resolver errors, like in the permission extension. [Strawberry/GraphQL-Core]GQLErrors should automatically be added to the execution context while all other errors should cause an Internal Server error.

The Schema Extension error should in any case lead to stopping the request. There should be no way to throw an error during on_operation and have the execution continue

@patrick91
Copy link
Member

Hey @staubina, with our current implementation this seems to be expected.

You might want to try to add your validation error to the execution context, similar to how our built-in ValidationCache and MaskErrors extensions do it. Based on your code this would roughtly look like this:

MAX_QUERY_LENGTH = 8192 # 8KB

class MaxQueryLengthExtension(SchemaExtension):
    async def on_validate(self):
        execution_context = self.execution_context

        if len(self.execution_context.query) > MAX_QUERY_LENGTH:
            error = StrawberryGraphQLError(message="Query too large")
            execution_context.errors = [error]
        yield

I missed your reply before answering! I think this won't work well because we'd have parsed the query by then, and we want to prevent to parse huge strings 😊

@patrick91 for continuity this should be handled the same way we handle field extension errors or normal resolver errors, like in the permission extension. [Strawberry/GraphQL-Core]GQLErrors should automatically be added to the execution context while all other errors should cause an Internal Server error.

So you'd suggest to only handle GraphQL errors in our execute function?

The Schema Extension error should in any case lead to stopping the request. There should be no way to throw an error during on_operation and have the execution continue

Yup, that makes sense!

@DoctorJohn
Copy link
Member

DoctorJohn commented Nov 10, 2023

I missed your reply before answering! I think this won't work well because we'd have parsed the query by then, and we want to prevent to parse huge strings 😊

Well spotted! Just noticed too that only adding the error does not stop further execution.

@jkimbo @DoctorJohn do you remember why we didn't send these errors to the response object?

As far as I can tell, rrrors raised during on_operation (previously on_request) were never handled since extensions were added. However, on_parse used to handle errors until #1324 unintentionally broke handling which probably went unnoticed due to the lack of tests.

I don't see a reason not catch those errors (at least GraphQL errors) and to add them to the result and end execution. Pretty sure this was just an oversight or that we thought there was no need for raising GraphQL errors on_operation.

@DoctorJohn
Copy link
Member

So you'd suggest to only handle GraphQL errors in our execute function?

Wondering about that too :) At least from the perspective of integrations, all "normal resolvers errors" are eventually converted into GraphQL(core)Errors and included in the results errors. For consistency we probably want any error raised from within extensions to be handled similarly?

@erikwrede
Copy link
Member

So you'd suggest to only handle GraphQL errors in our execute function?

For consistency we probably want any error raised from within extensions to be handled similarly [to one raised from withing resolvers]?

After looking at it again, I believe we should handle all errors the same, no matter if they originate in the extension or in the resolver. The detailed decision of how non-GraphQL Errors are handled could be moved to an ExceptionFilter in the web views.

@patrick91
Copy link
Member

After looking at it again, I believe we should handle all errors the same, no matter if they originate in the extension or in the resolver. The detailed decision of how non-GraphQL Errors are handled could be moved to an ExceptionFilter in the web views.

I think so too, we already treat any error inside resolvers as GraphQL errors, it is weird to not do that for parsing and operation errors 😊

@DoctorJohn what do you think?

@DoctorJohn
Copy link
Member

I think so too, we already treat any error inside resolvers as GraphQL errors, it is weird to not do that for parsing and operation errors 😊

@DoctorJohn what do you think?

I agree. Already started working on it. Turns out 83 testc cases depend on the current behaviour. Looks like mainly for assertions within tests. I'll make sure to check whether any public facing and tested behaviour changes.

@DoctorJohn DoctorJohn self-assigned this Nov 11, 2023
@staubina
Copy link
Author

staubina commented Nov 13, 2023

After reading the docs here: https://strawberry.rocks/docs/guides/custom-extensions a few more times, I was able to get the following to work for now.

Any potential pitfalls I am not seeing yet?

from graphql.error import GraphQLError
from graphql import ExecutionResult as GraphQLExecutionResult
...

MAX_QUERY_LENGTH = 8192 # 8KB


class MaxQueryLengthExtension(SchemaExtension):
    def on_execute(self):
        if len(self.execution_context.query) > MAX_QUERY_LENGTH:
            self.execution_context.result = GraphQLExecutionResult(
                data=None,
                errors=[GraphQLError("Query too large")],
            )


Response

{
    "data": null,
    "errors": [
        {
            "message": "Query too large"
        }
    ]
}

@DoctorJohn
Copy link
Member

I was able to get the following to work for now.

Good job! You're correct, supplying a result from within the on_excute hook will prevent further evaluation.

This is because of this check:

with extensions_runner.executing():
if not execution_context.result:

Any potential pitfalls I am not seeing yet?

Your query will be fully parsed and validated even when it's too long. If your goal is to mitigate DDoS attacks, performing the parsing and validation steps might be unwanted.

@staubina
Copy link
Author

That was my fear, I would like to stop it at an earlier stage for DDoS protection, but as you are all discussing any level above execute throws a 500. I was not able to determine an approach to catching the exception and handling it in my FastAPI app code.

This is a start at least.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants