diff --git a/README.md b/README.md index 7119ec97..46901720 100644 --- a/README.md +++ b/README.md @@ -436,6 +436,7 @@ The [Executor](./docs/executor.md) component builds atop the Ruby fiber-based im This repo includes working examples of stitched schemas running across small Rack servers. Clone the repo, `cd` into each example and try running it following its README instructions. - [Merged types](./examples/merged_types) +- [File uploads](./examples/file_uploads) ## Tests diff --git a/docs/http_executable.md b/docs/http_executable.md new file mode 100644 index 00000000..6b3b9109 --- /dev/null +++ b/docs/http_executable.md @@ -0,0 +1,48 @@ +## GraphQL::Stitching::HttpExecutable + +A `HttpExecutable` provides an out-of-the-box convenience for sending HTTP post requests to a remote location, or a base class for your own implementation with [GraphQL multipart uploads](https://github.com/jaydenseric/graphql-multipart-request-spec?tab=readme-ov-file#multipart-form-field-structure). + +```ruby +exe = GraphQL::Stitching::HttpExecutable.new( + url: "http://localhost:3001", + headers: { + "Authorization" => "..." + } +) +``` + +### GraphQL Uploads via multipart forms + +The [GraphQL Upload Spec](https://github.com/jaydenseric/graphql-multipart-request-spec) defines a multipart form structure for submitting GraphQL requests that include file upload attachments. It is possible to flow these requests through a stitched schema using the following steps: + +1. File uploads must be submitted to stitching as basic GraphQL variables with `Tempfile` values assigned. The simplest way to recieve this input is to install [apollo_upload_server](https://github.com/jetruby/apollo_upload_server-ruby) into your stitching app's middleware so that multipart form submissions arrive unpackaged and in the expected format. + +```ruby +client.execute( + "mutation($file: Upload) { upload(file: $file) }", + variables: { "file" => Tempfile.new(...) } +) +``` + +2. Stitching will route the request and its variables as normal. Then it's up to `HttpExecutable` to re-package any upload variables into the multipart form spec before sending them upstream. This is enabled with an `upload_types` parameter to tell the executable what scalar names must be extracted: + +```ruby + +client = GraphQL::Stitching::Client.new(locations: { + products: { + schema: GraphQL::Schema.from_definition(...), + executable: GraphQL::Stitching::HttpExecutable.new( + url: "http://localhost:3000", + upload_types: ["Upload"], # << extract "Upload" scalars into multipart forms + ), + }, + showtimes: { + schema: GraphQL::Schema.from_definition(...), + executable: GraphQL::Stitching::HttpExecutable.new( + url: "http://localhost:3001" + ), + }, +}) +``` + +Note that `upload_types` adds request processing, so it should only be enabled for locations that actually recieve file uploads. Those locations can again leverage [apollo_upload_server](https://github.com/jetruby/apollo_upload_server-ruby) to unpack the multipart form sent by stitching. diff --git a/examples/file_uploads/Gemfile b/examples/file_uploads/Gemfile new file mode 100644 index 00000000..8ede7a05 --- /dev/null +++ b/examples/file_uploads/Gemfile @@ -0,0 +1,9 @@ +# frozen_string_literal: true + +source 'https://rubygems.org' + +gem 'rack' +gem 'rackup' +gem 'foreman' +gem 'graphql' +gem 'apollo_upload_server', '2.1' diff --git a/examples/file_uploads/Procfile b/examples/file_uploads/Procfile new file mode 100644 index 00000000..17089898 --- /dev/null +++ b/examples/file_uploads/Procfile @@ -0,0 +1,2 @@ +gateway: bundle exec ruby gateway.rb +remote: bundle exec ruby remote.rb diff --git a/examples/file_uploads/README.md b/examples/file_uploads/README.md new file mode 100644 index 00000000..542faa32 --- /dev/null +++ b/examples/file_uploads/README.md @@ -0,0 +1,13 @@ +```shell +curl -X POST http://localhost:3000 \ + -H 'Content-Type: application/json' \ + -d '{"query":"{ gateway remote }"}' +``` + +```shell +curl http://localhost:3000 \ + -H 'Content-Type: multipart/form-data' \ + -F operations='{ "query": "mutation ($file: Upload!) { gateway upload(file: $file) }", "variables": { "file": null } }' \ + -F map='{ "0": ["variables.file"] }' \ + -F 0=@file.txt +``` diff --git a/examples/file_uploads/file.txt b/examples/file_uploads/file.txt new file mode 100644 index 00000000..c57eff55 --- /dev/null +++ b/examples/file_uploads/file.txt @@ -0,0 +1 @@ +Hello World! \ No newline at end of file diff --git a/examples/file_uploads/gateway.rb b/examples/file_uploads/gateway.rb new file mode 100644 index 00000000..66e02b09 --- /dev/null +++ b/examples/file_uploads/gateway.rb @@ -0,0 +1,37 @@ +# frozen_string_literal: true + +require 'rackup' +require 'json' +require 'graphql' +require_relative '../../lib/graphql/stitching' +require_relative './helpers' + +class StitchedApp + def initialize + @client = GraphQL::Stitching::Client.new(locations: { + gateway: { + schema: GatewaySchema, + }, + remote: { + schema: RemoteSchema, + executable: GraphQL::Stitching::HttpExecutable.new( + url: "http://localhost:3001", + upload_types: ["Upload"] + ), + }, + }) + end + + def call(env) + params = apollo_upload_server_middleware_params(env) + result = @client.execute( + query: params["query"], + variables: params["variables"], + operation_name: params["operationName"], + ) + + [200, {"content-type" => "application/json"}, [JSON.generate(result)]] + end +end + +Rackup::Handler.default.run(StitchedApp.new, :Port => 3000) diff --git a/examples/file_uploads/helpers.rb b/examples/file_uploads/helpers.rb new file mode 100644 index 00000000..02eeeee7 --- /dev/null +++ b/examples/file_uploads/helpers.rb @@ -0,0 +1,62 @@ +# frozen_string_literal: true + +require 'action_dispatch' +require 'apollo_upload_server/graphql_data_builder' +require 'apollo_upload_server/upload' + +# ApolloUploadServer middleware only modifies Rails request params; +# for simple Rack apps we need to extract the behavior. +def apollo_upload_server_middleware_params(env) + req = ActionDispatch::Request.new(env) + if env['CONTENT_TYPE'].to_s.include?('multipart/form-data') + ApolloUploadServer::GraphQLDataBuilder.new(strict_mode: true).call(req.params) + else + req.params + end +end + +# Gateway local schema +class GatewaySchema < GraphQL::Schema + class Query < GraphQL::Schema::Object + field :gateway, Boolean, null: false + + def gateway + true + end + end + + class Mutation < GraphQL::Schema::Object + field :gateway, Boolean, null: false + + def gateway + true + end + end + + query Query + mutation Mutation +end + +# Remote local schema, with file upload +class RemoteSchema < GraphQL::Schema + class Query < GraphQL::Schema::Object + field :remote, Boolean, null: false + + def remote + true + end + end + + class Mutation < GraphQL::Schema::Object + field :upload, String, null: true do + argument :file, ApolloUploadServer::Upload, required: true + end + + def upload(file:) + file.read + end + end + + query Query + mutation Mutation +end diff --git a/examples/file_uploads/remote.rb b/examples/file_uploads/remote.rb new file mode 100644 index 00000000..99b78731 --- /dev/null +++ b/examples/file_uploads/remote.rb @@ -0,0 +1,21 @@ +# frozen_string_literal: true + +require 'rackup' +require 'json' +require 'graphql' +require_relative './helpers' + +class RemoteApp + def call(env) + params = apollo_upload_server_middleware_params(env) + result = RemoteSchema.execute( + query: params["query"], + variables: params["variables"], + operation_name: params["operationName"], + ) + + [200, {"content-type" => "application/json"}, [JSON.generate(result)]] + end +end + +Rackup::Handler.default.run(RemoteApp.new, :Port => 3001) diff --git a/lib/graphql/stitching/http_executable.rb b/lib/graphql/stitching/http_executable.rb index 80ecf598..ebd8824f 100644 --- a/lib/graphql/stitching/http_executable.rb +++ b/lib/graphql/stitching/http_executable.rb @@ -7,18 +7,148 @@ module GraphQL module Stitching class HttpExecutable - def initialize(url:, headers:{}) + def initialize(url:, headers:{}, upload_types: nil) @url = url @headers = { "Content-Type" => "application/json" }.merge!(headers) + @upload_types = upload_types end - def call(_location, document, variables, _request) - response = Net::HTTP.post( + def call(_location, document, variables, request) + multipart_form = if request.variable_definitions.any? && variables&.any? + extract_multipart_form(document, variables, request) + end + + response = if multipart_form + post_multipart(multipart_form) + else + post(document, variables) + end + + JSON.parse(response.body) + end + + def post(document, variables) + Net::HTTP.post( URI(@url), JSON.generate({ "query" => document, "variables" => variables }), @headers, ) - JSON.parse(response.body) + end + + def post_multipart(form_data) + uri = URI(@url) + req = Net::HTTP::Post.new(uri) + @headers.each_pair do |key, value| + req[key] = value + end + + req.set_form(form_data.to_a, "multipart/form-data") + Net::HTTP.start(uri.hostname, uri.port, use_ssl: uri.scheme == "https") do |http| + http.request(req) + end + end + + # extract multipart upload forms + # spec: https://github.com/jaydenseric/graphql-multipart-request-spec + def extract_multipart_form(document, variables, request) + return unless @upload_types + + path = [] + files_by_path = {} + + # extract all upload scalar values mapped by their input path + variables.each do |key, value| + ast_node = request.variable_definitions[key] + path << key + extract_ast_node(ast_node, value, files_by_path, path, request) if ast_node + path.pop + end + + return if files_by_path.none? + + map = {} + files = files_by_path.values.tap(&:uniq!) + variables_copy = variables.dup + + files_by_path.keys.each do |path| + orig = variables + copy = variables_copy + path.each_with_index do |key, i| + if i == path.length - 1 + map_key = files.index(copy[key]).to_s + map[map_key] ||= [] + map[map_key] << "variables.#{path.join(".")}" + copy[key] = nil + elsif orig[key].object_id == copy[key].object_id + copy[key] = copy[key].dup + end + orig = orig[key] + copy = copy[key] + end + end + + form = { + "operations" => JSON.generate({ + "query" => document, + "variables" => variables_copy, + }), + "map" => JSON.generate(map), + } + + files.each_with_object(form).with_index do |(file, memo), index| + memo[index.to_s] = file.respond_to?(:tempfile) ? file.tempfile : file + end + end + + private + + def extract_ast_node(ast_node, value, files_by_path, path, request) + return unless value + + ast_node = ast_node.of_type while ast_node.is_a?(GraphQL::Language::Nodes::NonNullType) + + if ast_node.is_a?(GraphQL::Language::Nodes::ListType) + if value.is_a?(Array) + value.each_with_index do |val, index| + path << index + extract_ast_node(ast_node.of_type, val, files_by_path, path, request) + path.pop + end + end + elsif @upload_types.include?(ast_node.name) + files_by_path[path.dup] = value + else + type_def = request.supergraph.schema.get_type(ast_node.name) + extract_type_node(type_def, value, files_by_path, path) if type_def&.kind&.input_object? + end + end + + def extract_type_node(parent_type, value, files_by_path, path) + return unless value + + parent_type = Util.unwrap_non_null(parent_type) + + if parent_type.list? + if value.is_a?(Array) + value.each_with_index do |val, index| + path << index + extract_type_node(parent_type.of_type, val, files_by_path, path) + path.pop + end + end + elsif parent_type.kind.input_object? + if value.is_a?(Enumerable) + arguments = parent_type.arguments + value.each do |key, val| + arg_type = arguments[key]&.type + path << key + extract_type_node(arg_type, val, files_by_path, path) if arg_type + path.pop + end + end + elsif @upload_types.include?(parent_type.graphql_name) + files_by_path[path.dup] = value + end end end end diff --git a/lib/graphql/stitching/supergraph.rb b/lib/graphql/stitching/supergraph.rb index ee2b3dbe..b0e40655 100644 --- a/lib/graphql/stitching/supergraph.rb +++ b/lib/graphql/stitching/supergraph.rb @@ -90,7 +90,7 @@ def from_definition(schema, executables:) attr_reader :schema, :boundaries, :locations_by_type_and_field, :executables - def initialize(schema:, fields:, boundaries:, executables:) + def initialize(schema:, fields: {}, boundaries: {}, executables: {}) @schema = schema @boundaries = boundaries @possible_keys_by_type = {} diff --git a/test/graphql/stitching/http_executable_test.rb b/test/graphql/stitching/http_executable_test.rb new file mode 100644 index 00000000..f0d39f00 --- /dev/null +++ b/test/graphql/stitching/http_executable_test.rb @@ -0,0 +1,116 @@ +# frozen_string_literal: true + +require "test_helper" + +describe "GraphQL::Stitching::HttpExecutable" do + + class UploadSchema < GraphQL::Schema + class Upload < GraphQL::Schema::Scalar + graphql_name "Upload" + end + + class FileInput < GraphQL::Schema::InputObject + graphql_name "FileInput" + + argument :file, Upload, required: false + argument :files, [Upload], required: false + argument :deep, [[Upload]], required: false + argument :nested, FileInput, required: false + end + + class Root < GraphQL::Schema::Object + field :upload, Boolean, null: true do + argument :input, FileInput, required: true + end + field :uploads, Boolean, null: true do + argument :inputs, [FileInput], required: true + end + end + + query Root + end + + DummyFile = Struct.new(:tempfile) + + def setup + @supergraph = GraphQL::Stitching::Supergraph.new(schema: UploadSchema) + end + + def test_extract_multipart_form + file1 = DummyFile.new("A") + file2 = DummyFile.new("B") + document = %| + mutation($input: FileInput!, $inputs: [FileInput]!) { + upload(input: $input) + uploads(inputs: $inputs) + } + | + variables = { + "input" => { + "file" => file1, + "files" => [file1, file2], + }, + "inputs" => [{ + "file" => file1, + "files" => [file1, file2], + },{ + "file" => file1, + "files" => [file1, file2], + }] + } + + request = GraphQL::Stitching::Request.new( + @supergraph, + document, + variables: variables + ) + + exe = GraphQL::Stitching::HttpExecutable.new( + url: "", + upload_types: ["Upload"], + ) + + result = exe.extract_multipart_form(document, variables, request).tap do |r| + r["operations"] = JSON.parse(r["operations"]) + r["map"] = JSON.parse(r["map"]) + end + + expected = { + "operations" => { + "query" => document, + "variables" => { + "input" => { + "file" => nil, + "files" => [nil, nil], + }, + "inputs" => [{ + "file" => nil, + "files" => [nil, nil], + }, { + "file" => nil, + "files" => [nil, nil], + }] + } + }, + "map" => { + "0" => [ + "variables.input.file", + "variables.input.files.0", + "variables.inputs.0.file", + "variables.inputs.0.files.0", + "variables.inputs.1.file", + "variables.inputs.1.files.0", + ], + "1" => [ + "variables.input.files.1", + "variables.inputs.0.files.1", + "variables.inputs.1.files.1", + ] + }, + "0" => "A", + "1" => "B", + } + + assert_equal expected, result + end +end diff --git a/test/graphql/stitching/request_test.rb b/test/graphql/stitching/request_test.rb index 4f076e6a..d5788eca 100644 --- a/test/graphql/stitching/request_test.rb +++ b/test/graphql/stitching/request_test.rb @@ -7,9 +7,6 @@ def setup @supergraph = GraphQL::Stitching::Supergraph.new( schema: Schemas::Example::Products, - fields: {}, - boundaries: {}, - executables: {}, ) end diff --git a/test/graphql/stitching/supergraph_test.rb b/test/graphql/stitching/supergraph_test.rb index 635e7e02..dde7a45b 100644 --- a/test/graphql/stitching/supergraph_test.rb +++ b/test/graphql/stitching/supergraph_test.rb @@ -114,7 +114,6 @@ def test_fields_by_type_and_location schema: ComposedSchema, fields: FIELDS_MAP.dup, boundaries: BOUNDARIES_MAP, - executables: {}, ) mapping = supergraph.fields_by_type_and_location @@ -128,7 +127,6 @@ def test_locations_by_type schema: ComposedSchema, fields: FIELDS_MAP.dup, boundaries: BOUNDARIES_MAP, - executables: {}, ) mapping = supergraph.locations_by_type @@ -142,7 +140,6 @@ def test_possible_keys_for_type_and_location schema: ComposedSchema, fields: FIELDS_MAP.dup, boundaries: BOUNDARIES_MAP, - executables: {}, ) assert_equal ["upc"], supergraph.possible_keys_for_type_and_location("Product", "products") @@ -155,7 +152,6 @@ def test_adds_supergraph_location_for_expected_introspection_types schema: ComposedSchema, fields: FIELDS_MAP.dup, boundaries: BOUNDARIES_MAP, - executables: {}, ) ["__Schema", "__Type", "__Field"].each do |introspection_type| diff --git a/test/test_helper.rb b/test/test_helper.rb index 1fc9ec3e..e6173444 100644 --- a/test/test_helper.rb +++ b/test/test_helper.rb @@ -6,7 +6,7 @@ Gem.path.each do |path| # ignore warnings from auto-generated GraphQL lib code. Warning.ignore(/.*mismatched indentations.*/) - Warning.ignore(/.*lib\/graphql\/language\/nodes.rb.*/) + Warning.ignore(/.*lib\/graphql\/language\/nodes.rb:.*/) end require 'bundler/setup'