Glob and multiple schema pointers as schema: **/*.graphqls (#142)

* It works. All pass * fix: should include configHash - Update snapshots * README.md * removing __generated__ in "yarn clean" doesn't make sense * fix: Care CRLF before creating hash * The order must always be same
piglovesyou · Aug 8, 2020 · 3802ece · 3802ece
1 parent 3f2a3d5
commit 3802ece
Show file tree

Hide file tree

Showing 20 changed files with 306 additions and 416 deletions.
diff --git a/README.md b/README.md
@@ -11,7 +11,7 @@ that integrates graphql-let.
 -   [Why it exists](#why-it-exists)
 -   [How it works](#how-it-works)
 -   [Get started with webpack loader](#get-started-with-webpack-loader)
--   [Configuration options are mosly same as `codegen.yml` except:](#configuration-options-are-mosly-same-as-codegenyml-except)
+-   [Configuration options are mosly same as `codegen.yml` except:](#configuration-options-are-mosly-same-as-codegenyml-except)
 -   [Setup Babel Plugin for inline GraphQL documents](#setup-babel-plugin-for-inline-graphql-documents)
 -   [Jest Transformer](#jest-transformer)
 -   [Experimental feature: Resolver Types](#experimental-feature-resolver-types)
@@ -194,7 +194,7 @@ const News: React.FC = () => {
 }
 ```
 
-## Configuration options are mosly same as `codegen.yml` except:
+## Configuration options are mosly same as `codegen.yml` except:
 
 graphql-let passes most of options to GraphQL code generator, so
 **`.graphql-let.yml` is mostly compatible to `codegen.yml`. However**, some of
@@ -269,17 +269,32 @@ cacheDir: __generated__
 TSConfigFile: tsconfig.json
 TSConfigFile: tsconfig.compile.json
 
-# #gqlDtsEntrypoint", optional.
+# "gqlDtsEntrypoint", optional.
 # `node_modules/@types/graphql-let/index.d.ts` by default. Needs to end with ".d.ts".
 # Used as an entrypoint and directory of generated type declarations for `gql()` calls.
 gqlDtsEntrypoint: node_modules/@types/graphql-let/index.d.ts
 
+# "schemaEntrypoint", optional. You need this if you want to use Resolver Types.
+# Since you could point to multiple schemas, this path is
+# used to generate `.d.ts` to generate `*.graphqls.d.ts`. If you do this,
+#
+#   schema: **/*.graphqls
+#   schemaEntrypoint: schema.graphqls
+#
+# you can import the generated resolver types like below.
+#
+#   import { Resolvers } from '../schema.graphqls'
+#
+# It doesn't matter if the file of the path exists. I recommend
+# you to specify a normal relative path without glob symbols (`**`) like this.
+schemaEntrypoint: schema.graphqls
+schemaEntrypoint: lib/schema.graphqls
 ```
 
 Simple example:
 
 ```yaml
-schema: schema/type-defs.graphqls
+schema: "schema/**/*.graphqls"
 documents:
     - "**/*.graphql"
     - "!shouldBeIgnored1"
@@ -431,27 +446,28 @@ schema. Just use what you need, it's most likely to be `jest-transform-graphql`.
 
 ## Experimental feature: Resolver Types
 
-If:
+If you meed the following conditions, graphql-let generates Resolver Types.
 
--   your `schema` in .graphql-let.yml points to a single local GraphQL schema
-    file (`.graphqls`)
--   you have installed
+-   You have `schemaEntrypoint` in the config
+-   You have file paths including glob patterns in `schema`
+-   You have
     [`@graphql-codegen/typescript-resolvers`](https://graphql-code-generator.com/docs/plugins/typescript-resolvers)
-    in dependencies
+    installed
+-   your `schemaEntrypoint` in .graphql-let.yml points to a single local GraphQL
+    schema file (`.graphqls`)
 
-, graphql-let will generate `.graphqls.d.ts` to help you type your GraphQL
-resolvers. Run:
+Run:
 
 ```bash
 yarn add -D @graphql-codegen/typescript-resolvers
 
 yarn graphql-let
 ```
 
-then you will get `Resolver` type from the schema file.
+Then you will get `${schemaEntrypoint}.d.ts`. Import the types from it.
 
 ```typescript
-import { Resolvers } from "./type-defs.graphqls";
+import { Resolvers } from "../schema.graphqls";
 
 const resolvers: Resolvers = {
   Query: {
@@ -465,8 +481,9 @@ const resolvers: Resolvers = {
 export default resolvers;
 ```
 
-`graphql-let/schema/loader` is also available. It just pass GraphQL Content to
-the next loader but it updates resolver types in HMR. Set it up as below:
+`graphql-let/schema/loader` is also available. It generates/updates
+`${schemaEntrypoint}.d.ts` but it doesn't transpile anything; just passes the
+file content to the next webpack loader. Set it up as below:
 
 ```diff
  const config: Configuration = {
@@ -500,13 +517,14 @@ my attention🍩🍦
 
 These are the states/tools for the syntaxes.
 
-| states/tools for syntax                            | File import as<br>`import './a.graphql';`                            | Inline GraphQL as<br>`import gql from 'graphql-tag';`<br>`` gql(`query {}` ); `` |
-| -------------------------------------------------- | -------------------------------------------------------------------- | -------------------------------------------------------------------------------- |
-| generating `.d.ts`s by command `graphql-let`       | ✅                                                                   | ✅                                                                               |
-| webpack loader `graphql-let/loader`                | ✅                                                                   | [Vote by issuing](https://github.com/piglovesyou/graphql-let/issues)             |
-| Bable Plugin `graphql-let/babel`                   | [Vote by issuing](https://github.com/piglovesyou/graphql-let/issues) | ✅                                                                               |
-| Jest Transformer `graphql-let/jestTransfomer`      | ✅                                                                   | [Vote by issuing](https://github.com/piglovesyou/graphql-let/issues)             |
-| Experimental: Resolver Types for<br>GraphQL schema | ✅ by<br>`import './schema.graphqls'`                                | [Vote by issuing](https://github.com/piglovesyou/graphql-let/issues)             |
+| states/tools for syntax                                                      | File import as<br>`import './a.graphql';`                            | Inline GraphQL as<br>`import gql from 'graphql-tag';`<br>`` gql(`query {}` ); `` |
+| ---------------------------------------------------------------------------- | -------------------------------------------------------------------- | -------------------------------------------------------------------------------- |
+| generating `.d.ts`s by command `graphql-let`                                 | ✅                                                                   | ✅                                                                               |
+| importing GraphQL content from another as<br>`# import A from './a.graphql'` | ✅                                                                   | ✅                                                                               |
+| webpack loader `graphql-let/loader`                                          | ✅                                                                   | [Vote by issuing](https://github.com/piglovesyou/graphql-let/issues)             |
+| Bable Plugin `graphql-let/babel`                                             | [Vote by issuing](https://github.com/piglovesyou/graphql-let/issues) | ✅                                                                               |
+| Jest Transformer `graphql-let/jestTransfomer`                                | ✅                                                                   | [Vote by issuing](https://github.com/piglovesyou/graphql-let/issues)             |
+| Experimental: Resolver Types for<br>GraphQL schema                           | ✅ by<br>`import './schema.graphqls'`                                | [Vote by issuing](https://github.com/piglovesyou/graphql-let/issues)             |
 
 #### Is this a tool only for React?
 
@@ -531,8 +549,9 @@ loaders with fewer pitfalls. Another reason for `.graphqls` is that it's one of
 
 #### How to import `.graphql` from another document, especially GraphQL Fragment?
 
-Thanks to `graphql-tools/import`, the syntax
-`# import X from './fragment.graphql'` is supported.
+Thanks to
+[`graphql-tools/import`](https://www.graphql-tools.com/docs/schema-loading/#using-import-expression),
+the syntax `# import X from './fragment.graphql'` is supported.
 
 Define your fragment named as `partial.graphql`
 

diff --git a/package.json b/package.json
@@ -31,6 +31,7 @@
     "log-update": "^4.0.0",
     "make-dir": "^3.1.0",
     "minimist": "^1.2.5",
+    "p-map": "^4.0.0",
     "rimraf": "^3.0.2",
     "slash": "^3.0.0",
     "yaml": "^1.10.0"
@@ -155,7 +156,7 @@
     "typecheck": "tsc --noEmit",
     "compile": "tsc --project tsconfig.compile.json",
     "compile-develop": "tsc --project tsconfig.develop.json",
-    "clean": "rimraf \"**/__generated__\" dist",
+    "clean": "rimraf dist",
     "test": "jest",
     "build": "yarn clean && yarn compile",
     "prepublishOnly": "yarn lint && yarn typecheck && yarn build && yarn test"

diff --git a/src/init.ts b/src/init.ts
@@ -10,7 +10,6 @@ const DEFAULT_CONFIG: UserConfigTypes = {
   schema: 'lib/type-defs.graphqls',
   documents: '**/*.graphql',
   plugins: ['typescript'],
-  respectGitIgnore: true,
 };
 
 const defaultYamlContent = yamlStringify(DEFAULT_CONFIG);

diff --git a/src/lib/config.ts b/src/lib/config.ts
@@ -9,18 +9,12 @@ import { printError } from './print';
 export type PartialGraphqlCodegenOptions = Omit<Types.Config, 'generates'>;
 
 export type GraphQLLetAdditionalOptions = {
-  // Required. Goes to `codegen.generates[outFile].plugins`.
   plugins: Array<string | Record<string, any>>;
-  // Optional. "true" is the default value.
   respectGitIgnore?: boolean;
-  // Optional. "node_modules/graphql-let/__generated__" is the default value.
+  schemaEntrypoint?: string;
   cacheDir?: string;
-  // Optional. "tsconfig.json" is the default value.
   TSConfigFile?: string;
-  // Optional. "node_modules/@types/graphql-let/index.d.ts" is the default value.
-  // Necessary if you use Babel Plugin "graphql-let/babel".
   gqlDtsEntrypoint?: string;
-  // Optional.
   generateOptions?: Types.ConfiguredOutput;
 };
 
@@ -52,12 +46,13 @@ export function buildConfig(raw: UserConfigTypes): ConfigTypes {
     // Normalized codegen options
     documents,
     // Set graphql-let default values
-    respectGitIgnore: true,
+    respectGitIgnore: raw.respectGitIgnore || true,
     cacheDir: raw.cacheDir || 'node_modules/graphql-let/__generated__',
     TSConfigFile: raw.TSConfigFile || 'tsconfig.json',
     gqlDtsEntrypoint:
       raw.gqlDtsEntrypoint || 'node_modules/@types/graphql-let/index.d.ts',
     generateOptions: raw.generateOptions || Object.create(null),
+    schemaEntrypoint: raw.schemaEntrypoint || '',
   };
 }
 

diff --git a/src/lib/hash.ts b/src/lib/hash.ts
@@ -1,8 +1,20 @@
+import { EOL } from 'os';
 import crypto from 'crypto';
 
+const shouldCareNewline = EOL !== '\n';
+const RegexCRLF = /\r\n/g;
+function normalizeNewline(input: string | Buffer): string {
+  const str = Buffer.isBuffer(input) ? input.toString() : input;
+  if (shouldCareNewline) return str.replace(RegexCRLF, '\n');
+  return str;
+}
+
 export function createHash(s: string | Buffer): string {
-  return crypto
-    .createHash('sha1')
-    .update((Buffer.isBuffer(s) ? s.toString() : s).replace(/\r\n/g, '\n'))
-    .digest('hex');
+  return crypto.createHash('sha1').update(normalizeNewline(s)).digest('hex');
+}
+
+export function createHashFromBuffers(ss: (string | Buffer)[]): string {
+  const hash = crypto.createHash('sha1');
+  for (const s of ss) hash.update(normalizeNewline(s));
+  return hash.digest('hex');
 }
diff --git a/src/lib/literals/literals.ts b/src/lib/literals/literals.ts
@@ -20,10 +20,7 @@ import { readFile } from '../file';
 import { join } from 'path';
 import { createHash } from '../hash';
 import * as t from '@babel/types';
-import {
-  prepareGenResolverTypes,
-  shouldGenResolverTypes,
-} from '../resolver-types';
+import { createSchemaHash, shouldGenResolverTypes } from '../resolver-types';
 import { LiteralCache, PartialCacheStore } from './cache';
 import { CodegenContext, LiteralCodegenContext } from '../types';
 import { appendExportAsObject, createPaths, parserOption } from './fns';
@@ -134,10 +131,7 @@ export async function processLiteralsWithDtsGenerate(
   const execContext = createExecContext(cwd, config, configHash);
   let schemaHash = configHash;
   if (shouldGenResolverTypes(config)) {
-    const { schemaHash: _schemaHash } = await prepareGenResolverTypes(
-      execContext,
-    );
-    schemaHash = _schemaHash;
+    schemaHash = await createSchemaHash(execContext);
   }
 
   const codegenContext: LiteralCodegenContext[] = [];

diff --git a/src/lib/resolver-types.ts b/src/lib/resolver-types.ts
@@ -1,44 +1,66 @@
-import { extname, join as pathJoin } from 'path';
+import { Types } from '@graphql-codegen/plugin-helpers/types';
+import globby from 'globby';
+import slash from 'slash';
 import { ConfigTypes } from './config';
 import { ExecContext } from './exec-context';
 import { readFile, readHash } from './file';
-import { createHash } from './hash';
-import { createPaths } from './paths';
-import { PRINT_PREFIX, updateLog } from './print';
+import { createHashFromBuffers } from './hash';
+import { createPaths, isURL } from './paths';
+import { printError, updateLog } from './print';
 import { processGraphQLCodegen } from './graphql-codegen';
 import { CodegenContext, FileCodegenContext } from './types';
-
-// Currently glob for schema is not allowed.
-function isLocalFilePathWithExtension(s: any): boolean {
-  if (typeof s !== 'string') return false;
-  if (extname(s).length) return true;
-  return false;
-}
+import pMap from 'p-map';
 
 export function shouldGenResolverTypes(config: ConfigTypes): boolean {
   try {
+    if (!config.schemaEntrypoint) return false;
     require('@graphql-codegen/typescript');
     require('@graphql-codegen/typescript-resolvers');
-
-    if (isLocalFilePathWithExtension(config.schema)) return true;
-    console.info(
-      PRINT_PREFIX +
-        'Failed to generate Resolver Types. You have to specify at least one schema (glob) path WITH an extension, such as "**/*.graphqls"',
+    const hasFilePointer = getSchemaPointers(config.schema!).some(
+      (p) => !isURL(p),
     );
-    return false;
+    if (!hasFilePointer) {
+      printError(
+        new Error(
+          `To use Resolver Types, you should have at least one file in "schema".`,
+        ),
+      );
+      return false;
+    }
+    return true;
   } catch (e) {
     // Just skip.
     return false;
   }
 }
 
-export async function prepareGenResolverTypes(execContext: ExecContext) {
-  const { cwd, config, configHash } = execContext;
-  const fileSchema = config.schema as string;
-  const schemaFullPath = pathJoin(cwd, fileSchema);
-  const content = await readFile(schemaFullPath);
-  const schemaHash = createHash(configHash + content);
-  return { schemaFullPath, schemaHash };
+function getSchemaPointers(
+  schema: Types.InstanceOrArray<Types.Schema>,
+  _acc: string[] = [],
+): string[] {
+  if (typeof schema === 'string') {
+    _acc.push(schema);
+  } else if (Array.isArray(schema)) {
+    for (const s of schema) getSchemaPointers(s, _acc);
+  } else if (typeof schema === 'object') {
+    for (const s of Object.keys(schema)) getSchemaPointers(s, _acc);
+  }
+  return _acc;
+}
+
+export async function createSchemaHash(execContext: ExecContext) {
+  const { config, configHash, cwd } = execContext;
+  const schemaPointers = getSchemaPointers(config.schema!);
+  const filePointers = schemaPointers.filter((p) => !isURL(p));
+
+  // XXX: Should stream?
+  const files = await globby(filePointers, { cwd, absolute: true });
+  const contents = await pMap(
+    files.map(slash).sort(),
+    (file) => readFile(file),
+    { concurrency: 10 },
+  );
+  return createHashFromBuffers([configHash, ...contents]);
 }
 
 export async function processResolverTypesIfNeeded(
@@ -51,11 +73,8 @@ export async function processResolverTypesIfNeeded(
   let schemaHash = configHash;
 
   if (shouldGenResolverTypes(config)) {
-    const fileSchema = config.schema as string;
-    const schemaFullPath = pathJoin(cwd, fileSchema);
-    const content = await readFile(schemaFullPath);
-    schemaHash = createHash(schemaHash + content);
-    const createdPaths = createPaths(execContext, fileSchema);
+    schemaHash = await createSchemaHash(execContext);
+    const createdPaths = createPaths(execContext, config.schemaEntrypoint);
 
     const shouldUpdate =
       schemaHash !== (await readHash(createdPaths.tsxFullPath)) ||

diff --git a/src/loader.ts b/src/loader.ts
@@ -1,15 +1,12 @@
 import logUpdate from 'log-update';
 import { loader } from 'webpack';
-import { relative as pathRelative } from 'path';
+import { relative as pathRelative, join } from 'path';
 import { processDocumentsForContext } from './lib/documents';
 import { processDtsForContext } from './lib/dts';
 import createExecContext from './lib/exec-context';
 import loadConfig from './lib/config';
 import memoize from './lib/memoize';
-import {
-  prepareGenResolverTypes,
-  shouldGenResolverTypes,
-} from './lib/resolver-types';
+import { createSchemaHash, shouldGenResolverTypes } from './lib/resolver-types';
 import { PRINT_PREFIX, updateLog } from './lib/print';
 import { readFile } from './lib/file';
 import { CodegenContext } from './lib/types';
@@ -29,11 +26,8 @@ const processGraphQLLetLoader = memoize(
     let schemaHash = configHash;
 
     if (shouldGenResolverTypes(config)) {
-      const {
-        schemaHash: _schemaHash,
-        schemaFullPath,
-      } = await prepareGenResolverTypes(execContext);
-      schemaHash = _schemaHash;
+      schemaHash = await createSchemaHash(execContext);
+      const schemaFullPath = join(cwd, config.schemaEntrypoint);
 
       // If using resolver types, all documents should depend on all schema files.
       addDependency(schemaFullPath);

diff --git a/test/__fixtures/babel/fixtures/basic/.graphql-let.yml b/test/__fixtures/babel/fixtures/basic/.graphql-let.yml
@@ -1,4 +1,5 @@
 schema: "schema/type-defs.graphqls"
+schemaEntrypoint: "schema/type-defs.graphqls"
 documents: "**/*.graphql"
 plugins:
     - typescript

diff --git a/test/__fixtures/babel/fixtures/basic/output.mjs b/test/__fixtures/babel/fixtures/basic/output.mjs
@@ -1,7 +1,7 @@
-import * as V1b047c726c8d0878f30e33cdadada1a79f3a6c62 from "./node_modules/graphql-let/__generated__/input-1b047c726c8d0878f30e33cdadada1a79f3a6c62.tsx";
+import * as V1ffee34aad94b8da4e33faf385ded9238d7affef from "./node_modules/graphql-let/__generated__/input-1ffee34aad94b8da4e33faf385ded9238d7affef.tsx";
 const {
   useViewerQuery
-} = V1b047c726c8d0878f30e33cdadada1a79f3a6c62;
+} = V1ffee34aad94b8da4e33faf385ded9238d7affef;
 export default function Viewer() {
   const {
     data