gatsby-source-contentful: Next Major Version Public Test

[axe312ger]

The Next Major Version of gatsby-source-contentful: Public Test

We're very happy to unveil the upcoming version of the gatsby-source-contentful plugin. This release tackles several architectural issues and brings forth significant changes.

Key Highlight:

Schema types will be dynamically generated based on the Content Model in Contentful. This ensures that your project remains robust, even if a field becomes devoid of values at any given moment.

In essence, the absence of values in Contentful fields will no longer break your page.

Moreover, this update aligns the plugin's GraphQL schema more closely with Contentful's GraphQL Content API.

For a detailed overview of the changes, please consult this issue.

1. Installation 🚀

npm i gatsby-source-contentful@ctf-next

For those utilizing the SQIP plugin:

npm i gatsby-transformer-sqip@ctf-next

2. Migration and Codemods ⚙️

This release brings numerous new naming conventions, and to assist your transition to this upcoming version, we've provided several code modifications. Furthermore, you'll need to review the alterations these modifications make and also perform additional manual updates to your codebase.

See our new Migration.md: https://github.com/gatsbyjs/gatsby/blob/feat/contentful-schema-generation/packages/gatsby-source-contentful/MIGRATION.md

3. Manual Review 👀

1. Dive into the breaking changes. Though they might seem numerous, the primary focus is on renaming and reorganizing to resonate with the Contentful GraphQL Content API schema.
2. Examine the modifications introduced by the codemods using git diff. While most are spot-on, occasional discrepancies may arise.
3. Should any misalignments occur due to codemods, use git checkout -p to revert them.
4. Address any lingering breaking changes, especially in the domains of Rich Text and JSON fields, since the codemods don't encompass these.
5. Kickstart your revamped site with gatsby develop. However, don't forget to validate that gatsby build also concludes successfully.
6. As a final measure, stage your modifications incrementally via git add -p to certify each change's purpose.

4. Your Feedback is Valuable 🌟

We believe in the power of community collaboration. While we've extensively tested these big changes with various projects, there's nothing quite like real-world testing to ensure robustness and compatibility.

Here's how you can help:

1. Testing: Try integrating this version with your projects. Every unique project setup brings a fresh perspective and uncovers potential nuances.

2. Suggestions: Do you have ideas for improvements or enhancements? Maybe a feature you've always wished for? Now's the time to let us know.

3. Feedback: Positive feedback can accelerate this version's transition from alpha to beta and beyond. If things are working smoothly for you, please share your experience.

4. Bug Reports: Encountered any issues? Report them immediately, so we can ensure a polished final release.

Your input plays a crucial role in shaping the future of gatsby-source-contentful. Let's make it the best version yet, together!

[axe312ger]

Yeah all old config options are still present :)

Not entirely true, we have one rename and 2 new config options:

Actually, typePrefix was renamed to contentTypePrefix, and will not affect the generic type names that will be shared between multiple spaces anyways. (Like ContentfulAsset, ContentfulSys, ...)

The plugin readme + joi options do not reflect this yet, I will include this in the next alpha variant.

Additionally, there are two new config options:

1. enableMarkdownDetection

   • Type: Boolean

   • Default: true

   • Description: This option makes an assumption that every long text field in Contentful is a markdown field. It can be used to auto-detect if the content from a long text field in Contentful should be treated and processed as Markdown. This can be useful because it streamlines the process of marking down long text fields, but with a cautionary note: on large projects, this could potentially be a performance bottleneck. It also has a requirement: you need to have gatsby-transformer-remark in your project.

     The default setting for this is true, which means by default, the plugin will treat all long text fields as Markdown. With this set to true, the plugin will behave like old versions of the plugin.

2. markdownFields

   • Type: Array

   • Default: []

   • Description: This is a more specific configuration option compared to the previous one. It provides a way for developers to specify exactly which fields (by their IDs) in Contentful contain Markdown content. This is done by providing an array of pairs, where each pair is an array. The first element of the pair is the ID of the content type, and the second element is another array containing the field IDs which have Markdown content.

     For example, the given .example for this config option is:

     [
       [`product`, [`description`, `summary`]],
       [`otherContentTypeId`, [`someMarkdownFieldId`]]
     ]

     Here, the product content type has two fields, description and summary, which contain Markdown. Another content type otherContentTypeId has a field someMarkdownFieldId that contains Markdown.

[axe312ger]

Note to myself:

When updating the readme/docs, make sure to remove forceFullSync (and look for other documented but unsupported features and remove them)

[axe312ger]

These are now properly reflected in the updated readme:

https://github.com/gatsbyjs/gatsby/blob/feat/contentful-schema-generation/packages/gatsby-source-contentful/README.md

[ascorbic]

Never mind: found it as sys.id

[axe312ger]

Correct! :)

All old fields still exist, but might be renamed/moved to match the Contentful GraphQL API.

A table of all the meta field renames can be found here:

#31385 (comment)

[ascorbic]

Great, thanks

[ascorbic]

@axe312ger I've been looking at custom type prefixes and they don't seem to be working. If I set "contentTypePrefix": "CustomPrefix" then it creates the types with that prefix, but no nodes. The nodes are created with the default ContentfulContentType prefix. e.g.

query MyQuery {
  allContentfulContentTypePost {
    nodes {
      title
    }
  }
  allCustomPrefixPost {
    nodes {
      title
    }
  }
}

..returns

{
  "data": {
    "allContentfulContentTypePost": {
      "nodes": [
        {
          "title": "Lorem Ipsum is simply dummy text of printing and typesetting"
        },
        {
          "title": "Friends of Sydney: Flora Fricker"
        },
        {
          "title": "This is a title"
        },
        {
          "title": "This is a blog post by Theo"
        }
      ]
    },
    "allCustomPrefixPost": {
      "nodes": []
    }
  },
  "extensions": {}
}

[ascorbic]

My config is:

module.exports = {
  plugins: [
    {
      "resolve": "gatsby-source-contentful",
      "options": {
        "spaceId": "<snip>",
        "accessToken": "<snip>",
        "downloadLocal": false,
        "contentTypePrefix": "CustomPrefix"
      }
    },
    {
      "resolve": "gatsby-plugin-image"
    }
  ]
}

[ascorbic]

And package.json:

{
  "name": "my-project",
  "version": "1.0.0",
  "private": true,
  "dependencies": {
    "gatsby": "latest",
    "gatsby-plugin-image": "latest",
    "gatsby-plugin-sharp": "latest",
    "gatsby-source-contentful": "^9.0.0-alpha.0",
    "react": "latest",
    "react-dom": "latest"
  }
}

[axe312ger]

Ohh thats an issue for sure. Thanks for figuring it out.

I'll fix it and include it in the test suite :)

[axe312ger]

@ascorbic we just released [email protected] which will patch the issue. Please give it a try :)

[axe312ger]

@ascorbic maybe test it again? :)

[me4502]

Hi, I'm super keen for this to officially release. In testing, one issue I've noticed is that Gatsby node fields seem to no longer end up in the schema, practically breaking them.

If I call createNodeField on all nodes of a type, I'm unable to access them the normal Gatsby way via the fields object, and get Cannot query field "fields" on type "ContentfulContentTypeX..

[FrivalszkyP]

@tony-gutierrez In the past, I used createNodeField to add a field to the fields property and then created a resolver using createResolver to expose it at the root (return source.fields.whatever is the key). But I'm guessing schema customization also works similarly.

[axe312ger]

I think if you directly create a fields + resolver, you will have more simple code

[axe312ger]

Should we mention this in the readme? WDYT?

[FrivalszkyP]

There could be a Best practices chapter for sure!

[axe312ger]

We already cover a good amount of best practices in the readme, I tried to improve further with this version:

See here: https://github.com/gatsbyjs/gatsby/blob/feat/contentful-schema-generation/packages/gatsby-source-contentful/README.md

Anything else coming to your mind besides of a "Adding custom fields to Contentful types" section?

[tony-gutierrez]

Are the codemods supposed to traverse your code? Mine only looks at gatsby-browser.js and does nothing.
If I force it to gatsby-node.js, it makes changes there but doesn't touch any other files.

UPDATE: it looks like the command in the doc does not traverse the code, but if I remove gatsby-*.js it will traverse the src folder

[FrivalszkyP]

Interestingly, it does the same thing for me. Using zsh on a mac. If I pass both gatsby-.js and src, it'll only update my gatsby-.js files, and if I just pass src, it'll traverse everything in my src folder.

[axe312ger]

Maybe the codemods version is different, the last time i executed this is months ago but it was 100% working 😱

[FrivalszkyP]

I'll try passing the paths in quotes and I'll get back to you. Thanks!

[axe312ger]

I will double test this now again and adjust the docs accordingly.

[axe312ger]

Alright, looks like using the extensions parameter is much more reliable as trying to include all files as path slugs:

This will work for JS and TS projects:

npx jscodeshift -t ../packages/gatsby-codemods/transforms/gatsby-source-contentful.js --extensions=js,ts,tsx gatsby-* src

[axe312ger]

@tony-gutierrez ohh. Thanks! This is very likely a API bug from Contentful, I have to check with them.

(Or a small chance I mixed sth up but I dont think so, every field i get from the /content-types API endpoint should end up in the schema)

[axe312ger]

Can confirm, that it does not show up.

Also can confirm, that the data is there in the Contentful API response.

So the error is on my side, thanks for catching it. Will fix!

[axe312ger]

Confirmed with Contentful, fix is pushed. New version to test will come soon :)

[axe312ger]

Here we go, its included in the latest update: [email protected]

[axe312ger]

Great, thanks!

I currently create a space that reflects a huge rich text space that also links assets. Will then check and search for issues, for example that map you mention should only contain things that are actually linked.

Plus we have other reports here about big Rich Text using spaces having memory/performance issues.

So, will also look into it (today!)

[FrivalszkyP]

I pinpointed the issue down to a query in create-schema-customization.js. Here's what I did (see the console.log lines):

(nodeType, entityType) => async (source, _args, context) => {
    const links = (0, _richTextLinks.getRichTextEntityLinks)(source, nodeType)[entityType].map(({
      id
    }) => id);
    console.log('makeRichTextLinksResolver links', links);
    const node = context.nodeModel.findRootNodeAncestor(source);
    if (!node) return null;
    const res = await context.nodeModel.findAll({
      query: {
        sys: {
          id: {
            in: links
          },
          spaceId: {
            eq: node.sys.spaceId
          }
        }
      },
      type: `Contentful${entityType}`
    });
    console.log('makeRichTextLinksResolver res', res.totalCount());
    return res.entries;
  }

See the output:

makeRichTextLinksResolver links [
  'j3wZF0TMHeokBPESqCcDX',
  '4q9uVIDxPtaZJoYxiB4YqY',
  '6T9Nhpy1tCAIJov7aEbnSS',
  '7IzGkLjrr6CERvb8JQc6wH',
  '29mtMgApQH37BPfcUZX646',
  '1YL5YfGqr4ema4Oc1K9ENw',
  'kGIJDSAxG0c74Ih2nmRgj',
  '2LUSa8SHVaPY8Mjcqit5u6',
  '2ZEzJzgFPRwsZsEpzq8WNi',
  'agi97iNpOiZSKnMRcyBjY',
  '70u4WYnmPVW6zqqOtK7hJL',
  't43Y4SDMcTcXfmGXMqxmC',
  '6HzPiSD7TKq0YpE37sv7tC',
  '11iGR9aANKNkaTAmZZydjU',
  '1G1wk3Q0PRQUsGBVcwCzm2',
  '5Zd90Df2OWeluL9rXG72ZW',
  '5AlI2FV5kz8ZRyp3KPaLK0',
  'r9ygOzD6stVrgQdn8jZD2'
]
makeRichTextLinksResolver links [ '4gGAJf3uE1sZYxTUsbOvHB' ]
makeRichTextLinksResolver res Promise { 5290 }
makeRichTextLinksResolver res Promise { 947 }

So basically: the first getRichTextEntityLinks returns only a handful of asset IDs, and the second call returns one. Compared to that, the findAll query returns 5290 and 947 items. Looks like, for some reason, the query: { sys: { id: { in: links } } } query doesn't work as expected.

[FrivalszkyP]

I figured it out @axe312ger!! :D the findAll query should look like this:

{
      query: {
        filter: {
          sys: {
            id: {
              in: links
            },
            spaceId: {
              eq: node.sys.spaceId
            }
          }
        }
      },
      type: `Contentful${entityType}`
    }

The query has to be wrapped in a filter: {} object. Damn Gatsby, why can't your API be typesafe. 🗡️

Can I create a PR for this? I couldn't figure out where the source code for the alpha version lives. Is it npm published locally from your working copy?

[axe312ger]

Ahh great!

The branch is here: feat/contentful-schema-generation (The umbrella issue was talking about an non existing branch... adjusted the description there now)

Feel free to provide a PR :)

[axe312ger]

The PR for this is merged :). Thank you again @FrivalszkyP

[FrivalszkyP]

EDIT: I solved this one by myself, see at the end.

One more thing. I noticed that gatsby-source-contentful exports a function renderRichText so I thought I should use that. That function calls a generateLinkMaps function, I thought that could solve my issue with the linked assets being all over the place. Problem is, it produces an error:

Here is how I'm calling the function:

export default function RichText({ text, siteUrl, isWide }) {
  if (!text.json) {
    return '';
  }
  return renderRichText(text, richTextOptions(
    siteUrl,
    {
      isWide,
      references: text.links,
    },
  ));
}

My richTextOptions function returns a function that accepts the output of the generateLinkMaps function, so it looks something like this:

const richTextOptions = (siteUrl, { isWide = true, references } = {}) => {
return ({
    assetBlockMap,
    assetHyperlinkMap,
    entryBlockMap,
    entryInlineMap,
    entryHyperlinkMap,
  }) => ({
renderNode: {
// ...etc...
}
})

EDIT:

Some additional information. Logging out the entry variable in the generateLinkMaps function, I'm getting this:

{id: '7orLdboQQowIUs22KAW4U'}

So no sys object and no contentful_id either.

I figured this one out. I was querying the { sys { id } } within the ... on ContentfulContentType... blocks, but I should have queried if on the fragment itself:

fragment RichTextReferences on ContentfulEntry {
  sys {
    id
  }
  ... on ContentfulContentTypeInlinePromo {
....

[FrivalszkyP]

And one last thing, which probably relates to the previous issue:

npm run dev runs fine but npm run build makes my CPU go brrr and my processor usage goes 110% and the build never finishes. It usually displays something like this for a long-long time and then I hit Ctrl+C:

[                            ]   25.281 s 2/705 0% run queries in workers
[=======================     ]   24.883 s 29/34 85% Running gatsby.IMAGE_CDN jobs

The image_cdn job never reaches 100%, looking like it "finds" new jobs over and over again, as if it got into a recursive endless loop. I think this relates to the linked assets containing all 5000+ images all the time.

[axe312ger]

As this is not the only performance complain, I am currently investigating and creating a data set in Contentful that should match your 5000+ images and assumable a space with a lot of rich text content. Will update as soon I found sth.

[FrivalszkyP]

I have some good news for you, Benedikt. The small fix that I did for the query in my other comment helped a little bit. I'm still getting a 100% or more CPU usage, and I also got an error like this: FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory. But the build did go through. Subsequent builds go through in less than 30 seconds (so the cache helps a lot). So there are definitely performance issues still, but it got better. I'll try and help with this issue as well if I can dedicate more time to it this week. Hope my research helps a little.

[grgcnnr]

Something in one of me contentful spaces is causing createPages to fail with a fairly cryptic message. (commenting it out leads to a successful build so definitely a data problem in the environment) my best guess is that it's being caused by missing linked entities.

[grgcnnr]

thanks for taking the time to look @axe312ger
This is the space i believe is the offender: https://gist.github.com/grgcnnr/ce100762f07838972b3c453634cc3a90

For the sake of completeness: we also run this smaller space on the same site: https://gist.github.com/grgcnnr/f88298cf02febc997530503bd2af58e7

[grgcnnr]

Also possibly related.
ever since we upgraded to Gatsby5 we are intermittently having issues with netlify builds

This seems to be some sort netlify cache issues the only fix seems to be to clear netlify cache and rebuild. Clearing Gatsby's cache ain doing it.

TLDR, i think there's something seriously messed up with our data

[grgcnnr]

Status update, I've unpublished deleted every single bit of content in the space and its still happening. In the process of unpublishing assets
So it must be in the content model

furthur update:
I've tracked it down to this content type: templateCuratedSection

Scratch that. Can't replicate the fix with production data.

[grgcnnr]

Ok this time I have actually tracked it down.
@axe312ger you were on the right track, This was being caused by a deeply nested content type that had validation rules on a linked entity type that didn't exist any more.

I didn't spot it at first because if an the failing rule is on a grandchild of a content type it fails up the tree all the way to the top.

All pretty obvious in retrospect. Thanks for pointing me in the right direction.

[axe312ger]

@grgcnnr good catch!

I'll make sure the schema checks for content types to exist before including them into a union :)

[tony-gutierrez]

After some number of builds I start to get errors like this, and must clear cache:

3:37:57 PM:   Error: Nodes can only be updated by their owner. Node "890aaef1-96bd-5f5c-8a2e  -630204aed593" is
3:37:57 PM:   owned by "undefined" and another plugin "gatsby-source-contentful"
3:37:57 PM:   tried to update it.
3:37:57 PM:   
3:37:57 PM:   - type-owners.ts:115 rawReducer
3:37:57 PM:     [repo]/[gatsby]/src/redux/reducers/type-owners.ts:115:15
3:37:57 PM:   
3:37:57 PM:   - index.ts:44 acc.<computed>
3:37:57 PM:     [repo]/[gatsby]/src/redux/index.ts:44:14
3:37:57 PM:   
3:37:57 PM:   - redux.js:568 combination
3:37:57 PM:     [repo]/[redux]/lib/redux.js:568:29
3:37:57 PM:   
3:37:57 PM:   - redux.js:296 next
3:37:57 PM:     [repo]/[redux]/lib/redux.js:296:22
3:37:57 PM:   
3:37:57 PM:   - index.ts:104 
3:37:57 PM:     [repo]/[gatsby]/src/redux/index.ts:104:68
3:37:57 PM:   
3:37:57 PM:   - index.js:27 
3:37:57 PM:     [repo]/[redux-thunk]/lib/index.js:27:16
3:37:57 PM:   
3:37:57 PM:   - redux.js:699 dispatch
3:37:57 PM:     [repo]/[redux]/lib/redux.js:699:28
3:37:57 PM:   
3:37:57 PM:   - public.js:837 
3:37:57 PM:     [repo]/[gatsby]/src/redux/actions/public.js:837:5
3:37:57 PM:   
3:37:57 PM:   - index.js:23 
3:37:57 PM:     [repo]/[redux-thunk]/lib/index.js:23:18
3:37:57 PM:   
3:37:57 PM:   - redux.js:586 boundActionCreator
3:37:57 PM:     [repo]/[redux]/lib/redux.js:586:12
3:37:57 PM:   
3:37:57 PM:   - api-runner-node.js:108 doubleBoundActionCreators.<computed>
3:37:57 PM:     [repo]/[gatsby]/src/utils/api-runner-node.js:108:20
3:37:57 PM:   
3:37:57 PM:   - source-nodes.js:54 createNode
3:37:57 PM:     [repo]/[gatsby-source-contentful]/dist/source-nodes.js:54:12
3:37:57 PM:   
3:37:57 PM:   - source-nodes.js:419 Object.sourceNodes
3:37:57 PM:     [repo]/[gatsby-source-contentful]/dist/source-nodes.js:419:15
3:37:57 PM:   
3:37:57 PM:   - api-runner-node.js:509 runAPI
3:37:57 PM:     [repo]/[gatsby]/src/utils/api-runner-node.js:509:16

[pieh]

@tomlschmidt did you manage to figure out what steps/setup is needed to reproduce this error? I will do some theorycrafting now, but it would be much easier if I knew how to reproduce it

[pieh]

Ok, I think I see what's happening.

@axe312ger

gatsby/packages/gatsby-source-contentful/src/source-nodes.ts

Line 474 in 9e61200

delete nodeToUpdate.internal.owner

this is causing it - we actually don't have that in current major of the plugin. Git history says it was added when source-nodes module was converted to typescript so possibly this was done just to appease typescript?

What happens is when nodeToUpdate is from getNode few lines above - it is the same reference/object as oldNode in

gatsby/packages/gatsby/src/redux/reducers/type-owners.ts

Lines 114 to 122 in aa60f1c

	if (oldNode && oldNode.internal.owner !== owner) {
	throw new Error(
	stripIndent`
	Nodes can only be updated by their owner. Node "${node.id}" is
	owned by "${oldNode.internal.owner}" and another plugin "${owner}"
	tried to update it.
	`
	)
	}

and because nodeToUpdate is mutated - oldNode.internal.owner will also be undefined after deletion causing the error.

We should not mutating objects we get from getNode so we do need to get rid of those delete calls that are there now and if they were added to appease TS, we need to find different way of making typescript happy there

[pieh]

Above I can reproduce with this:

exports.sourceNodes = async ({ actions, webhookBody, cache, getNode }) => {
  const oldNode = getNode("123");
  if (oldNode) {
    // this is causing the problem
    delete oldNode.internal.owner;
  }

  const counter = ((await cache.get("counter")) ?? 0) + 1;

  const node = {
    id: "123",
    internal: {
      type: "Test",
      contentDigest: `${counter}`,
    },
    counter,
  };

  actions.createNode(node);

  await cache.set("counter", counter);
};

[axe312ger]

Great thanks! Yeah good chance I did this to make TS happy. I'll adjust :)

[axe312ger]

Okay, removed the faulty deletions now: 5e719c2

[tony-gutierrez]

Am I correct that we are no longer dependent on the gatsby-plugin-image?

[axe312ger]

Yeah its fully optional now :)

[axe312ger]

Hey everyone,

@tony-gutierrez @grgcnnr @FrivalszkyP @me4502 @ascorbic (and others...)

Thanks for trying the new version, your feedback really helps me. We are getting closer and closer to a "hopefully bug free" release version :)

I updated the README and the steps to migrate, if some of you could check it, maybe dog-food my new "getting started guide", I would be very happy.

https://github.com/gatsbyjs/gatsby/blob/feat/contentful-schema-generation/packages/gatsby-source-contentful/README.md
https://github.com/gatsbyjs/gatsby/blob/feat/contentful-schema-generation/packages/gatsby-source-contentful/MIGRATION.md

Thanks a bunch!

[axe312ger]

Next Question. Currently renderRichText() accepts either an Options object or an Options factory but the docs only mention the factory method. Is the object literal way of passing options getting phased out?

I was thinking, for very simple setups, just having a Options object could be beneficial. But in the end, its just adding (undocumented) complexity, so I might remove the non-factory variant soon.

[me4502]

Thanks for the various fixes, bumping from alpha 1 to alpha 7 has alleviated all the memory issues I was hitting.

My current two issues/points of confusion are:

• Type-wise, it feels a bit messy adding types to rich text data from queries. In the prior version we had the ContentfulRichTextGatsbyReference & RenderRichTextData types which allowed at least getting the structure of this data, but it seems there's no direct equivalent in the new version. The closest I found is Parameters<typeof renderRichText>[0], but that types links and json as unknown rather than typing them as objects with optional fields. As far as I know the structure of links especially is known. Not a major issue as it doesn't directly affect functionality, but it does make it harder to migrate a typescript site.
• This one might be my fault as I wouldn't say my current migration is "perfect" (still a few bugs to work out that are likely on my end due to the size of the site I'm migrating), but our query times seem a lot slower on this version than prior. I'm getting the 15s+ query warning for every page, whereas I got it for around ~3-4 pages beforehand

[axe312ger]

@me4502 Thanks for sharing!

About the typing of rich text, I hope that we can use the further improved upcoming types from the official Contentful SDK. I'll keep an eye on it, but can't promise any improved rich text related typing for this release.

Performance wise we found an issue with the queries, which is about to be merged. Will give you a hint when the next alpha is out that includes that fix.

[axe312ger]

The performance update is released:

• [email protected]
• [email protected]

[me4502]

Thanks for that. It definitely does seem to improve performance (35m build down to 25m), but it is still a little slower than the released version (~18m). It's not a massive deal as it's a much smaller regression than before this fix though.

[FrivalszkyP]

Hi @axe312ger, any news on this thing going into beta maybe? Let us know if we, open-source contributors can help! A roadmap or a checklist could be helpful, I'd be glad to contribute.

[axe312ger]

@FrivalszkyP We plan to get it out ASAP now, thing is, Gatsby core team is really busy with a lot of things.

We will have another alpha as soon we got #38805 merged it. After another round of tests, I hope to release that version then :)

[axe312ger]

We finally published new alpha versions. As we updated Gatsby core, make sure to install the alpha there as well:

• [email protected]
• [email protected]

This should fix performance issues that did arise here in our testing phase.

[FrivalszkyP]

Hi @axe312ger, I've been testing these versions for weeks now and everything looks all right 👍 The performance looks okay, although I haven't been able to test it on Netlify because it believes the gatsby version is outdated. Anyway, thanks for all the work, I just wanted to give some feedback to let you know that I didn't experience any issues.

[axe312ger]

@FrivalszkyP THANKS for the info!

We are currently preparing a final review and will then push this version. Gatsby core team is under high pressure and super low availability, so we have to wait a little more. But it will come out soonish 🤞

[tony-gutierrez]

I just updated from alpha3 to alpha7, and can no longer build.

It appears now that no 2 content types can use the same name for a field unless those fields return the exact same type?

GraphQLError: Fields "items" conflict because they return conflicting types "[UnionContentfulBrochurePerspectiveProjectServiceSolution]" and "[ContentfulContentTypeSubcomponentIconAndText]!". Use different aliases on the fields to fetch both if this was intentional.

GraphQLError: Fields "verticalMargin" conflict because they return conflicting types "Boolean" and "Boolean!". Use different aliases on the fields to fetch both if this was intentional.

[tony-gutierrez]

@FrivalszkyP what version of Gatsby?

[FrivalszkyP]

5.13.1

[tony-gutierrez]

Same, thx

[FrivalszkyP]

Interesting! The difference might be that in your case, the "required" flag might mean the difference (if my assumption is correct that the "!" flag means "required"). I don't remember having that turned on for any of my fields that share the same name. Try turning that off for some of the impacted fields and see what happens!

[FrivalszkyP]

If all else fails, as a "quick" fix until this gets sorted out, you can add your own resolvers that simply return the field's original value, but cast the type to "Node". That might help.

[me4502]

After some testing, I feel like I've found the source of another performance degradation (and a bug). Based on what I've read on the contentful docs, this appears to be more a shortcoming in the SDK overall, but could be resolved within this plugin. Currently, if you filter a query for an entry down to a specific locale, this isn't reflected in rich text links. It also doesn't seem it's possible to filter these currently either (at the query level, can of course be done in code after getting the data). This results in significantly more data being fetched in queries, and therefore much slower queries. As the rich text SDK & link systems also only expect a single reference entry for each ID, it means the wrong locale can be selected in some situations.

If either the resolver only fetched entries with a locale matching the parent locale, or an option was added to the resolver to allow filtering by locale, this would resolve both the performance issue and the "wrong locale content selection" issue from this plugin's side.

[tomasbillborn]

I am getting this error, any advice on how to solve this?

"gatsby-source-contentful" threw an error while running the
createSchemaCustomization lifecycle:

Unable to create schema for Contentful Content Type My Pages Block:
Contentful field type ResourceLink is not supported

I am using
[email protected]
[email protected]

This Content Type have a cross reference to another space if that helps.

I am handling the cross-reference myself with the supplied "urn" so I am not expecting the real data from the other space to be available to me. I know that is a current limitation

Added information

I looked at my current implementation with gatsby-source-contentful 8.13.0 and

The type ResourceLink is from the cross-reference.

"node": {
            "settings": {
              "__typename": "ContentfulMyPagesBlockSettings",
              "sys": {
                "type": "ResourceLink"
                "linkType": "Contentful:Entry",
                "urn": "crn:contentful:::content:spaces/my_space_id/environments/sit/entries/entry_id"
              }
    }
}

and the current schema generated for it looks like this

type ContentfulMyPagesBlockSettings @derivedTypes {
  sys: ContentfulMyPagesBlockSettingsSys
}

type ContentfulMyPagesBlockSettingsSys {
  type: String
  linkType: String
  urn: String
}

[tomasbillborn]

Suggestion to add to the migration guide:

We have several Contentful apps in our project and they save their data as the type JSON. In the previous version of gatsby-source-contentful you could do this in your graphql

city is defined as JSON

city {
      alias
      name
      latitude
      longitude
 }

in gatsby-source-contentful v9 I get the error

Field "city" must not have a selection since type "JSON" has no subfields.

and it should just be defined only as

city

without any proprties. I might have missed something, but it was not obvious to me when I read the migration guide.

[tony-gutierrez]

Alpha 8 gives me ERROR #gatsby-source-contentful_111005: Used gatsby version is too old and doesn't support required features. Please update to gatsby@>=5.X.0 despite being on the latest Gatsby.

[FrivalszkyP]

Is the environment Netlify or something else? Netlify does the same thing for me but works well locally. I guess Netlify has a built-in check that cannot handle alpha version numbering schemes.

[tony-gutierrez]

Local (osx). Reverted to Alpha 3. I could never use 7 due to a schema bug detailed above.

[tony-gutierrez]

[tomasbillborn]

I get the same error "gatsby@>=5.X.0" as you and had to go with --legacy-peer-deps to get things to install locally

[tomasbillborn]

Is this totally on ice/stopped or is there a chance that there will be fix implemented for cross-space references @axe312ger ?

[FrivalszkyP]

I think your answer lies here: #39062

I appreciate all the work @axe312ger did on this project, but I realized a while ago that it's time for me to jump ship for Astro. That's what Netlify did.

[tomasbillborn]

I appreciate your answer. You can still run v9 with a single contentful space and it would be great if there would be a fix for the space cross-reference types and we could still use all this work that was done by @axe312ger for the current project with Gatsby/Contentful.