How to use Asynchronous Plugins in Remark?

[fwx5618177]

1. I am currently refactoring two plugins for Remark. One is a code sandbox parser for ESM modules, and the other involves embedding different types of links, such as videos, Excalidraw, images, etc.
2. I've encountered an issue where our Remark setup does not seem to support asynchronous plugins, which leads to numerous problems.

Here are the links:

• remark-excalidraw
• remark-codesandbox-newest

Here are sample:

index.ts

import { Providers, TextNode, Options } from 'remark-excalidraw';
import { visit } from 'unist-util-visit';

import { RuntimeFactory } from './http/RuntimeFactory';
import { COMMON } from './constants';
import { HelperFactory } from './helpers/HelperFactory';

const remarkExcalidraw = (options?: Options) => {
    return async (tree: TextNode) => {
        RuntimeFactory.getInstance();

        const providers = await RuntimeFactory.getHttpStrategy().requestProvider<Providers>(
            COMMON.OEMBED_PROVIDERS_URL,
        );

        const ctx = {
            ...options,
            providers: [...providers, ...COMMON.DEFAULT_DRAW_PROVIDER],
        };

        visit(tree, 'text', (node: TextNode) => {
            Promise.all([HelperFactory.getInstance().process(ctx, node)]);
        });
    };
};

export default remarkExcalidraw;

[remcohaszing]

Plugins can be asynchronous, but unist-util-visit is not. You can collect the promises and await them in the transformer.

import { Providers, TextNode, Options } from 'remark-excalidraw';
import { visit } from 'unist-util-visit';

import { RuntimeFactory } from './http/RuntimeFactory';
import { COMMON } from './constants';
import { HelperFactory } from './helpers/HelperFactory';

const remarkExcalidraw = (options?: Options) => {
    return async (tree: TextNode) => {
        RuntimeFactory.getInstance();

        const providers = await RuntimeFactory.getHttpStrategy().requestProvider<Providers>(
            COMMON.OEMBED_PROVIDERS_URL,
        );

        const ctx = {
            ...options,
            providers: [...providers, ...COMMON.DEFAULT_DRAW_PROVIDER],
        };

        let promises: PromiseLike[] = []
        visit(tree, 'text', (node: TextNode) => {
            promises.push(HelperFactory.getInstance().process(ctx, node));
        });
        await Promise.all(promises)
    };
};

export default remarkExcalidraw;

[fwx5618177]

If I use these asynchronous plugins in react-markdown, I need to use UseEffect?

[ChristianMurphy]

Async plugins are not currently supported in react-markdown. remarkjs/react-markdown#680

react-remark can, though it's running a version behind currently so typescript may yell (even of it works).

You can also leverage remark directly in react using state and callbacks.

[fwx5618177]

@ChristianMurphy @remcohaszing

Besides that, I have some other confusions:

1. My handling of nodes is non-intrusive, and I will add an extra key where I will append data.
2. Regarding the asynchronous plugin mentioned above, I use mocha and chai for testing, but when using visit, I found that the node's data has not changed.

Here is the plugin's handling module:

import { ContentStrategy } from 'remark-excalidraw';

import { BaseContentStrategy } from './BaseContentStrategy';

export class ExcalidrawStrategy extends BaseContentStrategy implements ContentStrategy {
    override handleContent(data: any): string {
        if (!this.node) {
            this.logInfo('No node found');
            throw new Error('No node found');
        }

        this.node.data = {
            extra: {
                type: this.node.data?.extra?.type || 'html',
                sorts: 'excalidraw',
                value: data.toString(),
            },
        };

        // TODO: it's test case
        // this.node = {
        //     type: 'html',
        //     value: data.toString(),
        // };

        return data;
    }
}

Here is the test file:

    it('should render markdown with asynchronous processing', function () {
        this.timeout(10000);
        const ast = remark().use(remarkExcalidraw).parse(markdownContent);

        visit(ast, 'text', (node: any) => {
            console.log(node);
        });
    });

[ChristianMurphy]

@fwx5618177 there's a lot to unpack in this comment, breaking it down into pieces

1. What you're showing isn't a plugin, I'm not sure what it is, but consider making it a plugin.
2. If you want to modify a node, you need to pass it in by reference. My best guess is this.node is a copy rather than a reference.
3. I'd avoid structuring JavaScript/TypeScript code in an object oriented way, prefer functional code where possible. But if you choose to go object oriented, follow object oriented practices: composition over inheritance.
4. If you are creating an async plugin (based off your initial comment) you should await the result in usage (like in tests).
5. parse does not run transformer plugins https://github.com/unifiedjs/unified?tab=readme-ov-file#overview
6. Avoid any in TypeScript, it largely defeats the purpose of having TypeScript in the first place (for type checking)

[fwx5618177]

Thank you, my friend, for all your assistance.

1. This is a test case designed to evaluate the AST after modifying node values in an asynchronous plugin.
2. When I refer to this.node, I am indicating that particular node. I have employed a strategy design pattern to enhance scalability.
3. I prefer utilizing inheritance. It's not a concern; it’s simply employed within a design pattern of strategies.
4. Regarding how to add await, follow this example:

remarkPlugins: [
                // @ts-ignore
                async options => {
                    await remarkExcalidraw(options);
                },
            ],

5. The use of Any is not critical at this stage; the project is still in development. I am considering what structures to integrate into this data.

[ChristianMurphy]

This is a test case designed to evaluate the AST after modifying node values in an asynchronous plugin

Then use unified.process or unified.run with async await https://github.com/unifiedjs/unified?tab=readme-ov-file#overview

When I refer to this.node, I am indicating that particular node.

I understand, but have you passed a reference to the node or a copy of the node?

It's not a concern; it’s simply employed within a design pattern of strategies.

It is a concern, misuse of inheritance has been known to be an anti-pattern for over 30 years.
There's a whole chapter dedicated to it in the Gang of Four Patterns book, the same book defines the Strategy Pattern https://en.wikipedia.org/wiki/Design_Patterns

Regarding how to add await, follow this example:

Huh? That's not how react-markdown or async plugins work.
Again, react-markdown doesn't support async plugins.
And when using a wrapper that does support async plugins, it doesn't need an extra wrapper around it.

The use of Any is not critical at this stage; the project is still in development. I am considering what structures to integrate into this data

Understanding the data you are passing around is important at all stages of the project, especially early on, when you are still getting used to how the data is structured.

[fwx5618177]

Okie, I would re-use FC, and discard class-based. But it still does not give the changed data. I have tried to use unified.process and run.

[ChristianMurphy]

But it still does not give the changed data.

Could you share a minimal reproducible example of what you've tried?
Ideally in a sandbox like:

• https://stackblitz.com/github/remarkjs/.github/tree/main/sandbox-templates/remark-with-vite
• https://stackblitz.com/github/remarkjs/.github/tree/main/sandbox-templates/remark-rehype-with-vite
• https://stackblitz.com/github/remarkjs/.github/tree/main/sandbox-templates/react-markdown-with-vite

[fwx5618177]

I would publish it. Recently I refactor the project to component. Here is one new version: https://github.com/fwx5618177/remark-codesandbox-sandpack