Mod File Generation

[harrysolovay]

This may be out of scope, but I believe it's enough of a productivity boon that I'd like to propose it just incase.

I frequently utilize a tool called moderate to regenerate mod.ts files. The workflow is very simple:

1. whenever I create a mod.ts that I'd like to "moderate", I add a // moderate comment at the very top.
2. whenever I wish to regenerate the file, I run a command deno task mod, which writes re-export statements for all files and mod.ts-containing subdirectories.

One can even specify excludes in the directive + add leading statements to be left untouched before the // moderate directive. Ie.

export { SomethingElse } from "./something_else.ts"

// moderate --exclude something_else.ts

export * from "./file.ts"
export * from "./dir/mod.ts"

This utility saves me significant time.

I was wondering if anyone thought a workflow such as this could make it into Deno toolchain, where it could be standardized and made to benefit the broader ecosystem. Dealing with mod/barrel files / and reexport statements is just such a pain for so many developers. I think we can do better by making this or a similar workflow a first class citizen.

[BlackAsLight]

So it just takes all your files and re-exports them in a mod.ts file? How does this save significant time? Why would you need to regenerate them and not update it whenever you create or delete a file? Since you've essentially switched it from manually including to manually excluding, I don't see how it saves any time.

[harrysolovay]

It's easy to forget to add/remove/update reexports. Consider projects that have dozens of files and many contributors.

Also: auto-import updates come to mind: they often result in mod files that contain undesirable reexports. For instance, let's say we have a mod.ts that looks as follows.

a/b/mod.ts

export * from "./c.ts

If we move c.ts into the parent directory (aka. it will be at a/c.ts), the mod.ts may auto-update to the following.

export * from "../c.ts"

Moderate ensures that this reexport is no longer in a/b/mod.ts, and ensures that it present in a/mod.ts.

"moderating" is a way to simplify managing exports in large projects. The Deno team already somewhat "moderates" denoland/std. Although it's manual, there is an agreed upon convention for organizing and excluding files from different subpackages/mods.

I'm not suggesting a specific // moderate directive, nor approach to generating mod files, but rather that mod file generation / convention-checking is useful and may merit consideration in the Deno toolchain itself.

[marvinhagemeister]

This sounds like a useful features for certain projects, but I'm not sure if putting that into Deno itself is the ideal place. Personally, I haven't encountered projects where everything of the sibling files should be exported. There is always an exception here or there which makes automating it more tricky. That said it's a perfect feature to put into a tool on jsr.io and share it across your projects that way. That's something you can do today and don't need to wait for Deno to add something.

[harrysolovay]

I would agree if not for the possibility that a proper solution may utilize the Deno module graph (to prevent issues such as used-before-initialization, misc).

[jaydenseric]

Index/mod/barrel modules are a serious anti pattern and should not be encouraged. Projects and packages should take steps to delete them from their codebase and move to optimal deep imports. To learn why, here is a detailed explanation:

https://jaydenseric.com/blog/optimal-javascript-module-design

[harrysolovay]

Hi @jaydenseric, thanks for your thoughts. I too disagree with dependency barrel files. However, I'm suggesting something different.

This issue is about conventions and workflows for exposing exports from subdirectories. We JS devs don't have visibility modifiers beyond export. There's no pub(super) nor pub(crate). Mod files help us declare what exports of a given directory should be visible to siblings and ancestors.

I just read your article and I disagree with your arguments, especially in the context of Deno.

Importing from an index module creates a waterfall loading step, as the index module has to be loaded by the JavaScript runtime

I haven't benched it myself, but my understanding––at least when using Deno––is that this performance overhead is negligible (caching included).

When importing only a subset of the exports available in an index module, the JavaScript runtime still has to waste time and system hardware resources loading the remaining exports.

I think this point is the same as the first, yes?

Dev tools like type checkers and editor IntelliSense have to work harder analyzing the dependencies and code for other exports in the index module you might not even use.

When you create a reexport file such as an index.ts, the reexported symbols are the same. There is no additional overhead for the checker.

Index module files significantly bloat hard drives whether it be Node.js and node_modules, Deno cache, or browser cache.

This overhead is negligible.

If there is a syntax error or any other other problem preventing the JavaScript runtime or build tool from loading one of the modules re-exported in an index module, then the entire index module fails to load

I'd recommend ensuring there are no parse errors before running your code. You can run deno check ..

If a module re-exported in an index module contains malicious code or imports from a compromised dependency, then the hack can affect all users of the package that import from the index module instead of only users of the particular export that’s compromised.

This is an interesting thought. But a bit of a reach, no?

In summary: files that reexport are not fundamentally an anti-pattern. @jaydenseric I think you may be coming to this issue with some faulty assumptions.

I think @dsherret addresses your faulty assumptions quite well in denoland/std#2321

Refactoring out code to a common module is a very common coding practice to encourage code reuse and to make the code cleaner. Internal code maintainability shouldn't be sacrificed to prevent waterfalling since people should use automated tools that distribute the code bundled (again, this does not require a build step because it can be done JIT).

---

The question remains: is there a general-enough mod file convention and/or workflow that could satisfy the needs of large projects like deno_std

[dsherret]

I think this is better left for a separate tool and too specific to have a command in Deno. For example, something could do this on the fly in the editor.

IIRC, I think someone proposed a general purpose code generator sub-command and that might be a better option, though it's something that can be easily achived with deno task (especially once it has stuff like cache busting support), so I'm not sure it's worth it.

[harrysolovay]

That's completely reasonable. Thank you @dsherret.

[dburles]

"since people should use automated tools that distribute the code bundled"

This statement is indicative of the very strong paradigm that you both may be constrained by. Please give deeper consideration and thought to environments where bundling is not (or no longer) required. There are no strong arguments on the side of barrel files. The majority of the counter arguments to @jaydenseric's article are weak or require some other solution to solving a problem created by barrel modules that simply doesn't need to exist.

[harrysolovay]

This statement is indicative of the very strong paradigm that you both may be constrained by.

Apologies I really don't know what you mean by this. By "paradigm", do you mean the decision to occasionally use re-exports to organize code / improve legibility?

On another note, calling counterarguments "weak" without technical explanation is dismissive (as is suggesting a lack of "deeper consideration"). Not sure why this convo devolved... perhaps because of (what appears to be) baggage in denoland/std#2321.

Anyways, the issue's closed. Be well guys!