Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Asset Manager Sourcemaps #17533

Draft
wants to merge 22 commits into
base: main
Choose a base branch
from
Draft

Asset Manager Sourcemaps #17533

wants to merge 22 commits into from

Conversation

Skrypt
Copy link
Contributor

@Skrypt Skrypt commented Feb 26, 2025

Adds sourcemaps output to assets compilation/minification.
Adds a .prod.js that can be used to not share the .map file on prod.
Makes the "Resource Debug Mode" configuration usable with Parcel output assets.

Adds sourcemaps to assets.
Adds a .prod.js that can be used to not share the .map file on prod.
@Skrypt
Copy link
Contributor Author

Skrypt commented Feb 26, 2025

This is no longer a \r\n issue but a swc compiler issue where it outputs a different sourcemap if you build it from Windows or Linux.

@Skrypt
Copy link
Contributor Author

Skrypt commented Feb 26, 2025

Opened ticket here and linking for tracking: swc-project/swc#10108

@Skrypt
Copy link
Contributor Author

Skrypt commented Feb 26, 2025

For info, my last commit is sent from a compilation done in Windows 11.

@@ -78,37 +78,30 @@ glob(config.source).then((files) => {
if (fileInfo.ext === ".js") {
let reader = await fs.readFile(file, "utf8");
Copy link
Contributor Author

@Skrypt Skrypt Feb 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here the issue I think is that fs on windows will read this file with \r\n in it and on linux it will read it with \n
Then when it creates the .map file it generates different mappings hash because that file length is longer on Windows making it fall on more lines making the hashes to differ.

@gvkries
Copy link
Contributor

gvkries commented Feb 27, 2025

Opened ticket here and linking for tracking: swc-project/swc#10108

When reading the comments on that issue, we just need to specify the line endings for the related files (e.g. JS):

*.js text eol=lf

@Skrypt
Copy link
Contributor Author

Skrypt commented Feb 27, 2025

Me and @sebastienros tried that 2 weeks ago and it did not work. We tried a lot of different combinations of that git parameter without success. The differentiation is that this is a text file with \r\n actually written litterally in them. It's not the "newline" char itself. When you read my post here I'm typing \r\n and it doesn't make it fall to a newline because it is litteral.

Also, whether you are on Windows or Linux when you run yarn build it will create these .map files with what it gets from reading the source file. So, we need to parse the \r\n before compiling/minifying the asset so that the .map files are consistently compiling as the same on every OS.

I made it work now.

@Skrypt Skrypt marked this pull request as ready for review February 27, 2025 12:11
@gvkries
Copy link
Contributor

gvkries commented Feb 27, 2025

Yes, I know it is about the escaped \r\n in the map files.
But as far as I understand the * text=auto git attribute, it adjusts the line endings in the local working copy when cloning. You will get LF on Linux and CRLF on Windows in your js files and when creating the maps files it will correctly output the escaped EOLs.
By forcing git to always used e.g. LF on JS files, this problem should be avoided.

But your current solution is also fine for me.

@Skrypt
Copy link
Contributor Author

Skrypt commented Feb 27, 2025

Yes, the issue would not be fixed because we generate these files on the CI and locally when developping. Of course it can alter that file when cloning but that's part of the issue.

I've had to read this to understand how a sourcemap works:
https://www.bugsnag.com/blog/source-maps/

The thing is that on Windows when we read that file with the Asset Manager. It has \r\n chars which makes that file being different. When the sourcemap mappings are getting created, it does a hash that basically has in it position of lines and chars position on that line which makes the hash differ from Windows or Linux when you compile it with SWC.

Is it an issue in SWC? ... not 100%. SWC generates from what it gets. But I'd expect maybe at least an option to make sure that it make it consistent in compiling assets and .map files.

We do not have this issue with Parcel because they probably figured it out already. And I believe Parcel uses SWC.

@gvkries
Copy link
Contributor

gvkries commented Feb 27, 2025

The thing is that on Windows when we read that file with the Asset Manager. It has \r\n chars which makes that file being different.

That's what I think we can avoid. We can just use \n on Windows as well (or \r\n on Linux respectively, I don't care) and therefore avoid the whole problem. This is what the git attribute does and the map files should be the same because the source files will never have \r\n but only \n.

@gvkries
Copy link
Contributor

gvkries commented Feb 27, 2025

Me and @sebastienros tried that 2 weeks ago and it did not work. We tried a lot of different combinations of that git parameter without success. The differentiation is that this is a text file with \r\n actually written litterally in them. It's not the "newline" char itself. When you read my post here I'm typing \r\n and it doesn't make it fall to a newline because it is litteral.

Just note that if you change the gitattributes, the local working copy must be updated, e.g. with

git rm --cached -r . # Remove every file from Git's index.
git reset --hard # Rewrite the Git index to pick up all the new line endings.

Because of that your current solution is much simpler for the OC devs.

@Skrypt
Copy link
Contributor Author

Skrypt commented Feb 27, 2025

I remember that we tried that also locally on my PC. I may retry it to see what happens now.

@@ -1,5 +1,7 @@
# Gulp Pipeline

Will be deprecated. Use new [Assets Manager](../assets-manager/README.md) instead.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here as discussed

.SetDependencies("bootstrap", "admin-main", "theme-manager", "jQuery", "Sortable")
.SetUrl("~/TheAdmin/js/theadmin/TheAdmin.prod.js", "~/TheAdmin/js/theadmin/TheAdmin.js")
.SetVersion("1.0.0");
```
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doc for file.prod.js

Fix yarn clean to remove Scripts and Styles folders since we removed the necessity of the dest param in assets.json file.
@Skrypt Skrypt marked this pull request as draft February 28, 2025 00:54
@Skrypt
Copy link
Contributor Author

Skrypt commented Feb 28, 2025

Arrghhh!!! Did a yarn clean and found that we were not copying everything from the Assets.json to the wwwroot folders in the OrchardCore.Resources module. Now, I need to clean this mess before getting back to this PR.

🫨

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants