incremental compilation behavior on single file change

[joprice]

I'm curious whether the current behavior I'm seeing is correct when rebuilding after a single line change. I find that when I run fable with a command like

dotnet fable cli/Cli.fsproj -e .fs.js -o cli/js --runScript "notifications"

the first build takes about 15s. The second is less than a second and finishes with Skipped compilation because all generated files are up-to-date!,

but after a single line change, I see output that seems to indicate the entire project is being recompiled (Compiled 221/224, etc), including files in the fable_modules folder, which, of course, I didn't modify. The time taken is the same as the full recompile, which makes me think that build caching only works for the exact same input, and not on incremental changes, or my environment or build is somehow misconfigured causing this.

Is this to be expected and if so, is there any way to improve it? It's my main frustration right now working with fable, so I'd be willing to contribute some time if it could help.

[joprice]

Found this log which I assume points to underlying issue:

Cached project info (12/31/1600 7:03:00PM) will be discarded because cli/Cli.fsproj (12/17/2024 8:22:02AM) is newer

The cached date is incorrect, so the cache is being thrown away on every run.

[joprice]

Haha. I've been holding onto that file for quite a while now! It's odd though: I don't see that on every run. A few other things I noticed:

• Sometimes, I am see a dotnet restore on every run, even with the --noRestore flag. (I wonder if somehow fsautocomplete could be causing issues by running it's own restore that invalidates the one run by fable?)
• the timestamp of fable_modules/project_cracked_debug.json gets bumped on every run, so perhaps something is modifying it unintentionally?
• Event when I get Project and references (226 source files) parsed in 99ms, I still see all files compiling, so there's two issues of the project metadata sometimes being invalidate unintentionally, and the other of fable not incrementally compiling a single change. Just so I'm not making too many assumptions, is there logic to handle resumption of incremental compilation from disk, or is it only available when already loaded in memory when using watch mode?

[MangelMaxime]

• Sometimes, I am see a dotnet restore on every run, even with the --noRestore flag. (I wonder if somehow fsautocomplete could be causing issues by running it's own restore that invalidates the one run by fable?)

I think the restore you are seeing is caused by this line, which according to the comment is needed.

Fable/src/Fable.Cli/BuildalyzerCrackerResolver.fs

Lines 112 to 121 in 53fda11

	finally
	File.safeDelete csprojFile
	// Restore the original fsproj because when restoring/analyzing the
	// csproj implicit references added by F# SDKs are not included
	// See https://github.com/fable-compiler/Fable/issues/3719
	// Restoring the F# project should restore the bin/obj folders
	// to their expected state
	let projDir = IO.Path.GetDirectoryName projectFile
	Process.runSync projDir "dotnet" [ "restore"; projectFile ] |> ignore

Note that if you pass --test:MSBuildCracker you should not have this issue. And with Fable 5, MSBuildCracker is now the default.

• the timestamp of fable_modules/project_cracked_debug.json gets bumped on every run, so perhaps something is modifying it unintentionally?

Hum, in theory we only write the cache when we get the project options

https://github.com/fable-compiler/Fable/blob/53fda11730817e08cdf252ac71dbe771dcd8d770/src/Fable.Compiler/ProjectCracker.fs#L815C5-L815C23

• Event when I get Project and references (226 source files) parsed in 99ms, I still see all files compiling, so there's two issues of the project metadata sometimes being invalidate unintentionally, and the other of fable not incrementally compiling a single change. Just so I'm not making too many assumptions, is there logic to handle resumption of incremental compilation from disk, or is it only available when already loaded in memory when using watch mode?

I don't know well this mechanism but I would expect Fable to still TypeCheck the files at minimum because it needs to restore the compiler state. But indeed according to the --verbose log, it seems to do both the TypeChecking and compilation.

[joprice]

I'll try MSBuildCracker. I remember I enabled it a few months back and got some odd binary compat type of issues where dependencies weren't found. I'll let you know if I hit them again.

As for incremental compilation, I assumed it was serialized to disk. I think other ones I'm aware of like scala's Zinc comiler https://github.com/sbt/zinc and rust's model https://rust-lang.github.io/rfcs/1298-incremental-compilation.html#persisting-the-graph persist the data to avoid re-analyzing the files.

[joprice]

After switching to MSBuildCracker, I randomly get this error in watch mode:

error EXCEPTION: Cannot find user entity Fable.React.ReactElement

So perhaps something related to compiler plugins?

[MangelMaxime]

This is strange, I don't have trouble with using MSBuildCracker and had been using since it has been introduced.

Trying to search for Cannot find in Fable source doesn't help to narrow down to the error you are seeing. Could you perhaps try to make a reproduction issue?