doc-use-cases.md

Use Cases

The purpose of a build process is to generate "target" files from "source" files by running other programs (originally mostly compilers), whenever the targets files are absent or older than the source files they depend on.

Development typically involves changing at any one time a small number out of a potentially large set of source code files, and then checking the results, by either manual or automatic testing.

Efficient development comes partly from speeding up that loop, so a good build process ensures that only the minimum amount of work is done each time it is invoked - by accurately mapping the relationship between sources and targets.

The build process typically involves a generic make program and a "makefile" that specifies the mappings for the project, which the make program reads when running. Each mapping has a corresponding script, or "recipe", that is executed if necessary, to create the targets from the sources.

These mappings will usually be of the following forms:

One-to-One

A single target file is generated from a single source file. E.g. an isolated HTML file is generated from an EJS template using a JS script.

Typically, the makefile specifies the source and target files directly.

  task("make_special_html_file", "path/to/target.html", "path/to/source.ejs", async () => {
    await exec("convert.js path/to/source.ejs > path/to/target.html");
  });

Many-to-One

A single target file is generated from multiple source files. E.g. a JS bundle is generated from a collection of Typescript files (.ts and .tsx).

Typically, the set of source files is either (a) listed out in full, or (b) expressed as a "glob" or wildcard expression. Ultimake requires an array, which can be obtained from the "glob" tool.

  // src/entry.ts is the bundle "entry-point",
  // but it is assumed the bundle might require any .ts file
  const source_files = glob("**/*.ts");

  task("make_js_bundle", "path/to/bundle.js", source_files, async () => {
    await exec("npx parcel build src/entry.ts --out-file path/to/bundle.js");
  });

Many-to-Many

Sometimes, multiple target files are generated by a single program execution from multiple source files, E.g. compiling JS files from Typrescript files using the "tsc" command, or CSS files from SCSS files using "node-sass".

Often, these really multiple instances of the One-to-One case above, but the commands that perform the generation is more efficiently run on multiple files are the same time, as per the two examples above.

In the original make tool, this situation was dealt with through string transformation functions that operated on glob expressions, such as $(patsubst pattern,replacement,text). The approach here is to use arrays of strings and existing JavaScript string manipulation tools (e.g. RegExp) that are already familiar to the developer.

  // discover all the relevant source files
  const source_files = glob("**/*.ts");

  // work out what the corresponding target files are
  const target_files = source_files
    .map((source_file) => source_file.replace(/.ts$/, ".js"));

  task("compile_typescript", target_files, source_files, async () => {
    await exec("npx tsc");
  });

So here, if any of the target files are missing, or if any one of the source files is newer than any one of the target files (not necessary its own target file) then the whole recipe is run. The recipe program is (hopefully) written to be efficient itself, not performing unnecessary work to generate targets that already exist and are up-to-date.

Multiple One-to-One

If the recipe program is NOT efficient, then the Ultimake script can break the many-to-many mapping down into separate tasks, so that the make program is responsible for identifying the tasks that need to be executed based on target file existence and target vs source file last-modified timestamps. This is done through creating the tasks inside a loop:

  // discover all the relevant source files
  const source_files = glob("**/*.ejs");

  // create a task for each source file
  source_files.forEach((source_file) => {
    const target_file = source_file.replace(/.ejs$/, ".html"));
    task(null, target_file, source_file, async () => {
      await exec(`convert.js ${source_file} > ${target_file}`);
    });
  });

In this case, the tasks are left unnamed - Ultimake creates a name for each task.