[@osd/pm] Automate multi-target bootstrap and build

[AMoo-Miki]

Description

Automate multi-target bootstrap and build to build both web and Node artifacts without custom build scripts.

Also:

• build @osd/std for multiple targets
• convert @osd/i18n and @osd/ace from custom build scripts to targeted build process
• have @osd/optimizer ignore already built artifacts

Check List

• All tests pass
  • yarn test:jest
  • yarn test:jest_integration
  • yarn test:ftr
• New functionality includes testing.
• New functionality has been documented.
• Update CHANGELOG.md
• Commits are signed per the DCO using --signoff

[codecov]

Codecov Report

Merging #4650 (1b9a7ef) into main (4bc1f55) will decrease coverage by 0.01%.
The diff coverage is 58.33%.

@@            Coverage Diff             @@
##             main    #4650      +/-   ##
==========================================
- Coverage   66.10%   66.09%   -0.01%     
==========================================
  Files        3315     3316       +1     
  Lines       63913    63927      +14     
  Branches    10125    10132       +7     
==========================================
+ Hits        42248    42252       +4     
+ Misses      19256    19190      -66     
- Partials     2409     2485      +76     

Flag	Coverage Δ
Linux_1	34.75% <100.00%> (+<0.01%)	⬆️
Linux_2	55.07% <58.33%> (+0.03%)	⬆️
Linux_3	42.92% <100.00%> (-0.12%)	⬇️
Linux_4	35.06% <100.00%> (-0.05%)	⬇️
Windows_1	34.76% <100.00%> (+<0.01%)	⬆️
Windows_2	55.04% <58.33%> (+0.03%)	⬆️
Windows_3	42.92% <100.00%> (-0.12%)	⬇️
Windows_4	35.06% <100.00%> (-0.05%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed	Coverage Δ
packages/osd-pm/src/utils/package_json.ts	66.66% <ø> (ø)
packages/osd-pm/src/utils/targeted_build.ts	16.66% <16.66%> (ø)
packages/osd-pm/src/utils/project.ts	66.26% <100.00%> (+4.62%)	⬆️
packages/osd-std/src/index.ts	100.00% <100.00%> (ø)

... and 32 files with indirect coverage changes

[ashwin-pc]

Why does our coverage drop so much?

[AMoo-Miki]

Why does our coverage drop so much?

Codecov failed to upload the reports.

[ashwin-pc]

Can we rerun it?

[AMoo-Miki]

Can we rerun it?

I reran the Linux ones one by one.

[BSFishy]

Don't have enough context on build process to feel confident in an approval but everything seems to look good to me, aside from a few nitpicks

[ashwin-pc]

The change looks good, just have a few questions around the output directory for the tsconfig files

[joshuarrrr]

@AMoo-Miki This seems like a great improvement! But I'm still a little fuzzy on the contours of the problem and solution. It's a case where a separate issue that describes the current problem and its impact would be helpful context for evaluating the solution. I can see how it does things differently but I'd like to have a better sense of the concrete benefit this will provide.

[joshuarrrr]

How will this distinction between type exports be enforced/detected in future packages?

[AMoo-Miki]

Without this, the built artifact for index.ts would have

// packages/osd-std/target/node/index.js
Object.defineProperty(exports, "Freezable", {
  enumerable: true,
  get: function () {
    return _deep_freeze.Freezable;
  }
});

... for the CommonJS builds but _deep_freeze doesn't export Freezable in their JS, or

// packages/osd-std/target/web/index.js
export { deepFreeze, Freezable } from './deep_freeze';

... for ES modules but ./deep_freeze doesn't export Freezable.

By telling the compilers that this is a type export, we get them to not re-export it. Of course, no one uses them in the compiled code but they are misleading and take space.

[joshuarrrr]

Oh yeah, that part I get - what I was asking is whether there's anything preventing a future dev unfamiliar with that distinction from making a similar mistake and exporting/re-exporting the type needlessly. But it sounds like the answer is that it's not really a problem, so we're just fixing the known instances.

[AMoo-Miki]

Since I am slightly obsessed with optimization, I would call it a problem but I am sure I would be in the minority if I wanted to prioritize this. I am biting my tongue for when the typescript upgrade happens; just choosing my battles.

[joshuarrrr]

OK, that's fine. It's not like we're frequently adding or creating new packages to the project, was just trying to avoid the problem coming back.

[AMoo-Miki]

@AMoo-Miki This seems like a great improvement! But I'm still a little fuzzy on the contours of the problem and solution. It's a case where a separate issue that describes the current problem and its impact would be helpful context for evaluating the solution. I can see how it does things differently but I'd like to have a better sense of the concrete benefit this will provide.

Purely to save on my carbon footprint, lemme respond here:
Some packages need to be made available to both node or web. We do that today by adding a manual scripts/build.js to each (like i18n did). This PR streamlines this approach so we don't have the same stuff happening in multiple scripts/build.js files; it also saves on missing to propagate fixes.
An additional benefit of it is that if we needed babel transformers applied during compiling, this could be used as opposed to the a specific build script.

[kavilla]

Can we let this bake in main. Verified locally and with plugins.

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch-Dashboards/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch-Dashboards/backport-2.x
# Create a new branch
git switch --create backport/backport-4650-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 1674c97b7650cfe7549a39b0ce4527f9692c7b67
# Push it to GitHub
git push --set-upstream origin backport/backport-4650-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch-Dashboards/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-4650-to-2.x.

[joshuarrrr]

@AMoo-Miki @manasvinibs Looks like this will need a manual backport.