Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Embed flameshow into the chdig binary #6

Closed
1 task done
azat opened this issue Dec 1, 2023 · 1 comment
Closed
1 task done

Embed flameshow into the chdig binary #6

azat opened this issue Dec 1, 2023 · 1 comment
Assignees
@azat
Labels
type::tech-debt

Comments

@azat
Copy link
Owner

azat commented Dec 1, 2023 •
edited
Loading

Using PyOxidizer this can be mostly done, but there is some trickery since Starlack in PyOxidizer language does not support building libraries.

Prerequisites

But also there are some things that should be checked first:

Stability

  • does embedded python is stable enough
    • no memory leaks
    • no crashes
@azat azat self-assigned this Dec 2, 2023
azat added a commit to azat-archive/chdig that referenced this issue Dec 2, 2023
@azat
Embedded flameshow (fixes azat#6)
24a44e1 
flameshow is a written in python library, that right now is compiled
into a single binary using PyOxidize.

However it will be "cool" to embed it into chdig binary, so that we can
ship one single binary -- chdig.

But PyOxidize does not provide support for compiling libraries, so some
trickery is required:
- we need python distribution with resources that is defined in Starlack
  (pyoxidize resources)
- we need python distribution (pyoxidize generate-python-embedding-artifacts)
- pyo3 configure incorrect config that has libm, this should be replaced
- PyOxidize configure config that uses incorrect packed-resources (not
  the one that has our resources from Starlack, but a simple one with
  python only)
- and the cherry on the top is that you need to have pyo3 config
  *before* running main cargo build -- that's why there is a separate
  `cargo build -p flameshow`

See also comments in flameshow/src/lib.rs and flameshow/build.rs (but in
a nutshell Python in Rust is not that stable, maybe I will have to
revert this patch eventually, but this was fun anyway!)
azat added a commit to azat-archive/chdig that referenced this issue Dec 2, 2023
@azat
Embedded flameshow (fixes azat#6)
7cf1624 
flameshow is a written in python library, that right now is compiled
into a single binary using PyOxidize.

However it will be "cool" to embed it into chdig binary, so that we can
ship one single binary -- chdig.

But PyOxidize does not provide support for compiling libraries, so some
trickery is required:
- we need python distribution with resources that is defined in Starlack
  (pyoxidize resources)
- we need python distribution (pyoxidize generate-python-embedding-artifacts)
- pyo3 configure incorrect config that has libm, this should be replaced
- PyOxidize configure config that uses incorrect packed-resources (not
  the one that has our resources from Starlack, but a simple one with
  python only)
- and the cherry on the top is that you need to have pyo3 config
  *before* running main cargo build -- that's why there is a separate
  `cargo build -p flameshow`

See also comments in flameshow/src/lib.rs and flameshow/build.rs (but in
a nutshell Python in Rust is not that stable, maybe I will have to
revert this patch eventually, but this was fun anyway!)
@azat
Copy link
Owner Author

azat commented Dec 2, 2023

Another downside is that I cannot strip flameshow only easily

@azat azat closed this as completed in ccfa72d Dec 2, 2023
azat added a commit that referenced this issue Dec 2, 2023
@azat
Merge branch 'embed-flameshow' - fixes #6
fcb6fae 
* embed-flameshow:
  Bump ubuntu version in attempt to overcome linking problems
  Embedded flameshow (fixes #6)
@azat azat mentioned this issue Dec 2, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@azat azat

Labels
type::tech-debt
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@azat