Skip to content
This repository has been archived by the owner on Jun 14, 2023. It is now read-only.

Some issues I am getting when running wapm2pirita #279

Open
ayys opened this issue Sep 27, 2022 · 6 comments
Open

Some issues I am getting when running wapm2pirita #279

ayys opened this issue Sep 27, 2022 · 6 comments
Labels
🐞 bug

Comments

@ayys
Copy link
Member

ayys commented Sep 27, 2022 •
edited
Loading

Describe the bug

When running wapm2pirita on packages on wapm.dev, I got the following errors.

wapm2pirita-errors.txt

The file contains package-name@version, followed by the error.

Expected behavior

I shouldn't get these errors.
@ayys ayys added the 🐞 bug label Sep 27, 2022
@fschutt
Copy link
Contributor

fschutt commented Sep 27, 2022

Lets categorize the errors:

syrusakbary/[email protected] Error while running webc conversion (volumes): b"
thread 'main' panicked at 'slice index starts at 8 but ends at 7', 
library/core/src/slice/index.rs:92:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: failed to run `/tmp/wax/felix/[email protected]/wapm_packages/felix/[email protected]/wapm2pirita.wasm`
\xe2\x94\x82   1: RuntimeError: unreachable
           at __rust_start_panic (wapm2pirita.wasm[8037]:0x48b1ce)
           at rust_panic (wapm2pirita.wasm[8019]:0x48a41e)
           at std::panicking::rust_panic_with_hook::h5af4a166307aff48 (wapm2pirita.wasm[8018]:0x48a39b)
           at std::panicking::begin_panic_handler::{{closure}}::h42a9871ead2de5e8 (wapm2pirita.wasm[8005]:0x489601)
           at std::sys_common::backtrace::__rust_end_short_backtrace::h42a6bde96d4a4a1f (wapm2pirita.wasm[8004]:0x489540)
           at rust_begin_unwind (wapm2pirita.wasm[8013]:0x489ceb)
           at core::panicking::panic_fmt::h33d7d4c3033d60da (wapm2pirita.wasm[8203]:0x496e7d)
           at core::slice::index::slice_index_order_fail_rt::ha2fc7ec577df8764 (wapm2pirita.wasm[8195]:0x4967bb)
           at core::ops::function::FnOnce::call_once::h2bc9f1f054f89741 (wapm2pirita.wasm[8194]:0x496734)
           at core::intrinsics::const_eval_select::h729d922677a4fc46 (wapm2pirita.wasm[8245]:0x49e9ca)
           at core::slice::index::slice_index_order_fail::h190cb0e30bcd5afd (wapm2pirita.wasm[8230]:0x49c918)
           at webc::FsEntry::parse::h7c118f1bd03a1efd (wapm2pirita.wasm[2654]:0x1cd386)
           at webc::Volume::specialsort_append_to_target::hd676de5279c8d2ee (wapm2pirita.wasm[2657]:0x1cd828)
           at webc::Volume::specialsort_append_to_target::hd676de5279c8d2ee (wapm2pirita.wasm[2657]:0x1cda75)
           at webc::Volume::specialsort_append_to_target::hd676de5279c8d2ee (wapm2pirita.wasm[2657]:0x1cda75)
           at webc::Volume::specialsort_append_to_target::hd676de5279c8d2ee (wapm2pirita.wasm[2657]:0x1cda75)
           at webc::Volume::specialsort_append_to_target::hd676de5279c8d2ee (wapm2pirita.wasm[2657]:0x1cda75)
           at webc::Volume::specialsort_append_to_target::hd676de5279c8d2ee (wapm2pirita.wasm[2657]:0x1cda75)
           at webc::Volume::specialsort_append_to_target::hd676de5279c8d2ee (wapm2pirita.wasm[2657]:0x1cda75)
           at webc::Volume::specialsort_append_to_target::hd676de5279c8d2ee (wapm2pirita.wasm[2657]:0x1cda75)
           at webc::Volume::specialsort_append_to_target::hd676de5279c8d2ee (wapm2pirita.wasm[2657]:0x1cda75)
           at webc::Volume::get_all_file_entries_directorysorted::h8710c19f61fb1be3 (wapm2pirita.wasm[2655]:0x1cd5db)
           at <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter::h3c58f29165eea2d3 (wapm2pirita.wasm[355]:0x3a098)
           at <alloc::collections::btree::map::BTreeMap<K,V> as core::iter::traits::collect::FromIterator<(K,V)>>::from_iter::hcba8d3c61baeaa04 (wapm2pirita.wasm[374]:0x3ca5d)
           at wapm_to_webc::main::h9090c1a1bac0dd6c (wapm2pirita.wasm[292]:0x2a633)
           at std::sys_common::backtrace::__rust_begin_short_backtrace::h67807de7f0a56d09 (wapm2pirita.wasm[64]:0xdc06)
           at std::rt::lang_start::{{closure}}::h6714bb1bd88194b1 (wapm2pirita.wasm[65]:0xdc2e)
           at std::rt::lang_start_internal::hb6c9ed857072d0ac (wapm2pirita.wasm[7856]:0x47f0b1)
           at __original_main (wapm2pirita.wasm[302]:0x3311a)
           at _start (wapm2pirita.wasm[23]:0x40d7)
           at _start.command_export (wapm2pirita.wasm[8404]:0x4a8843)
           \xe2\x95\xb0\xe2\x94\x80\xe2\x96\xb6 2: unreachable
"

This is an actual error and I'm currently debugging it.

mark2/[email protected] Error while running webc conversion: b'Error: Could not transform module python3.7: file bin/python3.7.wasm not found\n'
mark2/[email protected] Error while running webc conversion: b'Error: Could not transform module python3.7: file bin/python3.7.wasm not found\n'
syrusakbary/[email protected] Error while running webc conversion: b'Error: Could not transform module wasm2wat: file bin/wasm2wat.wasm not found\n'
mackenzie/[email protected] Error while running webc conversion: b'Error: Could not transform module app: file target/wasm32-unknown-wasi/release/wasi-example.wasm not found\n'
mackenzie/[email protected] Error while running webc conversion: b'Error: Could not transform module app: file target/wasm32-unknown-wasi/release/wasi-example.wasm not found\n'
mackenzie/[email protected] Error while running webc conversion: b'Error: Could not transform module app: file target/wasm32-unknown-wasi/release/wasi-example.wasm not found\n'
mark/[email protected] Error while running webc conversion: b'Error: Could not transform module wasi-example: file target/wasm32-unknown-wasi/release/wasi-example.wasm not found\n'
mark/[email protected] Error while running webc conversion: b'Error: Could not transform module wasi-example: file target/wasm32-unknown-wasi/release/wasi-example.wasm not found\n'
mark/[email protected] Error while running webc conversion: b'Error: Could not transform module wasi-example: file target/wasm32-unknown-wasi/release/wasi-example.wasm not found\n'
mark/[email protected] Error while running webc conversion: b'Error: Could not transform module wasi-example: file target/wasm32-unknown-wasi/release/wasi-example.wasm not found\n'
mark/[email protected] Error while running webc conversion: b'Error: Could not transform module wasi-example: file target/wasm32-unknown-wasi/release/wasi-example.wasm not found\n'
mark/[email protected] Error while running webc conversion: b'Error: Could not transform module wasi-example: file target/wasm32-unknown-wasi/release/wasi-example.wasm not found\n'
mark/[email protected] Error while running webc conversion: b'Error: Could not transform module wasi-example: file target/wasm32-unknown-wasi/release/wasi-example.wasm not found\n'
mackenzie/[email protected] Error while running webc conversion: b'Error: Could not transform module d_mod: file target/wasm32-unknown-wasi/release/d.wasm not found\n'
mackenzie/[email protected] Error while running webc conversion: b'Error: Could not transform module d_mod: file target/wasm32-unknown-wasi/release/d.wasm not found\n'
mark/[email protected] Error while running webc conversion: b'Error: Could not transform module wapm-override-test: file target/wasm32-unknown-wasi/release/wapm-override-test.wasm not found\n'
mark/[email protected] Error while running webc conversion: b'Error: Could not transform module wapm-override-test: file target/wasm32-unknown-wasi/release/wapm-override-test.wasm not found\n'
mark/[email protected] Error while running webc conversion: b'Error: Could not transform module wasi-example: file target/wasm32-unknown-wasi/release/wasi-example.wasm not found\n'
mark/[email protected] Error while running webc conversion: b'Error: Could not transform module wasi-example: file target/wasm32-unknown-wasi/release/wasi-example.wasm not found\n'
mark/[email protected] Error while running webc conversion: b'Error: Could not transform module wasi-example: file target/wasm32-unknown-wasi/release/wasi-example.wasm not found\n'
mark/[email protected] Error while running webc conversion: b'Error: Could not transform module wasi-example: file target/wasm32-unknown-wasi/release/wasi-example.wasm not found\n'
mark/[email protected] Error while running webc conversion: b'Error: Could not transform module rsign: file target/wasm32-unknown-wasi/release/rsign.wasm not found\n'
mark2/[email protected] Error while running webc conversion: b'Error: Could not transform module dog: file target/wasm32-unknown-wasi/release/dog.wasm not found\n'
mark2/[email protected] Error while running webc conversion: b'Error: Could not transform module dog: file target/wasm32-unknown-wasi/release/dog.wasm not found\n'
mark2/[email protected] Error while running webc conversion: b'Error: Could not transform module dog: file target/wasm32-unknown-wasi/release/dog.wasm not found\n'
mark2/[email protected] Error while running webc conversion: b'Error: Could not transform module dog: file target/wasm32-unknown-wasi/release/dog.wasm not found\n'
mark2/[email protected] Error while running webc conversion: b'Error: Could not transform module dog: file target/wasm32-unknown-wasi/release/dog.wasm not found\n'
mark2/[email protected] Error while running webc conversion: b'Error: Could not transform module dog: file target/wasm32-unknown-wasi/release/dog.wasm not found\n'
mark2/[email protected] Error while running webc conversion: b'Error: Could not transform module dog: file target/wasm32-unknown-wasi/release/dog.wasm not found\n'
mark2/[email protected] Error while running webc conversion: b'Error: Could not transform module dog: file target/wasm32-unknown-wasi/release/dog.wasm not found\n'
mark2/[email protected] Error while running webc conversion: b'Error: Could not transform module dog: file target/wasm32-wasi/release/dog.wasm not found\n'
evangao/[email protected] Error while running webc conversion: b'Error: Could not transform module python3.7: file bin/python3.7.wasm not found\n'
mark2/[email protected] Error while running webc conversion: b'Error: Could not transform module dog: file target/wasm32-wasi/release/dog.wasm not found\n'
mark2/[email protected] Error while running webc conversion: b'Error: Could not transform module dog: file target/wasm32-wasi/release/dog.wasm not found\n'
mark2/[email protected] Error while running webc conversion: b'Error: Could not transform module dog: file target/wasm32-wasi/release/dog.wasm not found\n'
mark2/[email protected] Error while running webc conversion: b'Error: Could not transform module dog: file target/wasm32-wasi/release/dog.wasm not found\n'
mark2/[email protected] Error while running webc conversion: b'Error: Could not transform module dog: file target/wasm32-wasi/release/dog.wasm not found\n'
syrusakbary/[email protected] Error while running webc conversion: b'Error: Could not transform module wasm-matrix: file build/optimized.wasm not found\n'
syrusakbary/[email protected] Error while running webc conversion: b'Error: Could not transform module wasm-matrix: file build/optimized.wasm not found\n'
mark2/[email protected] Error while running webc conversion: b'Error: Could not transform module dog: file target/wasm32-wasi/release/dog.wasm not found\n'
syrusakbary/[email protected] Error while running webc conversion: b'Error: Could not transform module wasm-matrix: file build/optimized.wasm not found\n'
mark2/[email protected] Error while running webc conversion: b'Error: Could not transform module dog: file target/wasm32-wasi/release/dog.wasm not found\n'
syrusakbary/[email protected] Error while running webc conversion: b'Error: Could not transform module lolcat: file target/wasm32-unknown-wasi/release/lolcat.wasm not found\n'
syrusakbary/[email protected] Error while running webc conversion: b'Error: Could not transform module lolcat: file target/wasm32-unknown-wasi/release/lolcat.wasm not found\n'
namespace-example/[email protected] Error while running webc conversion: b'Error: Could not transform module cowsay: file target/wasm32-unknown-wasi/release/cowsay.wasm not found\n'
namespace-example/[email protected] Error while running webc conversion: b'Error: Could not transform module cowsay: file target/wasm32-unknown-wasi/release/cowsay.wasm not found\n'

These are all packages with broken paths. We need to make sure we validate file paths when doing wapm publish.

syrusakbary/[email protected] Error while running webc conversion: b'Error: Could not parse wapm.toml: unknown variant `Emscripten`, expected one of `emscripten`, `none`, `wasi`, `wasm4` for key `module.abi` at line 13 column 1\n'
syrusakbary/[email protected] Error while running webc conversion: b'Error: Could not parse wapm.toml: unknown variant `Emscripten`, expected one of `emscripten`, `none`, `wasi`, `wasm4` for key `module.abi` at line 40 column 1\n'
syrusakbary/[email protected] Error while running webc conversion: b'Error: Could not parse wapm.toml: unknown variant `Emscripten`, expected one of `emscripten`, `none`, `wasi`, `wasm4` for key `module.abi` at line 41 column 1\n'
syrusakbary/[email protected] Error while running webc conversion: b'Error: Could not parse wapm.toml: unknown variant `Emscripten`, expected one of `emscripten`, `none`, `wasi`, `wasm4` for key `module.abi` at line 13 column 1\n'
syrusakbary/[email protected] Error while running webc conversion: b'Error: Could not parse wapm.toml: unknown variant `Emscripten`, expected one of `emscripten`, `none`, `wasi`, `wasm4` for key `module.abi` at line 13 column 1\n'
syrusakbary/[email protected] Error while running webc conversion: b'Error: Could not parse wapm.toml: unknown variant `Emscripten`, expected one of `emscripten`, `none`, `wasi`, `wasm4` for key `module.abi` at line 13 column 1\n'
php/[email protected] Error while running webc conversion: b'Error: Could not parse wapm.toml: unknown variant `Emscripten`, expected one of `emscripten`, `none`, `wasi`, `wasm4` for key `module.abi` at line 13 column 1\n'

This is not an error, it should be emscripten, not Emscripten.

aysjha/[email protected] Error while running webc conversion: b'Error: temp-17743464512547646874/wapm-to-webc/7621705207724584350: invalid gzip header\n\nCaused by:\n    invalid gzip header\n'
aysjha/[email protected] Error while running webc conversion: b'Error: temp-16224566972135981221/wapm-to-webc/17322268698711490168: invalid gzip header\n\nCaused by:\n    invalid gzip header\n'
aysjha/[email protected] Error while running webc conversion: b'Error: temp-9712864914341127160/wapm-to-webc/6220651120030053499: invalid gzip header\n\nCaused by:\n    invalid gzip header\n'

These were test packages and are invalid .tar.gz files. Can be removed.

Which leaves us with:

mackenzie/[email protected] Error while running webc conversion: b'Error: Dependency ["nginx"] is not in standard owner/name format\n'
Fawadkhan/[email protected] File does not exist: packages/Fawadkh/matrixs/matrixs-0.8.1.tar.gz
Fawadk/[email protected] Error while running webc conversion: b'Error: Command cowsay is defined more than once\n'
felix/[email protected]+8e75ba653 Error while running webc conversion (manifest): b"Error: Invalid WebC file (can't get magic header)\n"
Fawadk/[email protected] Error while running webc conversion: b'Error: Command cowsay is defined more than once\n'

@fschutt
Copy link
Contributor

fschutt commented Sep 27, 2022

Broken packages:

https://registry-cdn.wapm.dev/packages/Fawadk/matrix/matrix-2.2.0.tar.gz

https://registry-cdn.wapm.dev/packages/mackenzie/test_wamp/test_wamp-0.1.0.tar.gz

https://registry-cdn.wapm.dev/packages/Fawadkh/matrixs/matrixs-0.8.1.tar.gz (broken)

https://registry-cdn.wapm.dev/packages/felix/zig/zig-0.10.0-dev.30718e75ba653.tar.gz

https://registry-cdn.wapm.dev/packages/Fawadk/python/python-4.0.0.tar.gz

@fschutt
Copy link
Contributor

fschutt commented Sep 28, 2022

@ayys:

  • mackenzie/[email protected] Error while running webc conversion: b'Error: Dependency ["nginx"] is not in standard owner/name format\n': correct, the dependency is specified as nginx while it should be namespace-example/nginx
  • Fawadkhan/[email protected] File does not exist: packages/Fawadkh/matrixs/matrixs-0.8.1.tar.gz: correct, the link is broken for whatever reason
  • Fawadk/[email protected] Error while running webc conversion: b'Error: Command cowsay is defined more than once\n': correct, the command "cowsay" is specified more than once
  • Fawadk/[email protected] Error while running webc conversion: b'Error: Command cowsay is defined more than once\n': same here

So the only two real errors are:

  • felix/[email protected]+8e75ba653 Error while running webc conversion (manifest): b"Error: Invalid WebC file (can't get magic header)\n": not sure why this is happening, will try to reproduce
  • the other issue:
syrusakbary/[email protected] Error while running webc conversion (volumes): b"
thread 'main' panicked at 'slice index starts at 8 but ends at 7', 
library/core/src/slice/index.rs:92:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: failed to run `/tmp/wax/felix/[email protected]/wapm_packages/felix/[email protected]/wapm2pirita.wasm`
\xe2\x94\x82   1: RuntimeError: unreachable
           at __rust_start_panic (wapm2pirita.wasm[8037]:0x48b1ce)
           at rust_panic (wapm2pirita.wasm[8019]:0x48a41e)
           at std::panicking::rust_panic_with_hook::h5af4a166307aff48 (wapm2pirita.wasm[8018]:0x48a39b)
           at std::panicking::begin_panic_handler::{{closure}}::h42a9871ead2de5e8 (wapm2pirita.wasm[8005]:0x489601)
           at std::sys_common::backtrace::__rust_end_short_backtrace::h42a6bde96d4a4a1f (wapm2pirita.wasm[8004]:0x489540)
           at rust_begin_unwind (wapm2pirita.wasm[8013]:0x489ceb)
           at core::panicking::panic_fmt::h33d7d4c3033d60da (wapm2pirita.wasm[8203]:0x496e7d)
           at core::slice::index::slice_index_order_fail_rt::ha2fc7ec577df8764 (wapm2pirita.wasm[8195]:0x4967bb)
           at core::ops::function::FnOnce::call_once::h2bc9f1f054f89741 (wapm2pirita.wasm[8194]:0x496734)
           at core::intrinsics::const_eval_select::h729d922677a4fc46 (wapm2pirita.wasm[8245]:0x49e9ca)
           at core::slice::index::slice_index_order_fail::h190cb0e30bcd5afd (wapm2pirita.wasm[8230]:0x49c918)
           at webc::FsEntry::parse::h7c118f1bd03a1efd (wapm2pirita.wasm[2654]:0x1cd386)
           at webc::Volume::specialsort_append_to_target::hd676de5279c8d2ee (wapm2pirita.wasm[2657]:0x1cd828)
           at webc::Volume::specialsort_append_to_target::hd676de5279c8d2ee (wapm2pirita.wasm[2657]:0x1cda75)
           at webc::Volume::specialsort_append_to_target::hd676de5279c8d2ee (wapm2pirita.wasm[2657]:0x1cda75)
           at webc::Volume::specialsort_append_to_target::hd676de5279c8d2ee (wapm2pirita.wasm[2657]:0x1cda75)
           at webc::Volume::specialsort_append_to_target::hd676de5279c8d2ee (wapm2pirita.wasm[2657]:0x1cda75)
           at webc::Volume::specialsort_append_to_target::hd676de5279c8d2ee (wapm2pirita.wasm[2657]:0x1cda75)
           at webc::Volume::specialsort_append_to_target::hd676de5279c8d2ee (wapm2pirita.wasm[2657]:0x1cda75)
           at webc::Volume::specialsort_append_to_target::hd676de5279c8d2ee (wapm2pirita.wasm[2657]:0x1cda75)
           at webc::Volume::specialsort_append_to_target::hd676de5279c8d2ee (wapm2pirita.wasm[2657]:0x1cda75)
           at webc::Volume::get_all_file_entries_directorysorted::h8710c19f61fb1be3 (wapm2pirita.wasm[2655]:0x1cd5db)
           at <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter::h3c58f29165eea2d3 (wapm2pirita.wasm[355]:0x3a098)
           at <alloc::collections::btree::map::BTreeMap<K,V> as core::iter::traits::collect::FromIterator<(K,V)>>::from_iter::hcba8d3c61baeaa04 (wapm2pirita.wasm[374]:0x3ca5d)
           at wapm_to_webc::main::h9090c1a1bac0dd6c (wapm2pirita.wasm[292]:0x2a633)
           at std::sys_common::backtrace::__rust_begin_short_backtrace::h67807de7f0a56d09 (wapm2pirita.wasm[64]:0xdc06)
           at std::rt::lang_start::{{closure}}::h6714bb1bd88194b1 (wapm2pirita.wasm[65]:0xdc2e)
           at std::rt::lang_start_internal::hb6c9ed857072d0ac (wapm2pirita.wasm[7856]:0x47f0b1)
           at __original_main (wapm2pirita.wasm[302]:0x3311a)
           at _start (wapm2pirita.wasm[23]:0x40d7)
           at _start.command_export (wapm2pirita.wasm[8404]:0x4a8843)
           \xe2\x95\xb0\xe2\x94\x80\xe2\x96\xb6 2: unreachable
"

@fschutt
Copy link
Contributor

fschutt commented Sep 28, 2022

I can add tests to wapm publish to block users from doing this, but really this should be handled on the backend, too (sanity checks).

@syrusakbary We should think about a wapm2pirita verify {package} to verify that the package can be executed. That way we can immediately block packages that have invalid keys / etc. and will cause issues later on.

@fschutt
Copy link
Contributor

fschutt commented Sep 28, 2022

https://registry-cdn.wapm.dev/packages/felix/zig/zig-0.10.0-dev.30718e75ba653.tar.gz is a broken link. I think this happens when the publish process fails. It would be best to scan the registry continuously for broken package links, so that this doesn't happen.

@fschutt
Copy link
Contributor

fschutt commented Sep 28, 2022

Last failing package: https://registry-cdn.wapm.dev/packages/syrusakbary/ruby/ruby-0.1.1.tar.gz

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
🐞 bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@fschutt @ayys