eDSL Kernel Macro Prototype #1121

TlatoaniHJ · 2024-12-20T00:20:42Z

INT-2278

jonathanpwang · 2024-12-28T23:04:13Z

crates/toolchain/edsl-deserialize/src/castf_extension.rs

+pub struct CastF;
+
+#[derive(ChipUsageGetter, Chip, InstructionExecutor, From, AnyEnum)]
+pub enum CastFExecutor<F: PrimeField32> {


structure of this crate is weird: CastF should be in openvm-native-circuit.

This crate should be renamed openvm-native-transpiler

jonathanpwang · 2024-12-28T23:05:57Z

crates/toolchain/edsl-deserialize/src/lib.rs

+pub mod config;
+pub mod deserialize_instruction;
+
+const LONG_FORM_INSTRUCTION_INDICATOR: u32 = (1 << 31) + 115115115;


This should be some RISC-V instruction with a 6-bit opcode that's one of the designated custom opcodes

jonathanpwang · 2024-12-28T23:06:41Z

crates/toolchain/edsl-deserialize/src/lib.rs

+pub mod deserialize_instruction;
+
+const LONG_FORM_INSTRUCTION_INDICATOR: u32 = (1 << 31) + 115115115;
+const GAP_INDICATOR: u32 = (1 << 31) + 113113113;


same as above

jonathanpwang · 2024-12-28T23:08:21Z

crates/toolchain/edsl-kernel/Cargo.toml

this should be moved to extensions/native/guest-macro, I think we can call the crate openvm-native-guest-macro since kernel functions are intended to be called from guest programs

jonathanpwang · 2024-12-28T23:11:16Z

crates/toolchain/edsl-kernel/src/generate_edsl.rs

+edition = \"2021\"
+
+[dependencies]
+openvm-native-compiler = { path = \"../../../../extensions/native/compiler\" }


we need a way to have dependency versions consistent with the workspace without hardcoding

harcoding path is also not good

jonathanpwang · 2024-12-28T23:12:09Z

crates/toolchain/edsl-kernel/src/generate_edsl.rs

+type F = BabyBear;
+type EF = BinomialExtensionField<BabyBear, 4>;
+
+<===>


use format! to include the OUTSIDE_PLACEHOLDER by constant name

jonathanpwang · 2024-12-28T23:14:38Z

crates/toolchain/edsl-kernel/src/lib.rs

+const IMMEDIATE_ADDRESS_SPACE: usize = 0;
+const RUST_REGISTER_ADDRESS_SPACE: usize = 1;
+const KERNEL_ADDRESS_SPACE: usize = 5;
+const LONG_FORM_INSTRUCTION_INDICATOR: u32 = (1 << 31) + 115115115;


these should be defined in a common crate that can be imported by both this crate and the transpiler crate

jonathanpwang · 2024-12-28T23:15:03Z

crates/toolchain/edsl-kernel/src/lib.rs

+mod transportation;
+
+const IMMEDIATE_ADDRESS_SPACE: usize = 0;
+const RUST_REGISTER_ADDRESS_SPACE: usize = 1;


we already have this constant in openvm-instructions

jonathanpwang · 2024-12-28T23:15:16Z

crates/toolchain/edsl-kernel/src/lib.rs

+
+const IMMEDIATE_ADDRESS_SPACE: usize = 0;
+const RUST_REGISTER_ADDRESS_SPACE: usize = 1;
+const KERNEL_ADDRESS_SPACE: usize = 5;


this should be moved to openvm-instructions

jonathanpwang · 2024-12-28T23:20:13Z

crates/toolchain/edsl-kernel/src/generate_edsl.rs

General thoughts:

This is actually a bit messy and potentially shows the benefits of having a DSL instead of eDSL (then you can just take in a text file and compile it)

For now I think we should not include the generation of the serialized openvm assembly file in the proc-macro, and just let the macro load it from some pre-existing file

jonathanpwang · 2024-12-28T23:20:35Z

crates/toolchain/edsl-kernel/src/lib.rs

+const REGISTER_LIMB_SIZE: usize = 256;
+
+#[proc_macro]
+pub fn edsl_kernel(input: proc_macro::TokenStream) -> proc_macro::TokenStream {


native_kernel is a better name

jonathanpwang · 2024-12-28T23:23:08Z

crates/toolchain/edsl-kernel/src/transportation.rs

+    result
+}
+
+fn u32_to_directive(x: u32) -> String {


unnecessary, .insn supports a direct 32-bit version

jonathanpwang · 2024-12-28T23:26:58Z

crates/toolchain/edsl-kernel/src/transportation.rs

directioinally correct, but we should use quote! and paste! whenever possible. only the raw asm! block should be generated using string manipulation

jonathanpwang · 2024-12-28T23:28:56Z

crates/toolchain/transpiler/src/extension.rs

@@ -10,5 +10,33 @@ pub trait TranspilerExtension<F> {
    /// the next contiguous section of RISC-V instructions into an [`Instruction`].
    /// It returns `None` if it cannot transpile. Otherwise it returns `(instruction, how_many_u32s)` to indicate that
    /// `instruction_stream[..how_many_u32s]` should be transpiled into `instruction`.


update comment

jonathanpwang · 2024-12-28T23:30:07Z

crates/toolchain/transpiler/src/extension.rs

@@ -10,5 +10,33 @@ pub trait TranspilerExtension<F> {
    /// the next contiguous section of RISC-V instructions into an [`Instruction`].
    /// It returns `None` if it cannot transpile. Otherwise it returns `(instruction, how_many_u32s)` to indicate that
    /// `instruction_stream[..how_many_u32s]` should be transpiled into `instruction`.
-    fn process_custom(&self, instruction_stream: &[u32]) -> Option<(Instruction<F>, usize)>;
+    fn process_custom(&self, instruction_stream: &[u32]) -> Option<TranspilerOutput<F>>;


noting the return type is a breaking change, but is necessary because we want one Option<Instruction<F>> per 32-bit chunk

jonathanpwang · 2024-12-28T23:39:26Z

benchmarks/programs/test_kernel/macro_output.txt

I would prefer if we just had a file with 32-bits per line, and then the macro loaded that file and converted into asm! block

jonathanpwang · 2024-12-28T23:41:08Z

crates/toolchain/transpiler/src/extension.rs

+}
+
+impl<F> TranspilerOutput<F> {
+    pub fn one_to_one(instruction: Instruction<F>) -> Self {


I think we could implement From<Instruction<F>> for TranspilerOutput<F>

jonathanpwang · 2024-12-28T23:41:55Z

extensions/ecc/transpiler/src/lib.rs

-                    ),
-                    1,
-                ));
+                return Some(TranspilerOutput::one_to_one(Instruction::phantom(


see my note above about using into()

jonathanpwang

I think the general mechanisms are in place, but a lot of refactoring / adjustments to be made.

The compilation of eDSL within proc-macro seems like it will have versioning issues, and I'm also concerned it is too slow for rust-analyzer. I think for this first version, we should focus on the case where you just compile the eDSL separately into serialized 32-bit instruction format, and then the kernel proc-macro is focused on inserting those serialized instructions into the ELF.

jonathanpwang · 2025-01-08T08:13:26Z

replaced by #1191

Commit

b6281dd

TlatoaniHJ marked this pull request as draft December 20, 2024 00:20

jonathanpwang reviewed Dec 28, 2024

View reviewed changes

jonathanpwang requested changes Dec 28, 2024

View reviewed changes

TlatoaniHJ mentioned this pull request Jan 7, 2025

feat: native guest macro for kernel functions #1191

Merged

jonathanpwang closed this Jan 8, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

eDSL Kernel Macro Prototype #1121

eDSL Kernel Macro Prototype #1121

TlatoaniHJ commented Dec 20, 2024

jonathanpwang Dec 28, 2024

jonathanpwang Dec 28, 2024

jonathanpwang Dec 28, 2024

jonathanpwang Dec 28, 2024 •

edited

Loading

jonathanpwang Dec 28, 2024

jonathanpwang Dec 28, 2024

jonathanpwang Dec 28, 2024

jonathanpwang Dec 28, 2024

jonathanpwang Dec 28, 2024

jonathanpwang Dec 28, 2024

jonathanpwang Dec 28, 2024

jonathanpwang Dec 28, 2024

jonathanpwang Dec 28, 2024

jonathanpwang Dec 28, 2024

jonathanpwang Dec 28, 2024

jonathanpwang Dec 28, 2024

jonathanpwang Dec 28, 2024

jonathanpwang Dec 28, 2024

jonathanpwang Dec 28, 2024

jonathanpwang left a comment

jonathanpwang commented Jan 8, 2025

eDSL Kernel Macro Prototype #1121

eDSL Kernel Macro Prototype #1121

Conversation

TlatoaniHJ commented Dec 20, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonathanpwang Dec 28, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonathanpwang left a comment

Choose a reason for hiding this comment

jonathanpwang commented Jan 8, 2025

jonathanpwang Dec 28, 2024 •

edited

Loading