Testing

[utterances-bot]

This is a general purpose comment thread for the Testing post.

[leggettc18]

I'm not sure if I did something wrong, but by following along with this guide exactly I was receiving an error about fmt being undeclared after setting up the CompareMessage implementation. I fixed it by adding use core::fmt; and use core::fmt::Write; at the top of the file, and increasing the PANIC_LINE by 2 to compensate. Did I miss a step along the way or is there a missing step in the guide?

[phil-opp]

@leggettc18 You did nothing wrong, I forgot to mention these imports. Fixed in 1f97103. Thanks for reporting!

[Beidah]

I'm unable to run cargo xtest. Instead I keep keeping the message

error[E0463]: can't find crate for std
|
= note: the x86_64-blog_os-1954185736335484285 target may not be installed

cargo xbuild works fine, though.

[AntoineSebert]

Try cargo xtest --target x86_64-blog_os.json.

I have the same problem but couldn't figure out why the file isn't automatically found.

[phil-opp]

@AntoineSebert Do you have a build.target key defined in your .cargo/config file?

@Beidah Sounds like forgot a #![no_std] attribute (perhaps the conditional attribute in your lib.rs?). Or you accidentally use a dependency that uses std.

[AntoineSebert]

@phil-opp Yes it's there. The problem seems to have disappread today, and anyway it's not a big deal.

[AntoineSebert]

The recent improvements in the test framework now produce text outputs that are not colored in green, thus making the test result less visible within the terminal.
Is there a way to specify a color for the text when calling print/println macros ?

[phil-opp]

Yes, there is! You can use ANSI escape codes for coloring. For example, you can create a type to print something in green:

use core::fmt;

struct Green(&'static str);

impl fmt::Display for Green {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { 
        write!(f, "\x1B[32m")?; // prefix code
        write!(f, "{}", self.0)?;
        write!(f, "\x1B[0m")?; // postfix code
        Ok(())
    }
}

Now you can do serial_println!("{}", Green("[ok]")); at the end of your test.

I opened #603 for adding this to the post.

[bytesnail]

In the process of adding tests, I encountered some strange errors. In this test, I simply created a writer, but failed, and judging from the error message, it seems that it did not execute any tests at all, because I did not see the information printed by test_runner or panic_handler on the console.

lws@lsw:/document/project/flandre-os/FlandreOS$ cargo xtest test_default_writer
   Compiling flandre_os v0.0.1 (/document/project/flandre-os/FlandreOS)
    Finished dev [unoptimized + debuginfo] target(s) in 0.33s
     Running target/x86_64-flandre_os/debug/deps/flandre_os-d120248f92b70111
Building bootloader
   Compiling bootloader v0.6.0 (/work/tool/rust/cargo/registry/src/mirrors.ustc.edu.cn-61ef6e0cd06fb9b8/bootloader-0.6.0)
    Finished release [optimized + debuginfo] target(s) in 0.72s
Running: `qemu-system-x86_64 -drive format=raw,file=/document/project/flandre-os/FlandreOS/target/x86_64-flandre_os/debug/deps/bootimage-flandre_os-d120248f92b70111.bin -device isa-debug-exit,iobase=0xf4,iosize=0x04 -serial stdio -display none test_default_writer`
qemu-system-x86_64: -display none: drive with bus=0, unit=0 (index=0) exists
error: test failed, to rerun pass '--lib'

[bytesnail]

I did all kinds of printing tests and the printing function seemed to work normally, but when I tried to add some tests to ensure that these functions would not be damaged in future modifications, I got the above errors.

[bjorn3]

I think passing a test name to cargo xtest doesnt work. Try running just cargo xtest.

[phil-opp]

I think passing a test name to cargo xtest doesnt work. Try running just cargo xtest.

Yes, as you see from the "Running: …" output, the test_default_writer is directly passed as an argument to QEMU, which does not understand it and exits with an error. Passing the name of a single test is a feature of the default test runner and not available in our bare-metal environment because our kernel does not get any arguments passed when it's started.

What you can do is to run the tests of the library component through cargo xtest --lib and the tests of the binary through cargo xtest --bin blog_os. You can also run an individual integration test through cargo xtest --test test_name.

[xj42]

Now we can run the test using cargo xtest --test panic_handler. We see that it passes, which means that the reported panic info is correct. If we use a wrong line number in PANIC_LINE or panic with an additional character through panic!("{}x", MESSAGE), we see that the test indeed fails.

I tried adding the "x" as you indicated in the above paragraph. The test did not fail. WIth some more fooling about i see that the fn write_str(&mut self, s: &str) -> fmt::Result {} function is called 4 times. The third time it is called, it sets equals to true. Because there is no else setting false, the test does not work as expected.

I have submitted a pull request #610

[phil-opp]

@xj42 Thanks a lot for reporting! I created #611 to fix this.

[ta5een]

Hi, thank you for taking your time and effort into making this brilliant blog! I've learnt so much about operating systems and the Rust language in general! Can't wait to see how this blog unfolds and what new things you'll bring.

I just wanted to ask if it is the expected behaviour for serial_print! and serial_println! to only work in test mode? With cargo xtest, the serial print macros work as expected; they print to my host's terminal. However, with cargo xrun, nothing is shown in my host's terminal, nor my QEMU instance. Oddly enough, the system hasn't panicked, so it somehow worked (but the bytes don't show up).

It seems to me that perhaps this was the desired behaviour? If so, is there a way for me to send data to the serial port while the QEMU instance is running as normal?

[zhanlidegui]

[package.metadata.bootimage] test-args = ["-device", "isa-debug-exit,iobase=0xf4,iosize=0x04","-serial", "stdio","-display","none"] test-success-exit-code = 33 test-timeout = 120

This setting is only valid for testing

[nyabinary]

Its extremely hard to find a tutor for Rust that does OS Development lol

[bingxinliu]

Hi! First of all appreciate for your absolutely awesome tutorial!

I met a problem at very beginning of this section. After replacing the default test framework by inserting below announcement and test_main function

#![feature(custom_test_frameworks)]
#![test_runner(crate::test_runner)]
#![reexport_test_harness_main = "test_main"]

Then I met the problem of “duplicate lang item” errors. So I remove all panic = "abort" settings. But after that the eh_personality error talked at section A Freestanding Rust Binary appears.
Any idea of how to go around this problem?

Thank you so much!

[bingxinliu]

Sorry, never mind. I forgot a line in target specification json file. After adding the panic strategy line, all things work smoothly!

"panic-strategy": "abort",

[ochudi]

On macOS, running cargo test gave a lot of errors; but it's in line with it not being able to find the crate for test:

error: could not compile `spin` (lib) due to 118 previous errors```

[CalvoM]

@ochudi any update on this?

[bjorn3]

Do you have

#![feature(custom_test_frameworks)]
#![test_runner(crate::test_runner)]
#![reexport_test_harness_main = "test_main"]

in main.rs?

[NightProg]

Hello, when i run the cargo test command i got this error :

qemu-system-x86_64: --format=json: invalid option
error: test failed, to rerun pass `--bin hello_os`

my config.toml file :

[target.'cfg(target_os = "linux")']
rustflags = ["-C", "link-arg=-nostartfiles"]

[target.'cfg(target_os = "windows")']
rustflags = ["-C", "link-args=/ENTRY:_start /SUBSYSTEM:console"]

[target.'cfg(target_os = "macos")']
rustflags = ["-C", "link-args=-e __start -static -nostartfiles"]

[unstable]
build-std-features = ["compiler-builtins-mem"]
build-std = ["core", "compiler_builtins"]

[build]
target = "x86_64-hello_os.json"

[target.'cfg(target_os = "none")']
runner = "bootimage runner"

my json file :

{
  "llvm-target": "x86_64-unknown-none",
  "data-layout": "e-m:e-i64:64-f80:128-n8:16:32:64-S128",
  "arch": "x86_64",
  "target-endian": "little",
  "target-pointer-width": "64",
  "target-c-int-width": "32",
  "os": "none",
  "executables": true,
  "linker-flavor": "ld.lld",
  "linker": "rust-lld",
  "panic-strategy": "abort",
  "disable-redzone": true,
  "features": "-mmx,-sse,+soft-float"
}

and yes i have :

#![feature(custom_test_frameworks)]
#![test_runner(crate::test_runner)]
#![reexport_test_harness_main = "test_main"]

Thanks you.

[NightProg]

It's fixed apparently the bug was clion that add the option --format=json to the command so i renamed test by t and its working

[CodeKitty-max]

I'm getting a segfault on the first step of adding the test_runner function and running the cargo test. Is there anything that could explain this behavior??

[olest]

I am seeing the same error signal: 11, SIGSEGV: invalid memory reference

[CodeKitty-max]

At what point do you get this?

[olest]

Same as you, after running cargo test

[phil-opp]

Could you try cloning this repo, checking out the post-04 branch, and then running cargo test?

[CodeKitty-max]

It now works. Is there something in particular that you suspect causing this

[mr-hespicable]

hi!
this project builds and tests fine on my home computer, but when i run it on github (using the workflow), i get this error:

Error: Failed to execute QEMU test command `"qemu-system-x86_64" "-drive" "format=raw,file=/home/runner/work/sketchOS/sketchOS/target/x86_64-sketchOS/debug/deps/bootimage-sketch_os-118bcc2728fa5037.bin" "-no-reboot" "-device" "isa-debug-exit,iobase=0xf4,iosize=0x04," "-serial" "stdio" "-display" "none"`: An I/O error occured: No such file or directory (os error 2)
error: test failed, to rerun pass `--lib`

My rust.yml looks like this:

name: Rust

on:
  push:
    branches: [ "main" ]
  pull_request:
    branches: [ "main" ]

env:
    CARGO_TERM_COLOR: always

jobs:
    build:
        runs-on: ubuntu-latest
        steps:
          
            - uses: actions/checkout@v4
            - name: Get Nightly Build
              run: rustup override set nightly
            - name: Add rust-src
              run: rustup component add rust-src --toolchain nightly-x86_64-unknown-linux-gnu
            - name: Add llvm-tools 
              run: rustup component add llvm-tools-preview
            - name: Install Bootimage
              run: cargo install bootimage
            - name: Build
              run: cargo build --target x86_64-sketchOS.json --verbose
            - name: Run tests
              run: cargo test --target x86_64-sketchOS.json --verbose

anyone know why this is happening?

[tsatke]

Correct me if I'm wrong, but it seems to me that qemu-system-x86_64 is not installed?

Try to add these steps before cargo test:

      # install QEMU
      - name: Install QEMU
        run: sudo apt update && sudo apt install -y qemu-system-x86
      - name: "Print QEMU Version"
        run: qemu-system-x86_64 --version

[mr-hespicable]

oh.
obviously, haha. thanks a ton! it works fine now... first successful build :D

[tsatke]

Congrats!

[cmanlh]

Hi, when I try to pass arguments "-display none" to QUME, and then run the cargo test, it will below error message.

QEMU 8.1.90 monitor - type 'help' for more information
qemu-system-x86_64: -serial stdio: qemu_add_wait_object: failed
qemu-system-x86_64: -serial stdio: could not connect serial device to character backend 'stdio'

[tsatke]

Can you share the full command line?

[cmanlh]

Hi,

PS D:\git\xos> cargo test
warning: unused variable: `tests`
  --> tests\basic_boot.rs:16:16
   |
16 | fn test_runner(tests: &[&dyn Fn()]) {
   |                ^^^^^ help: if this is intentional, prefix it with an underscore: `_tests`
   |
   = note: `#[warn(unused_variables)]` on by default

warning: function `test_runner` is never used
  --> tests\basic_boot.rs:16:4
   |
16 | fn test_runner(tests: &[&dyn Fn()]) {
   |    ^^^^^^^^^^^
   |
   = note: `#[warn(dead_code)]` on by default

warning: `xos` (test "basic_boot") generated 2 warnings (run `cargo fix --test "basic_boot"` to apply 1 suggestion)
    Finished test [unoptimized + debuginfo] target(s) in 0.14s
     Running unittests src\lib.rs (target\x86_64_xos\debug\deps\xos-17d47f4dbcc45181)
Building bootloader
    Finished release [optimized + debuginfo] target(s) in 0.14s
Running: `qemu-system-x86_64 -drive format=raw,file=D:\git\xos\target\x86_64_xos\debug\deps\bootimage-xos-17d47f4dbcc45181.bin -no-reboot -device isa-debug-exit,iobase=0xf4,iosize=0x04 -serial stdio -display none`
QEMU 8.1.90 monitor - type 'help' for more information
qemu-system-x86_64: -serial stdio: qemu_add_wait_object: failed
qemu-system-x86_64: -serial stdio: could not connect serial device to character backend 'stdio'
error: test failed, to rerun pass `--lib`

Caused by:
  process didn't exit successfully: `bootimage runner D:\git\xos\target\x86_64_xos\debug\deps\xos-17d47f4dbcc45181` (exit code: 1)    
note: test exited abnormally; to see the full output pass --nocapture to the harness.

[phil-opp]

Look like a QEMU-specific error, not sure what's the issue. Have you tried some other QEMU versions?

[wintertempest]

The section where we try to print the names of the test functions is not functioning correctly for me. Whenever I use core::any::type_name::(), it only prints the actual type, which is &dyn core::ops::function::Fn() instead of the function's name.

[bjorn3]

Can you show your Testable implementation? And your test_runner?

[wintertempest]

Hi, thanks for replying, this is my implementation

[wintertempest]

pub trait Testable {
fn run(&self) -> ();
}

impl Testable for T
where
T: Fn(),
{
fn run(&self) {
serial_print!("{}...\t", core::any::type_name::());
self();
serial_println!("[ok]");
}
}

#[cfg(test)]
fn test_runner(tests: &[&dyn Fn()]) {
serial_println!("Demarage de {} tests", tests.len());
for test in tests {
test.run();
}
serial_println!("Fin des tests");
exit_qemu(QemuExitCode::Success);
}

[bjorn3]

You need to change the function signature of test_runner to accept &[&dyn Testable] instead of &[&dyn Fn()].

[wintertempest]

Thank you very much, it was indeed the issue.

[Cooper-Pilot]

#[cfg(test)] never returns true when compiling with cargo t.
This is my cargo.toml:

[package]
name = "xarbaros"
version = "0.1.0"
edition = "2018"

#[profile.dev]
#panic = "abort"
#[profile.release]
#panic = "abort"

[dependencies.lazy_static]
version = "1.0"
features = ["spin_no_std"]

[dependencies]
spin = "0.5.2"
volatile = "0.2.6"
bootloader = "0.9.23"

main.rs:

#![no_std]
#![no_main]

#![feature(custom_test_frameworks)]
#![test_runner(crate::test_runner)]

#![reexport_test_harness_main = "test_main"]

use core::{panic::PanicInfo, fmt::Write};
mod vga_buffer;

#[no_mangle]
pub extern "C" fn _start() -> ! {
    println!("Hello World{}", "!");
    
    #[cfg(test)]
    test_main();
    
    loop {}
}

#[panic_handler]
fn panic(_info: &PanicInfo) -> ! {
    loop {}
}

#[cfg(test)]
pub fn test_runner(tests: &[&dyn Fn()]) {
    println!("Running {} tests", tests.len());
    for test in tests {
        test();
    }
}

[toolateralus]

same problem, the tests never run/ get compiled

[AtomicGamer9523]

test_main doesn't exist

• it doesn't look like you have any tests to run

[Cooper-Pilot]

When I change #[cfg(test)] test_main(); to #[cfg(test)] println!("test");, I expect it to print “test” to the buffer when ran with cargo t, but the results are exactly the same as when ran with cargo r, that is it doesn’t print “test”. It seems to just be completely ignoring tests for me.

[toolateralus]

same here, and i also had some meaningless test cases and none ran.

[phil-opp]

Could you try with the post-04 branch of this repo?

[JiaFengYu]

Is it possible to move the testing module such as test_panic_handler, exit_qemu, Testable trait from lib.rs into its own dedicated file? It would make the code more structured and but I've had a problem with test_main(), where I've tried importing it via use crate::test_main() and also included these attributes in the dedicated testing module file:

#![feature(custom_test_frameworks)]
#![test_runner(blog_os::test_runner)]
#![reexport_test_harness_main = "test_main"]

Anyone have any thoughts?

[Vrajs16]

Is there a solution so that if the the test panics, can we continue?

[tsatke]

You would have to be able to catch a panic, which means you would have to implement unwinding in the kernel. You may want to look into https://github.com/nbdd0121/unwinding for that

[VincentBerthier]

The step after testing is to get coverage data.
Tryning to run llvm-cov on the tests fails however:

error: `profiler_builtins` crate (required by compiler options) is not compatible with crate attribute `#![no_core]`

error[E0463]: can't find crate for `profiler_builtins`
  |
  = note: the compiler may have been built without the profiler runtime```

I couldn’t find anything about how to solve that issue. Is there a solution to this?

[dedestem]

cargo test gives this: (signal: 11, SIGSEGV: invalid memory reference)

[Fincap]

If anyone else is experience the same issue that I was with cargo test failing, and the posted workaround of removing panic = "abort" from Cargo.toml not working, I have a workaround that worked for me.

Insert into your .cargo/config.toml in the [unstable] section:
panic-abort-tests = true

Make sure you're using the latest nightly compiler.

[delguoqing]

both lib.rs and basic_boot.rs contain a entry point function "_start", but cargo test still compile. Why is that?

[delguoqing]

ok, I think I know why. When we're running test cases for basic_boot.rs, cfg(test) is not used for lib.rs?