diff --git a/content/blog/2024-06-16-unikraft-gsoc-port-mimalloc-to-unikraft.mdx b/content/blog/2024-06-16-unikraft-gsoc-port-mimalloc-to-unikraft.mdx index b5e6f92a..09cf4e59 100644 --- a/content/blog/2024-06-16-unikraft-gsoc-port-mimalloc-to-unikraft.mdx +++ b/content/blog/2024-06-16-unikraft-gsoc-port-mimalloc-to-unikraft.mdx @@ -1,5 +1,5 @@ --- -title: "GSoC'24: Porting the mimalloc memory allocator to Unikraft" +title: "GSoC'24: Porting the mimalloc Memory Allocator to Unikraft" description: | Different applications have different memory usage patterns and perform better when using dedicated memory allocators. Right now, the buddy allocator is the only @@ -13,6 +13,7 @@ tags: - gsoc - gsoc24 - allocators +- synchronization --- ## Project Overview @@ -77,3 +78,9 @@ My future work includes: Special thanks to Răzvan Vîrtan and Radu Nichita, my two amazing mentors, for their support along the way. I would also like to thank Razvan Deaconescu, Hugo Lefeuvre, Ștefan Jumarea, Marco Schlumpp, Sergiu Moga, and the entire Unikraft community for all the work, discussions, and guidance which made this project possible. + +## About Me + +I'm [Yang Hu](https://linkedin.com/yanghuu531), a first year graduate student at the University of Toronto. +I am enthusiastic about operating systems and building infrastructure software in general. +In my free time, I really love traveling and swimming. diff --git a/content/blog/2024-07-10-gsoc-multiboot2.mdx b/content/blog/2024-07-10-gsoc-multiboot2.mdx new file mode 100644 index 00000000..4284064f --- /dev/null +++ b/content/blog/2024-07-10-gsoc-multiboot2.mdx @@ -0,0 +1,128 @@ +--- +title: "Multiboot2 Support in Unikraft" +description: | + In my previous blog post, I focused on the first steps of the project: understanding the Multiboot2 protocol and preparing the development environment. Over the following three weeks, I progressed to implementing the necessary changes/additions and then testing the code. +publishedDate: 2024-07-10 +tags: +- gsoc +- gsoc24 +- multiboot2 +- booting +authors: +- Maria Pana +--- + + + +In my [previous blog post](https://unikraft.org/blog/2024-06-17-gsoc-multiboot2), I outlined a general strategy for software enhancement, focusing on the first steps of the project: understanding the Multiboot2 protocol and preparing the development environment. +Naturally, over the following three weeks, I progressed to the next stages: implementing the necessary changes/additions and then testing the code. + +## Structuring for Seamless Integration + +In order to have Multiboot2 integrated in Unikraft, I mirrored the existing file organization established for Multiboot and extended it to accommodate the new protocol. +Three core files handle the protocol itself, while the rest of the codebase is updated around them to ensure proper integration and functionality. +Let's take a closer look: + +### `multiboot.S` + +This assembly file plays a pivotal role in preparing a standardized operating environment. +It verifies the system is booted through a compliant Multiboot/Multiboot2 bootloader and manages essential memory relocations. +In this context, the only modifications required were replacing `multiboot.h` and `MULTIBOOT_BOOTLOADER_MAGIC` with their Multiboot2 counterparts, featured depending on the chosen configurations. + +### `multiboot2.py` + +The Python script generates and adds the Multiboot2 and updated ELF headers to the original ELF file. +This ensures that the bootloader information is strategically positioned at the start, facilitating correct system initialization. +Since the Multiboot protocol is limited to 32-bit systems, the `multiboot.py` script also required transforming 64-bit ELFs into 32-bit ones. +This is no longer the case when it comes to Multiboot2, only needing to incorporate the Multiboot2 and updated ELF headers into the file, without any other alterations. + +### `multiboot2.c` + +This C program is responsible for processing boot information from a Multiboot2-compliant bootloader. +It meticulously manages memory regions, integrates boot parameters, and prepares the system for kernel execution by consolidating memory configurations and allocating essential resources. + +Alongside these core files, I made adjustments to adjacent files to seamlessly link everything together and ensure successful booting under Multiboot2 (e.g. duplicating the `mkukimg` script's menuentry to use `multiboot2` instead of `multiboot` etc.). + +## Testing the Waters: Progress and Challenges + +My initial focus was to lay the groundwork by creating the `multiboot.S` and `multiboot2.py` files and testing them with the simple `helloworld` application. +This allowed me to verify that the Multiboot2 header was inserted correctly and the system could boot without major hiccups. + +Of course, debugging became my constant companion during this process. +Tools like `hexdump` and attaching `gdb` to my custom GRUB build proved invaluable. + +I encountered an initial hurdle regarding the ELF file. +After playing around with GRUB error messages and making a "comparative study" of the binaries with Multiboot, Multiboot2, or no additional header, I realized the ELF was lacking the Multiboot2 header altogether. +Some detective work later, I traced the issue to an unlinked `multiboot2.py` script. + +## Multiboot2 Header: A Deep Dive + +So far, I really emphasized the need of the Multiboot2 header. +But what is its role? + +The Multiboot2 header is a structured data block embedded within the bootable system image. +Its purpose is to facilitate the transition of control and information from the bootloader to the kernel during the boot process. +This header acts as a standardized communication channel, ensuring the kernel can access essential details about the system's hardware, memory layout, and other boot-related parameters. + +### Format and Contents + +The Multiboot2 header follows a specific format, starting with a fixed-size header followed by a series of variable-length tags. + +The fixed-size header includes: + +- The magic number `0xE85250D6` indicating a Multiboot2-compliant image +- The target architecture (e.g. `i386`, `x86_64`) +- The header length, including all tags +- The checksum of the header, used for error detection + +I talked a little about the tags in my previous blog post, but to recap, they provide detailed information about the system's memory layout, boot command line, and other essential parameters. +Commonly, each tag has a type, size and payload. +There is a multitude of tags to choose from, depending on the specific requirements of the system, adding to the flexibility of Multiboot2. + +In the context of this project, it may be valuable to note that simply adding the Multiboot2 header to the ELF file is not sufficient. +On top of that, both `multiboot.py` and `multiboot2.py` scripts add an extra ELF header with updated offsets, "sandwiching" the Multiboot2 header between it and the original ELF. +To better visualize this, we can refer to the following diagram: + +```text + ┌────────────────────┐ + ┌───────│ Updated EHDR |─────────────┐ + | ├────────────────────┤ | + └──────>| Updated PHDR | | + ├────────────────────┤ | + | MB2HDR | | + ├────────────────────┤ | + ┌───────| Original EHDR |────────┐ | + | ├────────────────────┤ | | + └──────>| Original PHDR | | | + ├────────────────────┤ | | + | Original SHDR |<───────┘ | + ├────────────────────┤ | + | ... | | + ├────────────────────┤ | + | Updated SHDR |<────────────┘ + └────────────────────┘ +``` + +This ensures that GRUB identifies the overall file as an ELF and avoids any potential complications along the way. + +### Interaction between Bootloader and Kernel + +The bootloader locates the Multiboot2 header within the ELF file. +After verifying the magic number and architecture compatibility, it parses the header and tags. +Next, the information requested by the tags is placed into the kernel memory and the bootloader passes control to the kernel. + +The kernel, in turn, accesses the Multiboot2 header using the information provided by the bootloader. +It iterates through the tags to extract the necessary data, such as memory regions, boot command line, and other parameters. +This information is used to initialize its own data structures, set up memory management, and configure the system for execution. + +## Next Steps + +With the Multiboot2 header issue resolved, the ELF file is now generated correctly. +However, the system still encounters a roadblock, getting stuck in what appears to be an infinite loop. +More debugging using `gdb` for me it is! + +After I pinpoint and fix the problem, I will move on to: + +- Refactoring `multiboot2.py` to handle tag inclusion more generically +- Implementing the `multiboot2.c` file +- Testing the new modifications once again diff --git a/content/blog/2024-07-10-gsoc-uefi-gop.mdx b/content/blog/2024-07-10-gsoc-uefi-gop.mdx new file mode 100644 index 00000000..e8c7f5db --- /dev/null +++ b/content/blog/2024-07-10-gsoc-uefi-gop.mdx @@ -0,0 +1,60 @@ +--- +title: "GSoC'24: UEFI Graphics Output Protocol Support in Unikraft, Part II" +description: | + This is the second post in a series of posts where I talk about my progress with the project. +publishedDate: 2024-07-10 +image: /images/unikraft-gsoc24.png +authors: +- Sriprad Potukuchi +tags: +- gsoc +- gsoc24 +- uefi +- booting +--- + +## Project Overview + +The widely available and standardized [UEFI Graphics Output Protocol](https://uefi.org/specs/UEFI/2.10/12_Protocols_Console_Support.html#efi-graphics-output-protocol) (GOP) interface is an excellent alternative to VGA or serial port consoles for printing logs to the screen. + +This project aims to implement a UEFI GOP based console. +For more information, check out [Part I](https://unikraft.org/blog/2024-06-18-gsoc-uefi-gop) of this series. + +## Progress + +- Using a placeholder [font](https://github.com/dhepper/font8x8), it is now possible to print null-terminated strings to the screens. + + +- It is also possible to scale the font in both the axes independently + - No scaling + + - Scaled Y-axis + + +- I've also implemented scrolling. When all the lines are filled and a new log needs to be printed, the current logs on the screen are scrolled up (using `memcpy`) + + + +Right now, scrolling moves all the framebuffer data in place. +Reads and writes in framebuffer memory are slower because the video adapter actually syncs the framebuffer with the screen. +This needs to be optimized! + +## Next Steps + +- Finalize a font! +- Optimize scrolling (by maintaining an additional buffer) +- Look into [this PR](https://github.com/unikraft/unikraft/pull/1464), which adds a generic console interface. + +## Acknowledgement + +I would like to thank all the great Unikraft folk for being a great community! diff --git a/content/blog/2024-07-10-unikraft-gsoc-lxboot_x86.mdx b/content/blog/2024-07-10-unikraft-gsoc-lxboot_x86.mdx new file mode 100644 index 00000000..e5395b66 --- /dev/null +++ b/content/blog/2024-07-10-unikraft-gsoc-lxboot_x86.mdx @@ -0,0 +1,86 @@ +--- +title: "GSoC'24: Linux x86 Boot Protocol Support" +description: | + The Linux boot protocol plays an important role in the initialization of the Linux operating system, emphasizing the importance of system optimization and scalability. +publishedDate: 2024-07-10 +image: /images/unikraft-gsoc24.png +tags: +- gsoc +- gsoc24 +- booting +authors: +- Mihnea Firoiu +--- + +As I wrote in the previous blog post, my first objective is to make Unikraft boot on QEMU using the Linux x86 boot protocol. +Here I will present my progress so far and different challenges I faced while working towards my goal. + +## Where does QEMU handle the lxboot header? + +By using the `git grep` command, I found that `QEMU 9.0.1` handles the Linux x86 boot protocol in the [x86.c](https://github.com/qemu/qemu/blob/stable-9.0/hw/i386/x86.c) file. +Everything happens in the `x86_load_linux` function. + +### Fields used by QEMU + +These are as follows: + +* 0x202: HEADER +* 0x206: VERSION +* 0x211: LOADFLAGS +* 0x236: XLOADFLAGS +* 0x22c: INITRD_ADDR_MAX +* 0x228: CMD_LINE_PTR +* 0x1fa: VID_MODE +* 0x210: TYPE_OF_LOADER +* 0x224: HEAP_END_PTR +* 0x1f1: SETUP_SECTS +* 0x250: SETUP_DATA +* 0x218: RAMDISK_IMAGE +* 0x21c: RAMDISK_SIZE + +As you can see, only 13 out of 39 fields are used. +Out of these, 6 are read by QEMU: + +* 0x202: HEADER +* 0x206: VERSION +* 0x211: LOADFLAGS +* 0x236: XLOADFLAGS +* 0x22c: INITRD_ADDR_MAX +* 0x1f1: SETUP_SECTS + +And 7 are written by QEMU: + +* 0x228: CMD_LINE_PTR +* 0x1fa: VID_MODE +* 0x210: TYPE_OF_LOADER +* 0x224: HEAP_END_PTR +* 0x250: SETUP_DATA +* 0x218: RAMDISK_IMAGE +* 0x21c: RAMDISK_SIZE + +## Debugging and testing + +The application I am using to test the `mklinux_x86.py` file is [app-helloworld](https://github.com/unikraft/app-helloworld/tree/stable). +By using multiple `printf` and `exit` calls inside the `x86_load_linux` function, I was able to determine the way QEMU uses the header. +Another useful tool was `hexdump`,that was used to look at how my script builds the header. + +## Challenges + +For starters, I could not figure out why the header created by the script did not align to what QEMU expected. +The problem was that the header, as presented in the documentation, starts at offset `0x1f1`, not `0x0`. +To fix it I had to add 0x1f1 zeros. + +Another issue was with integrating it in Unikraft. +Eventually I figured it out, with help from my mentors. + +## Integration with Unikraft + +Although debugging could be done only by running the `mklinux_x86.py` script and looking at the header, I wanted to test by integrating everything in Unikraft. +For doing this I had to modify the following files: [Linker.uk](https://github.com/unikraft/unikraft/blob/staging/plat/kvm/Linker.uk), [Config.uk](https://github.com/unikraft/unikraft/blob/staging/plat/kvm/Config.uk) and [Makefile.rules](https://github.com/unikraft/unikraft/blob/staging/plat/common/Makefile.rules). + +## Next steps + +At the moment, when I try to run, it loops indefinitely. +I have to debug and find out what does not work. + +Additionally I am going to look into SeaBIOS/qboot/GRUB2 to see how the jumping to the kernel happens and I will write the needed 16-bit and 32-bit assembly stubs. diff --git a/content/blog/2024-07-12-unikraft-gsoc-test-mimalloc-on-unikraft.mdx b/content/blog/2024-07-12-unikraft-gsoc-test-mimalloc-on-unikraft.mdx new file mode 100644 index 00000000..1ebc845a --- /dev/null +++ b/content/blog/2024-07-12-unikraft-gsoc-test-mimalloc-on-unikraft.mdx @@ -0,0 +1,146 @@ +--- +title: "GSoC'24: Testing the mimalloc Memory Allocator on Unikraft" +description: | + Different applications have different memory usage patterns and perform better + when using dedicated memory allocators. Right now, the buddy allocator is the only + available memory allocator used in Unikraft, so this GSoC project aims to + port more memory allocators to it, starting with mimalloc. +publishedDate: 2024-07-12 +image: /images/unikraft-gsoc24.png +authors: +- Yang Hu +tags: +- gsoc +- gsoc24 +- allocators +- synchronization +--- + +## Project Overview + +### Memory allocation in Unikraft + +Memory allocators have a significant impact on application performance. +[Research](https://dl.acm.org/doi/10.1145/378795.378821) have shown that programs can run as much as 60% faster just by switching to an appropriate memory allocator. +Unikraft currently supports only one general-purpose memory allocator, the buddy allocator. +This project aims to port [mimalloc](https://github.com/microsoft/mimalloc) (pronounced "me-malloc"), a high-performance general-purpose memory allocator developed by Microsoft, to Unikraft. +This is the first step in a series of efforts to provide Unikraft users with more memory allocators to choose from to optimize the performance of their applications. + +### Objectives + +[Hugo Lefeuvre](https://github.com/hlef) has made an effort to port mimalloc to Unikraft back in 2020 (see [this repo](https://github.com/unikraft/lib-mimalloc)). +However, as Unikraft has evolved significantly over the years, more work is needed to adapt the `lib-mimalloc` port to the latest Unikraft core. + +The steps of porting the memory allocators are as follows: + +1. Adapt the existing port, which uses `lib-newlib` and `lib-pthread-embedded` as the C library to use `lib-musl` instead +1. Patch the existing port of mimalloc (v1.6.3) to adapt to the latest Unikraft's memory allocation interface +1. Patch the latest mimalloc (v2.1.7) to the latest Unikraft version (v0.17.0) + +## Current Progress + +Last month, I successfully ported the mimalloc memory allocator to Unikraft, marked by a successful compilation of mimalloc `v1.6.3` against the latest Unikraft core (v0.17.0) with `lib-musl`. +Following that, this month I am focused on stress-testing the `mimalloc` memory allocator to validate its usability and correctness. + +### Benchmark + +After porting was completed, I tested the allocator by running a few `malloc()` and `free()` calls in a single threaded `main()` function. + +That having no problem, I ported `cache-scratch`, a multi-threaded memory allocation benchmark, to Unikraft. +This benchmarkthat exercises a heap's cache-locality and tests for [passive false sharing](https://psy-lob-saw.blogspot.com/2014/06/notes-on-false-sharing.html). +It creates a number of worker threads that allocate a given-sized object, repeatedly write on it, then free it. +An allocator that allows multiple threads to re-use the same small object (possibly all in one cache-line) will scale poorly, while an allocator like mimalloc, which gives each thread a private and page-aligned heap, will exhibit near-linear performance scaling. + +### Porting `cache-scratch` + +It is fairly intuitive to port the C program itself to Unikraft as the Musl library already provides most of what we need, including the `pthread` interface. +It would be ideal if we could separate the helper functions from the main logic and build the benchmark as a library (as we do need to test the memory allocator against more benchmarks in the future), because then we could just build an all-in-one image and run a large number of tests with different benchmarks and different parameters using an automated script. +However, I haven't figured out a good way to do it as [Unikraft's build system](https://unikraft.org/docs/internals/build-system#makefileuk) is quite complex. +So for now, the benchmark is ported as [a single `main.c` file](https://gist.github.com/huyang531/28a31fc2d9f348fafb5b9d8e6c9493d5) that will run as an application. + +### Testing + +I ran the `cache-scratch` benchmark with `-nthreads 4 -iterations 1000 -objSize 8 -repititions 100000`. +That is, we will create four worker threads, each iterating 1000 times the following operations: + +1. Allocate an 8-byte object +1. Repeatedly scratch (write then read) the bytes in the object, one byte at a time, for 100,000 times + +When the main thread waits for the worker threads by calling `pthread_join()`, a general protection fault would occur: + +```console +[ 19.545757] dbg: {r:0x1f5634,f:0x11d40} [libmimalloc] allocating 24 bytes from mimalloc +[ 19.560961] dbg: {r:0x1f5634,f:0x11d40} [libmimalloc] allocating 24 bytes from mimalloc +[ 19.576167] CRIT: {r:0x107d37,f:0x289f10} [libkvmplat] Unhandled Trap 13 (general protection), error code=0x0 +[ 19.593306] CRIT: {r:0x106e82,f:0x289ef0} [libkvmplat] RIP: 00000000001c2842 CS: 0008 +[ 19.609267] CRIT: {r:0x106ed7,f:0x289ef0} [libkvmplat] RSP: 0000000000011d70 SS: 0010 EFLAGS: 00010246 +[ 19.626078] CRIT: {r:0x106f23,f:0x289ef0} [libkvmplat] RAX: fffe796000000000 RBX: 0000000410450018 RCX: 0000000000000000 +[ 19.643706] CRIT: {r:0x106f6f,f:0x289ef0} [libkvmplat] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 +[ 19.661342] CRIT: {r:0x106fbb,f:0x289ef0} [libkvmplat] RBP: 0000000000011da0 R08: 00000000ffffffff R09: 0000000000000000 +[ 19.678988] CRIT: {r:0x107007,f:0x289ef0} [libkvmplat] R10: 0000000000133c42 R11: 000000000000002d R12: 0000000000011df0 +[ 19.696691] CRIT: {r:0x107053,f:0x289ef0} [libkvmplat] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 +[ 19.714323] CRIT: {r:0x107d8f,f:0x289f20} [libkvmplat] Crashing +``` + +### Debugging + +A big portion of my work involves [debugging the unikernel](https://unikraft.org/guides/debugging). +For instance, I used `addr2line` to reveal the above error: + +```console +addr2line -e workdir/build/helloworld-pg_qemu-x86_64.dbg 00000000001c2842 +/home/huyang/code/gsoc/apps/helloworld-pg/workdir/build/libmusl/origin/musl-1.2.3//src/thread/pthread_join.c:16 +``` + +Then I used GDB to probe deeper and confirm the error: + +```console +(gdb) n +16 while ((state = t->detach_state) && r != ETIMEDOUT && r != EINVAL) { +(gdb) p t +$1 = (pthread_t) 0xfffe796000000000 +(gdb) p t->detach_state +Cannot access memory at address 0xfffe796000000038 +``` + +This indicates that `t` is a corrupted pointer, and more investigation is needed to find out how memory was tampered. + +### Other work + +While testing the allocator, I noticed that sometimes the boot process gets stuck at this test: +`posix_futex_testsuite->test_cmp_requeue_two_waiters_no_requeue`. +Backtracing the call stack reviews that it happens because once one of the Waiter threads blocks at the `futex()` call, other threads (including the `init` thread, the other Waiter, and the Requeuer thread) will never be given a chance to run. +Experiments show that it only happens when the QEMU virtual machine's memory size is smaller than a threshold (in my case, it is 427M), which is interesting because it isn't straightforward how the VM's memory size could impact the behaviour of the `futex()` system call. + +I also added tips about turning off compiler optimization for debugging [here](https://github.com/unikraft/docs/pull/442). + +## Next steps + +First, I will continue working on the `cache-scratch` benchmark: + +1. Debug `cache-scratch`: Figure out what went wrong in the multi-threaded test +1. Run `cache-scratch` with different sets of arguments to validate the memory allocator under different conditions + +After that, I will test mimalloc with other more complicated benchmarks, which may include: + +- `threadtest`: tests general scalability of the allocator; +Each thread repeatedly allocates a number of number of objects, does a fixed amount of computation, and then deletes its objects. +- `linux-scalability`: a basic scalability test from Linux libc malloc testing, similar to `threadtest` except that there is no extra 'work' between allocations and frees so contention may be higher. +- `cache-thrash`: tests for active false sharing +- `larson`: simulates a server by having each thread allocate and deallocate objects and then transfer some objects to other threads to be freed +- `phong` - a benchmark that randomly selects allocation sizes, and randomly chooses an allocated item to free + +Once fully tested, I will continue on with the work items outlined in [my previous post](https://unikraft.org/blog/2024-06-16-unikraft-gsoc-port-mimalloc-to-unikraft#next-steps). + +Meanwhlie, I will also try to debug the `test_cmp_requeue_two_waiters_no_requeue` to the best of my ability. + +## Acknowledgements + +Special thanks to Răzvan Vîrtan and Radu Nichita, my two amazing mentors, for their support along the way. +I would also like to thank Răzvan Deaconescu, Hugo Lefeuvre, Ștefan Jumarea, Marco Schlumpp, Sergiu Moga, and the entire Unikraft community for all the work, discussions, and guidance which made this project possible. + +## About Me + +I'm [Yang Hu](https://linkedin.com/yanghuu531), a first year graduate student at the University of Toronto. +I am enthusiastic about operating systems and building infrastructure software in general. +In my free time, I really enjoy traveling and swimming. diff --git a/content/blog/2024-08-01-unikraft-gsoc-debugging-mimalloc.mdx b/content/blog/2024-08-01-unikraft-gsoc-debugging-mimalloc.mdx new file mode 100644 index 00000000..22ee8873 --- /dev/null +++ b/content/blog/2024-08-01-unikraft-gsoc-debugging-mimalloc.mdx @@ -0,0 +1,134 @@ +--- +title: "GSoC'24: Debugging the Multithreaded Memory Allocation Interface" +description: | + Different applications have different memory usage patterns and perform better + when using dedicated memory allocators. Right now, the buddy allocator is the only + available memory allocator used in Unikraft, so this GSoC project aims to + port more memory allocators to it, starting with mimalloc. +publishedDate: 2024-08-01 +image: /images/unikraft-gsoc24.png +authors: +- Yang Hu +tags: +- gsoc +- gsoc24 +- allocators +- synchronization +--- + +## Project Overview + +### Memory allocation in Unikraft + +Memory allocators have a significant impact on application performance. +There have been some [research papers](https://dl.acm.org/doi/10.1145/378795.378821) which compared different memory allocators. +According to their work, Switching to the appropiate memory allocator may improve the application performance by 60%. +Unikraft currently supports only one general-purpose memory allocator, the buddy allocator. +This project aims to port [mimalloc](https://github.com/microsoft/mimalloc) (pronounced "me-malloc"), a high-performance general-purpose memory allocator developed by Microsoft, to Unikraft. +This is the first step in a series of efforts to provide Unikraft users with more memory allocators to choose from to enhance the performance of their applications. + +### Objectives + +[Hugo Lefeuvre](https://github.com/hlef) made an initial work to port mimalloc to Unikraft back in 2020 (see [this repo](https://github.com/unikraft/lib-mimalloc)). +However, as Unikraft has evolved significantly over the years, more work has to be done in order to adapt the `lib-mimalloc` port to the latest Unikraft core. + +So far, I have successfully ported the mimalloc memory allocator to Unikraft, marked by a successful compilation of mimalloc `v1.6.3` against the latest Unikraft core (v0.17.0) with `lib-musl`. +Now, I am focused on validating the functionality and benchmarking the performance of the allocator. +The next step would be to port the latest mimalloc (v2.1.7) to the latest Unikraft version (v0.17.0). + +## Current Progress + +I have successfully ported the multithreaded `cache-scratch` stress test to Unikraft (as outlined in the [previous blog post](https://unikraft.org/blog/2024-07-12-unikraft-gsoc-test-mimalloc-on-unikraft)). +For the past three weeks, I have been debugging it to get it running. + +### Debugging `pthread` + +The [first error we ran into](https://unikraft.org/blog/2024-07-12-unikraft-gsoc-test-mimalloc-on-unikraft#testing) was a memory corruption error with the `pthread` interface. +Turns out, this was due to the lack of virtual memory support so it was easily solved by enabling the `ukvmem` library. +> By the way, this is an error that would have never happened had I checked the return value of `pthread_create()` in the first place - lessons learned! + +### Debugging boot time virtual memory initialization + +There is a reason why we turned off virtual memory support when first porting the memory allocator. +Unikraft's virtual memory interface requires the memory allocators to behave in certain ways that are not intrinsically supported by *mimalloc*. +For example, this is what will happen when initializing the heap with virtual memory turned on: + +```c + /* In addition to paging, we have virtual address space management. We + * will thus also represent the heap as a dedicated VMA to enable + * on-demand paging for the heap. However, we have a chicken-egg + * problem here. This is because we need an allocator for allocating + * VMA objects but to do so, we need to create the heap VMA first. + * Theoretically, we could use a simple region allocator to bootstrap. + * But then, we would have to cope with VMA objects from different + * allocators. So instead, we just initialize a small heap using fixed + * mappings and then create the VMA on top. Afterwards, we add the + * remainder of the VMA to the allocator. + */ +``` + +This is because Unikraft's Virtual Memory Areas (VMAs, described by `struct uk_vma`) are managed by a Virtual Address Space (VAS, described by `struct uk_vas`), which has one dedicated memory allocator assigned to it. +So it would be impractical to allocate two VMAs under a same VAS using two different memory allocators. + +The code for this logic though, is customized for the buddy allocator: + +1. Initialize a physically-mapped small heap size of 16 pages +1. Initialize the allocator based on this heap +1. Initialize the VAS +1. Map the rest of the heap as an anonymous VMA +1. Call `uk_alloc_addmem()` to let the buddy allocator know that this anonymous region that is at its disposal + +This design accommodates the fact that the buddy allocator needs initial memory space to store its metadata and that it needs information about the total available memory for initializing its free lists. +However, for *mimalloc*, we are not initializing the allocator per-se at this time because the Thread Local Storage (TLS) infrastructure that it needs is not yet ready. +Instead, we initialize an internal *regions* allocator, and satisfy boot-time memory allocations using it until TLS is ready. +It remains to be seen if this workaround is the best approach, though. +> See [Hugo's thesis](https://os.itec.kit.edu/downloads/2020_BA_Lefeuvre_Toward_Specialization_of_Memory_Management_in_Unikernels.pdf) for more on this topic. + +Therefore, I have adapted the logic here to accommodate both allocators. + +### Debugging virtual memory interface + +Unikraft's current virtual memory interface is not without its flaws. +I ran into multiple "Cannot handle read page fault" errors during boot time. + +```console +[ 11.290098] Info: {r:0x175768,f:0x11e70} test: posix_futex_testsuite->test_wait_timeout +[ 11.304300] dbg: {r:0x130da1,f:0x11ba0} [libposix_futex] (int) uk_syscall_r_futex((uint32_t *) 0x11e48, (int) 0x0, (uint32_t) 0xa, (const struct timespec *) 0x11e30, (uint32_t *) 0x0, (uint32_t) 0x0) +[ 11.326513] dbg: {r:0x13094b,f:0x11af0} [libposix_futex] FUTEX_WAIT: Condition met (*uaddr == 10, uaddr: 0x11e48) +[ 11.342579] dbg: {r:0x1309ae,f:0x11af0} [libposix_futex] FUTEX_WAIT: Wait 12326512446 nsec for wake-up +[ 11.358184] dbg: {r:0x207f6f,f:0x119d0} [libcontext] Sysctx 0x1000025fc8 GS_BASE register updated to 0x29e140 (before: 0x11bf0) +[ 11.375007] CRIT: {r:0x17dde9,f:0x28ce70} [libukvmem] Cannot handle exec page fault at 0x0 (ec: 0x10): Bad address (14). +[ 11.392395] CRIT: {r:0x106f17,f:0x28cf00} [libkvmplat] RIP: 0000000000000000 CS: 0008 +[ 11.407999] CRIT: {r:0x106f6c,f:0x28cf00} [libkvmplat] RSP: 000000100003afe8 SS: 0010 EFLAGS: 00010246 +[ 11.424430] CRIT: {r:0x106fb8,f:0x28cf00} [libkvmplat] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000 +[ 11.441611] CRIT: {r:0x107004,f:0x28cf00} [libkvmplat] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000 +[ 11.458821] CRIT: {r:0x107050,f:0x28cf00} [libkvmplat] RBP: 0000000000011a70 R08: 0000000000000000 R09: 0000000000000000 +[ 11.476011] CRIT: {r:0x10709c,f:0x28cf00} [libkvmplat] R10: 0000000000000000 R11: 000000000000002d R12: 0000000000000000 +[ 11.493210] CRIT: {r:0x1070e8,f:0x28cf00} [libkvmplat] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 +[ 11.510388] CRIT: {r:0x107240,f:0x28cf00} [libkvmplat] base is 0x11a70 caller is 0x1732e6 +``` + +After extensive debugging, I realized that this bug is not unique to the *mimalloc* port. +I reproduced a similar error with the same library configurations on the buddy allocator as well and I have submitted [an issue](https://github.com/unikraft/unikraft/issues/1475) addressing this. +More work needs to be done to fix the virtual memory infrastructure. + +## Next steps + +In order to resolve the abovementioned issues, I still have much to study about Unikraft's virtual memory management subsystem, its boot process, and how *mimalloc* works under the hood - and I will focus on just that in the following weeks. + +Hopefully, with those issues fixed, we could finally get the `cache-scratch` test running on multiple cores. +Once that is done, I will get back on track with my "global" TODO list: + +1. Run more benchmarks on the current port to validate its correctness and performance (see more [here](https://unikraft.org/blog/2024-07-12-unikraft-gsoc-test-mimalloc-on-unikraft#next-steps)) +1. Port the latest version of mimalloc to Unikraft + +## Acknowledgements + +Special thanks to Răzvan Vîrtan and Radu Nichita, my two amazing mentors, for their support along the way. +I would also like to thank Răzvan Deaconescu, Hugo Lefeuvre, Ștefan Jumarea, Marco Schlumpp, Sergiu Moga, and the entire Unikraft community for all the work, discussions, and guidance which made this project possible. + +## About Me + +I'm [Yang Hu](https://linkedin.com/yanghuu531), a first year graduate student at the University of Toronto. +I am enthusiastic about operating systems and building infrastructure software in general. +In my free time, I really enjoy traveling and swimming. diff --git a/content/blog/2024-08-02-unikraft-gsoc-lxboot_x86.mdx b/content/blog/2024-08-02-unikraft-gsoc-lxboot_x86.mdx new file mode 100644 index 00000000..db06ae33 --- /dev/null +++ b/content/blog/2024-08-02-unikraft-gsoc-lxboot_x86.mdx @@ -0,0 +1,106 @@ +--- +title: "GSoC'24: Linux x86 Boot Protocol Support" +description: | + The Linux boot protocol plays an important role in the initialization of the Linux operating system, emphasizing the importance of system optimization and scalability. +publishedDate: 2024-08-02 +image: /images/unikraft-gsoc24.png +tags: +- gsoc +- gsoc24 +- booting +authors: +- Mihnea Firoiu +--- + +## Setup in Unikraft + +First of all, I added the following lines to the [lxboot.S](https://github.com/unikraft/unikraft/releases/tag/RELEASE-0.17.0) file from Unikraft: + +```diff +diff --git a/plat/kvm/x86/lxboot.S b/plat/kvm/x86/lxboot.S +index 0ebe18a5..8140804f 100644 +--- a/plat/kvm/x86/lxboot.S ++++ b/plat/kvm/x86/lxboot.S +@@ -21,6 +21,13 @@ lcpu_boot_startup_args: + .quad 0 + .quad 0 + ++.code32 ++.section .text.32.boot ++ENTRY(_lxboot_entry32) ++ cli ++ hlt ++END(_lxboot_entry32) ++ + /** + * 64-bit Linux Boot Protocol entry function + * +``` + +This way, if my header is correct and I do indeed reach Unikraft, the guest should hang. + +## Starting to debug + +There are 2 binary files that are of interest here: `linuxboot.bin` and `linuxboot_dma.bin`. +The `linuxboot.bin` file is optained by compiling [linuxboot.S](https://github.com/qemu/qemu/blob/stable-9.0/pc-bios/optionrom/linuxboot.S) and the `linuxboot_dma.bin` file is optained by compiling [linuxboot_dma.c](https://github.com/qemu/qemu/blob/stable-9.0/pc-bios/optionrom/linuxboot_dma.c). +As you can see, one is written in x86 AT&T assembly and the other is written in C, by using inline assembly. +Naturally, the one written in C would be easier to debug, and luckily, it seems like I do no need the other one. + +By adding in [linuxboot.S](https://github.com/qemu/qemu/blob/stable-9.0/pc-bios/optionrom/linuxboot.S) the following code, the QEMU instance should stay in a loop: + +```diff +diff --git a/pc-bios/optionrom/linuxboot.S b/pc-bios/optionrom/linuxboot.S +index ba821ab922..26df825db8 100644 +--- a/pc-bios/optionrom/linuxboot.S ++++ b/pc-bios/optionrom/linuxboot.S +@@ -30,6 +30,8 @@ run_linuxboot: + + cli + cld ++0: ++ jmp 0b + + jmp copy_kernel + boot_kernel: + +``` + +I observed that nothing changes in its behaviour, so I added the following lines in [linuxboot_dma.c](https://github.com/qemu/qemu/blob/stable-9.0/pc-bios/optionrom/linuxboot_dma.c): + +```diff +diff --git a/pc-bios/optionrom/linuxboot_dma.c b/pc-bios/optionrom/linuxboot_dma.c +index cbcf6679d9..4f12fe359e 100644 +--- a/pc-bios/optionrom/linuxboot_dma.c ++++ b/pc-bios/optionrom/linuxboot_dma.c +@@ -55,6 +55,8 @@ asm( + "_bev:\n" + " cli\n" + " cld\n" ++"some_label:\n" ++" jmp some_label\n" + " jmp load_kernel\n" + ); +``` + +This time the QEMU instance stayed in the loop, so everything was fine. +Now, all I had to do was to change the `jmp` with two `nop` instructions in gdb and from there to start debugging. +Sounds easy, right? + +## Unexpected challenges + +For some (at the time) unknown reason, when compiling the `.bin` files didn't update. +Was it my fault? +Did I have something wrong with my setup? + +After a lot of time, I figured out along with my mentors that I needed to have `gcc-multilib` instead of the one that I had previously. +As different versions of gcc clash (installing one uninstalls the other), I had to replace it. + +I observed that there are 3 `linuxboot.bin` and 3 `linuxboot_dma.bin` in the filesystem. +Only one of each was updated when compiling, and it wasn't the one that it searched for by default. +Using `strace`, I figured out that QEMU looks for the `.bin` file in the directory from which the command runs. +Because of this, I copied the `linuxboot_dma.bin` that was updated where I needed it. + +## Next steps + +I am going to look what and where the problem is inside the `linuxboot_dma.bin` file, using gdb. +Everything took much more than I expected, so I have to do this now. diff --git a/content/blog/2024-08-07-gsoc-uefi-gop.mdx b/content/blog/2024-08-07-gsoc-uefi-gop.mdx new file mode 100644 index 00000000..e6e605d0 --- /dev/null +++ b/content/blog/2024-08-07-gsoc-uefi-gop.mdx @@ -0,0 +1,98 @@ +--- +title: "GSoC'24: UEFI Graphics Output Protocol Support in Unikraft, Part III" +description: | + This is the third post in a series of posts where I talk about my progress with the project. +publishedDate: 2024-08-07 +image: /images/unikraft-gsoc24.png +authors: +- Sriprad Potukuchi +tags: +- gsoc +- gsoc24 +- uefi +- booting +--- + +## Project Overview + +The widely available and standardized [UEFI Graphics Output Protocol](https://uefi.org/specs/UEFI/2.10/12_Protocols_Console_Support.html#efi-graphics-output-protocol) (GOP) interface is an excellent alternative to VGA or serial port consoles for printing logs to the screen. + +This project aims to implement a UEFI GOP based console. +For more information, check out [Part I](https://unikraft.org/blog/2024-06-18-gsoc-uefi-gop) and [Part II](https://unikraft.org/blog/2024-07-10-gsoc-uefi-gop) of this series. + +## Progress + +All the work referred to here can be found in [this draft PR](https://github.com/unikraft/unikraft/pull/1448). + +### Arbitrary fonts + +The driver can now work with any arbitrary bitmap font! The bitmap font array, along with the width and height of each character can be defined in `drivers/uktty/gop/font.c` as follows: + +```c +__u8 char_width = 8; +__u8 char_height = 16; + +__u8 font[256][16] = { + {0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}, // control char + /* ... */ + {0x00, 0x00, 0x7C, 0xC6, 0xC6, 0x0C, 0x18, 0x18, 0x18, 0x00, 0x18, 0x18, 0x00, 0x00, 0x00, 0x00}, // question + {0x00, 0x00, 0x00, 0x7C, 0xC6, 0xC6, 0xDE, 0xDE, 0xDE, 0xDC, 0xC0, 0x7C, 0x00, 0x00, 0x00, 0x00}, // at + {0x00, 0x00, 0x10, 0x38, 0x6C, 0xC6, 0xC6, 0xFE, 0xC6, 0xC6, 0xC6, 0xC6, 0x00, 0x00, 0x00, 0x00}, // A + {0x00, 0x00, 0xFC, 0x66, 0x66, 0x66, 0x7C, 0x66, 0x66, 0x66, 0x66, 0xFC, 0x00, 0x00, 0x00, 0x00}, // B + /* ... */ + {0x00, 0x00, 0x00, 0x00, 0x10, 0x38, 0x6C, 0xC6, 0xC6, 0xC6, 0xFE, 0x00, 0x00, 0x00, 0x00, 0x00}, // char127 +}; +``` + +The driver code in `drivers/uktty/gop/gop.c` externs these symbols and directly accesses them. + +```c +/* Font metadata */ +extern __u8 char_width; +extern __u8 char_height; +extern __u8 font[256][16]; /* Default font */ +``` + +### Generic console device interface + +A generic console device interface was proposed by [PR #1464](https://github.com/unikraft/unikraft/pull/1464) submitted by [thass0](https://github.com/thass0) on GitHub. This interface, exposed through a library called `ukconsole`, allows you to: + +- Register console devices, each of which uses its own underlying interfaces (serial, VGA, UEFI GOP etc). +- Query registered console devices. +- Write bytes to either or all registered console devices to print logs. +- Read bytes from a given console device to take user input. + +All of my work has been rebased onto the work done in [PR #1464](https://github.com/unikraft/unikraft/pull/1464). +The GOP driver has been modified into a console device that can now be registered to the `ukconsole` library. + + + +This means that the kernel can be configured to transparently use this GOP console device without having to change any of the existing print statements! + +### Refactoring + +The code was moved to `drivers/uktty/gop/` from `plat/common/` which was a temporary location. +This was done in order to integrate with `ukconsole`. + +In addition to this, the code underwent a substantial overhaul. +It is now much cleaner! + +## Next Steps + +- Support for all ASCII codes (like `\a`, `\t` etc) must be added. + +- The logs are not colored! + The image above shows some weird characters being printed along with the actual logs. + + These are escape codes used to color the logs themselves, which must be interpreted as such and not actually printed onto the console. + +- The console device must be tested against all configuration options such as `CONFIG_LIBUKDEBUG_ANSI_COLOR`, which dictate some logging parameters. + +- The code must be brought to a mergeable state by adding license information, checking the formatting etc. + +## Acknowledgement + +I would once again like to thank all the great Unikraft folk for their continued support! This work would not have been possible without them :heart: diff --git a/content/community/about.mdx b/content/community/about.mdx index 5661c079..4c8751a7 100644 --- a/content/community/about.mdx +++ b/content/community/about.mdx @@ -11,7 +11,7 @@ software engineers, researchers, teachers, students and hobbyists. Periodic meetings, hackathons and a consistent presence in open source events are central to the well functioning of the community. -We use a guidelines for development and maintenance to ensure the creation of +We use guidelines for development and maintenance to ensure the creation of high quality code. Public releases are planned to happen once every two months. In general, we aim diff --git a/content/docs/cli/install.mdx b/content/docs/cli/install.mdx index 8ca4906f..f6de8029 100644 --- a/content/docs/cli/install.mdx +++ b/content/docs/cli/install.mdx @@ -20,7 +20,7 @@ Simply run the following command to get started: curl --proto '=https' --tlsv1.2 -sSf https://get.kraftkit.sh | sh ``` -The above script will identify your host and guide you through an the +The above script will identify your host and guide you through the installation process, including suggestions for how to receive updates. The installer will try to use your native package manager. diff --git a/content/docs/cli/running.mdx b/content/docs/cli/running.mdx index 2cb9690c..81d2f53d 100644 --- a/content/docs/cli/running.mdx +++ b/content/docs/cli/running.mdx @@ -10,7 +10,7 @@ At present, `kraft` only supports running instances that target the KVM hypervis We plan on adding support for additional hypervisors including Xen, VMware and Hyper-V. For more information the progress of these hypervisors, you can [track the relevant GitHub issue](#). - + There are multiple ways to instantiate a unikernel with `kraft`. The subcommand `kraft run` has been designed to be context-aware and can be used in different ways to launch unikernels depending on what is provided to it. diff --git a/content/docs/concepts/virtualization.mdx b/content/docs/concepts/virtualization.mdx index 2d15f04a..b594fec1 100644 --- a/content/docs/concepts/virtualization.mdx +++ b/content/docs/concepts/virtualization.mdx @@ -8,38 +8,41 @@ description: | Virtualization can be done using a hypervisor, which is a low-level software that virtualizes the underlying hardware and manages access to the real -hardware, either directly or through the host Operating System. There are 2 main -virtualized environments: virtual machines and containers, each with pros and -cons regarding complexity, size, performance and security. Unikernels come -somewhere between those 2. +hardware, either directly or through the host Operating System. +There are 2 main virtualized environments: virtual machines and containers, +each with pros and cons regarding complexity, size, performance and security. +Unikernels come somewhere between those 2. ## Virtual Machines A virtual machine represents an abstraction of the hardware, over which an operating system can run, thinking that it is alone on the system and that it -controls the hardware below it. Virtual machines rely on hypervisors to run -properly. +controls the hardware below it. +Virtual machines rely on hypervisors to run properly. A hypervisor incorporates hardware-enabled multiplexing and segmentation of compute resources in order to better utilize, better secure and better -facilitate the instantenous runtime of user-defined programs. By the -means of a virtual machine, an operating system is unaware of the multiplexing -which happens to facilitate its existence. The hypervisor emulates devices on -behalf of the guest OS, including providing access to virtual CPUs, virtual -Interrupt Controllers and virtual NICs, which are segmented from the underlying -hardware. +facilitate the instantenous runtime of user-defined programs. +By the means of a virtual machine, an operating system is unaware of the +multiplexing which happens to facilitate its existence. +The hypervisor emulates devices on behalf of the guest OS, including +providing access to virtual CPUs, virtual Interrupt Controllers and virtual +NICs, which are segmented from the underlying hardware. The hypervisors can be classified in 2 categories: Type 1 and Type 2: * The **Type 1 hypervisor**, also known as **bare-metal hypervisor**, has direct access to the hardware and controls all the operating systems that are running - on the system. The guest OSes have access to the physical hardware, and the - hypervisor arbiters this accesses. KVM is an example of Type 1 hypervisor. + on the system. + The guest OSes have access to the physical hardware, and the hypervisor + arbiters this accesses. + KVM is an example of Type 1 hypervisor. -* The **Type 2 hypervisor**, also known as **hosted hypervisor**, has to go - through the host operating system to reach the hardware. Access to the - hardware is emulated, using software components that behave in the same way - as the hardware ones. An example of Type 2 hypervisor is VirtualBox. +* The **Type 2 hypervisor**, also known as **hosted hypervisor**, has to go + through the host operating system to reach the hardware. + Access to the hardware is emulated, using software components that behave + in the same way as the hardware ones. + An example of Type 2 hypervisor is VirtualBox. Commit messages, along with all source files follow a 80-character rule. diff --git a/content/docs/getting-started/index.mdx b/content/docs/getting-started/index.mdx index 22a57d50..ba4bc1fc 100644 --- a/content/docs/getting-started/index.mdx +++ b/content/docs/getting-started/index.mdx @@ -49,7 +49,7 @@ We've put together a number of guides in case you'd like to know more about Unik * [Using the Application Catalog](/guides/using-the-app-catalog) * [Behind the Scenes with the Application Catalog](/guides/catalog-behind-the-scenes) - + ## Contribute diff --git a/content/docs/internals/booting.mdx b/content/docs/internals/booting.mdx index e562b977..f771f303 100644 --- a/content/docs/internals/booting.mdx +++ b/content/docs/internals/booting.mdx @@ -1,12 +1,12 @@ --- title: Booting description: | - Unikraft has an prograrmmable boot sequence which provides the ability to inject functionality at different moments of system initialization. Learn how to how and where to introduce custom functionality. + Unikraft has a prograrmmable boot sequence which provides the ability to inject functionality at different moments of system initialization. Learn how to and where to introduce custom functionality. --- ## Unikraft Boot Sequence -The Unikraft boot sequence is dependent on the selected platform (linuxu, kvm, xen), but is very similar to how any other operating system would behave. +The Unikraft boot sequence is dependent on the selected platform (`qemu`, `firecracker` or `xen`), but is very similar to how any other operating system would behave. In the case of the `KVM` platform, the booting process involves more steps. First, the image is loaded from the disk into memory and the program sections are defined (see `plat/kvm/x86/link64.lds.S`, as example for the x86 architecture). diff --git a/content/docs/internals/debugging.mdx b/content/docs/internals/debugging.mdx index 34a7ef1a..49b7d2e3 100644 --- a/content/docs/internals/debugging.mdx +++ b/content/docs/internals/debugging.mdx @@ -40,14 +40,6 @@ We recommend 3, the highest level. Once set, save the configuration and build your images. -#### Linuxu - -For the Linux user space target (`linuxu`) simply point GDB to the resulting debug image, for example: - -```console -$ gdb build/app-helloworld_linuxu-x86_64.dbg -``` - #### KVM For KVM, you need to start the guest with the kernel image that does not include debugging information. diff --git a/content/docs/internals/writing-tests.mdx b/content/docs/internals/writing-tests.mdx index c93ee61d..d439b174 100644 --- a/content/docs/internals/writing-tests.mdx +++ b/content/docs/internals/writing-tests.mdx @@ -46,7 +46,7 @@ Generally, the following pattern is used for test suites: `$LIBNAME_$TESTSUITENAME_testsuite` -An the following for test cases: +And the following for test cases: `$LIBNAME_test_$TESTCASENAME` diff --git a/content/guides/baby-steps.mdx b/content/guides/baby-steps.mdx index 703012ae..f18117f8 100644 --- a/content/guides/baby-steps.mdx +++ b/content/guides/baby-steps.mdx @@ -5,12 +5,12 @@ description: | --- It does so in the context of a vibrant community consisting of highly skilled software engineers, researchers, teachers, students and hobbyists. -Periodic meetings, hackathons and a consistent presence in open source events are central to the well functioning of the community. +Periodic meetings, hackathons and a consistent presence in open-source events are central to the well functioning of the community. -We use a guidelines for development and maintenance to ensure the creation of high quality code. +We use guidelines for development and maintenance to ensure the creation of high-quality code. Public releases are planned to happen once every two months. -In general, we aim for a public release to happen at the last Monday of each odd month (January, March, May, etc.) +In general, we aim for a public release to happen on the last Monday of each odd month (January, March, May, etc.) -We welcome contributors and users on Discord and on GitHub. -If you are looking for a fun technical project in the area of operating systems, virtualization, come aboard on Discord. +We welcome contributors and users on Discord and GitHub. +If you are looking for a fun technical project in the area of operating systems, and virtualization, come aboard on Discord. diff --git a/content/guides/catalog-behind-the-scenes.mdx b/content/guides/catalog-behind-the-scenes.mdx index 37b85ee0..f18a0742 100644 --- a/content/guides/catalog-behind-the-scenes.mdx +++ b/content/guides/catalog-behind-the-scenes.mdx @@ -5,7 +5,7 @@ description: | It shows what is happening behind the scenes and how you can get more control on the build and run phases. --- -The [guide on using the application catalog](/guides/using-the-app-catalog) provides the user-friendly information on using the Unikraft application registry and the [`catalog` repository](https://github.com/unikraft/catalog). +The [guide on using the application catalog](/guides/using-the-app-catalog) provides user-friendly information on using the Unikraft application registry and the [`catalog` repository](https://github.com/unikraft/catalog). It presents some hints into what's happening behind the scenes, but it aims to keep the user away for this. This guide takes a deep dive into the internals of configuring, building and running Unikernel applications from the catalog. It is aimed for those more technically inclined who would be interested in understanding what's happening behind the scenes and maybe [contribute to the application catalog](/docs/contributing/adding-to-the-app-catalog). @@ -38,7 +38,7 @@ The `Kraftfile` defines the: - root filesystem used to build the (embedded) initrd The root filesystem is generated from a `Dockerfile` specification, as configured in the `Kraftfile`. -The `Dockerfile` specification collects the required files (binary executable, depending libraries, configuration files, data files): +The `Dockerfile` specification collects the required files (binary executable, depending libraries, configuration files, and data files): ```dockerfile FROM --platform=linux/x86_64 nginx:1.25.3-bookworm AS build @@ -141,13 +141,13 @@ Query it using: curl http://localhost:8080 ``` -If you want use a bridge interface, first create the bridge interface as `root` (prefix with `sudo` if required): +If you want to use a bridge interface, first create the bridge interface as `root` (prefix with `sudo` if required): ```bash kraft net create -n 172.44.0.1/24 virbr0 ``` -An the run manually with `qemu-system-x86_64` as `root` (prefix with `sudo` if required): +And then run manually with `qemu-system-x86_64` as `root` (prefix with `sudo` if required): ```bash qemu-system-x86_64 \ @@ -171,7 +171,7 @@ To close the running QEMU instance, use `Ctrl+a x` in the QEMU console. ## HTTP Go Server For the [`http-go1.21` bincompat example](https://github.com/unikraft/catalog/tree/main/examples/http-go1.21), there is no build phase, only a run phase. -The example it's using a prebuilt kernel image. +The example is using a prebuilt kernel image. The prebuilt [`base` kernel image](https://github.com/unikraft/catalog/tree/main/library/base) is pulled from the registry, from `unikraft.org/base`. This happens during the run phase. @@ -197,7 +197,7 @@ The `Kraftfile` defines: - the available run targets: currently only x86_64-based builds are available, and only KVM-based builds, using QEMU or Firecracker The root filesystem is generated from a `Dockerfile` specification, as configured in the `Kraftfile`. -The `Dockerfile` specification collects the required files (binary executable, depending libraries, configuration files, data files): +The `Dockerfile` specification collects the required files (binary executable, depending libraries, configuration files, and data files): ```dockerfile FROM golang:1.21.3-bookworm AS build @@ -243,7 +243,7 @@ kraft run -W -p 8080:8080 . 1. It pulls the kernel package from the registry, from `unikraft.org/base:latest`. 1. It generates the root filesystem, via BuildKit from the `Dockerfile` specification. - The generation of the root filesystem implies the building the Go source code files into a binary executable (`ELF`). + The generation of the root filesystem implies the building of the Go source code files into a binary executable (`ELF`). The executable, together with the depending libraries is then extracted into the root filesystem. 1. It packs the root filesystem in an initial ramdisk (initrd). 1. It runs the kernel attaching the initrd and using the command line in the specification: `/http_server`. @@ -316,13 +316,13 @@ Query it using: curl http://localhost:8080 ``` -If you want use a bridge interface, first create the bridge interface as `root` (prefix with `sudo` if required): +If you want to use a bridge interface, first create the bridge interface as `root` (prefix with `sudo` if required): ```bash kraft net create -n 172.44.0.1/24 virbr0 ``` -An the run manually with `qemu-system-x86_64` as `root` (prefix with `sudo` if required): +And then run manually with `qemu-system-x86_64` as `root` (prefix with `sudo` if required): ```bash qemu-system-x86_64 \ diff --git a/content/hackathons/usw24/index.mdx b/content/hackathons/usw24/index.mdx index 6a5fd06b..b2468f6d 100644 --- a/content/hackathons/usw24/index.mdx +++ b/content/hackathons/usw24/index.mdx @@ -137,3 +137,391 @@ In case there are issues with KraftCloud / KraftKit, you can use Docker to see i Follow the steps [here](https://unikraft.org/docs/contributing/adding-to-the-app-catalog) to see how you can port a new application on top of Unikraft. Mark the items as completed [here](https://docs.google.com/spreadsheets/d/1_dOqYnHKQgkVJn0Tiudawna4d1SV_PGgS1D8oLmGD34/edit?usp=sharing). + +### Session 04: Binary Compatibility + +In the previous sessions, we managed to run some applications on top of Unikraft, with only a minimal filesystem required. +We extracted the filesystem making use of Docker and KraftKit. +We made use of the already existing kernel images from the registry, but sometimes we want to configure our kernel in a particular way, so we want to have manual control over the build process. + +In this session, we will take a look at what `kraft` does behind the scenes in order to build the Unikraft kernel image. + +To run the application that we have inside the minimal filesystem, we will use an application called [`elfloader`](https://github.com/unikraft/app-elfloader/), together with the [Unikraft core](https://github.com/unikraft/unikraft/) and some external libraries. +All of them will be cloned by `kraft`, so we don't have to worry about that. + +#### `helloworld-c` + +Let's start with the [`helloworld-c`](https://github.com/unikraft/catalog/tree/main/examples/helloworld-c) application. +We need to update the `Kraftfile`, so it build a kernel image locally, whithout pulling it directly from the registry. +You can copy the [`Nginx` Kraftfile](https://github.com/unikraft/catalog/blob/main/library/nginx/1.25/Kraftfile), change the `name:` to `helloworld` and the `cmd:` to `["/helloworld"]`. + +Let's run `kraft build` and notice what happens. +First, we will see some messages that look like this: + +```text +[+] pulling app/elfloader:staging ••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••• 100% [0.0s] +[+] finding core/unikraft:staging... done! [0.5s] +[+] finding lib/lwip:staging... done! [0.3s] +[+] finding lib/libelf:staging... done! [0.3s] +[+] pulling lib/libelf:staging ••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••• 100% [0.0s] +[+] pulling lib/lwip:staging ••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••• 100% [0.0s] +[+] pulling core/unikraft:staging ••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••• 100% [0.1s] +``` + +This tells us that `kraft` successfully cloned all the required dependencies to build the kernel. +They are placed under `.unikraft/`: + +```console +$ tree -L 1 .unikraft/ +.unikraft/ +|-- apps/ +|-- build/ +|-- libs/ +`-- unikraft/ +``` + +The default configuration will be placed by `kraft` in `.config.helloworld_qemu-x86_64`. +The final image will be placed under the `build/` directory, as shown by the output of `kraft build`: + +```text +[*] Build completed successfully! + | + |---- kernel: .unikraft/build/helloworld_qemu-x86_64 (2.8 MB) + `- initramfs: .unikraft/build/initramfs-x86_64.cpio (2.1 MB) +``` + +To tweak the configuration of the kernel, we need to add a `Makefile` and choose what we want to include in the final image. +The `Makefile` will look like this: + +```make +UK_APP ?= $(PWD)/workdir/apps/elfloader +UK_ROOT ?= $(PWD)/workdir/unikraft +UK_LIBS ?= $(PWD)/workdir/libs +UK_BUILD ?= $(PWD)/workdir/build +LIBS ?= $(UK_LIBS)/lwip:$(UK_LIBS)/libelf + +all: + @$(MAKE) -C $(UK_ROOT) A=$(UK_APP) L=$(LIBS) O=$(UK_BUILD) + +$(MAKECMDGOALS): + @$(MAKE) -C $(UK_ROOT) A=$(UK_APP) L=$(LIBS) O=$(UK_BUILD) $(MAKECMDGOALS) +``` + +All it does it call the `Makefile` from `.unikraft/unikraft/` with the right parameters, so we can just copy-paste it any time we want to configure the kernel. +To enter the configuration menu, we can run `make C=$(pwd)/.config.helloworld_qemu-x86_64 menuconfig`. +This will prompt us with a text interface that allows us to select certain features. +Let's select `Library Configuration -> ukdebug: Debugging and tracing -> Enable debug messages globally`. +We then exit by repeatedly pressing `ESC` on our keyboard. + +To build the image, we run `make C=$(pwd)/.config.helloworld_qemu-x86_64 -j$(nproc)`. +The final image will be placed under `.unikraft/build/elfloader_qemu-x86_64`. +We can run it manually, using `qemu-system-x86`, which is what `kraft` does behind the scenes. + +```console +$ qemu-system-x86_64 -cpu max -nographic -kernel .unikraft/build/elfloader_qemu-x86_64 --append "/helloworld" + +[...] +[ 0.431128] dbg: [appelfloader] brk @ 0x407821000 (brk heap region: 0x407800000-0x407a00000) +[ 0.432070] dbg: [libposix_fdio] (ssize_t) uk_syscall_r_write((int) 0x1, (const void *) 0x4078002a0, (size_t) 0xc) +Bye, World! +[ 0.433875] dbg: [libposix_process] (int) uk_syscall_r_exit_group((int) 0x0) +[ 0.434095] dbg: [libposix_process] Terminating PID 1: Self-killing TID 1... +[...] +``` + +To close the application, press `Ctrl+a`, then `x` on the keyboard. + +You can toy around with the configuration, enable different features and see how the application changes. + +#### `nginx` + +Let's move to [`nginx`](https://github.com/unikraft/catalog/tree/main/library/nginx/1.25), a more complex application. +To configure and build it, we follow the same steps. +This time we don't have to modify the `Kraftfile`, just add a `Makefile` with the same content as the one above, and run `make C=$(pwd)/.config.helloworld_qemu-x86_64 -j$(nproc)`. + +To run `nginx`, we also need to setup the networking. +Let's create a script called `run.sh`. + +```bash +# Remove previously created network interfaces. +sudo ip link set dev tap0 down +sudo ip link del dev tap0 +sudo ip link set dev virbr0 down +sudo ip link del dev virbr0 + +# Create bridge interface for QEMU networking. +sudo ip link add dev virbr0 type bridge +sudo ip address add 172.44.0.1/24 dev virbr0 +sudo ip link set dev virbr0 up + +sudo qemu-system-x86_64 \ + -kernel .unikraft/build/elfloader_qemu-x86_64 \ + -nographic \ + -m 1024M \ + -netdev bridge,id=en0,br=virbr0 -device virtio-net-pci,netdev=en0 \ + -append "netdev.ip=172.44.0.2/24:172.44.0.1::: -- /usr/bin/nginx" \ + -cpu max +``` + +You can see that we firstly remove the network interfaces, then we recreate them and run the application. +We give more memory to the application and also give an ip address. +To test that this works, we can open another terminal and run `curl 172.44.0.2`. +We close the application by pressing `Ctrl+a` then `x`. + +#### `redis` + +Follow the same steps with [`redis`](https://github.com/unikraft/catalog/tree/main/library/redis/7.2). +Create a `Makefile`, build the application and then run it. + +#### `hugo` + +Follow the same steps with [`hugo`](https://github.com/unikraft/catalog/tree/main/library/redis/0.122). +Create a `Makefile`, build the application and then run it. + +#### `redis` + +Follow the same steps with [`node/21`](https://github.com/unikraft/catalog/tree/main/library/node/21). +Create a `Makefile`, build the application and then run it. + +#### `redis` + +Follow the same steps with [`PHP`](https://github.com/unikraft/catalog/tree/main/library/php/8.2). +Create a `Makefile`, build the application and then run it. + +#### Custom Application + +Create an application of your choice in a **compiled** programming language (i.e. obtain an executable) and run it with Unikraft in binary-compatibility mode. +Use compiled programming languages such as C, C++, Go, Rust. + +#### Add System Call Tracing + +Uncomment the syscall tracing feature in the `Kraftfile` of one of the applications above: + +```text +CONFIG_LIBSYSCALL_SHIM_STRACE: 'y' +``` + +Build and run the application with Unikraft with the syscall tracing feature enabled. +See the system calls. + +Compare the system calls from the Unikraft-based run, with those from a native Linux run. +They are identical, since the application is run unmodified on Linux and on Unikraft. + +### Session 05: Debugging + +Many times, when we try to port an application and to use it on top of Unikraft, we will run into issues, as we do when we use any other platform. +Unikernels can seem harder to debug, since they function as virtual machines, but having the kernel code in the same address space as the application makes it easy to jump from the application code to the kernel code.. +In this sessions, we will look at different ways we can debug our Unikraft applications, from simple debug messages to using `gdb` to attach to the guest. + +#### Enable Debug Messages + +To enable debug messages, we will need to change the unikernel configuration, like we did in session 04. +We will use the [`nginx`](https://unikraft.org/hackathons/usw24#nginx) application from the last session. + +First, let's enter the configuration menu using `make C=$(pwd)/.config.helloworld_qemu-x86_64 menuconfig`. +There are multiple types of debugging messages we can enable. +For now, let's go for the `strace`-like output. +We do that by enabling `Library Configuration -> Syscall Shim -> Debugging -> strace-like messages`. +After that, we can rebuild the application and run it, using the script from the last session. +We should get an output similar to what `strace` will show on a usual linux setup: + +```text +close(fd:7) = OK +socketpair(0x1, 0x1, ...) = 0x0 +epoll_ctl(0x3, 0x1, ...) = 0x0 +close(fd:8) = OK +epoll_wait(0x3, 0x1000158844, ...) = 0x1 +close(fd:7) = OK +epoll_ctl(0x3, 0x1, ...) = 0x0 +gettimeofday(0x1000158980, 0x0, ...) = 0x0 +``` + +This is very useful when the application requires certain files to be present in the filesystem, and we have no way of determining that at build time. +If that is the case, we will likey see a message like: + +```text +openat(AT_FDCWD, "/etc/localtime", O_RDONLY|O_CLOEXEC) = No such file or directory (-2) +``` + +If the application crashes after that, we can assume that the file is a requirement and we can add it to the filesystem, using the `Dockerfile`. +If the application continues to run properly without that file, then most likely the file is not needed, it might be part of extra functionalities and we can choose if we want to add it or not. + +Another option is to enable all available debug messages. +To do this, enable `Library Configuration -> ukdebug -> uk_printd`. +This will show a lot of output, enabling debug messages globaly. + +You can toy around the configurations under `ukdebug` and see what they do and how they affect the printed messages. + +#### Using GDB + +Since we are running the applications using `qemu`, we can attach `gdb` and debug it like any other application. +To do that, we need to update the run script accordingly. + +Let's start with the [`helloworld` application](https://unikraft.org/hackathons/usw24#session-04-binary-compatibility) that we used in the last session. +The new run command will be: + +```console +qemu-system-x86_64 -cpu max -nographic -kernel .unikraft/build/elfloader_qemu-x86_64 --append "/helloworld" -S -s +``` + +Notice the extra `-S -s` flags. +The `-S` option will start the application in a paused state, while the `-s` will open a gdbserver on TCP port 1234. +After that, we can open another terminal and run gdb: + +```console +gdb --eval-command="target remote :1234" .unikraft/build/elfloader_qemu-x86_64.dbg +``` + +This will connect to the gdbserver, and we can go ahead and debug the application as usual. +Notice that we used the `.unikraft/build/elfloader_qemu-x86_64.dbg`, with the extra `.dbg` when we started `gdb`. +That is a non-stripped kernel image, that we can not run, but we will always use when debugging via gdb. +When debugging, instead of the usual breakpoints, use `hb` (hardware breakpoints). + +#### `nginx` with `gdb` + +Follow the same steps on the `nginx` application. +Attach gdb, toy around, place some breakpoints and see how the application flows. + +#### `redis` + +Follow the steps for debugging messages and gdb for [`redis`](https://github.com/unikraft/catalog/tree/main/library/redis/7.2). +Use the `redis` setup from the last session. + +#### `hugo` + +Follow the steps for debugging messages and gdb for [`hugo`](https://github.com/unikraft/catalog/tree/main/library/hugo/0.122). +Use the `hugo` setup from the last session. + +#### `node` + +Follow the steps for debugging messages and gdb for [`node`](https://github.com/unikraft/catalog/tree/main/library/node/21). +Use the `node` setup from the last session. + +### Session 06: Porting an Application + +Now that we have learned how to debug Unikraft applications, we can move on to porting some more complex applications, that might require more then we have seen in the third session. +The workflow for porting an application is the same as the one in the third session: create a `Dockerfile`, add a `Kraftfile`, create a minimal filesystem and run the application on top of Unikraft. +Sometimes, the minimal filesystem can not be created corectly whithout running the application, so we will make use of the debugging messages from the last session. + +Let's take `nginx` as an example. +We start from the already existing [`nginx` port](https://github.com/unikraft/catalog/blob/main/library/nginx/1.25) and we remove the `Dockerfile`, since we will write that ourselves. + +Next, we start a docker container from the `nginx` official image: + +```console +docker run --rm -it nginx:1.25.3-bookworm /bin/bash +``` + +We use `ldd` to get the dependencies: + +```console +$ ldd /usr/sbin/nginx + linux-vdso.so.1 (0x00007ffdf39e8000) + libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x000073162deb9000) + libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0 (0x000073162de1f000) + libssl.so.3 => /lib/x86_64-linux-gnu/libssl.so.3 (0x000073162dd75000) + libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3 (0x000073162d8f3000) + libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x000073162d8d4000) + libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x000073162d6f1000) + /lib64/ld-linux-x86-64.so.2 (0x000073162e133000) +``` + +Next, we create a `Dockerfile` where we copy the dependencies to a scratch image: + +```text +FROM nginx:1.25.3-bookworm AS build + +FROM scratch + +COPY --from=build /usr/sbin/nginx /usr/bin/nginx +COPY --from=build /usr/lib/nginx /usr/lib/nginx + +# Libraries +COPY --from=build /lib/x86_64-linux-gnu/libcrypt.so.1 /lib/x86_64-linux-gnu/libcrypt.so.1 +COPY --from=build /lib/x86_64-linux-gnu/libpcre2-8.so.0 /lib/x86_64-linux-gnu/libpcre2-8.so.0 +COPY --from=build /lib/x86_64-linux-gnu/libssl.so.3 /lib/x86_64-linux-gnu/libssl.so.3 +COPY --from=build /lib/x86_64-linux-gnu/libcrypto.so.3 /lib/x86_64-linux-gnu/libcrypto.so.3 +COPY --from=build /lib/x86_64-linux-gnu/libz.so.1 /lib/x86_64-linux-gnu/libz.so.1 +COPY --from=build /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libc.so.6 +COPY --from=build /lib64/ld-linux-x86-64.so.2 /lib64/ld-linux-x86-64.so.2 +COPY --from=build /etc/ld.so.cache /etc/ld.so.cache +``` + +Then, to enable debug messages, we uncomment [this line](https://github.com/unikraft/catalog/blob/main/library/nginx/1.25/Kraftfile#L112) from the `Kraftfile`. + +Now, we can run `kraft build` and `kraft run`, to see if the application works. +We will get an error message: + +```text +openat(AT_FDCWD, "/etc/nginx/nginx.conf", O_RDONLY) = No such file or directory (-2) +gettid() = pid:1 +2024/07/12 08:11:50 [emerg] 1#1: open() "/etc/nginx/nginx.conf" failed (2: No such file or directory) +``` + +This tells us that `nginx` will look for a config file. +We have a minimal config in the [`conf/`](https://github.com/unikraft/catalog/tree/main/library/nginx/1.25/conf) directory, so we copy that too, adding this line in the `Dockerfile`: + +```text +COPY ./conf/nginx.conf /etc/nginx/nginx.conf +``` + +We do `kraft build` and `kraft run` again, and we get another message: + +```text +openat(AT_FDCWD, "/etc/passwd", O_RDONLY|O_CLOEXEC) = No such file or directory (-2) +gettid() = pid:1 +2024/07/12 08:19:32 [emerg] 1#1: getpwnam("root") failed (2: No such file or directory) in /etc/nginx/nginx.conf:4 +``` + +This tells us that `getpwnam("root")` failed, because no `/etc/passwd` and `/etc/group` files are provided, so we also add those in the `Dockerfile`: + +```text +COPY --from=build /etc/passwd /etc/passwd +COPY --from=build /etc/group /etc/group +``` + +We rebuild and run again, and get: + +```text +2024/07/12 08:25:31 [emerg] 1#1: open() "/etc/nginx/mime.types" failed (2: No such file or directory) in /etc/nginx/nginx.conf:11 +``` + +So we add `/etc/nginx/` to the filesystem: + +```text +COPY --from=build /etc/nginx /etc/nginx +``` + +We repeat the same process and find out more requirements for our application: + +```text +COPY --from=build /var/cache/nginx /var/cache/nginx +COPY --from=build /var/run /var/run +COPY --from=build /usr/lib/nginx /usr/lib/nginx +COPY --from=build /var/log/nginx /var/log/nginx +``` + +After that, the application seems to run properly. +We use `kraft run -p 8080:80` and then, from another terminal, we run `curl localhost:8080` and we get a `404` page response. +This is because we also need to add an initial page for `nginx` to serve. +We have that already under `wwwroot/`, and we add that to the `Dockerfile`: + +```text +COPY ./wwwroot /wwwroot +``` + +After that, everything should work properly. + +#### `node` + +Now that you have seen how porting an application works, you can try it yourself with the [`node`](https://github.com/unikraft/catalog/tree/main/library/node/21) application. +Remove the `Dockerfile`, start from the `node:21-alpine` image and follow the same steps as above. + +#### `memcached` + +Do the same for [`memcached`](https://github.com/unikraft/catalog/blob/main/library/memcached/1.6/). +Remove the `Dockerfile` and start from `memcached:1.6.23-bookworm`. + +### Session Recordings + +You can check the recordings of the initial presentations for each session on [YouTube](https://www.youtube.com/playlist?list=PL0ZXUYDfkQ61ezmByQNLlzJ8s_dkJmQ1S). diff --git a/i18n/ui.json b/i18n/ui.json index b3232a3f..25957472 100644 --- a/i18n/ui.json +++ b/i18n/ui.json @@ -179,7 +179,7 @@ "description": "Feel free to ask questions, report issues, and meet new people.", "join-discord": "Join us on Discord!" }, - "kraftcloud-strip": { + "unikraft-cloud-strip": { "heading": "Get the most out of your application and Unikraft", "description": "The only cloud platform scaling to zero, with 'only on' pricing, and cold-starting in 20ms.", "learn-more": "Learn more" diff --git a/package.json b/package.json index 43353dd8..472be0a1 100644 --- a/package.json +++ b/package.json @@ -30,7 +30,7 @@ "@chakra-ui/utils": "2.0.14", "@codesandbox/sandpack-react": "^1.2.2", "@codesandbox/sandpack-themes": "^1.0.0", - "@docsearch/react": "^3.0.0", + "@docsearch/react": "3.6.0", "@docusaurus/utils": "2.3.0", "@emotion/react": "^11.10.0", "@emotion/styled": "^11.9.3", diff --git a/pages/blog/index.tsx b/pages/blog/index.tsx index 12864197..1b40b9a1 100644 --- a/pages/blog/index.tsx +++ b/pages/blog/index.tsx @@ -33,7 +33,7 @@ import { relativeTimeFromDates } from 'utils/to-relative-time'; import { IconType } from 'react-icons/lib' import { ReactElement } from 'react' import { DiscordCard } from 'components/discord-card' -import { KraftCloudAd } from 'components/kraftcloud-ad' +import { UnikraftCloudAd } from 'components/unikraft-cloud-ad' const BlogTags = (props: any) => { const { marginTop = 0, tags } = props @@ -320,7 +320,7 @@ function Blogs() { - + diff --git a/pages/index.tsx b/pages/index.tsx index ae1b5cec..7a35df58 100644 --- a/pages/index.tsx +++ b/pages/index.tsx @@ -1,7 +1,7 @@ import * as React from 'react' import { Footer } from 'components/footer' import LandingHero from 'components/landing-hero' -import { KraftCloudStrip } from 'components/landing/kraftcloud-strip' +import { UnikraftCloudStrip } from 'components/landing/unikraft-cloud-strip' import { BlueprintStrip } from 'components/landing/blueprint-strip' import { KraftKitStrip } from 'components/landing/kraftkit-strip' import { CatalogStrip } from 'components/landing/catalog-strip' @@ -20,7 +20,7 @@ const HomePage = () => { />
- + diff --git a/public/images/uefi-gop-after-scrolling.png b/public/images/uefi-gop-after-scrolling.png new file mode 100644 index 00000000..ea69227e Binary files /dev/null and b/public/images/uefi-gop-after-scrolling.png differ diff --git a/public/images/uefi-gop-before-scrolling.png b/public/images/uefi-gop-before-scrolling.png new file mode 100644 index 00000000..65483fee Binary files /dev/null and b/public/images/uefi-gop-before-scrolling.png differ diff --git a/public/images/uefi-gop-first-text-render.png b/public/images/uefi-gop-first-text-render.png new file mode 100644 index 00000000..44942fff Binary files /dev/null and b/public/images/uefi-gop-first-text-render.png differ diff --git a/public/images/uefi-gop-scaled-text-1.png b/public/images/uefi-gop-scaled-text-1.png new file mode 100644 index 00000000..7b4fec83 Binary files /dev/null and b/public/images/uefi-gop-scaled-text-1.png differ diff --git a/public/images/uefi-gop-scaled-text-2.png b/public/images/uefi-gop-scaled-text-2.png new file mode 100644 index 00000000..496c45bc Binary files /dev/null and b/public/images/uefi-gop-scaled-text-2.png differ diff --git a/public/images/uefi-gop-with-ukconsole.png b/public/images/uefi-gop-with-ukconsole.png new file mode 100644 index 00000000..d7ce42a2 Binary files /dev/null and b/public/images/uefi-gop-with-ukconsole.png differ diff --git a/src/components/landing/catalog-strip.tsx b/src/components/landing/catalog-strip.tsx index 9d832b4d..29f9a505 100644 --- a/src/components/landing/catalog-strip.tsx +++ b/src/components/landing/catalog-strip.tsx @@ -235,7 +235,7 @@ export function CatalogStrip(props: BoxProps) { */} diff --git a/src/components/landing/kraftcloud-strip.tsx b/src/components/landing/unikraft-cloud-strip.tsx similarity index 82% rename from src/components/landing/kraftcloud-strip.tsx rename to src/components/landing/unikraft-cloud-strip.tsx index 10882377..864318a9 100644 --- a/src/components/landing/kraftcloud-strip.tsx +++ b/src/components/landing/unikraft-cloud-strip.tsx @@ -6,11 +6,11 @@ import { BoxProps, chakra, } from '@chakra-ui/react'; -import { CompanyKraftCloud } from 'components/logos' +import { CompanyUnikraftCloud } from 'components/logos' import Container from 'components/container' import { t } from 'utils/i18n' -export function KraftCloudStrip(props: BoxProps) { +export function UnikraftCloudStrip(props: BoxProps) { return ( - + - {t('component.kraftcloud-strip.heading')} + {t('component.unikraft-cloud-strip.heading')} - {t('component.kraftcloud-strip.description')} + {t('component.unikraft-cloud-strip.description')} @@ -60,7 +60,7 @@ export function KraftCloudStrip(props: BoxProps) { justifyContent='center' display='inline-flex' alignItems='center' - href='https://kraft.cloud' + href='https://unikraft.cloud' rel='noopener' target='_blank' fontWeight='bold' @@ -74,7 +74,7 @@ export function KraftCloudStrip(props: BoxProps) { transition='all 0.1s ease-in-out' _hover={{ textDecoration: 'none', translateY: '-2px', shadow: 'lg' }} > - {t('component.kraftcloud-strip.learn-more')} + {t('component.unikraft-cloud-strip.learn-more')} diff --git a/src/components/logos/company-kraftcloud.svg b/src/components/logos/company-kraftcloud.svg deleted file mode 100644 index 04d23d43..00000000 --- a/src/components/logos/company-kraftcloud.svg +++ /dev/null @@ -1,29 +0,0 @@ - - - - - - - - - - - - - - - - - diff --git a/src/components/logos/company-unikraft-cloud.svg b/src/components/logos/company-unikraft-cloud.svg new file mode 100644 index 00000000..cbcbec4b --- /dev/null +++ b/src/components/logos/company-unikraft-cloud.svg @@ -0,0 +1,40 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/src/components/logos/index.ts b/src/components/logos/index.ts index 78ddda3c..200bb73c 100644 --- a/src/components/logos/index.ts +++ b/src/components/logos/index.ts @@ -11,7 +11,7 @@ export { default as CompanyNEC } from './company-nec.svg'; export { default as CompanyOpensynergy } from './company-opensynergy.svg'; export { default as CompanyOvhcloud } from './company-ovhcloud.svg'; export { default as CompanyOvhcloudSm } from './company-ovhcloud-sm.svg'; -export { default as CompanyKraftCloud } from './company-kraftcloud.svg'; +export { default as CompanyUnikraftCloud } from './company-unikraft-cloud.svg'; export { default as LinuxFoundation } from './linux-foundation.svg'; export { default as PoweredByUnikraftInverse } from './powered-by-unikraft-inverse.svg'; export { default as UnicoreProject } from './unicore-project.svg'; diff --git a/src/components/mdx-components/mdx-components.tsx b/src/components/mdx-components/mdx-components.tsx index 815c22ce..44f0d5e7 100644 --- a/src/components/mdx-components/mdx-components.tsx +++ b/src/components/mdx-components/mdx-components.tsx @@ -11,7 +11,7 @@ import CodeBlock from './codeblock/codeblock' import ComponentLinks from './component-links' import { VideoPlayer } from './video-player' import AsciinemaPlayer from 'components/asciinema-player' -import { KraftCloudBanner } from './kraftcloud-banner' +import { UnikraftCloudBanner } from './unikraft-cloud-banner' const { Accordion, @@ -180,7 +180,7 @@ export const MDXComponents = { LanguageLinks, VideoPlayer, AspectRatio, - KraftCloudBanner, + UnikraftCloudBanner, JoinCommunityCards, FaqList: ({children, ...props}) => { return ( diff --git a/src/components/mdx-components/kraftcloud-banner.tsx b/src/components/mdx-components/unikraft-cloud-banner.tsx similarity index 79% rename from src/components/mdx-components/kraftcloud-banner.tsx rename to src/components/mdx-components/unikraft-cloud-banner.tsx index bc2e88e1..28ef8786 100644 --- a/src/components/mdx-components/kraftcloud-banner.tsx +++ b/src/components/mdx-components/unikraft-cloud-banner.tsx @@ -1,9 +1,9 @@ import * as React from 'react'; import { Box, Heading, Flex, chakra } from '@chakra-ui/react'; -import { CompanyKraftCloud } from 'components/logos'; +import { CompanyUnikraftCloud } from 'components/logos'; import { t } from 'utils/i18n'; -export const KraftCloudBanner = ({ title, url, children, ...rest }: { +export const UnikraftCloudBanner = ({ title, url, children, ...rest }: { title: { string } url: { string } children: React.ReactNode @@ -29,7 +29,7 @@ export const KraftCloudBanner = ({ title, url, children, ...rest }: { alignItems='start' > - + - {t('component.kraftcloud-strip.learn-more')} + {t('component.unikraft-cloud-strip.learn-more')} - {title || t('component.kraftcloud-strip.heading')} + {title || t('component.unikraft-cloud-strip.heading')} - {children || t('component.kraftcloud-strip.description')} + {children || t('component.unikraft-cloud-strip.description')} ) diff --git a/src/components/kraftcloud-ad.tsx b/src/components/unikraft-cloud-ad.tsx similarity index 81% rename from src/components/kraftcloud-ad.tsx rename to src/components/unikraft-cloud-ad.tsx index b64f43a9..a8846f71 100644 --- a/src/components/kraftcloud-ad.tsx +++ b/src/components/unikraft-cloud-ad.tsx @@ -1,9 +1,9 @@ import * as React from 'react'; import { Box, VStack, Heading, Flex, chakra } from '@chakra-ui/react'; -import { CompanyKraftCloud } from 'components/logos'; +import { CompanyUnikraftCloud } from 'components/logos'; import { t } from 'utils/i18n'; -export const KraftCloudAd = () => { +export const UnikraftCloudAd = () => { return ( { w={'100%'} > - + - {t('component.kraftcloud-strip.heading')} + {t('component.unikraft-cloud-strip.heading')} - {t('component.kraftcloud-strip.description')} + {t('component.unikraft-cloud-strip.description')} { transition='all 0.1s ease-in-out' _hover={{ textDecoration: 'none', translateY: '-2px', shadow: 'lg' }} > - {t('component.kraftcloud-strip.learn-more')} + {t('component.unikraft-cloud-strip.learn-more')} )