Skip to content

Latest commit

 

History

History
387 lines (316 loc) · 15.6 KB

README.md

File metadata and controls

387 lines (316 loc) · 15.6 KB

LensorOS

A 64-bit operating system with everything built from scratch!

Table of Contents

Booting into LensorOS using QEMU

Get QEMU

Before beginning, ensure you have the LensorOS bootloader and kernel binaries.
See the build section.

To change the font, replace dfltfont.psf in the kernel/res folder with any PSF1 font (not PSF2).
For a few fonts that are compatible, check out this repository.

Ensure the following dependencies are installed on your system:

  • Linux: sudo apt install mtools
  • Windows

GNU mtools is a set of tools for manipulating MS-DOS style files and filesystems. This is necessary as the UEFI specification requires FAT32 to be used as it's boot device's filesystem.

To build a disk image that will boot into LensorOS, run the following included helper script:

bash /Path/to/LensorOS/kernel/mkimg.sh

Upon completion, this will have generated a disk image file LensorOS.img that follows the UEFI standards, meaning any UEFI-supporting machine could boot from this image, given it is a valid boot device.

Alternatively, Create a disk image file that is GPT formatted (aka supports partitions) by utilizing the mkgpt.sh script that is included. See the scripts README for more info.

On Linux

To run QEMU with the correct command line options automatically, use the following helper script to launch QEMU booting into LensorOS:

bash /Path/to/LensorOS/kernel/run.sh

For debugging with gdb, run bash rundbg.sh instead. This will launch QEMU but wait to start cpu execution until gdb has connected on port 1234 of localhost.
See a note about debugging.

On Windows

To launch QEMU from the generated disk image, A run.bat file is included.
The batch file requires the directory that the QEMU executable resides in be added to the system's PATH variable. See this stackoverflow thread for help.
If editing the PATH variable isn't a valid option, the batch script could always be edited to use the path to the QEMU executable on your local machine. \

By simply double clicking this batch file, QEMU will open and boot into LensorOS.

All serial output will be re-directed to stdin/stdout, which is likely a cmd.exe window that opened along with QEMU. The problem with cmd.exe is that it leaves the output looking rather mangled (ie. left-pointing arrows, open square-brackets, etc). This is due to the terminal not supporting ANSI color codes.

To get around this, I use the (rather life-changing) Windows Terminal. It should be the default terminal, and should have been for five years now, but I'm glad it is available and open source none-the-less.

This new terminal allows both WSL and PowerShell to be open in the same terminal, but separate tabs. By the run.bat file from within a PowerShell in the new Windows Terminal, you will experience glorious full-color, formatted serial output.

If you are having none of this, and would prefer to have a very monotone serial output that is also not mangled with ANSI color codes, define LENSOR_OS_UART_HIDE_COLOR_CODES during compilation and all color codes will be hidden from serial output by the kernel itself.

There is also a rundbg.bat that will launch QEMU with the appropriate flags to wait for gdb to connect on port 1234 of localhost.
NOTE: When debugging with gdb, the kernel must be built with debug symbols ("-g" compile flag). To achieve this, run cmake with the following command line argument: -DCMAKE_BUILD_TYPE=Debug

Booting into LensorOS using VirtualBox

Get VirtualBox

VirtualBox is less lenient when it comes to virtual drive file formats.
Because of this fact, VirtualBox can not use the .img file raw like QEMU, and either a .bin GPT formatted disk image with an EFI System partition or .iso file with an ISO filesystem must be prepared from it first.

Ensure you have the LensorOS bootloader and kernel binaries built.
See the build section.

To change the font, replace dfltfont.psf in the kernel/res folder with any PSF1 font (not PSF2).
For a few fonts that are compatible, check out this repository, or even GNU Unifont.

There are two possible pathways that LensorOS bootable media can be generated.

GPT

Utilize the mkgpt.sh script that is included in the /scripts/ directory. See the scripts README for more information on this script.

If all has went well, there will be a LensorOS.img and a LensorOS.bin in the /kernel/bin/ directory of the repository. While this binary .bin file is a perfectly valid image of a real GPT formatted disk, VirtualBox does not accept it as valid. When VirtualBox is installed, it also installs a lot of command line tools, one of which is called VBoxManage. This tool has a subcommand convertfromraw that we will utilize to create a .vdi virtual disk image from our .bin binary disk image.

VBoxManage convertfromraw /Path/to/LensorOS/kernel/bin/LensorOS.bin /Path/to/LensorOS/kernel/bin/LensorOS.vdi --format VDI

Continue to the VirtualBox VM Configuration, and replace any mention of Optical Drive with Hard Disk, and any mention of LensorOS.iso with LensorOS.vdi.

Live CD

NOTE: On Windows, complete the following shell commands from within WSL, or the Windows Subsystem for Linux.

Install the tool necessary to create .iso files and ISO-9660 filesystems, as well as a tool to create and format MS-DOS style filesystems:

sudo apt install mtools xorriso

Next, simply run the mkiso.sh script with Bash:

bash /Path/to/LensorOS/kernel/mkiso.sh

If all goes well, this will first generate a FAT32 EFI-compatible boot disk image, then create a bootable ISO-9660 CD-ROM disk image.

Virtual Machine Configuration

To actually use the generated bootable media in a VM in VirtualBox, it requires some setup:

  1. Open VirtualBox.
  2. Click the New button to create a new virtual machine (VM).
  3. Give the VM a name and a file path you are comfortable with.
  4. Select Type of Other and Version of Other/Unknown (64-bit).
  5. Leave the memory size how it is; 64MB is plenty at this time.
  6. Select the Do not add a virtual hard disk option.
  7. Click the Create button to create the new virtual machine.
  8. Select the new VM in the list on the left, then click the Settings button.
  9. Navigate to System within the list on the left.
    1. Change Chipset to ICH9.
    2. Enable Extended Feature Enable EFI (special OSes only).
    3. In the Processor tab, check the Enable Nested VT-x/AMD-V checkbox.
  10. Navigate to Storage within the list on the left.
    1. Right click the storage controller (IDE), and select Optical Drive.
    2. Click Add in the new Optical Disk Selector window that pops up.
    3. Browse to Path/To/LensorOS/kernel/bin/ and select LensorOS.iso.
    4. Ensure LensorOS.iso is selected, then click the Choose button.
  11. Navigate to Network within the list on the left.
    1. Disable all network adapters.

After all of this has been done, you are ready to click Start on the VirtualBox VM; the bootloader should run automatically.

Booting into LensorOS on hardware

DISCLAIMER: LensorOS IS IN NO WAY GUARANTEED TO BE 'SAFE'; RUN AT YOUR OWN RISK! (see LICENSE)

Before beginning, ensure you have the LensorOS bootloader and kernel binaries.
See the build section.

Ensure you have a USB storage device that is working. Remove all data from the USB. This can be done by formatting, or simply deleting/moving everything off of it.

Next, create a folder called EFI, then inside that a folder called BOOT.
Move main.efi from /gnu-efi/x86_64/bootloader/ directory into the BOOT folder on the USB.
Rename main.efi in the usb's BOOT folder to bootx64.efi, as per the UEFI specification.

Finally, navigate back to the root directory of the USB (where the EFI folder resides).
Create a folder named LensorOS, then move the following resources into the directory:

  • kernel.elf from /kernel/bin/
  • Any .psf version 1 font renamed to dfltfont.psf
    • A font comes included; it can be found at kernel/res/dfltfont.psf.
    • Many fonts can be found here.
    • GNU Unifont provides PSF1.
Final Directory Structure:
USB
|-- efi
|   `-- boot
|       `-- bootx64.efi
|-- LensorOS
|   |-- kernel.elf
|   `-- dfltfont.psf
`-- startup.nsh

Building LensorOS

There are three steps to building LensorOS:

  1. the Toolchain
  2. the Bootloader
  3. the Kernel

On Windows, use the Windows Subsystem for Linux (WSL) to emulate Linux exactly, no drawbacks.
See this walkthrough on how easy it is to communicate between Linux and Windows processes.

If you don't want to use WSL, you'll need pre-built binaries of the LensorOS Toolchain, the LensorOS bootloader, and GNU mtools; ask and we will help you on your journey :^).

Keep in mind the bootloader can't be built natively on Windows (yet), so skip straight to step #3 if not using WSL.

Alternatively, one could use a virtual machine that emulates a linux distro (ie. VirtualBox running Linux Mint, or something), and develop from there.

Alternatively alternatively, one could use Cygwin for a Unix-like environment on native Windows, but I am not knowledgable on this topic.

Linux

If you are on Windows 10+, you are able to use WSL to complete the following Linux steps as-is.

Get dependencies:

sudo apt install build-essential git make nasm

You will need CMake version 3.20 or greater, which is why we didn't install it through apt; most distros only have up to version 3.16. Check your current installation of CMake's version using cmake --version. See the CMake downloads page for pre-built binaries. Simply download the latest release for Linux and your machine type, then (after extracting it), add the bin subdirectory to the PATH environment variable. CMake are nice guys and even provide self extracting options. If you do not see your machine type, you must download the source and build CMake from it.

Obtain the source code:

git clone https://github.com/LensPlaysGames/LensorOS

1. Build The LensorOS Toolchain

Follow the instructions in the toolchain README

2. Build the bootloader

The bootloader source code resides in the gnu-efi directory, for now.

cd /Path/to/LensorOS/gnu-efi/
make
make bootloader

This will generate main.efi in the /gnu-efi/x86_64/bootloader/ directory within the repository. This is the UEFI compatible executable file.

NOTE: One only needs to run make for the bootloader once; it generates libgnuefi.a, which the bootloader itself links with. Following that, simply using the bootloader target will be sufficient to update the bootloader; even that only needs to be done if the bootloader code was changed.

3. Build the kernel

The LensorOS Kernel uses CMake to generate the build system; beware, as not all the build systems CMake can generate honor the request to use the LensorOS Toolchain (ahem Visual Studio ahem).
To prepare a build system that will build the kernel with GNU make, run the following:

cd /Path/to/LensorOS/kernel/
cmake -G "Unix Makefiles" -S . -B rls -DCMAKE_BUILD_TYPE=Release
cmake -G "Unix Makefiles" -S . -B rls -DCMAKE_BUILD_TYPE=Release

Yes, I always recommend generating the build system twice in a row (weird bugs may appear otherwise).

The above command should generate an out-of-source GNU make build system in the rls subdirectory. Alternatively, generate any build system of your choice that is supported by CMake (I recommend Ninja, it's fast).

To build the kernel, invoke the build system that was generated by CMake. No matter the build system you choose, invoke it using the following command:

cmake --build /Path/to/LensorOS/kernel/rls

NOTE: It's not necessary to provide an absolute path, but it means the command can be run from any working directory.

This final build step will generate kernel.elf within the /kernel/bin directory, ready to be used in a boot usb or formatted into an image.
See the sections starting with "Booting into LensorOS" above.

If building the kernel for real hardware, ensure to set the MACHINE CMake variable within config.cmake to PC, and not any of the virtual machines (ie. QEMU, VBOX). This allows for the hardware timers to be used to their full potential (asking QEMU for 1000hz interrupts from multiple devices overloads the emulator and guest time falls behind drastically; to counter-act this, very slow frequency periodic interrupts are setup as to allow the emulator to process them accordingly, allowing for accurate time-keeping in QEMU).

For faster build times, and better incremental building, download ccache and add the directory in which it's executable resides in to the PATH system environment variable. The kernel CMake script will detect and use it automatically.

Acknowledgements

At first, development followed tutorials that can be found at Poncho's GitHub.
Those tutorials were abandoned just after setting up a very basic AHCI driver, so I've taken the wheels from there.

A large thanks to the huge portions of inspiration, knowledge, and help that came from the OSDev wiki and forums (sorry, Terry).

A huge amount of entertainment and inspiration has come from SerenityOS, an operating system being built by the OS development hobbyist community.

The compeletely-from-scratch ToaruOS has also been an amazing source of knowledge, inspiration, and information.

The FreeBSD and Linux codebases have been a huge help in seeing how hard-to-solve problems have been solved in the past.

LensorOS Birthday

My birthday is January 14th! I am still a wittle baby.