-
Notifications
You must be signed in to change notification settings - Fork 107
Porting UFS (using Spatch)
Harvey needs a file system, and after investigating various file systems, we decided to port the FreeBSD implementation of UFS2 to Harvey.
This is not an easy task - it involves transplanting a major system from one operating system to another. Once the port is complete, we want to ensure we can incorporate any improvements to the FreeBSD implementation with minimal effort.
To achieve this, it was suggested we use spatch to create semantic patches. These patches are describe patterns of code that can be found across a code base and then modified as described in the patch. Where spatches are not possible, we can use sed, or if necessary, modify the code by hand.
The process we intend to follow is:
- Fork FreeBSD. (Done)
- Copy the ufs code into a new library (libufs). (Done)
- Add an initially empty build file to allow us to build the library as a kernel library in Harvey. (Done)
- Develop simple spatches that can be applied to the code to slowly transform it into something that can be built. Combine with hand editing and commenting out code where spatches are less useful. (In Progress)
- Implement a driver to integrate libufs with Harvey.
Spatch sometimes has problems parsing code with macros. An example is the use of the TAILQ_ENTRY() macro in the inode struct in ufs/inode.h. If spatch cannot parse this entry in the struct, it cannot transform any part of the struct. We can resolve this by adding the definition of the macro to a macro file, and pass this to spatch when applying the patches. We will build up this macro file as we go, from the actual macros in FreeBSD.
You can check that a file can be fully parsed by spatch as so: spatch --parse-c ../ufs/acl.h. This will report whether spatch can fully parse the file or not.
Snapshots can be made of the filesystem at the current time. These can then be mounted as frozen images. There is a limit of 20 snapshots, linked to the superblock.
Soft updates ensure files system metadata remains consistent in the case of a crash by strictly ordering operations. I've seen some doubts about whether this is compatible with the disk writeback cache, with some people suggesting that the cache needs to be disabled, with a noticeable performance impact.
Journalled soft updates were added to FreeBSD 9. This extension prevents the inconsistencies not covered by soft updates, and avoids the need for a background fsck after a crash. If journalled soft updates are enabled on the filesystem, then snapshots are not supported.
gjournal is another journalling method, but it seems to be implemented in the GEOM layer, outside UFS. If so, it won't be supported initially, and probably not at all.
Quotas is another UFS feature implemented in FreeBSD. We won't be supporting it in the initial version.
Devufs will be the kernel device implemented over libufs. First pass will be a blank device (e.g. based on devregress) that errors for every message. Next comes the init and reset message implementations. Eventually it will have to manage multiple UFS filesystems, but we can focus on handling a single filesystem initially.
When we first bind, it will read in the superblock, and perform any additional initialisation.
Assuming we name it #U as the device name for now. At the very least we need to be able to mount, unmount and access UFS filesystems.
To mount a partition as a UFS filesystem: echo #S/sdE0/whatever > #U/mount
To mount an arbitrary file as a UFS filesystem: echo path/to/some/file > #U/mount
In both cases, a new directory will be created for the mounted filesystem with a numeric name.
Assuming a mounted filesystem has been given the name '0', it can be unmounted as so: echo unmount > #U/0/ctl.
Assuming a mounted filesystem has been given the name '3', the directory corresponding to the root can be accessed as so: ls #U/3/root.