We are excited to announce the update version of Unikraft, v0.16.0 (Telesto).
This release is the result of constant testing, debugging, fixing and improvements during the past three months. It comes packed with plenty of fixes that improve the stability and application support of Unikraft as well as new features we're excited to show off.
For a full breakdown, please check out the full changelog.
This feature was championed by Andrei Tătar, with great support from Simon Kuenzer.
Unikraft's vfscore, a key element of the Unikraft project since its inception, has historically excelled in virtualizing file systems but faced limitations in terms of modularity and addressing new constraints made possible via the unikernel model.
Acknowledging these challenges, the Unikraft team has completely redesigned and reimplemented the virtual filesystem abstractions which has previously represented vfscore.
The goal is to enhance modularity, scalability, and overall adaptability, addressing existing limitations and ensuring Unikraft remains a pioneering force in lightweight, specialized operating systems.
This effort aims to create a more flexible and extensible VFS system, forming a solid foundation for continued innovation within the Unikraft ecosystem.
These new-style VFS changes are fully compatible with existing KConfig and Unikraft internal APIs.
To address the challenge of modularity, vfscore has been radically re-imagined and broken down into more primitive interfaces representing individually bespoke microlibraries which are testament to the Unikraft philosophy of "one library one purpose" model.
Such compensation facilitates the construction of even more bespoke unikernel images.
These new libraries are:
| New Library | PR | Purpose |
|---|---|---|
ukfile | #1165 | A new low-level abstraction for "file descriptors" and "open file descriptors". |
posix-fdio | #1170 | Offers POSIX-compatible file I/O, metadata, and control functions for files and the POSIX syscalls for (p)read(v)*, (p)write(v)*, lseek, fstat, fcntl, ioctl. |
posix-fdtab | #1168 | A file descriptor table and POSIX compatibility for Unix file types (and registration macros for new-style file types) as well as the POSIX syscalls for close, dup, dup2 and dup3. |
posix-poll | #1171 | Support for polling files and POSIX syscalls for (p)poll, (p)select(6) and epoll_*. |
posix-eventfd | #1171 | Support for event file descriptors, eventfd files, and the POSIX syscalls for eventfd and eventfd. |
posix-timerfd | #1179 | Support for timer file descriptors, timerfd files, and the POSIX syscalls for timerfd_create, timerfd_settime and timerfd_gettime. |
posix-pipe | #1172 | Support for pipe files and the POSIX syscalls for pipe and pipe2. |
To understand the challenge of modularity we must start with an introduction of a new fundamental internal library, ukfile, which contains basic primitives for Unikraft abstractions of a "file" as well as an "open file" (a.k.a. "open file descriptions").
These are low-level internals which do not have direct correspondents in any userspace-facing API.
Not to be confused with "file descriptors" or other similar POSIX-y concepts; these are later introduced via posix-fd* and others.
To overuse a classical *NIX idiom, "everything is a file". In the context of Unikraft, however, a file is considered an abstraction for any resource that offers a combination of input, output, and/or control operations. A file in Unikraft is a combination of an immutable identity coupled with mutable state.
Files are now represented in Unikraft by the struct uk_file type, referenced in APIs as const struct uk_file * to enforce immutability. Identity consists of:
File state is used for bookkeeping purposes and includes: reference counting (strong & weak references); locks for synchronization; and, event set and queue for polling operations.
Files allow for a defined set of operations, some of which are driver-implemented, while others are common across all files.
Driver-specific operations have a well-defined interface and are implemented by file drivers. These are:
read and write.getstat and setstat.ctl.Common operations are implemented centrally for all file objects:
Open files are stateful and mutable references to a file that is "in use". The precise definition of "in use" is intentionally left vague and up to client code. Open file state consists of:
lseek())Open files are represented in Unikraft by the struct uk_ofile type. A single struct uk_file may be referenced by an arbitrary number of ofiles, each of which acts independently from the others.
Open files do not expose any operations themselves, instead providing only a base data structure for higher-level abstractions, such as file descriptor tables and POSIX I/O syscalls.
This feature was championed by Simon Kuenzer with support and feedback from Sergiu Moga and Alexander Jung.
In conjunction with the embedded initrd support, this feature allows a secondary filesystem to add and potentially overwrite items in the primary (root) filesystem.
This is implemented by passing the extract option to the vfstab command line
argument, e.g.:
vfs.fstab="initrd:/:initrd::extract:"
With this in place, user can customize their own application on top of an existing application / language frameworks. A user would pull the base application from the registry and then run it with a secondary filesystem, that adds / overwrites required files.
This feature was championed by Simon Kuenzer with great support from Sergiu Moga.
This release introduces support for safely and gracefully shutting down Unikraft via the introduction of system-wide destructors via an extension to inittab.
Up until now, Unikraft VMs were left in a no-operation state, but didn't fully stop, when a shutdown signal received.
The inittab serves as a section of entries which are executed at different lifecycle moments in Unikraft's boot process and offers a convenient callback mechanism.
Each entry is extended to two function pointers: an initialization function pointer and a termination function pointer, which are handled in pairs and share the same priority.
While initialization means iterating over the inittab and calling the initialization functions, termination is done by iterating over the inittab in reverse order.
Initialization and termination are handled by the "init" thread of ukboot.
With scheduling, there is an option to start the main() of the application in a separate "main" thread.
This allows shutdown requests to be handled independently of the state of main().
A shutdown request driver (e.g., ACPI button) or any other application thread can send a request to the "init" thread to initiate a shutdown.
ukgcov) (#906)#This feature was championed by Eduard Mihăilescu and Michalis Pappas.
This ukgcov library introduces initial setup with config options for console, binary file, and memory output.
It takes advantage of GCC's Profiling and Test Coverage in Freestanding Environments.
This feature was introduced in GCC-13, as such, this library won't work with any version lower than that.
To utilize this library, you will need to select and configure some options:
ukgcov from Library Configuration KConfig menu.gcov information from the kernel.To extract the coverage file, you will need external filesystem support (9pfs):
plat -> kvm -> virtio, select:
virtio PCI device supportvirtio 9P devicelib -> vfscore, select:
Default root filesystem (9PFS)fs0/ where the resulting filename with the name provided in KConfig will reside.To extract the coverage information, you can call the gcov_dump_info() function at the end of the program
This feature was championed by Alexander Jung and Razvan Deaconescu with feedback from Ștefan Jumărea.
A new repository for building, running and deploying application is available, as the catalog repository.
This is a replacement for the application repositories (app-*) and for the repositories for binary compatible applications (static-pie-apps, dynamic-apps).
Applications and examples are to be aggregated in the catalog.
The user will use kraft commands to build, run and deploy applications, without other steps required.
The catalog will be the focus of CI-based testing and, generally, any testing of submitted features and fixes in the Unikraft core and library repositories.
This feature was championed by Andrei Tatar with great support from Simon Kuenzer.
This feature adds UNIX socket support (i.e. local domain sockets), up until now not-supported in Unikraft.
It adds posix-unixsocket, a library implementing AF_UNIX sockets on top of posix-socket using new VFS design, along with prerequisite changes:
posix-socket<sys/un.h> in nolibcThis work is part of the VFS redesign work, relying on the newly introduced ukfile, posix-fdio, posix-fdtab libraries.
This feature was championed by Cezar Crăciunoiu with greate support from Alexander Jung.
New GitHub workflows have been added to the Unikraft core repository. These perform various checks on submitted work to ensure consistency and compliance. They affect C, YAML, Python, shell code and Git commits.
This work was championed by Rareș Miculescu and Michalis Pappas.
The plat re-arch track continued with three new items:
plat/ to arch/ (#937)ukatomic library and move atomic code to it (#950essentials.h to compiler.h (#960)These further assist in the overall improved design of the Unikraft source code and a better separation between platform and architecture code.
This feature was championed by Ștefan Jumărea, Radu Nichita, Delia Pavel with support from Răzvan Deaconescu.
Further updates to the internal documentation, in the form of code comments and library README files, have been integrated.
This is part of the ongoing effort to have up-to-date documentation of the interfaces and inner workings of Unikraft internal libraries.
lib-libgo#9)#This feature was championed by Eduard Vintilă with support from Michalis Pappas and Ștefan Jumărea.
Previously updated Go support was only available for x86_64. This feature enables it for ARM64.
This now makes it possible to run native Go applications on ARM64 with lib-libgo.
This feature was championed by Sergiu Moga with support from Michalis Pappas.
Before, whenever a system call would be invoked through the syscall instruction on x86_64, Unikraft would continue using the application's stack to save context state, which would end up sometimes contaminating the Red Zone of the trapped application or even write past it in cases such as those of Goroutines that have a very small stack size.
In order to solve this, a per-thread and per-LCPU (logical CPU) auxiliary stack pointer has been introduced that we switch to on binary system call entry through the swapgs instruction.
Furthermore, ARM64 was suffering from the same thing, but worse: on every Asynchronous or Synchronous Exception, we were using the same application stack, since ARM64 does not have Interrupt Stack Tables like x86_64.
To fix this, similarly to how we store LCPU data into the GS_BASE/KERNEL_GSB_BASE MSR's on x86_64, we use TPIDR_EL1 to store this structure on ARM64 and TPIDRRO_EL0 as a scratch register used to switch to the auxiliary stack without touching anything from the current context.
The auxiliary stack pointer is mainly used for system calls, but for IRQ's/traps the unused SP_EL0 register will be used to hold a double buffer: a stack for IRQ's and a stack for traps.
clone(5) and UK_LLSYSCALL_R_U_DEFINE#This feature was championed by Sergiu Moga with support from Michalis Pappas.
On some rare cases, some application runtimes would prefer that after a clone system call the child's context will be the exact same as the parent's before the call, except of course for the scratch registers or the return value registers.
In our case we would not do this and instead have the child's context zeroed out.
Therefore, we would need to give access to the caller's context to both native and binary system calls.
To make tracking of this context across system calls easier, two new context structures have been added: ukarch_sysregs (TLS and, for x86_64, GS_BASE) and uk_syscall_ctx (for ukarch_sysregs, extended context and general purpose registers).
While in the case of binary system calls we had the generic syscall instruction entry, for the native ones system calls are just a function call so in order to also be able to store this context on native system calls, a new system call registration macro has been introduced: UK_LLSYSCALL_R_U_DEFINE.
This new macro would append a context saving prologue for both x86_64 and ARM64 to the actual function call of native system calls as well as define a hidden additional argument, usc (struct uk_syscall_ctx), to registered system calls.
An example on how this system call registration macro can be used is the new clone system call registration, as well as the documentation in the commit message itself.
Unikraft was part of GSoC Mentors Summit 2023 taking place in Sunnyvale, California, US. Răzvan Deaconescu represented the Unikraft community and its taking part in Google Summer of Code during the past two years.
It was a great time to interact with representatives from other open source projects and share ideas and experiences related to Google Summer of Code and beyond.
As part of the un-conference at the summit, Răzvan hosted the "Balancing Acts in Open Source Projects", engaging the participants in sharing their own views on potentially conflicting actions in open source communities, and best practicies in handling them.
There is now an ongoing discussion on Zulip channel to further strengthen the collaboration between open source communities. And to increase the quality of future GSoC projects.
We organized the Unikraft SOSP Tutorial hackathon as part of the SOSP 2023 conference. The tutorial was focused on recent updates to the Unikraft ecosystem, particularly application support and usability via KraftKit.
Participants were mostly researchers and academia staff, interested in using Unikraft as part of their research and teaching work.
We continue organizing monthly hackathons, generally taking place in the last Saturday of each month. Hackathons take place both online and in person, generally at the National University of Science and Technology POLITEHNICA of Bucharest.
We organized a monthly hackathon on Sunday, December 17, 2023.
It was targeted on the 0.16 release, application support, CI improvements and fixes.
It featured an extensive participation of 16 members of the community.
The next monthly hackathon is scheduled to take place on Saturday, January 27, 2024, ahead of a scheduled new release.
As part of the Unikernel Alliance community, we now have a brand new website, created from a repository. The website presents information about member projects in the community, topics of interest and further information.
We continue organizing meetings in the community, generally taking place in the last Thursday of each month. The next meeting is scheduled for Thursday, January 25, 2024, 6pm CET.
Current discussions focus on unikernel specifics and challenges in topics such as usability, performance, security, efficiency.
Unikraft will be part of FOSDEM'24, taking place on February 3 & 4, 2024, at the Microkernel and Component-based OS devroom. There will be two talks featured:
It will be a great time to meet in person with other members of the unikernel community and the larger open source community.
See you in Brussels!
Google Summer of Code 2024 (GSoC'24) has been announced. We are making preparations for the future GSoC'24 submission period. The aim is to get future applicants involved in the community such that they would be able to make consistent contributions as part of the application portfolio.
If you are interested in taking part in GSoC'24, and work on operating-systems, cloud and tooling items, join Unikraft Discord, take part in discussions, make contributions and ask for support.
Feel free to ask questions, report issues, and meet new people.