We are thrilled to announce the new edition of Unikraft, v0.11.0 (Janus)!
This release is the result of around 3 months of hard work in the entire community, with a focus on integrating the long awaited musl support.
In this blog post, we highlight just some of the new features available in Unikraft. For a full breakdown, please check out the changelog.
New Scheduling API
This feature was championed by Simon Kuenzer with extensive support from Răzvan Deaconescu, Adina Smeu, Dragoș Argint, Cezar Crăciunoiu and Florin Postolache.
We are releasing a revised
uksched API. The driving motivation for the design was to remove unneeded functionality, simplification, and introducing the concept of separating thread creation from scheduling. Threads are independently handled instances with references to a stack and thread-local storage (TLS). They can be assigned, unassigned, and reassigned to a scheduler instance. A
uksched thread is minimal by design and executes only a non-returning thread function. Extended functionality such as POSIX-typical thread properties, thread IDs, and APIs are to be implemented on top of it, for example by
The revision has also enabled the
clone system call which was needed by
musl. With the revision, the API for handling context, extended context (floating point, vector units, …) and TLS/TCB are provided by a new architecture library called
The following highlights are implemented by
- Support for lightweight contexts (threads) with and without TLS/TCB
- Support for lightweight contexts (threads) that run only ISR-safe code
- Support for user-defined stack sizes
- Support for embedding custom TCBs by third-party libraries (e.g., libc, pthread-embedded)
Musl support, POSIX-process, and the
clone system call
This feature was championed by Simon Kuenzer and Dragoș Argint with extensive support from Răzvan Deaconescu, Adina Smeu, Florin Postolache, Maria Sfîrăială, Ștefan Jumărea, Robert Kuban, Delia Pavel, Eduard Vintilă, Cezar Crăciunoiu, Marco Schlumpp, Răzvan Vîrtan and Sergiu Moga.
We are proudly announcing that with this release
musl is becoming our default libc for application builds.
musl replaces the
pthread-embedded pair. Our goal is a vanilla integration that leaves
musl’s codebase untouched as much as possible. Updating the library to a newer version in the future should be as simple as possible.
Instead of doing system calls, our integration is utilizing the
syscall_shim library: complex but automatic macros allow us to replace every system call invocation with a direct function call at compile time. This required a fairly complete system call interface: We refactored posix-process to provide proper handling of threads and process IDs, and an implementation of the
clone system call which is used by
musl to create threads behind the POSIX thread API.
The update to
musl also required changes to
lwip. The internal
nolibc library was updated for better compatibility and to support the required changes for
musl in the core repository.
We updated the
kraft.yaml file for applications to use
musl as the default libc:
musl in your Unikraft application, update the
kraft.yaml file to feature
musl as a dependency (remove
newlibc and / or
pthread-embedded if present):
libraries: [...] musl: version: stable
Or update the application
[...] UK_LIBS ?= $(PWD)/../../libs LIBS := $(UK_LIBS)/musl [...]
This feature was championed by Florin Postolache with extensive input from Simon Kuenzer, Delia Pavel and Răzvan Deaconescu.
libc-test as an external library for Unikraft in the
libc-test tests the standard C library API. We updated
libc-test to make use of the
uktest framework and test standard C library implementations. We used it to test
musl, the new default libc for Unikraft and
KraftKit – Our New CLI Companion Tool
This feature was championed by Alexander Jung with extensive support from Cezar Crăciunoiu, Gabriel Mocanu, Ștefan Jumărea, Costi Răducanu, Andrei Albișoru.
We are excited to announce the release of KraftKit, our new companion tool for building Unikraft unikernels! KraftKit aims to ease the use of building and using Unikraft, streamlining the process of retrieving relevant sources and abstracting the internals of Unikraft so that you can focus on building your application as a unikernel.
You can get started today with KraftKit by installing via:
$ curl --proto '=https' --tlsv1.2 -sSf https://get.kraftkit.sh | sh $ kraft --help
. /^\ Build and use highly customized and ultra-lightweight unikernels. :[ ]: | = | /|/=\|\ Documentation: https://kraftkit.sh/ (_:| |:_) Issues & support: https://github.com/unikraft/kraftkit/issues v v ' ' USAGE kraft [SUBCOMMAND] [FLAGS] BUILD COMMANDS build Configure and build Unikraft unikernels clean Remove the build object files of a Unikraft project configure Configure a Unikraft unikernel its dependencies fetch Fetch a Unikraft unikernel's dependencies menu Open's Unikraft configuration editor TUI prepare Prepare a Unikraft unikernel properclean Completely remove the build artifacts of a Unikraft project set Set a variable for a Unikraft project unset Unset a variable for a Unikraft project PACKAGING COMMANDS pkg Package and distribute Unikraft unikernels and their dependencies pkg list List installed Unikraft component packages pkg pull Pull a Unikraft unikernel and/or its dependencies pkg source Add Unikraft component manifests pkg update Retrieve new lists of Unikraft components, libraries and packages RUNTIME COMMANDS ps List running unikernels rm Remove one or more running unikernels run Run a unikernel stop Stop one or more running unikernels FLAGS -h, --help help for kraft
On Rewriting It in Go
Our companion tool for Unikraft started life with an initial implementation written in Python. The original version (now known as
pykraft) has been a great adventure building and using over the last 2 years. However, it has had many limitations which maintainers of the Unikraft project believe limits its potential. As a result, we have spent the last few months re-building it from the ground-up in Go, now designed following the lessons we have learnt from this original project.
Apart from speed, size and efficiency, one of the main reasons for the reimplementation has been the ecosystem that Unikraft finds itself situated in. Many of the tools and services which Unikraft finds itself well-suited for are written in Go. Many of the usecases for manipulating unikernels are represented as libraries which are written in Go and many of the APIs for managing services in the cloud have APIs which are more rich in nature, and are written or provided in Go. These possible integrations are listed below under future plans.
pykraft project will always remain open-source and accessible to the community. However, maintainers of the Unikraft project will no longer maintain the project. Instead, the project today can only serve existing use cases outside of being an official command-line companion tool for building Unikraft, including: usage for research (such as with FlexOS) as well as Python bindings for building Unikraft unikernels programmatically. Note that since
pykraft will no longer receive updates to be compatible newer specification versions its last acceptable
kraft.yaml specication version which is v0.5. Newer versions of Unikraft’s core may also be incompatible. We will still accept valid PRs which make the repository more library-friendly as we are always seeking for contributors!
For anyone who is using the original
kraft program, please reach out to us so we can help you during this transition. KraftKit has been designed to feel very much like the original
kraft utility but with re-implementation and design aimed at making the workflow easier to use. Importantly, all original
kraft.yaml files are compatible.
We have a lot of plans and are very excited for the future of KraftKit towards making Unikraft easier to use and useful for developers and this journey has already begun. Some highlights of what’s to come:
- Integration into major software build systems, e.g. Packer and CloudFoundry BOSH;
- Using the built in machine monitor to facilitate runtime integrations, e.g. with Kubenetes and Nomad.
Memory Tagging Extension (MTE)
This feature was championed by Michalis Pappas with extensive support from Cezar Crăciunoiu and Marc Rittinghaus.
Memory tagging is a hardware protection against memory safety violations in the scope of both spatial (eg buffer overflows) and temporal (eg use-after-free) safety.
The idea behind MTE is to restrict pointer access to predefined blocks of memory. This is done by tagging memory regions and pointers, and check for match upon access. A mismatch generates a Tag Check Fault (TCF). Specifically, the architecture introduces the concepts of:
Physical Address Tags, which are and are stored into the upper bits of pointers. Allocation Tags, which tag blocks of memory. These are stored into the Tag PA Space, an IMPLEMENTATION DEFINED memory space that is logically different from the Data PA Space.
Combined together, these ensure that only memory within a given block is accessed. For instance, in the case of a buffer overflow, memory access outside the allocated bufer will cause a tag mismatch. Similarly, assuming that freeing memory re-tags the freed region, a use-after-free attempt will also cause a tag mismatch. Other accesses like prefetches, cache maintenance, translation table walks, accesses to the tag PA space, etc are always tag unchecked.
MTE requires support from the operating system. For example, addresses returned by malloc() must be tagged. Notice that the tags are only 4-bits wide, ie tags can have 16 possible values. Consequently, the same tag may be used on different regions on a given time, or that a region may have different tag values during time. This release introduces instrumentation to ukalloc. This is a fairly simple implementation that tags memory on uk_malloc_ifpages(), uk_free_ifpages(), and uk_posix_memalign_ifpages(). The region tagged is equal to the size requested by the user aligned to the MTE granule, ie 16 bytes. Any attempt to access memory outside that region, including allocation metadata, will cause a fault.
When it comes to hardware support, until recently MTE has been only known to be implemented in emulators QEMU, and in Arm FVP. Lately vendor support for MTE has started to appear on the news. Notable examples include the DSU-110 DynamIQ cluster from Arm for Armv9, and Samsung’s Exynos 2200.
Reaching 1k+ Stars and 500 Discord Members!
October was an interesting month for many reasons. One of those is that we have been able to reach an exciting threshold: we are beyond 1k Github stars on our main repository! In addition to this, our Discord server reached 500 members! A huge thank you and warm welcome to all stargazers, members and new contributors of the project!
We are very happy because we can see that Unikraft experienced a clear acceleration in the stars number in the last few months. More about this in the accompanying blog post.
Unikraft Community Meetup
This autumn we had the opportunity to organize the first in-person meeting of the Unikraft community members. The event was organized in the shape of a three-day trip to Sinaia, Romania between the 14th and the 16th of October.
The trip was a great opportunity for both technical talks about the future of Unikraft and for the community members to get to know each other more.
For details about this event, check the related blogpost.
In October, one of our traditional Saturday hackathons was a bit different, since it was accepted to be part of Hacktoberfest 2022.
The event was a hybrid one, with participants being able to join both remote, on our Discord server, or in person at University POLITEHNICA of Bucharest.
You can find more insides about the hackathon outcomes in the Hacktoberfest 2022 blogpost.
With awesome support from Technische Universität München (TUM) we organized the Unikraft Munich Hackathon, a two days event (October 22-23, 2022) of technical work on Unikraft. Around 30 active participants took part in tutorials and then on hackathon practical items that resulted in PRs and issues for the project.
The hackathon took place in person, at TUM, with support being provided both live and via the Unikraft Discord server.
You can find more about the Munich Hackathon on our blogpost.