We are thrilled to announce the new edition of Unikraft, v0.11.0 (Janus)!
This release is the result of around 3 months of hard work in the entire community, with a focus on integrating the long awaited musl support.
In this blog post, we highlight just some of the new features available in Unikraft. For a full breakdown, please check out the changelog.
This feature was championed by Simon Kuenzer with extensive support from Răzvan Deaconescu, Adina Smeu, Dragoș Argint, Cezar Crăciunoiu and Florin Postolache.
We are releasing a revised uksched API. The driving motivation for the design was to remove unneeded functionality, simplification, and introducing the concept of separating thread creation from scheduling. Threads are independently handled instances with references to a stack and thread-local storage (TLS). They can be assigned, unassigned, and reassigned to a scheduler instance. A uksched thread is minimal by design and executes only a non-returning thread function. Extended functionality such as POSIX-typical thread properties, thread IDs, and APIs are to be implemented on top of it, for example by pthread-embedded or musl.
The revision has also enabled the clone system call which was needed bymusl. With the revision, the API for handling context, extended context (floating point, vector units, ...) and TLS/TCB are provided by a new architecture library called libcontext.
The following highlights are implemented by libcontext:
clone system call#This feature was championed by Simon Kuenzer and Dragoș Argint with extensive support from Răzvan Deaconescu, Adina Smeu, Florin Postolache, Maria Sfîrăială, Ștefan Jumărea, Robert Kuban, Delia Pavel, Eduard Vintilă, Cezar Crăciunoiu, Marco Schlumpp, Răzvan Vîrtan and Sergiu Moga.
We are proudly announcing that with this release musl is becoming our default libc for application builds. musl replaces the newlib and pthread-embedded pair. Our goal is a vanilla integration that leaves musl's codebase untouched as much as possible. Updating the library to a newer version in the future should be as simple as possible.
Instead of doing system calls, our integration is utilizing the syscall_shim library: complex but automatic macros allow us to replace every system call invocation with a direct function call at compile time. This required a fairly complete system call interface: We refactored posix-process to provide proper handling of threads and process IDs, and an implementation of the clone system call which is used by musl to create threads behind the POSIX thread API.
The update to musl also required changes to lwip. The internal nolibc library was updated for better compatibility and to support the required changes for musl in the core repository.
We updated the kraft.yaml file for applications to use musl as the default libc:
To use musl in your Unikraft application, update the kraft.yaml file to feature musl as a dependency (remove newlibc and / or pthread-embedded if present):
libraries:[...]musl:version: stable
Or update the application Makefile:
[...]UK_LIBS ?= $(PWD)/../../libsLIBS := $(UK_LIBS)/musl[...]
This feature was championed by Florin Postolache with extensive input from Simon Kuenzer, Delia Pavel and Răzvan Deaconescu.
We added libc-test as an external library for Unikraft in the lib-libc-test repository. libc-test tests the standard C library API. We updated libc-test to make use of the uktest framework and test standard C library implementations. We used it to test musl, the new default libc for Unikraft and nolibc.
This feature was championed by Alexander Jung with extensive support from Cezar Crăciunoiu, Gabriel Mocanu, Ștefan Jumărea, Costi Răducanu, Andrei Albișoru.
We are excited to announce the release of KraftKit, our new companion tool for building Unikraft unikernels! KraftKit aims to ease the use of building and using Unikraft, streamlining the process of retrieving relevant sources and abstracting the internals of Unikraft so that you can focus on building your application as a unikernel.
You can get started today with KraftKit by installing via:
$ curl --proto '=https' --tlsv1.2 -sSf https://get.kraftkit.sh | sh$ kraft --help
./^\ Build and use highly customized and ultra-lightweight unikernels.:[ ]:| = |/|/=\|\ Documentation: https://kraftkit.sh/(_:| |:_) Issues & support: https://github.com/unikraft/kraftkit/issuesv v' 'USAGEkraft [SUBCOMMAND] [FLAGS]BUILD COMMANDSbuild Configure and build Unikraft unikernelsclean Remove the build object files of a Unikraft projectconfigure Configure a Unikraft unikernel its dependenciesfetch Fetch a Unikraft unikernel's dependenciesmenu Open's Unikraft configuration editor TUIprepare Prepare a Unikraft unikernelproperclean Completely remove the build artifacts of a Unikraft projectset Set a variable for a Unikraft projectunset Unset a variable for a Unikraft projectPACKAGING COMMANDSpkg Package and distribute Unikraft unikernels and their dependenciespkg list List installed Unikraft component packagespkg pull Pull a Unikraft unikernel and/or its dependenciespkg source Add Unikraft component manifestspkg update Retrieve new lists of Unikraft components, libraries and packagesRUNTIME COMMANDSps List running unikernelsrm Remove one or more running unikernelsrun Run a unikernelstop Stop one or more running unikernelsFLAGS-h, --help help for kraft
Our companion tool for Unikraft started life with an initial implementation written in Python. The original version (now known as pykraft) has been a great adventure building and using over the last 2 years. However, it has had many limitations which maintainers of the Unikraft project believe limits its potential. As a result, we have spent the last few months re-building it from the ground-up in Go, now designed following the lessons we have learnt from this original project.
Apart from speed, size and efficiency, one of the main reasons for the reimplementation has been the ecosystem that Unikraft finds itself situated in. Many of the tools and services which Unikraft finds itself well-suited for are written in Go. Many of the usecases for manipulating unikernels are represented as libraries which are written in Go and many of the APIs for managing services in the cloud have APIs which are more rich in nature, and are written or provided in Go. These possible integrations are listed below under future plans.
The original pykraft project will always remain open-source and accessible to the community. However, maintainers of the Unikraft project will no longer maintain the project. Instead, the project today can only serve existing use cases outside of being an official command-line companion tool for building Unikraft, including: usage for research (such as with FlexOS) as well as Python bindings for building Unikraft unikernels programmatically. Note that since pykraft will no longer receive updates to be compatible newer specification versions its last acceptable kraft.yaml specication version which is v0.5. Newer versions of Unikraft's core may also be incompatible. We will still accept valid PRs which make the repository more library-friendly as we are always seeking for contributors!
For anyone who is using the original kraft program, please reach out to us so we can help you during this transition. KraftKit has been designed to feel very much like the original kraft utility but with re-implementation and design aimed at making the workflow easier to use. Importantly, all original kraft.yaml files are compatible.
We have a lot of plans and are very excited for the future of KraftKit towards making Unikraft easier to use and useful for developers and this journey has already begun. Some highlights of what's to come:
This feature was championed by Michalis Pappas with extensive support from Cezar Crăciunoiu and Marc Rittinghaus.
Memory tagging is a hardware protection against memory safety violations in the scope of both spatial (eg buffer overflows) and temporal (eg use-after-free) safety.
The idea behind MTE is to restrict pointer access to predefined blocks of memory. This is done by tagging memory regions and pointers, and check for match upon access. A mismatch generates a Tag Check Fault (TCF). Specifically, the architecture introduces the concepts of:
Physical Address Tags, which are and are stored into the upper bits of pointers. Allocation Tags, which tag blocks of memory. These are stored into the Tag PA Space, an IMPLEMENTATION DEFINED memory space that is logically different from the Data PA Space.
Combined together, these ensure that only memory within a given block is accessed. For instance, in the case of a buffer overflow, memory access outside the allocated bufer will cause a tag mismatch. Similarly, assuming that freeing memory re-tags the freed region, a use-after-free attempt will also cause a tag mismatch. Other accesses like prefetches, cache maintenance, translation table walks, accesses to the tag PA space, etc are always tag unchecked.
MTE requires support from the operating system. For example, addresses returned by malloc() must be tagged. Notice that the tags are only 4-bits wide, ie tags can have 16 possible values. Consequently, the same tag may be used on different regions on a given time, or that a region may have different tag values during time. This release introduces instrumentation to ukalloc. This is a fairly simple implementation that tags memory on uk_malloc_ifpages(), uk_free_ifpages(), and uk_posix_memalign_ifpages(). The region tagged is equal to the size requested by the user aligned to the MTE granule, ie 16 bytes. Any attempt to access memory outside that region, including allocation metadata, will cause a fault.
When it comes to hardware support, until recently MTE has been only known to be implemented in emulators QEMU, and in Arm FVP. Lately vendor support for MTE has started to appear on the news. Notable examples include the DSU-110 DynamIQ cluster from Arm for Armv9, and Samsung's Exynos 2200.
October was an interesting month for many reasons. One of those is that we have been able to reach an exciting threshold: we are beyond 1k Github stars on our main repository! In addition to this, our Discord server reached 500 members! A huge thank you and warm welcome to all stargazers, members and new contributors of the project!
We are very happy because we can see that Unikraft experienced a clear acceleration in the stars number in the last few months. More about this in the accompanying blog post.
This autumn we had the opportunity to organize the first in-person meeting of the Unikraft community members. The event was organized in the shape of a three-day trip to Sinaia, Romania between the 14th and the 16th of October.
The trip was a great opportunity for both technical talks about the future of Unikraft and for the community members to get to know each other more.
For details about this event, check the related blogpost.
In October, one of our traditional Saturday hackathons was a bit different, since it was accepted to be part of Hacktoberfest 2022.
The event was a hybrid one, with participants being able to join both remote, on our Discord server, or in person at University POLITEHNICA of Bucharest.
You can find more insides about the hackathon outcomes in the Hacktoberfest 2022 blogpost.
With awesome support from Technische Universität München (TUM) we organized the Unikraft Munich Hackathon, a two days event (October 22-23, 2022) of technical work on Unikraft. Around 30 active participants took part in tutorials and then on hackathon practical items that resulted in PRs and issues for the project.
The hackathon took place in person, at TUM, with support being provided both live and via the Unikraft Discord server.
You can find more about the Munich Hackathon on our blogpost.
Feel free to ask questions, report issues, and meet new people.