Unikraft releases v0.14.0 (Prometheus)

This release is the result of extensive hard work during the last months in the entire community.

Unikraft v0.14.0 (Prometheus) is out and comes packed with tons of fixes and new features that improve the overall Unikraft ecosystem.

In this blog post, we describe some of the new features available in Unikraft. For a full breakdown, please check out the changelog.

Native macOS (Darwin) Support (#1034)#

This feature was championed by Simon Kuenzer, Mihailescu Eduard-Florin, and Răzvan Deaconescu with important advice from Alexander Jung.

With the release of Unikraft 0.14, the build system officially supports a macOS-based development environment. Unikernel images can be developed, compiled, and tested directly on macOS. Both types of Macs are supported: you can do this on a Mac with an Intel processor or with an Apple silicon. Only the following packages are expected to be installed on the system via Homebrew:

brew install gnu-sed make m4 gawk grep wget qemu socat git
# ...for compiling x86_64 unikernels
brew install x86_64-elf-binutils x86_64-elf-gcc
# ...for compiling aarch-64 unikernels
brew install aarch64-elf-binutils aarch64-elf-gcc

Behind the scenes, Unikraft uses the Python-based Kconfiglib to configure Unikraft on a Mac. The build system automatically chooses the Linux-equivalent command-line tools on Mac to perform a successful build.

Within a terminal session, Unikraft can be cloned in the same way as on Linux:
```
git clone https://github.com/unikraft/unikraft.git
cd unikraft
```
Configuration and compilation can be invoked with gmake:
```
# Configure (select an architecture and platform)
gmake menuconfig
# Build
gmake
```
Also on Mac, qemu-guest can support in running built Unikernels with QEMU:
```
support/scripts/qemu-guest -t arm64v -k build/unikraft_qemu-arm64
```

UEFI Support on `x86_64` and `AArch64` (#910, #909)#

This feature was championed by Sergiu Moga with amazing support from Michalis Pappas and advice from Răzvan Deaconescu, Ștefan Jumărea and Răzvan Vîrtan.

Unikraft bootable through UEFI on arm64 / AArch64 and x86_64.

Features:

custom, minimalist built-in bootloader to act as a UEFI stub capable of loading Unikraft straight from UEFI firmware on both x86_64 and AArch64 by relocating the kernel, fetching the command-line, initial RAM disk, Devicetree blob and setting up the memory region descriptors as well as bridging the gap between the components initialized by UEFI and those supported by Unikraft
script to create UEFI based disk and ISO Unikraft bootable images

Linux Boot Protocol on `arm64` (#988)#

This feature was championed by Michalis Pappas with the inputs of Xingjian Zhang and Sergiu Moga.

The arm64 Linux boot protocol defines the layout of a Linux image, the kernel parameters, and the system configuration expected by the boot environment.

Unikraft 0.14 introduces support for the arm64 Linux boot protocol, which allows booting Unikraft into any environments that expects an arm64 linux image, such as boot loaders or VMMs like Firecracker.

Firecracker Support on `arm64` (#989)#

This feature was championed by Michalis Pappas with the inputs of Xingjian Zhang and Sergiu Moga.

This release introduces the ability to boot arm64 builds of Unikraft on Firecracker. Firecracker is an ultra-light VMM that allows creating and running microVMs, combining the security and isolation properties of VMs with the flexibility and ease of use of containers.

Dynamic `uk_store` API (#939)#

This feature was championed by Cezar Crăciunoiu and Michalis Pappas with inputs from Simon Kuenzer.

The uk_store API provides micro-libraries with the ability to register getters and setters for various properties. Unikraft 0.14 enhances uk_store with the ability for microlibraries to dynamically create and destroy uk_store objects. This paves the road for features like per-device statics, dynamic entries in virtual file systems like procfs / sysfs, and more.

SUBBUILD Build System Feature (#938)#

This feature was championed by Simon Kuenzer with support from Michalis Pappas.

This release introduces a so called SUBBUILD feature to the build system. It adds the ability to place generated files (e.g., pre-processed, compiled units) in a subdirectory of a library build directory. For this purpose, the build system looks for a file-scoped variable with the suffix _SUBBUILD. Such a variable contains a path that is created below the library build directory.

The feature is especially useful for generating headers with a pre-processor, like AWK or M4.

An example Makefile.uk:

# Create `myheader.h` under `build/libmylib/include/uk/`
LIBMYLIB_SRCS-y            += $(LIBMYLIB_BASE)/myheader.m4>.h
LIBMYLIB_MYHEADER_SUBBUILD += include/uk

# Register the include folder of the library with the generated header
CINCLUDES-$(CONFIG_LIBMYLIB) += -I$(LIBMYLIB_BUILD)/include

TREE BUILD Build System Feature (#1001)#

This feature was championed by Andrei Tatar with support from Marco Schlumpp, Maria Sfîrăială, Ștefan Jumărea, and Simon Kuenzer.

TREE BUILD is an alternative library registration mode for the build system which automatically handles subdirectory paths for build products. The work adds addlib_tree and addlib_tree_s as library registration functions. For each library in tree mode, the build system places build products in a subdirectory structure that mirrors the directory structure of the corresponding source file. This is a very handy tool to simplify the porting of libraries whose source files span multiple directories.

`uklibid`: Library Identifiers (#938)#

This feature was championed by Simon Kuenzer with support from Michalis Pappas and Robert Kuban.

Based on the SUBBUILD build feature, this release introduces library identifiers with uklibid. The new library generates at compile-time unique IDs for each library. Such a ID starts from 0 and increases linearly so that they can be used for table lookups with O(1) complexity. The library provides helpers to map identifiers to names and vice versa. lib/ukdebug and lib/ukstore are first users of this library and adopted accordingly.

A clear separation between runtime and compile-time resolution of library identifiers is done so that libraries can potentially be built off-tree and shipped in binary form. Each library's own identifier (uk_libid_self()) is stored in a constant variable that is resolved during link-time. The owner of the identifier variable is uklibid, so that the identifier is external to a binary library.

`ukreloc`: Self-Relocation and Symbol-based Relocations (#772)#

This feature was championed by Sergiu Moga with amazing support from Michalis Pappas, Marco Schlumpp and advice from Răzvan Deaconescu and Dragoș Petre.

Unikraft is now position independent, can be built as a static PIE and self relocate on both x86_64 and ARM64.

Features:

CONFIG_OPTIMIZE_PIE to build Unikraft as a static PIE
CONFIG_LIBUKRELOC automatically selected by CONFIG_OPTIMIZE_PIE and implements Unikraft's custom relocation system:
- custom macro-definitions to be used as relocation friendly alternatives to various instructions (ldr -> ur_ldr, mov -> ur_mov) as well as numerical values declarations GNU Assembler directives (.quad/.long/.short/etc -> ur_data) and a way to create relocatable static Page Table entries (ur_pte).
- self relocator written in C to be used in 64-bit operating mode
- 32-bit self relocator written in assembly to be used by early code that runs in x86 Protected Mode (e.g. when booting from Multiboot)

Individual Volume Automounting (#979)#

This feature was championed by Sergiu Moga with support from Simon Kuenzer and advice from Radu Nichita and Ștefan Jumărea.

It is now possible to dynamically mount additional volumes via the introduction of a new command-line argument vfs.fstab that is meant to contain a list of whitespace separated strings with the following format:

vfs.fstab=[
    "<src_dev>:<mntpoint>:<fsdriver>[:<flags>:<opts>]"
    "<src_dev>:<mntpoint>:<fsdriver>[:<flags>:<opts>]"
    ...
]

The core will parse the provided strings as volume information and attempt to mount them accordingly, if CONFIG_LIBVFSCORE_FSTAB is provided.

vDSO and vsyscall Support for Binary Compatibility (#23)#

This feature was championed by Tianyi Liu with great support from Simon Kuenzer, and insights from Teodor Țeugea and Ștefan Jumărea.

The vsyscall (virtual system call) is the first and oldest mechanism used to accelerate system calls. Due to security concerns, it has been deprecated, and vDSO (virtual dynamic shared object) serves as its successor. However, in the context of unikernels, kernel isolation is not a concern, allowing us to utilize both of them. The app-elfloader now supports both of these features.

If you don't have the source code of your application but desire the performance of Unikraft, app-elfloader can load unmodified ELF files. However, there will be some performance overhead compared to a native build, while the two newly introduced features can help mitigate this difference.

The vDSO is available for both dynamically linked applications and statically linked ones, to accelerate certain commonly used time-related system calls (clock_gettime, gettimeofday, time, clock_getres). As a result, some server programs can experience significant performance improvements.

If your application is dynamically linked, it can further utilize vsyscall to avoid costly binary syscalls (since vsyscall has been deprecated in the latest version of libc, we provide patched dynamic runtime libraries for glibc and musl).

Application Updates#

This feature was championed by in the community by Ștefan Jumărea, Radu Nichita, Andrei Tatar, Răzvan Deaconescu, Alexander Jung and participants at various Unikraft hackathons.

The Unikraft applications now contain README.md files with very easy to use instructions for running the applications, both using kraftkit and the Make-based build system. We take advantage of the defconfig build rule, that can generate a full .config file starting from a given minimal configuration. The minimal configurations (for multiple supported platforms and architectures) are already provided in the application repositories, and can be used as such:

UK_DEFCONFIG=$(pwd)/.config.helloworld_qemu-x86_64 make defconfig

After the configuration file is generated, the application can be build using make and run using the instructions from the README.md files.

Along with that, many applications have been ported to use Musl as the default libc. Some of the more important ones are app-lua, app-duktape, app-click, app-wamr.

As part of the Unikraft Summer of Code final hackathon, we also focused on the binary compatibility side, porting a lot of new applications and adding them in the dynamic-apps repository. Among the new ported applications are lua, ruby, iputils, redis7.

In order to have a one to one version match between the binary compatibility and native application, app-redis was also updated to version 7.0.11, from 5.0.6.

GitHub Actions for CI (#1012)#

This feature was championed by Alexander Jung and Cezar Crăciunoiu with support from Răzvan Deaconescu.

Preliminary steps in using GitHub actions as a continuous integration have been undertaken. These allow the building and running of tests at PR submissions, as a complement and future replacement of the current Concourse-based system.

In conjunction to this, a new application catalog repository will store application files and serve as the sole go-to place for grabbing applications that can be built and run on top of Unikraft.

Improved Python Support (lib-python3:15)#

This feature was championed by Andrei Tatar with support from and Ștefan Jumărea, Radu Nichita and Maria Sfîrăială.

The Unikraft port of Python 3 has been updated to upstream version 3.10.11, with the Python 3 Demo App similarly updated.

Notable changes:

Functional asyncio
More robust root filesystem build
Many standard lib functionality & stability improvements

We are also excited to announce the first native ports of third-party Python packages:

Updated Go Support (lib-libgo:#23)#

This feature was championed by Marc Rittinghaus and Eduard Vintilă with support from Ștefan Jumărea and Radu Nichita.

We update the Go support to version 1.18 an fully integrated lib-go with Musl. Go applications are now fully functional on top of Unikraft for x86_64.

In the near future Go support will be updated for AArch64.

Integrated Clang Support (#23)#

This feature was championed by Maria Sfîrăială with input from Michalis Pappas and Ștefan Jumărea.

We now provide Clang support for compiling for x86_64 and cross-compiling for AArch64 targets. This is exciting news as it opens the door for cool compiler enforced security features and better integration for platforms such as Darwin (unikraft#685).

Use the following command to compile with Clang, both on x86_64 and on AArch64 (depending on the configuration):

$ make CC=clang

Coding Conventions (docs:#248)#

This feature was championed by Marc Rittinghaus, Michalis Pappas, Simon Kuenzer, Marco Schlumpp and Radu Nichita based on numerous meetings where the Unikraft community provided insight.

We're excited to announce an official coding conventions guideline! This comprehensive document outlines that one should have in mind when submitting contributions to out codebase, whether it's a first-time contributor or a well-known veteran.

By introducing these conventions, we want to ensure consistent and clean code across all our repositories and make maintenance more efficient. The document includes naming conventions, indentation guidelines, commenting practices, and many more. Happy coding!

External Build System Integration (`kraftld`) (#957)#

This feature was championed by Martin Kröning with support from Marco Schlumpp and Simon Kuenzer.

You can now influence the Unikraft build process via a set of environment variables.

UK_ASFLAGS: explicit Unikraft-specific additions to the assembler flags (the ASFLAGS variable is ignored)
UK_CFLAGS: explicit Unikraft-specific additions to the C compiler flags (the CFLAGS variable is ignored)
UK_CXXFLAGS: explicit Unikraft-specific additions to the C++ compiler flags (the CXXFLAGS variable is ignored)
UK_GOCFLAGS: explicit Unikraft-specific additions to the GO compiler flags (the GOCFLAGS variable is ignored)
UK_LDFLAGS: explicit Unikraft-specific additions to the linker flags (the LDFLAGS variable is ignored)
UK_LDEPS: explicit, space-separated link-time file dependencies (changes to these files will trigger relinking on subsequent builds)

This info is also shown on make help.

The UK_LDFLAGS and UK_LDEPS flags are of special interest, as they allow injecting externally built object files into the Unikraft image. This approach will be further integrated into KraftKit via kraftld. kraftld is a drop-in replacement for a gcc-flavored linker, allowing the transparent linking of object files into Unikraft images. A follow-up blog post will describe this approach and its design in more detail and will show how this is used to bring Rust support to Unikraft.

`ADOPTERS.md` File (#892)#

This feature was championed by Alexander Jung with support from Răzvan Deaconescu.

We want to keep track of entities using Unikraft and foster collaboration. Inside the unikraft core repository there is now an ADOPTERS.md file storing Unikraft users and, ideally, contributors (companies, universities, institutions) that benefit from using Unikraft as part of their commercial, research or teaching activities.

If you are an entity using Unikraft, follow the submission guidelines and submit a pull request to update the ADOPTERS.md file. And expect full support from the Unikraft community in assisting your work and integrating contribution in the Unikraft project.

Community Activities#

Unikraft Summer of Code 2023#

This activity was lead by Ștefan Jumărea with the active involvement of Cezar Crăciunoiu, Răzvan Deaconescu, Eduard Vintilă, Radu Nichita, Vlad Bădoiu, Sergiu Moga, Luca Serițan, Teodor Țeugea.

The third edition of Unikraft Summer of Code took place this July, had about 45 participants, and consisted in 6 sessions and a final hackathon. The sessions covered topics like building and running unikernels, debugging, porting applications, etc. The final hackathon was a full day event, and the participants had their chance to apply the things they've learned and to contribute to Unikraft. It took place both in person, at University POLITEHNICA of Bucharest, and online, on Discord. A lot of work was done on application porting and binary compatibility, with more than 35 issues and pull requests opened.

We plan to keep organising the Unikraft Summer of Code in the following years, and make people eager to learn more about low-level topics, operating systems, unikernels, and, most importantly, the open source world.

Full-day Hackathon at UPB, 2023

Unikernels Alliance#

As we want to speed up the unikernels ascent, we decided to organize a discussion group involving all the people that are interested and bring their contribution into this field.

We decided to name our group the "Unikernel Alliance". This is a community formed from existing unikernel and libOS communities, aimed at popularizing and promoting unikernel-like solutions in research, industry and in technical communities.

We organize monthly meetings, featuring technical talks from the group members. Until now, we have organized three meetings. You can find the recordings and slides here.

Feel free to join the discussion group, our Slack channel and to be part of our next meeting on Thursday, August 31, 2023, 6pm CEST, online, on Zoom.

Unikernels in the Wild#

Together with our friends from RWTH Aachen, we organized the first edition of Unikernels in the Wild, an event centered on knowledge sharing in the Unikernels area. Răzvan Deaconescu and Răzvan Vîrtan participated from our side, having presentations and offering support during the practical workshops.

The day started with a few presentations about the general view and latest updates in RustyHermit. Then, we moved on with two presentations about binary compatibility and memory allocation in Unikraft. Also, our collaborators from Nubificus (remember the Athens Hackathon?) gave a presentation about their work on getting unikernels into the DevOps mainstream. The day ended with two hands-on sessions for Unikraft and RustyHermit.

This event has been a great chance for getting together people from the Unikernels community and we hope it will become a tradition!

Full-day Hackathon at UPB, 2023

If you are using Unikraft, consider adding your affiliation in our ADOPTERS.md file. We use this file to keep a track of entities that use Unikraft for their commercial and / or research work.

Google Summer of Code 2023#

We are proud to be one of the Google Summer of Code organizations for the second year in a row!

During these last months, our 5 students have started their work and already achieved some amazing intermediary results! You can find more about the progress of our mentees in the following blog posts:

Stay tuned for news regarding the final status of their projects!