We are proud to announce the update version of Unikraft, v0.17.0.
This release comes packed with features related to security, virtual filesystem and general application support. It contains plenty of fixes that improve the stability and application support of Unikraft.
In this blog post, we present the new features available in Unikraft. For a full breakdown, please check out the changelog.
linuxu platform (#1422)#This feature was championed by Simon Kuenzer with support from Marco Schlumpp.
Given the limited scope and reduced use of the linuxu platform, it has been removed.
This simplifies maintenance effort.
Applications should no longer be built for the linuxu platform.
This feature was championed by Michalis Pappas with great support from Sergiu Moga and assistance from Șerban Sorohan.
The initialization of paged memory is now reworked to provide a more flexible implementation that is capable of handling regions beyond the limits defined in the boot pagetables. The motivation for this change is to allow mapping device regions that are unknown at compile-time, such as device regions of Arm CCA Realms accessed via their Unprotected IPA Alias, the address of which cannot be derived at compile-time.
The new implementation replaces the previous method of updating the boot pagetables with the initialization of a new pagetable from regions defined in bootinfo. This simplifies the implementation and avoids expensive TLB flushes associated with updating the existing pagetable. To support this rework additional changes have been made to bootinfo and the way device MMIO regions are mapped.
With the initialization of a new pagetable bootinfo is now simplified to only contain descriptors that correspond to valid memory regions. This deprecates the UKPLAT_MEMRF_MAP / UKPLAT_MEMRF_UNMAP flags in bootinfo regions as well as the hard-coded unmap_mrd region in the boot pagetables.
The initialization of a new pagetable additionally requires that early devices add descriptors for their MMIO regions to bootinfo, so that they remain mapped when transitioning to the new pagetable. To address this requirement, a new early init boot-stage has been added to kvm/arm64, that is invoked at the end of early boot before passing control to the platform. x86_64 is not affected by this change.
Finally, devices that register at later stages of init are also updated to map their MMIO regions at runtime.
This changeset comes with additional changes and cleanup, most importantly on the drivers of the pl011 and ns16550 devices.
This feature was championed by Alexandru Apostolescu and Michalis Pappas with support from Sergiu Moga and Delia Pavel.
The uk_swrand library has been redesigned to exclusively provide cryptographically secure random numbers and has been renamed to uk_random. The random numbers are generated using the ChaCha20 algorithm, seeded by a hardware TRNG. Both /dev/random and /dev/urandom provide access to the output of the software CSPRNG, while direct access to hardware generated randomness is exposed through the newly added /dev/hwrng, which provides additional compatibility with Linux.
For non security-critical use cases, software-generated non cryptographically-secure random numbers are available via the standard library's rand() function, available in nolibc and musl.
This set of features were championed by Andrei Tatar with support from Andrei Stan, Andrei Tatar, Delia Pavel, Eduard Vintilă, Marco Schlumpp, Mihnea Firoiu, Mihnea Popeanga, Radu Nichita, Razvan Deaconescu, Razvan Virtan, Serban Sorohan, Stefan Jumarea.
As part of the ongoing effort to deprecate vfscore in favor of an improved and more modular file/filesystem layer, the new-style ukfiles introduced in v0.16.0 (Telesto) have seen new additions and marked improvements. In addition to many under-the-hood changes, users and developers alike can expect several new features.
The handling of standard I/O files (stdin, stdout, stderr) has been moved out of vfscore into its own core library: posix-tty, responsible for ukfile implementations of serial- and pseudo-files, as well as the initialization of the first 3 file descriptors on boot.
With this change, file descriptors 0, 1, and 2 are no longer special and can be closed like any other files.
Support for packet-mode pipes (O_DIRECT) has been introduced. This Linux extension has pipes behave analogous to datagram sockets, handling data in indivisible "packets" and matching each read with a singular write.
UNIX Sockets, introduced in the previous release have seen improvements:
SOCK_DGRAM) and sequential-packet (SOCK_SEQPACKET) socketssendto & equivalentgetsockopt/setsockoptFile polling functions (select/poll/epoll) can now be opted into forcefully yielding execution back to the scheduler when called. This may help mitigate hangs on workloads that (perhaps inadvertently) rely on a pre-emptive/starvation-free scheduler to work correctly. This behavior is enabled by the CONFIG_LIBPOSIX_POLL_YIELD Kconfig option.
epoll (#1419)#Files that are closed while being monitored by epoll will now automatically remove themselves from being monitored. This prevents resource leaks on workloads such as Nginx that do not remove files from epoll before closing.
ukfile API Changes & Improvements (#1397)#The (still in development) ukfile API continues to stabilize and improve in response to the needs of consumer libraries:
epoll feature of autodeleting closed files is implemented using file finalizers;iovec-based I/O.This feature was championed by Andrei Tatar with support from Marco Schlumpp, Delia Pavel and Mihnea Firoiu.
Implementations of splay and rank-balanced trees have been imported from FreeBSD 13.3, enabling their use in Unikraft libraries for implementing efficient mappings.
This feature was championed by Sergiu Moga with support from Michalis Pappas and Simon Kuenzer.
Implement a generic stack allocation wrapper over a given Unikraft memory allocator. The library receives as arguments an allocator and, if libukvmem is enabled, a virtual address space and an initial size. The library works in three ways:
libukvmem is disabled, the library will make use of the allocator received as argument to allocate the initial allocator structure and Unikraft compliant stacks (16-byte alignment).libukvmem is enabled, the library will make use of the allocator received as argument to allocate the initial allocator structure and Unikraft compliant stacks. The difference from 1. is that the virtual address space and the initial size arguments are now available. The two arguments will hint how much of the stack to premap (page-in early) whenever they are allocated first, i.e. what is the initial stack size before on-demand paging kicks in.libukvmem is enabled AND the configuration option of the library integrating page guards is enabled, the allocator given as argument will only be used once when allocating the initial allocator structure, while the virtual address space and initial size arguments will be used to create stack VMA's, specific to libukvmem.Additionally, now all libukvmem stacks have their guards end-to-end with the help of two new configurations: CONFIG_LIBUKVMEM_STACK_GUARD_PAGES_TOP (Number of guard pages for the top of the stack) and CONFIG_LIBUKVMEM_STACK_GUARD_PAGES_BOTTOM (Number of guard pages for the bottom of the stack (low address)).
swapgs Logic (#1346, #77)#This feature was championed by Sergiu Moga with great support from Simon Kuenzer.
Following the introduction of the concept of auxiliary stack pointers, swapgs, struct uk_syscall_ctx and struct ukarch_sysregs, a number of things have emerged:
libcontext (arch/)swapgs introduces a significant inconsistency between ARM64 and x86_64 as we never know during an exception the state of MSR_GS_BASE / MSR_KERNEL_GS_BASEThus, this commit does the following:
libcontext and document themlib/syscall_shim/arch/x86_64/sysregs.c -> arch/x86/sysctx.clib/syscall_shim/arch/x86_64include/arch/sysregs.h -> arch/x86/x86_64include/uk/asm/sysctx.hs/struct ukarch_sysregs/struct ukarch_sysctx/ (and all related defs)struct uk_syscall_ctx from lib/syscall_shim/include/uk/syscall.h to include/uk/arch/ctx.h as struct ukarch_execenvs/struct uk_syscall_ctx/struct ukarch_execenv/ (and all related defs)swapgs, architecture specific holdback by exploiting the flexibility of auxiliary stacks through the introduction of a new always existing control block at their top end:struct ukarch_auxspcb under libcontextauxspscb will be part of the auxiliary stack, we need to know the safe place where we can start using the auxiliary stack area as a stack (this is also helpful in cases where we need to nest on the auxstack)swapgs pair will only be done very early during system call entry (and only there, not on clone child exit anymore either) just enough so that we, first things first, switch to auxstack and push auxsp so that on entry to C handler we will know that we must do a call to ukarch_sysctx_load on the Unikraft sysctx we can get from the pushed auxsp (another benefit of this is we get rid of MSR read/writes)We continue organizing monthly coding sessions (formerly hackathons), generally taking place in the last Saturday of each month. Hackathons take place both online and in person, generally at the National University of Science and Technology POLITEHNICA of Bucharest.
We organized a monthly coding session on Saturday, March 30, 2024. And then we organized one on Sunday, April 28, 2024. It was targeted on the 0.17 release.
The next monthly coding session is scheduled to take place on Saturday, June 22, 2024.
Unikraft Summer Workshop 2024 (USW'24) is a free and virtual workshop held by members of the Unikraft community. It focuses on cloud-native applications and on the unikernel technology that powers efficient and high performance cloud deployments.
The three-week event holds a number of starter tutorials and workshops on how to configure, build, run, deploy and debug cloud applications using Unikraft.
There will be 6 sessions taking place in the first two weeks (between July 1 and July 12, 2024). Each session takes place for 3 hours, in the 4pm-7pm CEST, in English. Sessions will consist of talks and demos delivered by members of the Unikraft community, followed by practical tutorials that you will work on with support and supervision. Sessions take place on Unikraft's Discord server.
Topics include building unikernels, benchmarking, debugging, porting applications, virtualization and platform specifics. The first 3 sessions (first week) will focus on using KraftKit, Unikraft's companion tool, to manage cloud applications. The next 3 sessions (second week) will focus on the internals of Unikraft: the build system, native configuration options, application porting.
The two weeks with sessions will be followed by a week of working on the final project. You will work on the project in teams of 2-3 people. We will have support sessions online to help with the project.
On Saturday, 20 July 2024, 9am-5pm CEST, we will have the final hackathon, that consists of adding final touches to the project. The hackathon will take place in hybrid format, in person, at the The National University of Science and Technology POLITEHNICA of Bucharest, and, online, Unikraft's Discord server. Participants will receive a participation diploma. The first three teams will get special prizes. All hackathon in-person participants will get a Unikraft T-shirt.
Unikraft was part of DORS/CLUC, an open source conference, taking place in Zagreb, Croatia, on May 14-17, 2024, where:
It was a great occasion to get in touch with other members of the open source community and advertise the Unikraft project, both to technical enthusiasts and business professionals.
The Zagreb Unikraft Hackathon was organized in conjunction with DORS/CLUC on May 18-19, 2024. This was the first event focused on cloud applications, KraftKit and KraftCloud.
Participants were able to deploy existing application and also try their own, using unikernel technology and KraftCloud.
Unikraft took part in Xen Summit 2024, the yearly gathering of the Xen community, in Lisbon, Portugal, on June 4-6, 2024. We gave the Keynote, titled [Unikraft Has Arrived: The Lightweight, Fast, Usable Solution for Cloud Deployments] (https://docs.google.com/presentation/d/1bOskgOIW5P0yQjybuiEqWolXwuCoTzwBy5e_OqGoRi8/edit?usp=sharing).
We also headed the "Xen and Unikraft (and unikernels)" design session, where we captured insights in improving Unikraft's Xen support and where we clarified use cases for Unikraft for Xen (such as stub domains and Qubes OS).
Unikraft GSoC'24 projects have been announced! 5 projects are currently under way in the Unikraft community:
The 5 applicants, and their projects are:
Mihnea Firoiu from POLITEHNICA Bucharest in Bucharest, Romania
Yang Hu from University of Toronto in Toronto, Canada
Ujjwal Mahar from Sharda University in Greater Noida, NCR, India
Maria Pană from POLITEHNICA Bucharest in Bucharest, Romania
Sriprad Potukuchi from PES University in Bengaluru, India
Congratulations Mihnea, Yang, Ujjwal, Mia, Sriprad! 🥳
Feel free to ask questions, report issues, and meet new people.