Reminders

Print system

This print system in implemented in lib/ukdebug and can be activated using make menuconfig (Library Configuration -> ukdebug: Debugging and Tracing).

There are two types of messages:

• Kernel messages
  • Information(uk_pr_info)
  • Warnings(uk_pr_warn)
  • Errors(uk_pr_err)
  • Critical Messages(uk_pr_crit)
• Debug messages(uk_pr_debug)

Assertions

We can use assertions to check if the system is in a defined and stable state. Can be compiled-in or compiled-out and it can be activated from Library Configuration -> ukdebug: Debugging and Tracing -> Enable assertions.

The macros used can be:

• UK_ASSERT (condition)
• UK_BUGON (negative condition)
• UK_CTASSERT (condition)(used for compile-time assertions)

GDB

To use GDB we need the symbols from the gdb file generated at build time. For this we need to set Debug information level to Level 3 from make menuconfig (Build Options -> Debug information level -> Level 3).

Linux

For the Linux user space target (linuxu), simply point GDB to the resulting debug image:

$ gdb path_to_unikraft_gdb_image

KVM

For KVM we need to go through few steps:

1. Run guest in paused state

   Using qemu:

   $ qemu-guest -P -g 1234 -k path_to_unikraft_gdb_image
   

   Using kraft:

   $ kraft run -d -g 1234 -P
   

2. Attach debugger

   $ gdb --eval-command="target remote :1234" path_to_unikraft_gdb_image
   

3. Disconnect GDB

   disconnect
   

4. Set GDB’s machine architecture to x86_64

   $ set arch i386:x86-64:intel
   

5. Re-connect

   tar remote localhost:1234
   

Tracepoints

Tracepoints are provided by lib/ukdebug. To enable Unikraft to collect trace data, enable the option CONFIG_LIBUKDEBUG_TRACEPOINTS in your configuration (via make menuconfig under Library Configuration -> ukdebug -> Enable tracepoints).

Instrumenting

Instrumenting your code with tracepoints is done in two steps:

• Define and register a tracepoint handler with the UK_TRACEPOINT() macro.
• Place calls to the generated handler at the places in your code where your want to trace an event.

Reading traces

Unikraft is storing trace data to an internal buffer that resides in the guest’s main memory. To access that data you need to configure the GDB and add source /path/to/your/build/uk-gdb.py to ~/.gdbinit

Commands available in GDB:

Any saved trace file can be later processed with the trace.py .

$ support/s/uk_trace/trace.py list <file>

Work Items

Support Files

Session support files are available in the repository. If you already cloned the repository, update it and enter the session directory:

$ cd path/to/repository/clone

$ git pull --rebase

$ cd content/en/community/hackathons/sessions/complex-applications/


$ ls -F
images/  index.md  sol/  work/

If you haven’t cloned the repository yet, clone it and enter the session directory:

$ git clone https://github.com/unikraft/docs

$ cd docs/

$ cd content/en/community/hackathons/sessions/complex-applications/

$ ls -F
images/  index.md  sol/  work/

00. Qemu Wrapper

As we saw during the other sessions, qemu-guest is a wrapper script over the qemu-system-x86_64 executable, to make the use of binary less painful. In the following session, it will be very handy to use it. To see the options for this wrapper you can use qemu-guest -h.

It is possible to run a lot of complex applications on Unikraft. In this session we analyze 3 of them:

• SQLite
• Redis
• Nginx

01. SQLite (Tutorial)

The goal of this tutorial is to get you to set up and run SQLite on top of Unikraft. Find the support files in the work/01-set-up-and-run-sqlite/ folder of the session directory.

SQLite is a C library that implements an encapsulated SQL database engine that does not require any setting or administration. It is one of the most popular in the world and it differs from other SQL database engines because it is simple to administer, use, maintain, and test. Thanks to these features, SQLite is a fast, secure, and (most crucial) simple application.

The SQLite application is built using an external library, lib-sqlite, that depends on another library that is ported for Unikraft: Musl (a standard C library). To successfully compile and run the SQLite application for the KVM platform on the x86-64 architecture, we follow the steps below.

Setup

First, we make sure we have the directory structure to store the local clones of Unikraft, library and application repositories. The structure should be:

workdir
|-- unikraft/
|-- libs/
`-- apps/

We clone the lib-sqlite repository in the libs/ folder. The libraries on which lib-sqlite depends (Musl) are also to be cloned in the libs/ folder.

We clone the app-sqlite repository in the apps/ folder. In this directory, we need to create two files:

• Makefile: containing rules for building the application, as well as specifying the libraries that the application needs
• Makefile.uk: used to define variables needed to compile the application or to add application-specific flags

In the Makefile, the order in which the libraries are mentioned in the LIBS variable is important to avoid the occurrence of compilation errors.

UK_ROOT ?= $(PWD)/../../unikraft
UK_LIBS ?= $(PWD)/../../libs
LIBS := $(UK_LIBS)/lib-musl:$(UK_LIBS)/lib-sqlite

all:
	@$(MAKE) -C $(UK_ROOT) A=$(PWD) L=$(LIBS)

$(MAKECMDGOALS):
	@$(MAKE) -C $(UK_ROOT) A=$(PWD) L=$(LIBS) $(MAKECMDGOALS)

For the moment, Makefile.uk should look like this:

$(eval $(call addlib,appsqlite))

Configure

We configure the application by running:

$ make menuconfig

We select the SQLite library from the configuration menu, Library Configuration section. For starters, we select the option to generate the main source file used to run the application.

To import or export databases or CSV/SQL files, the SQLite application needs to configure a filesystem. The filesystem we use is 9pfs. Hence, in the Library Configuration section, we select the 9pfs filesystem within the vfscore library options.

As we are going to use the qemu wrapper to launch the app, we’ll need to name the Default root device fs0 (Library Configuration -> vfscore -> Default root device). This is due to how the qemu-guest script automatically tags the FS devices attached to qemu.

Make sure that both options Virtio PCI device support and Virtio 9P device are selected. They can be found in Platform Configuration -> KVM guest -> Virtio.

Build

We build the application by running:

$ make

Test

For testing we can use the following SQLite script, which inserts ten values into a table:

CREATE TABLE tab (d1 int, d2 text);
INSERT INTO tab VALUES (random(), cast(random() as text)),
(random(), cast(random() as text)),
(random(), cast(random() as text)),
(random(), cast(random() as text)),
(random(), cast(random() as text)),
(random(), cast(random() as text)),
(random(), cast(random() as text)),
(random(), cast(random() as text)),
(random(), cast(random() as text)),
(random(), cast(random() as text));

Up next, create a folder in the application folder called sqlite_files/ and write the above script into a file. When you run the application, you can specify the path of the newly created folder to the qemu-guest script as following:

$ ./qemu-guest -k ./build/app-sqlite_qemu-x86_64 \
               -e ./sqlite_files \
               -m 500

The SQLite start command has several parameters:

• k indicates the executable resulting from the build of the entire system together with the SQLite application
• e indicates the path to the shared directory where the Unikraft filesystem will be mounted
• m indicates the memory allocated to the application

To load the SQLite script, we use the following command .read <sqlite_script_name.sql>. In the end, we run select * from tab to see the contents of the table.

If everything runs as expected, then we’ll get the following output:

SeaBIOS (version 1.13.0-1ubuntu1.1)
Booting from ROM...
Powered by
o.   .o       _ _               __ _
Oo   Oo  ___ (_) | __ __  __ _ ' _) :_
oO   oO ' _ `| | |/ /  _)' _` | |_|  _)
oOo oOO| | | | |   (| | | (_) |  _) :_
 OoOoO ._, ._:_:_,\_._,  .__,_:_, \___)
                  Phoebe 0.10.0~9bf6e63
SQLite version 3.30.1 2019-10-10 20:19:45
Enter ".help" for usage hints.
sqlite> .read script.sql
sqlite> select * from tab;
-1758847760864160102|2718837905630364326
-1339730570270182734|-413022835704168293
899003099627700560|-5446400296487656477
3986823405912376844|-3683968660549484071
5750141151993138490|-949527979363852620
2608659443316808689|3543024197312456352
-2195896775588749426|6838623081517951948
8293933456345343304|6460961935619776014
6827842764477913763|7025795551657688644
4026439721321663478|8364502757469924828
sqlite> .exit

02. SQLite New Filesystem (Tutorial)

In the previous work item, we have chosen to use 9PFS as the filesystem. For this work item, we want to change the filesystem to InitRD and load the SQLlite script as we have done in the previous work item. Find the support files in the work/02-change-filesystem-sqlite/ folder of the session directory. Make sure to create the Makefile and Makefile.uk files in a similar manner as with the previous work item before proceeding in configuring the application.

First, we need to change the filesystem to InitRD. We can obtain that by using the command make menuconfig and from the vfscore: Configuration option, we select the default root filesystem as InitRD.

The InitRD filesystem can load only cpio archives. In order to load our SQLite script into InitRD, we need to create a cpio out of it. This can be achieved in the following way:

Create a folder, move the SQLite script in it, and cd into it.

After that we run the following command:

$ find -type f | bsdcpio -o --format newc > ../archive.cpio

We’ll obtain a cpio archive called archive.cpio in the parent directory.

Next, we run the following qemu-guest command to run the instance:

$ ./qemu-guest -k build/app-sqlite_qemu-x86_64 -m 100 -i archive.cpio

If everything runs as expected, then we’ll get the following output:

SeaBIOS (version 1.13.0-1ubuntu1.1)
Booting from ROM...
Powered by
o.   .o       _ _               __ _
Oo   Oo  ___ (_) | __ __  __ _ ' _) :_
oO   oO ' _ `| | |/ /  _)' _` | |_|  _)
oOo oOO| | | | |   (| | | (_) |  _) :_
 OoOoO ._, ._:_:_,\_._,  .__,_:_, \___)
                  Phoebe 0.10.0~9bf6e63
SQLite version 3.30.1 2019-10-10 20:19:45
Enter ".help" for usage hints.
sqlite> .read script.sql
sqlite> select * from tab;
-2854471077348014330|8890688652355553061
6848326607576863720|8057668357382476232
-4851485256049611772|1080284340194216118
3617801119133923790|-3742008368926465716
-8000990739986823138|603753214333179605
-1492560099439825568|-8062818652230049204
8818728981714743313|-1714591670076544373
-1304043959596685652|557566099797623154
-9196798118140052834|3433881783117867716
-4291436294037928857|6810153594571143752
sqlite> .exit

03. Redis (Tutorial)

The goal of this tutorial is to get you to set up and run Redis on top of Unikraft. Find the support files in the work/03-set-up-and-run-redis/ folder of the session directory.

Redis is one of the most popular key-value databases, with a design that facilitates the fast writing and reading of data from memory, as well as the storage of data on disk in order to be able to reconstruct the state of data in memory in case of a system restart. Unlike other data storage systems, Redis supports different types of data structures such as lists, maps, strings, sets, bitmaps, streams.

The Redis application is built using an external library, lib-redis, that depends on other ported libraries for Unikraft (musl and lwip library), all of which you should be familiar with by now. To successfully compile and run the Redis application for the KVM platform and x86-64 architecture, we follow the steps below.

Setup

As above, we make sure we have the directory structure to store the local clones of Unikraft, library and application repositories. The structure should be:

workdir
|-- unikraft/
|-- libs/
`-- apps/

We clone the lib-redis repository in the libs/ folder. We also clone the library repositories which lib-redis depends on (musl and lwip) in the libs/ folder.

We clone the app-redis repository in the apps/ folder. In this directory, we need to create two files:

• Makefile: it contains rules for building the application, as well as specifying the external libraries that the application needs
• Makefile.uk: used to define variables needed to compile the application or to add application-specific flags

In the Makefile, the order in which the libraries are mentioned in the LIBS variable is important to avoid the occurrence of compilation errors.

UK_ROOT ?= $(PWD)/../../unikraft
UK_LIBS ?= $(PWD)/../../libs
LIBS := $(UK_LIBS)/lib-musl:$(UK_LIBS)/lib-lwip:$(UK_LIBS)/lib-redis

all:
	@$(MAKE) -C $(UK_ROOT) A=$(PWD) L=$(LIBS)

$(MAKECMDGOALS):
	@$(MAKE) -C $(UK_ROOT) A=$(PWD) L=$(LIBS) $(MAKECMDGOALS)

For the moment, Makefile.uk should look like this:

$(eval $(call addlib,appredis))

Configure

We configure the application by running:

$ make menuconfig

We select the Redis library from the configuration menu, in the Library Configuration section. For starters, we select the option to generate the main source file used to run the application.

To connect to the Redis server, the network features should be configured. Hence, in the configuration menu in the Library Configuration section, within the lwip library the following options should be selected:

• IPv4
• UDP support
• TCP support
• ICMP support
• DHCP client
• Socket API

The Redis application needs a configuration file to start. Thus, a filesystem should be selected in the configuration menu. The filesystem we previously used was 9PFS.

As such, in the Library Configuration section of the configuration menu, the following selection chain should be made in the vfscore library: VFSCore Interface -> vfscore Configuration -> Automatically mount a root filesystem -> Default root filesystem -> 9PFS. Same as before, since we’ll be using the qemu-guest script, we’ll need to name the Default root device fs0 (Library Configuration -> vfscore -> Default root device).

Nevertheless, don’t forget to select the posix-event library: Library Configuration -> posix-event.

Build

We build the application by running:

$ make

Test

Following the steps above, the build of the entire system, together with the Redis application will be successful. We can create a redis_files directory in which we can place our configuration file for Redis, redis.conf. We used a script to run the application in which a bridge and a network interface (kraft0) are created. The network interface has an IP associated with it used by clients to connect to the Redis server. Also, the script takes care of starting the Redis server, but also of stopping it, deleting the settings created for the network.

sudo brctl addbr kraft0
sudo ifconfig kraft0 172.44.0.1
sudo ifconfig kraft0 up

sudo dnsmasq -d \
             -log-queries \
             --bind-dynamic \
             --interface=kraft0 \
             --listen-addr=172.44.0.1 \
             --dhcp-range=172.44.0.2,172.44.0.254,255.255.255.0,12h &> dnsmasq.logs &

./qemu-guest.sh -k ./build/app-redis_qemu-x86_64 \
                -a "/redis.conf" \
                -b kraft0 \
                -e ./redis_files \
                -m 100

The Redis server start command has several parameters:

• k indicates the executable resulting from the build of the entire system together with the Redis application
• e indicates the path to the shared directory where the Unikraft filesystem will be mounted
• b indicates the network interface used for external communication
• m indicates the memory allocated to the application
• a allows the addition of parameters specific to running the application

The following image is presenting an overview of our setup:

Consequently, after running the script, the Redis server will start and dnsmasq will act as a DHCP server and, therefore it will dynamically assign to our Unikraft instance an IP address. The IP can be seen in the output of qemu as bellow:

Booting from ROM...
en1: Added
en1: Interface is up
Powered by
o.   .o       _ _               __ _
Oo   Oo  ___ (_) | __ __  __ _ ' _) :_
oO   oO ' _ `| | |/ /  _)' _` | |_|  _)
oOo oOO| | | | |   (| | | (_) |  _) :_
 OoOoO ._, ._:_:_,\_._,  .__,_:_, \___)
                  Phoebe 0.10.0~9bf6e63
1:C 27 Aug 2022 12:37:07.023 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 27 Aug 2022 12:37:07.026 # Redis version=5.0.6, bits=64, commit=c5ee3442, modified=1, pid=1, just started
1:C 27 Aug 2022 12:37:07.031 # Configuration loaded
1:M 27 Aug 2022 12:37:07.049 * Increased maximum number of open files to 10032 (it was originally set to 1024).
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 5.0.6 (c5ee3442/1) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 1
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           http://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

1:M 27 Aug 2022 12:37:07.090 # Server initialized
1:M 27 Aug 2022 12:37:07.092 * Ready to accept connections
en1: Set IPv4 address 172.44.0.242 mask 255.255.255.0 gw 172.44.0.1

Another way of inspecting the received IP is through the dnsmasq.logs file:

$ cat dnsmasq.logs
[...]
dnsmasq-dhcp: DHCPOFFER(kraft0) 172.44.0.242 52:54:00:20:37:c1
dnsmasq-dhcp: DHCPDISCOVER(kraft0) 52:54:00:20:37:c1
dnsmasq-dhcp: DHCPOFFER(kraft0) 172.44.0.242 52:54:00:20:37:c1
dnsmasq-dhcp: DHCPREQUEST(kraft0) 172.44.0.242 52:54:00:20:37:c1
dnsmasq-dhcp: DHCPACK(kraft0) 172.44.0.242 52:54:00:20:37:c1

Using the received IP, it will be possible to connect clients to it using redis-cli (the binary redis-cli is the folder for this work item):

$ ./redis-cli -h 172.44.0.242 -p 6379
172.44.0.242:6379> PING
PONG
172.44.0.242:6379>

Nevertheless, after completing this task, you will need to check that the dnsmasq process is not running anymore, as it messes up your other connections. You can also disable and delete the bridge interface by running the following commands:

$ sudo ip l set dev kraft0 down
$ sudo brctl delbr kraft0

04. Redis Static IP Address

In tutorial above we have dynamically assigned an IP to the network interface used by Unikraft using the dnsmasq utility. Find the support files in the work/04-obtain-the-ip-statically/ folder of the session directory.

Modify the launching script and run the application with a static IP. Beware that the assigned IP address must differ from the one assigned on the bridge.

You can use redis-cli, found in the suport folder to test your changes. If everything runs as expected you should see the following output:

$ ./redis-cli -h 172.88.0.2 -p 6379
172.88.0.2:6379> PING
PONG
172.88.0.2:6379>

05. Redis Benchmarking (Tutorial)

We aim to benchmark the Redis app running on top of Unikraft and Redis running on top of Linux. Find the support files in the work/05-benchmark-redis/ folder of the session directory. There are three binaries: redis-cli, redis-benchmark, and redis-server.

First, we will start by benchmarking app-redis on Unikraft. Start Redis on top of Unikraft as we did before and in another terminal run the following command:

$ ./redis-benchmark --csv -q -r 100 -n 10000 -c 1 -h 172.88.0.2 -p 6379 -P 8 -t set,get

The description of the used options can be seen here:

Usage: redis-benchmark [-h <host>] [-p <port>] [-c <clients>] [-n <requests>] [-k <boolean>]

 -h <hostname>      Server hostname (default 127.0.0.1)
 -p <port>          Server port (default 6379)
 -c <clients>       Number of parallel connections (default 50)
 -n <requests>      Total number of requests (default 100000)
 -P <numreq>        Pipeline <numreq> requests. Default 1 (no pipeline).
 -q                 Quiet. Just show query/sec values
 --csv              Output in CSV format
 -t <tests>         Only run the comma separated list of tests. The test
                    names are the same as the ones produced as output.

If everything runs as expected, you’ll see the following output:

"SET","265252.00"
"GET","276701.72"

The printed values represent requests/second for the set and get operations.

Further, we will run the executable redis-server (./redis-server), which can be found in the support folder, and the following command (only the IP address of the redis server was changed):

$ ./redis-benchmark --csv -q -r 100 -n 10000 -c 1 -h 127.0.0.1 -p 6379 -P 8 -t set,get

The output should be similar to this:

"SET","495785.84"
"GET","514138.81"

06. Nginx

The aim of this work item is to set up and run Nginx, a popular open-source web server. Find the support files in the work/06-set-up-and-run-nginx/ folder of the session directory.

From the point of view of the library dependencies, the Nginx app has the same dependencies as the Redis app. Of course, instead of lib-redis, the Nginx app depends on an external library called lib-nginx. You can clone this library in the libs/ directory in a similar manner as with the previous work items. It’s your choice how you assign the IP to the VM.

In the support folder of this work item there is a subfolder called nginx with the following structure:

nginx_files
`-- nginx/
    |-- conf/
    |   |-- fastcgi.conf
    |   |-- fastcgi_params
    |   |-- koi-utf
    |   |-- koi-win
    |   |-- mime.types
    |   |-- nginx.conf
    |   |-- nginx.conf.default
    |   |-- scgi_params
    |   |-- uwsgi_params
    |   `-- win-utf
    |-- data/
    |   `-- images/
    |       `-- small-img100.png
    |-- html/
    |   |-- 50x.html
    |   `-- index.html
    `-- logs/
        |-- error.log
        `-- nginx.pid

The path to the nginx_files folder should be given as a parameter to the -e option of the qemu-guest. The html/ folder stores the files of the website you want to be run.

If everything works as expected, you should see the following web page in the browser.

07. Nginx Benchmarking (Tutorial)

Benchmarking Nginx running on top of Unikraft can be achieved with a utility called iperf. The package can be easily installed using the command:

sudo apt-get install -y iperf

Next, we will start the nginx app as we have done with the previous work item, and then we will open one additional terminal.

We’ll start an iperf client by connecting to the Nginx server running on top of Unikraft with the command:

$ iperf -c 172.44.0.76 -p 80

If everything runs as expected, then we will see the following output:

------------------------------------------------------------
Client connecting to 172.44.0.76, TCP port 80
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[  3] local 172.44.0.1 port 33262 connected with 172.44.0.76 port 80
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.28 GBytes  1.10 Gbits/sec

08. Quiz

If you reached this far, please do a 5 minutes tour through this quiz to assess your current understanding of the session.

09. Give Us Feedback

We want to know how to make the next sessions better. For this, we need your feedback. Thank you!