Emulating Embedded Linux Systems with QEMU
Emulating Embedded Linux Systems with QEMU
1. Introduction
Embedded software development relies on embedded hardware devices, such as development boards, external module devices, etc., but if the debugging work has nothing to do with peripherals, only the kernel debugging can be simulated using QEMU without purchasing hardware.
It’s available for Linux and Windows hosts and emulated PowerPC, ARM, MIPS, and SPARC targets. QEMU takes the approach of providing a minimal translation layer between the host and target processor. The host processor is the one running the emulator, and the target processor is what’s being emulated.
The following is a detailed introduction to the process of setting up QEMU development environment.
2. Environment
2.1 Used environment
* Ubuntu-18.04.1
OR:
* PC:Windows10
* Virtual Machine:VirtualBox-5.18
* Virtual OS:Ubuntu-18.04.1
* Simulated development board: vexpres
2.2 Tools used when setting up the environment
* qemu-4.2.0
* linux-4.14.172 (Linux Kernel)
* u-boot-2017.05
* busybox-1.31.1
* arm-linux-gnueabi-gcc
Put all related files in /home/joe/qemu
3. Install cross-compilation tools
# sudo apt install gcc-arm-linux-gnueabi
Check if the installation is successful
$ arm-linux-gnueabi-gcc -v
Using built-inspecs. COLLECT_GCC=arm-linux-gnueabi-gcc COLLECT_LTO_WRAPPER=/usr/lib/gcc-cross/arm-linux-gnueabi/7/lto-wrapper Target: arm-linux-gnueabi Configured with: ../src/configure -v –with-pkgversion=’Ubuntu/Linaro 7.5.0-3ubuntu1~18.04′–with-bugurl=file:///usr Thread model: posix gcc version 7.5.0(Ubuntu/Linaro 7.5.0-3ubuntu1~18.04) |
4. Configure and Compile the Linux kernel
4.1 Download Linux Kernel
Download the required kernel version from www.kernel.org.
Here I download the relatively latest long-term supported kernel version linux-4.4.157
wget https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.4.157.tar.xz to /qemu directory
4.2 Unzip the Linux kernel
# tar xvJf linux-4.4.157.tar.xz
4.3 Compile Linux Kernel
// Enter the kernel source file directory
# cd linux-4.4.157
make CROSS_COMPILE=arm-linux-gnueabi- ARCH=arm vexpress_defconfig
make CROSS_COMPILE=arm-linux-gnueabi- ARCH=arm menuconfig
If running menuconfig shows that the ncurses package is missing, just run the following command to install it)
$ sudo apt-get install libncurses5-dev
Enter the menu configuration and make the following settings
Compile with cross toolchain
After successful compilation, Generate a kernel image file under the directory
arch/arm/boot, zImage and dtb can be copied into a separate folder for convenient use
5. Install QEMU Tools
5.1 Install QEMU
* wget https://download.qemu.org/qemu-4.2.0.tar.xz
* tar xvJf qemu-4.2.0.tar.xz
* cd qemu-4.2.0
5.2 Install dependent packages before configuring QEMU
# apt install zlib1g-dev
# apt install libglib2.0-0 libglib2.0-dev
# apt install libsdl1.2-dev
# apt install libpixman-1-dev libfdt-dev
In order to prevent the files from being messy after compilation, create the builder directory as the intermediate target path for compilation.
Configure, compile and install QEMU.
5.3 Configure QEMU to support all boards under the arm architecture
# ../configure –target-list=arm-softmmu –audio-drv-list=
If pixman is missing when the following prompt appears,
use sudo apt-get install libpixman-1-dev to install it.
5.4 View QEMU version
5.5 View development boards supported by QEMU
5.6 Run QEMU
# qemu-system-arm -M vexpress-a9 -m 512M -kernel ./zImage -dtb ./vexpress-v2p-ca9.dtb -nographic -append “console=ttyAMA0”
OR:
$pwd
/home/joe/qemu
# qemu-system-arm -M vexpress-a9 -m 512M -kernel linux-.4.157/arch/arm/boot/zImage -dtb linux-4.4.157/arch/arm/boot/dts/vexpress-v2p-ca9.dtb -nographic -append “console=ttyAMA0”
In order to better testing and start qemu, you can create the startup script start.sh, and give the script permission to run chmod +x start.sh
#!/bin/bash
qemu-system-arm \
-M vexpress-a9 \
-m 512M \
-kernel /home/joe/jemu/linux-4.4.157/arch/arm/boot/zImage \
-dtb /home/joe/jemu/linux-4.4.157/arch/arm/boot/dts/vexpress-v2p-ca9.dtb \
-nographic \
-append “console=ttyAMA0”
6. Make a root file system
Use busybox to make a simple root file system.
6.1 Download busybox tool
Download busybox from https://busybox.net/downloads/
# wget https://busybox.net/downloads/busybox-1.31.1.tar.bz2
# tar xjvf busybox-1.31.1.tar.bz2
# cd busybox-1.31.1
# make defconfig
# make CROSS_COMPILE=arm-linux-gnueabi-
# make install CROSS_COMPILE=arm-linux-gnueabi-
The following information is prompted, indicating that the installation is successful.
After the installation is complete, the generated target file defaults to the ./_install directory.
6.2 Generate root file system
6.2.1 compile and install busybox
# mkdir rootfs
# sudo cp -r _install/* rootfs/
6.2.2 Add glibc library, add loader and dynamic library in the root file system
# sudo cp -r _install/* rootfs/
# sudo cp -p /usr/arm-linux-gnueabi/lib/* rootfs/lib/
6.2.3 Create 4 tty terminal devices (c stands for character device, 4 is the major device number, and 1~4 are the minor device numbers respectively)
6.3 Make SD card file system image
6.3.1 Generate an empty SD card image
# dd if=/dev/zero of=rootfs.ext3 bs=1M count=32
6.3.2 Format SD card as exts file system
# mkfs.ext3 rootfs.ext3
6.3.3 Burn rootfs to SD card
# sudo mount -t ext3 rootfs.ext3 /mnt -o loop
# sudo cp -rf rootfs/* /mnt/
# sudo umount /mnt
7. Verify
7.1 Start Qemu
Run the following command to test, check if the compiled kernel can run successfully
# sudo qemu-system-arm -M vexpress-a9 -m 512M -kernel ~/qemu/zImage –dtb ~/qemu/vexpress-v2p-ca9.dtb -nographic -append “console=ttyAMA0”
Or using Script:
In the above test, the kernel will report panic, suggesting that we lack the root file system.
The above problem is due to the busybox tool generated inn the x86 environment.
We used make install when install busybox, so you should use
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- install |
The compilation tool generates the busybox tool used by the arm platform
# file rootfs/bin/busybox
rootfs/bin/busybox: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-, for GNU/Linux 3.2.0, BuildID[sha1]=cbcd33b8d6c946cb19408a5e8e714de554c87f52, stripped
7.2 Verify again
Now, Qemu has started the Linux kernel and mounted the file system successfully, and can interact with the system with simple functions through the serial terminal. The problem of not being able to run /etc/init.d/rcS in the printing process, you only need to add the /etc/init.d/rcS file. The content of the file can be a prompt statement.
7.3 Exit QEMU
Two ways to exit qemu
* In another terminal input: kill all qemu-system-arm
* In Qemu input: Ctrl+ A; X
QEMU: Terminated
8. Start the Linux kernel through u-boot
Embedded system usually include: u-boot, kernel, rootfs, and appfs. The positional relationship of these parts on the ARM development board shown in the figure below
BootLoader | BootParameters | Kernel | Rootfs | Appfs |
Rootfs can run in board or PC
8.1 Prepare U-boot
8.1.1 Download u-boot
http://ftp.denx.de/pub/u-boot/, we use: u-boot-2021.01.tar.bz2
# tar -jxvf u-boot-2018.09.tar.bz2
8.1.2 Compile u-boot
# vim Makefile
CROSS_COMPILE = arm-linux-gnueabi-
# vim config.mk
ARCH = arm
# make vexpress_ca9x4_defconfig, error
Need : sudo apt install bison
sudo apt install flex
then: # make -j4 error
Need : export CROSS_COMPILE=arm-linux-gnueabi-
export ARCH=arm
again: # make vexpress_ca9x4_defconfig
# make -j4
8.1.3 Test, start u-boot
$ sudo qemu-system-arm -M vexpress-a9 -m 512M -kernel u-boot-2021.01/u-boot –nographic
8.2 Kernel configuration compilation
Use u-boot to boot the kernel image:
Need to compile the kernel into uImage format,
Need to specify the load address of uImage in memory
Specify when compiling the kernel: make LOADADDR=? uImage -j4
# cd /home/joe/qemu/linux-4.4.157
# make LOADADDR=0x60003000 uImage -j4
After u-boot compilation is finished, a mkimage file will be generated under the tool folder, copy this file to the bin folder under the cross compiler directory.
$ cd qemu/linux-4.4.157
Error:
$ sudo apt install u-boot-tools
Get uImage
9. QEMU network function settings
When the Qemu virtual machine starts on u-boot, uImage needs to be loaded into the memory, and uImage can be downloaded to the specified address in the memory through the TFTP server.
9.1 Check whether the host kernel supports the tun/tap module
// Install the two tools that the bridged network depends on
# sudo apt install uml-utilities bridge-utils
Create tun device file: /dev/net/tun (usually created automatically)
Modify the /etc/network/interfaces (configure the network, restart to take effect)
# sudo vim /etc/network/interfaces
auto loiface lo inet loopbackauto enp0s3 // name of virtual network cardauto br0iface br0 inet dhcpbridge_ports enp0s3 |
9.2 Reboot
# reboot
Then check Qemu’s network environment
The virtual network port br0 is the network port for the communication between the Qemu virtual machine and the Linux host.
10. Install TFTP server
Create a TFTP server to download uImage to the memory when launching uImage for the Qemu simulation development board
10.1 Install tftp tool
$ apt-get install tftp-hpa tftpd-hpa xinetd
10.2 Modify the configuration file and set the TFTP server directory
# sudo vim /etc/default/tftpd-hpa
…… TFTP_DIRECTORY=”/home/joe/tftpboot” …… |
10.3 Create a tftp directory on the Linux host
# mkdir /home/joe/tftpboot
# chmod 777 /home/joe/tftpboot
10.4 Restart the tftp service
# sudo /etc/init.d/tftpd-hpa restart
10.5 Set kernel startup parameters in u-boot
copy uImage and cexpress-v2p-ca9.dtb to tftpboot
Start Qemu to verify
$ sudo qemu-system-arm -M vexpress-a9 -m 512M -kernel u-boot-2021.01/u-boot –nographic -net nic,vlan=0 -net tap,vlan=0,ifname=tap0 -sd rootfs.ext3
Now, the rootfs directory is a simple root file system, which can be made into a mirror file, and the mirror file can be burned to the development board, or the Linux kernel can be started by u-boot in Qemu and mounted on the mirror file. It can also be set to boot via NFS network file system.
11. Mount NFS file system
11.1 Install and configure NFS service
11.1.1 Install
$ sudo apt install nfs-kernel-server
11.1.2 Configutation
$ sudo mkdir /home/joe/qemu/rootfs
$ sudo chown nobody:nogroup /home/joe/qemu/rootfs
$ sudo chmod 777 /home/joe/qemu/rootfs
$ sudo nano /etc/exports
Add: /home/joe/qemu/rootfs *(rw,sync,no_root_squash)
Restart nfs server:
$ sudo /etc/init.d/nfs-kernel-server restart
Or: $systemctl restart nfs-kernel-server
Check whether the NFS shared directory is created
$ sudo showmount –e
When using the NFS network file system, the Linux host needs to close the system firewall, otherwise, abnormalities will occur when the system is running.
Conclusion
Hopefully, with the help of this blog, you know more about QEMU. All the techniques demonstrated above were used in various submissions to our program. There’s not a single, fixed way to emulate with QEMU. Explore different techniques and see what works for you. Familiarize yourself with the knowledge and you will be surprised at how it can help you in unexpected ways.