Saturday, July 18, 2015

Creating a custom Linux BSP (including kernel) for the Raspberry Pi with Yocto 1.8 Fido


This guide provides directions for building a custom Linux kernel, bootloader, and root filesystem for the Raspberry Pi.  This guide was created with the following purposes.
  • To utilize the Yocto project and it's native support of the Raspberry Pi for building a custom kernel and board support package for the Raspberry Pi.
  • To Provide the ability to quickly revert to different (new and old) kernel versions and apply version specific patch sets with different kernel configurations when building a BSP image for the Raspberry Pi with Yocto.
  • To store kernel configurations and kernel patches that are specific to a a version of a kernel in a single directory that has been named with the version of the kernel. i.e. linux-*_3.18.5/
  • To follow the procedures that Freescale and Boundary Devices used for integrating multiple kernels versions into the meta-fsl and meta-fsl-arm layers.
  • To build the boot loader and kernel in separate pieces for the Raspberry Pi.
  • To exclusively utilize Yocto and bitbake for building the Linux kernel and boot loader without having to copy kernel config files between directories in the build tree.
  • Image size is not an issue since this image will be used for internal development and research. The image will contain all of the compilers and dev tools.

Creating a custom Linux distribution for the Raspberry Pi is very similar to creating a custom Linux distribution for the RIoTboard.  With only a few modifications to the Yocto supported meta-raspberry pi layer, a BSP can be created for the Raspberry Pi that will allow one to easily modify the GNU/Linux kernel and U-Boot bootloader for the Pi.  Rather than making direct modifications to the meta-raspberry pi layer, modifications will be made in a cloned layer.  The cloned layer will have a new name. Kernel and and boot loader configuration and compile changes can subsequently be made in the cloned layer.


The board support package that will be built for the Raspberry Pi will consist of U-boot, a GNU/Linux kernel, and a root filesystem.  A 64-bit Fedora 21 GNU/Linux host will be used to compile all of the source components.

The Yocto project already has support for the Raspberry Pi.  There is a hardware specific BSP overlay layer in the Yocto project called meta-raspberrypi.  The layer contains machine specific tunings, compile time parameters, kernel configuration options, and boot loader parameters that are specific to the Raspberry Pi hardware.  Other items such as the GPU/CPU memory split, ARM and GPU processor frequencies, and SDRAM frequency are also set in the pre-existing bsp layer.


This guide consists of the following steps.

1. Install required Packages for a 64-bit Fedora 21 Host development system
2. Pull the Yocto Raspberry Pi sources.
3. Clone the Yocto meta-raspberrypi layer.
4. Clone the two preexisting machines in the meta-raspberrypi layer.
5. Make everything compatible with the two new machines.
6. Replace raspberrypi with the name of the new layer, slicedpi, in layer.conf.
7. Clone the RPI hardware up image.
8. Modify the new SpecialPi image.
9. Add the meta-slicedpi layer to BBLAYERS.
10. Set the MACHINE name
11. Fix the kernel recipe include file
12. Configure local.conf.
13. Create a custom kernel defconfig file for GNU/Linux kernel 3.18.5.
14. Customize busybox.
15. Check the output of the build.
16. Kick off the build and burn the image to an SD/uSD card.
17. Validate the build.
18. A few notes.

Hardware and software prerequisites

 

Host Operating System

Fedora release 21 for x86_64
GNU/Linux kernel 4.0.7-200.fc21.x86_64 #1 SMP

 

Host Hardware

Intel(R) Core(TM) i7-4600U CPU @ 2.10GHz
Physical Memory 12 GB
My /home partition is 414 GB and resides on a solid state hard drive.



The Yocto build requires a large amount of space so I recommend keeping a scratch area on your /home partition of at least 100 GB.  You can get away with less space but if you decide to make changes to the source and binaries within the Yocto tree later on down the line, then it is a good idea to have extra space.

1. Install required Packages for a 64-bit Fedora 21 Host development system


from the Yocto Project Mega Manual Revision 1.8

Execute the following commands on the development host.

 host]$ sudo yum install gawk make wget tar bzip2 gzip python unzip perl patch \
        diffutils diffstat git cpp gcc gcc-c++ glibc-devel texinfo chrpath \
        ccache perl-Data-Dumper perl-Text-ParseWords perl-Thread-Queue socat \
        findutils which
 host]$ sudo yum install SDL-devel xterm perl-Thread-Queue
 host]$ sudo yum install make docbook-style-dsssl docbook-style-xsl \
        docbook-dtds docbook-utils fop libxslt dblatex xmlto xsltproc
 host]$ sudo yum install autoconf automake libtool glib2-devel

2. Pull the Yocto Raspberry Pi sources.

Execute the following commands on the development host.


host]$ mkdir $HOME/bin
host]$ curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
host]$ chmod a+x $HOME/bin/repo
host]$ echo "PATH=$PATH:$HOME/bin" >> $HOME/.bashrc
host]$ source .bashrc
host]$ mkdir -p $HOME/src/rpi
host]$ cd $HOME/src/rpi
host]$ git clone git://git.yoctoproject.org/poky -b fido
host]$ cd poky
host]$ git clone git://git.yoctoproject.org/meta-raspberrypi -b fido
host]$ git clone git://git.openembedded.org/meta-openembedded -b fido
host]$ source oe-init-build-env build
host]$ export RPIHOME=$HOME/src/rpi/poky

3. Clone the Yocto meta-raspberrypi layer.

Execute the following commands on the development host.

host]$ cd $RPIHOME
host]$ cp -R meta-raspberrypi meta-slicedpi

4. Clone the two preexisting machines in the meta-raspberrypi layer.

Execute the following commands on the development host.


host]$ cd meta-slicedpi/conf/machine/
host]$ mv raspberrypi.conf slicedpimach.conf
host]$ mv raspberrypi2.conf slicedpi2mach.conf

5. Make everything compatible with the two new machines.


It appears that COMPATIBLE_MACHINE has been set to raspberrypi in more than a handful of files. So let's fix COMPATIBLE_MACHINE by replacing the rval in the assignment expression with the names of the two new machines OR'd together.  All of the recipes need to be machine compatible with both the raspberrypi and raspberrypi2. These two machines were cloned and renamed slicedpimach and slicedpi2mach in step 4.

Execute the following commands on the development host.

host]$ find . -type f -exec sed -i \
's/COMPATIBLE_MACHINE = "raspberrypi"/COMPATIBLE_MACHINE = "(slicedpimach|slicedpi2mach)"/g' {} +

6. Replace raspberrypi with the name of the new layer, slicedpi, in layer.conf

Execute the following commands on the development host.


host]$ find . -name layer.conf -exec sed -i \
's/raspberrypi/slicedpi/g' {} +

7. Clone the RPI hardware up image.

Copy the rpi hardware up image to a new image called rpi-slicedpi-image as follows. Since this image is almost empty, having it in place means it can easily be customized later on.

Execute the following commands on the development host.

host]$ cp meta-slicedpi/recipes-core/images/rpi-hwup-image.bb \
meta-slicedpi/recipes-core/images/rpi-specialpi-image.bb

8. Modify the new specialpi Image

Modify meta-slicedpi/recipes-core/images/rpi-specialpi.bb so that rpi-specialpi-image.bb looks like the following

include recipes-core/images/core-image-minimal.bb
IMAGE_FEATURES += "dev-pkgs tools-sdk tools-debug tools-profile tools-testapps \
                   debug-tweaks ssh-server-openssh package-management"
# Include modules in rootfs
IMAGE_INSTALL += " \
    packagegroup-core-boot \
    packagegroup-core-full-cmdline \
    packagegroup-core-tools-profile \
    packagegroup-core-buildessential \
    kernel-modules \
    ${CORE_IMAGE_EXTRA_INSTALL} \
    kernel-devsrc \
    "
IMAGE_ROOTFS_EXTRA_SPACE_append += "+ 3000000"

9. Add the meta-slicedpi layer to BBLAYERS

Add the meta-slicedpi layer path to the BBLAYERS variable in build/conf/bblayers.conf so that bblayers.conf looks like the following


# LAYER_CONF_VERSION is increased each time build/conf/bblayers.conf
LCONF_VERSION = "6"

BBPATH = "${TOPDIR}"
BBFILES ?= ""

BBLAYERS ?= " \
  /home/ns/src/rpi/poky/meta \
  /home/ns/src/rpi/poky/meta-yocto \
  /home/ns/src/rpi/poky/meta-yocto-bsp \
  \
  /home/ns/src/rpi/poky/meta-slicedpi \
  "
BBLAYERS_NON_REMOVABLE ?= " \
  /home/ns/src/rpi/poky/meta \
  /home/ns/src/rpi/poky/meta-yocto \
  "

10. Set the MACHINE name

For Raspberry Pi Model B,
set MACHINE in build/conf/local.conf above the MACHINE ?? = "qemux86" line as follows

MACHINE ?= "slicedpimach"

For Raspberry Pi 2 or Model B+,
set MACHINE in build/conf/local.conf above the MACHINE ?? = "qemux86" line as follows

MACHINE ?= "slicedpi2mach"

11. Fix the kernel recipe include file

Execute the following commands on the development host.

host]$ find . -name linux-raspberrypi.inc -exec sed -i \
's/KERNEL_DEFCONFIG_raspberrypi/KERNEL_DEFCONFIG_slicedpimach/g' {} +
host]$ find . -name linux-raspberrypi.inc -exec sed -i \
's/KERNEL_DEFCONFIG_raspberrypi2/KERNEL_DEFCONFIG_slicedpi2mach/g' {} +

12. Configure local.conf

A source mirror will be setup along with source licences to accompany each binary that is built, and source archives for all of the packages. Add the following lines to the end of build/conf/local.conf file

PREFERRED_PROVIDER_virtual/libgl ?= "userland"
PREFERRED_PROVIDER_virtual/libgles2 = "mesa"
PREFERRED_PROVIDER_virtual/egl = "mesa"
# CONF_VERSION is increased each time build/conf/ changes incompatibly and is used to
# track the version of this file when it was generated. This can safely be ignored if
# this doesn't mean anything to you.
CONF_VERSION = "1"
DL_DIR ?= "${TOPDIR}/downloads"
ACCEPT_FSL_EULA = ""
INHERIT += "archiver"
ARCHIVER_MODE[src] = "original"
COPY_LIC_MANIFEST = "1"
COPY_LIC_DIRS = "1"
SOURCE_MIRROR_URL ?= "file://${TOPDIR}/source-mirror/"
INHERIT += "own-mirrors"
BB_GENERATE_MIRROR_TARBALLS = "1"

13. Create a custom kernel defconfig file for GNU/Linux kernel 3.18.5

Execute the following commands on the development host

host]$ cd $RPIHOME/meta-slicedpi/recipes-kernel
host]$ mkdir linux-raspberrypi-3.18.5
host]$ touch linux-raspberrypi-3.18.5
host]$ cd linux-raspberrypi-3.18.5

If you are building for the Raspberry Pi B
execute the following commands on the development host

host]$ curl https://raw.githubusercontent.com/raspberrypi/linux/rpi-3.18.y/arch/arm/configs/bcmrpi_defconfig \
> defconfig

If you are building for the Raspberry Pi 2 or Model B+
execute the following commands on the development host

host]$ curl https://raw.githubusercontent.com/raspberrypi/linux/rpi-3.18.y/arch/arm/configs/bcm2709_defconfig \
> defconfig

Customize the kernel configuration by statically building AES XTS, AES CTS, and SHA512 support into the kernel.

host]$ sed -i -e 's/CONFIG_CRYPTO_XTS=m/CONFIG_CRYPTO_XTS=y/g' defconfig
host]$ sed -i -e 's/CONFIG_CRYPTO_CTS=m/CONFIG_CRYPTO_CTS=y/g' defconfig
host]$ sed -i -e 's/CONFIG_CRYPTO_SHA512=m/CONFIG_CRYPTO_SHA512=y/g' defconfig

Create meta-slicedpi/recipes-kernel/images/linux-raspberrypi-3.18.5.bbappend with the following contents

EXTRAPATHS_prepend := "${THISDIR}/${PN}-${PV}:"
SRC_URI += "file://defconfig"

It makes more sense to keep version specific kernel patches in a directory that is named in accordance with the kernel patch versions that are stored within it.  Thus, move the i2c kernel patch for the 3.18.5 kernel from the linux-raspberrypi directory to the linux-raspberrypi_3.18.5 directory.

host]$ cd $RPIHOME/meta-slicedpi/recipes-kernel/linux/linux-raspberrypi_3.18.5
host]$ mv ../linux-raspberrypi/*.patch .

Modify meta-slicedpi/recipes-kernel/images/linux-raspberrypi-3.18.bb with the following contents

require linux.inc

DESCRIPTION = "Linux Kernel for Raspberry Pi"
SECTION = "kernel"
LICENSE = "GPLv2"
LIC_FILES_CHKSUM = "file://COPYING;md5=d7810fab7487fb0aad327b76f1be7cd7"

SRC_URI += " \
        file://defconfig \
        "
COMPATIBLE_MACHINE = "(slicedpimach|slicedpi2mach)"
PV = "${LINUX_VERSION}"

# NOTE: For now we pull in the default config from the RPi kernel GIT tree.
KERNEL_DEFCONFIG_slicedpimach ?= "bcmrpi_defconfig"
KERNEL_DEFCONFIG_slicedpi2mach ?= "bcm2708_defconfig"

# CMDLINE for raspberrypi
CMDLINE = "dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 root=/dev/mmcblk0p2 rootfstype=ext4 rootwait"

UDEV_GE_141 ?= "1"

# Set programmatically some variables during recipe parsing
# See http://www.yoctoproject.org/docs/current/bitbake-user-manual/bitbake-user-manual.html#anonymous-python-functions
python __anonymous () {
    kerneltype = d.getVar('KERNEL_IMAGETYPE', True)
    kerneldt = d.getVar('KERNEL_DEVICETREE', True)

    # Add dependency to 'rpi-mkimage-native' package only if RPi bootloader is used with DT-enable kernel
    if kerneldt:
        if kerneltype != 'uImage' and len(kerneldt.strip()) > 1:
            depends = d.getVar("DEPENDS", True)
            depends = "%s rpi-mkimage-native" % depends
            d.setVar("DEPENDS", depends)
}

do_kernel_configme_prepend() {
    install -m 0644 ${S}/arch/${ARCH}/configs/${KERNEL_DEFCONFIG} ${WORKDIR}/defconfig || die "No default configuration for ${MACHINE} / ${KERNEL_DEFCONFIG} available."
}

do_install_prepend() {
    install -d ${D}/lib/firmware
}

do_deploy_append() {
    # Deploy cmdline.txt
    install -d ${DEPLOYDIR}/bcm2835-bootfiles
    echo "${CMDLINE}" > ${DEPLOYDIR}/bcm2835-bootfiles/cmdline.txt
}

do_rpiboot_mkimage() {
    if test "x${KERNEL_IMAGETYPE}" != "xuImage" ; then
        if test -n "${KERNEL_DEVICETREE}"; then
            # Add RPi bootloader trailer to kernel image to enable DeviceTree support
            ${STAGING_DIR_NATIVE}/usr/lib/rpi-mkimage/mkknlimg --dtok ${KERNEL_OUTPUT} ${KERNEL_OUTPUT}
        fi
    fi
}
addtask rpiboot_mkimage before do_install after do_compile

PREFERRED_VERSION_linux-raspberrypi ?= "3.18.%" is currently set in meta-slicedpi/conf/machine/include/rpi-default-version.inc

Additional kernel patches can be placed in meta-slicedpi/recipes-kernel/images/linux-raspberrypi and then added to SRC_URI in meta-specialpi/recipes-kernel/images/linux-raspberrypi-3.18.bb

14. Customize Busybox

Configuration fragments can be created for customizing the Busybox configuration.  To start with, copy over the example configuration fragment from the busybox directory in the meta-skeleton layer.

For now, there isn't a need to customize busybox so the bbappend file is copied in from the skeleton layer directory in case it is needed later on.  There is a configuration fragment in the busybox subdirectory for setting the norfkill configuration option in busybox.

Execute the following commands on the development host.

host]$ cd $RPIHOME/meta-slicedpi
host]$ cp -R ../meta-skeleton/recipes-skeleton/recipes-core/busybox recipes-core/
host]$ cd recipes-core/busybox


Additional configuration options can be added by creating a configuration fragment file in the busybox/ subdirectory, inserting the configuration option from the busybox .config file into the configuration fragment file, and then appending the name of the configuration fragment file to SRC_URI in busybox_%.bbappend.

15. Kick off the build and burn the image to an SD/uSD card


For Raspberry Pi Model B, insert an SD card into the host.

Execute the following commands on the development host.

host]$ cd $RPIHOME/build
host]$ bitbake rpi-specialpi-image
host]$ cd tmp/deploy/images/raspberrypi
host]$ dd if=specialpi-image-raspberrypi.rpi-sdimg off=/dev/sd<X> bs=1M
host]$ sync

For Raspberry Pi 2 or Model B+, insert a uSD card into the host.

Execute the following commands on the development host.

host]$ cd $RPIHOME/build
host]$ bitbake rpi-specialpi-image
host]$ cd tmp/deploy/images/raspberrypi
host]$ dd if=rpi-specialpi-image-slicedpimach.rpi-sdimg of=/dev/sdb bs=1M
host]$ sync

16. Check the output of the build

The build output directory should contain multiple files - a kernel uImage, a rootfs image, a manifest files, etc.

Execute the following command on the development host and validate the output of the command.  Depending on how many builds have been executed, there will be more than one type of each file in the directory. For example, there are multiple sd card images in the slicedpimach directory below because mult

host]$ ls -lR build/tmp/deploy/images/slicedpimach
total 18070236
drwxr-xr-x. 2 ns ns       4096 Jul 18 21:36 bcm2835-bootfiles
-rw-r--r--. 1 ns ns 3196059648 Jul 18 18:23 rpi-specialpi-image-slicedpimach-20150718221335.rootfs.ext3
-rw-r--r--. 1 ns ns      60339 Jul 18 18:23 rpi-specialpi-image-slicedpimach-20150718221335.rootfs.manifest
-rw-r--r--. 1 ns ns 3221225472 Jul 18 18:24 rpi-specialpi-image-slicedpimach-20150718221335.rootfs.rpi-sdimg
-rw-r--r--. 1 ns ns   21262054 Jul 18 18:23 rpi-specialpi-image-slicedpimach-20150718221335.rootfs.tar.bz2
-rw-r--r--. 1 ns ns 5125439488 Jul 18 20:38 rpi-specialpi-image-slicedpimach-20150718233110.rootfs.ext3
-rw-r--r--. 1 ns ns     153004 Jul 18 20:36 rpi-specialpi-image-slicedpimach-20150718233110.rootfs.manifest
-rw-r--r--. 1 ns ns 5150605312 Jul 18 20:43 rpi-specialpi-image-slicedpimach-20150718233110.rootfs.rpi-sdimg
-rw-r--r--. 1 ns ns  291233698 Jul 18 20:38 rpi-specialpi-image-slicedpimach-20150718233110.rootfs.tar.bz2
-rw-r--r--. 1 ns ns 5125439488 Jul 18 21:24 rpi-specialpi-image-slicedpimach-20150719021250.rootfs.ext3
-rw-r--r--. 1 ns ns     153004 Jul 18 21:22 rpi-specialpi-image-slicedpimach-20150719021250.rootfs.manifest
-rw-r--r--. 1 ns ns 5150605312 Jul 18 21:28 rpi-specialpi-image-slicedpimach-20150719021250.rootfs.rpi-sdimg
-rw-r--r--. 1 ns ns  290871927 Jul 18 21:24 rpi-specialpi-image-slicedpimach-20150719021250.rootfs.tar.bz2
lrwxrwxrwx. 1 ns ns         59 Jul 18 21:26 rpi-specialpi-image-slicedpimach.ext3 -> rpi-specialpi-image-slicedpimach-20150719021250.rootfs.ext3
lrwxrwxrwx. 1 ns ns         63 Jul 18 21:26 rpi-specialpi-image-slicedpimach.manifest -> rpi-specialpi-image-slicedpimach-20150719021250.rootfs.manifest
lrwxrwxrwx. 1 ns ns         64 Jul 18 21:29 rpi-specialpi-image-slicedpimach.rpi-sdimg -> rpi-specialpi-image-slicedpimach-20150719021250.rootfs.rpi-sdimg
lrwxrwxrwx. 1 ns ns         62 Jul 18 21:26 rpi-specialpi-image-slicedpimach.tar.bz2 -> rpi-specialpi-image-slicedpimach-20150719021250.rootfs.tar.bz2
lrwxrwxrwx. 1 ns ns         30 Jul 18 18:15 u-boot.img -> u-boot-slicedpimach-git-r0.img
-rwxr-xr-x. 2 ns ns     286128 Jul 18 18:15 u-boot-slicedpimach-git-r0.img
lrwxrwxrwx. 1 ns ns         30 Jul 18 18:15 u-boot-slicedpimach.img -> u-boot-slicedpimach-git-r0.img

./bcm2835-bootfiles:
total 6720
-rw-rw-r--. 2 ns ns       0 Jul 18 18:02 bcm2835-bootfiles-20150206.stamp
-rw-r--r--. 2 ns ns   17856 Jul 18 18:02 bootcode.bin
-rw-r--r--. 2 ns ns   32843 Jul 18 17:56 config.txt
-rw-r--r--. 2 ns ns    2345 Jul 18 18:02 fixup_cd.dat
-rw-r--r--. 2 ns ns    6133 Jul 18 18:02 fixup.dat
-rw-r--r--. 2 ns ns    9191 Jul 18 18:02 fixup_x.dat
-rw-r--r--. 2 ns ns  554008 Jul 18 18:02 start_cd.elf
-rw-r--r--. 2 ns ns 2641752 Jul 18 18:02 start.elf
-rw-r--r--. 2 ns ns 3598760 Jul 18 18:02 start_x.elf

17. Validate the build

Insert the SD card or uSD card in to the Raspberry Pi and validate that the Raspberry Pi properly boots the image on the SD or uSD card.

18.  A few notes

The modifications that were made by cloning the meta-raspberrypi layer loosen up the recipes and the ability to make changes to the raspberry pi BSP.

If you are used to building the GNU/Linux kernel the normal way, then the  environment-setup-* script can be used to set the cross compiler and arch before building the kernel from within the kernel source tree.  The same goes for u-boot. The u-boot recipe is in meta-slicedpi/recipes-bsp/u-boot.
Lastly, I intentionally did not include kernel configuration fragments for the kernel configuration piece but included an example for the busybox configuration.

meta-slicedpi layer on github.

No comments:

Post a Comment