Lab 5: Kernel Driver for Playing Audio

Table of Contents

In this lab, you will add a new dimension to Space Invaders: sound! The PYNQ board contains a codec chip that can play sounds under software control. With this chip, and a little effort, you will be able to hear those aliens marching back and forth; hear the strange sounds of the alien space ship as it appears and tries to get away, and hear the explosion when those evil aliens blast your tank.

Objectives

  • Learn how to write a kernel driver for a hardware device.
  • Experience interacting with a large, complex software system (the Linux kernel).
  • Experience interacting with a FIFO-based hardware device, requiring interrupt-driven code.

Preliminary

  • Make sure you have completed the assigned reading from the LDD3 textbook.
  • Read the documentation on Linux Platform Devices and the Device Tree. Note that for this lab you don’t need to modify the device tree, but you will be querying information from the device tree, so it’s good to be familiar with it.
  • Review the Platform Devices & Device Tree slides. They cover the device tree node for a device, connecting your driver to that node, querying it for addresses and IRQs, creating the /dev file, and what belongs in init vs. probe.
  • Read the documentation on Linux Device Drivers.
  • Read the documentation on the Audio Hardware.

Github Repository

You will go back to working individually for this lab and the remaining labs. You should copy your space invaders game code from your group repository to your individual repository. Do not commit your code for labs 5+ to your shared repository.

Overview

You will be creating a loadable kernel module (background), which allows you to write code that runs in kernel space. This kernel module will be a driver for the audio codec.

Your driver code will need to do several things. Some of the major tasks:

  • Register as a platform device driver with Linux, so that Linux will call your code when it finds the audio device in the device tree.
  • Create a character device, which will allow user space programs to communicate with your driver, along with a device file in /dev (/dev/audio_codec).
  • Support the read() and write() file operations for your character device:
    • In Milestone 1, the read() and write() functions will only print log messages.
    • In Milestone 2, the read() and write() functions will interact with the audio device.
    • In Milestone 3, you will add another file operation, ioctl.
  • Obtain the physical address of the audio device from the device tree, reserve that address space, and map it to a virtual address pointer so that your driver can read and write registers in the audio device.
  • Handle interrupts from the audio device, which will be used to keep the audio FIFOs full so that the audio plays without popping or skipping.

Handling Interrupts

The audio hardware will send an interrupt when it needs more audio data. Unlike previous labs, this interrupt line is not connected to an interrupt controller that you manage from user space. Instead, it is connected directly to the Global Interrupt Controller (GIC) that is part of the ARM processor. The GIC is managed by Linux. As such, you will be interacting directly with the Linux interrupt system.

To handle and respond to interrupts generated by the audio hardware, you will need to query the device tree and get the virtual IRQ number, register your interrupt service routine with Linux, and then implement the ISR to respond to the interrupt.

Driver vs. Device

Keep the driver (your kernel module) and the device separate in your code:

  • module_init()/module_exit() run once, when the driver is loaded (sudo insmod audio_codec.ko) and unloaded (sudo rmmod audio_codec).
  • .probe/.remove run once per device Linux finds in the device tree. There is only one audio device, so each is called once, right after load and right before unload.

The Driver vs Device slide shows what goes in each. Your driver only needs to support one device, but organize it so it could be extended to more.

<Resources> is an important reference for each milestone!

Milestone 1

Implementation

⚠️ Before you start: the starter code may have been updated since you cloned your repository. Merge the latest starter code into your repository before working on this lab. See Add Starter Code Remote if you have not set up the remote yet.

git fetch startercode
git merge startercode/main

In this milestone you will create the skeleton of your kernel driver: create your character device, gain access to the device registers, and hook up the hardware interrupt. You will also create a simple user space program to test your driver.

Your kernel driver needs to:

  • Implement init and exit functions for the driver.
  • Register itself as a platform driver, with probe and remove functions for the device.
  • Upon device probe, the driver should create a character device as described above.
    • The character device should implement read() and write() functions; however, for this milestone, these functions should simply print a message to the kernel log.
    • Name the device audio_codec in device_create(), which creates /dev/audio_codec. A udev rule on the board (like the ones for your UIO devices) matches that exact name, makes the file accessible without sudo, and adds the symlink /dev/ecen427/audio_codec. Use the /dev/ecen427/audio_codec path in your user space programs. If you name the device anything else, the rule won’t apply and opening it will need sudo.
  • Also upon device probe, the driver should:
    • Print the physical address read from the device tree, and confirm in the kernel log when the memory region has been reserved.
    • Setup a virtual address pointer as described above, and print the physical and virtual address to the kernel log.
    • Print the IRQ number read from the device tree, and confirm in the kernel log when the interrupt handler has been registered.
  • Provide helper functions to read and write registers in the audio device.
  • At the end of your probe function, enable the interrupt output of the audio core. Since there is no data in the FIFOs, this should immediately trigger your ISR. In your ISR print a message to the kernel log and then disable the interrupt output of the audio core (or you will be stuck in an endless loop).
  • Make sure that:
    • When the driver is unloaded, undo all appropriate actions that were performed when the driver was loaded.
    • When a device is removed, undo all appropriate actions that were done when the device was added.
  • Add messages to print:
    • When your driver is loaded and unloaded.
    • When your class is created and destroyed.
    • When your device is added and removed.
    • The allocated major number of the driver and minor number of the device, both when allocated and when unregistered.
    • When you create and destroy your device using device_create()/device_destroy().
    • When your character device is registered (cdev_add()) and removed (cdev_del()).
    • When read() and write() are called.

Create a new user space program in userspace/apps/audio_driver_test1, with appropriate CMake file to create an executable named audio_driver_test1 (we use automated scripts when grading to make sure your executable is created at userspace/build/apps/audio_driver_test1/audio_driver_test1).

  • This program should open your device file, perform a read() and write(), and then close it.
  • Your audio_driver_test1/CMakeLists.txt only needs to add_executable() your source file and call deploy_to_board(), like the other apps. Then uncomment the add_subdirectory(audio_driver_test1) line in userspace/apps/CMakeLists.txt.

  • The program is only a few lines. You can use the following code exactly as your main.cpp:

    #include <fcntl.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int main() {

  // Open device file
  int fd = open("/dev/ecen427/audio_codec", O_RDWR);
  if (fd < 0) {
    fprintf(stderr, "Cannot open /dev/ecen427/audio_codec.  Is the audio_codec "
                    "module loaded?\n");
    return 1;
  }

  // Each call should produce a log message from your driver (check dmesg)
  uint8_t buf[100];
  ssize_t bytes_read = read(fd, &buf, sizeof(buf));
  ssize_t bytes_written = write(fd, &buf, sizeof(buf));

  // Close
  close(fd);
  return 0;
}

Pass Off

To grade your submission we will:

  1. Load and unload your driver (insmod and rmmod) TWICE.

    Make sure it works without error and has appropriate logging messages as described above. A simple script is provided that loads and unloads the module twice, prints recent kernel log entries, and prints details of your device file.

  2. Compile and run your audio_driver_test1 program.

    Make sure your cmake files are set up to correctly build the executable. We will run it after loading your driver and inspect the kernel logs to see that the read() and write() functions in your driver were executed.

    Make sure you have committed both your kernel driver code and your user space test program and CMake files. There will be points deducted if these don’t build correctly.

Milestone 2

In this milestone you will write code to read and parse WAVE files, and update your kernel driver to play sound data.

Specifications

  • Initialize the audio codec chip via I2C from userspace using the provided functions. See Audio Hardware.
  • Update your kernel driver such that:
    • Your device struct contains a statically-sized buffer (ie, an array) to store audio samples the user will write() to your driver (512*1024 entry array should be large enough).
    • In your write() function, you accept a buffer containing an audio clip. When this occurs your driver will stop playing any existing audio clip, and immediately begin playing the newly provided audio clip. In the write() function you will need to:
      • Immediately disable interrupts from the audio core.
      • Copy the audio data from user space to your buffer (including safety checks on the user space pointer) - LDD page 64.
      • Make sure the audio core has interrupts enabled.
    • Your ISR will be responsible for actually playing the audio clip. In your ISR:
      • Fill the audio FIFOs with the next audio samples to be played. You can either determine how much free space is in the FIFOs, or simply write enough samples to fill 75% of each FIFO, since that is the condition that triggers the interrupt (see Audio Hardware).
      • Once the end of the audio clip is reached, disable interrupts on the audio core.
    • In your read() function, read back one byte of data, with boolean value 0 or 1, indicating whether an audio sample is currently being played (this is different than the return value of the read() function, which should be the number of bytes read).
  • Write a user space program that parses WAVE files and sends the PCM data to the audio driver.
    • The format of the provided WAVE files is described in Audio Hardware.
    • The audio data you read from the WAVE files (in user space) will be an array of 16-bit samples.
    • The audio data you write to the hardware FIFOs (in kernel space) needs to be 24-bit samples (there’s no 24-bit data type in C, so use a 32-bit type)
    • This means you will need to convert each sample. For example, for a 16-bit PCM sample (int16_t), you will want to left-shift the data by 8 bits and then store it in a 32-bit data type (int32_t). This should be done in user space as you read the file into memory.
    • This means that your kernel write() function will take an array of 32-bit samples.

Passing Off

  • Create a user space executable that takes a WAVE file path as a command-line argument, and plays the audio clip, twice in a row, on the speakers using your audio driver. The TAs will run this executable when doing your pass-off.
    • This should use read() to check if audio is currently playing, and if so, wait until it finishes before sending the next clip to the driver. Do not use a sleep to wait for the audio to finish; make sure you are testing the driver read() function to determine when the audio is done.
  • This new user space program should be located in userspace/apps/audio_driver_test2, with an appropriate CMake file to create an executable named audio_driver_test2. Thus, when the TAs build your userspace code, it should produce an executable userspace/build/apps/audio_driver_test2/audio_driver_test2, which takes a single command-line argument.

Milestone 3

In this milestone you will add an ioctl interface to your driver to allow userspace to send special commands to your audio driver. You will also update your Space Invaders code to play sound effects. This is demonstrated in this video.

Specifications

  • Extend your kernel driver to add ioctl to the list of file operations supported by your character device. You should support two ioctl commands:
    • Turn on looping for the current audio clip.
    • Turn off looping for the current audio clip.
  • Integrate sound into Space Invaders by generating the following sounds during game operation:
    • WAVE files are provided here.
    • The “marching” sound the aliens make as they move back and forth across the screen is comprised of four separate walk1, walk2, walk3, and walk4 sounds. You start with walk1 the first time an alien moves and then play the next sound in the sequence on each successive move, cycling back to walk1. You will need to have a pause between each sound.
    • The sound that the red flying saucer makes as it flies across the screen (use your looping functionality for this)
    • The sound that occurs when your tank is hit by an alien bullet.
    • The sound that occurs when an alien is hit by a tank bullet.
    • The sound that occurs when you fire a bullet.
    • the sound the flying saucer makes if you hit it with a bullet.
  • Sound Priority:
    • The UFO sound has the highest priority. When it is present, no other sound should be played.
    • The walking sounds have the lowest priority. They should be played only if no other sound is currently playing.
    • The other sounds have medium priority. When played, they should interrupt any other sound that is currently playing, except for the UFO sound.

Passing Off

Your Space Invaders game should be operating with all of the sound effects.

Important: Your space_invaders executable must take one command-line argument: the path of the directory that holds the WAVE files. The TAs will run your game with the resources/wavFiles directory of your repository, for example:

./space_invaders ~/ecen427/resources/wavFiles

Build the path of each WAVE file from this argument. Do not hard-code an absolute path, and do not locate the files relative to your executable (for example with readlink()); the TAs run your executable from a different location than you do. You will need to update main.cpp to read this argument (argv[1]) and pass it to your audio code.

Code Submission

Follow the Submission Instructions. Reminder: your code must compile with zero warnings. The valgrind requirement does not apply to kernel modules.

Resources, Tips, and Hints

Milestone 1

audio_codec.c provides a starting framework for your driver software. Read over it carefully before coding anything.

A few notes about the provided code:

  • Your driver only needs to support a single audio device, so we have allocated a single struct audio_device as a global variable at compile time.
    • You might want to add a check in your probe function to make sure it isn’t being called more than once and overwriting the data in your struct.
    • If you need to add new variables related to the device they should be placed within this struct. Variables related to the driver should be placed as static global variables.
  • In our driver we are going to assume that the character device is only ever opened by one user program at a time. Otherwise, we would need to handle parallel accesses and deal with race conditions. LDD3, Ch 6, Single-Open Devices discusses how you can enforce this property. You can implement this enforcement if you like.

Things to remember:

  • MAKE SURE you pay close attention to the return values of every function you call, and handle any errors. Not all functions return a negative number on error. Functions that return pointers often encode errors differently. (See LDD3, Ch 10 Pointers and Error Values)
  • Add lots of kernel logging messages to help with debugging

Error handling:

  • In init and probe, if you encounter an error, you will need to roll back any changes you have made. A common technique to do this is using labels and gotos as shown in the pseudocode below:
  int err;

  err = doTask1();
  if (err)
    goto errTask1;

  err = doTask2();
  if (err)
    goto errTask2;

  err = doTask3();
  if (err)
    goto errTask3;

  return 0;  // success: skip the cleanup below

  errTask3:
    undoTask2();
  errTask2:
    undoTask1();
  errTask1:
    return err;

Some resources to help you with the kernel function calls:

  • alloc_chrdev_region:
    • see LDD3, Ch3, Major and Minor Numbers and Allocating and Freeing Device Numbers.
  • class_create:
    • All devices need to belong to a class, so you will need to create a device class for your audio code. This class pointer will be used when you create the device file. See the Creating a device file slide.
    • You can store the struct class* as a static global variable.
    • On our kernel, class_create() takes only the class name. LDD3 and many online examples pass THIS_MODULE as a first argument; that form no longer compiles.
  • platform_driver_register
    • There is quite a bit of background you need to read before understanding the significance of this call. Read the course documentation on linux platform devices.
    • You will need to pass in a pointer to a struct platform_driver. You can create a static global variable for this struct.
    • The Connecting device tree node to driver slide shows how to initialize this struct and its struct of_device_id match table. Remember that the .compatible field needs to exactly match the string in our device tree.
    • xillybus.com walks through a longer example. Note: This webpage targets an older version of the kernel: you don’t need to provide the __devinitdata attribute on your struct of_device_id.
    • Read about and study device-tree source files for this project here. Search for a compatible string that looks like it is related to an audio codec. The Anatomy of a node slide explains each line of a node.
  • cdev_init and cdev_add
    • LDD3, Ch3 discusses these functions in the section Char Device Registration.
  • device_create
  • platform_get_resource
  • platform_get_irq
    • Use this, not platform_get_resource with IORESOURCE_IRQ, to retrieve the IRQ number. (Note: It will be a virtual IRQ number, and will not be the same as the physical IRQ number listed in the device tree). See the Querying the Device Tree slide.
  • request_mem_region and ioremap
    • See LDD3, Ch 9, I/O Memory Allocation and Mapping.
  • request_irq
    • See LDD3, Ch 10, Installing an Interrupt Handler.
  • iowrite32
    • See LDD3, CH 9, Accessing I/O Memory.
    • Don’t forget the rules of Pointer Arithmetic.
    • Basically, the register offsets are byte-aligned, but if you’re not careful, the program will calculate addresses as if they were word-aligned.

Milestone 2

  • Consult the documentation on the Audio Hardware.
  • Consult the LDD3 textbook.

Milestone 3

  • Read LDD3 chapter 6, the first section on ioctl.
  • Use .unlocked_ioctl in the struct file_operations (.ioctl as the text suggests is out of date). Even with this update, you still use the ioctl() system call in your user-space code for space invaders.
  • The ioctl interface must be implemented as described in LDD3. Make sure the ioctl command values are created using the _IO* macros. An example of these are given on pages 138-139 of LDD3.
  • You probably want to read all of the WAVE files and store them in arrays at the startup of your game. Then each time you want to play a sound effect you can pass the appropriate array buffer to the driver. This avoids repeatedly reading the WAVE files each time you play a sound.
  • No sound mixing is required for this lab, simply play one sound at a time.

Other

Lifelong Learning: What’s Next?

Your ISR has one job: keep the FIFOs from running dry. The audio core interrupts when a FIFO drops to 256 samples, and the codec drains 48,000 samples per second, so your ISR has about 5 ms to run before the speakers pop. That is a deadline, and nothing in Linux promises you will meet it.

Explore this question: What could make your ISR miss its deadline, and how do real-time systems prevent that?

  • What else on the board could delay your handler? (Other interrupts, code that disables interrupts, the scheduler.)
  • Linux is not a real-time operating system. What does an RTOS guarantee that stock Linux does not?
  • What are some popular RTOSs, and what timing guarantees do they actually provide?

AI use is encouraged for this section. You are welcome to paste your completed audio_codec.c into an AI tool to give it context, then discuss the question with it. See the Lifelong Learning: What’s Next? page.

When you’re done, record what you learned in a few sentences in lifelong_learning/lab5.txt. Submit this file with Milestone 3; it is not required for Milestones 1 or 2.