Yocto Project Expert Tips: The Power of devshell

Reading Time: 4 minutes

Introduction

Building a Yocto project based distribution often means encountering frustrating errors when building recipes. While bitbake logs these failures, deciphering the root cause can be like finding a needle in a haystack. Each recipe and build process is unique, so relying solely on logs can be inefficient and overwhelming. Luckily, the Yocto project provides powerful tools and features to help pinpoint the source of these issues and streamline your development process.

Though often overlooked, devshell is a secret weapon in the Yocto project developer’s arsenal. This powerful tool drops you into a shell within the recipe’s source directory (${S}), prepped and ready for action. It executes all tasks leading up to do_patch, setting all the necessary OpenEmbedded build environment variables. This means you can dive right in and run commands like configure or make, giving you hands-on control to debug and troubleshoot build issues.

https://docs.yoctoproject.org/dev-manual/development-shell.html

But the true magic of devshell unfolds when you encounter build errors. By allowing you to execute individual do_ scripts directly within the recipe’s temporary directory, devshell transforms debugging from a guessing game into a surgical operation. You can insert debugging statements, enable echo mode, or step through the script line by line, effectively pinpointing the exact location and cause of the failure. This level of granular control empowers you to resolve issues swiftly and efficiently.

Debugging Failing Tasks

I’m a big believer in learning by doing, so let’s dive into a real-world example! We’ll create a simple recipe that intentionally fails. Now, this recipe is definitely simpler than what you’d see in a typical build, but it’s perfect for illustrating the debugging process:

# devshell-demo.bb
LICENSE = "MIT"
DESCRIPTION = "Example using devshell to debug recipe failures"

do_compile() {
    echo "build a recipe"
    foo
    echo "recipe is done"
}

After building the recipe, to nobody’s surprise, the build fails:

Now bear with me, I understand that the problem is obvious in this case reading the log. However, there will be real world cases where it is not so obvious, as the failure may be deep down in some other script, or the result of a utility which does not return useful failure information.

Let’s begin debugging the failure by first launching devshell, this is as simple as doing:

bitbake <recipe-name> -c devshell

This should open a new shell and you should be in the source directory within the work directory for the recipe. In this example, our recipe did not have any source, so we see an empty directory. However, we can list the contents of one directory up and see the rest of the work directory. This is where all the magic happens normally by bitbake itself when building the recipe.

Note: Depending on the terminal you are using, this may or may not open a new “tab” or “pane.”

Within the “temp” sub-directory, there are a series of both logs and scripts produced. The scripts actually hold the content of the various bitbake tasks themselves:

This means we can execute those scripts to replicate the exact steps bitbake would normally take for a given task. This gives us the power to run the task directly within our shell, allowing us to see what’s happening in real-time! How powerful is that?

Let’s see this in action. Our problematic task was do_compile, so we’ll run the corresponding script directly in our devshell instance:

$ ../temp/run.do_compile
../temp/run.do_compile: 141: foo: not found
WARNING: exit code 127 from a shell command.

The log clearly shows the same failure we saw before. But here’s where it gets even better: we can manually enable echo in the script to get a real-time trace of the commands as they execute in the shell.

Let’s give it a shot. First, we’ll edit the run.do_compile script to turn on tracing messages:

vim ../temp/run.do_compile

# Add the set -x near the top
#!/bin/sh
set -x

# :wq to save and quit

Now let’s run it again:

../temp/run.do_compile
....
+ cd /home/ksloat-engineering/engineering/cornersoft/yocto-env/poky/build/tmp/work/core2-64-poky-linux/devshell-demo/1.0/devshell-demo-1.0
+ do_compile
+ echo build a recipe
build a recipe
+ foo
../temp/run.do_compile: 142: foo: not found
+ bb_sh_exit_handler
+ ret=127
+ [ 127 != 0 ]
+ echo WARNING: exit code 127 from a shell command.
WARNING: exit code 127 from a shell command.
+ exit 127

The actual output is quite extensive, so I’ve trimmed it down to highlight the crucial part. As you can see, the call to foo fails because it doesn’t exist.

It’s important to note that this debugging technique isn’t limited to compile tasks. It can be applied to a wide range of tasks within your Yocto recipes.

A quick caveat: This method focuses on shell-based tasks. Some bitbake tasks, like do_fetch, are Python-based and would require additional steps to debug in this manner, which are outside the scope of this article

Summary

This article demonstrated how devshell can be a powerful tool for debugging bitbake/Yocto task failures during recipe builds. But devshell’s capabilities go far beyond debugging. The Yocto project offers a wealth of built-in techniques for streamlining your development workflow. We hope this exploration of devshell adds another valuable tool to your Yocto development arsenal.

The Yocto Project can be a complex and challenging endeavor, but there are strategies and tools that can simplify the process. By providing clear explanations and free practical guidance, we hope to equip you with valuable knowledge that will smooth your Yocto journey.

Remember that you don’t have to navigate the complexities of Yocto alone. Partnering with a seasoned Yocto project expert, like Cornersoft Solutions, can provide invaluable support and expertise. Our organization can guide you through the intricacies of the project, helping you avoid costly mistakes and time-consuming setbacks. Engaging a partner like Cornersoft Solutions can enable you to confidently tackle the challenges of the Yocto Project and achieve your embedded Linux development goals.

Contact linux@cornersoftsolutions.com to learn how our Yocto expertise, development ability, coaching and guidance can benefit your organization.