Real-time Linux (Xenomai)
Exercise #1: The Basics of Real-Time
Real-Time Linux Introduction
Linux is a free Unix-like operating system that runs on a variety
of platforms, including PCs. Numerous Linux distributions such as Red Hat, Debian and Mandrake bundle the Linux OS with tools,
productivity software, games, etc.
- The Linux scheduler, like
that of other OSes such as Windows or MacOS, is designed for best average response, so it
feels fast and interactive even when running many programs.
- However, it doesn't
guarantee that any particular task will always run by a given deadline. A
task may be suspended for an arbitrarily long time, for example while a
Linux device driver services a disk interrupt.
- Scheduling guarantees
are offered by real-time operating systems (RTOSes),
such as QNX, LynxOS or VxWorks.
RTOSes are typically used for control or
communications applications, not for general purpose computing.
- Linux has been adapted for
real-time support. These adaptations are termed "Real-Time
Linux" (RT Linux).
- Numerous versions of RT
Linux are available, free or commercial. Three commonly available free RT
Linux versions are
- RTLinux, developed by
New Mexico Tech and now maintained by Wind River Systems.
- RTAI, The Real-Time
Application Interface (RTAI), developed by the Milan Polytechnical
University and available at https://www.rtai.org/.
a spin off of RTAI Linux. While RTAI is
focused on lowest technically feasible latencies, Xenomai
also considers clean extensibility (RTOS skins), portability, and
maintainability as very important goals. It is available at www.xenomai.org.
- These RT Linux systems are
patches to the basic Linux kernel source code. Instructions for building
an RT Linux system from a the Linux source code
are provided with these RT Linux systems. Briefly, this involves setting
up the basic Linux system, getting the latest Linux kernel source code
from www.kernel.org, patching the
kernel source code, and compiling the patched kernel.
The following are the primary objectives of this exercise:
- To learn how Xenomai real-time Linux works and to build a small Xenomai real-time task.
How does RT Linux work?
The general idea of RT Linux is that a small real-time
kernel runs beneath Linux, meaning that the real-time kernel has a higher
priority than the Linux kernel. Real-time tasks are executed by the real-time
kernel, and normal Linux programs are allowed to run when no real-time tasks
have to be executed. Linux can be considered as the idle task of the real-time
scheduler. When this idle task runs, it executes its own scheduler and
schedules the normal Linux processes. Since the real-time kernel has a
higher priority, a normal Linux process is preempted when a real-time task
becomes ready to run and the real-time task is executed immediately.
How is the real-time kernel given higher priority than Linux kernel?
Basically, an operating system is driven by interrupts,
which can be considered as the heartbeats of a computer:
- All programs running in an
OS are scheduled by a scheduler which is driven
by timer interrupts of a clock to reschedule at certain times.
- An executing program can block
or voluntary give up the CPU in which case the scheduler is informed by
means of a software interrupt (system call).
- Hardware can generate
interrupts to interrupt the normal scheduled work of the OS for fast
handling of hardware.
RT Linux uses the flow of interrupts to give the real-time
kernel a higher priority than the Linux kernel:
- When an interrupt
arrives, it is first given to the real-time kernel, and not to the Linux
kernel. But interrupts are stored to give them later to Linux when the
real-time kernel is done.
- As first in row, the
real-time kernel can run its real-time tasks driven by these interrupts.
- Only when the real-time
kernel is not running anything, the interrupts which were stored are
passed on to the Linux kernel.
- As second in row, Linux
can schedule its own processes driven by these interrupt.
Hence, when a normal Linux program runs and a new interrupt
- it is first handled by an
interrupt handler set by the real-time kernel;
- the code in the interrupt
handler awakes a real-time task;
- immediately after the
interrupt handler, the real-time scheduler is called ;
real-time scheduler observes that another real-time task is ready to run,
so it puts the Linux kernel to sleep, and awakes the real-time task.
- Hence, to the real-time
kernel and Linux kernel coexist on a single machine a special way of
passing of the interrupts between real-time kernel and the Linux kernel is
needed. Each flavor of RT Linux does this is in its own way. Xenomai uses an interrupt pipeline from the Adeos project.
For more information, see also Life
RT Linux Tasks are not Linux Programs
- In real-time Linux we can
make a real-time program by programming real-time threads. None-real-time
programs in real-time Linux just use the conventional Linux threads.
- A real-time task can be
executed in kernel space using a kernel module, but it can also be
executed in user space using a normal C program.
- Running in user space
instead of in kernel space gives a little extra overhead, but with the
following advantages :
- A bug in a kernel
module can crash the whole system, however a bug
in a user space program can only crash the program.
- In a kernel model
one can only use the real-time API
and the limited kernel API, but in a
real-time user space program the real-time API
and the whole Linux API can be used!
However when using the Linux API in
user space, the program cannot be scheduled by the real-time scheduler
(HARD real-time) but must be scheduled by the Linux scheduler (SOFT
real-time). So calling a Linux API
call from a real-time user space task will degrade its performance from
HARD to SOFT real-time. After the call
the task will return to the real-time scheduler.
- The Xenomai project
was launched in August 2001.
is based on an abstract RTOS core, usable for building any kind of real-time
interfaces, over a nucleus which exports a set of generic RTOS services.
Any number of RTOS personalities called “skins” can then be built over the
nucleus, providing their own specific interface to the applications, by
using the services of a single generic core to implement it.
- The following skins on
the generic core are implemented :
allows to run real-time threads either strictly
in kernel space, or within the address space of a Linux process. A
real-time task in user space still has the benefit of memory protection,
but is scheduled by Xenomai directly,
and no longer by the Linux kernel. The worst case scheduling latency of
such kind of task is always near to the hardware limits and predictable,
since Xenomai is not bound to synchronizing with
the Linux kernel activity in such a context, and
can preempt any regular Linux activity with no delay. Hence, he preferred
execution environment for Xenomai applications
is user space context.
- But there might be a few
cases where running some of the real-time code embodied into kernel
modules is required, especially with legacy systems or very low-end
platforms with under-performing MMU hardware. For this reason, Xenomai's native API
provides the same set of real-time services in a seamless manner to
applications, regardless of their execution space. Additionally, some
applications may need real-time activities in both spaces to cooperate, therefore special care has been taken to
allow the latter to work on the exact same set of API
- In our terminology, the
terms "thread" and "task" have the same meaning. When
talking about a Xenomai task we refer to
real-time task in user space, i.e., within the address space of a Linux
process, not to be confused with regular Linux task/thread.
- In our exercise we will
use the native Xenomai skin and run real-time tasks
in user space because working with them is easier and still HARD
- The Native Xenomai API is flexible and well documented.
- The most important
characteristics of the Xenomai API
Xenomai's native API
provides the same set of real-time services in a seamless manner to application
regardless of their execution mode is user or kernel mode.
Xenomai objects (eg. a semaphore, task, etc..) are
always reachable even when the location of the object is another execution
space than were the task using it is running. e.g.
user space task running in user space can use a semaphore which is stored in
To this end, each category of services in Xenomai's
native API defines a uniform descriptor type
to represent the objects it manages. For instance, a task will always be
represented by a RT_TASK descriptor, a message queue by a RT_QUEUE descriptor,
and a semaphore by a RT_SEM descriptor,
regardless of the execution space from which they are used.
Xenomai uses a unified registry, to index each object
descriptor under a symbolic name given by the user. By doing this, Xenomai enables you to retrieve any descriptor associated
with a registered object in any location of your application.
C is the Preferred Language
- Linux is written in the C
programming language (with some assembly language), as is Xenomai, so C is the preferred language for writing RT
- Other languages (e.g.,
C++) can be used. For instance, see the Xeno--
project which is providing a thin object-oriented
software interface to Xenomai’s
- The examples in this
tutorial are all written in C.
- C programs are normally
compiled into full executable programs
- In C, a program's
"entry point" where execution begins is a function called 'main()'.
- Each real-time task is
associated with a C function that is called when the task is scheduled to
Creating a task
- When you create a
real-time task in Xenomai the RT_TASK structure
is used as the descriptor to refer to this task.
- An RT_TASK data structure
is used to hold all the information about a task:
- the task
function to be executed by the real-time task,
- any initial argument
passed to it,
- the size of the
stack allocated for its variables,
- its priority,
- whether or not it
uses floating-point math,
a "signal handler" that will be called when the task becomes
- The task is created by calling
int rt_task_create (RT_TASK *task, const char *name, int stack_size, int priority, int mode)
is a pointer to an RT_TASK type structure which must have been declared
before and whose structure is filled.
is an ASCII string standing for the symbolic name of the
task. This symbolic name you can use to retrieve the task structure
anywhere else in your code by calling the rt_task_bind() function.
is the size of the stack to be used by the new task.
is the priority to be given the task. The highest priority is 99, while
the lowest is 1.
- 'mode' is a set of
flags which affect the task :
allows the task to use the FPU
(Floating Point Unit) whenever available on the platform. This flag is
forced for user-space tasks.
- T_SUSP causes the
task to start in suspended mode. In such a case, the thread will have to
be explicitly resumed using the rt_task_resume() service for its execution to actually begin.
- T_CPU(cpuid) specifies that
the new task runs on the CPU with number cpuid.
CPU identifiers range from 0 to RTHAL_NR_CPUS - 1 (inclusive).
- T_JOINABLE (user-space
only) allows another task to wait on the termination of the new task.
This implies that rt_task_join() is actually called for this task to clean up
any user-space located resources after its termination.
Starting a task
- A task can be started by
int rt_task_start (RT_TASK *task, void(*task_func)(void *arg), void *arg)
is a pointer to an RT_TASK type structure which must be already
initialized by a call to rt_task_create().
- 'task_function' is the the task function to be
executed by this real-time task.
- 'arg' is the void pointer
argument given to the task function.
If you did specify T_SUSP as a
mode flag to rt_task_create(), the task will be started in a suspended mode.
In that case you have to explicitly resume the
task with rt_task_resume(). Otherwise, the real time task is
ready to execute immediately.
The following "ex01.c" C code illustrates the creating and start of
a "demo" task which writes a message.
void demo(void *arg)
// hello world
// inquire current task
// print task name
rt_printf("Task name : %s \n", curtaskinfo.name);
argc, char* argv)
char str ;
// Perform auto-init
of rt_print buffers if the task doesn't do so
// Lock memory :
avoid memory swapping for this program
* Arguments: &task,
stack size (0=default),
mode (FPU, start suspended, ...)
rt_task_create(&demo_task, str, 0, 50, 0);
* Arguments: &task,
rt_task_start(&demo_task, &demo, 0);
In the program above, memory is locked by calling
so that it cannot be swapped to
disk. Preventing swapping is a must when real-time performance must be reached.
Hence, Xenomai generates a warning when this is
forgotten when its API is called.
The rdtk real-time printing library
Note that in the above code we used the command 'rt_printf()'
instead of the familiar 'printf()' command. We did
not use 'printf()' because
it uses Linux syscalls and therefore terribly
slows down the real-time thread. Instead we use the 'rt_printf'
command from the rdtk
real-time printing library. This is a stand-alone real-time library for printf services. It is embedded into the Xenomai user space part but actually doesn't depend on any Xenomai service, just using the plain POSIX API.
The librtprint API
looks much like the printf(3) man page:
The basic idea of librdtk is to
keep the print side as cheap as possible: no locks, no syscalls
at all. Each thread that uses rt_printf & friends
has its own local ring buffer. A central (per process) non-RT output thread
takes care of forwarding the content of all thread ring buffers to the output
streams. The message order is preserved loosely by a global sequence counter
(arch-dependent CPU-wide or system-wide atomic_inc_return
might be used in the future). The output thread uses periodic polling (default:
10 Hz) to avoid the need for special signalling
To start the non-RT output thread one always has to call the following at the
beginning of a Xenomai programma :
This performs auto-init of the rt_print buffers and start the non-RT output thread.
Compiling a Xenomai program
To compile the above program, first obtain the CFLAGS and LDFLAGS by
> xeno-config --xeno-cflags
> xeno-config --xeno-ldflags
To set these flags, use
> export CFLAGS=`xeno-config --xeno-cflags`
> export LDFLAGS=`xeno-config --xeno-ldflags`
Finally, compile the ex01.c program against the 'native' library
implementing the Xenomai Native API
and the 'rtdk' print library
> gcc $CFLAGS $LDFLAGS -lnative -lrtdk ex01.c -o ex01
By default the executable 'ex01' is linked against the
dynamic libraries in /usr/xenomai/lib. Because these
are special and not known to linux, we first have to
tell Linux where to find them by setting a library path environment variable :
> export LD_LIBRARY_PATH=/usr/xenomai/lib
Than run the program :
Compile and run program ex01.c to get some basic experience
in working with Xenomai tasks. First, try a manual
build by compiling with the commands above. Next, try an automated the build
with this Makefile (using
commands "make" and "./ex01").
Introduce an error by commenting out the assignment to the current task as follows:
Save the resulting program as ex01b.c
Build and run this program, observe the result.
Next introduce a return value for the 'rt_task_inquire' call and print the error code as described in the part about "Error handling" of the Tips page on the Xenomai pages of Robot Lab site.
Build and run the new version of the program.
Last Updated: 22 August 2011 (Jozef
Created by: Harco Kuppens