May 28, 2008

Controlling desktop applications with six degrees of freedom

Author: Ben Martin

Six degrees of freedom (6DOF) devices allow for movement in three axes, tilt in two axes, and rotation in the third. Some of these devices look like small joysticks -- for example, the SpaceNavigator. These devices are typically used for computer-aided design and manufacturing (CAD/CAM) and other 3-D applications. With a new programming library, you can now also use them with Linux applications.

You might wonder why you would ever want to use such a device with a desktop application. Well, imagine using the rotation of the device to control your video playback speed, or moving the x-axis of the device to see the next image in a slideshow.

Sure you can map keys on the keyboard to increase playback speed 10% in a video player, and pressing Space will show you the next image in a slideshow viewer. But with a 6DOF device, moving the x-axis is not a binary operation. If you move the axis with more force, you should be able to skip through more images per second than if you move the device only 25% in the x-axis. For video playback you might want to move all the way to get playback to happen 20 times faster than real time. As soon as you let go of the dial you might want to revert to normal playback speed or paused video. You can makes the 6DOF device act in a similar manner to a jog shuttle.

I created libsixdof to let applications easily support 6DOF devices. One key to the design of libsixdof was to have a minimal impact on an application (small patch size) in order for it to support control by a 6DOF device. I also wanted to be able to set up complex configurations for a 6DOF device -- for example, using a button on the device to switch axis movement and tilt to control a different function in the application until the base mode was activated again. Some folks may frown on modal input designs like this but I don't see why you should not have the freedom to map device actions to functions in an application any way you want.

The three core things that an application must do in order to support libsixdof are to register an XWindow with libsixdof, let libsixdof see XEvents so it can monitor for events from the device, and provide a callable function table. The libsixdof library takes care of mapping device activity into functions in the application and calling them.

You can register two types of functions with libsixdof: a trivial and a callable function. A trivial function takes no arguments, while a callable function takes a single floating point number as an argument. The floating point number passed to a callable function specifies what pressure is currently put on the currently active axis of the device. For example, if the x-axis is set to call a pan-x function and the device is moved 50% of the total distance in the x-axis, then the argument to the pan-x method will be 0.5. Having callable functions that take the amount of pressure as an argument like this allows libsixdof to perform smoother panning because you can make fewer function calls into the application in order to pan the canvas the same distance.

Apart from letting the user configure what device movements actually do in the application, keeping the device XEvent handling separate from the application lets the user set up calibration of the device and non-linear mappings. An application developer might provide a default configuration of bindings for a 6DOF device to get the user started, but it is ultimately up to the application user, not the developer, as to how device movements control the application when the 6DOF device is used with libsixdof.

As an example of calibration, the GIMP supports Linux /dev/input devices. If you are not using a 6DOF device with XWindow and you have read access to the right /dev/input device, you can tell the GIMP to monitor that device and use the Input plugins to perform GIMP functions when the device receives some events. But there is no support for calibration in the Input plugin. This means that if you set up z-axis movement on a 6DOF device to zoom the current window in, a slight touch of the device will zoom you in beyond any level that might be interesting. Because libsixdof configuration supports calibration, you can tell the library that z-axis movement should have less effect on the zooming function. The result is that you can make zooming less sensitive to the touch of the device. Another example: you could tell libsixdof that an x-axis movement is to move only five images per second when the device is held at 100% on its x-axis.

Non-linear mappings are not implemented yet, but the idea is that you can make movement in an axis be reported to the application in ways that are non-linear, so as you move more on the x-axis, the application function is called more and more aggressively.

You configure libsixdof through a .libsixdofrc file in your home directory. For each application you want to control you have a section in this file telling libsixdof what device actions to map to what application functionality. Part of a libsixdofrc file is shown below. The preamble includes some global device configuration information. The axis_ settings set up the overall calibration of the device. The axis_move_minval_cutoff setting tells libsixdof not to report any movement in any of the degrees of freedom where the movement is less than 40% of the axis. Having a high setting for axis_move_minval_cutoff means that beginner users will not call multiple functions because they have moved 10% in a second axis as well is 40% in the axis that they are primarily interested in.

The axis and buttons sections allow you to set up mnemonics for both the axis movement and buttons of the device. It can be much simpler when binding movement to a function if you are able to use yaxisback to specify the axis instead of needing to know that Linux reports that as axis 7 movement. Some 6DOF devices also come with more than 10 buttons, and rather than remembering that button 17 is on the left side of the device, it is much simpler to give a descriptive name to each device button. These axis and button mnemonics can then be used throughout the rest of the configuration file.

Each application has a section of its own in the configuration file. In the below example you can see the settings for controlling the feh image viewer. Various axis movements have been assigned to call application functions. The period argument tells libsixdof the maximum number of times that a function can be called per second. This lets you set up how quickly the axis movement will effect functionality in the application. A button configuration example is also included. There can of course be multiple app sections to specify the bindings for each libsixdof-controlled application.

axis_max_per_second_neg={ 46000, 46000, 21000, 25000, 25000, 35000 }
axis_max_per_second_pos={ 57000, 57000, 28000, 25000, 25000, 18000 }

number=6 name=xaxisleft
number=0 name=xaxisright
number=7 name=yaxisback
number=1 name=yaxisforward
number=8 name=zaxisup
number=2 name=zaxisdown
number=9 name=xtiltaway
number=3 name=xtilttowards
number=4 name=ytiltleft
number=10 name=ytiltright
number=11 name=zrotationanti
number=5 name=zrotationclock

number=17 name=left
number=18 name=right

axis=xaxisright period=10 function=slideshow-next
axis=xaxisleft period=10 function=slideshow-prev
axis=yaxisforward period=6 function=yaxis
axis=yaxisback period=6 function=yaxis
axis=zaxisdown period=10 function=zoom-in
axis=zaxisup period=10 function=zoom-out
axis=xtilttowards period=20 function=move-up
axis=xtiltaway period=20 function=move-down
axis=ytiltleft period=20 function=move-left
axis=ytiltright period=20 function=move-right
axis=zrotationclock period=6 function=rotate-clockwise
axis=zrotationanti period=6 function=rotate-anticlockwise

The function table registration in libsixdof also lets an application provide a description of the function that is being registered. In the future, developers will be able to present this information to users to allow them to set up libsixdof bindings from a user-friendly graphical interface.

For each window that an application registers with libsixdof it should provide a name. In the future the configuration file will expand to allow device bindings to be specified independently for each managed window in an application by adding subsections to the app sections in the configuration.

Wrap up

Currently, the CVS version of feh has support for libsixdof, and there is a patch to add support to KDE 3.x KPhotoAlbum. MPlayer and the GIMP are on the list for upcoming patches as time permits.

If libsixdof, or some library like it, catches on, then support for six degrees of freedom devices will become more mainstream. In the future you may be able to plug in such a device and immediately control your music player, video player, or even browse the Web with it. With technology like DBus allowing an application to expose its functionality through callable functions, the burden of adding libsixdof support to an application should be even smaller in the future.


  • Desktop Hardware
Click Here!