Using a Wiimote as a 6-DOF Vrui Input Device

Current versions of Vrui contain a tracking driver for the Wiimote using the camera-based tracking approach described on the technology page. As this driver is part of Vrui's device driver (VRDeviceDaemon), getting the Wiimote into Vrui applications requires a few simple configuration steps.

Step 1: Build IR Beacon

The Wiimote tracking driver relies on a beacon consisting of four infrared LEDs in a known, non-planar configuration. The Wiimote can only be tracked while its camera sees all four LEDs. A detailed description of a canonical beacon's layout, and a simple wiring diagram, are available on the IR beacon page.

Step 2: Configure VRDeviceDaemon

VRDeviceDaemon is configured via its own configuration file, VRDevices.cfg. For more information on Vrui, VRDeviceDaemon, and their configuration files, see the Vrui HTML documentation.

Step 2.1: Create New VRDevices.cfg

The recommended process is to create a new VRDevices.cfg file in Vrui's configuration directory, and create a new root section named after the local computer, i.e., the result of running echo $HOSTNAME, or, if that fails, echo $HOST in a terminal. The computer name needs to be enclosed in double quotes. In the following, we will use "localhost.localdomain" as a placeholder for the actual computer name. Inside that root section, create the skeleton file shown below:

section "localhost.localdomain"
	section DeviceManager
		deviceNames (Wiimote)

		section Wiimote
		endsection
	endsection

	section DeviceServer
		serverPort 8555
	endsection
endsection

The deviceNames setting in section DeviceManager contains a list of section names which in turn contain device driver specifications. This file uses only a single device driver, the Wiimote driver in the following section of the same name. The Wiimote driver's details will be filled in later.
The serverPort setting in section DeviceServer specifies the TCP port on which VRDeviceDaemon will listen for connections from Vrui applications. This port number is arbitrary, but it needs to match the same setting in Vrui's configuration file, and any local firewalls need to be configured to allow connections to that port at least from the local computer.

Step 2.2: Configure Wiimote Tracking Driver

VRDeviceDaemon needs to be told what driver module to use for the Wiimote, and the Wiimote tracking driver needs to know the bluetooth ID of the Wiimote device to which to connect, and some basic information about the device itself. In the skeleton VRDevices.cfg file, insert the following in section Wiimote:

section "localhost.localdomain"
	section DeviceManager
		...
		section Wiimote
			deviceType WiimoteTracker
			deviceName "xx:xx:xx:xx:xx:xx"
			enableTracker true

			homeTransform translate (0.0, -12.0, -2.0)

			cameraCenter (512, 384)
			cameraFocalLength 1280
		endsection
	endsection
	...
endsection

The deviceType setting tells VRDeviceDaemon to use the Wiimote tracking driver.
The deviceName setting specifies the Wiimote's bluetooth ID as a string of six hexadecimal numbers, separated by colons and enclosed in double quotes. The easiest way to find the Wiimote's bluetooth ID is to use the bluetooth panel applet from the desktop, enter device discovery, and press buttons "1" and "2" on the Wiimote simultaneously.
The enableTracker settings tells the driver to enable the camera-based tracking algorithm. If this setting is set to false, the Wiimote will only report the state of its buttons and joystick axes.
The homeTransform setting specifies the position and orientation in Vrui physical space, given as a rigid body transformation composed from a sequence of translations and rotations, from which to start the iterative tracking algorithm if there is no valid position or orientation from which to start. This position and orientation should correspond to the typical position and orientation in which a user would hold the Wiimote in space.
The cameraCenter and cameraFocalLength settings specify the Wiimote's intrinsic camera parameters (see the technology page). Both settings use pixel-space coordinates, and assume that the camera's pixels are square. The values given here are based on experiments using one particular Wiimote, but seem to apply in general. Proper camera calibration procedures are left as an exercise to the reader.

Step 2.3: IR Beacon Configuration

The Wiimote driver needs to know the absolute positions of the IR beacon's four LEDs in Vrui's physical space in order to calculate the Wiimote's position and orientation in the same physical space. Because a beacon is typically internally rigid but movable, it is configured via two steps: the positions of the four LEDs are first defined in some arbitrary local coordinate system, and are then transformed to Vrui physical space via a rigid body transformation. This simplifies updating the driver's configuration when the beacon is moved to a different position or orientation.

Vrui physical space is the (arbitrary) coordinate space in which an environment's screens and viewers are defined, and is described in more detail in Vrui's HTML documentation. The typical physical space layout for a desktop environment has the coordinate origin in the center of the screen, the X axis going to the right, and the Z axis going up. The Y axis, then, points into the screen.

The configuration settings for the IR beacon are:

section "localhost.localdomain"
	section DeviceManager
		...
		section Wiimote
			...
			targetPoints ((-2.0, 0.0, 0.0), (0.0, -1.0, 2.0), (2.0, 0.0, 0.0), (0.0, -1.0, -2.0))
			targetTransformation translate (0.0, 0.0, 10.0) * rotate (1.0, 0.0, 0.0), 15.0
		endsection
	endsection
	...
endsection

The targetPoints setting is a list of four points, defining the positions of the four LEDs in beacon-local coordinates. The order of the points matters: the beacon should form a configuration with left, top, right, and bottom LEDs, and the positions should be specified in that order. The current tracking algorithm uses orientation information from the Wiimote's linear accelerometers to create associations between observed LED images and beacon LEDs, and assumes that the first and third LEDs are the leftmost and rightmost, respectively, when looking at the beacon, and the second and fourth LEDs are the topmost and bottommost, respectively. The LED positions are specified using the same coordinate units as Vrui's physical space, which are inches in the default configuration. The positions should be as accurate as possible.
The targetTransformation setting specifies the rigid body transformation from local beacon space to Vrui physical space. The transformation given above corresponds to a beacon sitting on top of a 20" LCD monitor, with the beacon rotated downwards by 15°, using Vrui's default physical space as described above.

Step 3: Configuring Vrui

To use the Wiimote, Vrui needs to be told to connect to a VRDeviceDaemon instance on start-up, and to create a 6-DOF input device using the Wiimote's position and orientation, buttons, and analog axes. This is done by adding another input device adapter to Vrui's configuration file, Vrui.cfg. One can either modify Vrui.cfg itself, or create a "patch" configuration file which is merged into the main configuration file on request. Using the second approach, one first creates an empty configuration file, called Vrui-Wiimote.cfg, in the same directory as Vrui.cfg, and inserts the following content:

section Vrui
	section Desktop
		inputDeviceAdapterNames (MouseAdapter, DeviceDaemonAdapter)

		section DeviceDaemonAdapter
			inputDeviceAdapterType DeviceDaemon
			serverName localhost
			serverPort 8555
			inputDeviceNames (Wiimote)
			
			section Wiimote
				name Wiimote
				trackType 6D
				trackerIndex 0
				numButtons 13
				numValuators 2
				deviceGlyphType Cone
			endsection
		endsection
	endsection
endsection

The inputDeviceAdapterNames setting lists the names of sections which in turn define one input device adapter each. MouseAdapter represents mouse and keyboard in desktop environments, and DeviceDaemonAdapter is the name of the new section representing the Wiimote.
The inputDeviceAdapterType setting defines the type of the input device adapter. DeviceDaemon is the adapter type to connect to Vrui's external device driver, VRDeviceDaemon.
The serverName and serverPort settings define the Internet host name and TCP port under which the device driver instance can be reached. A serverName of localhost indicates that the device driver is running on the same computer. The port number of 8555 is arbitrary, but must match the port number configured in VRDevices.cfg, and must not be blocked by a local firewall.
The inputDeviceNames setting lists the names of sections defining one input device each.
The name setting defines an arbitrary name for the section's input device.
The trackType setting indicates how many degrees of freedom the device has; a value of 6D means the device reports a 3D position and a full 3D orientation.
The trackerIndex setting identifies the tracker defining the device's position and orientation. Since VRDevices.cfg contains only a single device, the tracker index is zero.
The numButtons and numValuators settings define the numbers of buttons and analog axes, respectively, on the device. The Wiimote tracking driver always exports 13 buttons (11 on the Wiimote itself, 2 on the optional Nunchuck extension) and two valuators (the thumbstick axes on the optional Nunchuck extension).
The deviceGlyphType setting defines which glyph type Vrui should use to draw the position and orientation of the device.

Step 4: Starting VRDeviceDaemon

In order to use the Wiimote, one first has to start VRDeviceDaemon, Vrui's external device driver. Running VRDeviceDaemon from a terminal, without command line arguments, will start the driver and configure it according to the VRDevices.cfg file. On start-up, the driver will print several status messages, and then ask the user to press buttons "1" and "2" on the Wiimote simultaneously on the Wiimote. This will put the Wiimote into discovery mode, indicated by four flashing LEDs. After a few seconds, the driver should pick up the connection, turn off the flashing LEDs, and print Waiting for client connection... At this point the driver is ready to accept connections from Vrui applications, and serve tracking data. The driver can be kept running throughout multiple Vrui application sessions, but the Wiimote will use battery power while the driver is running, even when idle. To preserve battery life, it is best to stop the driver by pressing Ctrl-C in its terminal window when noone will be using it for a longer time.

Step 5: Starting Vrui Applications

While VRDeviceDaemon is running and ready, Vrui applications can be started in the usual manner. If the Wiimote was configured via a "patch" configuration file, as recommended, one has to add -mergeConfig <config file name> to the application's command line, where <config file name> is the absolute or relative path to the created configuration file. If invoked in that way, VRDeviceDaemon should print connection and disconnection messages as applications are started and terminated, and a grey cone indicating the Wiimote's position and orientation should appear in the main window whenever the Wiimote's camera can see all four LEDs on the IR beacon.

Step 6: Pre-Assigning Tools

Up to this point, the Wiimote should appear as a 6-DOF device, but it will not yet have any functionality associated with it. As a result, pressing and holding any of its buttons should show Vrui's tool selection menu. It is possible to pre-assign common tools to several buttons via the same patch configuration file. How this is done varies between 1.0-xxx versions of Vrui and 2.x versions. In all versions, one starts by inserting the following sections into the patch configuration file:

section Vrui
	section Desktop
		...

		section Tools
			toolNames (MouseNavTool, \
			          FPSNavTool, \
			          MenuTool, \
			          WiiNavTool, \
			          WiiMenuTool)
			
			section WiiNavTool
				toolClassName WandNavigationTool
			endsection
			
			section WiiMenuTool
				toolClassName RayScreenMenuTool
			endsection
		endsection
	endsection
endsection

The toolNames setting lists the names of sections pre-assigning one tool each. MouseNavTool, FPSNavTool, and MenuTool are the default tools assigned in desktop environments. WiiNavTool and WiiMenuTool are two new tools assigned to the Wiimote.
The WiiNavTool and WiiMenuTool sections define the new tools' bindings, and their contents depend on whether the Vrui version is smaller than 2.0 or larger than or equal to 2.0
The toolClassName settings define the type of their respective sections' tools. WandNavigationTool is a navigation tool for 6-DOF devices using one button to grab space, and a second button to scale around the grabbed point if pressed together with the first button. RayScreenMenuTool is a tool class to pop up and interact with application menus.

Vrui Versions Prior To 2.0

In pre-2.0 versions of Vrui, tool bindings were specified by button and valuator indices. To bind a wand navigation tool to the "A" button and the direction pad's down arrow, and to bind a menu tool to the "Home" button, one would insert:

section Vrui
	section Desktop
		...

		section Tools
			...

			section WiiNavTool
				toolClassName WandNavigationTool
				deviceName0 Wiimote
				device0ButtonIndex0 3
				device0ButtonIndex1 8
			endsection
			
			section WiiMenuTool
				toolClassName RayScreenMenuTool
				deviceName0 Wiimote
				device0ButtonIndex0 5
			endsection
		endsection
	endsection
endsection

The deviceName0 settings specify the name of the first device to which to bind the tool. This name must match the name setting in the configuration file's DeviceDaemonAdapter section.
The device0ButtonIndex0 and device0ButtonIndex1 settings define the indices of the first and second buttons on the first device to which to bind the tool. WandNavigationTool uses two buttons on the first device, in this case the "A" button and the down arrow. RayScreenMenuTool uses only a single button, in this case the "Home" button.

Vrui Versions 2.0 And Later

In Vrui version 2.0 and later, tool bindings designate buttons and valuators by name instead of index, and using a list of assignments instead of individual settings. Additionally, Vrui 2.0 allows stacking of button bindings, which makes it possible to use the down arrow to scale space when pressed together with the "A" button, and to pop up and interact with an application's main menu if pressed by itself. To achieve this binding, which is more comfortable than the one in the previous section, one would insert:

section Vrui
	section Desktop
		...

		section Tools
			...

			section WiiNavTool
				toolClassName WandNavigationTool
				bindings ((Wiimote, A, Down))
			endsection
			
			section WiiMenuTool
				toolClassName RayScreenMenuTool
				bindings ((Wiimote, Down))
			endsection
		endsection
	endsection
endsection

The bindings settings specify nested lists of device assignments. Each device assignment is enclosed in parentheses, and starts with the name of the device to use, followed by an arbitrary number of names of buttons or valuators on that device. Any number of assignments can be specified in order to bind buttons or valuators from multiple devices, and the entire list of assignments is enclosed in parentheses as well. In this case, since both WandNavigationTool and RayScreenMenuTool expect only a single device, there is only one device assignment item in either list.