Blue Tunes Video Overview

Basic algorithm for using Wiimote to control cursor

Using the Wiimote as a way to operate a mouse cursor on a PC is an idea that has come across the minds of many in the home brew community. GlovePIE allows for this task to be done using algorithms related to the camera within the Wiimote and the IR sensor bar used with the Wii console.

This is a recent project I have started and am hoping to expand upon in order to provide a precise and easy-to-use experience for PC users who may opt to using a Wiimote to operate their PCs.

This entry will focus on the basic algorithm I will be using to perform the function of operating a mouse cursor.

These algorithms simply allow you to move the cursor around the screen. There are no button assignments made yet, thus there is no "left-clicking" or "right-clicking." The following are descriptions of the functions and numbers found in the first equation:

• mouse.CursorPosY : The mouse cursor position on the Y axis. Assigning a correct algorithm to this variable allows the user to move the cursor up and down.
• round : Allows us to round up the equation within the first set of parenthesis, thus giving the user smoother mouse movement on the screen.
• wiimote1.dot1y : The Wiimote relies on two IR points provided by the sensor bar in order to give more accurate readings. In GlovePIE, these points are referred to as "dots." Thus, dot1y is the y-position of one of the points being detected.
• wiimote1.dot2y : This is the y-position of the second IR point being detected.
• ((wiimote1.dot1y+wiimote1.dot2y)/2)/760*1200 : This equation allows us to average the two y-positions, thus giving a more accurate reading.
• ((wiimote1.dot1y+wiimote1.dot2y)/2)/760*1200 : 760 is the maximum height reading that can be achieved using the Wiimote, with 0 being the lowest. This number can be determined by doing a simple test with the debug function where debug = wiimote1.dot1y. The average is divided by this number in order to create a percentage and a number between 0 and 1.
• ((wiimote1.dot1y+wiimote1.dot2y)/2)/760*1200 : 1200 is the pixel resolution of the height of the monitor currently being used. The previous part of the equation is multiplied by the height of the resolution in order to pinpoint exactly It is important that this number is adjusted according to the resolution of the monitor being used.

The variables in the algorithm reflecting the x-axis position are similar to the algorithm of the y-axis. However, due to the way the Wiimote was designed, the x-axis is inverted. In order to reflect this, the length resolution, 1920, is subtracted by round(((wiimote1.dot1x+wiimote1.dot2x)/2)/1012*1920)

What is Human-computer interaction?

The definition for human-computer interaction is one that is not agreed upon, but in order to put this research project in perspective and understand HCI on a broad level, the definition provided by HCI Bibliography will be used:

Human-computer interaction is a discipline concerned with the design, evaluation and implementation of interactive computing systems for human use and with the study of major phenomena surrounding them.

Human-computer interaction deals with primarily two things: the user and the computer. The user will use some sort of input device to issue commands, and the interaction is displayed in an output device. Typical input devices include a mouse and a keyboard. An output device typically paired with a mouse and keyboard would be a computer monitor. When commands are entered into the keyboard, or if the scroll wheel is rotated on the mouse, these actions are reflected on the monitor through animations being displayed.

Why is home brew development so popular with the Wiimote?
According to VGchartz, Wii hardware sales worldwide are estimated to be 54.58 million as of October 24, 2009. This comes out to be 48.4% of the video game console market share, with the Xbox 360 holding 29% and the Playstation 3 holding 22.6% respectively. Also, this number far exceeds sales of other computer devices including tablet PCs.

The innovation behind the Wiimote is the main obvious factor which has resulted in the high volume of sales. People are excited by unique methods of controlling ones character interactively through the use of motion. Thus with the number of sales and the excitement the Wiimote brings to its customers, people started to evaluate other methods of using the Wiimote technology.

What types of programs are available?
Currently, when it comes to interaction with the Wiimote, most home brew applications rely on the motion of the user controlling the Wiimote as well as the commands entered by pushing buttons on the controller. There seem to be two different types of programs available for the use of the Wiimote. The first are scripts written to allow the user to use the Wiimote for existing PC applications. This is primarily done through GlovePIE. The second would be ideas created entirely from scratch, and typically run via Microsoft's Visual Studio. Johnny Lee is responsible for the popular applications that fit into this category, including the virtual head tracker application, a finger tracking application, and an artificial smart board application.

These different applications may be difficult to run for some, as there are some prerequisites to having the applications work properly, especially with Johnny Lee's work. However, anyone who is at least somewhat smart when it comes to working with computers should not have any issues.

Some applications are easier to modify and understand than others. GlovePIE scripts tend to be easy to interpret and programming a script for it can be done in at a quick pace. Johnny Lee's projects tend to be more complex and more difficult to understand without any extensive prior programming experience.

Google Earth

One of the samples scripts that comes along with the free download of glovePIE allows a user to take control of the application Google Earth using the Wiimote.

The following comments are found at the beginning of the script:

Google Earth Interface using Wii Remote
by J.Coulston

Instructions:
Start the GlovePIE script, then calibrate your Wiimote (see next section).
After calibrating, make sure Google Earth is the top window.
Tilting up, down, left and right will simulate arrow keys.
Holding B while tilting up and down will tilt the view up and down.
Holding B while tilting left and right will rotate the view.
Pressing + and - will zoom when the Wiimote is level.
Pressing Home will center the view.
Pressing 1 will toggle fullscreen.

After starting the script I calibrated the Wiimote by referring to these instructions left by the author:

//Calibrate your Wiimote!
//Place the Wiimote face up on a flat surface. Change these values until the
//debug line next to the run button shows zero for each axis.
var.xOffset = 10
var.yOffset = -38
var.zOffset = 8

I then ran Google Earth as my top screen to see how user friendly this script actually was.

Tilting up and down worked as expected. But being able to rotate the globe left and right can be a little tricky at first. Tilting was probably not an accurate description of how to accomplish rotating left in right. I had to roll the Wiimote left or right in order to get the desired effects. One other issue I noticed was that there was only one speed setting. Speed was not reflected by how much I tilted and rolled.

Holding the B trigger down while tilting and rolling worked, but the same complaints listed above reflect on this function as well.

The - and + buttons did not perform the described function even after calibrating the Wiimote. These buttons seemed to just tilt the view of the camera up and down. I was not able to easily zoom in and out like I would with a mouse.

The Home button did center the view as expected, and pushing 1 did put the application in and out of full-screen mode.

A look at the code:
Some functions can be easily modified by simply making easy changes in the script.

The most noticeable problem users will probably face is issues with zooming in and out. The code responsible for this function looks like this:

if Wiimote.minus then
Ctrl+Down = not(abs(var.zRot)>10)
else
Ctrl = false
endif

if Wiimote.plus then
Ctrl+Up = not(abs(var.zRot)>10)
else
Ctrl = false
endif

Since the Wiimote is obviously replacing only keyboard commands, I decided to evaluate what Ctrl+Down and Ctrl+Up do when applied to Google Earth. As expected, these commands pan the camera up in down rather than zoom.

A recent post describes what is necessary to apply the keyboard buttons as an output source, so by using this knowledge, I can alter the code to get the desired results when using the Wiimote.

if Wiimote.minus then
key.Minus = not(abs(var.zRot)>10)
else
key.Minus = false
endif

if Wiimote.plus then
key.Equals = not(abs(var.zRot)>10)
else
key.Equals = false
endif

Unfortunately, I have yet to figure out a way to vary the speed of rotation/zooming using the Wiimote (similar to how a mouse would display such an effect). It is possible to mimic the mouse buttons and movement, however, I have not been able to calculate an algorithm in which the Wiimote can create an analog form of speed.

Blue Tunes

The first application tested was an interesting and simple one entitled Blue Tunes.

What is it?

Blue Tunes is a software application that allows a user to control his or her music and video library via Wii remote.

What is required?

• Connection between a Windows based PC and Wiimote
• Blue Tunes software (Download)
• Media player

What media players can be used? (* indicates which media players were tested)

iTunes*
Winamp
Napster
Windows Media Player*
VLC*
Quintessential Player
Media Player Classic
Foobar2000
Yahoo! Music Jukebox
MusikCube
MediaMonkey
Apollo
BS.Player
Songbird (beta v0.3)
GOM Player
Creative MediaSource
JetAudio
Hulu Desktop

Impressions:

Blue Tunes is easy to download and install. Being able to run it successfully is simple and self-explanatory, too. There are a variety of control options available which allow the user to play, pause, skip forward/backward, increase/decrease volume, increase/decrease bass, go to the next/previous track, add to a playlist, and turn the shuffle on/off. Each individual button has two assignments. Its assignment is determined by whether or not the B-trigger is being held. Blue Tunes does provide a graphic layout explaining the functions of each button.

Blue Tunes would be a great and easy tool to use during a party scene or whenever music needs to be played in general. With iTunes, the response is nearly flawless, and the distance that one can use it from is exceptional. (Approx 50 yards)

General complaints: Using the Bluetooth connection in general can drain the Wiimotes battery rather quickly. If the user isn't careful, the Wiimote can be left on for a couple of hours without being used, thus requiring a fresh pair of batteries soon after. I see this as being an issue with multiple related applications, and wish there could be an easy way to turn the Wiimote on/off while being able to maintain a connection to the PC.

iTunes: Blue Tunes was designed around iTunes, thus no noticeable issues were found while using the Wiimote to perform based on the default button layout. The response time between the application and the Wiimote was great. All the buttons worked as described.

Windows Media Player: Although most of the basic functions worked like they were suppose to with WMP (play/pause, next/previous track, volume), some wouldn't work at all. Skipping forward/backward didn't work as expected, and once the shuffle was turned on, the only available working function was the play/pause button. Turning the shuffle back off didn't fix the issue.

VLC: Although VLC is a great media player for multiple media formats, using Blue Tunes with it does not seem practical. The play/pause and volume functions work fine. There are no playlists with VLC, so being able to go to the previous/next track is useless. The skip forward/backward functions did not seem to work either.

GlovePIE 1: "Hello"

Being able to write a script in GlovePIE is pretty simple, especially with previous programming experience. Watching Youtube tutorial videos provided by user theriddler24 can be quite useful for newbies. As of the writing of this entry, no detailed text tutorials have been found.

The buttons on the Wiimote obviously correspond to a boolean value. If the button is pushed down, its value becomes true. Likewise, if the button is left up, its value becomes false.

Being able to map the buttons of an input device to the buttons of an output device is done by typing and running:

[outputdevice].buttonname = [inputdevice].buttonname

When coding scripts which allow the Wiimote to work with PC applications, we typically will treat the keyboard as the output device and the Wiimote as the input device. Thus, if we want to output the letter 'a' on a text program using a Wiimote, we would write a line of script in GlovePIE that would look something like this:

keyboard.a = wiimote1.a

*Note that wiimote1 represents the first Wiimote connected via bluetooth. Wiimote2 would represent the second, and so on. GlovePIE allows one to program up to eight Wiimotes.

Running the following script allows us to output the text "hello " to an output window GlovePIE provides. To access the output window, on the toolbar select TroubleShooter -> Remote Test.

Notice that this script also allows us to use the backspace key by pressing the B-trigger on the Wiimote. The cursor can also be moved by pressing the corresponding button on the direction pad. On the output screen, the text "hello " is displayed every time the A-button on the Wiimote is pressed.

GlovePIE

GlovePIE is a popular input emulator which allows for users to play PC based games and other applications through other means, rather than just a keyboard and mouse.

GlovePIE (Glove Programmable Input Emulator) was created by Carl Kenner and was originally intended to be a way of emulating keyboard and mouse using a virtual glove. The project grew to be more than just allowing a virtual glove, and there is now tremendous support for using the emulator with a Wiimote. Once downloaded from the creator's site, dozens of scripts are available which allow one to play first person shooters, manipulate Google Earth, or operate Firefox all through the use of just a Wiimote. These scripts are openly available to look at and modify. An example of what the scripts look like can be seen by clicking the image below: