TactoSonix is a multitouch environment for musical expression. It consists of a hardware interface that was designed and built by the author and an Industrial Design consultant, as well as software that is being developed as part of the author's M.A. thesis research.

The TactoSonix hardware stands out from most other multitouch devices in that it supports two methods of implementing multitouch technology: Frustrated Total Internal Reflection and Rear Diffused Illumination. As such, it is an excellent platform for prototyping ideas about tangible objects, and also offers a more robust finger tracking method in its FTIR configuration.

Thanks to the arrival of certain consumer products on the market, multitouch technology is now gaining exposure to a much wider audience than ever before in the 40 years of its existence.

Long before Apple's iPad, researchers at the University of Illinois and at the University of Toronto were designing hardware for single-touch and multitouch applications. It was, however, Jeff Han's breakthrough demo at TED 2005 that showed the world the possibilities offered by this technology, when coupled with the power of today's computers. A few years on, one might wonder why this technology hasn't yet delivered on the promise of revolutionizing human-machine interaction.

Multitouch capability in most current software applications is often limited to tasks such as the manipulation of buttons and sliders derived from the desktop metaphor inherited from Xerox PARC. A highly popular example of multitouch capability consists of zooming in/out and manipulating images, although it is another instance of the desktop metaphor.

Given this restrictive trend in software design for multitouch interfaces, we ask the following questions:

• What is the importance of metaphor in multitouch musical software design?
• How does software design vary in multitouch interfaces when compared to the traditional WIMP (Window, Icon, Menu, Pointing Device) setup?
• Why choose one implementation of multitouch technology, and not another?

There are several ways to implement multitouch support from a hardware perspective. Without going into too much detail, here are a few:

• Camera-based solutions
  • Front Diffused Illumination
  • Rear Diffused Illumination (RDI)
  • Laser Light Plane
  • Frustrated Total Internal Reflection (FTIR)
  • Diffused Surface Illumination
  • Stereoscopy
• Capacitive sensing
• Resistive technology
• Mechanical displacement
• Optical fibre
• Acoustic sensing
• Dispersive signal technology

These approaches represent the way to implement touch sensing. On top of this distinction, multitouch devices vary with respect to the visual feedback provided:

• None
• Tangibles (cardboard, fabric, objects with fiducials, etc.)
• Front display (projector above)
• Rear display (projector below)
• Embedded display (LCD)

TactoSonix is a multitouch table prototype that implements two types of camera-based approaches: FTIR and RDI. It is an aesthetically pleasant (or so we hope!) device with larger dimensions than consumer-ready products, and was built with low cost (by Canadian standards) components from home renovation and electronics stores.

The table is meant to be connected to an external computer through its VGA (for the projector) and USB (for the webcam) connectors. Existing multitouch software can therefore be run on our hardware, provided the webcam's image can be interpreted to generate TUIO events.

When switching from one configuration to the other, the toggle must be moved to the appropriate position. The tabletop must also be changed to the right one. RDI technology uses a more transparent surface than FTIR (as can be seen below), so that the camera can identify fiducials on the interaction surface. An FTIR tabletop consists of a projection surface (vellum paper) onto which liquid silicon has been poured and spread. To protect the exposed vellum (FTIR) and semitransparent surface (RDI) from dirty fingers and scratches, the topmost layer of the plexiglass frames consists of a thin transparent layer of nonadhesive plastic.

Although it is used to investigate the potential of multitouch technology in music, the TactoSonix hardware can also be used with any multitouch software. Whereas commercial platforms (*cough cough* Apple) tend to restrict you to their approved API, a DIY system gives you the full freedom to write your own software.

Since extensive information is available from DIY communities interested in multitouch devices, initial guidance was obtained from such websites (most notably from the NUIGroup forums).

Initially an FTIR setup with vellum paper as a projection surface, the TactoSonix hardware has evolved into an FTIR-or-RDI configuration. This change came from the need to support tangible objects with fiducials, which were invisible to the webcam in a setup with vellum paper. Additionally, heat from the projector (see pictures above) had caused ripples in the vellum paper, which served as an incentive to find another solution.

The first FTIR implementation yielded poor blob quality, so it was decided to use a silicon layer as a compliant surface. Tinkerman's method was used, and the quality of the blobs was greatly improved.

To (try to) eliminate the risk of damaging the electronic wiring of the table, it was decided to connect the FTIR circuit to the bottom part of the table via contact connections. These are implemented using simple screws, as can be seen below.

To implement diffused illumination, a few infrared LEDs were assembled onto a small breadboard, and the circuit was covered in silicon, with the aim of diffusing the infrared light. This component is shown below. At the moment, the light is not diffused enough, and a bright spot appears on the webcam image, so a better solution will have to be found.