Multi-touch technology is becoming more common place in both industrial and casual computing. With this rise in use developers are producing a wider range of multi-touch interfaces, each with different elements in their design. One element that most have in common is the shape of the display.

However, with the growing desire for non-rectangular displays to facilitate ubiquitous computing it is becoming apparent that multi-touch systems will need to support a range of display shapes. Several technologies already exist which can facilitate this. Other display technologies can easily be adapted to different display shapes as well.

Displays adapted to non-rectangular shapes

However most software is currently designed assuming a rectangular visual output. This results in the developed software being reliant on a particular display shapes. This work focused on finding ways of adapting software to be able to adapt its visual content to any display shape.

It was discovered that when software which was dependant on a rectangular visual output were used with a non-rectangular display, occlusion of regions of the output occurred. This occlusion resulted in visual information becoming lost and software becoming unusable.

Several different methods of positioning visual content to avoid occlusion were developed. One of the methods involved mapping the layout of content items to a virtual rectangle. This rectangle was then scaled, rotated and posited to fit the display (using a largest empty rectangle finding method). Alternatives to this method discussed in the work included deforming the shape of the visual contents to match the output or squashing the layout of the content items to pull content items into the display shape. These methods did resolve the occlusion problem but did result in the layout of content items being lost. This may not be desirable for some content items which are positioned relative to each other in a specific way – for example if the content items were laid out in a grid these methods would deform the appearance of the grid.

The method which returned the best results was discovered to be a combination of two methods. First the virtual rectangle method is used to resolve occlusion. Then for content items to which layout is not important (in multi-touch systems these items are normally those which users can manipulate directly through gestures) their initial layout is stretched. This pulls these content items out of the virtual rectangle into previously unused areas of the display. This maximises the initial use of the display’s real estate.

A technique for adapting content to a non-rectangular display

These methods of resolving occlusion from different display shapes was implemented into the SynergyNet framework. With the methods implemented at a framework level, any applications supported by the software became capable of utilising different shaped displays. Footage captured from the software during the evaluation of the methods can be seen in the following videos:

The results of the evaluation were promising and show that it is possible to allow multi-touch software to dynamically adapt its visual contents to different display shapes. Of course the methods of allowing software to dynamically adapt itself to different display shapes are not constrained to multi-touch systems. Methods for resolving occlusion caused by different display shapes could be used in any software. Substantially more information on the work done is available in the related thesis:

The Thesis in which this work is discussed can be read here