In this previous post I mentioned that we at TEL in Durham had been running some studies using the Kinect with SynergyNet. Though data analysis is still being carried out on the results I’ve decided to provide some details on the system, its working and its capabilities, in addition to some of the initial findings.
The SynergyNet 3 Tracking project allows for a teacher to issue commands to instances of the SynergyNet framework by performing gestures. These commands can be the initiating of content transfers between instances, the creation of snapshots of work being done, freezing/unfreezing of the interfaces and clearing of content. These are common commands often carried out in the SynergyNet classroom environment.
The design of the system was influenced through the use of a focus group. This guessability study consisted of over 70 trainee teachers who were asked to perform the first gesture that came to mind when given one of the common SynergyNet commands.
To evaluate the tracking system when implemented, a study was carried out where performance using the Kinect was compared with alternative SynergyNet control technologies. The alternative technologies included an instance of a SynergyNet projector app made accessible through a multi-touch board and SynergyNet’s web interface made accessible through a tablet. Performance was defined by the time taken to issue commands to SynergyNet and the error rate. The error rate was defined by the number of commands which instigated what a teacher using the system wanted vs. the number of commands which resulted in a undesired consequence.
The study was undertaken by a teacher and a class of 9-11 year olds. The class were asked to carry out a task called Mysteries using the SynergyNet framework on multi-touch tables. Mysteries comprises of a number of clues which the students must use to answer a question. The studies were recorded and the teachers’ use of the control technologies was analysed with help from the SynergyView tool.
Despite the speed of commands issued with the Kinect being much quicker than that observed for the multi-touch board and tablet based controls, the error rate for the Kinect observed in the first study was extremely high, making it unsuitable for classroom use.
A second study was carried out. For this study a number of improvements were made to the SynergyNet 3 Tracking project. The most significant of these improvements was the support of multiple Kinects. In the first study only one Kinect was used. This resulted in the teacher often leaving the area observed by the device. This led to erroneous commands being issued as the Kinect could not correctly interpret the gestures performed by the teacher outside of the accurately observed area. The use of multiple Kinect would expand the accurately observed area and reduce the chance of this happening.
To allow for multiple Kinects to be used several issues had to be overcome. The first of these was the interference caused by two Kinects’ projected light patterns overlapping. To avoid this, the Kinects were positioned perpendicular to each other. If spaced out correctly this could allow for four Kinects to be used with the system. However, in the study two were sufficient for covering the entire classroom.
The next issue was the merging of observed data. Though the stitching of Kinect’s depth data has been used in several projects before, it was decided that the positional data derived by the Kinect from the depth data would be merged to reduce network traffic. The instances of SynergyNet which manage the Kinects are made aware of the devices’ positions relative to other instances of SynergyNet through their config settings. This allows any observed positional information to normalised. Therefore, where entities (a joint, limb or centre of mass) viewed by multiple Kinects are observed to have the same normalised location, they can be assumed to be the same entity and merged. This allows for the tracking of users, limbs and joints across multiple Kinects.
The second study used the same setup and measured the same parameters as the first with a different teacher and students. A short video showing some moments from the second study can be seen below:
The results of the second study indicated a much lower error rate. However, the error rate appeared to still be too high to be deemed suitable for classroom use.
The results from both the studies carried out appear to imply that the Kinect is the faster technology for issuing commands. However, its error rate is far too high to make it a viable choice for classroom use, even with the improvements made for the second study. The results seem to imply that the Kinect’s accuracy is the cause of most of the errors, especially in the second study. The new Kinect, to be released with the Xbox One, or alternative technologies may resolve the issues caused by this shortcoming. Future studies may focus on using the same system with better tracking technologies.
To try running the SynergyNet 3 Tracking software yourself follow the steps given on this wiki page.
To try developing with the system follow the steps given on this wiki page.
J.A.McNaughton 