This might be very intersting for the NUI-Community and MT development.
Today i received a master-thesis from a friend who graduated two days ago.

Here is a summary:

“Abstract
Even though multi-touch screens based on frustrated total internal reflection
(FTIR) principle have gained significant popularity over the last years their
overall performance is often limited by design. In particular, we identified robustness
towards ambient light, high-speed tracking and precise calibration as key
issues in many systems. In this thesis we propose a set of software and hardware
extensions for a custom-built FTIR screen that improve on these aspects by reducing
the impact of ambient light, optimizing tracking performance and performing
an automated calibration at high precision. Comparisons with existing approaches
reveal great optimization potential: Our system performs well under intense and
rapidly changing lighting conditions, its tracking runtime is up to 30x faster than
Touchlib and calibration achieves sub-pixel resolution. The results are general
enough to apply equally well to other camera-based technologies and might influence
the next generation of multitouch processing software.

Despite its research history, only in recent years with the introduction of the
FTIR1 approach the number of researchers and enthusiasts realizing the potential
of the technology grew significantly. Several companies [Mic07, Per07, IB07]
have also developed products based on FTIR or closely related technologies. Even
a small selection of libraries and SDKs dedicated to multitouch-specific processing
tasks are openly available and the acceptance of the technology is increasing.
Still, the used software and hardware components are in a very early stage.
Over the course of a year we have seen many different system configurations, but
were mostly disappointed of the overall performance. First, either tracking accuracy
or latency or both altogether were not satisfying enough to use the touch
screen as a primary human computer interface (HCI). Second, the lack of robustness
towards ambient light limit their applicability. Third, systems exhibit errorprone
calibration methods based on tedious manual procedures. We identified
these three aspects as key issues that deserve a more thorough discussion.
This thesis aims to present ideas from established research areas, translate
them to this domain and discuss how they apply and perform.”

Basically we connected the cameras with LED frame...simple explanation

cheerz
goat

Could you please attach the pdf?

Cheers Chris

Im sorry i cant publish the complete PDF - These are parts which were already published:

Chapter 3 - “Tracking Performance:

Among the most crucial requirement of a HCI device is its ability to quickly respond
to user input, because if the latency between user input and its processing
result is too high, the device feels unattractively slow and lagging. At the same
time, the device should not react randomly, but exactly in the particular way the
user envisioned it. Thus, the goal is to achieve low latency and low error rate
simultaneously. The tracking performance of an FTIR screen plays an important
role in achieving these goals. This chapter discusses a system design that is able
to achieve fast and robust multitouch tracking.
In the following sections we will give a problem definition, provide a solution
using the forward/reverse auction algorithm, evaluate the system and compare
runtime and accuracy with a widely-used open source multitouch library.
..
.

3.5 Conclusion
We presented a design for FTIR signal tracking based on the forward/reverse auction
algorithm. The system was shown to achieve low latency and low error rates
even for larger tracking problems. Alternative approaches based on established
tracking algorithms such as KLT feature tracker and optical flow are either not
compatible with FTIR signal properties or computationally prohibitive. We reduce
raw images to trackable points and showed that point tracking can have low
computational complexity while a robust tracking with an error rate around 2%
can be achieved.
We compared our approach to a widely-used open source multitouch library.
The forward/reverse auction algorithm performs between 5x to 30x better in runtime
and equally for small and even for better larger problem sizes. In a brief
analysis of the Touchlib library we discussed its limitations.”

@all

so what do you think ?

“Our system performs well under intense and
rapidly changing lighting conditions, its tracking runtime is up to 30x faster than
Touchlib and calibration achieves sub-pixel resolution. The results are general
enough to apply equally well to other camera-based technologies and might influence
the next generation of multitouch processing software.”

last week i saw the table in action. it is by far the most responsive system… even faster than the PQLabs IR-Frame i saw at CEBIT. he also presents a fast&easy-30-sec-autocalibration - it works perfect!

isnt it somehow possible to “adapt” this method to CCV?

@Christian & AlexP and other developers?

hi goat
i think it is possible when he developers see the algorithm. did you ask your friend to share this beautiful algorithm?

....

the_goat - 27 March 2010 01:49 PM

here is the extracted chapter that explains the used algorithm.

i would love to get some feedback on this from christian, alexp and other developers.

Could you please post the whole thesis. The Chapter 2 does not explain the “forward/reverse auction algorithm” used in your project. As far as the LED strobing and the background subtraction outlined in chapter 2.4, this is doable but requires some hardware to be built in addition to code modifications to the CCV.

Wow,

This is some really super impressive stuff. So the question is, with this be made public either by way of open source with a breakdown of hows its done or commercially. Ether way I am sure there are many folks here that would love to see this in action. I for one am always looking for alternative ways of doing things.

Killer work

Sam

Hi,

LED strobing and syncing of camera captures/shutters are widely used in the motion capture areas for film and games. Typically cameras are synced via some TTL signal to a master clock such that multiple camera setups have correlating 2D data to process. This is very important for 3D reconstruction techniques.

As for the auction algorithm. Info can be found here: http://wapedia.mobi/en/Auction_algorithm as well as here: http://en.wikipedia.org/wiki/Auction_algorithm.
It is mainly a bipartite graph matching technique of which the ‘Hungarian’ method (http://en.wikipedia.org/wiki/Hungarian_algorithm) is the most well known. The time complexity of the Hungarian algorithm is O(n^4), although there are heuristic adaptations to make it run in O(n^3). Here is a nice algorithmic explanation of the Hungarian algorithm: http://www.topcoder.com/tc?module=Static&d1=tutorials&d2=hungarianAlgorithm

ciao,
Sean.
PS: this is my first post on this forum, and yes, I’ll be building my own multi-touch surface able to track fingers and fiducials (the hardware is just the start - I’m more interested in the software side of things....)

unfortunately i didnt have the time to do some more research on this subject even though it is really interesting because it could lead to very stable (terms of ambient light) mt-setups. but does any of you know a working setup with “pulsed leds” or similar technology?

Hey the_goat, this is from a Masters thesis right? You should be able to publish the entire PDF as it becomes public domain after graduation (at least it does here in Australia) for general reading. It would be interesting to fill in the blanks.