Abstract: Current tabletop applications, which go beyond mouse-like input, mostly use gestures and/or tangible objects. Gestures are mostly recognized by matching input to a given set of predeï¬ned (scripted) gestures, and tangible input is implemented by recognizing shapes or markers. While this allows exactly the interactions foreseen by the programmer, it can be observed in the real world, that physical artefacts are used much more ï¬exibly and often in unforeseen ways. We describe two of our explorations into tabletop interaction styles, discuss their respective strengths and shortcomings, and then derive a new model for tabletop interaction based on a physics simulation and rich multi touch input. Interaction with this model achieves a very high ï¬delity to the physical world and allows appropriation of the interface, i.e., the development of individual and unforeseen interactions with the system. We have evaluated our model in different variations and believe that it holds a strong promise for future, highly ï¬exible tabletop interfaces.
Abstract: HoloDesk is an interactive system combining an optical see through display and Kinect camera to create the illusion that users are directly interacting with 3D graphics. A virtual image of a 3D scene is rendered through a half silvered mirror and spatially aligned with the real-world for the viewer. Users easily reach into an interaction volume displaying the virtual image. This allows the user to literally get their hands into the virtual display and to directly interact with an spatially aligned 3D virtual world, without the need for any specialized head-worn hardware or input device. We introduce a new technique for interpreting raw Kinect data to approximate and track rigid (e.g., books, cups) and non-rigid (e.g., hands, paper) physical objects and support a variety of physics-inspired interactions between virtual and real. In particular the algorithm models natural human grasping of virtual objects with more fidelity than previously demonstrated. A qualitative study highlights rich emergent 3D interactions, using hands and real-world objects. The implementation of HoloDesk is described in full, and example application scenarios explored. Finally, HoloDesk is quantitatively evaluated in a 3D target acquisition task, comparing the system with indirect and glasses-based variants.
Abstract: We present a system for accurate real-time mapping of complex and arbitrary indoor scenes in variable lighting conditions, using only a moving low-cost depth camera and commodity graphics hardware. We fuse all of the depth data streamed from a Kinect sensor into a single global implicit surface model of the observed scene in real-time. The current sensor pose is simultaneously obtained by tracking the live depth frame relative to the global model using a coarse-to-fine iterative closest point (ICP) algorithm, which uses all of the observed depth data available. We demonstrate the advantages of tracking against the growing full surface model compared with frame-to-frame tracking, obtaining tracking and mapping results in constant time within room sized scenes with limited drift and high accuracy. We also show both qualitative and quantitative results relating to various aspects of our tracking and mapping system. Modelling of natural scenes, in real-time with only commodity sensor and GPU hardware, promises an exciting step forward in augmented reality (AR), in particular, it allows dense surfaces to be reconstructed in real-time, with a level of detail and robustness beyond any solution yet presented using passive computer vision.
Abstract: The Family Archive device is an interactive multi-touch tabletop technology with integrated capture facility for the archiving of sentimental artefacts and memorabilia. It was developed as a technology probe to help us open up current family archiving practices and to explore family archiving in situ. We detail the deployment and study of three of these devices in family homes and discuss how deploying a new, potentially disruptive, technology can foreground the social relations and organizing systems in domestic life. This in turn facilitates critical reflection on technology design.
Abstract: In this paper we discuss some of our recent research work designing tabletop interfaces for co-located photo sharing. We draw particular attention to a specific feature of an interface design, which we have observed over an extensive number of uses, as facilitating an under-reported but none- the-less intriguing aspect of the photo-sharing experience namely the process of getting sidetracked. Through a series of vignettes of interaction during photo-sharing sessions we demonstrate how users of our tabletop photoware system used peripheral presentation of topically incoherent photos to artfully initiate new photo-talk sequences in on-going discourse. From this we draw implications for the design of tabletop photo applications, and for the experiential analysis of such devices.
Abstract: Although interactive surfaces have many unique and compelling qualities, the interactions they support are by their very nature bound to the display surface. In this paper we present a technique for users to seamlessly switch between interacting on the tabletop surface to above it. Our aim is to leverage the space above the surface in combination with the regular tabletop display to allow more intuitive manipulation of digital content in three-dimensions. Our goal is to design a technique that closely resembles the ways we manipulate physical objects in the real-world; conceptually, allowing virtual objects to be âpicked upâ off the tabletop surface in order to manipulate their three dimensional position or orientation. We chart the evolution of this technique, implemented on two rear projection-vision tabletops. Both use special projection screen materials to allow sensing at significant depths beyond the display. Existing and new computer vision techniques are used to sense hand gestures and postures above the tabletop, which can be used alongside more familiar multi-touch interactions. Interacting above the surface in this way opens up many interesting challenges. In particular it breaks the direct interaction metaphor that most tabletops afford. We present a novel shadow-based technique to help alleviate this issue. We discuss the strengths and limitations of our technique based on our own observations and initial user feedback, and provide various insights from comparing, and contrasting, our tabletop implementations
Abstract: On traditional tables, people often manipulate a variety of physical objects, both 2D in nature (e.g., paper) and 3D in nature (e.g., books, pens, models, etc.). Current advances in hardware technology for tabletop displays introduce the possibility of mimicking these physical interactions through direct-touch or tangible user interfaces. While both promise intuitive physical interaction, they are rarely discussed in combination in the literature. In this paper, we present a study that explores the advantages and disadvantages of tangible and touch interfaces, specifically in relation to one another. We discuss our results in terms of how effective each technique was for accomplishing both a 3D object manipulation task and a 2D information visualization exploration task. Results suggest that people can more quickly move and rotate objects in 2D with our touch interaction, but more effectively navigate the visualization using tangible interaction. We discuss how our results can be used to inform future designs of tangible and touch interaction.
Abstract: We present a new approach for rapidly prototyping multi-touch and object sensing surfaces. It works by liquid displacement inside a malleable projection surface. The system provides both touch and pressure information and a distinct organic quality when touched. The system is easy to build and produces a clean signal revealing multiple fingers, whole hands and other object outlines that can be processed using computer vision techniques. This approach provides an easy mechanism to build interactive surfaces, requiring no infrared edge lighting or soldering. In this paper we provide an overview of the approach, some of its unique capabilities, and uncover some of the tradeoffs between viscosity of liquid, air pressure, surface malleability and the volume of liquid used. Our aim is to allow practitioners - from DIY enthusiasts to researchers - to build and experiment with such systems more readily.
Abstract: Photohelix: = Takuma = Ting-Ting Hu and Yi-Wei Chia and Li-Wei Chan and Yi-Ping Hung and Jane Hsu, title = i-m-Top: An Interactive Multi-Resolution Tabletop System Accommodating to Multi-Resolution Human Vision, booktitle = Proceedings of the 3rd IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP 2008), year = 2008, abstract = i-m-Top: âxqrWC\^NeBu âxeugbvVXe qrWAoFx S A( )AoxB oNIeBA xfBXvC AXAvâeugbvVXeqrW lxpB Xi-m-Top(C\^NeBu âx
Abstract: Collaborative creativity is traditionally supported by formal techniques, such as brainstorming. These techniques improve the idea-generation process by creating group synergies, but also suffer from a number of negative effects. Current electronic tools to support collaborative creativity overcome some of these problems, but introduce new ones, by either losing the benefits of face-to-face communication or the immediacy of simultaneous contribution. Using an interactive environment as a test bed, we are investigating how collaborative creativity can be supported electronically while maintaining face-to-face communication. What are the design-factors influencing such a system? We have designed a brainstorming application that uses an interactive table and a large wall display, and compared the results of using it to traditional paper-based brainstorming in a user study with 30 participants. From the considerations that went into the design and the observations during the study we derive a number of design guidelines for collaborative systems in interactive environments.
Abstract: This dissertation describes an exploration of digital tabletop interaction styles, with the ultimate goal of informing the design of a new model for tabletop interaction. In the context of this thesis the term digital tabletop refers to an emerging class of devices that afford many novel ways of interaction with the digital. Allowing users to directly touch information presented on large, horizontal displays. Being a relatively young field, many developments are in flux; hardware and software change at a fast pace and many interesting alternative approaches are available at the same time. In our research we are especially interested in systems that are capable of sensing multiple contacts (e.g., fingers) and richer information such as the outline of whole hands or other physical objects. New sensor hardware enable new ways to interact with the digital. When embarking into the research for this thesis, the question which interaction styles could be appropriate for this new class of devices was a open question, with many equally promising answers. Many everyday activities rely on our hands ability to skillfully control and manipulate physical objects. We seek to open up different possibilities to exploit our manual dexterity and provide users with richer interaction possibilities. This could be achieved through the use of physical objects as input mediators or through virtual interfaces that behave in a more realistic fashion. In order to gain a better understanding of the underlying design space we choose an approach organized into two phases. First, two different prototypes, each representing a specific interaction style namely gesture-based interaction and tangible interaction have been implemented. The flexibility of use afforded by the interface and the level of physicality afforded by the interface elements are introduced as criteria for evaluation. Each approaches suitability to support the highly dynamic and often unstructured interactions typical for digital tabletops is analyzed based on these criteria. In a second stage the learnings from these initial explorations are applied to inform the design of a novel model for digital tabletop interaction. This model is based on the combination of rich multi-touch sensing and a three dimensional environment enriched by a gaming physics simulation. The proposed approach enables users to interact with the virtual through richer quantities such as collision and friction. Enabling a variety of fine-grained interactions using multiple fingers, whole hands and physical objects. Our model makes digital tabletop interaction even more natural. However, because the interaction the sensed input and the displayed output is still bound to the surface, there is a fundamental limitation in manipulating objects using the third dimension. To address this issue, we present a technique that allows users to conceptually pick objects off the surface and control their position in 3D. Our goal has been to define a technique that completes our model for on-surface interaction and allows for as-direct-as possible interactions. We also present two hardware prototypes capable of sensing the users interactions beyond the tables surface. Finally, we present visual feedback mechanisms to give the users the sense that they are actually lifting the objects off the surface. This thesis contributes on various levels. We present several novel prototypes that we built and evaluated. We use these prototypes to systematically explore the design space of digital tabletop interaction. The flexibility of use afforded by the interaction style is introduced as criterion alongside the user interface elements physicality. Each approaches suitability to support the highly dynamic and often unstructured interactions typical for digital tabletops are analyzed. We present a new model for tabletop interaction that increases the fidelity of interaction possible in such settings. Finally, we extend this model so to enable as direct as possible interactions with 3D data, interacting from above the tables surface.
