Abstract: The cockpit of the future will not only digital – it will be networked and combine several displays. Windscreen projections, which today present prototype augmented reality head-up displays of the future will be shown on the entire instrument panel and side windows where they can be used interactively. This requires novel and intelligent tracking systems. RTT uses such systems and provides state-of-the-art 3D visualisation services. What is succeeding impressively today in presentation technology will in future form part of extensively networked driver assistance systems.
Abstract: This paper introduces a system for real-time physiological measurement, analysis, and metaphorical visualization within a virtual environment (VE). Our goal is to develop a method that allows humans to unconsciously relate to parts of an environment more strongly than to others, purely induced by their own physiological responses to the virtual reality (VR) displays. In particular, we exploit heart rate, respiration, and galvanic skin response in order to control the behavior of virtual characters in the VE. Such unconscious processes may become a useful tool for storytelling or assist guiding participants through a sequence of tasks in order to make the application more interesting, e.g., in rehabilitation. We claim that anchoring of subjective bodily states to a virtual reality (VR) can enhance a person’s sense of realism of the VR and ultimately create a stronger relationship between humans and the VR.
Abstract: Brain–computer interfaces (BCIs) are becoming more and more popular as an input device for virtual worlds and computer games. Depending on their function, a major drawback is the mental workload associated with their use and there is significant effort and training required to effectively control them. In this paper, we present two studies assessing how mental workload of a P300-based BCI affects participants' reported sense of presence in a virtual environment (VE). In the first study, we employ a BCI exploiting the P300 event-related potential (ERP) that allows control of over 200 items in a virtual apartment. In the second study, the BCI is replaced by a gaze-based selection method coupled with wand navigation. In both studies, overall performance is measured and individual presence scores are assessed by means of a short questionnaire. The results suggest that there is no immediate benefit for visualizing events in the VE triggered by the BCI and that no learning about the layout of the virtual space takes place. In order to alleviate this, we propose that future P300-based BCIs in VR are set up so as require users to make some inference about the virtual space so that they become aware of it, which is likely to lead to higher reported presence.
Abstract: Abstract This work investigates novel alternative means of interaction in a virtual environment (VE). We analyze whether humans can remap established body functions to learn to interact with digital information in an environment that is cross-sensory by nature and uses vocal utterances in order to influence (abstract) virtual objects. We thus establish a correlation among learning, control of the interface, and the perceived sense of presence in the VE. The application enables intuitive interaction by mapping actions (the prosodic aspects of the human voice) to a certain response (i.e., visualization). A series of single-user and multiuser studies shows that users can gain control of the intuitive interface and learn to adapt to new and previously unseen tasks in VEs. Despite the abstract nature of the presented environment, presence scores were generally very high.
Abstract: A brain-computer interface (BCI) is a new communication channel between the human brain and a digital computer. Such systems have been designed to support disabled people for communication and environmental control. In more recent research also BCI control in combination with Virtual Environments (VE) gains more and more interest. Within this study we present experiments combining BCI systems and VE for navigation and control purposes just by thoughts. Results show that the new P300 based BCI system allows a very reliable control of the VR system. Of special importance is the possibility to select very rapidly the specific command out of many different choices. The study suggests that more than 80% of the population could use such a BCI within 5 minutes of training only. This eliminates the usage of decision trees as previously done with BCI systems.
Abstract: This thesis is concerned with the development and evaluation of whole-body interfaces and their application in virtual reality. In particular we examine two orthogonal elements of presence, namely place illusion and plausibility. The research presented in this thesis comprises four experiments that examine different types of whole-body interaction – physical, physiological or mental - and assess them in terms of place illusion and plausibility.
For the first experiment our hypothesis was that whole-body movements influencing the behaviour of an abstract environment give rise to plausibility even the environment and effects of user actions were previously unknown. The experiment concerned correlations between whole-body movements and a virtual environment in which participants used a Hula Hoop in order to interact with a particle system displayed consisting of hoop-shaped objects that would individually and collectively respond to the participant’s actions. The immersive environment was displayed on a powerwall. The hypothesis was supported indicating that people can quickly adapt to a new environment and experience plausibility.
The goal of the second experiment was to assess the feasibility of using subjective and mostly unconscious physiological response as a means to modify or enhance certain elements of the virtual environment and thus enhance plausibility. The ultimate goal is to use physiology as an additional tool for storytelling, for example in order to modify or enhance the narrative by increasing the tie between a human and events or other elements of a virtual environment. The underlying assumption was that real and recognizable behavioural constituents, in this case physiological responses of the participant, visualized in a virtual environment should be identifiable by him or her. The second experiment thus also addressed plausibility and we explored how unconscious physiological interactions linked to the behaviour of virtual characters can increase the bond between participant and that character in an environment displaying several virtual characters all exhibiting similar behaviour. The behaviour of one of them was controlled by the participant’s physiology while the remaining ones were automated. Our results suggest that own physiological responses visualized through the behaviour of a virtual character cannot be discerned from a pool of similar but automated stimuli.
The third and fourth experiment both aim to quantify whether brain-computer interfaces used as a universal input device for a virtual environment provide a feasible and stable method for interaction and also if their use affects place illusion, the illusion of being located in a virtual environment. A secondary aim was to demonstrate the feasibility of virtual reality for rapid prototyping of a smart home containing fully automated appliances that can be controlled remotely. In the first of the two experiments we measured performance and subjective presence scores of 12 participants using a P300 brain-computer interface to navigate and interact with objects in a virtual apartment. In the second experiment we collected subjective reports on presence from 12 participants in the same environment, although this time it was controlled via a combination of wand navigation and gaze-based object selection techniques. Our results clearly indicate that when operating the environment via a P300 brain-computer interfaces participants’ place illusion are significantly lower, possibly due to high mental workload.
The research presented in this thesis examined physical, physiological and mental means of interaction with virtual environments. The overall conclusion is that while most tools developed yield adequate performance levels, not all of the assessed environments give rise to the desired effect of enhancing presence and this is true in particular when using a brain-computer interface possibly due to high mental workload and split attention.
