Abstract: Displaying anatomical and physiological information derived from preoperative medical images in the operating room is critical in image-guided neurosurgery. This paper presents a new approach referred to as augmented virtuality (AV) for displaying intraoperative views of the operative field over three-dimensional (3-D) multimodal preoperative images onto an external screen during surgery. A calibrated stereovision system was set up between the surgical microscope and the binocular tubes. Three-dimensional surface meshes of the operative field were then generated using stereopsis. These reconstructed 3-D surface meshes were directly displayed without any additional geometrical transform over preoperative images of the patient in the physical space. Performance evaluation was achieved using a physical skull phantom. Accuracy of the reconstruction method itself was shown to be within 1 mm (median: 0.76 mm +/- 0.27), whereas accuracy of the overall approach was shown to be within 3 mm (median: 2.29 mm +/- 0.59), including the image-to-physical space registration error. We report the results of six surgical cases where AV was used in conjunction with augmented reality. AV not only enabled vision beyond the cortical surface but also gave an overview of the surgical area. This approach facilitated understanding of the spatial relationship between the operative field and the preoperative multimodal 3-D images of the patient.

Abstract:

Abstract: During an image-guided neurosurgery procedure, the neuronavigation system is subject to inaccuracy because of anatomical deformations which induce a gap between the preoperative images and their anatomical reality. Thus, the objective of many research teams is to succeed in quantifying these deformations in order to update preoperative images. Anatomical intraoperative deformations correspond to a complex spatio-temporal phenomenon. Our objective is to identify the parameters implicated in these deformations and to use these parameters as constrains for systems dedicated to updating preoperative images. In order to identify these parameters of deformation we followed the iterative methodology used for cognitive system conception: identification, conceptualization, formalization, implementation and validation. A state of the art about cortical deformations has been established in order to identify relevant parameters probably involved in the deformations. As a first step, 30 parameters have been identified and described following an ontological approach. They were formalized into a Unified Modeling Language (UML) class diagram. We implemented that model into a web-based application in order to fill a database. Two surgical cases have been studied at this moment. After having entered enough surgical cases for data mining purposes, we expect to identify the most relevant and influential parameters and to gain a better ability to understand the deformation phenomenon. This original approach is part of a global system aiming at quantifying and correcting anatomical deformations.

Abstract: Anatomical intra operative deformation is a major limitation of accuracy in image guided neurosurgery. Approaches to quantify these deforamations based on 3D reconstruction of surfaces have been introduced. For accurate quantification of surface deformation, a robust surface registration method is required. In this paper, we propose a new surface registration for video-based analysis of intraoperative brain deformations. This registration method includes three terms: the first term is related to image intensities, the second to Euclidean distance and the third to anatomical landmarks continuously tracked in 2D video. This new surface registration method can be used with any cortical surface textured point cloud computed by stereoscopic or laser range approaches. We have shown the global method, including textured point cloud reconstruction, had a precision within 2 millimeters, which is within the usual rigid registration error of the neuronavigation system before deformations.

Abstract: Les développements considérables de la neuro-imagerie moderne permettent une meilleure préparation des procédures de neurochirurgie guidée par l'image en zones éloquentes [9]. Ces informations, de nature multimodale, anatomiques et fonctionnelles, sont également accessibles durant la procédure chirurgicale grâce à l'utilisation de systèmes de neuronavigation permettant la mise en correspondance géométrique entre les images pré opératoires et le patient. La visualisation des images pré opératoires durant la procédure chirurgicale est un point crucial en neurochirurgie moderne. La plupart des systèmes de neuronavigation permettent d'afficher, dans les oculaires du microscope, les images pré opératoires du patient en superposition avec la vue du champ opératoire, définissant ainsi le concept de réalité augmentée [3]. Les images superposées sont les intersections des volumes d'intérêt et du plan focal du microscope. En outre, elles sont le plus souvent bidimensionnelles et monochromes alors que ces informations sont tridimensionnelles et multimodales [6]. Cette situation limite considérablement la reconnaissance et l'interprétation des images superposées. Dans ce papier, nous décrivons une méthode originale dite de virtualité augmentée. Le principe est de reconstruire la surface tridimensionnelle du champ opératoire à partir des vues stéréoscopiques obtenues par l'intermédiaire du microscope opératoire et de superposer cette surface à la scène virtuelle pré opératoire pendant l'intervention.

Abstract: This paper reports the performance evaluation of a method for visualisation and quantification of intraoperative cortical surface deformations. This method consists in the acquisition of 3D surface meshes of the operative field directly in the neuronavigatorrsquos coordinate system by means of stereoscopic reconstructions, using two cameras attached to the microscope oculars. The locations of about 300 surfaces are compared to the locations of two reference surfaces from a physical phantom: a segmented CT scan with image-to-physical fiducial-based registration, used to compute the overall system performance, and a cloud of points acquired with the neuronavigatorrsquos optical localiser, used to compute the intrinsic error of our method. The intrinsic accuracy of our method was shown to be within 1mm.

Abstract: Image guidance mainly consists of displaying preoperative images related to the neurosurgeon's view of the operative field. In most available commercial neuronavigation systems, limitations are limited point of view and difficulty to understand 3D complex scenes. Furthermore, after opening the arachnoid, cortical surface deformation is significant and the preoperative information no longer corresponds to the anatomical reality of the patient. We present a new approach referred to as augmented virtuality for displaying intraoperative views of the operative field over 3D multimodal preoperative images onto an external screen during surgery. 3D surfaces meshes of the operative field were then generated using stereopsis. An approach for correcting the intraoperative location of regions of interest near the surface, based on 3D surface meshes registration and on tracking in video image sequences is also presented. The approach consists in using surface meshes obtained by stereoscopic reconstructions from the microscope oculars. A visible light image is associated with this surface. Between two acquisitions of surfaces meshes, landmarks are tracked in the video. The cost function for surface matching is then composed by a dissimilarity metric based on both Euclidian distance and intensity correlation and an additional term which is the deformation field representing the respect of landmark matching. Good performance of our methods was assessed by reference comparison, using phantoms in clinical settings and on some real clinical cases.