Abstracts

 

Abstracts of papers in refereed journals or conferences since 1992

1. Generation of mental images from verbal descriptions

2. Cognitive processes and linguistic strategies in the production of spatial descriptions

3. Cognitive processes involved in memorizing and using spatial descriptions

4. Brain structures involved in the generation of visuo-spatial representations

 

 

1. Generation of mental images from verbal descriptions

Previous research has found a linear relation between distance and scanning times for spatial configurations when the spatial configurations were learned perceptually and when they were constructed from well-structured verbal descriptions. The current research replicated the time-distance relation when the images were constructed from repetitions of well-structured descriptions but not when the images were generated from three repetitions of a description that presented information in a random order. Six exposures to the randomly ordered information yielded the expected time-distance relation in image scanning. We posited that additional exposure to the poorly structured information allowed the image to develop the structural coherence and resolution needed to support consistent scanning. Thus, the structure of descriptions can affect the intrinsic structure of images of described objects and hence the mental operations performed subsequently on these images. Another experiment indicated that image coherence and resolution improves even after the verbal description is accurately recalled.

  • Denis, M., & Zimmer, H. D. (1992). Analog properties of cognitive maps constructed from verbal descriptions. Psychological Research, 54, 286-298.

Six experiments were conducted to test whether the spatial properties of cognitive maps constructed from verbal descriptions are similar to those of representations derived from visual experience. Data from spatial priming, distance comparison, and mental scanning experiments all suggest that people are able to convert verbal descriptions of configurations into mental representations that reflect the spatial extension of these configurations. In addition, cognitive maps constructed from descriptions exhibit reliable metric properties which make them structurally isomorphic to the corresponding spatial configurations.

  • Denis, M., Gonçalves, M.-R., & Memmi, D. (1995). Mental scanning of visual images generated from verbal descriptions: Towards a model of image accuracy. Neuropsychologia, 33, 1511-1530.

Recent mental scanning experiments have shown that subjects are not only able to construct mental images from verbal descriptions, but that these images have structural properties similar to those of images derived from perception. In addition, the specific sequencing of a description can affect the internal structure of images of described objects, in particular their metric properties. Discontinuous descriptions require additional exposure to achieve the structural coherence of images constructed from continuous descriptions. Thus, the capacity of images to reflect accurately the objects they refer to is not an all-or-nothing property, but rather results from stepwise elaboration. This study describes a quantitative model designed to account for the gradual process of image elaboration and the progressive increase in image accuracy. The model posits that the location of a landmark mentioned in a description is not represented as a sharp point in the mental image, but is instead associated with a region around this point. Learning the description essentially consists of progressively narrowing each « region of uncertainty » associated with a landmark to its exact location. Additional experimental data were collected to provide a more fine-grained understanding of image elaboration. Computations of the regions of uncertainty associated with each landmark were used to develop a computer program simulating the whole mental scanning protocol, which provided support for our account of image accuracy.

Subjects were presented with nouns and required to form visual mental images in response to them. The generation latencies and durations of images, as indicated by subjects by pressing a key, were recorded. The results confirmed that the higher the noun imagery value, the shorter the generation latencies, but that imagery value does not affect image duration. This pattern was unchanged in two sessions held three weeks apart. Depressed subjects were also tested with the same task. These subjects failed to form many images, especially in response to low imagery nouns, but this effect was significantly reduced by antidepressant treatment. Generation latencies were significantly longer for depressed subjects than for controls. They were not affected by antidepressant treatment. The image durations in control and depressed subjects were of similar magnitudes. These findings are interpreted as indicating that functionally distinct processes are required for generating and maintaining visual images.

When subjects mentally scan across visual images of spatial configurations, the « mental scanning effect » is said to occur when there is a linear relationship between distances scanned and scanning times. This effect has been documented in studies where configurations were learned perceptually, and also when mental images of spatial configurations were constructed from verbal descriptions. The scanning effect is generally taken to indicate that visual images incorporate the metric structure of represented objects or configurations in an analog fashion. This article reports three experiments designed to test whether the cognitive salience of landmarks in a configuration can alter the mental scanning effect. Three manipulations of landmark salience were used, but there was no evidence of such an alteration. The scanning times towards salient and non-salient landmarks were quite similar, and the experimental manipulations had no effect on the time-distance correlation coefficients. We conclude that the structural organization of visual images constructed from verbal descriptions is robust, since semantic variations in the descriptions did not affect the mental scanning effect. The experiments showed that high visuo-spatial imagers (as classified on the basis of their scores on the Minnesota Paper Form Board) consistently had shorter scanning times than low imagers, and that only the responses of high imagers gave the time-distance correlation coefficients characteristic of the mental scanning effect. These findings suggest that the visual images constructed by high imagers include more accurate metric information than those constructed by low imagers.

The process of mental scanning corresponds to the systematic shifting of attention over visualized objects. The first part of this article focuses on the role of mental scanning as an empirical method to assess the structural properties of the representations that underlie visual mental imagery. One theory of imagery posits that the metric properties of the surfaces of objects are made explicit in visual images. If so, then the time to scan across imaged objects should increase linearly with the distance scanned, and such results have been reported in a number of experiments. However, these results proved controversial. Various alternative accounts were proposed, and new studies conducted. The present review shows that the alternative accounts were not compelling, and that results from image scanning studies are best interpreted as reflecting the metric properties of imagery representations and also showing that imagery uses mechanisms that are used to encode and interpret objects during perception. The second part of this article focuses on the use of image scanning to examine whether verbal descriptions may be used to construct images with structural properties similar to those of images of previously seen objects. Newly constructed images have been shown to possess a structure very much like the structure of representations that arise from visual perception. Moreover, the scanning technique has been used to study which properties of descriptions allow them to be converted easily into images. Finally, the article considers the use of the mental scanning paradigm in neuroimaging studies. By examining the neural foundations of mental scanning, researchers are in a good position not only to learn more about imagery, but also to discover more about the roles of particular brain mechanisms.

Mental scanning was used to assess the metric properties of mental spatial representations derived from visual experience or the processing of a verbal description, and either from survey or route perspective. Participants were asked to mentally scan their images of a spatial environment they had learned in one of the following four conditions: Visual-Survey, Visual-Route, Verbal-Survey, and Verbal-Route. No difference was found between the scanning times of the visual and verbal conditions, but scanning times were shorter after survey than route acquisition, and they consistently increased as a function of the Euclidean distances between locations in the environment, with steeper slopes in the route conditions. These results demonstrate that mental spatial representations derived from different sources and perspectives are endowed with similar properties and preserve the Euclidean characteristics of the original environment, but that they are easier to access when they have been constructed from a survey perspective.

In this study, we investigated the metric properties of spatial representations built from the verbal description of a spatial layout in early blind, late blind, and transiently visually deprived sighted participants. We adapted the verbal descriptions designed by Denis and Zimmer (1992). Participants had to mentally compare distances separating pairs of landmarks. The analysis of the frequency of correct responses suggests that visual experience does not play a crucial role in the preservation of the topology of a memorized spatial configuration. However, response times differed significantly among groups, with participants who experienced transient visual deprivation being overall faster than those suffering permanent loss of vision. Lastly, for all groups, the smaller the difference between two pairs of distances, the longer the response time, which attests to the presence of a symbolic distance effect. To conclude, if mental representations can be considered as reflecting described spatial layouts analogically, our data do not provide any strong evidence in favor of the visual character of these analog representations.

This study presents the combined efforts of three research groups toward the investigation of a cognitive issue through the development and implementation of a general purpose VR environment that incorporates a high quality virtual 3D audio interface. The psychological aspects of the study concern mechanisms involved in spatial cognition, in particular to determine how a verbal description of an environment or the active exploration of that environment affects the building of a mental spatial representation. Another point is to investigate the role of vision by observing whether or not participants without vision (blind from birth, late blind or blindfolded sighted individuals) can benefit from these two learning modalities. This paper presents the preliminary results of this study. Additionally is a description of the generic toolkit and companion architecture that has been developed and used for modeling the environment and interface in a cohesive manner. Details for generating an immersive multimodal experimental environment for this platform are also included.

Mental image scanning is generally assumed to be a single process that allows people to shift attention across visualized objects. However, this implicit assumption is open to question. We report a set of three experiments based on the tasks originally designed by Kosslyn, Ball, and Reiser (1978) and by Finke and Pinker (1982). Participants scanned the identical images of an array of dots in the two tasks. Nevertheless, participants required more time to shift their focus over the imaged stimulus in the Kosslyn et al. (1978) paradigm. Moreover, correlational analyses revealed no consistent relationship between the slopes of the increases in scanning times with increasing distances in the two paradigms. We conclude that in the Kosslyn et al. (1978) paradigm, the participants draw primarily on transformational processes to scan, whereas in the Finke and Pinker (1982) paradigm, they draw primarily on attentional processes. Both processes, transforming the image and shifting an attention window, produce linear increases in time with increases in distance, but for different reasons.

Mental images constructed after visual examination of a spatial configuration or after processing a verbal description of that configuration have been shown to share similar properties, in particular the capacity to preserve metric information contained in the configuration represented. In the present study, we investigated the properties of mental images constructed under learning conditions resulting from the combination of a visual or a verbal mode of acquisition and a survey or route perspective. Participants memorized a virtual environment (a garden containing six objects) under one of four learning conditions: (a) viewing a map of the garden (visual–survey); (b) viewing a video presentation of a journey along the path around the garden (visual–route); (c) listening to a verbal description of the map of the garden (verbal–survey); and (d) listening to a verbal description of the journey around the garden (verbal–route). The participants were then invited to compare the distances separating objects in the garden mentally. Experiment 1, where the pairs of distances to be compared had a common starting point, revealed that the frequency of correct responses was higher, and response times were shorter when participants had learned about the environment visually rather than by a verbal description. The conditions involving a survey perspective resulted in a higher frequency of correct responses and shorter response times than those involving a route perspective. Lastly, a symbolic distance effect was obtained in the first three conditions, in that the greater the difference between the two distances being compared, the higher the frequency of correct responses, and the shorter the response times. Experiment 2, where the pairs of distances had different starting points, replicated these results, although longer response times revealed that the comparison process was more costly. Taken together, these findings support the view that mental spatial representations derived from different sources and adopting different perspectives contain genuine metric properties, except when the verbal modality and the route perspective are combined during learning.

In two experiments, undergraduates processed a verbal description of a spatial configuration on the periphery of which six landmarks were located. The participants were then invited to generate visual images of the configuration, and to visualize the distances between pairs of landmarks. Their task consisted of deciding which of the two specified distances was longer. The results showed that as the magnitude of the differences in distance increased, the frequency of correct responses was higher, and response times were shorter. This pattern of results is characteristic of the symbolic distance effect, which is especially interesting in the present experiment where the images generated by the participants were constructed after processing a verbal description (rather than reconstructed from previous perceptual experience). In order to assess the role of visual imagery in the comparison of distances, the performance of participants with the highest scores on a visuo-spatial test (the Minnesota Paper Form Board) was compared to that of those with the lowest scores. High visuo-spatial imagers had higher frequencies of correct responses and shorter response times than the low imagers in the distance-comparison task. They outperformed their counterparts even more clearly on the items where the distance differences were the smallest, suggesting that visual imagery is especially important for items requiring the most attentive examination of a visual image. These data are interpreted as reflecting the fact that visual imagery mediates the process of mentally comparing distances, even when learning has been essentially based on verbal input. These findings support the view that a representation constructed from a verbal description may incorporate metric information about distances, and they offer evidence suggesting that visual images constructed from descriptive texts have genuine analog properties.

When people scan mental images, their response times increase linearly with increases in the distance to be scanned, which is generally taken as reflecting that their internal representations incorporate the metric properties of the corresponding objects. In view of this finding, we investigated the structural properties of spatial mental images created from nonvisual sources in three groups (blindfolded sighted, late blind, and congenitally blind). In Experiment 1, blindfolded sighted and late blind participants created metrically accurate spatial representations of a small-scale spatial configuration under both verbal and haptic learning conditions. In Experiment 2, late and congenitally blind participants generated accurate spatial mental images after both verbal and locomotor learning of a full-scale navigable space (created by an immersive audio virtual reality system), whereas blinfolded sighted participants were selectively impaired in their ability to generate precise spatial representations from locomotor experience. These results attest that in the context of a permanent lack of sight, encoding spatial information on the basis of the most reliable currently functional system (the sensorimotor system) is crucial for building a metrically accurate representation of a spatial environment. The results also highlight the potential of spatialized audio-rendering technology for exploring spatial representations of visually impaired participants.

 

2. Cognitive processes and linguistic strategies in the production of spatial  descriptions

This paper provides a general framework for the analysis of descriptions of routes, in order to account for the way in which spatial cognition is externalized through discourse. Three cognitive operations are assumed to be involved in the generation of this form of spatial discourse: (a) activation of an internal representation of the environment in which navigation will take place; (b) the planning of a route in the subspace of the mental representation currently activated; and (c) the formulation of the procedure that the user should execute to reach the goal. Two major components of the descriptions of routes are considered, those by which speakers refer to landmarks and those which consist of prescribing actions. Descriptions of two routes in a natural environment were collected from 20 undergraduates. A detailed analysis of the protocols was used to establish a classification of items, in which five classes were defined: prescription of actions without referring to any landmark, prescription of actions with reference to a landmark, reference to landmarks without referring to any associated action, description of landmarks, commentaries. Individual protocols were then used to construct more abstract (skeletal) descriptions, reflecting the essentials of the navigational procedure. Skeletal descriptions confirmed that landmarks and their associated actions were key components of route descriptions. Additional analyses indicated that the similarity of individual descriptions to the skeletal description of a route predicted the rated quality of these descriptions. Female subjects provided descriptions containing more landmarks than did males. Lastly, the descriptions provided by high visuo-spatial imagers contained more frequent references to landmarks, which is consistent with the expectation that subjects who are more likely to retrieve visuo-spatial information from their memories should include more references to such information in their verbal productions.

The analysis of route directions provides a relevant context for the study of relationships between language and spatial cognition. This research examines the theoretical issue of the cooperation between differing cognitive representational systems possessing contrasting functional properties. In addition, this research domain offers a unique opportunity for psychology, linguistics, computer science, and human-machine communication to work together, thereby contributing to the understanding of cognitive processes and improving the design of navigational systems adjusted to the cognitive capacities of their users. This paper outlines the cognitive operations involved in producing route directions and identifies certain invariant features of route descriptions beyond the diversity of individual protocols. Data collected in natural environments were analyzed and used to construct « skeletal descriptions » that reflected the essentials needed for navigating along a route. The functional value of these descriptions was assessed by their ability to guide navigation in unfamiliar environments.

We report four studies in which we investigated the production of spatial discourse designed to help people move around in unfamiliar environments. In Study 1, descriptions of three routes were collected from residents of the city of Venice. Analysis of the descriptions revealed the variety of ways used to describe each route. Typical features of route directions were found, in particular the uneven distribution of the landmarks mentioned, which tended to concentrate at critical points where an orientation problem had to be solved. Study 2 used individual protocols to construct more abstract (« skeletal ») descriptions, reflecting the essentials needed for navigation. New subjects selected those units of information they judged necessary and sufficient to guide a person travelling along the routes. The contents of the skeletal descriptions were very similar, whether they were established by people familiar with Venice or complete strangers, suggesting that people can judge the relevance of information in route directions, regardless of their knowledge of the environment described. Study 3 showed that the ratings of the communicative value of the original individual protocols also resulted in very similar responses from familiar and unfamiliar judges. Finally, Study 4 assessed the value of individual descriptions for assisting navigation by testing the navigational performance elicited by these descriptions. Subjects unfamiliar with the city of Venice were given skeletal descriptions or descriptions which had been rated « good » or « poor » in the previous study. Navigation with good descriptions gave significantly lower error scores than navigation with poor descriptions, and skeletal descriptions gave scores similar to those of good descriptions. Poor descriptions also resulted in more errors from subjects who tended to use a survey perspective than from subjects expressing a preference for visual memories of landmarks. We suggest that the efficiency of route directions as navigational aids depends not only on their intrinsic characteristics, but also on the mode of processing adopted by the users.

The investigation reported here is part of a project to design navigational aids for the Paris subway system. Users of the subway were asked to describe the route to be followed from the platforms of subway stations to specific buildings in the city of Paris. The descriptions thus included two parts: one underground (from the train platform to the exit) and the other in the urban environment (from the exit to the building). Most studies of route directions have been conducted in open environments (campus, city, etc.). The aim here was to contrast the types of description specific to the two contexts. Forty-eight participants (24 male, 24 female) took part in the experiment. Verbal descriptions were analyzed using the procedure proposed by Denis (1997). The results showed that participants relied extensively on signs for the underground part of the route. This was true for subway stations displaying either newly designed or older signs. The paths were rarely referred to in the subway environment, but they were frequently used in the urban environment. The patterns of landmark distribution along the routes were similar in both environments, in that they were more frequent at the nodes where reorientation was needed.

Route directions describe the sequence of actions a moving person needs to take to reach a goal in an environment. When generating directions, speakers not only specify what to do. They also refer to landmarks located along the route. We report two studies intended to identify the cognitive functions of landmarks. In the first study, participants learned a route in an urban environment. They then generated route directions to help pedestrians unfamiliar with this environment to find their way. We found that landmarks were reported more frequently at specific points on the route, especially at reorientation points. The second study showed that pedestrians perceived landmarks as a useful part of route directions. We conclude that reference to landmarks is intended to help movers to construct a mental representation of an unfamiliar environment in advance and to prepare them cognitively to get through difficult or uncertain parts of that environment.

Route directions to reach a target point on a campus were collected from undergraduates. A « good » description and a « poor » one were selected, based on ratings provided by judges in terms of their value for navigational assistance. People unfamiliar with the campus were then required to navigate to the target after studying one of these descriptions. In addition, a « skeletal » description, which contained the essentials needed for navigating, was constructed and used in the experiment. During navigation, we measured the frequencies of stops and of directional errors (whether these errors were self-corrected or corrected by the experimenter). Overall, the good and the skeletal descriptions resulted in better performance than the poor one. Their value as navigational aids was confirmed by measuring the navigation times. Analyzing the structure and content of the descriptions confirmed that the effectiveness of route directions depends on their ability to connect actions to landmarks, that is, to closely link the prescriptive and the descriptive parts of this specific type of spatial discourse.

The study reported here investigated the effects of conditions expected to favour conciseness in the production of route directions. In Experiment 1, two groups of undergraduates were invited to give written descriptions of the route connecting two well-known places on their university campus. The control group received standard instructions, and the other group was instructed to produce route directions that were as concise as possible, while remaining sufficiently informative to guide a pedestrian to the goal. Not surprisingly, instructions to be concise resulted in an overall shortening of descriptions, but this occurred in a selective manner. In particular, actions and action-landmark combinations were the least affected, whereas the number of landmarks mentioned without being associated with actions, as well as of details describing landmarks, was considerably reduced. Furthermore, landmarks situated at points on the route involving a decision about changing direction underwent less reduction than other landmarks. In Experiment 2, conditions were created where conciseness was brought into play without any explicit instructions to be concise. Undergraduates were asked to work in groups of three, and each group was assigned the task of producing a single description. The results showed that the descriptions produced by a group were shorter than those produced by individuals. This was interpreted as reflecting that in the absence of any conciseness instructions, the feedback developed within groups during the production of route directions led to effective selection of the information content. Interestingly, the items relating actions and landmarks were almost fully preserved in the group descriptions. In Experiment 3, conciseness was shown to be further increased by combining instructions for conciseness and group production. Altogether, the three experiments revealed the primary role of propositions linking action prescriptions and landmarks at points on the route where key actions have to be taken.

  • Przytula-Machrouh, E., Ligozat, G., & Denis, M. (2004). Vers des ontologies transmodales pour la description d’itinéraires: Le concept de « scène élémentaire ». Revue Internationale de Géomatique, 14, 285-302.

The work reported here investigates the knowledge used by people generating route directions, the modes of representation of that knowledge, and their use. Our initial work on a corpus of data including verbal and graphic route directions showed the existence of common conceptual structures which we called « elementary scenes ». We propose a specific approach to the development of transmodal ontologies adapted to the representation and the use of route directions.

Three experiments investigated expert and non-expert knowledge of a familiar but loosely structured spatial environment as revealed through the production of sketch maps. In the first experiment, experts and non-experts in geomatics sketched maps of a well-known park. The analysis of the maps revealed that experts and non-experts used different drawing strategies that reflected different mental representations. In the second experiment, new participants identified good and poor examples from the previous maps. Expert and non-expert evaluators agreed, indicating that experts and non-experts alike agree on what constitutes a « good map ». In the third experiment, people familiar and unfamiliar with the park were asked to remove non-essential features from a consolidated map that incorporated all the features drawn by the participants of the first experiment. Those familiar and unfamiliar with the environment retained the same features, notably, the paths in the park. Together, the research shows that experts produce superior maps to non-experts, but that people, irrespective of expertise and familiarity, concur on the features that make a map effective. Even for relatively unstructured environments like a large park, people seek structure in the configuration of paths. These findings have implications for the design of maps.

Twenty outpatients who fulfilled the criteria for a diagnosis of schizophrenia and 28 control participants were invited to learn a route through a complex outdoor environment. They were then tested in tasks intended to explore various aspects of their memorized representation of the navigational episode. Compared to controls, the patients showed significant impairment in both the verbal production of route directions and the drawing of sketch maps. They referred to fewer landmarks and provided fewer directional instructions than the controls, while making a greater number of irrelevant comments. When invited to distinguish between photographs showing views of landmarks encountered along the route and distractors, they performed as well as the controls, and they had similar response times. However, when they were presented with pairs of actual photographs taken along the route, they displayed special difficulty in deciding which of the two landmarks was encountered first along the route. This difficulty in retrieving the sequential structure of the navigational episode suggests that the patients’ memories were not accurately linked to one another in their mental representation of the route. These findings are interpreted in the context of current hypotheses about the hippocampal impairment that affects schizophrenic patients.

 

3. Cognitive processes involved in memorizing and using spatial descriptions

  • Denis, M., & Briffault, X. (2000). Analyse des dialogues de navigation à bord d’un véhicule automobile. Le Travail Humain, 63, 59-88.

The study reported in this article was designed to contribute to the development of on-board navigational aid systems. In order to improve the functional value of such systems, it is crucial to have information on how people actually exchange information in navigational tasks. With this purpose in mind, we collected a corpus of navigational dialogues in realistic situations involving a driver and a co-driver in a car in an urban environment. Twenty-seven couples (driver plus co-driver) were assigned to one of four conditions resulting from the combination of two factors. The first factor was whether or not the co-drivers had any prior knowledge of the route to be followed. The second factor was related to the mode of expression used by co-drivers (in particular, whether they had no constraints or were required to communicate only verbally without gestures). The analysis of the data provided evidence that co-drivers’ prior knowledge influenced the content of dialogues, in particular, the selection of landmarks and the expression of orienting instructions. The dialogues involving co-drivers without any prior knowledge contained more sentences, and each sentence was longer than those in dialogues involving co-drivers with prior knowledge. Constraints on the mode of expression used by co-drivers also moderately affected the dialogues. A detailed classification of the drivers’ and co-drivers’ utterances was established. Drivers’ utterances were classified into six categories: confirmations; queries; requests for repetitions or precisions; requests for confirmation; reformulations; information to the co-driver. Co-drivers’ utterances included six types: marking; descriptions of actions; descriptions of scenes; advance descriptions; confirmations; requests for information. The study also analyzed the descriptive strategies used by co-drivers when approaching and traversing complex spaces, such as intersections. Three distinct patterns of dialogues were described, depending on whether they occurred at a distant point (before the section in which the intersection would be found), in a nearby area (in which the intersection was visible) or in an immediate area (where the vehicle had reached the intersection). The results of this study are discussed for their implications on the specification of navigational aid systems.

  • Chalmé, S., Denis, M., Briffault, X., Gaunet, F., & Nathan, F. (2000). Aides verbales à la navigation automobile: L’impact des instructions directionnelles sur le comportement d’un pilote à l’approche de carrefours. Le Travail Humain, 63, 353-376.

Recent innovations in microcomputer and display technology have resulted in sophisticated route guidance systems that help drivers in selecting and maintaining routes. However, the efficacy of these systems still depends on the drivers’ cognitive characteristics. These characteristics imply constraints on what kind of information drivers need and how such information can be best displayed. In general, it is recognised that voice guidance makes the driving task easier and safer. In particular, drivers need guiding information before every required turn. The most obvious criterion of effectiveness of guiding information is the time left available before the maneuver. Using the driving simulator SHERPA, we tested the influence of time to maneuver on automobile drivers’ behaviour at the approach of two reorientation intersections. We designed four conditions of verbal announcements involving various degrees of anticipation with respect to the locations where the directional changes should occur. We intended to identify the announcements allowing for the most appropriate anticipatory behaviour with the lowest risk of accident. Behavioural measures included velocity, pressure on the brake and the clutch, acceleration of the steering wheel, and position of hands on the steering wheel. The results showed that two of the four conditions did not allow drivers to anticipate their actions in the intersection and resulted in dangerous behaviour. The other two, which approximated information deliverance by road signs, allowed drivers to anticipate more effectively. However, none of the latter announcements taken separately were really optimal. In one case, the drivers tended to forget information, while in the other, the announcements failed to provide confirmation of previously memorised instructions. A combination of types of announcements should thus be recommended. Based on the results of this investigation, we propose specifications that designers might use to develop an effective and safe guiding system.

When describing routes in urban environments, speakers usually refer to both street names and visual landmarks. However, a navigational system can be designed which only refers to streets or, alternatively, only to landmarks. Does it make any difference which type of information users are provided with? The answer to this question is crucial for the design of navigational aids. We report two experiments. The first one showed that in a wayfinding task, route directions referring to streets were less effective than those referring to landmarks for guidance purposes. The second experiment showed that when people generate route directions, they tend to produce less street than landmark information. These studies provide a further illustration of the critical role of landmarks in route directions.

Two experiments were conducted to investigate the processes involved when people use spatial descriptions intended to assist navigation. More specifically, we compared the effectiveness of route directions in an urban environment based on references either to landmarks or to street names. In the first experiment, the participants learned route directions that referred either to landmarks or to streets named after landmarks (e.g., a hospital vs. « Hospital Street »). Processing times were shorter for instructions based on landmarks than for those based on street names. When the participants subsequently drew the route described, their memory was better when they had processed landmark rather than street information. The same route directions were used in the second experiment, in which the participants’ memories were tested in a recognition task. The results showed that when target words referred to landmarks, the participants were more accurate and took less time to respond than when the same words were used to refer to streets. This finding indicates that the results of the previous experiment cannot be attributed to differing costs of the retrieval processes. Overall, the results of these experiments confirm the special cognitive status of landmarks in the mental representation of routes.

The authors conducted two experiments to study the metrics of spatial distance in a mental imagery task. In both experiments, participants first memorized the layout of a building containing 10 rooms with 24 objects. Participants then received mental imagery instructions and imagined how they walked through the building from one room to another. The authors manipulated Euclidean distance involved in these imaginary motions: Spatial distance measured in centimeters on the layout was either short or long. Independently, they varied categorical distance: The motions led through one room or two rooms. The time needed to imagine motions and response times to test probes indicated that both Euclidean distance and categorical distance affected mental imagery. The authors discuss the new finding of categorical distance effects in mental imagery and relate the results to earlier failures to find Euclidean distance effects in formally equivalent studies of narrative comprehension.

The dual-task paradigm was used to determine whether the spatial, visual and verbal components of working memory are engaged in the processing of spatial descriptions. Participants listened to route or survey descriptions of urban-like spatial environments, and then drew corresponding maps. The position of each new landmark was described either in terms of the direction to move toward this landmark (route descriptions) or its relative location with regard to the previously mentioned landmark (survey descriptions). Route and survey descriptions resulted in similar recall performance in the absence of an interfering task, and landmarks were consistently less well recalled than their associated moves/locations. The pattern of interference resulting from the secondary tasks indicated that the processing of landmarks called upon both the visual and spatial components of working memory in the route perspective, whereas the processing of moves/locations essentially relied on the spatial component in both the route and the survey perspectives. The verbal component of working memory was only involved in the processing of landmarks in the survey perspective. The results suggest that distinct cognitive processes support memory for route and survey descriptions, and that distinct working memory resources support the processing of landmarks and landmark positions.

Navigation within a closed environment requires analysis of a variety of acoustic cues, a task that is well developed in many visually impaired individuals, and for which sighted individuals rely almost entirely on visual information. Focusing on the needs of the blind, the creation of cognitive maps for spaces such as home or office buildings can be a long process, for which the individual may repeat various paths numerous times. While this action is typically performed by the individual on-site, it is of some interest to investigate to what point this task can be performed off-site, at the individual’s discretion. In short, is it possible for an individual to learn an architectural environment without being physically present? If so, such a system could prove beneficial for preparing for navigation in new and unknown environments. A comparison of three learning scenarios has been performed: in-situ real displacement, passive playback of recorded navigation (binaural and Ambisonic), and active navigation in virtual auditory environment architecture. For all conditions, only acoustic cues are employed. This research is the result of collaboration between researchers in psychology and acoustics on the issue of interior spatial cognition.

Navigating complex routes and finding objects of interest are challenging tasks for the visually impaired. The project NAVIG (Navigation Assisted by artificial VIsion and GNSS) is directed toward increasing personal autonomy via a virtual augmented reality system. The system integrates an adapted geographic information system with different classes of objects useful for improving route selection and guidance. The database also includes models of important geolocated objects that may be detected by real-time embedded vision algorithms. Object localization (relative to the user) may serve both global positioning and sensorimotor actions such as heading, grasping, or piloting. The user is guided to his/her desired destination through spatialized semantic audio rendering, always maintained in the head-centered reference frame. This paper presents the overall project design and architecture of the NAVIG system. In addition, details of a new type of detection and localization device are presented. This approach combines a bio-inspired vision system that can recognize and locate objects very quickly and a 3D sound rendering system that is able to perceptually position a sound at the location of the recognized object. This system was developed in relation to guidance directives developed through participative design with potential users and educators for the visually impaired.

Four experiments investigated whether directional spatial relations encoded by reading narratives are updated following described protagonist rotations. Participants memorized locations of objects described in short stories that placed them, as the protagonist, in remote settings. After reading a description that the protagonist rotated to the left or the right of the initial orientation, participants made judgements about object relations in the described environment (Experiment 1). Before making these judgements, participants were instructed to physically rotate to match (Experiment 2) or mismatch (Experiment 4) the protagonist’s described rotation and in Experiments 3 and 4 to also visualize the changed relations following rotation. Participants’ performance suggested that they relied on the initial representation they constructed during encoding rather than on the updated protagonist-to-object relations. Participants’ physical movement to match the described rotation and additional visualization instructions did not facilitate updating through a sensorimotor process. In these respects, updating spatial relations in situation models constructed from narratives differs from updating in perceptually experienced environments.

Navigation within a closed environment requires analysis of a variety of acoustic cues, a task that is well developed in many visually impaired individuals, and for which sighted individuals rely almost entirely on visual information. For blind people, the act of creating cognitive maps for spaces, such as home or office buildings, can be a long process, for which the individual may repeat various paths numerous times. While this action is typically performed by the individual on-site, it is of some interest to investigate at which point this task can be performed off-site, at the individual’s discretion. In short, is it possible for an individual to learn an architectural environment without being physically present? If so, such a system could prove beneficial for navigation preparation in new and unknown environments. The main goal of the present research can therefore be summarized as investigating the possibilities of assisting blind individuals in learning a spatial environment configuration through the listening of audio events and their interactions with these events within a virtual reality experience. A comparison of two types of learning through auditory exploration has been performed: in situ real displacement and active navigation in a virtual architecture. The virtual navigation rendered only acoustic information. Results for two groups of five participants showed that interactive exploration of virtual acoustic room simulations can provide sufficient information for the construction of coherent spatial mental maps, although some variations were found between the two environments tested in the experiments. Furthermore, the mental representation of the virtually navigated environments preserved topological and metric properties, as was found through actual navigation.

We tested the hypothesis that a route’s memorability is dependent on the frequency with which people are exposed to visual landmarks. Undergraduates learned either a route through an urban area lacking visually salient features, or a route in a neighborhood with many shops and urban objects. They were then asked to recall the learned route in the form of route directions and sketch maps. The results showed higher recall performance for the richer environment. When presented with photographs depicting scenes along the route, participants exposed to the richer environment had higher recognition scores and shorter response times than the others. The data confirm the functional role of landmarks in route memory and wayfinding.

 

4. Brain structures involved in the generation of visuo-spatial representations

Using regional cerebral blood flow (rCBF) imaging, two populations having high and low imagery abilities were compared at rest and while performing two cognitive tasks: silent verb conjugation and mental imagery. The imagery task produced an rCBF increase in the left visual association and left frontal cortices in both groups. Differences between high and low imagers were observed on global and regional flow responses to cognitive tasks: low imagers showed a whole cortex CBF increase during both tasks; high imagers showed a right dominance in the visual association cortex in all conditions, and in the parietal association cortex at rest.

We measured normalized regional cerebral blood flow (NrCBF) using positron emission tomography (PET) and oxygen-15-labeled water in eight young right-handed healthy volunteers selected as high-imagers, during 2 runs of 3 different conditions: 1, rest in total darkness; 2, visual exploration of a map; 3, mental exploration of the same map in total darkness. NrCBF images were aligned with individual magnetic resonance images (MRI), and NrCBF variations between pairs of measurements (N = 15) were computed in regions of interest having anatomical boundaries that were defined using a three-dimensional (3-D) reconstruction of each subject’s MRI. During visual exploration, we found bilateral activations of primary visual areas, superior and inferior occipital gyri, fusiform and lingual gyri, cuneus and precuneus, bilateral superior parietal, and angular gyri. The right lateral premotor area was also activated during this task while superior temporal gyri and Broca’s area were deactivated. By contrast, mental exploration activated the right superior occipital cortex, the supplementary motor area, and the cerebellar vermis. No activation was observed in the primary visual area. These results argue for a specific participation of the superior occipital cortex in the generation and maintenance of visual mental images.

Positron emission tomography (PET) was used to monitor regional cerebral blood flow variations while subjects were constructing mental images of objects made of three-dimensional cube assemblies from auditorily presented instructions. This spatial mental imagery task was contrasted with both passive listening (LIST) of phonetically matched nonspatial word lists and a silent rest (REST) condition. All three tasks were performed in total darkness. Mental construction (CONS) specifically activated a bilateral occipitoparietal-frontal network, including the superior occipital cortex, the inferior parietal cortex, and the premotor cortex. The right inferior temporal cortex also was activated specifically during this condition, and no activation of the primary visual areas was observed. Bilateral superior and middle temporal cortex activations were common to CONS and LIST tasks when both were compared with the REST condition. These results provide evidence that the so-called dorsal route known to process visuospatial features can be recruited by auditory verbal stimuli. They also confirm previous reports indicating that some mental imagery tasks may not involve any significant participation of early visual areas.

Positron emission tomography was used to investigate the functional anatomy of mental simulation of routes (MSR) in five normal volunteers. Normalized regional cerebral blood flow was measured while subjects mentally navigated between landmarks of a route which had been previously learned by actual navigation. This task was contrasted with both static visual imagery of landmarks (VIL) and silent rest. MSR appears to be subserved by two distinct networks: a non-specific memory network including the posterior and middle parts of the hippocampal regions, the dorsolateral prefrontal cortex and the posterior cingulum, and a specific mental navigation network, comprising the left precuneus, insula and medial part of the hippocampal regions.

The functional anatomy of the interactions between spoken language and visual mental imagery was investigated with PET in eight normal volunteers during a series of three conditions: listening to concrete word definitions and generating their mental images (CONC), listening to abstract word definitions (ABST) and silent REST. The CONC task specifically elicited activations of the bilateral inferior temporal gyri, of the left premotor and left prefrontal regions, while activations in the bilateral superior temporal gyri were smaller than during the ABST task, during which an additional activation of the anterior part of the right middle temporal gyrus was observed. No activation of the occipital areas was observed during the CONC task when compared either to the REST or to the ABST task. The present study demonstrates that a network including part of the bilateral ventral stream and the frontal working memory areas is recruited when mental imagery of concrete words is performed on the basis of continuous spoken language.

This study had two purposes. First, in order to address the controversy regarding activation of the primary visual area (PVA) during visual mental imagery, regional cerebral blood flow (rCBF) was recorded while subjects performed a task that required high-resolution visual mental imagery. Second, in order to discover whether verbal descriptions can engage visual mechanisms during imagery in the same way as visual stimuli, subjects memorized 3D scenes that were visually presented or were based on a verbal description. Comparison of the results from the imagery conditions to a non-imagery baseline condition revealed no activation in PVA for imagery based on a verbal description and a significant decrease of rCBF in this region for imagery based on visual learning. The pattern of activation in other regions was very similar in the two conditions, including parietal, midbrain, cerebellar, prefrontal, left insular, and right inferior temporal regions. These results provide strong evidence that imagery based on verbal descriptions can recruit regions known to be engaged in high-order visual processing.

There are two major sources of information to build a topographic representation of an environment, namely actual navigation within the environment (route perspective) and map learning (survey perspective). The aim of the present work was to use positron emission tomography (PET) to compare the neural substrate of the topographic representation built from these two modes. One group of subjects performed a mental exploration task in an environment learned from actual navigation (mental navigation task). Another group of subjects performed exploration in the same environment learned from a map (mental map task). A right hippocampal activation common to both mental navigation and mental map tasks was evidenced and may correspond to the neural substrate of a « dual-perspective » representation. The parahippocampal gyrus was additionally activated bilaterally during mental navigation only. These results suggest that the right hippocampus involvement would be sufficient when the representation incorporates essentially survey information while the bilateral parahippocampal gyrus would be involved when the environment incorporates route information and includes « object » landmarks. The activation of a parietofrontal network composed of the intraparietal sulcus, the superior frontal sulcus, the middle frontal gyrus, and the pre-SMA was observed in common for both mental navigation and mental map and is likely to reflect the spatial mental imagery components of the tasks.

In the majority of investigations of representational neglect, patients are asked to report information derived from long-term visual knowledge. In contrast, studies of perceptual neglect involve reporting the contents of relatively novel scenes in the immediate environment. The present study aimed to establish how representational neglect might affect (a) immediate recall of recently perceived, novel visual layouts, and (b) immediate recall of novel layouts presented only as auditory verbal descriptions. These conditions were contrasted with reports from visual perception and a test of immediate recall of verbal material. Data were obtained from 11 neglect patients (9 with representational neglect), 6 right-hemisphere lesion control patients with no evidence of neglect, and 15 healthy controls. In the perception, memory following perception, and memory following layout description conditions, the neglect patients showed poorer report of items depicted or described on the left than on the right of each layout. The lateralised error pattern was not evident in the non-neglect patients or healthy controls, and there was no difference among the three groups on immediate verbal memory. One patient showed pure representational neglect, with ceiling performance in the perception condition, but with lateralised errors for memory following perception or following verbal description. Overall, the results indicate that representational neglect does not depend on the presence of perceptual neglect, that visual perception and visual mental representations are less closely linked than has been thought hitherto, and that visuospatial mental representations have similar functional characteristics whether they are derived from visual perception or from auditory linguistic descriptive inputs.

Humans have the ability to build and to inspect an internal visual image of an environment built from a verbal description. We used positron emission tomography (PET) to investigate the brain areas engaged in the mental scanning of a map that subjects built from reading of a descriptive text. This task engaged a parieto-frontal network known to deal with spatial representations. Additional activations were evidenced in the angular gyrus and in Broca’s and Wernicke’s areas. In order to examine the neural impact of the learning modality, these PET results were compared to those obtained in another group of six subjects who performed a similar mental scanning task on a topographic representation built from visual inspection of a map. Both scanning tasks engaged the parieto-frontal network. However, the bilateral activation of the angular gyrus as well as the involvement language areas appeared specific to the mental scanning of the topographic representation built from textual information. On the opposite, the right medial temporal lobe was activated only when a map had been visually learned. These results suggest that although both tasks involved visuo-spatial internal representations, a trace of the learning modality remained present in the brain.

Patients with representational unilateral neglect were impaired in immediate recall of novel material on the neglected side as presented (viewed and removed or verbally described) or following mental rotation. Transforming material from the neglected (left) side to the non-neglected (right) side resulted in no additional loss; patients were unimpaired in directing attention to the neglected side of their representation in order to perform the mental rotation. These findings cannot be explained by the widely adopted attention deficit hypothesis for representational neglect.  It is suggested that the disorder arises from damage to temporary storage functions of visuo-spatial working memory.

In this study, we used event-related functional magnetic resonance imaging to investigate whether visual mental images retinotopically activate early visual cortex. Six participants were instructed to visualize or view horizontally or vertically oriented flashing bow-tie shaped stimuli. When compared to baseline, imagery globally activated Area V1. When the activation evoked by the stimuli at the different orientations was directly compared, distinct spatial activation patterns were obtained for each orientation in most participants. Not only was the topography of the activation patterns from imagery similar to the topography obtained with a corresponding visual perception task, but it closely matched the individual cortical representation of either the horizontal or the vertical visual field meridians. These findings strongly support that visual imagery and perception share low-level anatomical substrate and functional processes. Binding of spatial features is suggested as one possible mechanism.

A study is reported of visuo-spatial working memory in two individuals suffering from a cognitive deficit known as unilateral spatial neglect, and seven healthy control participants. Both patients have difficulties reporting details on the left side of imaged representations, and one has an additional difficulty with perceptual input to the left of his body mid line. All participants were asked to report the location and identity of objects presented in novel 2 x 2 arrays that were either present throughout or were described orally by the experimenter, with no visual input. On half of the trials, the report was to be made from the opposite perspective, requiring 180 degree mental rotation of the mentally represented array. The patients showed an impaired ability to report details from the presented or the imagined left, but had no difficulty with mental rotation. Results point to a clear separation between the processes of perception and those of visuo-spatial working memory. Results also suggest that the patients might be suffering from damage to the system used for holding visuo-spatial representations rather than a difficulty with attending to elements of that representation.