When researchers in artificial intelligence started trying to teach computers how to identify visual objects in natural scenes, they soon realized that this is a very complex and difficult problem indeed and yet we do it all the time by means of our visual system, and effortlessly during most of the time. An important (first) step of object recognition in most natural environments is the segregation of objects from their surround, the so-called figure-ground segregation. Examples such as finding the Dalmatian dog on a spotted background and more natural examples of camouflage in the animal kingdom demonstrate that even the abilities of our visual system to discriminate figure and ground are limited. Typical factors allowing to discriminate between figure and ground include differences in luminance, colour, motion speed and/or motion direction, stereoscopic depth, and texture (such as different orientations of line elements or different size or spacing of texture elements). Examples for figure-ground segregation are shown in figure 1.
Figure 1: Examples for figure-ground segregation based on differences in a) Luminance; b) Colour; c) Orientation of line elements; d) Presentation time (here, the points within the rectangle were presented with a time delay relative to the points of the surround - the lines outlining the rectangle were not shown).
While Gestalt psychology contributed important insights into the phenomenology of figure-ground discrimination, the underlying neuronal mechanisms are far from clear, partly since this topic did not receive sufficient research efforts during the last decades, and clinical tests for figure-ground discrimination so far exist only for colour- and depth stimuli (the Ishihara and Lang tests).
Project aims: We want to better understand the neuronal mechanisms underlying figure-ground discrimination in the human visual system. More specifically, we investigate a) the way temporal parameters can lead to figure-ground discrimination, b) how different parameters such as colour, luminance, and orientation differences between figure and ground are combined, and c) which parts of the visual system are crucial for figure-ground discrimination. The investigations combine experiments in normals using functional Magnetic Resonance Imaging (fMRI), multichanel EEG recordings, and MEG-recordings with studies in patients suffering from circumscribed lesions of the brain.
Temporal parameters and visual perception: Regarding the temporal parameters of visual perception in general and more specifically of figure-ground segregation, we found that a figure consisting, for example, of line elements or dots as in Fig. 1d can be discriminated from its background if the elements within the figure are presented slightly before or after the ones in the background. This is to say that in the original stimulus, the thin lines delineating the figure in 1d were not present; a photograph of the stimulus would not allow to discriminate between figure and surround. Surprisingly, the time differences required to discriminate between figure and ground were below one hundredth of a second in trained observers, i.e. the elements of the figure were presented 0.01 sec before or after the ones of the surround. This time difference is clearly below the flicker fusion frequency of around 60 Hz as we all know, an image presented 60 times per second, as on a computer monitor, appears as constant. Therefore, one can denote the time differences resolved in temporal figure-ground discrimination as temporal hyperacuity (Fahle, 1993). This result, in line with recent results of single cell recordings in cat and monkey (Singer, 1993) show that temporal factors play an important and hitherto somewhat neglected role in the organisation of (visual) perception.
Combination of parameters for figure-ground segregation and synergy: While a large number of studies show that different aspects or components of visual stimuli are processed, in the visual system, partly independent from each other, the goal of visual perception, of course, is to attain a single, unified impression of the visual world- and after all, this is what our introspection tells us as being achieved. So it is to be expected that in the real world, as opposed to laboratory experiments, cues from different aspects or components of visual perception have to be combined in order to allow a fast, and reliable differentiation of an object from its surroundings and eventually the identification of the object. Another reason to investigate the process of combination of different cues for figure-ground segmentation is that the type of summation between different cues or components allows important conclusions about the underlying neuronal mechanisms. If figure-ground discrimination is performed separately in the cortical areas known to be specialized for colour- and motion perception, and is subsequently combined, we should expect probability summation of the cues at the most. If, however, figure-ground segmentation is achieved by a combined stage, such as by cells sensitive to both colour and line orientation, then we might find summation of the cues surpassing probability summation and that is actually what we find in psychophysical experiments. Some cues, such as colour and line orientation, show a kind of synergy effect: presenting both cues simultaneously improves figure-ground segmentation even more than to be expected on the basis of probability summation (Abele & Fahle, 1995; Gordon & Fahle, 1996). Hence, there seems to exist a processing stage in the visual cortex that is sensitive for both of these features and able to extract figure-ground boundaries.
Localisation of brain areas responsible for figure ground segregation: We tried to find the location of this processing stage in the human visual system. There are electrophysiological results indicating that figure-ground segregation might be achieved already in the primary visual cortex (V1) (Bach & Meigen, 1997). Checkerboard stimuli differing in either luminance, colour, or motion direction of points within the check types served as figure-ground tests. These stimuli specifically activated the primary and secondary visual cortex more than homogeneous stimuli (i.e. those without checks) consisting of homogeneous versions of the colour, luminance, etc. of each of the check types. This was true both for multi-channel EEG recordings, and for functional MRI (Fig. 2). This result confirms the above hypothesis that early mechanisms exist that are able to extract a figure from its background simultaneously on the basis of different visual attributes, such as colour and texture. The synergy found between different cues has implications not only for the understanding of the intact human brain, but also for its disturbances in patients, and even for the design of instruments, e.g. in (fast) vehicles where fast and reliable identification of displayed information is important.
Figure 2: Difference between activation of the human brain achieved by a green/red checkerboard versus homogeneous green and red stimuli. The figure shows an inflated view of the areas of the human visual cortex specifically activated by this type of stimulus.