We discuss a model of memory for visual form which treats the memory 'trace' as a set of procedures for reconstructing earlier visual experience. The procedures, Barnsley's Iterated Function System (IFS), construct an image from a collection of operators (affine transformations). From this perspective, remembering and imagining are processes whose dynamics are captured by the iterative rules. Changes in memory for visual experience are described as changes in the parameters and weights of the reconstruction operators. The model is used to discuss known phenomena and effects in the empirical literature on memory for visual form.
Fractals, as all present know, are irregular geometric objects that yield detail at all scales. Unlike Euclidean, differentiable, objects that smooth out when zoomed into, fractals continue to reveal features as more closely regarded. Fractals also have "self-similarity" which, in one meaning refers to the presence of parts that resemble the whole, or to the continual repetition of a feature. Benoit Mandelbrot (1983) not only invented the term fractal, but advanced the position that fractal geometry is the geometry of nature. Eliot Porter, the nature photographer, upon reading Gleick's (1988) account of Mandelbrot's work, realized he had been taking pictures of fractals in nature for decades. To promote the point, he collaborated with Gleick on a collection of photographs for a book titled Nature's Chaos (Porter & Gleick, 1990). The slides shown here are samples from that book.
From a Darwinian perspective, we propose that our sensory receptors evolved in the presence of fractal objects, bathed in and powerfully shaped by them. It makes sense to us, then, that fractal geometry should be adopted in the study of perception and memory for visual form (cf. Gilden, Schmuckler, & Clayton, 1993). Yet contemporary psychophysical studies of perception are dominated by Euclidean measures, and modern theories of visual form, such as Biederman's (1987) object recognition theory, have Euclidean objects (spheres, cubes, etc.) as primitives ("geons").
A general model. We first present a general, abstract, model of memory for visual form from this perspective, and then consider a more specific version (see figure 1). We assume, first, that memory is not a passive list of attributes, but rather is a set of mental procedures. These procedures, when activated, allow a reconstruction of a semblance of the original experience. There are two classes of memory dynamics; those involved in remembering and imagining, and those involved in forgetting. These are very different procedures. The first is initiated by some query of memory, arising internally or externally. It is relatively fast and interacts with ongoing cognitive demands and simultaneous visual experience. The second process, altering memory, is the result of subsequent experience. It acts to alter procedure parameters.
Visual experience results in a set of mental
Remembering and Imagining
Activation of the procedures allow a reconstruction of a
Forgetting occurs with change in parameter value
Figure 1. General Model of Memory for Visual Form.
Given this general, sketchy, non-controversial view, how might fractal geometry be brought in to flesh out the details? We were encouraged to use the concept of Iterated Function System (IFS), introduced by Michael Barnsley (1988). Two things were compelling about IFSs. The first was their success at constructing natural-looking images. Examples are shown in the next few slides, which are not photographs, but are images produced by a computer IFS routine. One interesting feature of some of these images, which are fractals, is that they may be endlessly zoomed into without loss of their natural appearance.
A second attractive feature of IFS is that they provide a solution to the computer science problem of image compression. This is illustrated by Barnsley's Fern, shown in figure 2, which requires but 24 numbers to generate, in contrast, say, to the 100,000 or more that would be needed to store a bit-map.
Iterated Function Systems. So what are IFSs? Well, the function part of the name refers to affine transformations. These are functions that alter geometric forms and are illustrated in figure 3. Three kinds of transformations are shown: displacement, compression, and rotation.
Displacement is a change in location, a change in x or y, or both.
Compression is reduction in scale, in x or y or both.
Rotation is accomplished with two parameters reflecting degree changes from the x and y axes.
There are a total of six parameters reflecting these transformations, three for changes in x, three for changes in y. So two equations, as shown in figure 3, allow a complete description of one transformation, or operator.
These are systems, by which is meant we have a collection of transformations, or 'operators'. The resulting image is the set of images obtained after applying every operator.
And they are iterated systems, which means the operators are repeatedly applied to the results of the previous application.
We illustrate with a simple case. Three transformations, all involving compression in x and y by one-half, and displacement. We start with a square and apply the set (see figure 4). The result is a fractal known as the Sierpinski Triangle. The resulting image is an attractor for the iterative system. One non-obvious point: the form of the attractor is independent of the starting image. I could have started with any image other than the square, and the Sierpinski Triangle would have resulted.
A Specific IFS Model. Now we map this onto the general model. We assume that the representation of visual experience is a stored set of operators and their parameter values. The representations are activated by memory probes, and when activated allow the construction of an image in working memory (or a "visuo-spatial sketchpad", see Baddeley, 1986). The constructed image emerges over iterations, or real time.
Different memory tasks require different degrees of completion. For example, 'imagining', literally forming a mental image, would likely require more detail than would recognition - judging the form had been presented earlier. In the case of recognition, reconstruction would presumably proceed only until some matching criterion is satisfied. Herein lies a line of research, because response time should depend on the specifics of the dynamic reconstruction. For example, if this characterization is correct, low-frequency spatial information should emerge first, whereas high-frequency information (detail) show require more time.
Those are the dynamics of remembering. What about forgetting? According to the Gestalt view and data (e. g., Goldmeier, 1982), the goodness of a form depends in part on whether it is symmetrical and continuous. Memory for a good form is stable, while memory for a nearly good form moves toward goodness. This principle is illustrated in figure 5, which also shows a strength in this overall approach, namely, how a qualitative, global, change in a form results from a small change in a single IFS parameter. In this case, the figure on the left, a nearly-good figure, is the inspection figure, the one to be remembered. According to the Gestalt view, the memory for this figure would move toward goodness, perhaps to that on the right. This is accomplished by modifying one parameter. We plan research along these lines.
What about forgetting produced by other experience? A possible approach is illustrated in figure 6. Suppose Form A is experienced at one time, and Form B later. One way to store both experiences, the way, for example of a neural network approach, would be to store an average of their IFS parameters (Stucki & Pollack, 1992). Now if we activate the resulting memory, we would get something like Form C. Form C illustrates how the approach nicely handles qualitative merging of two or more forms, and suggests a way to handle prototype learning.
All this could easily be set into any of a number of contemporary models of memory. For example, the parameters could be features in a global model (for example, Hintzman's MINERVA, 1988) or a connectionist model (e. g., Chappel & Humphreys, 1994). The difference, of course, is that the features in global and connectionist models are used to describe experienced stimuli. Here we use the values to dynamically reconstruct earlier experience.
One substantial problem remains. It is known as the fractal inversion problem and refers to the fact that procedures are needed to encode any image. It is one thing to construct an image from a known IFS, quite another to determine the IFS needed to encode a given image. We are familiar with two notable developments on this problem. First, Stucki & Pollack (1992) have shown that neural networks can learn the IFS parameters. However, this occurs when the image is presented to the network along with its parameter values. It cannot find the solution on its own.
Barnsley has achieved one solution to the inversion problem (Barnsley & Hurd, 1993) in sufficient detail to encourage him to form a computer hardware company and to patent the technique. But the solution involves splitting the image into numerous sub-sections and finding IFSs independently for each. Since the IFSs are computed locally, the global features are entirely ignored. But it is exactly the global features that are remembered and altered. So, even though Barnsley's technique satisfactorily compresses images automatically and is technically useful, it cannot serve as an adequate theory of the stored mental image.
In summary, we have offered a few details about how Iterated Function Systems may be used as a part of a theory of memory for form. The approach raises a number of empirical and theoretical questions which we plan to pursue.
Baddeley A. D. Working Memory. Oxford, Oxford University Press.
Barnsley, M. F. (1988). Fractals Everywhere. Boston, Mass.: Academic Press.
Barnsley, M. F. & Hurd, L. P. (1993). Fractal Image Compression. Wellsley, Massachusetts: A K Peters.
Biederman, I. (1987). Recognition-by-components: A theory of human image understanding. Psychological Review, 94,115-147.
Chappel, M. & Humphreys, M. S. (1994). An auto-associative neural network for sparse representations: Analysis and application to models of recognition and cued recall. Psychological Review, 101, 103-128.
Gilden, D. L., Schmuckler, M. A., & Clayton, K. (1993). The percep-tion of natural contour. Psychological Review, 100, 460-478.
Gleick, J. (1988). Chaos: Making a New Science. New York, NY: Penguin Books.
Goldmeier, E. (1982). The Memory Trace: Its Formation and its Fate. Hillsdale, NJ : L. Erlbaum Associates.
Hintzman, D. L. (1988). Judgments of frequency and recognition memory in a multiple-trace memory model. Psychological Review, 95, 528-551.
Mandelbrot, B. B. (1983). The Fractal Geometry of Nature. New York : W. H.. Freeman.
Porter, E. & Gleick, J. (1990). Nature's Chaos. New York, NY: Viking Penguin.
Stucki, D. J. & Pollack, J. B. (July, 1992). Fractal (reconstructive analogue) memory. Proceedings of the 14th Annual Conference of the Cognitive Science Society, Bloomington, IN.