Presented to the Society for Chaos Theory in Psychology and the Life Sciences,
Berkeley, CA
June 1996

Abstract:

     We discuss a model of memory for visual form which treats the memory 
'trace' as a set of procedures for reconstructing earlier visual experience. The 
procedures, Barnsley's Iterated Function System (IFS), construct an image 
from a collection of operators (affine transformations). From this perspective, 
remembering and imagining are processes whose dynamics are captured by 
the iterative rules. Changes in memory for visual experience are described as 
changes in the parameters and weights of the reconstruction operators. The 
model is used to discuss known phenomena and effects in the empirical 
literature on memory for visual form. 

     Fractals, as all present know, are irregular geometric 
objects that yield detail at all scales. Unlike Euclidean, differentiable, 
objects that smooth out when zoomed into, fractals continue to reveal 
features as more closely regarded. Fractals also have "self-similarity" 
which, in one meaning refers to the presence of parts that resemble the 
whole, or to the continual repetition of a feature. Benoit Mandelbrot (1983) 
not only invented the term fractal, but advanced the position that fractal 
geometry is the geometry of nature. Eliot Porter, the nature photographer, 
upon reading Gleick's (1988) account of Mandelbrot's work, realized he 
had been taking pictures of fractals in nature for decades. To promote the 
point, he collaborated with Gleick on a collection of photographs for a book 
titled Nature's Chaos (Porter & Gleick, 1990). The slides shown here are 
samples from that book.

          From a Darwinian perspective, we propose that our sensory 
receptors evolved in the presence of fractal objects, bathed in and 
powerfully shaped by them. It makes sense to us, then, that fractal 
geometry should be adopted in the study of perception and memory for 
visual form (cf. Gilden, Schmuckler, & Clayton, 1993). Yet contemporary 
psychophysical studies of perception are dominated by Euclidean measures, 
and modern theories of visual form, such as Biederman's (1987) object 
recognition theory, have Euclidean objects (spheres, cubes, etc.) as 
primitives ("geons").


     A general model. We first present a general, abstract, 
model of memory for visual form from this perspective, and then consider 
a more specific version (see figure 1). We assume, first, that memory is not 
a passive list of attributes, but rather is a set of mental procedures. These 
procedures, when activated, allow a reconstruction of a semblance of the 
original experience. There are two classes of memory dynamics; those 
involved in remembering and imagining, and those involved in forgetting. 
These are very different procedures. The first is initiated by some query of 
memory, arising internally or externally. It is relatively fast and interacts 
with ongoing cognitive demands and simultaneous visual experience. The 
second process, altering memory, is the result of subsequent experience. It 
acts to alter procedure parameters.

     Given this general, sketchy, non-controversial view, how might 
fractal geometry be brought in to flesh out the details? We were 
encouraged to use the concept of Iterated Function System (IFS), 
introduced by Michael Barnsley (1988). Two things were compelling about 
IFSs. The first was their success at constructing natural-looking images. 
Examples are shown in the next few slides, which are not photographs, but 
are images produced by a computer IFS routine. One interesting feature of 
some of these images, which are fractals, is that they may be endlessly zoomed 
into without loss of their natural appearance.


     A second attractive feature of IFS is that they provide a solution to 
the computer science problem of image compression. This is illustrated by 
Barnsley's Fern, shown in figure 2, which requires but 24 numbers to 
generate, in contrast, say, to the 100,000 or more that would be needed to 
store a bit-map.

Figure 2. Barnsley's Fern.

     Iterated Function Systems. So what are IFSs? Well, the 
function part of the name refers to affine transformations. These are 
functions that alter geometric forms and are illustrated in figure 3. Three 
kinds of transformations are shown: displacement, compression, and 
rotation.

          Displacement is a change in location, a change in x or y, or both.

          Compression is reduction in scale, in x or y or both.

          Rotation is accomplished with two parameters reflecting degree 
          changes from the x and y axes.

Figure 3. Three affine transformations.

     There are a total of six parameters reflecting these transformations, 
three for changes in x, three for changes in y. So two equations, as 
shown in figure 3, allow a complete description of one transformation, or 
operator.

     These are systems, by which is meant we have a collection of 
transformations, or 'operators'. The resulting image is the set of images 
obtained after applying every operator.

     And they are iterated systems, which means the operators are 
repeatedly applied to the results of the previous application.


     We illustrate with a simple case. Three transformations, all 
involving compression in x and y by one-half, and displacement. We start 
with a square and apply the set (see figure 4). The result is a fractal known as 
the Sierpinski Triangle. The resulting image is an attractor for the iterative 
system. One non-obvious point: the form of the attractor is independent of 
the starting image. I could have started with any image other than the square, 
and the Sierpinski Triangle would have resulted.

Figure 4. Constructing the Sierpinski Triangle.

     A Specific IFS Model. Now we map this onto the general model. 
We assume that the representation of visual experience is a stored set of 
operators and their parameter values. The representations are activated by 
memory probes, and when activated allow the construction of an image in 
working memory (or a "visuo-spatial sketchpad", see Baddeley, 1986). The 
constructed image emerges over iterations, or real time.

     Different memory tasks require different degrees of completion. 
For example, 'imagining', literally forming a mental image, would likely 
require more detail than would recognition - judging the form had been 
presented earlier. In the case of recognition, reconstruction would 
presumably proceed only until some matching criterion is satisfied. Herein 
lies a line of research, because response time should depend on the specifics 
of the dynamic reconstruction. For example, if this characterization is 
correct, low-frequency spatial information should emerge first, whereas 
high-frequency information (detail) show require more time.


     Those are the dynamics of remembering. What about forgetting? 
According to the Gestalt view and data (e. g., Goldmeier, 1982), the 
goodness of a form depends in part on whether it is symmetrical and 
continuous. Memory for a good form is stable, while memory for a nearly 
good form moves toward goodness. This principle is illustrated in figure 5, 
which also shows a strength in this overall approach, namely, how a 
qualitative, global, change in a form results from a small change in a 
single IFS parameter. In this case, the figure on the left, a nearly-good figure, 
is the inspection figure, the one to be remembered. According to the Gestalt 
view, the memory for this figure would move toward goodness, perhaps to 
that on the right. This is accomplished by modifying one parameter. We plan 
research along these lines.

Figure 5. A nearly-good figure becomes good over time.

     What about forgetting produced by other experience? A possible 
approach is illustrated in figure 6. Suppose Form A is experienced 
at one time, and Form B later. One way to store both experiences, the way, 
for example of a neural network approach, would be to store an average of 
their IFS parameters (Stucki & Pollack, 1992). Now if we activate the 
resulting memory, we would get something like Form C. Form C illustrates 
how the approach nicely handles qualitative merging of two or more forms, 
and suggests a way to handle prototype learning.

Figure 6. Merging (C) the memories of two forms, A and B.

     All this could easily be set into any of a number of contemporary models 
of memory. For example, the parameters could be features in a global model 
(for example, Hintzman's MINERVA, 1988) or a connectionist model (e. g., 
Chappel & Humphreys, 1994). The difference, of course, is that 
the features in global and connectionist models are used to describe 
experienced stimuli. Here we use the values to dynamically reconstruct 
earlier experience.

     One substantial problem remains. It is known as the fractal 
inversion problem and refers to the fact that procedures are needed to 
encode any image. It is one thing to construct an image from a known IFS, 
quite another to determine the IFS needed to encode a given image. We 
are familiar with two notable developments on this problem. First, 
Stucki & Pollack (1992) have shown that neural networks can learn the 
IFS parameters. However, this occurs when the image is presented to the 
network along with its parameter values. It cannot find the solution on 
its own.

     Barnsley has achieved one solution to the inversion problem 
(Barnsley & Hurd, 1993) in sufficient detail to encourage him to form a 
computer hardware company and to patent the technique. But the solution 
involves splitting the image into numerous sub-sections and finding IFSs 
independently for each. Since the IFSs are computed locally, the global 
features are entirely ignored. But it is exactly the global features that are 
remembered and altered. So, even though Barnsley's technique satisfactorily 
compresses images automatically and is technically useful, it cannot serve as 
an adequate theory of the stored mental image.



     In summary, we have offered a few details about how Iterated 
Function Systems may be used as a part of a theory of memory for form. 
The approach raises a number of empirical and theoretical questions which 
we plan to pursue.




References


Baddeley A. D. Working Memory. Oxford, Oxford University Press.


Barnsley, M. F. (1988). Fractals Everywhere. Boston, Mass.: 
     Academic Press.


Barnsley, M. F. & Hurd, L. P. (1993). Fractal Image Compression. 
     Wellsley, Massachusetts: A K Peters.


Biederman, I. (1987). Recognition-by-components: A theory of human 
     image understanding. Psychological Review, 94,115-147.


Chappel, M. & Humphreys, M. S. (1994). An auto-associative neural 
     network for sparse representations: Analysis and application to models 
     of recognition and cued recall. Psychological Review, 101, 103-128.


Gilden, D. L., Schmuckler, M. A., & Clayton, K. (1993). The percep-tion of 
     natural contour. Psychological Review, 100, 460-478.


Gleick, J. (1988). Chaos: Making a New Science. New York, NY: 
     Penguin Books.


Goldmeier, E. (1982). The Memory Trace: Its Formation and its 
     Fate. Hillsdale, NJ : L. Erlbaum Associates.


Hintzman, D. L. (1988). Judgments of frequency and recognition memory in 
     a multiple-trace memory model. Psychological Review, 95, 528-551.


Mandelbrot, B. B. (1983). The Fractal Geometry of Nature. New 
     York : W. H.. Freeman.


Porter, E. & Gleick, J. (1990). Nature's Chaos. New York, NY: Viking 
     Penguin. 


Stucki, D. J. & Pollack, J. B. (July, 1992). Fractal (reconstructive analogue) 
     memory. Proceedings of the 14th Annual Conference of the Cognitive 
     Science Society, Bloomington, IN.