Thinking and Reasoning with Diagrams

According to Duncker (1935/1945, p. 1), "a problem arises when a living creature has a goal but does not know how this goal is to be reached. Whenever one cannot go from the given situation to the desired situation simply by action [i.e., by the performance of obvious operations], then there has to be recourse to thinking." Novick and Bassok (2005) argue that to understand this thinking, researchers must distinguish between representations and solution procedures. Solvers typically begin their problem-solving efforts by trying to understand, at least at a rudimentary level, the underlying structure of the problem. In doing so, they construct some type of *problem representation,* which may be either internal or external (or both). Then they perform operations on the information in the representation in an attempt to get from the given situation to the goal (i.e., to determine the solution). That is, they apply a *solution procedure.* Although these processes are clearly interrelated, it is possible to study their separate contributions to the success of problem solving and reasoning.

I am interested in students' knowledge and use of three, external, spatial diagram representations -- matrices, networks (i.e., path diagrams), and hierarchies (i.e., trees) -- that are important tools for thinking both in everyday situations and in formal domains (Novick, 2001). Like other abstract diagrams, much of their usefulness can be attributed to three sources: (a) They simplify the complex, (b) they make the abstract more concrete, and (c) they substitute easier perceptual inferences for more computationally-intensive search processes and sentential deductive inferences. It is not surprising, therefore, that numerous researchers have shown that using abstract diagrams, including the three spatial diagrams, often facilitates learning and problem solving.

An important advantage of these types of representations is that they are applicable across a wide variety of contexts. That is, they highlight structural commonalities across situations that are superficially quite different. For example, a hierarchy can be used to represent evolutionary relationships among a set of taxa, a basketball tournament, or a corporate power structure. Similarly, a network can be used to represent the flight paths for an airline, the friendships between people at a conference, or the (hypothesized) structure of semantic memory. More generally, these types of representations are useful for solving a wide variety of problems involving analytical (including mathematical) reasoning. By successfully constructing (appropriate) spatial diagram representations, problem solvers would be led to see deep similarities among diverse problems that otherwise might not be salient. There is a large literature documenting the importance of structural understanding as a key factor underlying expertise.
**The Nature of Students' Knowledge**

It is critical to study solvers' knowledge of spatial diagram representations to fully understand their use of such representations to support reasoning and problem solving. One source of information about appropriate spatial diagram representations for different types of problem structures is examples of specific situations in which these types of representations have proven useful in the past (e.g., matrices are used for multiplication tables, seating charts, time schedules, police maps of a city, etc.). That is, besides transferring a sequence of solution operators from an example problem to a related test problem, solvers might transfer the type of representation used to describe the problem's underlying structure. My early research on spatial diagram representations examined the processes involved in and the factors affecting representational transfer in comparison to procedural transfer (Novick, 1990; Novick & Hmelo, 1994). The most important difference between these two types of transfer is that in procedural transfer the test problem's solution procedure is constructed by adapting the procedure from the example problem, whereas in representational transfer the representation for the test problem appears to be constructed from scratch (rather than by adapting the specific representation provided with the example problem). This result brings into focus the importance of studying the process of representation construction, a topic that is considered below.

A second source of information also is available to solvers to help guide their attempts to select and construct (appropriate) spatial diagram representations. The results of several experiments (Novick, Hurley, & Francis, 1999) suggest that college students have at least rudimentary abstract, rule-based knowledge concerning the applicability conditions for the three spatial diagrams. The results of a more recent study in which subjects had to choose the most appropriate type of spatial diagram for scenarios written in a specific content domain versus completely abstractly provide more direct evidence that students' representation selections are based, at least in part, on abstract, rule-based knowledge (Hurley & Novick, 2006). We found that subjects' representation selections were as accurate for scenarios that were written using completely abstract language as for scenarios that described a specific concrete situation involving familiar objects and concepts. In a follow-up study in which subjects provided think-aloud protocols while selecting diagrams for the abstractly-worded scenarios, we found that students almost never referred to concrete, real world situations. In contrast, they often referred to abstract features of the diagrams. It is unclear at this point, however, which aspects of using spatial diagram representations to support problem solving and reasoning rely more on abstract, rule-based knowledge about these representations and which rely more on the details of specific examples encountered previously.

Diagrams are among the oldest preserved examples of written mathematics. The past 15 years or so has seen increased attention in the mathematics education community to the goal of developing numeracy -- the mathematical counterpart to literacy -- among school children. Most educational theorists view representational, spatial, graphical, visual, or diagrammatic competence as included among the requisite skills that comprise numeracy. The National Council of Teachers of Mathematics (2000) standards emphasize that K-12 students need to obtain more sophisticated, and explicit, knowledge of culturally-significant types of abstract, mathematical diagrams -- including the three types of diagrams I have investigated in my research. In a recent manuscript
(Novick, 2004), I proposed a model of diagram literacy that specifies six types of knowledge that students should possess to demonstrate diagrammatic competence -- implicit, construction, similarity, structural, metacognitive, and translational. I also reported the results of a study that examined the diagram literacy of students from three distinct populations -- pre-service, secondary level, math teachers; computer science majors; and typical undergraduates -- with respect to matrices, networks, and hierarchies. The results of the study are reassuring in some ways with respect to the level of diagram literacy exhibited by students at the culmination of current K-12 instruction, as well as that possessed by teachers of the upcoming generation of secondary students. However, the results also point to areas in which pre-service math teachers should be better prepared if the goals for diagram literacy proposed by the National Council of Teachers of Mathematics are to be met.
**Structural Analysis of Matrices, Networks, and Hierarchies**

Determining the appropriate representation to use for the situation at hand depends on assessing the degree of fit between the structure of the information to be represented and the structures of various representations. Just as hammers, screwdrivers, and wrenches work best in specific types of situations, so do matrices, networks, and hierarchies (see Novick, 2001, for further discussion of this analogy). It is critical, therefore, to specify the problem structures for which each type of spatial diagram is best suited. Novick and Hurley (2001) did just that. We hypothesized that 10 properties (e.g., global structure, link type, linking relations, traversal) distinguish when to use each of the three types of representations. Each representation has a value for each property, and these property values constitute the hypothesized applicability conditions for the representations. For example, matrices are particularly useful when (a) all possible combinations of the items across two sets must be considered, (b) the links between items in the different sets are non-directional (i.e., purely associative), and (c) it is important to be able to explicitly mark pairs of items that cannot be linked. In contrast, networks are particularly useful when (a) there are no constraints on which items may be linked, (b) the links between items are many-to-many, and (c) multiple different routes may sometimes be followed to travel between two particular items. Finally, hierarchies are particularly useful when (a) the items are distinguished according to different levels, (b) the links between items are either one-to-many or many-to-one (but not both), and (c) only one route exists between any two items.

The results of a study in which subjects had to verbally justify their selection of the most appropriate type of representation to use for each of 18 scenarios (all of which were set in a vaguely medical context) provided good support for the structural analysis (Novick & Hurley, 2001), both at the level of the properties and at the level of the individual property values (i.e., the applicability conditions). This support came from an analysis of subjects' representation choices, as well as detailed analyses of their verbal justifications. The results of this study also provided important preliminary information concerning how students' knowledge of the structural properties is organized in memory and how the organization varies as a function of expertise (Novick, 2001; Novick & Hurley, 2001).

One limitation of this study, however, is that verbal protocols are useful with respect to what subjects say, but no firm conclusions can be drawn from what they fail to say. With respect to the structural analysis of the three spatial diagrams, this limitation leaves two important questions unanswered. The first concerns the status of the (9 of 30) proposed property values that were mentioned either not at all or only by a tiny handful of students: Do these property values constitute applicability conditions for the representations or not? The second question concerns the relative importance or diagnosticity of the applicability conditions; presumably, not all applicability conditions are equally diagnostic cues for the use of a particular type of diagram. For example, the fact that the items in a particular situation are organized into levels is probably a more diagnostic cue for a hierarchy representation than is the fact that there are directional links between the items.

Two recent studies addressed these issues (Novick, 2006b). In one study, subjects at different levels of expertise (typical undergraduates and advanced computer science majors) were asked to rate the diagnosticity of each of the proposed applicability conditions with respect to each of the three types of representations. This task required students to discriminate among the three types of diagrams for each applicability condition. In a second study, students from these same two populations rated the diagnosticity of the applicability conditions for each representation separately. This task required students to discriminate among the applicability conditions for a particular type of diagram. For example, subjects were given a list of all the proposed applicability conditions for the hierarchy representation, and they had to rate how diagnostic each of those property values is for that type of diagram. The results of these two studies validated 24-26 of the 30 hypothesized applicability conditions and provided evidence regarding the relative importance, or diagnosticity, of the validated properties for each type of diagram. A different set of properties was identified as most highly diagnostic for each type of diagram, indicating that the three spatial diagrams are optimized to serve different representational functions: The matrix stores static information about the kind of relation that exists between pairs of items in different sets, the network conveys dynamic information by showing the local connections and global routes connecting the items being represented, and the hierarchy depicts a rigid structure of power or precedence relations among items.
**Constructing and Reasoning from Matrices, Networks, and Hierarchies**

The applicability conditions specify which type of diagram best fits the structure of a particular problem. But selecting the right type of diagram is only the first step. Next, one must draw that diagram so that it accurately and efficiently represents the given situation (Novick, 2001). For his dissertation, my graduate student, Sean Hurley, conducted three studies concerning diagram construction and use (Hurley & Novick, in press). We generated several hypotheses concerning the conventions for mapping problem information onto matrix, network, and hierarchy diagrams (e.g., objects are mapped onto nodes in networks and hierarchies, and onto rows/columns in matrices). In two experiments, we tested the validity of the hypothesized conventions by analyzing (a) college students' verbal descriptions of how they would construct these diagrams and (b) their drawings of these diagrams. Strong support was found for the conventions hypothesized to be associated with constructing all three diagrams. In Experiment 3, we examined the importance of the conventions for successful and efficient diagram use by evaluating students' reasoning time and accuracy when they answered questions using diagrams that either followed or violated the hypothesized conventions. Strong effects of convention adherence on reasoning time and accuracy were found for matrices and networks, but not for hierarchies.

Abstract diagrams are important not only in mathematics but in science as well (Novick, 2006a). In biology, hierarchical diagrams are especially common. For the past several years, I have been working with Kefyn Catley, a biologist and science educator at Western Carolina University, to investigate college students' understanding of *cladograms,* the most important tool that contemporary scientists use to reason about evolutionary relationships. A cladogram is a type of hierarchical diagram that depicts the distribution of characters (i.e., physical, molecular, and behavioral characteristics) among a set of taxa. More information about this research may be found here.
**Assembly Diagrams**

In a project conducted in collaboration with
Doug Morse,
I investigated the role of iconic diagrams in facilitating the execution of a set of procedural instructions.
Although diagrams are ubiquitous in instructions for assembling a wide variety of objects (e.g., bicycles, bookcases, Lego vehicles), there is surprisingly little research on the role of diagrams in supporting object assembly. We examined this issue in the domain of origami, a Japanese paper-folding task enjoyed by both children and adults. This research had two principle aims. First, it extended research on diagrams as instructional aids to the case of object assembly. Second, it provided new data on the effectiveness of adding step-by-step versus completed-object diagrams to text instructions as a function of the difficulty of the assembly task. We hypothesized that completed-object diagrams are primarily helpful in situations in which the steps needed to construct the objects can easily be extracted mentally from the diagrams. In such situations, a completed-object diagram can substitute for a large number of step-by-step diagrams. When the individual steps cannot easily be extracted from the completed-object diagram, however, the diagram will have little or no benefit for object assembly. The results of three experiments (Novick & Morse, 2000) supported these predictions.
**Learning a Visual Computer Language**

In another project, Kirsten Whitley, Doug Fisher, and I examined the effectiveness of the visual representation in LabVIEW, a visual programming language (Whitley, Novick, & Fisher, 2006). We compared the performance of students taught a subset of the LabVIEW language, with its circuit-diagram type of representation, to that of students taught a textual equivalent of that language. Performance was assessed on three types of problems. For the tracing problems, students were given program code and had to evaluate what output would be produced given certain input. For the parallelism problems, students had to determine which functions could execute immediately following a designated function. For the debugging problems, students were given a description of the function the code was supposed to compute, as well as some code ostensibly written to perform that function. Students had to locate and identify the (single) bug in the code. For the tracing problems, accuracy was comparable in the two representation conditions. The high level of accuracy in both conditions indicates that our instruction was effective. For the parallelism and debugging problems, however, the students in the visual condition were reliably more accurate than those in the textual condition. The better performance of the visual subjects on the parallelism problems is consistent with the power of diagrammatic representations to make readily accessible information that must be laboriously inferred from equivalent textual representations. The results for the debugging problems indicate that, as in other domains, (a) the visual representation facilitated global understanding, and (b) the advantage of the visual representation was largest for the most difficult problems.

Hurley, S. M., & Novick, L. R. (2010). Solving problems using matrix, network, and hierarchy diagrams: The consequences of violating construction conventions. The Quarterly Journal of Experimental Psychology, 63, 275-290.

Hurley, S. M., & Novick, L. R. (2006). Context and structure: The nature of students' knowledge about three spatial diagram representations. Thinking & Reasoning, 12, 281-308.

Novick, L. R. (2006b). Understanding spatial diagram structure. The Quarterly Journal of Experimental Psychology, 59, 1826-1856.

Novick, L. R. (2006a). The importance of both diagrammatic conventions and domain-specific knowledge for diagram literacy in science: The hierarchy as an illustrative case. In D. Barker-Plummer, R. Cox, & N. Swoboda (Eds.), Diagrams 2006, LNAI 4045 (pp. 1-11). Berlin: Springer-Verlag.

Whitley, K. N., Novick, L. R., & Fisher, D. (2006). Evidence in favor of visual representation for the dataflow paradigm: An experiment testing LabVIEW's comprehensibility. International Journal of Human-Computer Studies, 64, 281-303.

Novick, L. R., & Bassok, M. (2005). Problem solving. In K. J. Holyoak & R. G. Morrison (Eds.), Cambridge handbook of thinking and reasoning (Ch. 14, pp. 321-349). New York, NY: Cambridge University Press.

Novick, L. R. (2004). Diagram literacy in pre-service math teachers, computer science majors, and typical undergraduates: The case of matrices, networks, and hierarchies. Mathematical Thinking and Learning, 6, 307-342.

Novick, L. R., & Hurley, S. M. (2001). To matrix, network, or hierarchy, that is the question. Cognitive Psychology, 42, 158-216.

Novick, L. R. (2001). Spatial diagrams: Key instruments in the toolbox for thought. In D. L. Medin (Ed.), The psychology of learning and motivation (Vol. 40, pp. 279-325). San Diego, CA: Academic Press.

Novick, L. R., & Morse, D. L. (2000). Folding a fish, making a mushroom: The role of diagrams in executing assembly procedures. Memory & Cognition, 28, 1242-1256.

Novick, L. R., Hurley, S. M., & Francis, M. D. (1999). Evidence for abstract, schematic knowledge of three spatial diagram representations. Memory & Cognition, 27, 288-308.

Novick, L. R., & Hmelo, C. E. (1994). Transferring symbolic representations across nonisomorphic problems. Journal of Experimental Psychology: Learning, Memory, and Cognition, 20, 1296-1321.

Novick, L. R. (1990). Representational transfer in problem solving. Psychological Science, 1, 128-132.

Return to Lab Home