Verified by 
Key points
- Memory is influenced by what you believe is true of the world.
- Memory and perception are influenced by how you interpret the world.
- Your ability to see is there to help you figure out what is going on in the world.
A core belief for most people is that what they see is really what is happening in the world. Obviously, there are a few perceptual illusions that creep in, but most of those involve images that play on specific biases of the visual system. Two examples shown here are the Muller-Lyer illusion, in which lines of the same length appear different sizes depending on the angles at the end and the illusory contours you can see in a Kanizsa square.
We still assume that in more complex situations, what we see (and what we remember about what we have seen) is a good indicator of what actually happened. That is one reason why eyewitness testimony is part of criminal trials. What witnesses remember about a crime can be used as evidence.
For decades, studies have demonstrated situations in which eyewitness memory is inaccurate. In addition, the pioneering work of Elizabeth Loftus and her colleagues finds that words you hear describing an event you saw affect your later memory of it. Studies also suggest that our beliefs about the world influence what we remember later so that we may actually believe we encounter things that we did not experience directly.
More careful studies have been done recently to look at the interpretation of events. For example, suppose you saw three squares lined up a little distance apart on a computer screen. From left to right, there is a red, green, and blue rectangle. (For ease of discussing this here, I’ll call them rectangles A, B, and C.) Suppose A starts to move and contacts B, at which point it stops, and B starts to move, until it hits C, and then B stops and C moves. Most people seeing this sequence interpret it as A hitting B causing it to move until it hits C, which then causes it to move. They also remember this sequence of events.
Suppose, though, that you play with the timing a bit. So, A starts to move and then when it contacts B, C actually starts to move first and a short time later, C stops and then B starts to move. This is a weird order (which we’ll call ACB), because you would expect that A hitting B would cause B to move. Interestingly, if you show this sequence to people, and then ask them what they remember happening, they tend to misremember it as A hitting B, which moves and then hits C, causing it to move. That is, they remember a more sensible sequence of events.
What causes this mistaken memory? One obvious explanation is that people see the sequence correctly, but then misremember it. That explanation is reasonable, because of all the demonstrations of memory errors based on knowledge and expectations. An alternative that is worth considering, though, is that people actually see the event in a way that is consistent with the sensible sequence. That is, when they see the ACB sequence, perhaps they actually perceive it as the more reasonable ABC sequence.
A fascinating series of studies by Christos Bechlivanidis, Marc Buehner, Emma Tecwyn, David Lagnado, Chrisoph Hoerl, and Teresa McCormick in the February 2022 issue of Psychological Science explored this question.
They did a clever experiment in which people say sequences like ABC or ACB. The squares were presented against a white background. At some point, there is a brief flash in which the background turns black and then back to white. Participants can adjust when the flash happens. Sometimes, they are instructed to make the flash coincide with the moment when square B starts moving. Sometimes, they are instructed to make the flash coincide with the moment when square C starts moving. They adjust the flash by pressing keys on a computer keyboard to move the flash earlier or later. They can watch the video as often as they want and can adjust the flash as many times as they want until they are happy with where it lands.
The two accounts for the mistaken memory of the sequence make different predictions for this study. If people are perceiving the sequence in the way it happens, but misremembering it, then when viewing the sequences, people should time the flash about where each square starts moving. But, if people are actually misperceiving the sequence, then they should be accurate on the ABC sequence, but not for the ACB sequence. For the ACB sequence, people should put the flash for square B before it actually starts moving and the flash for square C after it actually starts moving. That would be consistent with the idea that they are seeing the ACB sequence as if A hits B, which then hits C.
So what happens?
Across the studies, people respond as though they are misperceiving the weird sequence. They respond to the ACB sequence by putting the flashes on the screen as if the timing were that A hits B, which hits C.
At first, it might seem strange that vision is filling in information that isn’t there. It isn’t as odd as it seems, though. Your ability to see is there to help you figure out what is going on in the world, and because objects don’t generally start moving for no reason, it makes sense that you would see things in a sensible way.
After all, think about what happens when you watch a movie. You are watching a stream of static pictures. You see things moving, because your visual system is trying to make sense of that movement. And think about this: You can’t have the experience of something moving until the second image has already appeared. Otherwise, how would you know in which direction the movement was happening? So, even that experience of motion from movies (or computer screens) requires the visual system to fill in an interpretation of what is happening slightly after it happened.
Finally, it is important to bear in mind that there are limits on this kind of misperception. In these studies, the squares are fairly close together, and the collisions happen fairly close together in time. If the squares were very far apart or moving very slowly so that the differences in movement were large, then you would see them correctly (and probably remember them correctly later).
References
Loftus, E.F. (1996). Eyewitness Testimony. Cambridge, MA: Cambridge University Press.
Sulin, R. A., & Dooling, D. J. (1974). Intrusion of a thematic idea in retention of prose. Journal of Experimental Psychology, 103(2), 255–262
Bechlivanidis, C. & Lagnado, D.A. (2016). Time reordered: Causal perception guides the interpretation of temporal order. Cognition, 146, 58-66.
Bechlivanidis, C., Buehner, M.J., Tecwyn, E.C., Lagnado, D.A., Hoerl, C., & McCormick, T. (2022). Human vision reconstructs time to satisfy causal constraints. Psychological Science, 33(2), 224-235.
