9 February 2013
5 February 2013
In Italy in the twenties the Corriere dei piccoli used to publish the best-known American comic strips of the time: Happy Hooligan, the Katzenjammer Kids, Felix the Cat, Maggie and Jiggs, all of them rebaptized with American names. And there were also Italian comic strips, some of them of excellent quality, according to the graphic taste and style of the period. In Italy they had not yet started to use balloons for dialogue (these began in the thirties with the importation of Mickey Mouse). The Corriere dei piccoli redrew the American cartoons without balloons, replacing them with two or four rhymed lines under each cartoon. However, being unable to read, I could easily dispense with the words — the pictures were enough. I used to live with this little magazine, which my mother had begun buying and collecting even before I was born and had bound into volumes year by year. I would spend hours following the cartoons of each series from one issue to another, while in my mind I told myself the stories, interpreting the scenes in different ways — I produced variants, put together the single episodes into a story of broader scope, thought out and isolated and then connected the recurring elements in each series, mixing up one series with another, and invented new series in which the secondary characters became protagonists.
When I learned to read, the advantage I gained was minimal. Those simple-minded rhyming couplets provided no illuminating information; often they were stabs in the dark like my own, and it was evident that the rhymster had no idea what might have been in the balloons of the original, either because he did not understand English or because he was working from cartoons that had already been redrawn and rendered wordless. In any case, I preferred to ignore the written lines and to continue with my favorite occupation of dayreaming within the pictures and their sequence.
– from Six Memos for the Next Millennium (1988) by Italo Calvino
* * *
I’m still a bit hacked off about Google no longer routing image searchers directly to the blog post where the image appears, instead steering traffic to a Google-generated simulacrum of the original blog post. This devious maneuver shifts vast numbers of hits from blogs to Google — probably they’re artificially juicing their traffic numbers to satisfy the investment community about business growth potential. In my prior post I noted that over the years a lot of people have visited the blog to look at the screengrabs I put up from movies I’d recently watched. So what’s the big deal about losing these visitors to Google’s insatiable attention-gobbling maw? It’s not like I shot the films from which I nab the still images. I do have to select the images to post, and it takes a bit of effort to isolate the right ones. For me though what’s important is the assemblage, the selection of multiple images from the same film.
The stills might trigger memories for people who have previously seen the movie. For those who haven’t, the stills could serve as a kind of teaser or trailer, perhaps stimulating them to give it a viewing. But for me the process of selecting and assembling the stills functioned as a kind of prismatic concentration, bringing into clearer focus a particular thematic element extending through the movie. I couldn’t necessarily name the theme; it’s more of a visual resonance. Take this one, for instance, or this one — both historically popular hits from Google image searches. They’re like cinematic triptychs, or Calvino’s cartoons without bubbles. If you don’t already know the stories that go with the pictures you might be able to make up your own.
If I were a different sort of psychologist I might suggest a self-help intervention. Find five photos of yourself, or — better — envision five situations you’ve experienced in your life that you regard as important in some way. Describe each situation in a paragraph, focusing on facts rather than interpretations — sort of like a screengrab without dialogue or a cartoon without a bubble. Now take those five scenarios and, disregarding as best you can the meaning you usually ascribe to these events, invent a new story linking them together.
Me, I’m trying to figure out whether this is a good fiction-writing maneuver: assemble a series of short situational stills involving a character without linking the situations together narratively. Then the (imagined) reader can assemble a story from the fragments.
10 December 2012
2 December 2012
Getting my own numerological obsession out of the way… Early in the film Oscar asks Céline, his chauffeuse, if he has a lot of appointments scheduled for the day. Nine, she tells him. But at this point Oscar has already had two appointments. So I counted them myself:
1. Hotel portal into movie theater
2. Exec leaving big white house with children and guards on the roof
3. Beggar woman
4. Point-light CGI dance…
5. Beauty and the beast at photo shoot…
6. Father retrieving daughter from party
7. Entr’acte with accordions…
8. Murder at warehouse
9. Murder at cafe
10. Old man dying in bed
11. Musical interlude with Kylie Minogue…
12. Home with chimps
One could split a couple of these appointments into two separate scenes, but each of the 12 requires Oscar to assume a different persona. Between appointments he is Oscar, riding in the limo getting ready for his next appointment. But is this the “real” Oscar? Or as limo-rider is he again playing a role? Surely he is, since it’s this role — Oscar as performer doing a variety of gigs — that holds the whole movie together. Make it 13 appointments then.
But what about the last scene, the titular scene, when the holy motors, the limos, are all gathered at the garage? Oscar isn’t in this scene, but like the interior of the limo, the garage is a setting that frames the whole movie. The cars are lamenting the lost age of the “visible machines” like themselves, the “holy motors,” replaced now by the unholy and invisible machines of CGI and the backroom banking mechanisms where the movie deals get done and the appointments get put on the books. We infer that Oscar too is a holy motor, a visible machine, a live actor, and that his time too is coming to an end. So this nostalgic garage, populated by no living humans, is a kind of mausoleum. We’ve had scenes shot in graveyards, we’ve seen Oscar killing his own double, twice, we’ve seen Kylie either suicided or playing a suicide, so this comically mournful scene of reminiscing limos is the future unreality toward which the rest of the movie points.
So let’s make it 14 appointments, the last one with Oscar absent except in postmortem spirit. And voilà, we have the fourteen Stations of the Cross, with Oscar playing the role of Jesus and the garage playing the tomb, the Fourteenth Station. And the limos are like the angels in the tomb, ready to bring Oscar back from the dead again tomorrow. I believe it was the Entr’acte that tipped off this idea for me, since it’s played inside a cathedral like a liturgical procession, and all of those old French churches have the Stations lining their walls.
21 November 2012
Following up on my last post about JJ Gibson‘s ideas on how people perceive pictures, here’s a video produced by psychologist Tim Smith that tracks the eye movements of 11 people watching a segment of the film There Will Be Blood.
Contemporary film editing relies largely on cuts and close-ups, alternating perspectives and montage, to direct the viewer’s attention. In contrast, the film excerpt in the video consists of one extended continuous take and a second shorter one. Despite the freedom afforded the viewer to look around these extended shots at leisure and at will, the cinematographer remains in control. The first scene illustrates how visual attention is drawn by the objects in the environment, their interrelationships, and their movements. Human faces and hands are particularly compelling. The second scene shows how the observer’s movement through the environment also directs visual attention. The eye is drawn to what’s present in the scene, but the observer also anticipates change in the scene. The viewer follows the character’s gaze toward the object of his attention; a pause in the conversation leads the eye toward the character who is likely to speak next; a camera tracking to the right directs the viewer’s attention to the right of the screen in anticipation of what will slide into view next.
Discussing his eye movement video on David Bordwell’s blog, Smith writes:
The most striking feature of the gaze behaviour when it is animated in this way is the very fast pace at which we shift our eyes around the screen. On average, each fixation is about 300 milliseconds in duration. (A millisecond is a thousandth of a second.) Amazingly, that means that each fixation of the fovea lasts only about 1/3 of a second. These fixations are separated by even briefer saccadic eye movements, taking between 15 and 30 milliseconds!
Looking at these patterns, our gaze may appear unusually busy and erratic, but we’re moving our eyes like this every moment of our waking lives. We are not aware of the frenetic pace of our attention because we are effectively blind every time we saccade between locations. This process is known as saccadic suppression. Our visual system automatically stitches together the information encoded during each fixation to effortlessly create the perception of a constant, stable scene…
The second most striking observation you may have about the video is how coordinated the gaze of multiple viewers is. Most of the time, all viewers are looking in a similar place… Viewers’ gazes are attracted by the sudden appearance of objects, moving hands, heads, and bodies. The greater the motion contrast between the point of motion and the static background, the more likely viewers will look at it. If there is only one point of motion at a particular moment, then all viewers will look at the motion, creating attentional synchrony.
This is a powerful technique for guiding attention through a film. But it’s of course not unique to film. Noticing points of motion is a natural bias which we have evolved by living in the real world. If we were not sensitive to peripheral motion, then the tiger in the bushes might have killed our ancestors before they had chance to pass their genes down to us.
19 November 2012
[From The Ecological Approach to Visual Perception by James J. Gibson, 1979 — Chapter 15, “Pictures and Visual Awareness”]
To see the environment is to extract information from the ambient array of light. What is it, then, to see a picture of something? The information in ambient light consists not of forms and colors but of invariants. Is it implied that the information in a picture does not consist of forms and colors but of invariants likewise? That sounds very odd, for we suppose that a picture is entirely composed of forms and colors. The kind of vision we get from pictures is harder to understand than the kind we get from ambient light, not easier…
* * *
When the young child sees the family cat at play the front view , side view, rear view, top view, and so on are not seen, and what gets perceived is the invariant cat. The child does not notice the aspects of perspectives of the cat until he is much older; he just sees the cat rolling over. Hence, when the child first sees a picture of a cat he is prepared to pick up the invariants, and he pays no attention to the frozen perspective of the picture, drawing, photograph, or cartoon. It is not that he sees an abstract cat, or a conceptual cat, or the common features of the class of cats, as some philosophers would have us believe; what he gets is the information for the persistence of that peculiar, furry, mobile layout of surfaces.
When the young child sees the cat run away, he does not notice the small image but the far-off cat. Thus, when he sees two adjacent pictures of Felix in the comic book, a large Felix at the bottom of its picture and another small Felix higher up in its picture, he is prepared to perceive the latter as farther off. When he sees the cat half-hidden by the chair, he perceives a partly hidden cat, not a half-cat, and therefore he is prepared to see the same thing in a drawing.
The child never sees a man as a silhouette, or as a cutout like a paper doll, but probably sees a sort of head-body-arms-legs invariant. Consequently, any outline drawing of this invariant is recognized as a man, and the outlines tend to be seen as the occluding edges of a man with interchangeable near and far sides. Even when the outlines give way to line segments, as in so-called stick figures, the invariant may still be displayed and the man perceived…
To summarize, a picture is a surface so treated that it makes available an optic array of arrested structures with underlying invariants of structure. The cross-sections of the visual angles of the array are forms, but the invariants are formless. The array is delimited, not ambient. The array is arrested in time, except for the case of the motion picture… A picture is a surface that always displays something other than what it is…
* * *
By gradual stages human children begin to draw in the full meaning of the term — to draw a man or a woman, a house, a flower, or the sun in the sky. The child is still making lines on a surface that record the movements of the tool in his hand, but he is now also recording an awareness in terms of the invariants he has picked up. He delineates for himself and others something he has apprehended or experienced. The traces he leaves on the paper are not just lines, or the outlines of forms, but the distinguishing features of the environment. While drawing, he may be looking at something real, or thinking about something real, or thinking about something wholly imaginary; in any case, the invariants of his visual system are resonating. The same is true of the artist as of the child. The invariants are not abstractions or concepts. They are not knowledge; they are simply invariants…
Drawing is never copying. It is impossible to copy a piece of the environment. Only another drawing can be copied. We have been misled for too long by the fallacy that a picture is similar to what it depicts, a likeness, or an imitation of it. A picture supplies some of the information for what it depicts, but that does not imply that it is in projective correspondence with what it depicts.
* * *
If this new theory is correct, the term representation is misleading. There is no such thing as a literal re-presentation of an earlier optic array. The scene cannot be reestablished; the array cannot be reconstituted. Some of its invariants can be preserved, but that is all…
The efforts made by philosophers and psychologists to clarify what is meant by a representation have failed, it seems to me, because the concept is wrong. A picture is not an imitation of past seeing. It is not a substitute for going back and looking again. What it records, registers, or consolidates is information, not sense data.
* * *
A picture is not like perceiving. Nevertheless, a picture is somehow more like perceiving an object, place, or person than is a verbal description… The picture is both a scene and a surface, and the scene is paradoxically behind the surface. This duality of the information is the reason the observer is never quite sure how to answer the question, “What do you see?” For he can perfectly well answer that he sees a wall or a piece of paper. It is this duality in the optic array from a picture that makes the drawing a bad way to begin the study of perception…
A picture is both a surface in its own right and a display of information about something else. The viewer cannot help but see both, yet this is a paradox, for the two kinds of awareness are discrepant. We distinguish between the surface of the picture and the surfaces in the picture. In such paintings as those of the impressionists, we can see the difference between the illumination of the picture and the illumination in the picture. The two sets of surfaces are not comparable, and the two kinds of illumination are not commensurable…
* * *
Psychologists have long been showing inkblots to their subjects and asking what they saw. A set of such random blobs on cards devised by Hermann Rorschach has now been standardized and is in use by clinical psychologists. Faced with a card, a sensible patient might very well say simply that she saw a blot, but she seldom does. She attends to the nameless squiggles, contours, textures, and colors and says, “A bleeding heart” or “A pair of dancing bears,” allowing the psychologist to diagnose her fantasy life. I have argued that a Rorschach blot is a picture of sorts containing information not only for bleeding hearts and dancing bears but for dozens of other events. It is different from a regular picture in that the invariants are all mixed up together and are mutually discrepant instead of being mutually consistent or redundant. It is rather like a mass of scribbles for a child in this respect.
The old mentalistic explanation of perceiving objects in clouds and inkblots, incidentally, is projection, the projecting outward of fantasy images from the unconscious mind as if by a mental magic lantern. Hence, the Rorschach is called a “projective” test. This is mischievous nonsense. But the dogma of two different contributions to perception, one objective and one subjective, one coming from the world and the other coming from the mind, is so strong that the notion of a picture being thrown outward to mix with a picture being thrown inward is widely believed.
What are we to call the tree in the photograph, or the bleeding heart in the inkblot? Neither is an object in my terminology. I am tempted to call them virtual objects. They are not perceived, and yet they are perceived. The duality of the information in the array is what causes the dual experience… I conclude that a picture always requires two kinds of apprehension that go on at the same time, one direct and the other indirect. There is a direct perceiving of the picture surface, along with an indirect awareness of virtual surface — a perceiving, knowing, or imagining, as the case may be.
* * *
The invariants display a world with nobody in it, and the perspective displays where the observer is in that world… What is induced in these pictures is not an illusion of reality but an awareness of being in the world. This is no illusion. It is a legitimate goal of depiction, if not the only one…
Visual scientists with all their theorizing know little about the actual art of painting. A fine art should not be subject to rules and regulations. This is the attitude of many modern painters and most schools of art. The theory that artificial perspective is no more than a convention of Western art is a way of justifying this attitude. E. Panovsky (1924-1925) asserted that perspective is “symbolic.” G. Kepes (1944) has written about the “language” of vision. R. Arnheim (1954) believes that we will learn to see what is represented by abstract painters even if we now cannot. And N. Goodman (1968) in Languages of Art assumes that depiction is fundamentally description, that we learn to read a picture as we learn a language, and that linear perspective could just as well be reversed from the way we have become accustomed to interpret it.
Now it is one thing to argue that the use of perspective is not necessary for a painting, but it is quite another to say that perspective is a language. That says that both the perspective and the invariants of a picture must be analogous to words and that, just as we can learn a new vocabulary, so we can learn a new mode of perception. If a language of words can be invented such as Esperanto, why not a language of art? But the essence of a picture is just that its information is not explicit. The invariants cannot be put into words and symbols. The depiction captures an awareness without describing it. The record has not been forced into predications and propositions. There is no way of describing the awareness of being in the environment at a certain place. Novelists attempt it, of course, but they cannot put you in the picture in anything like the way the painter can.
* * *
It has been generally believed that even adults can become conscious of their visual sensations if they try. You have to take an introspective attitude, or analyze your experience into its elements, or pay attention to the data of your perception, or stare at something persistently until the meaning fades away. I once believed it myself. I suggested that the “visual field” could be attended to, as distinguished from the “visual world,” and that it was almost a flat patchwork of colors, like a painting on a plane surface facing the eye. The awareness of depth in the scene could not be wholly eliminated, I thought, but it could be reduced. The similarity to a painting could be enhanced by not rotating the head and not displacing it, by closing one eye, and by avoiding any scene with motion. I recognized even then that the normal field of view of an ocular orbit is continually changing and that an arrested pattern is exceptional.
My comparison of the visual field to a perspective painting, although guarded, now seems to me a serious mistake. No one ever saw the world as a flat patchwork of colors — no infant, no cataract patient, and not even Bishop Berkeley or Baron von Helmholtz, who believed firmly that the cues for depth were learned. The notion of a patchwork of colors comes from the art of painting, not from any unbiased description of visual experience. What one becomes aware of by holding still, closing one eye, and observing a frozen scene are not visual sensations but only the surfaces of the world that are viewed now from here. They are not flat or depthless but simply unhidden. One’s attention is called to the fact of occlusion, not to the pseudofact of the third dimension. I notice the surfaces that face me, and what I face, and thus where I am. The attitude might be called introspective or subjective, but it is actually a reciprocal, two-way attitude, not a looking inward…
The young child learning to draw has long interested both psychologists and artists. When he first draws a man or a truck or a table, I suggest, he depicts the invariants that he has learned to notice. He does not draw a patchwork perspective, for he never had the experience of a patchwork. He may not yet draw in edge perspective because he has not noticed it. Hence, he may draw a table with a rectangular top and four legs at the corners because those are the invariant features of the table he has noticed. This is a better explanation than saying he draws what he knows about the table, his concept, instead of what he sees of the table, his sensation.
2 November 2012
31 October 2012
3 October 2012
26 September 2012
24 September 2012
22 September 2012
20 September 2012
14 September 2012
8 March 2012
Recent neural imaging research has demonstrated that, when watching someone else perform a particular action, the viewer experiences neural firing patterns in the brain that are similar to those associated with actually performing the action. It has been proposed that these “mirror neurons” serve as the structural and functional underpinnings for mutual empathy, understanding, and imitation. In effect we unconsciously simulate others’ actions, and the intentions motivating those actions.
As a side benefit, mirror neural activity enables the observer to live vicariously through those they observe. It’s one reason why movies and TV are so engaging: what we watch characters doing on-screen we simulate neurally as if we ourselves were doing it.
It turns out that the mirror neurons are activated not only when watching. Reading works too.
Here’s the abstract from this 2009 article by Speer et al., informatively entitled “Reading Stories Activates Neural Representations of Visual and Motor Experiences” (emphases mine):
To understand and remember stories, readers integrate their knowledge of the world with information in the text. Here we present functional neuroimaging evidence that neural systems track changes in the situation described by a story. Different brain regions track different aspects of a story, such as a character’s physical location or current goals. Some of these regions mirror those involved when people perform, imagine, or observe similar real-world activities. These results support the view that readers understand a story by simulating the events in the story world and updating their simulation when features of that world change.
In the study, 28 native English speakers read excerpts from One Boy’s Day, a nonfiction observational account of the everyday activities of Raymond, a 7-year-old boy. In the four excerpts, ranging from 8 to 11 minutes, Raymond wakes up, plays before going to school, performs an English lesson at school, and participates in a music lesson. The text was displayed on an LCD screen. The experimental subjects were hooked up to a functional MRI machine, which recorded their brain activity while reading. And it turned out that the readers’ neural patterns changed simultaneously with their reading about Raymond’s activities changing. The subjects’ fMRIs lit up in the same areas of the brain that would be activated if they themselves had been performing the activities instead of textual Raymond. The study authors summarize their key findings:
These results suggest that readers dynamically activate specific visual, motor, and conceptual features of activities while reading about analogous changes in activities in the context of a narrative, while reading: Regions involved in processing goal-directed human activity, navigating spatial environments, and manually manipulating objects in the real world increased in activation at points when those specific aspects of the narrated situation were changing. For example, when readers processed changes in a character’s interactions with an object, precentral and parietal areas associated with grasping hand movements increased in activation. Previous studies of motor execution and motor imagery provide strong evidence that the portion of premotor cortex identified in this study performs computations that are specific to motor planning and execution (Ehrsson et al., 2003; Michelon, Vettel, & Zacks, 2006; Picard & Strick, 2001). These results suggest that readers use perceptual and motor representations in the process of comprehending narrated activity, and these representations are dynamically updated at points where relevant aspects of the situation are changing.
Overall, these data make a strong case for embodied theories of language comprehension, in which readers’ representations of situations described in language are constructed from basic sensory and motor representations (Barsalou, 1999; Glenberg, 1997; Zwaan, 2004). However, the use of perceptual and motor representations to guide story comprehension may be an example of a more general, fundamental principle of cognitive function. Brain regions involved in motor function are active when viewing another person execute an action (Rizzolatti & Craighero, 2004). When viewing a movie, somatosensory and motor cortices increase in activity during scenes showing close-ups of features such as hands and faces (Hasson, Nir, Levy, Fuhrmann, & Malach, 2004), and similar correspondences exist between the regions involved in perceiving and later remembering auditory and visual information (Wheeler & Buckner, 2004). Thus, the use of sensory and motor representations during story comprehension observed in the current study may reflect a more general neural mechanism for grounding cognition in real-world experiences. Language may have adopted this general mechanism over the course of human evolution to allow individuals to communicate experiences efficiently and vividly.
Now doesn’t that just set your readerly and writerly neurons aquiver?