How we focus attention in picture viewing, picture description, and during mental imagery
Författare
Redaktör
- Klaus Sachs-Hombach
- Rainer Totzke
Summary, in English
We cannot directly uncover the contents of our mind but we can come closer to cognitive processes via overt manifestations. I am interested in how speakers perceive, conceptualise and spontaneously describe complex pictures on higher levels of discourse and I use eye tracking methodology along with verbal protocols and a multimodal scoring method to study these mental processes.
In the studies on picture viewing and picture description (Holsanova 2001, 2008), the combination of visual and verbal data revealed how picture objects are focused on and conceptualised at different levels of specificity and how objects’ location, activities and attributes are evaluated. We can witness a process of stepwise specification, evaluation, interpretation and re-conceptualisation of picture elements and of the picture as a whole. During their successive picture discovery, viewers describe not only scene-inherent objects with spatial proximity but also clustered elements distributed across the scene and create new mental groupings based on abstract concepts. The process of mental zooming in and out is documented, where concrete objects are re-fixated and viewed with another concept in mind. In sum, the comparison of visual and verbal foci in the process of picture viewing and picture description shows us the ways information is acquired and processed in the human mind.
As the studies on mental imagery suggest (Johansson 2006 et al., Holsanova et al., forthc.), inner visualisations and mental images are important for speakers who describe pictures and visual scenes. When analysing attentional shifts in mental image inspection, a significant similarity was found between the eye movement patterns during picture viewing and those produced during picture description (when the picture was removed and the informants were looking at a white screen). The eye movements closely reflected the content and the spatial relations of the original picture, suggesting that the informants created a sort of mental image as an aid for their descriptions from memory. Eye movements were thus not dependent on a present visual scene but on a mental record of the scene. In addition, even verbal scene descriptions evoked mental images and elicited eye movements that reflect spatiality. In sum, picture viewing, picture description and mental imagery can be used as windows to the mind showing how language and vision, in concert, elucidate covert mental processes.
In the studies on picture viewing and picture description (Holsanova 2001, 2008), the combination of visual and verbal data revealed how picture objects are focused on and conceptualised at different levels of specificity and how objects’ location, activities and attributes are evaluated. We can witness a process of stepwise specification, evaluation, interpretation and re-conceptualisation of picture elements and of the picture as a whole. During their successive picture discovery, viewers describe not only scene-inherent objects with spatial proximity but also clustered elements distributed across the scene and create new mental groupings based on abstract concepts. The process of mental zooming in and out is documented, where concrete objects are re-fixated and viewed with another concept in mind. In sum, the comparison of visual and verbal foci in the process of picture viewing and picture description shows us the ways information is acquired and processed in the human mind.
As the studies on mental imagery suggest (Johansson 2006 et al., Holsanova et al., forthc.), inner visualisations and mental images are important for speakers who describe pictures and visual scenes. When analysing attentional shifts in mental image inspection, a significant similarity was found between the eye movement patterns during picture viewing and those produced during picture description (when the picture was removed and the informants were looking at a white screen). The eye movements closely reflected the content and the spatial relations of the original picture, suggesting that the informants created a sort of mental image as an aid for their descriptions from memory. Eye movements were thus not dependent on a present visual scene but on a mental record of the scene. In addition, even verbal scene descriptions evoked mental images and elicited eye movements that reflect spatiality. In sum, picture viewing, picture description and mental imagery can be used as windows to the mind showing how language and vision, in concert, elucidate covert mental processes.
Avdelning/ar
Publiceringsår
2011
Språk
Engelska
Sidor
291-313
Publikation/Tidskrift/Serie
Bilder - sehen - denken : zum Verhältnis von begrifflich-philosophischen und empirisch-psychologischen Ansätzen in der bildwissenschaftlichen Forschung
Dokumenttyp
Del av eller Kapitel i bok
Förlag
von Halem
Ämne
- Philosophy
Nyckelord
- picture viewing
- eye movement protocol
- cognitive processes
- mental imagery
- verbal protocol
- picture description
Status
Published
Projekt
- Thinking in Time: Cognition, Communication and Learning
ISBN/ISSN/Övrigt
- ISBN: 978-3-86962-006-0