The Creative Science Classroom

This post will be a bit theoretical, as it is lifted from a draft of my dissertation proposal but is written in journalistic style. I hope it helps you understand the multi-literacy theories behind the Cosmic Creator Challenge.

Achieving competence in any language (literacy) requires four components: fluency of listening, speaking, reading, and writing. It is more than just understanding the meaning of words, one must be able to recognize the written phonemes of a language as words with correct pronunciation and have the skills to recreate and draw out phonetic letter forms or ideograms spontaneously to form words and sentences. This is difficult enough in English with its strange spellings and grammars, but some languages are extremely challenging. Having studied Mandarin Chinese for three years including two years living in Taiwan, I achieved a passable fluency with spoken Mandarin with its vocabulary, grammar, and tonal qualities and learned enough Chinese ideograms to read common words. At my best I could read about 1500 characters, which is at about a 3^rd or 4^th grade level. I never reached full literacy, which would require learning over 10,000 characters and how to write them using a Chinese maubi (writing brush) with correct stroke order.

Writing is simply one form of media; there are many others, each with its own technical skills, design elements, and communication methods. Each one has its own form of literacy. To understand this requires going a bit deeper.

Meaning-Making Modes, Triadic Semiotics, Media Forms, and Message Communication

In 1996 the New London Group proposed a revision of our concept of literacy. Before reading or writing were invented, people conveyed meaning through speech, storytelling, song, dance, and gestures among other methods. What the New London Group suggested is that literacy should be considered broadly; there are many forms of literacy (Cazden et al., 1996; Kalantzis & Cope, 2012) and our discussions of how to promote literacy should apply to all media modes using a common metalanguage. Since that time digital media forms have proliferated and greatly expanded our methods for conveying meaning between people. New social media platforms sprout up almost daily.

As a general model of the process of communication, the diagram at right shows that meaning begins as a mental representation in the mind and memory of a message originator or sender. In Peirce’s triadic semiotic model (1931 & 1955), this part of the message is called the object or referent and can be a science fact or phenomenon (science content). A prompt or need initiates the originator to construct and send a message, which can be communicated through a variety of meaning-making and communication or representation modes including gesture, body language, song, speech, visualization, and touch. Each of these modes of communication have accompanying media forms that are used as the message conveyance or delivery system, which Peirce would define as the sign or representamen. If one uses the analogy of radio, a signal such as music or speech is applied to a carrier wave (frequency) through the heterodyning process, or modulation. The signal can be modulated by changing the frequency (FM), the amplitude (AM), or the phase (PM) of the carrier wave. That modulated signal gets transmitted through space as electromagnetic radiation (the delivery system, sign, or medium) and picked up by a receiving antenna, where the signal is stripped from the carrier wave, converted back into sound, heard through the ear (sensory organ), filtered by perceptional need, and interpreted by the listener. This is the third corner of Peirce’s triad, the meaning, concept, or interpretation in the mind of the receiver.

All communication occurs in this manner. We modulate our meanings onto a medium through its affordances and conventions, be it traditional or digital. The medium conveys the signal to the sensory organs of the receiver but must then be processed through perceptual filters to strip out the signal from the noise. Our sensory organs are constantly receiving information, but we do not perceive most of it unless it comes to our attention as being important, for whatever reason. Information is coming at you constantly, just as billboards and other media forms bombard us daily, but we have learned to filter most of it out. We always sense but we do not always perceive.

English is not nuanced enough to make fine distinctions between sensing and perceiving, but other languages are better at this. For example, Mandarin Chinese uses modifier words to make the distinction between sensing, perceiving, and understanding. One can say, “Wo ting de jyan” which means “I hear and perceive” or say, “Wo ting bu dung,” which means “I hear but do not understand” or even “Wo ting bu jyan,” which means “I was listening but did not perceive.” (Note: I am using Yale Romanization instead of mainland Pinyin as I am more accustomed to it).

Educational implications. This model of communication through media forms has important implications for education. How we choose to communicate a concept to our students must include our choice of meaning-making mode and media form with an understanding of the sensory limitations and perceptual filters of our students and the affordances and limitations of the medium. If a student doesn’t successfully interpret our message, is it because they did not or cannot hear (they weren’t listening, they are deaf or wearing their ear buds), they cannot perceive (the room is too noisy), or they cannot understand (they lack English language skills)? And how will we know if they interpret our message as we intend it in the first place? Does the interpretant truly signify the object? How will they demonstrate their understanding? Will they write it on a test, or are there other modes that are more appropriate? If the message is unsuccessful, what can we do as teachers to choose different communication modes, to provide additional modes through different media, or to improve students’ language or perceptual skills? This is what is meant by differentiation.

Student-created digital media projects allow for differentiation because students can choose their communication modalities, even blended modalities. With so many new forms of media available, we are not limited to writing our messages on a chalkboard or speaking to them verbally. The reverse is true for our students; for them to demonstrate their understanding and learning of content, they are not limited to giving oral or written reports or writing short essay answers on a test. Yet to visit some classrooms one would think we are still living in the pre-digital society of the 1950s. Nor are scientists and engineers limited to the traditional methods of communicating the results of their research through scientific conference presentations or through writing articles for journals. They frequently communicate through podcasts, YouTube videos, or other means. Yet the Next Generation Science Standards and Utah’s SEEd standards do not adequately address these alternative forms of science communication which have become popular and have useful affordances beyond mere text.

Just as learning to read is incomplete without learning to write, so are the multi-literacies of the new media modes. We can teach students how to be media consumers, or how to “read,” interpret, and analyze the various types of digital messages they receive daily, but this must also include how to create digital media for full literacy. As students gain literacy, they move from consuming educational content to creating their own messages. Learning skills for creating multiple forms of media allows students to have expanded avenues for self-expression, creativity, communication, and demonstration of concept mastery. This is just as true in STEM fields as in the arts or humanities.

As stated by the New London group:

. . . literacy pedagogy now must account for the burgeoning variety of text forms associated with information and multimedia technologies. This includes understanding and competent control of representational forms that are becoming increasingly significant in the overall communications environment, such as visual images and their relationship to the written word – for instance, visual design in desktop publishing or the interface of visual and linguistic meaning in multimedia (Cazden et al., 1996, p. 61).

Many new forms of media have developed since 1996, and to be considered literate in the new forms, one must not only understand the affordances, limitations, and semiotics of a medium but also have “competent control” over it as a media producer or author. Editing a video into a tight narrative that effectively conveys meaning is a qualitatively different skill than writing words on a page or in a word processor; developing directable, interactive 3D models in an augmented reality scene is yet another entirely different skill set. As students work through the process of media creation, they also work through multiple representation modes. They first read and/or view the content, write notes, create a script and storyboard, build the models, and create the media. At each step, they reference the object or content and reinforce their learning (Hoban, Loughran, & Nielsen, 2011).

We must consider the extent to which public education should be responsible for teaching these competencies as new forms of media increase in popularity, reach, and educational potential. If public schools are the proper venue for teaching reading and writing literacy, should they not also teach video editing, game development, 3D animation, and desktop publishing literacies? Should schools of education require pre-service teachers to master these skills so that they can more competently teach concepts to their students? According to O’Brien, Chandler, & Unsworth, there is a “pedagogical chasm” (2010, p. 34) between the literacy pedagogies of the past that are based solely on reading and writing, and the new pedagogies required for teaching the multimodal digital media literacies that are part of students’ lives today.

Affordances, Conventions, Limitations, and Semiotics of Media Forms

For a medium to communicate meaning, it must include a common set of symbols or signs that are understood in the same way by the person communicating and the person receiving the communication (Peirce, 1931 & 1955; Cazden et al., 1996; Kalantzis & Cope, 2012). This is referred to as semiotics, or evocation of meaning through symbology or connotation and includes both the vocabulary of a language and its grammar, or rules for structuring the vocabulary to provide meaning-making (O’Brien, Chandler, & Unsworth, 2010). Any form of communication produced through a medium such as an online digital video has its own shared symbology, vocabulary, and grammar for conveying meaning. For example, a YouTube video can be “shared” and is “searchable” through an algorithm. Suggestions for further videos to watch are brought up related to the video one is currently watching but also based on past viewing patterns. This feature has evolved its own semiotics, including new vocabulary and cultural concepts such as “click bait,” “falling down the rabbit hole,” and “doom-scrolling.” The plethora of new digital media forms is having a profound influence on the way in which we entertain ourselves, learn new ideas, and communicate meaning to others. They are new forms of literacy, and are having an increasing effect on education pedagogy, content creation, and delivery. The changes that have occurred in the last 25 years have far outstripped anything the New London Group anticipated in 1996.

To support the argument that all media forms have their own affordances, semiotics, limitations, and conventions, I will use the analogy of the most ancient literary medium: petroglyphs, such as the ones near the town of Castledale in central Utah called the Rochester Panel, shown at left.

Semiotics. The Fremont culture which created this panel used common symbols that obviously had meaning to them. Could the rainbow be the dome of the sky, and the fantastical beasts carved above it constellations? Is the vertical line inscribed with circles meant to represent the pathway of the sun or moon across the constellations? Whatever shared semiotics this medium conveyed have been lost to time. As shown by common symbols used throughout Europe (Von Petzinger, 2015) and other symbols used in the American southwest (DuVal, 1999), petroglyphs used a common symbology that remained remarkably constant over many generations and even across cultures.

Digital media modes likewise share symbols. A vector shape made of Bezier curves consists of curve segments, anchor points, and control lines and points shown as a recognizable series of symbols. For pixel-based programs, the symbols are paintbrush and pencil icons; the semiotics are pixels, color modes, bit-depth, resolution, and layers. As Reyna’s framework suggests (2020), learning these symbols and conventions is essential for full digital media literacy.

Affordances. Affordances are a medium’s capabilities or advantages for communication compared with other media. If we take ancient petroglyphs as an example, they have an important affordance of relative permanence, lasting for hundreds of years and fading only slowly over centuries as new desert varnish forms in the symbols. These sites represent a continual thread of communication occurring over thousands of years, with succeeding generations adding their own ideas to the conversation. This longevity is the most important affordance of petroglyphs. Each digital medium has unique affordances as well, but permanence is not one of them; they are formed from effervescent ones and zeros that are easily deleted.

Limitations. Petroglyphs as a means of communication also have limitations just as any other literacy form. Crawling underground into caves in France or hiking through the steep slot canyons of the Colorado Plateau were acts of pilgrimage. Carving through the dark desert varnish required considerable patience and time to repeatedly hammer a stone chisel into the rocks. That they were so difficult to make and view suggests these markings are not just random graffiti; there was purpose in the communication. 

For digital media, the limitations are the power and speed of the computer systems used to run the software and the imaginations of the programmers who develop it. With browser-based software now having a smaller footprint because it runs off the Internet, the power and speed of computers is no longer a limitation.

Conventions. All forms of media have their own conventions for how they are created. Petroglyphs required using a stone hammer to pound a sharp rock into the surface of sandstone to chip away the desert varnish, and the symbols had styles that changed slowly between cultures. The trapezoidal human figures of the Fremont culture are distinct from the more elongated, ghostlike figures of the Barrier Canyon culture. Ute petroglyphs feature figures on horseback, unknown to previous cultures.

For digital media, their conventions are the tools, menus, and interface elements that allow the medium to be created. Shape-based vector drawing programs have different conventions than pixel-based image programs. Students need to learn these conventions and the affordances and limitations of each medium to become effective communicators and fully literate.

Old and new media. Petroglyph panels may represent the earliest literacy or meaning making known beyond verbal forms. However, their limitations are no longer relevant since travel to such sites is no longer required and photographs allow remote viewing. New media tools can bring the entire ambience of a site to life remotely. At Clark Planetarium I will be working with the PUNCH (Polarimeter to UNify the Corona and Heliosphere) mission outreach team to develop lesson plans regarding a series of petroglyphs in Chaco Canyon, New Mexico. In one location, a petroglyph of the sun clearly shows its corona with curved lines as it would appear during an eclipse at solar maximum and is linked to the total eclipse of 1097 CE. We have photographed the entire site and turned it into a 3D model using photogrammetry and will use this model with software developed by Infiniscope for teachers to create remote online explorations of the site. In this way, both old and new media are combining into a synthesis not possible before, with an entirely new set of affordances, conventions, semiotics, and limitations.

Leading Forms of Media

The possible forms of digital media have burgeoned in the past twenty years and have directly impinged on education. The table below outlines major forms of digital media and common examples of software used with semiotics and common terms, affordances, limitations, and educational potentials. I hope this table is useful for you as you and your students plan your Cosmic Creator Challenge projects, and that it helps to see what the different types of digital media can do.

References