The Multimodal Curriculum

Investigating The Multimodal Curriculum

Adolescent Literacy

LCRT 5029 E50

Chris Thomas

The Multimodal Curriculum

CONTENTS

I. Proposal

II. Abstract

III. Literature Review

IV. Introduction

Terminology

V. Research question

VI.The importance of multimodality in education

Learning styles

VII. The Cognitive Foundation for Multimodal Learning

Plasticity

VIII. Our Personalized Brains – Creativity

IX. Perception

X. The Multimodality Enhanced World

XI. Educational Approaches to Developing Multimodal Curricula

XII. Multimodality in the Classroom

Relevance

Culture

The Personal

XIII. The Action Plan

The Multimodal Curriculum

I. PROPOSAL

My classroom based project is concerned with the use of multi-sensory media to enhance learning. This

topic is based on the following observations:

1. that students are generally bored with school;

2. that students find school irrelevant,

3. that teaching methodologies used today are the sames ones devised when the U.S. Public

school system was invented at the turn of the 19th century and are no longer appropriate for

today's learners; and

4. that today's learners learn differently than they did even as recently as a generation ago.

Additionally, I was concerned about whether male students gain more from visual representations than

females as well as how students with differing levels of prior knowledge respond to multi-sensory

media?

My essential inquiry hasn't changed significantly. It is my contention that because people learn

differently, different tools are needed to engage them. I feel this is the reason so many kids seem

disinterested in school. As I read more and more, it became clear that the problem was more serious

than I had originally thought.

We often assume that kids in middle school and high school can choose to make the effort to access the

information we're disseminating if they just understand its importance to their lives. If we see they're

not doing well, we assume its because they either don't care, don't understand the importance, or have

some learning disability.

We blame the lack of cooperation at home. Home blames the school for not having adequate teaching,

and the whole process turns into an ugly stalemate. In view of the fact that we have a national high

school drop-out rate of nearly 40%, it's time we got to the bottom of this problem.

The Multimodal Curriculum

In truth, one's learning style either fits into the verbal/linear process of schooling, or - one somehow

figures out how to make some sort of sense of it even if we're not completely comfortable with it, or -

we're excluded from learning. It became clear to me that someone who is dominantly visual-spacial,

like a large percentage of our gifted students and an equally large percentage of boys, feels like they

don't speak the same language.

Personal experience. My own background and personal educational experience feeds heavily into this

area. First, I spent fifteen years developing learning modules designed to teach highly technical

subjects to non-technical learners so they could understand the material at a level that enabled them to

render an opinion in a patent trial. During that time, I learned how to design and sequence visual

information.

As a visual-learner, for every trial – and I did hundreds of them- I had to sit down with a stack of

patents and commit hours to drawing them out so I could understand them well enough to create the

modules; so necessity was the mother of invention for me.

I also remembered struggling in school to take notes. I couldn't get the words out of my head and onto

the paper, so I made drawings and created a sort of visual shorthand for myself. It was a wonderful day

for me when I learned how to use a word processor that would go as fast as my fingers needed to in

order to get my flood of thoughts down. It was an even happier day when I saw my first media wall

with twelve screens each playing something different. I was thrilled that I could learn so much at one

time. My brain was very happy.

When, twenty years hence, I found my own high schooler didn't have the patience to sit through classes

and kept telling me she already knew the material. She was an A-B student, but she hated school. I

understood. Her senior year we let her transfer to a computer-based school that allowed her to work at

her own pace. She had no trouble focusing, getting up at 7 to drive to Westminster to the computer lab,

and finishing all her courses on time.

The Multimodal Curriculum

Discussions with her group of ten or so friends made it clear that they all felt like they were wasting

their time. One girl said, I've just spent four years studying and I don't know how to DO ANYTHING!

I repeatedly heard kids tell me that what they were being taught had nothing to do with their current

lives and they knew it wouldn't have much to do with their future lives either. Over the course of my

graduate career, I've interviewed forty or fifty kids and the consensus was the same. They're very

aware that the educational system isn't meeting their needs.

So, based on the premise that new media: computer-based learning, cell phones, iphones, smart phones,

video games, ipad, - all the digital-technologies that we live with now, have significantly changed the

way we learn, I set out to find out how. My research was primarily through scholarly publications

because I don't have a classroom or school to work with. That notwithstanding, I did draw to some

degree, as I have indicated, on prior experiences to fuel my contentions.

I felt, based on those experiences, that the answers could be found in the pacing and structure of

material, and that there had been a substantive change – an upgrading - in student's neural capabilities

for scanning, sorting, choosing, storing and assimilating required media that needed to be

acknowledged.

To that end, I researched the most current approaches to the use of media in learning and also pursued

the cognitive aspects as well. Interestingly, both topics tend to migrate toward the linguistic theory of

literacy that tells us that literacy is not simply verbal, it is predominantly non-verbal and that with the

digital and cultural stimuli of our highly visual world, the visual side is taking over.

The outcome of this journey is somewhat surprising. Because the digital revolution has placed the

control of learning squarely in the hands of the learners, they have become knowledge-makers in their

own right. In a learning system where students are researching and creating their own learning under

the mentorship and guidance of their instructors (no longer “sages on stages,”) they also learn how

they learn. Their personal learning process is much more closely tied to their experiences of their own

learning interests and needs.

The surprising part is that the next step beyond learning about how one learns through creating learning

The Multimodal Curriculum

in which one is interested, has to do with appreciating oneself as a learner, gaining confidence in ones'

thoughts, and ultimately, understanding that others have the same kinds of interior experiences. This led

me to a session on education where physicist, Dr. Murray Gell-Mann, psychologist, Daniel Siegel,

educationalist, Sir Ken Robinson, metaphysician, Eckhart Tolle and the founders of the Blue Man

School sat with the Dalai Lama discussing “Educating the Heart and Mind.”

A reasonable process in changing the educational approach would be to personalize education,

encourage and guide research in the digital world, and open new avenues to accomplishment and

employment that afford young people as much respect as if they had attended a college (which would

mean de-emphasizing the purely academic path). The outcome would be people who find greater

fulfillment in creating their own life-long pathway of learning.

The Multimodal Curriculum

II. ABSTRACT

This paper addresses a series of issues having to do with the effects of the digital age and digital

technology on both education and the students in that system.

Most educational experts understand, if not agree, that the educational system we employ now is

radically outdated and to a great degree – a nearly 40% high school drop out rate – ineffectual. This

study examines the elements that have pointed the way to a new literacy, a form of meaning-making

that offers to change the conformity of the educational system to one that ignites the creativity and

ingenuity of each individual. Substantiated by linguistic theory and birthed by a generation of personal

digital technologies, the power and relevance of personalized education is permeating the edu-sphere

from the ground up.

Research was conducted on two main topics: the use of media, and the cognitive foundations for that

use; and in three realms, personal, at the library from scholarly books, and from scholarly resources on

the internet. The results of both the education and cognitive research indicated the importance of

personalization of the learning experience to maximize engagement and the construction of substantive

bodies of knowledge in learners' minds.

The Multimodal Curriculum

III. LITERATURE REVIEW

The literature I investigated centered around the primary topic of multimodality and was supported by

studies of cognition and the aspects of technology-derived thinking that caused multimodality to

congeal and veer off the trodden path and into new territory. In particular, I looked for information

that dealt with applications for secondary and post secondary students.

My original research question specified “multisensory” media, which I quickly found was referred to in

the literature as “multimodal” methodologies, a definition that was broader and sometimes somewhat

ambiguously applied.

Most particularly, I was interested in determining the cognitive foundations for the use of multimodal

processes. I wanted to be sure there was sound documentation concerning the effect of media on

learning. And while statistical information of learning was not conclusive, the foundations for the

effect of personalized learning was across the board, unanimously endorsed by published authorities.

I learned that because of the non-linear complexity of the brain processes, the statistical analysis of

information understanding was not very relevant. While researchers understand the general processes

of the brain and can surmise information from EEG's and fMRI's, they are seeing only a slight fragment

of what is actually happening.

The doctoral thesis of Ms. Anastopoulou, entitled: “Investigating multimodal interactions for the

design of learning environments” helped define some basic terms and pointed out some pedagogical

issues having to do with the use of media.

Rudolf Arnheim's study of visual thinking, provided a profound inspiration with its ground-breaking

The Multimodal Curriculum

exploration of the connection between perception and thinking. Dr. Arnheim, Professor Emeritus at

Harvard, in his book, “Visual Thinking,” seeks to expand our understanding of cognition. This work is

a pivotal link between the traditional approach to assimilating, processing, storing, and retrieving data

and the much more complex process of seeing, understanding contextualized information, and building

knowledge structures that characterizes multimodal learning.

Dr. Dispenza's book, “Evolve Your Brain,” helped elucidate some of the physiology of the brain and

helped me understand its changeability or neuroplasticity. His writing is detailed in such a way as to

break all our preconceptions of how we think we think and his description of the flexibility of our

knowledge building, and its effect on our physiology expands our conception of the brain and its

functions. These aspects supported my contention that the brains of young people today have

neurological capabilities that others before them have not had.

The study of the brain brought me to the topic of learning styles and whether there might be a brain-

related reason that handicapped learners in our predominantly verbal education system. As a basis for

understanding these styles, I read work by Ms. Cherry, in her psychology article, “Learning Styles

Based on Jung's Theory of Personality.” It provided a comprehensive overview of learning styles as

delineated by several educational authorities.

After reading so much about brain process which makes it sounds like a big computer, it was refreshing

to read Dr. Eagleman's newest book, “Incognito.” It provides the most recent update on some of the

more esoteric aspects of the brain. He detail the latest research which not only confirms the amazing

activity of the brain that deals with our uncanny awareness and the affects of the unconscious, likely

biochemically-caused aspects of human activity. It also provides a substantial “Ah_ha! I knew it! --

factor” which allowed me to more fully appreciate the incredible capabilities of our minds.

Richard Felder, provided, in his web resource, Index of Learning Styles, an overview of several

different approaches to the issues of learning styles. He elaborated more fully on the ramifications of

these in his article, “ Understanding student differences” as he discussed the problems of adapting

The Multimodal Curriculum

engineering curriculum to meet the needs of a changing student population.

Dr. Gee is one of my favorite sources on multimodal media and its effect on learning. His insightful

understanding of digital media's importance in developing higher learning skills continues to point out

that higher thinking is much more expansive and complex than we could have imagined. He, also,

underscores the personal aspect in learning, and as a linguists and cognitive scientist, his work, “Social

linguistics and literacies: ideology in discourses,” identifies the discoveries in language and culture that

Gee finds “beautiful, exciting, and important.”

Ms.Hornberger's article, “Biliteracy, transnationalism, multimodality, and

identity:Trajectories across time and space,” provided interesting information on a broader linguistic

perspective and its affect on other cultures.

One of the linguistic gurus of multimodality, Dr. Kress's of the University of London very explicitly in

his book, “Literacy in the new media age,” redefines new literacy. In fact, this book is probably the

bible for multimodal literacy. Although sometimes difficult to understand, his linguistic approach helps

ground the personalized approach to literacy.

Mayer and Moreno always have cut and dried ways of looking at media and its affect on learning, and

while I often don't agree with them, they have to be given credit for striving to discover reasons why

learning from words and pictures makes such a difference for some people. One of the theories I don't

agree with is the whole notion of cognitive overload. As I point out, it's far too simplistic in its

understanding of brain function and fails to address the real complexity of perception involved with

seeing not only an object but the entire world surrounding the object, including changing size, relative

size, calculating motion, determining tones of lighting as the object differentiates itself from its

surroundings, and a whole host of other small but vital pieces of data. We don't see parts of an object;

we understand that even though we don't see its back surfaces, that something has dimension. Parts of

objects don't get left out of our pictures because they didn't make it through the cognitive overloading

process. I find it illogical.

The Multimodal Curriculum

Palfrey and Gasser, and their book, “Born digital,” blazed the trail for naming and understanding the

digital generation. And although, I don't subscribe to everything they say, they nonetheless, understand

the importance of the digital revolution and the profound affects it brings.

Mr. Prensky, in his article, “Digital natives, digital immigrants,” is one of the few authorities to

proclaim the immense difference of brain processes between kids who grew up immersed in digital

technologies and those who didn't. It was refreshing to have my thoughts on this confirmed.

This article, “Brain Mechanisms of Serial and Parallel Processing during Dual-Task Performance, “ by

Sigman, M, Dahaene, S., in the Journal of Neuroscience confirmed my thoughts about the processing

abilities of the brain. I had read somewhere in a long lost article about the presence of both single and

parallel processing capabilities, and this article confirmed it for me.

Last - but first in my mind is Sir Ken Robinson, an international lecturer and consultant on education.

His work is consistent, inventive, and entertaining and I am probably one of his biggest fans. His book,

“The Element” is inspiring because it shows us what learning is supposed to do for learners: provide

them with enjoyment, a fulfilling career, and an understanding of themselves and the things that are

important in their lives. His ideas that a good education creates a resilient, resourceful, and happy

contributor to the community – and that it's not so hard to do, always makes me want to help that

happen in some way.

The Multimodal Curriculum

IV. INTRODUCTION

The contention among many linguists, communications experts, sociologist, and educationalists is that

literacy can no longer be thought of in isolation from a vast array of social, technological and economic

factors [12]. Gunther Kress, Professor of Semiotics and Education at the University of London, in his

book, “Literacy in the New Media Age,” tells us “Language-as-speech” will remain the major mode of

communication; “language-as-writing” will increasingly be displaced by image in many domains of

public education, though writing will remain the preferred mode of the political and cultural elites.”

[12]

And so we find ourselves in a new place – a new frontier of literacy where we are no longer only

addressing the reading and writing needs of our students. We are inextricably involved in the new

visual ways of communicating; ways which have upended the dominance of the written word.

Literacy instructors must now bridge the gap between what has always been considered standard

literacy in American schools and the burgeoning realm of culturally-relevant, technology-stimulated,

non-academic, real-world communication. This is the space bridged by multimodal meaning-making.

In fact, according to Dr. Anastapoulou in her doctoral thesis, the very purpose of multimodality is to

“structure the interactive space between the learner and the learning task with the aid of digital

technology.”[1]

Terminology. First, a re-definition. I have used the term, “multi-sensory” to refer to the inclusion of

information that is available to us through means other than our intellect. This includes visuals, sounds,

tactile, olfactory, taste, and any experiential activities that involve “doing.”

Educational and pedagogical research, however, refers to this genre as “multimodal” although it carries

a much broader definition – a means by which humans communicate sensory information. [13]

The Multimodal Curriculum

However, in the literature, it is widely used not only in reference to sensory elements, it also refers to

the different sorts of media used to convey learning, and, occasionally, it is used in a way that makes an

oblique connection to learning styles as personal modalities.

I find “modality” makes the most sense as a generalized conveyance, a delivery mechanism for

information and meaning-making in both the physiological realms and the media realms. e.g. Vision is

the dominant modality of the brain. In other words, information and meaning are delivered to the brain

through the modality of sight. And – interactive, massive, online computer games are modalities that

cause players to engage morally-based, decision-making processes. Again, information and meaning –

in this case, contextualized by the game – are presented to the brain in order to engage the player and

cause an action by the brain.

V. RESEARCH QUESTION

My research question, “how does a multi-sensory curriculum enhance learner engagement, increase

retention, and why,” in fact opens a Pandora's box of research and technology, sociological priorities

and political realities that profoundly affect and reflect personal and global change. I shall do my best

to present these issues in the context of pedagogical concerns.

As my research evolved, my inquiry took on deeper implications and wider scope. The change from

“multi-sensory” to “multi-modal” is a substantive expansion that by definition includes all aspects of

meaning-making, from words to sounds and images, and on to motion, symbols, shapes and all the

means for understanding them.

Investigation into the learning process inevitably leads to the realms of cognition as well as to the

morass of sociological and cultural implications that are as profound as they are overwhelming. In

response, I have tried to include an overview of these looming potentialities – some of which are

already upon us – because their impact is so important.

The applications of a digitally-determined, multimodal world in today's classroom are still, to a great

degree, “dreamware.” While many instructors often include video clips and other media, it isn't done

The Multimodal Curriculum

with purpose; that is, it isn't done knowing that it will make a difference to a specific set of students.

Multimodality , as an educational tool, is expansive, but it is meant to deliver methods of understanding

to specific groups of learners. Yes, it can be fun and entertaining, but it needs to be purposeful,

otherwise its very akin to throwing the proverbial plate of spagetti on the wall.

VI. THE IMPORTANCE OF MULTIMODALITY IN EDUCATION.

Multimodality typically refers to the application of multiple learning methods – technological and

otherwise – aimed at engaging students of diverse learning styles. These methodologies don't have to

always be new or flashy. What is novel about multimodality is the unprecedented access to information

afforded by the different digital approaches and the powerful synthesis of the verbal and symbolic

worlds in support of any and all learning objectives.

The impact of digital technologies can't be overstated. It has facilitated an economy of both visual and

verbal communication that has overturned the publishing and printing industries and stimulated the

world of visual communications. Computer software, cell phones, ipads, iphones, wii, video games,

and all interactive, digital devices, have changed the way we communicate, the way we generate

images, the way we manage businesses, the way we research and access information, and it has

irrevocably changed the way we create meaning from both the verbal and the visual worlds.

The synthesis of these two formerly separate lexicons - the spoken/written and the visual/felt – has

rendered meaning-making as exponentially greater than the sum of the parts. It is a linguistic equation

to which we may never understand the full answer because all of the operatives – the parts of the

equation – are dynamic; that is, always changing.

Still, we humans have to compartmentalize. In education, the concept of learning styles is one with

which we are all familiar. Even before the research came out, we all understood that there were

fundamental differences in the way people learned. We just didn't understand the full scope and

classroom ramifications.

Learning styles. Learning styles are “characteristic cognitive, affective, and psychological behaviors

The Multimodal Curriculum

that serve as relatively stable indicators of how learners perceive, interact with, and respond to the

learning environment.”[8]

The reality of learning styles is that they are personal ways of viewing the world and as such, affect the

ways in which we take in information and make our own meaning. As most of us who have taught

know, those styles include, but are not limited to:

-Active-Reflective

-Sensing-Intuitive

-Visual-Verbal

-Sequential-Global

The Jungian version of learning types is similar and is also the basis for the Briggs-Meyers personality-

typing. These include:

-Introvert-extrovert

-Sensing-intuition

-Thinking-feeling

-Judging-perceiving [7]

The only other type that is not explicitly listed here is the kinesthetic learner which has been included

in a “visual-spacial” category. As it turns out, the inclusion of “doing activities” is one of the elements

that is most easily understood by teachers in the classroom and one which they can, and often do,

implement on their own.

Visual-spacial learning. The concept of visual-spacial learning was developed by Dr. Linda Silverman

of the Gifted Development Center in Colorado and it describes kids who think in pictures. There is

evidence that more children than we imagined think visually. [10] In fact, Einstein noted that he

thought in images and sensations and that applying words to his ideas was a secondary effort.

The application of multimodal media that is predominantly visual has obvious applications here,

however, researchers weren't ready for this next set of results.

Gender/ethnicity. Interestingly, studies conducted in the areas of gifted students revealed that 62% of a

The Multimodal Curriculum

group of 750 gifted students preferred a visual-spacial approach to learning. In another study of Navaho

students in Arizona, 71% were identified as visual-spacial. And although we're careful not to make

assumptions that apply to minorities across the board, there's something in us that nods when we hear

this.

Studies have shown that while maturity in the areas of fine motor skills and verbal acuity is

significantly advance in girls, boys' maturity in the area of visual-spacial skills is more advanced than

girls'. Alexandra Shires Golon, author of the book, “Visual-Spacial Learners,” reports that as many as

80% of the boys she teaches are visual-spacial and raises the question of whether boys might need

different kinds of learning methodologies. It's also not surprising that the vast majority of “gamers,”

kids who play video games, are boys.

The important element here is to understand the profound effect learning style has on a person's ability

to learn. Personalized meaning-making directly impacts how individual's brains construct their

information structures or “schemas”.

The schema. One of the pivotal concepts of cognitive learning theory, the schema is a knowledge base

- an abstract mental framework. Piaget believed it was the accumulated, interconnected body of

information that individuals possess as a result of their life experiences. Still used by today's

researchers, the schema includes not only educational information, but also cultural knowledge, skills

and talents, philosophical understandings, memories, strengths, weaknesses, choices, and motivations.

The cognitivist approach to learning holds that the information in an individual's schema is organized

in a way that is personally meaningful. Researchers have found that people naturally choose topics and

directions that connect with their knowledge base - the content of their schema. In this way, the schema

becomes an influential part of an individual's personality and affects the way they learn.

The human mind learns and remembers by connecting new information to old. If a person does not

have or for some reason, cannot connect with existing knowledge, then learning is extremely difficult.

Godon quotes sobering statistics:

` ”Boys represent 99% of the discipline problems, 80% of the

The Multimodal Curriculum

dropouts, and 66% of the identified learning disabled in America's

classrooms.” [10]

These numbers suggest that many male students are somehow not able to understand and connect new

information with old. Why is that?

Learning connections are difficult if existing knowledge is encoded with highly personalized contexts,

such as culturally-impacted impressions, emotional biases, sociologically-based blind spots in

understanding, or inadequate language skills. In cases like these, other methods of engagement and

meaning making need to be used. These other methods help define a multimodal approach to learning.

Multimodality in education is a solution-based approach for reaching learners of all styles. It is a fuller,

richer approach to education that goes far beyond the occasional Youtube video on a web page of text.

It is nothing less than a re-visualization of primary, secondary and college curriculum incorporating

social, cognitive, and sensory principles that are born out of an explosion in the personal media

industries resulting in a fundamental change in the way students process information.

Consensus among educational researchers indicates that learners need to be able to relate themselves

and their everyday lives to the material they're studying in order to engage. This aspect of relevance is a

powerful one that I will discuss in greater detail later on; however, from a learning perspective, the

presentation of multiple representations of the material that incorporates learner's styles, is the

foundation of multimodal learning.

VII. THE COGNITIVE FOUNDATION FOR MULTIMODAL LEARNING

or....What are they thinking?

The examination of multimodality, must include an understanding of cognitive processes. Educational

research depends more and more on cognitive analysis involved in information processing. Yet, as we

shall see, even this leading edge science is rife with contradictions.

The Multimodal Curriculum

The cognitive process used by educational researchers is a monomodial, theoretical process that

describes what we know about the storing, accessing, and connection of bits of information in short-

term and longer-term memory. Current theory describes the fairly simple retrieval and storage system

seen below. The Atkinson-Shiffrin model has been criticized for being overly simplistic; nonetheless, it

has been used as the basis for much educational research.

The Atkinson-Shiffrin model

It does; however, give us an idea of how the theory of Cognitive Load could happen,[14 ] and the

concerns researchers have about overloading the brain's processor with too much information.

Unfortunately, this implies that most of the media that bombards students is wasted, never stored in

long term memory. And because of that, it further implies that massive amounts of information

gathered in the multimodal processes, while somewhat more effective, are not earth-shaking in their

abilities to change learning outcomes.

Much of this research points to the contention that our central cognitive processor only works serially,

processing one thing at a time. Too much of certain kinds of information, such as those items that need

to be connected to exiting schemas stored in long term memory, contribute to Sweller's [14 ] concept

of “cognitive overload.” This condition is apparently caused by crowding problems resulting from

thinking, decision-making, and cueing involved with accessing and retrieving longer-term memories.

This model of the human thinking process underlies much educational research but misrepresents the

scope of our very remarkable brains, possibly skewing the outcomes.

When we study cognitive overload and discuss the serial functioning of the brain, we get the

impression that although the human sensory organism can collect unlimited amounts of data

simultaneously – through the senses, a bottle neck of sorts occurs as it is forced to cue up and be

The Multimodal Curriculum

processed in a serial fashion. The implication that only one bit or piece of information is processed at a

time causes us to wonder how evolution could have created such a marvelously complex information

collection system and constrained it to a simple, single processor.

It's further perplexing when we know that the brain is the fastest known information processor on earth,

handling an estimated 300 billion bits of information per second. In fact, comparing the brain's

processing abilities to those of a computer vastly understates both its complexity and its potential.

There is simply no know computer that can approach the processing speed of the brain, nor its ability to

receive, appropriately understand content, create understanding and meaning, and re-route information.

According to the Journal of Neuroscience, the authors of the 2008 article, “Brain Mechanisms of Serial

and Parallel Processing during dual-task Performance,”[16] determined that the brain has, in fact,

multiple methods for processing information, two of which are, put simply: 1. utilizing a complex

system of dual processing, and 2. utilizing a highly complex and task-sensitive system of timing and

delays that allow the orderly flow of different kinds of information at different times.

The result is that information from the sensory collection systems, autonomic systems, and all the

other information input systems undergo a kind of evaluation and possible re-direction prior to

processing. These different executive functions mediate to a great extent our unconscious responses to

stimuli, supporting the theory that there is a sort of hierarchical assessment and sorting that happens to

all information, not one little gate whereby bits of data are cued up for storing in short term memory.

That said, researchers also determined that there is a crowding of task-related issues tantamount to a

kind of bottleneck, but in response, the brain has a series of executive functions and other evaluative

systems that help to resolve the problem. In any case, processing delays are calculated in the hundreds

of milliseconds, as opposed to the full stoppage implied by the term 'cognitive overload.'

In a study based on precisely timed tasks utilizing EEG and fMRI imaging, research “identified

networks with qualitatively different timing properties. The results provide physiological evidence for

the coexistence of serial and parallel processes within a cognitive task. “[16 ]

The Multimodal Curriculum

The reason it's important to understand that the brain is profoundly fast and complex and sophisticated

in its ability to sort, route and process information is because some argue that the brain can't keep up

with the massive amounts of stimuli it receives in today's communication-obsessed world. It not only is

quite capable, in the case of digital natives and many digital immigrants, it – the brain – prefers it.

Plasticity. One of the most important discoveries about the brain is its plasticity. Once thought to be a

lump of cells that gradually died off until the brain was incapable of healthy functioning, we now know

that the brain is continually expanding and that it does so in response to repeated neurological

stimulation.

It is likely that those of us who find media exciting, have developed new neural pathways that process

large stimulant loads. And it is also likely that those of us who have never known anything but the

digital world, have neurological configurations that accommodate quantities of information at high

speeds.

VIII. OUR PERSONALIZED BRAINS – CREATIVITY

“In each of us there is another whom we do not know.” Carl Jung

“There's someone in my head, but it's not me.” Pink Floyd

While educational research likes to tells us that learning depends on the mere mechanics of the brain,

the process fails to address more personal topics of creativity and perception, both of which, I would

argue, are pivotal to learning and are highly personalized aspects of our minds.

According to Sir Ken Robinson, “creativity is possible whenever we are using our intelligence.”[17] It

has to do with the processing of new material. We may think that our great ideas are spontaneous, but

that's not generally the case. When we come to a moment of epiphany, we may think it's an original

thought, but the truth is that your brain has been working on this idea for a very long time and has

finally made the cerebral connections that make the difference.

We think that our thinking brain is the center of the thinking process, but it is actually dwarfed by the

The Multimodal Curriculum

huge unconscious aspect that not only keeps the whole physiological aspect of the human organism

functioning, it does the behind-the-scenes grunt work of what we think of as our mind.

The processing of new information happens in the right hemisphere.

“Initially scientists speculated that the functions handled by the right hemisphere were more creative,

intuitive, spatial, nonlinear, meaning-oriented, emotional, and abstract than the activities of the left

lobe. According to our model dual-brain processing, this is correct. When we are creative, we are

embracing novelty. When we are intuitive, we are projecting unknown possibilities. When we are

nonlinear and abstract, we are not in routine or fixed in a pattern of familiarity. When we are searching

for meaning in reference to our own identity, we are projecting new ideas in relationship to old

concepts to advance the wisdom of self.” [3]

Sir Ken Robinson, prominent educationalist suggests that we need to rethink our ideas about

intelligence. According to him, we know three things about it. First, it's diverse. Humans think about

our world in all the ways we experience it. We think of what we see, hear, feel; we think kinesthetically,

and we draw abstract conclusions about our experiences. [19]

Secondly, intelligence is dynamic. It's very interactive. Creativity, which Robinson defines as the

process of having original ideas that have value, [18 ] typically comes through the connections of

different interdisciplinary ways of seeing things.

And thirdly, intelligence is distinct. It is unlimited in that it imagines. It has the power to bring to mind

things that aren't present. Creativity is applied imagination. It is a unique representation of each of us as

individuals.

IX. PERCEPTION

“Visual perception is visual thinking.'

Arnheim, “Visual Thinking”

Vision is the primary medium of thought and indispensable for the functioning of the mind. Yet,

perception, which is closely associated with vision, has long been thought to be something separate

from thinking. It has been reduced to a vague shadow of fact, one that is heavily imprinted with

The Multimodal Curriculum

subjectivity – a questionable personalization of whatever information might have been there. In the

face of EEG's and fMRI's, it has been marginalized even further; however, perception is an integral part

of our thinking process even though much of it takes place below the level of awareness.

Productive thinking is perceptual thinking. In science as in art, a student learns the importance of his

own direct experience and response. As his experience matures, he sees more. A young artists “learns to

see” and his abilities expand as his sight improves. A more experienced doctor or carpenter “sees” more

than a novice.

What makes perception a difficult concept is that its cognitive function tends to be both personal and

relational. It is a prime provider of contextual information. It gives the mind greater options for

attaching meaning which means that perception plays a major role in multimodal learning.

X. THE MULTIMODALITY ENHANCED WORLD

“The world told is a different world to the world shown.”

G. Kress, linguist, University of London

We humans have always perceived our world in a multimodal fashion; that is, through our senses and

with the help of whatever information conveyance system was handy. However, because the written

word was such a momentous step in the evolution of intellectual growth and education, everything of

any import, since the fourteenth century, was recorded in text. It was not until the onset of the digital

world that it became clear to us that our learning had been and still is to a large degree held hostage to

the linear sequence of letters and words.

Some of the advantages of the multimodality enhanced educational world are:

1. the augmentation of real-world interactions enabled by the student's ability to physically

interact with digital, symbolic entities; e.g., learning science by manipulating animated models;

2. the ability to function in a conceptual world, such as the world of architectural design or

The Multimodal Curriculum

sustainable economics where symbolic entities have explicit corollaries in the real world;

3. creating more universally accessible links between concrete understanding and abstract

thinking;

4. facilitating the construction and negotiation of emerging ideas;

5. connecting activities with existing understanding. [1 ]

Despite considerable research and positive outcomes in multimodal research, questions linger about the

findings. The complexity of the brain is the prime culprit; however, consensus is that “learners who

have access to multiple representations enhance their comprehension, learning, memory,

communications, and inference. [14]

In research conducted in the UK for a doctoral thesis on multimodal design, the researcher utilized

three groups of students, one with very limited access to real-world modalities [digital resources] where

they only read, one with greater access where they watched someone else manipulate the media and

one with full access to real-world modalities consisting of physical manipulations, looking at the

affects of their manipulations and reading and answering questions on the subject.

The results showed that learners who had the least access to real-world modalities had the poorest

results while with the other groups, the greater their access to modalities, the better their results on the

learning tasks. [1]

XI. EDUCATIONAL APPROACHES IN DEVELOPING MULTIMODAL CURRICULA.

The development of multimodal learning material typically depends on the coordination of visuals and

text. And while some of the principles listed below seem obvious, others depend on the material and

the kind of learners. This list of general principles is from the work of Richard Mayer and Roxanne

Moreno and applies more to design form than to content.[14]

1. Multimedia Principle: Retention is improved through words and pictures rather than through words

The Multimodal Curriculum

alone.

2. Spatial Contiguity Principle: Students learn better when corresponding words and pictures are

presented near each other rather than far from each other on the page or screen.

3. Temporal Contiguity Principle: Students learn better when corresponding words and pictures are

presented simultaneously rather than successively.

4. Coherence Principle: Students learn better when extraneous words, pictures, and sounds are

excluded rather than included.

5. Modality Principle: Students learn better from animation and narration than from animation and on-

screen text.

6. Redundancy Principle: Students learn better when information is not represented in more than one

modality – redundancy interferes with learning.

7a. Individual Differences Principle: Design effects are higher for low-knowledge learners than for

high-knowledge learners.

7b. Individual Differences Principle: Design effects are higher for high-spatial learners rather than for

low-spatial learners.

7c. Direct Manipulation Principle: As the complexity of the materials increase, the impact of direct

manipulation of the learning materials (animation, pacing) on transfer also increases.

Content is king when it comes to creating lesson plans or an online course. The following elements

pertain to the structure and sequencing of the information so that it helps learners build their

information structures or schemas. In order to engage learners we need to provide them with a

conveyance they relate to. Often it isn't easy to figure out what personalized methodologies are going

to work for which students, but often, careful attention to their media habits and their conversations

The Multimodal Curriculum

will clarify those issues.

On the other hand, building content can be tricky, especially when the target includes several different

cultures. Finding some sort of cultural common denominator can help add the personalized touch that

will make the difference.

1. Contextualized content that tells the student substantial, subtextual information about the subject;

this often includes sensory information such as visuals, sounds, smells, taste, and touch.

2. Well-thought out sequencing of information that allows the learner to understand the big picture,

the over all concept, before exploring the details. This relies on an emphasis on the visual or pictorial

aspect, followed by other methodologies.

3. Familiar media that is important to the learner;

4. A pace that doesn't bore the learner;

5. Content that carries cultural and social components that make it more relevant.

Each of these areas affects students engagement and warrant further comment.

1. Contextualized information is the matrix of relational information surrounding a topic that gives

the learner more understanding or meaning. The most common example is the visual surroundings of

an object that place it in three dimensional space, such as its size relative to other known objects or its

distance based not only on size, but in color that is lighter than foreground objects.

Sight is the most detailed of our senses, and as we'll see later, a faculty without which the brain is

severely handicapped. Every second, we take in enormous amounts of visual information which we

may not notice, but which gives us a more complete picture.

Showing context increases a sense of reality. Seeing a real time animation which shows the actual time

The Multimodal Curriculum

it takes for a puff of cigarette smoke to fill the lungs and molecules of nicotine to reach nicotine

receptors in the brain - a span of five seconds - is an eye opening realization. Seeing the immense speed

with which blood cells surge through the body's veins and arteries increases our understanding of the

dynamic nature of the body.

Hearing, touching, smelling, taste are senses that also inform the body of important information. These

are just as vital as our sight to survival and to a great degree, operate below the level of our awareness.

However, in education, the use of any of these in an experiential learning situation, increases the

amount of information about a topic and reinforces the learning moment.

Sound is often considered secondary to vision and is ubiquitous in digital media. Some sounds are

immediately recognizable, such as a heart beat, and can be utilized to great effect in the development of

multimodal learning tools. Music can evoke moods and is pivotal for engaging our emotions as we

experience storytelling in movies, television, and audio books.

2. The sequencing of information is a vital element in creating learning material. In the old teaching

paradigm, information is doled out in serial fashion, generally starting from the most basic and moving

with increasing complexity to the most complicated. A multimodal approach, however, might begin

with the presentation of the full “thing” or concept, demonstrated in three dimensions and with

important, contextual information or surroundings that give it relevance. A discussion might ensue

about its different applications shown in a video summary. After everyone understands what this

“thing” means, then the exterior of the computer model dissolves away and we start to look at the

pieces that make it tick.

An important element in this presentation is the opportunity for the learner to make his own

connections to his own understanding. This understanding may relate to concrete experiences, or it

may be an intuitive understanding that hangs on an abstract understanding of something in the brain;

e.g. a student seeing a giant condenser in an oil refinery might not have a full understanding of its

purpose, but the water droplets on the outside of the cold water pipes make him recall the water

droplets on his glass of iced tea at the family picnic last Sunday. He understands “cooling.”

The Multimodal Curriculum

3. Familiar media is an ever-expanding topic; one that is easy for digital natives and somewhat

perplexing to digital immigrants. And, it will continue to be a challenge. The important thing to

remember here, is that vilifying media such as video games is to miss the boat on important learning

opportunities.

As we will see, students today have very different ways of obtaining information and they relate to

those methodologies for reasons that are hardwired into their brains. For those us who are hardwired

into books, it's important to understand that the nature of the connection is the same; just different

media for different reasons.

4. Pace is one of those reasons. The speed with which digital natives access information is a non-

negotiable requirement. Their visual and audio acuity, their attention span, the kind and size of the

informational element they're looking for - all are determined by the speed of the digital technologies

they have been born into and use every day.

5. Relevance. While audio and visual media, digital and web technologies, video games and learning-

friendly instructional design techniques are pivotal to multimodal learning, they are simply

accompanists to the looming importance of relevance in curriculum content and in the educational

process, in general. Why is this?

Whereas pedagogical techniques traditionally tend to come from the top down; that is, from academia

to the classroom; new media - especially socially-relevant media - works in the other direction,

originating with the student, or as a response to youth-focused markets. We have achieved a critical

mass of learners, even those in junior high and high school, who feel they should be able to have a say

in what is meaningful and worthwhile to learn. They want to understand why a subject is important to

their lives. This contextualizing helps them associate new material with old; that is, it helps them build

their own schemas.

This is something they do everyday with computers, cell phones, ipads, video games and other

technologies. The speedy delivery of information gives them the specific piece or module of

information they need nearly instantly, without having to sit through a class. They have redefined

The Multimodal Curriculum

learning for themselves. They have become knowledge-makers and they have taken control of their

own edification, and unless the school system can present information in an equally efficient manner,

the student won't be around to get it.

The strength and breadth of this movement has not only caught much of academia unawares, it

portends a power shift wherein the academic world is, through the sheer weight of media economics,

being altered to accommodate today's multimodal student; that is, the digital native. [15 ]

The connection between understanding and personal experience is now an inexorably part of learning

for today's digital natives. These learners now require a personal element in their education, one that is

culturally imbued and linguistically vital. This not just a sociological phenomenon. It is linguistically

accurate to understand that literacy is inexorably tied to the language and the culture of the learner.

The strength and breadth of this movement has not only caught much of academia unawares, it

portends a power shift wherein the academic world is, through the sheer weight of media economics,

being altered to accommodate today's multimodal student; that is, the digital native. [15 ]

So, our definition of multimodality now must expand to include the space where linguistic issues and

cultural issues are married, however uncomfortably, to literacy. Multimodality, then, is not just the

accommodation of styles and assorted media. It is a linguistic concept, a content-rich and culturally-

vital approach that integrates personal context with learning.

Multimodality has been defined by McGinnis et al. as the “integration of words with visual images,

sound, streamed video, and/or paralinguistic symbols,” through which meaning-making is premised on

an understanding of literacy as “the ability to write oneself into an acquired fictional narrative.” That is,

to see oneself or aspects of one's life in the subjects one studies. [11]

Educationalist, Sir Ken Robinson, author of the New York Times bestseller, “The Element,” tells us

that our languages carry our cultural attributes and that as we learn other languages, we learn new ways

of “thinking, feeling, and relating.” [18]

The Multimodal Curriculum

Dr. James Paul Gee, in his book. “Social Linguistics and Literacies: Ideology in Discourses,” refers to

this approach as “discourses,” which are ways of being “people like us.” They are ways of “being in

the world”; they are “forms of life”; they are socially-situated identities.”[9]

Therefore, as a linguistic concept, multimodality is founded on more than the idea that there are many

different types of learners and that curriculum should include nods to kids' lives. Multimodality is all

about the individual. It is a reflection of a new way of perceiving the world, of processing and of

interacting with information. It's as if, while academia was occupied with its dusty dissemination of

information, the keys to education were stolen by generations of students whose ways of learning

evolved out of the technologies that changed the thinking of mankind.

A new kind of student.

Marc Presnsky, author of Digital Natives, Digital Immigrants [16 ] exclaims loudly that “our students

have changed radically. Today's students are no longer the people our educational system was designed

to teach.” The reasons for this can certainly be traced primarily to the average student's intense

involvement in computer games, texting, email, social media on computer, television, and other forms

of digital entertainment.

But those activities, however annoying, have wrought profound changes in kids' brains. We know that

with new experiences, the brain forms new neurological pathways and so there is every reason to

believe that today's students' brains are very different than the ones we educated even a generation ago.

They think and process information differently, and their thinking patterns have also changed.

The application of multimodality requires that we view learners differently. It may be sobering to

understand that most students learn differently from their teachers. There is a diversity of learners in

every classroom; students who are compilations of learning style, language or dialect, ethnic profile,

and socio-cultural attributes. They each learn differently.

Because of this, no one methodology effectively reaches all students and as a teacher with limited time

and an insubstantial curriculum, you can't possibly accommodate all the possible learning styles at the

same time. So, for the first time in over a hundred years of public education, students and educators

The Multimodal Curriculum

now meet on a sociologically even playing field, a field where students counter traditional education

with a powerful technologically-honed, culturally and linguistically-vital environment which must be

accommodated by educators if they intend to educate.

The impact of this shift is dramatically demonstrated in the events of the Arab Spring where social

media literally stimulated a grass-roots movement that brought down governments. The shift in

education is no less profound. The multimodal media methodologies for accessing information are

controlled by young people; no longer by the traditional academic systems.

XII. MULTIMODALITY IN THE CLASSROOM

The expansive nature of multimodality and the diversity of learning opportunities make designing

classroom objectives somewhat difficult. One project, conducted in Bristol, England started with two

objectives, both of which we have discussed earlier.

The first was the inclusion of students' cultures and their linguistically diverse forms of learning.

Secondly, it was important to include as many different forms of instructional and social media as

feasible in the lesson plan. [13] These are the two foundations of any multimodal project: relevance and

media.

Next, in designing a cohesive curriculum for a diversity of learning styles, it's important to understand

that some of the pieces are already in place and incorporating newer elements will round out or

enhance the current offering. This is the synergistic combination of multimodality. The development

included four steps:

1. Overt instruction – the use of standard methods of lecture and discussion.

2. Situated practice – the inclusion of student's own life experiences as a source for connecting with

the topic and also their experiences as knowledge producers such as creators of videos.

3. Critical framing – students will discuss what they have learned in the production of their

The Multimodal Curriculum

presentation both from the subject matter and from the standpoint of the media they used.

4. Transformed practice – the result of item 3 wherein students use higher cognitive skills to reflect on

what they did, what they learned, what they learned about their process, and what they would do

differently next time.

Perhaps the scariest aspect of this kind of integration is the freedom students are given to create their

own meaning and learning. This is referred to by researchers, Cope and Kalantzis, as “agency.”

Agency is one of the roadsigns of the digital age and acknowledges that students are now the creators

of their own learning with teachers as their guides. [5]

XIII. THE ACTION PLAN

Because I don't teach, my action plan can only be theoretical. I have long purported that education

should be personalized; that children should all be on their own personalized learning plan that takes

into account not only their changing skills, but more particularly, their learning styles and personal

interests. The major antagonist to this theory is that teachers don't have the time to do this for each of

their thirty second-graders or for all of the high school students who pass through their room every day.

Yet, this is a systemic change I feel needs to happen. School districts deal with numbers...numbers of

kids, numbers of staff, and numbers of dollars, yet, the real focus has to be on each child. We have

such a limited time with each child in order to help him grow, and if we fail, we send a young person

out into the world with less than optimum skills for creating a fulfilling life for himself. We have failed

so many.

I see literacy in its many forms as pivotal to the success of students, however, I see literacy as a set of

tools that helps student decode a wide range of learning situations. Those literacy tools are personally

determined, based on language, ethnicity, and culture with the goal of learning to function in the wider

society.

The Multimodal Curriculum

Multimodality is nothing more than another literacy tool, or a series of tools, that can be specifically

applied to take a student from a limited understanding to a wider appreciation of himself and his

world. Certainly the digital world will continue to challenge us and we will always find ourselves in

this “catch-up” mode, but getting through this transition from old education to new multimodal

education will enable us to re-tool more easily in the future.

The Multimodal Curriculum

REFERENCES

1. Anastopoulou, S., Investigating multimodal interactions for the design of learning environments.

Retrieved from: [http://etheses.bham.ac.uk/18/1/Anastopoulou04PhD.pdf

2. Arnheim, R., (1969). Visual thinking, Berkley, University of California Press

3. Dispenza, J., (2007). Evolve your brain, Deerfield Beach, Health Communications, Inc.

4. Cherry, K., Learning Styles Based on Jung's Theory of Personality

Retrieved from: http://psychology.about.com/od/educationalpsychology/ss/jung-styles.htm

5. Cope, B., Kalantzis, M., New media, new learning, The international journal of learning. Retrieved

from: http://newlearningonline.com/~newlearn//wp-content/blogs.dir/35/files/2009/03/L07_8792 _

NewMediaNewLearning_final.pdf .

6. Eagleman, D., (2011) Incognito, New York, Pantheon Books

7. Felder, Richard M., Soloman, B., Index of learning styles, accessed 24 January, 2006. Retrieved

from: http://serc.carleton.edu/NAGTWorkshops/earlycareer/teaching/learningstyles.html]

8. Felder, R., Brent, R., Understanding student differences, Retrieved from

http://www4.ncsu.edu/unity/lockers/users/f/felder/public/Papers/Understanding_Differences.pdf

9. Gee, J. (1990). Social linguistics and literacies: ideology in discourses. London: Falmer Press.

The Multimodal Curriculum

10. Golon, A., (2008). Visual-spacial learners, Waco, Prufrock Press, Inc.

11. Hornberger, Nancy H., Biliteracy, transnationalism, multimodality, and

identity:Trajectories across time and space, University of Pennsylvania,

nancyh@gse.upenn.edu; http://repository.upenn.edu/cgi/viewcontent.cgi?

article=1150&context=gse_pubs

12. Kress, G. (2003). Literacy in the new media age. London: Routledge.

13. Mathewman, S., What does multimodality mean for English? Creative tensions in teaching new

texts and new literacies. Retrieved from: http://www.interactiveeducation.ac.uk/out_mat.pdf

14. Mayer, R.E., Moreno, R. Nine ways to reduce cognitive load in multimedia learning. Retrieved

from: http://edtechnewsletter.wikispaces.umb.edu/

file/view/Cognitive+load+in+Multimedia+learning.pdf

15. Palfrey, J., Gasser, U., (2008) Born digital, New York, Basic Books.

16. Prensky, M, Digital natives, digital immigrants, From On the Horizon (MCB University Press, Vol.

9 No. 5, October 2001 Retrieved from: http://www.marcprensky.com/writing/ prensky%20-

%20digital%20natives,%20digital%20immigrants%20-%20part1.pdf

17. Sigman, M, Dahaene, S., Brain Mechanisms of Serial and Parallel Processing during Dual-Task

Performance, Journal of Neuroscience. http://www.jneurosci.org/content/28/30/7585.full

18. Robinson, K., (2011). Out of our minds – learning to be creative. United Kingdom, Capstone

Publishing.

The Multimodal Curriculum

19. Robinson, K., The World Peace Conference - Educating of the heart and mind, retreived from:

http://www.youtube.com/watch?v=suojNzKZ8ew