Northern Lights: Film and Media Studies Yearbook: Volume: 5 | Issue: 1

Northern Lights Film & Media Studies Yearbook 2007

Northern Lights: Film

& M

edia Studies Yearbook | Volume Five

ISSN 1601-829X

Volume 5

intellectwww.intellectbooks.com

Volume Five

intellect Journals | Film

Studies

Northern Lights Volume 5 – 2007

3 Introduction Arild Fetveit and Gitte Bang Stald

7–24 Mixed media: from digital aesthetics towards general communicationtheory

Klaus Bruhn Jensen

25–38 Remediation and the language of new media Jay David Bolter

39–56 Alan Kay’s universal media machine Lev Manovich

57–74 Convergence by means of globalized remediation Arild Fetveit

75–88 The website as unit of analysis? Bolter and Manovich revisited Niels Brügger

89–104 Gameplay as design: uses of computer players’ immaterial labour Adam Arvidsson and Kjetil Sandvik

105–118 On transdiegetic sounds in computer games Kristine Jørgensen

119–140 Power and personality: politicians on the World Wide Web Ib Bondebjerg

141–158 Online debate on digital aesthetics and communication Lev Manovich, Jay David Bolter, Klaus Bruhn Jensen, Arild Fetveit and Gitte

Bang Stald

159 Contributors

9 771601 829000

ISSN 1601-829X0 5

Northern Lights cover.indd 1Northern Lights cover.indd 1 8/22/07 4:51:16 PM8/22/07 4:51:16 PM

Northern Lights: Film and Media Studies YearbookVolume 5, 2007

Digital Aesthetics and Communication

Northern Lights: Film and Media Studies Yearbook is a peer-reviewed internationalyearbook started in 2002 and dedicated to studies of film and media. Each yearbookis devoted to a specific theme. In addition, every volume may include articles on othertopics as well as review articles. The yearbook wants to further interdisciplinarystudies of media with a special emphasis on film, television and new media. Since theyearbook was founded in Scandinavia, the editors feel a special obligation towardsScandinavian and European perspectives. But in a global media world it is importantto have a global perspective on media culture. The yearbook is therefore open to allrelevant aspects of film and media culture: we want to publish articles of excellentquality that are worth reading and have direct relevance for both academics in thebroad, interdisciplinary field of media studies in both humanities and social sciencesand for students in that area. But we also want to appeal to a broader public interestedin thorough and well-written articles on film and other media.

Editorial Board:Ib Bondebjerg, Editor-in-chief (Department of Media, Cognition and

Communication, Section of Film and Media Studies, University of Copenhagen),e-mail: [email protected]

Torben Kragh Grodal (Department of Media, Cognition and Communication,University of Copenhagen, Section of Film and Media Studies)

Stig Hjarvard (Department of Media, Cognition and Communication, University ofCopenhagen, Section of Film and Media Studies)

Anne Jerslev (Department of Media, Cognition and Communication, University ofCopenhagen, Section of Film and Media Studies)

Gunhild Agger (Department of Communication, University of Aalborg)Jens Hoff (Department of Political Science, University of Copenhagen)

Corresponding Editors:Daniel Biltereyst (Ghent University, Belgium), Edward Branigan (University ofCalifornia – Santa Barbara, USA), Carol Clover (University of California –Berkeley, USA), John Corner (University of Liverpool, UK), John Ellis (RoyalHolloway University of London, UK), Johan Fornäs (Linköping University,Sweden), Jostein Gripsrud (University of Bergen, Norway), Andrew Higson(University of East Anglia, UK), Mette Hjort (Lignan University, Hong Kong),Dina Iordanova (University of St Andrews, Scotland), Steve Jones (University ofIllinois, USA), Sonia Livingstone (London School of Economics, UK), UlrikeMeinhof (University of Southampton, UK), Guliano Muscio (University ofPalermo, Italy), Janet Murray (Georgia Institute of Technology, USA), HoraceNewcomb (University of Georgia, USA), Roger Odin (l’Université de Paris 3.,France), Dominique Pasquier (CEMS, France), Murray Smith (University of Kent,UK), Trine Syvertsen (University of Oslo, Norway), William Uricchio (MIT, USA),Lennart Weibull (Göteborg University, Sweden), Espen Aarseth (Copenhagen,Denmark), Eva Warth (University of Bochum, Germany)

Northern Lights is published once a year by Intellect, The Mill, ParnallRoad, Bristol, BS16 1JG, UK. The current subscription rates are £30(personal) and £140 (institutional). A postage charge of £8 is made forsubscriptions outside of Europe. Enquiries and bookings for advertisingshould be addressed to: [email protected]

© 2007 Intellect Ltd. Authorisation to photocopy items for internal orpersonal use or the internal or personal use of specific clients is grantedby Intellect Ltd to libraries and other users registered with the CopyrightLicensing Agency (CLA) in the UK or the Copyright Clearance Center (CCC)Transactional Reporting Service in the USA, provided that the base fee ispaid directly to the relevant organisation.

Journal EditorsIb BondebjergDepartment of Media, Cognition andCommunicationSection of Film and Media StudiesUniversity of CopenhagenNjalsgade 80, 2300 Copenhagen SPhone: +45 35328102Fax: +45 35328110Mobile: +4524421168Email: [email protected]: www.mef.ku.dk

Guest EditorsArild Fetveit, Department of Media,Cognition and Communication,University of Copenhagen,Email: [email protected]

Gitte Bang Stand, IT-university ofCopenhagen,Email: [email protected]

ISSN 1601-829X

Printed and bound in Great Britainby 4edge Ltd. Hockley.www.4edge.co.uk

Northern Lights.qxd 3/9/07 10:47 am Page 1

Notes for Contributors

Editorial processAll articles submitted for NL must be originalworks not published or considered for publicationelsewhere. The journal is a refereed, internationaljournal, and the editors and two anonymousreferees will evaluate all articles submitted for thejournal. Anonymity is also accorded to authors.

FormatArticles must not exceed 8000 words (50,000characters, including space), including notes andreferences – but introduction, keywords, abstractnot included.

Author-name, Institutional affiliation, address, ande-mail of the author(s) on a separate title page only.

Author-CV: On same page: short cv of author, max150 words

All articles should be made in Word. Font: TimesNew Roman size 12.

Top of article: authors name in italics.Before article: short introduction, in italics, max.75 words.

Keywords – six words, or two-word phrases, thatare at the core of what is being discussed. There isa serious reduction in an article’s ability to besearched for if the keywords are missing.

Insert abstract after notes and references, in italics,max 150 words.

Format specificationsHeadings, Paragraphs and sectionsBold is used for title of article (bold, size 14). Boldis also used for headings (size 12) in the article. Bysub-headings, use italics (size 12). If further levelis needed, use normal (size 12).

A new paragraph is indicated by a carriage returnand one tabulator indent. A new section isindicated by two carriage returns (a blank line).

OrthographyThe Journal follows standard British English. Butstandard American spelling may be used. Wordlanguage checking for UK-English or Americancan be used. Use ‘ize’ endings in stead of ‘ise’,when there is an option for that.

ReferencesAll references in the text should be according tothe Harvard system, e.g., (Bordwell, 1989: 9).Book titles are italicized, with the main wordscapitalized. The titles of articles are placed indouble quotation marks, with the main wordscapitalized, e.g., Gunning introduces these ideas inan article from 1983, “An Unseen EnergySwallows Space.” See also the sample referencesbelow.

Works mentionedTitles of films, TV-program, literary works etc.must be italicized. Works like this must befollowed by year. Original title in other languagethan English must be given, title in English afteryear in italics, if original title in English exists,otherwise translation to English in doublequotation marks, e.g. Italiensk for begyndere(2000, Italian for Beginners), or Barnet (1940,“The Child”).

QuotationsNL’s style for quotations embedded into aparagraph is single quote marks, with double quotemarks for a second quotation contained within thefirst. All long quotations (i.e. over four lines or 40words long) should be ‘displayed’ – i.e. set into aseparate indented paragraph with an additionalone-line space above and below, and without quotemarks at the beginning or end. Brief quotationswithin the main text are indicated by doublequotation marks. Quotations of more than 50words are treated as a separate section (blank linebefore and after, no quotation marks, no indent).

‘Scare quotes,’ highlighting or questioning the use ofa term, are indicated by single quotation marks, alsowithin an actual quotation, e.g: As Bordwell states,“To speak of ‘interpretation’ invites misunderstandingfrom the outset” (Bordwell 1989: 1).

Punctuation marks should always be placed withinquotation marks.

All omissions in a quotation are indicated thus: (...)

Italics may be used (sparingly) to indicate keyconcepts.

Images, Tables and DiagramsAll illustrations, photographs, diagrams, maps, etc.should follow the same numerical sequence and beshown as Figure 1, Figure 2, etc. The source has tobe indicated below. Copyright clearance should beindicated by the contributor and is always theresponsibility of the contributor. When they are ona separate sheet or file, an indication must be givenas to where they should be placed in the text.Reproduction will be in greyscale (sometimesreferred to as ‘black-and-white’). If you aresupplying any article images as hard copy, theseshould be prints between 10–20 cms wide ifpossible, and preferably greyscale if beingsubmitted as illustrations for articles. However,colour prints, transparencies and small images canbe submitted if you need to supply these.Photocopies are never advisable, but may be okayfor diagrams. They are never acceptable forphotographs. Line drawings, maps, diagrams, etc.should be crisp, clear and in a camera-ready state,capable of scanning and reduction. Although notideal, slides are certainly acceptable.

If images are supplied electronically, all imagesneed to have a resolution of at least 12 dpm (dotsper millimetre) – or 300 dpi (dots per inch). Thefigure showing the number of pixels across thewidth of the image, a figure independent ofmillimetres, centimetres or inches, is reached bymultiplying the width of the image in millimetresrequired for reproduction in the journal by 12, or ininches by 300. This is the actual informationavailable that allows the production team to offsetresolution (dpm or dpi) against width.

Images sent in as e-mail attachments should begreyscale to save time uploading and downloading.Tables should be supplied either within the Worddocument of the main text or as separate Worddocuments. These can then be extracted andreproduced. Reproducing text within imagessupplied separately is difficult: they need a highfinal resolution – around 48 dpm. An additionalAcrobat PDF document is encouraged. The PDF isa good proof copy that can also be used forreproduction if the table is exactly as it should be,but if editing is necessary, this can be done inWord if there is a small spelling error or if astatistical error is identified later. Diagrams aredifficult to construct in Word. Diagrams are bestconstructed in an object-oriented computerprogram rather than a text-oriented one. Diagramscan be supplied to us as JPEG, TIFF or AcrobatPDF documents. If a mistake is identified in adiagram, make the amendments and re-supply.

Bullets and numbered listsNL prefer that you use bullet points when listing isnecessary. If a numbered list is used they should beformatted as 1. 2. 3. Etc.

NotesNotes may be used for comments and additionalinformation only. Do not use footnotes for simplereference-purposes. Use the Word-program forfootnotes, and please do not use endnotes. Notesshould be used only in very special cases and onlyas footnotes. Footnotes must not exceed 30 words.

Dates 21 March 1978 1970s, 1980s 1964–67; 1897–1901 nineteenth century, twentieth century,

twenty-first century

Numbersone to twenty (words); 21–99 (figures); 100, 200 thirty, forty, fifty (if expressed as an

approximation) 15 years old 3 per cent, 4.7 per cent, 10 per cent,

25 per cent pp. 10–19, 19–21; 102–07, 347–49 16mm, 35mm

Abbreviations ibid., op. cit., Ph.D., BBC, UN, MA, PAR

(practice as research)

Foreign names Capitalized proper names of organizations,institutions, political parties, trade unions, etc.should be kept in roman type, not in italics.

Specific Names Names of art exhibitions, film festivals, etc. shouldbe in roman type enclosed in single quote marks.

ReferencesAll references are listed at the end of the article,alphabetically and beginning on a separate page. Ablank line is entered between references. Thereference list must follow the Harvard style ofreference, more specifically the APA-standard(http://www.apastyle.org) that should comply withEnd Note and other electronic standard referenceprograms. The following samples indicateconventions for the most common types ofreference:

Anon (1931). Les films de la semaine. Tribune deGenéve, p. 15 (January 28).

Cabrera, D. (1998a). Table Ronde de l’APA. LaFaute á Rousseau: ‘Le secret’, 18 (1), pp. 28-29.

Cabrera, D. (1998b). Une chambre á soi. Trafic, 26(1), 28-35.

Flitterman-Lewis, S. (1990). To Desire Differently:Feminism and the French Cinema. Urbana andChicago: University of Chicago Press.

Grande, M. (1998). Les Images non-dérivées. InFahle, O (ed.), Le Cinéma selon Gilles Deleuze.Paris: Presse de la Sorbonne Nouvelle, pp. 284-302.

Gibson, R., Nixon, P. & Ward, S. (eds.) (2003).Political Parties and the Internet: Net Gain?.London: Routledge.

Hayward, S. (1993). French National Cinema. 2ndedn. New York and Paris: Routledge.

Hottel, R. (1999). Including Ourselves: The Roleof Female Spectators in Agnés Varda’s ‘Lebonheur and L’une chante, l’autre pas. CinemaJournal, 38(2), 52-72.

Roussel, R. (1996), Locus Solus, Paris: Gallimard.(Originally published 1914).

Stroöter-Bender, J. (1995). L’Art contemporain dansles pays du ‘Tiers Monde’. (trans. O. Barlet). Paris:L’Harmattan.

Mendoza, A. (1994). Las communicaciones eningles y espanol [Communications in English andSpanish]. Madrid: Universidad de Madrid.?

(So this is for titles in other languages you want totranslate to English, and where an official Englishversion doesn’t exist, list title in roman and insquare brackets).

Website references are similar to other references.There is no need to decipher any place ofpublication or a specific publisher, but thereference must have an author, and the author mustbe referenced Harvard-style within the text. Unlikepaper references, however, web pages can change,so there needs to be a date of access as well as thefull web reference. In the list of references at theend of your article, the item should read somethinglike this:

Bondebjerg. (2005). Web Communication and thePublic Sphere in a European Perspective. Atwww.media.ku.dk, accessed February 15, 2005.


Northern Lights Volume 5 © 2007 Intellect LtdEditorial. English language. doi: 10.1386/nl.5.1.3/2

Introduction

Arild Fetveit and Gitte Bang Stald

Media proliferate and migrate across new technological devices in anongoing digital revolution. This involves changing aesthetics, refiguredcommunication and distribution patterns, challenges to copyright holders,and a new surge for audience creativity and sociality. The developmentraises a number of questions for scholars as well as for media practitionersand cultural commentators.

Are ‘new media’ new in a more fundamental way than previous media?To what extent and in which ways are media converging? What happens toother media when the computer is positioned as a metamedium, one thatcan handle and display most previous media? How does design andcreativity develop in the game industry and to what extent is user-driveninnovation becoming a factor in the assessment of productivity? Areplayers increasingly coming to produce the computer games they play, andin that case, how does this phenomenon relate to a new economic logiccharacteristic of Web 2.0? How are statesmen and -women dressing uptheir websites, and do these sites add to our democracies? And, to whatextent do classics like Remediation: Understanding New Media (1999),and The Language of New Media (2001) still hold up, and how could theybe updated to address current developments? This volume aims ataddressing such questions, and it also hopes to raise even more that can beproductive for further research in the field. But to what extent is there afield, and how should it be conceived?

From early attempts to view ‘new’ and ‘digital’ media as entirelydifferent from ‘old media’, new-media studies, to the extent that such a fieldexists, has come of age through being able to historicize and to see howearlier media forms in various ways revisit the new. Such an approach seemsproductive in a number of areas, whether it be the study of computer games,political discourse, websites, or other matters of interest in the realm ofdigital aesthetics and communication. One of the merits of an historicizingand comparative approach is also that it may gradually help overcome thetendency to treat ‘new media’ as a separate field of inquiry deserving its ownspecial methods and theories secluded from other approaches employed inthe study of aesthetics, culture, technology, and social life. Besides, as mostmedia in some way are coming to employ digital technologies in someaspect of their production, distribution and reception processes, an erosionof the concepts ‘digital media’, and ‘new media’ is about to take place. Thismay create an opening for a further integration between established researchfields and the field of ‘new media’.

Two of the classics, mentioned earlier, of the study of ‘new media’,which both incidentally use this concept in their titles, are Remediation:Understanding New Media (1999), by Jay D. Bolter and Richard Grusin,and The Language of New Media (2001), by Lev Manovich. Both books, in

NL 5 3–6 © Intellect Ltd 2007 3


their particular ways, take a historical and comparative approach, exploringcontinuities and relations between past and present rather than completebreaks. Thus, they position themselves against an early tendency to dismissthe importance of historical precedents for digital media in the interest ofemphasizing their revolutionary newness. Bolter and Grusin, as well asManovich, counter such tendencies in their own particular ways, and seekto position ‘new media’ in an aesthetic and cultural history of westernvisual culture spanning all the way back to the Renaissance, in the case ofBolter and Grusin, and with a major interest in how cinematic language iscontinued in digital media, in the case of Manovich. In the beginning of hisbook, Manovich (2001: xv) proposes an examination of the idea that‘cinematic ways of seeing the world, of structuring time, of narrating astory, of linking one experience to the next, have become the basic meansby which computer users access and interact with all cultural data’. Criticshave generally taken this to indicate that Manovich has a skewed view ofthe importance of cinema, rather than realizing the potential usefulness ofpushing such an idea for its explorative purposes, understanding that thecinematic paradigm is first of all prominent in the opening and the closingof the book, as Manovich points out in the discussion ending this volume.

Bolter and Grusin pursue a comparative perspective, not by opting forone particular medium and exploring how it informs ‘new media’, butthrough historicizing mediation across any previous divide between the artworld and the world of popular culture. Within this general perspective,they develop a tool for examining the ways in which media histories revisitnew media in the concept of ‘remediation’. Remediation entails ‘therepresentation of one medium in another’, they claim, and is ‘a definingcharacteristic of the new digital media’ (Bolter and Grusin 1999: 45).Through this definition, the concept structurally sets up a comparativeproject of assessing ways in which the past revisits the present, well awarethat this is a two-way street, so that, for example, televisual features notonly reappear in various ways in websites but television itself also adoptselements from the newer media. Bolter and Grusin further define theconcept of remediation as being based on a double logic of immediacy andhypermediacy. Where the former represents an effort to escape mediationand access reality or whatever is mediated directly, the latter represents anopposite logic. In this case, the experience of mediation is sought, andmultiplied, in ways that bring attention to the process of mediation itself.1

Bolter and Grusin’s concept also involves remediation in the sense ofrepairing, improving, and making something better.

The works of Bolter and Grusin, and that of Manovich, have beenproductive for the field, not only in offering concepts and insights, but alsoin raising issues in need of further consideration. The following volume istherefore conceived in part as a dialogue with these books, and with thework of Bolter and Manovich in general. By naming the volume ‘DigitalAesthetics and Communication’, by adding two concepts to that of thedigital, the multiplicity of the field is acknowledged. But perhaps evenmore important, a possible tension between an aesthetic approach and anapproach focusing on communication is suggested.

Invoking such a tension, Klaus Bruhn Jensen opens this volume byoffering an article in which he finds Remediation (Bolter and Grusin 1999),

4 Arild Fetveit and Gitte Bang Stald

1. However, it is notalways evident thatthe double logic ofimmediacy andhypermediacy is vitalto the wayremediation takesplace. This suggeststhat the concept ofremediation haswider relevance thanwhat the termsimmediacy andhypermediacy wouldinitially suggest.

According to theirpreface, the origin ofthe book was aseminar onmultimediacy (whichlater becamehypermediacy)offered by Grusin, inwhich Bolter visiteddiscussingimmediacy. Theconcept ofremediation was thendeveloped later. Thisdecent explains thecentral position thatimmediacy andhypermediacy havebeen granted. In fact,a theory ofremediation itselfcould work well alsoby allocating a lessprominent role tothese terms, althoughthe distinctionremains a powerfulone, as well as therelated distinctionbetween looking atand looking through(see Bolter andGrusin 1999: iii, 41).


and The Language of New Media (Manovich 2001) to represent a limitingtradition of ‘digital aesthetics’. He finds this tradition useful for researchingthe interface and assessing modes of representing reality. But, he argues, itfails to assess interactivity in terms of the different social meanings itgenerates, and to understand how digital media ‘enter into socialinteraction beyond the interface’. Bruhn Jensen proposes instead to placedigital media within the larger framework of a general communicationtheory, which allows greater attention to the social uses of media. A keychallenge for Bruhn Jensen’s paradigm, as for most paradigms addressingdigital media, is to explain how digital media are different from earliermedia. In order to address this challenge, he proposes to replace thedistinction between unmediated and mediated communication with a morerefined three-layer concept which distinguishes between media of first,second, and third degree.

Jay David Bolter is also preoccupied with historicizing digital media,but from within a computer-science perspective informed by cross-disciplinary aesthetics. His contribution reviews the way in which theconcept of remediation took shape, as well as how his and Grusin’s workrelate to that of Manovich. In his current contribution, Manovich locatesan effort to activate and use earlier medial forms directly in the statedaspirations of computer pioneers like Ivan Sutherland, Douglas Engelbart,Ted Nelson, and Alan Kay. In this context, Kay is the most interestingbecause he conceives of the computer as a metamedium capable ofhandling ‘already-existing and not-yet-invented media’. But Manovichalso emphasizes how these media get new properties with newfunctionalities in their digital remediations. Arild Fetveit also seeks toaddress this addition of properties in his interrogation of mediaconvergence. The manipulability of digital data has been taken to facilitateconvergence and to effect an erosion of differences between media. Fetveitquestions these assumptions by showing how there are considerableobstacles against such manipulability as well as against a convergence thatwill erase the differences between media, one of them being our affectionfor a variety of specific media aesthetics. He proposes a conception ofconvergence not so much as resulting from an erasure of differencesamong media, as resulting instead from an addition of new properties andaffordances that tend to be similar. This tendency, he claims, promotesconvergence by means of a globalized remediation.

The character of digital media and how they develop can also be studiedmore specifically by looking at a single phenomenon like the website. Thisis what Niels Brügger offers in an article which reviews Bolter’s andManovich’s contributions to understanding the website. He finds that animportant tension in discussions of the website revolve around whether itis embedded in a coherent structure, or subject to fragmentation. He makesa case for coherence. One of the ways in which websites have developedover the last decade is by facilitating new forms of sociality and productivecooperation. Online games have been among the major vehicles to promotesuch developments. Adam Arvidsson and Kjetil Sandvik investigate gamedesign fuelled by immaterial labour, and argue that the creative activity ofcomputer players is being put to work by the game industry. Theirargument feeds into larger issues concerning the way in which the Web is



developing, and the much talked about shift to a Web 2.0 logic, in whichusers become producers, exemplified by Wikipedia, and in part byYouTube, MySpace and other sites based on social software.

In spite of these interesting developments in the social area, there is stilla need to understand how digital media like games draw upon and employelements of previously developed audio-visual language like that of thecinema, in new and different ways. Kristine Jørgensen explores the case ofsound. She identifies what she calls a transdiegetic sound space incomputer games, sounds that neither stem from a source within the story(diegetic), nor are quite external to the story (extradiegetic). These soundsmay seem extradiegetic, but the fact that they communicate actively aboutthe story events to the player who then comes to act upon them, allowsJørgensen to label them transdiegetic.

The stakes in digital media are not merely aesthetic, social, economic ortechnological; they are also political. And this aspect is most explicitlybrought out in Ib Bondebjerg’s analysis of websites serving state leaders.As politics and politicians are increasingly remediated onto the Web, it isimportant to explore how these new media are used, and what bearing theymay have on the functioning of democracy.

The many questions concerning digital aesthetics and communicationprompted the editors to arrange a debate addressing challenges relating toWeb 2.0, the issue of remediation, how ‘old media’ now tend to fill up ‘newmedia’, and how the concept of media itself is affected by the instalment ofthe computer as a metamedium. The debate also addressed the relevance ofBolter and Grusin’s Remediation (1999) and that of Manovich’s TheLanguage of New Media (2001), especially in light of the Web 2.0paradigm in which social software gain increased prominence.

An interesting aspect in this debate proved also to be a bridging of thepossible tension between digital aesthetics and a general communicationtheory, prompted by the recent developments on the Web. The movetowards social software, Web 2.0, and the number of sites aiming togenerate sociality, as well as the social move within the art world itself,makes for a situation where researchers coming from an aesthetics and acommunication-theory paradigm are challenged to explore new forms ofsociality and communication, and where perspectives interrogatingimmaterial labour might well be considered more closely.

The dynamism in the chat format made the discussion touch on anumber of questions. We hope that the discussion will prove to be all themore valuable for inspiring future debates.

6 Arild Fetveit and Gitte Bang Stald


Northern Lights Volume 5 © 2007 Intellect LtdArticle. English language. doi: 10.1386/nl.5.1.7/1

Mixed media: from digital aestheticstowards general communication theory

Klaus Bruhn Jensen

AbstractDuring the last decade, many studies have reconsidered the definition of‘media’, frequently emphasizing how ‘new’digital media may be reproducingor reformulating ‘old’ analogue media. Through a critical examination oftwo key contributions – Bolter and Grusin (1999) on remediation andManovich (2001) on the language of new media – this article suggests thatmuch current work under a heading of ‘digital aesthetics’, approachingmedia as modes of representing reality, rather than as resources for acting inand on reality, is missing not one, but two opportunities – one of exploringinteractivity at the level of meaning as received and interpreted, the other ofspecifying how the discourses of digital media enter into social interactionbeyond the interface. Digital media should be understood in the widercontext of general communication theory, including issues of ‘mediated’ and‘unmediated’ social interaction.

IntroductionIn 1996, one of the world’s main professional organizations forcommunication research changed its name. The abbreviation, IAMCR,which used to denote the International Association for MassCommunication Research, came to refer to the International Association forMedia and Communication Research. Founded in 1957, at a time when the‘old’ mass media, with television at the forefront, were consolidatingthemselves as social institutions, the IAMCR, like communication researchat large, was coming to terms with another major shift in its object ofanalysis. New media of the digital and interactive variety had challengedthe field of research to reconsider the very definition of (mass)communication and (mass) media. During the ten years since 1996, a widevariety of studies have addressed this foundational issue, frequentlyemphasizing the question of how ‘new’ media may be reproducing orreformulating ‘old’ media. This article reviews some of the answers,identifying disciplinary as well as ideological fault lines, and proposing anagenda for continued interdisciplinary theory development.

In his important history of the idea of communication, John DurhamPeters showed how communication as a general category, including face-to-face interaction, ‘became thinkable only in the shadow of mediatedcommunication. Mass communication came first’ (Peters 1999: 6). Duringthe past few decades, the ongoing differentiation of mediated forms ofcommunication appears to have made a general category of media


KeywordsActionCommunicationtheoryDigital mediaHistory ofcommunicationModality


thinkable, as well. As in the case of communication, thereconceptualization of ‘media’ involves reconsideration, not just ofinformation and communication technologies, but of the very distinctionsand interrelations between humans, technological artefacts, and socialcontexts. The mass media, arguably, came first. At present, research isstruggling to explain what comes after mass media.

Following a brief genealogy of the concept of media, this article departsfrom two key contributions to recent media theory – Bolter and Grusin(1999) on remediation and Manovich (2001) on the language of new media– which provided some of the first comprehensive and most widelyinfluential accounts of how the discursive forms of new media differ fromthose of old media. A critical analysis of the two volumes serves to identifya premise that is commonly shared in much current work under a heading of‘digital aesthetics’, approaching media as modes of representing reality,rather than as resources for acting in and on reality. This article suggests thatsuch a premise may lead the broadly humanistic, text-oriented stream ofmedia and communication research to miss not one, but two opportunitiesin the face of new media – one of exploring interactivity at the level ofmeaning as received and interpreted, the other of specifying how thediscourses of digital media enter into social interaction beyond the interface.

The last part of this article outlines an approach to reinserting digitalaesthetics into general communication theory, drawing on a widerrepertoire of (new) media studies. First, while media show and tell, theyalso enable their users to do things in the world. All media, new and old,are vehicles of information, channels of communication, and means of bothinterpersonal and institutionally organized action. Second, no medium iscreated equal to any other in all of these respects, having been shaped in aninterplay of the modalities of human experience, the historically availabletechnologies, and the institutional conditions of communication. In order tolocate new media within contemporary culture, the final sectiondistinguishes three prototypes of media, each of which is programmable todifferent degrees and in different respects, including a very old medium –humans communicating in the flesh (Lakoff and Johnson 1999).

The means in the middleThe Oxford English Dictionary Online (OED) (accessed 5 January 2006)notes that while classical Latin ‘medium’ referred to some middle entity orstate, in post-classical Latin and in British sources from the twelfth centuryonwards, ‘medium’ and ‘media’ also came to denote the means of doingsomething. On the one hand, a medium can be understood as a more or lessincidental presence, linking natural phenomena or, for the spirituallyinclined, this world and the hereafter. On the other hand, a medium canserve as an intentional instrument of human action in a modern sense. Inthe latter respect, the OED distinguishes two conceptions – medium as anartistic modality, material, or technique; and medium as a channel of masscommunication – both of them from the mid-nineteenth century, when theidea of communication took hold (Peters 1999). By the mid-twentiethcentury, medium in the sense of ‘any physical material (as tape, disk, paper,etc.) used for recording or reproducing data, images, or sound’ becamecommon, presumably accelerated by digital media with diverse input and

8 Klaus Bruhn Jensen


output options. All three senses – mode of expression, material ofrecording, and means of transmission – can be retraced in the media-studiesliterature. In order to understand what are increasingly hybrid or mixedmedia, it is helpful to begin to unmix definitions of media.

In his Keywords (1983), Raymond Williams reminded researchers thatthe changing meanings of, for instance, ‘media’ bear witness to the culturesusing (and studying) them. Williams himself noted three senses of medium,including a middle entity and a technical means of transmission, adding‘the specialized capitalist sense’ in which it is ‘a medium for somethingelse, such as advertising’ (Williams 1983: 203). With or without the criticaltwist, the term has remained not just contested, but ambiguous. In a recentoverview, Ryan (2004: 16) noted the persistence in parallel of the two mid-nineteenth-century senses – mode of expression and means of transmission.Whereas social scientists commonly give priority to media as technologicaland institutional infrastructures (means of transmission), scholarsoriginating from the arts and humanities still tend to privilege mediadiscourses as aesthetic forms (modes of expression). Digital media provideone more opportunity for research to consider the potential of aninterdisciplinary, integrative ‘third culture’ (Brockman 1995) of mediastudies. One of the first movers behind the personal computer, Alan Kay,early on compared computing to music-making (Kay 1999: 129).

Comparing phenomena such as media is the business of scholarship.According to Beniger (1992: 35), ‘all social science research iscomparative’ because it compares across time, space, cultures, individuals– and media. Scholarly comparisons, in turn, depend on the availableconcepts and theories for the job, which vary with historical context. It wasnot until the early 1960s that ‘the media’ presented themselves as onephenomenon (Scannell 2002: 194), the elements of which called forcomparative analyses. Since the seminal contributions of MarshallMcLuhan (1962, 1964), research has been expected to account for differentmedia in terms of their distinctive and complementary contributions tocontemporary culture. Also outside academia, it is a common assumptionthat the media make up a networked cultural environment that conditionsand frames social interaction as well as individual existence. As such, themedia constitute the publicly accessible components of the contemporarycontrol society (Beniger 1986), which is increasingly dependent oninformation and communication technologies to regulate and reproduceitself. Regardless of terminology – control, information, media, or networksociety (Castells 1996) – social and cultural theory is asking how materialnetworks of communication afford and constrain imagined networks(Anderson 1991).

The material channels of communication set the terms for who knowswhat and when (Rogers 1962); the prevalent modes of expression shapehow people come to know. While research on who, what, and when in the‘social shaping and social consequences’ (Lievrouw and Livingstone 2002)of new media still predominates, the how of communication haspreoccupied a great deal of new-media theory, yielding findings with anaudience far beyond the arts and humanities, and into engineering circlesand boardrooms. The ongoing differentiation of media formats ischallenging traditional transmission models of communication – corporate



research entities can no longer depend on old-style development processes,from lab to launch, in the attempt to generate context-sensitive and, hence,viable products and services. Enter ordinary users, creative artists, anddigital aesthetics.

Remediation revisitedSituated within a historical perspective of medium theory (Meyrowitz1994), emphasizing the implications of shifting media forms for humanconsciousness and culture, the volume by Bolter and Grusin (1999) offereda vocabulary in which to examine new media discourses. CitingMcLuhan’s famous quip, that ‘the “content” of any medium is alwaysanother medium’, the authors set out to specify ‘a more complex process ofborrowing’, rejecting any ‘simple repurposing’ of one medium in another.To Bolter and Grusin, instead, ‘one medium is incorporated or representedin another’. As it turns out, this terminology provides a key to thetheoretical argument – small discursive differences make a difference, inmetatheory as in media discourse. A few lines on, representation ispreferred over incorporation in a central definition: ‘we call therepresentation of one medium in another remediation, and we will arguethat remediation is a defining characteristic of the new digital media’[original emphasis] (Bolter and Grusin 1999: 45). Whereas incorporationmight suggest functional integration, representation rather privilegesformal simulation – surface versus substance.

Remediation manifests itself, according to Bolter and Grusin, as a duallogic involving two general forms of representation, namely, immediacyand hypermediacy. Immediacy is the transparency of media as windows onthe world, informed by ‘the belief in some necessary contact point betweenthe medium and what it represents’ (Bolter and Grusin 1999: 30), andexemplified by linear perspective as well as photorealist computergraphics. Hypermediacy, in contrast, interferes with the subject’s line ofsight, as in modernist art seeking to defamiliarize the spectator’scomprehension of what is being represented, not least through the form ofthe artwork. In an art-historical perspective, the authors note, ‘the logic ofimmediacy has perhaps been dominant in Western representation’, and atthe end of the twentieth century, hypermediacy still was in a subordinateposition, even if it ‘has never been suppressed fully or for long periods oftime’ (Bolter and Grusin 1999: 34). The central accomplishment of thevolume was the application of this dual logic in a series of close analysesof new media genres and discourses – from digital photography to virtualworlds – through a non-sectarian postmodernist lens of study. Insubsequent publications, Bolter has extended some of the points to designpractices (Bolter and Gromala 2003), as well as reconceiving his ‘historyof writing’ (Bolter 1991) in a second edition with a subtitle referring to ‘theremediation of print’ (Bolter 2001).

Acknowledging that ‘the computational device’ only became a mediumwhen it acquired aesthetic forms and ‘social and cultural functions’, Bolterand Grusin (1999: 66) were early contributors to that growing body ofresearch that has challenged commercial as well as scientific hypeassuming the technological determination of culture and society, whatCarey and Quirk (1988) referred to as a fascination with the technological



sublime. As a theory for interpreting and explaining such social and culturalfunctions, however, for ‘understanding new media’ – the subtitle of thebook – Remediation presented several ambiguities.

The first issue concerns the systematics of the theoretical framework.Elaborating on the relationship between media and remediation, Bolter andGrusin (1999: 65) offer ‘this simple definition: a medium is that whichremediates’. But, media do not merely or primarily represent each other.And, if remediation is, indeed, the defining characteristic of new media, itis not clear what old media used to do. In some passages, the authors seemhard pressed to defend an immanent analysis of media representations, forexample, when they assert that ‘there is nothing prior to or outside the actof mediation’ (Bolter and Grusin 1999: 58). A few sections do considermaterial, economic, and other social aspects of media, in part to claim aparallel space for aesthetic and formal studies:

The social dimension of immediacy and hypermediacy is as important as theirformal and technical dimensions. However, there is no need to deny theimportance of the latter in order to appreciate the former, no need to reduce thetechnical and psychological dimensions to the social.

(Bolter and Grusin 1999: 73)

Still, the theory of remediation tends to choose sides, inviting an analyticalgaze at the surface of the interface, bracketing technologies, users, andsocial contexts.

Second, the place of history – the history of media, but also the historyof explanatory concepts – is in question. In support of the previousargument, that media are essentially remediators, it is said that ‘a mediumin our culture can never operate in isolation’ (Bolter and Grusin 1999: 70).Yet, the analytical examples in the volume cover much of the history ofwestern arts, raising questions of whether it might not be necessary toconsider several different kinds of remediation – discursive, technological,and institutional – in order to capture the processes by which humanexperience has been shaped and cumulated through shifting media forms.Bolter and Grusin do recognize the historical contingency of theirapproach, to such a degree, in fact, that readers may wonder what kind ofexplanatory value is being assigned to the framework. Having emphaticallysubordinated objects to representations as the field of study, the authorsnext relativize the concepts that serve as their lens of study. What remains,appears to be a set of ad hoc analytical surfaces or terms – with immediacyand hypermediacy as the central nodes – regarding the things people dowith media: ‘we see ourselves today in and through our available media’(Bolter and Grusin 1999: 231). Importantly, we are meant to includeresearchers trying to make sense of the media and signs of our times. Todayamounts to a rather brief window of opportunity through whichcontemporary media provide access to cultural history: ‘at this extendedhistorical moment, all current media function as remediators and […]remediation offers us a means of interpreting the work of earlier media aswell’ (Bolter and Grusin 1999: 55). Surely, the framework of Remediation,elaborating insights from Russian formalism onwards concerning the(de)familiarizing functions of media, has more lasting relevance; the



question is how its internalist perspective may be complemented tosubstantiate conclusions beyond the discourses of the media that are newhere and now.

A final, related ambiguity has to do with the pragmatics of remediation– what are the claims being made regarding the effects or implications ofnew media? Bolter and Grusin go on to draw quite far-reaching inferencesabout the impact of new media on users in terms of a ‘remediated self’(Bolter and Grusin 1999: 230). They further identify a ‘psychologicaleconomy of remediation’ (Bolter and Grusin 1999: 236), which is groundedin the processes and stages of Lacanian psychoanalysis. While this is inkeeping with the tradition of textual media studies spanning art history, filmtheory, and digital aesthetics, which infers from media representations toaudience responses, the line of argument appears problematic if one seeksto account for the distinctive features of specific historical media forms. Insome sections, the authors briefly consider other positions, including whatamounts to an alternative hypothesis, namely, that immediacy andhypermediacy might constitute different aspects or moments of onereception process. This is suggested by evidence presented by, for example,Messaris (1994: 73), that non-western spectators quickly learn to interpretand ‘see through’ unfamiliar, hypermediated images. The relative merits ofthis and other approaches, however, are not pursued. In an additionalreference to the psychological experiments by Reeves and Nass (1996),showing that people relate to media in the same way that they relate to otherpeople, Bolter and Grusin (1999: 58) find that this ‘supports andcomplements our contention that media and reality are inseparable’. Giventhe radically different epistemologies and methodologies of the twoapproaches, it remains to be seen in which senses media and reality mightbe inseparable.

On the dustjacket of Remediation (1999), the reader learns that thevolume challenges ‘the modernist myth of the new’ assuming that newmedia require ‘a new set of aesthetic and cultural principles’. Cover textsare not necessarily penned by authors; ‘modernist’ is a contested term. Inreference to modernity, the text nicely captures the historically reflexiveperspective of the volume on media as open-ended cultural forms. Inreference to modernism, however, the premise concerning the dual logic ofimmediacy and hypermediacy, operative since at least the Renaissance(Bolter and Grusin 1999: 21), embraces rather than challenges themodernist mainstream of contemporary art history and (digital) mediaaesthetics, seeking new insights and, perhaps, new forms of socialorganization in the cracks and crevices of aesthetic artefacts. Remediation,similarly, depends on internalist perspectives on media in order tosubstantiate conclusions about cultural history as well as audiencepsychology. New-media studies need perspectives gazing through theinterface in both directions – into machines and humans in context.

The functionalities of new mediaApproaching the machine architecture behind the computer interface,Manovich’s The Language of New Media (2001) offered another importantcontribution to new-media theory. Preparing his agenda for computeraesthetics, Manovich identifies five principles of new, digital media. First,



regardless of their immediate appearance, they are the product of numericalrepresentation or digital code. Second, new media are subject to modularityon a different scale than analogue media, being recomposable at the site ofproduction as well as in the context of use. Third, these first two principlesallow for ‘the automation of many operations’, be it ‘creation,manipulation, or access’ [emphasis added] (Manovich 2001: 32). Fourth, afurther consequence of numerical and modular computing is variability, forexample, interactivity as a form of user-driven variability. Fifth, new mediaenable cultural transcoding, or a translation back and forth between ‘acultural layer’ of familiar objects in recognizable forms and ‘a computerlayer’ processing these according to a digital common denominator(Manovich 2001: 46). To Manovich (2001: 45), this is ‘what is in my viewthe most substantial consequence of the computerization of media’. It isalso where his position has the strongest affinities to that of Bolter andGrusin: transcoding and remediation have a family resemblance, even ifthey do not share all the same theoretical ancestors. Manovich cites Bolterand Grusin approvingly when he, too, seeks to distance his position from ‘amodernist view that aims to define the essential properties of everymedium’ and from ‘old metaphors’ concerning interfaces in traditionalhuman-computer interaction research (Manovich 2001: 89).

Manovich’s argument joins two components ‘that today can be found inmost areas of new media’. On the one hand, both the Internet andcomputers as such constitute a database, ‘a collection of documents’, thathas been taken to a different, digital degree. On the other hand, access tothe database takes place through ‘a navigable space’, specified as ‘a virtualinteractive 3D space, employed in computer games, motion rides, VR,computer animation, and human-computer interfaces’ (Manovich 2001:214). One of Manovich’s main points is that display and narrative arebecoming less central in new media, compared to their role in classical artsand traditional mass media. In Manovich’s strong formulation (2001: 225),‘database and narrative are natural enemies’, even if he recognizes thatdigital narratives result from the user’s interaction with games orinteractive fiction. Perhaps database and narrative were cultural enemies insome previous media. Digital media facilitate links between databases andinterfaces, which further enable users to communicate and act.

The links between the two constituents of new media, however, areunderstood less as means of doing than as ways of showing. FromManovich’s perspective, cinema is experiencing a second coming as amodel of digital representation: ‘To summarize, the visual culture of acomputer age is cinematographic in its appearance, digital on the level ofits material, and computational (i.e., software driven) in its logic’ [originalemphasis] (Manovich 2001: 180). Even intuitively, however, it isquestionable whether cinema, in some definition, can account for the rangeof representations in computer interfaces. The GUI (graphic user interface)is clearly home to variants of cinema, television, and video; it is also a pointof access to other virtual 3D spaces. But, cinematography is hardly asufficient principle when it comes to matters of, for example, the layout ornavigation of a database.

In the last part of the volume, Manovich elaborates on his conception ofcinematography and film theory, as informed by aesthetics and semiotics.



With reference to the basic semiotic matrix of paradigms and syntagms, heargues that even if interactive interfaces present users with severalsimultaneous paradigms from which to choose, ‘the end result is a linearsequence of screens that […] unfolds along a syntagmatic dimension’. Theresulting syntagms are described, further, as a ‘language-like sequencing’which, to Manovich, suggests that new media ‘follow the dominantsemiological order of the twentieth century – that of cinema’ (Manovich2001: 232). Leaving aside the issue of whether cinema might qualify as thedominant cultural order of the last century, again it is intuitively far fromclear that the common experience of watching several screen images,interstitched by paradigmatic choices, resembles anything like cinema, ortelevision, or animation, for that matter. In theoretical terms, moreover, it isquite a stretch to batch verbal language and computer interfaces withcinema as sequential vehicles of meaning under a heading of ‘language-like’ characteristics. Especially against the background of film theory,which has had notorious struggles with the metaphor of film as language(Metz 1974), it is surprising to find the metaphor reinstated at this level ofgenerality for the field of new-media studies.

In specific analyses of interactive genres, especially games, the volumedoes recognize the various communicative interchanges linking system andusers, beyond their cinematic identity as spectators gazing at silver andother screens. In a key section examining the ways in which database andinterface map onto each other during an interchange, Manovich begins tofocus the performative aspect of using new media. Having noted thepotential conflict between efficient access to information and the users’psychological involvement, he generalizes the point in italics: ‘Along withsurface versus depth, the opposition between information and “immersion”can be thought of as a particular expression of the more general oppositioncharacteristic of new media – between action and representation’(Manovich 2001: 216). The implication seems to be that the category ofaction is associated with immersion or engagement – virtual action. Actionin the sense of interactivity with a database of content, with other users, orwith the system of communication itself, is not theorized explicitly and ona par with the other pole of ‘the more general opposition’ of new media –representation. And, the everyday actions that people perform withcomputers – from social networking and netbanking, to culturalengagement and political mobilization – fall outside the perspective of thiscinematographic theory of new media.

Compared to the approach of Bolter and Grusin, Manovich appearsrelatively more cautious in inferring from media formats to theirconsequences for users and historical contexts. Still, in addition toconceiving of cinema as the dominant cultural code of the last century, healso assumes that cinema holds the key to understanding twenty-first-century media, returning in his last chapter to André Bazin’s question,‘What is cinema?’ Manovich’s answer is that what we used to think of as‘cinema’s defining characteristics are now just default options, with manyothers available’ (Manovich 2001: 293). More ambitiously, cinema is takento provide both the default option and the source code for other options.Having reviewed how cinema was born from animation, which thenbecame marginalized, the author restates the question, ‘What is digital



cinema?’: ‘Digital cinema is a particular case of animation that uses live-action footage as one of its many elements’ [original emphasis] (Manovich2001: 302), a notion that Manovich has explored in a creative project on‘soft cinema’ (Manovich and Kratky 2005). Most important perhaps, aparticular subset of cinema is said to triumph with the computer:

One general effect of the digital revolution is that avant-garde aesthetic strategiescame to be embedded in the commands and interface metaphors of computersoftware. In short, the avant-garde became materialized in a computer […]collage reemerged as the ‘cut-and-paste’ command, the most basic operation onecan perform on digital data.

(Manovich 2001: 306f)

Modernism is back, not just as aesthetic logic, but as technological form.On the very last page of the book, Manovich adds that ‘cinema, along

with other established cultural forms, indeed becomes precisely a code. Itis now used to communicate all types of data and experiences, and itslanguage is encoded in the interfaces and defaults of software programs andin the hardware itself’ (Manovich 2001: 333). Leaving aside again thestrong and surprising claim that cinema is already encoded in the hardwareof computers, the present discussion has suggested that cinema willaccount for only certain dimensions of how digital media articulateinformation, enable communication, and facilitate action. Cinema,undoubtedly, is the source of some subset of the codes that are currentlybeing reworked in the software of digital media. Cinema may, or may not,have a language. But, it is the functionalities and practices that linkdatabases and users via interfaces that a theory of new media, above all,must account for. Manovich and Bolter and Grusin, in related ways, havebegun to explore how new media show and tell at the interface. Media alsodo things beyond the interface.

Media showing, telling, and doingThe position of digital aesthetics, as informed by cinematography and arthistory in the works of Manovich and of Bolter and Grusin, can besummarized with reference to recent interdisciplinary research that focuses,not on visuals, but on sound (Bull and Back 2003). Sound serves as areminder concerning the multimodal nature of new media and humancommunication as such. Examining sound in cinema and other screenmedia, Chion (1994) identified three modes of listening. Causal listeningseeks the source of a sound, for example, a human voice. Semantic listeninginterprets its message in terms of a code, i.e. a particular verbal statement.And, reduced listening, a term coined by Pierre Schaeffer, focuses on ‘thetraits of the sound itself, independent of its cause and of its meaning […]its own qualities of timbre and texture’ (Chion 1994: 29–31). In real-lifesettings, causal and semantic listening can be expected to predominate;people listen in order to orient themselves and understand events in context.In arts settings, and in the meta-analysis of sound by musicologists oracousticians, reduced listening is the defining practice. Digital aestheticshas given priority to reduced listening and viewing.



Over the last decade, much other research has sought to establish linksbetween the social, technological, and aesthetic aspects of new media (e.g.Bell and Kennedy 2000; Lister et al. 2003), notably studies arising from theAssociation of Internet Researchers, as reported in the Internet ResearchAnnual (2004ff). In order to advance an interdisciplinary dialogue on theseveral necessary constituents of a theory of new media, it is helpful toreturn to some of the basics of communication theory. Media are vehiclesof information; they are channels of communication; and they are means ofboth interpersonal and macrosocial action (Jensen, 2006). While all ofthese remain contested – as terms, concepts, and phenomena – togetherthey offer a set of what Blumer (1954) called ‘sensitizing concepts’ inconfiguring the domain of inquiry. The conceptual pair of ‘information’ and‘communication’, first of all, is familiar from several fields of research invarious terminological guises. Philosophy traditionally distinguishesbetween proposition and modality, i.e. a potential reference and the realitystatus being assigned to it in an assertion (Audi 1996). In structuralistliterary and film theory, enoncé covers a work as a statement or message,whereas enonciation refers to the act of enunciation (Stam et al. 1992: 105).And, in speech-act theory (Austin 1962; Searle 1969), a distinction wasintroduced between locution (propositional components), illocution (asocial act being performed, for example, a promise or a threat), andperlocution (the received implications of the act). In combination,information and communication enable socially coordinated actions – fromdiscussion and voting, to consumer purchases and investments, to politicaland aesthetic involvement.

Each of the three constitutive concepts can be exemplified withreference to sound:

• Information: Sound serves as an explicit and regularized vehicle ofdelimited items of information. This is the case in oral narratives, withfire alarms (no warning without an implied object of attention), as wellas for jingles and other ‘program music’ that seeks to generate ideas orvalues in the listener.

• Communication: Sound supports intersubjective relations ofcommunication. An oral narrative engages its listeners, young and old.A fire alarm, when activated by a person or by smoke, addresses awarning to the inhabitants of a building. And, program music produces,however tendentially and momentarily, some level of understanding andorientation in the audience.

• Action: Sound accomplishes physical as well symbolic actions, over andabove the (speech) act being performed in and of communication –sound becomes action as it is embedded in established social practicesand institutions. Storytelling is a classic part of primary socialization;fire alarms accomplish evacuations; and program music reactivatesimagined communities (Anderson 1991), ranging from nationalism toconsumerism.

Media, new and old, enable and constrain these uses, functions, orcharacteristics in different ways and shifting configurations. Informationcan be thought of as the potential articulation of insights and ideas, lending



itself to externalization and dissemination, through the modalities of humanexperience and communication technologies of human making; in moreformal terms, ‘information is data that have been organized andcommunicated’ (Porat 1977: 2). Artworks, digital and otherwise, may beunderstood as information waiting to make its mark on the world throughsome medium. Communication, next, minimally requires a mode ofexpression and a channel of transmission, as noted, both of which areprogrammable in different respects and to varying degrees. The act ofcommunication produces some more or less stable tokens to which twoparties make themselves available and, to a degree, internalize. Finally,through informational representation and communicative interaction, thecommunicators engage in action, cumulatively enacting themselves, theirsignificant others, and the social system of which both are components.

This potential widening of the field of media studies next suggests thequestion: what is not a medium? Anthropology, sociology, and otheradjoining fields note that people continuously ascribe significance tonatural objects, cultural artefacts, and social institutions. Even theboundary between ‘nature’ and ‘culture’ is negotiated from a positionwithin culture through the historically available media. As pinpointed byWatzlawick et al. (1967), humans cannot not communicate – the bodyshows itself, and it sounds. Equally, social arrangements from businesstransactions to interior decorating have, or are given, meanings.

The media that form the objects of analysis for media andcommunication research are distinguished by their ‘programmability’,being flexible resources for the articulation of information andcommunicative interaction as part of social structuration (Giddens 1984).The definition of media in terms of programmability can be specified inthree respects. First, media comprise modalities that make possible therendering of and interaction with worlds, past and present, real andimagined. Modalities amount to semiotic registers of verbal language,music, still and moving images, etc. enabling an immensely variedrepertoire of discourses and genres, and engaging the human senses inselective and culturally conventional ways. Second, media depend on amaterial substratum for articulating and presenting information, ascommonly associated with modern technologies of communication. (Thenext section considers the human body in context as a medium.) Likemodalities, technologies lend themselves to diverse aesthetic and socialadjustments – across time, space, and possible worlds. Third, mediacommunicate to, about, and on behalf of social institutions. Media andsocieties mutually shape – programme – each other in the course ofprevalent communicative and cultural practices (Meyrowitz 1994). Theagenda of new-media studies may be clarified with reference to these threeaspects, particularly how the modalities, technologies, and institutions ofdigital media relate to those of earlier (mediated) communication.

Media of three degreesThe traditional dichotomy of ‘mediated’ and ‘unmediated’ communication,as mentioned in the introduction, assumes that the human body does notqualify as a medium of contact and exchange, but somehow communicatesdirectly. As argued by Peters (1999: 264), neither messages nor people have



a simple, immediate presence in the world – even face-to-face ‘dialoguemay simply be two people taking turns broadcasting at each other’. Withthe rise of many more differentiated types of communicative interaction,the dichotomy is increasingly in question. At the present stage of research,it is helpful to distinguish conceptually and analytically between threedegrees of media (see further Jensen 2002a, 2006).

Media of the first degree can be defined, briefly, as the biologicallybased, socially formed resources that enable humans to articulate anunderstanding of reality, for a particular purpose, and to engage incommunication about it with others. The central example is verballanguage, or speech, as constitutive of oral cultures and subcultures (e.g.Scribner and Cole 1981) – additional examples include song and othermusical expression, dance, drama, painting, and creative arts generally,often relying on comparatively simple, mechanical techniques such asmusical instruments and artistic or writing utensils as necessary elements.Importantly, such media depend on the presence of the human body in localtime-space. While one might identify (spoken) language, or the humanvoice, as the medium, it seems helpful to differentiate between, forexample, speech and song as media with reference to their differentmodalities, sharing the same material substratum, but commonlyaddressing different social institutions, contexts, and practices.

Frequent references to the ongoing ‘mediatization’ of politics andculture tend to obscure the fact that embodied speech, music, and othersounds remain constitutive of everyday life. As noted by one standardtextbook of media studies (McQuail 2005: 18), the total number of face-to-face interactions that occur within the micro-coordination of daily life byfar outnumber those communicative events that are technologicallymediated. Moreover, speech became an integral part of the modern massmedia, notably radio and television, further stimulating conversationsabout and around media (Gumpert and Cathcart 1986; Scannell 1991).Indeed, Ong (1982) argued that the technological re-embedding of speechhad produced a new form of ‘secondary orality’. Speech delivers not justthe contents, but also many of the forms that have been remodelled asmedia genres – the town crier as news announcer, the court jester as talk-show host. Theorizing digital media, it is essential to consider not just thereworking of analogue into digital media, whether in the sense of‘remediation’ or ‘new languages’, but equally the human body as a sourceand medium of representation and interaction. Compared to a tendency insome cybercultural and digital aesthetics (e.g. Haraway 1997; Hayles 1999;Stone 1991) to discursify the body, it seems time for new-media studies toexamine users as historical and biological individuals, not just asabstractions and represented surfaces.

Media of the second degree come under the heading of Benjamin’stechnically reproduced and enhanced forms of representation andinteraction (1977) which support communication across space and time,irrespective of the presence and number of participants. Whereas Benjaminplaced the emphasis on photography, film, and radio, media of the seconddegree range from early modern examples including the standardizedreproduction of religious and political texts by the printing press(Eisenstein 1979), to television and video. The common features are, first,



one-to-one reproduction, storage, and presentation of a particular contentand, second, radically extended possibilities for dissemination across timeand space. In this regard, the technologies were key to a re-embedding,both of media of the first degree and of people in relation to distant others,issues, and arenas. At the same time, the specific adaptability orprogrammability of these media had important consequences for majorsocial institutions – from the Catholic Church to the nation state. And,modalities from media of the first degree were reworked – remediated: inradio talk shows, conversation took on new conventions, just as actingstyles were adapted from the theatre stage to cinema and television. (Afurther question is whether handwriting, fixing, for instance, speech andmusic in comparatively stable forms, should be understood as a separatecategory of media. In the present context, handwriting is considered withinmedia of the first degree: the production of manuscripts is embodied andlocal, laborious and error-prone, and their distribution is selective,commonly within established institutions, as supported by oralcommentary.)

Media of the third degree are the digitally processed forms ofrepresentation and interaction and, accordingly, of particular interest here.Digital technology enables reproduction and recombination of all media ofthe second degree on a single platform – computers, thus, can beunderstood as metamedia (Kay and Goldberg 1999) with an unprecedenteddegree of technical programmability, between as well as within previousmedia. The central current example is the networked personal computer,


Media offirst degree

Media ofthird degree

Media ofsecond degree

Figure 1: Media of three degrees.


although this interface, as well as that of mobile telephones, is likely tochange substantially as technologies are adapted further to the humansenses, and integrated into both common objects and social arrangements.Whereas classic mass media, such as illustrated magazines and television,combined modalities to a considerable degree, the scale and speed withwhich digitalization facilitates the incorporation and reconfiguration ofsecond-order modalities, supports the view that already the personalcomputer may represent a qualitative shift from media of the second degreethat is comparable to the shift from first-degree to second-degree media.The interrelations of digital technology and multimodality with theinstitutions of contemporary society are still in the making, withimplications to be determined through empirical research and in historicalperspective.

One characteristic of media of the third degree is their re-enactment orsimulation of face-to-face interaction. Computer networks enable forms ofinteraction that are more similar to interpersonal than to masscommunication, as exemplified by the informality of e-mail, chat, andgaming. In certain respects, humans are media; in certain respects, digitalmedia can substitute for the social roles of humans. Figure 1 seeks toillustrate the interrelations of the media of three different degrees as awheel of culture. The media types do not replace each other – theyrecirculate the forms and contents of shifting cultural traditions, and theyremain elements of the same historical media environment. They do,however, constitute different and ascending degrees of combinedprogrammability in terms of adaptable technologies, differentiatedmodalities, and institutions transcending time, space, and social actors.

While communication has always been pervasive, digital technologiesare making information and interaction more accessible and applicableacross contexts. Why communicate so much? As noted by Aristotle (Clarke1990: 11), words allow humans to consider that which is at leasttemporarily absent – in space, in time, and from one’s immediateexperience – through thought experiments and dialogue. Media canrepresent what is absent from, but imagined within, face-to-faceencounters, opening up universes of what is not yet, what might be, as wellas what ought never to come to pass. Why not communicate less? Wecannot not communicate (Watzlawick et al. 1967), because we are co-present with others in the real world, and necessarily share a culture. In adiscussion of communication and culture in relation to music, Meyer(2001: 348f) noted that we keep social complexity manageable throughculture: ‘what most significantly shaped human behaviour and gave rise tohuman cultures was not the presence, but the absence of adequate innateconstraints. It is because evolution resulted in such an animal that humancultures became indispensable.’ Culture is not icing on the layercake ofevolution and history; it is the preliminary outcome of communication inmanaging extreme social and cognitive complexities for endless practicalpurposes. We need all the media we can get, occasionally to appreciate theiraesthetics, but mostly to get by and go on.



ConclusionMedia and communication research is positioned to renew its theorydevelopment, having been challenged by digital technologies to reconsiderits core concepts of ‘media’ and ‘communication’. This article has arguedfor an inclusive agenda, incorporating interdisciplinary concepts andconcerns from several decades of humanistic as well as social-scientificresearch, as well as addressing humans as media. The traditional dividesbetween interpersonal, organizational, and mass communication studies areincreasingly counterproductive. Media content itself – from Frankenstein(1818) via Blade Runner (1982) to current massively multiplayer onlinerole-playing games (MMPORPG) – provides a cultural laboratoryregarding the status of humans and the realities about which theycommunicate. As a second-order laboratory or institution-to-think-with(Jensen 2002b), research – from hard-nosed artificial intelligence (e.g.Boden 1996) via semi-soft actor-network theory (e.g. Latour 1993) topostmodern philosophy dissolving the category of being human (e.g.Hayles 1999) – equally is at pains to define who or what communicates.

‘Mixed media’ that combine materials in more or less innovative waysare a familiar format in artistic practice and criticism. The aesthetic gaze andthe camera eye, as developed by Bolter and Grusin (1999) and by Manovich(2001), are valid perspectives on new, mixed media, as well. Appearing halfa decade after the popular breakthrough of the Internet, the two volumesoffered some of the first elaborate theories regarding digital technologies asmedia, and have contributed to digital aesthetics as a separate sub-specialityof study. In order to account for the wider implications of mixed mediatoday, however, as they reconfigure modalities, materials, as well asinstitutions, digital aesthetics need to reconsider their interfaces with otherexplanatory models. From within the art domain, the tradition ofcontemplative appreciation of media and culture has recently beencountered, for example, by Summers (2003) in a ‘post-formalist art history’,which examines the arts as thoroughly practical enterprises in a materialenvironment of really existing media and humans. The larger field of mediaand communication research itself is ripe with approaches to the texts andcontexts of new media, from their role in everyday life (Wellman andHaythornthwaite 2002) and sociocultural communities (Baym 2000) to theirplace in the infrastructures of economy and politics (Castells 1996). Inconjunction, these approaches may begin to address the key questionregarding any new medium for policy-makers, business leaders, culturalactivists, little boys and, increasingly, little girls: what does it do?

If the idea of communication has been a century and a half in the making(Peters 1999), it is not surprising that the definition of media has continuedto pose significant challenges for research since the 1960s, as restated bydigital media during the 1990s. The media of three degrees provide aframework in which to approach the distinctive affordances (Gibson 1979;Hutchby 2001) of different media, with implications for humancommunication and action over the longues durées of history. Mixed mediafill up art museums; metamedia saturate the everyday across platforms andcontexts. In order to focus historical and empirical studies of the social usesand implications of new media, further research is needed to unmixtheoretical definitions of media.



ReferencesAnderson, B. (1991), Imagined Communities: Reflections on the Origin and Spread of

Nationalism, 2nd edn., London: Verso.Audi, R. (ed.) (1996), The Cambridge Dictionary of Philosophy, Reprinted edn.,

Cambridge: Cambridge University Press.Austin, J.L. (1962), How To Do Things With Words, Oxford: Oxford University Press.Baym, N.K. (2000), Tune In, Log On: Soaps, Fandom, and Online Community,

Thousand Oaks, CA: Sage.Bell, D. and Kennedy, B.M. (eds) (2000), The Cybercultures Reader, London:

Routledge.Beniger, J. (1986) The Control Revolution, Cambridge, MA: Harvard University Press.———— (1992), ‘Comparison, yes, but – the case of technological and cultural

change’, in J.G. Blumler, J.M. McLeod and K.E. Rosengren (eds), ComparativelySpeaking: Communication and Culture Across Space and Time, Newbury Park, CA:Sage.

Benjamin, W. (1977 [1936]), ‘The work of art in the age of mechanical reproduction’,in J. Curran, M. Gurevitch and J. Woollacott (eds), Mass Communication andSociety, London: Edward Arnold.

Blumer, H. (1954), ‘What is wrong with social theory?’ American Sociological Review,19, pp. 3–10.

Boden, M.A. (ed.) (1996), Artifical Intelligence, San Diego, CA: Academic Press.Bolter, J.D. (1991), Writing Space: The Computer, Hypertext, and the History of

Writing, Hillsdale, NJ: Lawrence Erlbaum.———— (2001), Writing Space: Computers, Hypertext, and the Remediation of Print,

2nd edn., Mahwah, NJ: Lawrence Erlbaum.Bolter, J.D. and Gromala, D. (2003), Windows and Mirrors: Interaction Design, Digital

Art, and the Myth of Transparency, Cambridge, MA: MIT Press.Bolter, J.D. and Grusin, R. (1999), Remediation: Understanding New Media,

Cambridge, MA: MIT Press.Brockman, J. (ed.) (1995), The Third Culture, New York: Simon & Schuster.Bull, M. and Back, L. (eds) (2003), The Auditory Culture Reader, Oxford: Berg.Carey, J.W. and Quirk, J.J. (1988), ‘The mythos of the electronic revolution’, in J.W.

Carey (ed.), Communication as Culture, New York: Unwin Hyman.Castells, M. (1996), The Rise of the Network Society, Oxford: Blackwell.Chion, M. (1994), Audio-Vision: Sound on Screen, New York: Columbia University

Press.Clarke, D.S. (1990), Sources of Semiotic: Readings with Commentary from Antiquity to

the Present. Carbondale, IL: Southern Illinois University Press.Eisenstein, E.L. (1979), The Printing Press as an Agent of Change: Communication and

Cultural Transformation in Early-Modern Europe, Cambridge: CambridgeUniversity Press.

Gibson, J.J. (1979), The Ecological Approach to Perception, London: Houghton-Mifflin.

Giddens, A. (1984), The Constitution of Society, Berkeley, CA: University of CaliforniaPress.

Gumpert, G. and Cathcart, R. (eds) (1986), Inter/media: Interpersonal Communicationin a Media World, New York: Oxford University Press.

Haraway, D.J. (1997), Modest_Witness@Second_Millennium. Femaleman_Meets_Oncomouse: Feminism and Technoscience, New York: Routledge.



Hayles, N.K. (1999), How We Became Posthuman: Virtual Bodies in Cybernetics,Literature, and Informatics, Chicago: University of Chicago Press.

Hutchby, I. (2001), Conversation and Technology: From the Telephone to the Internet,Cambridge: Polity.

Jensen, K.B. (2002a), ‘Introduction: The state of convergence in media andcommunication research’, in K.B. Jensen (ed.), A Handbook of Media andCommunication Research: Qualitative and Quantitative Methodologies, London:Routledge.

———— (2002b), ‘The social origins and uses of media and communication research’,in K. B. Jensen (ed.), A Handbook of Media and Communication Research:Qualitative and Quantitative Methodologies, London: Routledge.

Jensen, K.B. (2006), ‘Sounding the media: An interdisciplinary review and a researchagenda for digital sound studies’, Nordicom Review, 27(2), 7-33.

Kay, A. (1999 [1984]), ‘Computer software’, in P.A. Mayer (ed.), Computer Media andCommunication: A Reader. Oxford: Oxford University Press, pp. 129-138.

Kay, A. and Goldberg, A. (1999 [1977]), ‘Personal dynamic media’, in P.A. Mayer (ed.),Computer Media and Communication: A Reader, Oxford: Oxford University Press,pp. 111–19.

Lakoff, G. and Johnson, M. (1999), Philosophy in the Flesh: The Embodied Mind andits Challenge to Western Thought, New York: Basic Books.

Latour, B. (1993), We Have Never Been Modern, Hemel Hempstead: HarvesterWheatsheaf.

Lievrouw, L. and Livingstone, S. (eds) (2002), Handbook of New Media: SocialShaping and Social Consequences, London: Sage.

Lister, M., Dovey, J., Giddings, S., Grant, I. and Kelly, K. (2003), New Media: ACritical Introduction, London: Routledge.

Manovich, L. (2001), The Language of New Media, Cambridge, MA: MIT Press.Manovich, L. and Kratky, A. (2005), Soft Cinema: Navigating the Database,

Cambridge, MA: MIT Press.McLuhan, M. (1962), The Gutenberg Galaxy, Toronto: University of Toronto Press.———— (1964), Understanding Media, New York: McGraw-Hill.McQuail, D. (2005), McQuail’s Mass Communication Theory, 5th edn., London: Sage.Messaris, P. (1994), Visual ‘Literacy’: Image, Mind, and Reality, Boulder, CO:

Westview Press.Metz, C. (1974), Language and Cinema, The Hague: Mouton.Meyer, L.B. (2001), ‘Music and emotion: Distinctions and uncertainties’, in P.N. Juslin

and J. Sloboda (eds), Music and Emotion: Theory and Research, Oxford: OxfordUniversity Press, pp. 341–60.

Meyrowitz, J. (1994), ‘Medium theory’, in D. Crowley and D. Mitchell (eds),Communication Theory Today, Cambridge: Polity Press.

Ong, W. (1982), Orality and Literacy, London: Methuen.Peters, J.D. (1999), Speaking into the Air: A History of the Idea of Communication,

Chicago: University of Chicago Press.Porat, M. (1977) The Information Economy: Definition and Measurement, Washington,

DC: Government Printing Office.Reeves, B. and Nass, C. (1996), The Media Equation: How People Treat Computers,

Television, and New Media Like Real People and Places, New York: CambridgeUniversity Press.

Rogers, E. M. (1962), The Diffusion of Innovations, Glencoe, IL: Free Press.



Ryan, M.-L. (2004), ‘Introduction’, in M.-L. Ryan (ed.), Narrrative Across Media: TheLanguages of Storytelling, Lincoln: University of Nebraska Press.

Scannell, P. (ed.) (1991), Broadcast Talk, London: Sage.———— (2002), ‘History, media, and communication’, in K.B. Jensen (ed.), A

Handbook of Media and Communication Research: Qualitative and QuantitativeMethodologies, London: Routledge.

Scribner, S. and Cole, M. (1981), The Psychology of Literacy, Cambridge, MA: HarvardUniversity Press.

Searle, J.R. (1969), Speech Acts, London: Cambridge University Press.Stam, R., Burgoyne, R. and Flitterman-Lewis, S. (1992), New Vocabularies in Film

Semiotics, London: Routledge.Stone, A.R. (1991), ‘Will the real body please stand up? Boundary stories about virtual

cultures’, in M. Benedikt (ed.), Cyberspace: First Steps, Cambridge, MA: MITPress.

Summers, D. (2003), Real Spaces: World Art History and the Rise of WesternModernism, London: Phaidon.

Watzlawick, P., Beavin, J.H. and Jackson, D.D. (1967), Pragmatics of HumanCommunication: A Study of Interactional Patterns, Pathologies, and Paradoxes,New York: Norton.

Wellman, B. and Haythornthwaite, C. (eds) (2002), The Internet in Everyday Life,Malden, MA: Blackwell.

Williams, R. (1983), Keywords, London: Fontana.




Remediation and the language of newmedia

Jay David Bolter

AbstractMany new-media enthusiasts have inherited from modernist aesthetic theorythe assumptions of essentialism and absolute originality. They assume thateach medium is constituted by a unique set of essential characteristics andthat the task of designers is to explore these characteristics by creatingartefacts that will ‘define the medium’. Bolter and Grusin’s Remediation isa study of intermedial relationships that rejects modernist aesthetics andcalls these assumptions into question. Manovich’s The Language of NewMedia does not simply accept these assumptions, but it does seek to derivenew-media artistic practice from modernism.

Remediation is a study of the relationships between ‘new media’ andtraditional media. Richard Grusin and I argued that these relationships wereoften ignored by popular new-media enthusiasts, who insisted on the utternovelty of digital technology as a medium. Their unstated assumption was(and is) that complete originality is a necessary condition for true creativity.In assuming that the goal of the artist or designer is (and should be) toreinvent the medium, new-media writers were adopting a popularizedversion of the rhetoric of high modernism. The attitude was typified by thefollowing quotation from Steven Holtzman’s Digital Mosaics (Holtzman1997), in which the author characterized repurposing (a kind ofremediation) as a passing phase in the development of a new medium:‘Repurposing is a transitional step that allows us to get a secure footing onunfamiliar terrain. But it isn’t where we’ll find the entirely new dimensionsof digital worlds. We need to transcend the old to discover completely newworlds of expression’ (Holtzman 1997: 15). It is revealing that, even in thetitle of a book on the originality of new media, Holtzman referred to theancient medium of the mosaic. Holtzman’s title was an unwittingacknowledgement that remediation is an unavoidable element in theprocess of mediation. But Holtzman was certainly not alone in themodernist assumption of utter originality, which is implied in the term ‘newmedia’ itself. By giving a name (remediation) to the process of borrowingamong media forms, we hoped to challenge this assumption and encouragereaders to examine the complex intermedial relationships of digital mediaforms to such older forms as film, television, radio and photography.

The term ‘remediation’ indicates a particular kind of intermedialrelationship, characterized by what Harold Bloom referred to long ago asthe ‘anxiety of influence’ (Bloom 1997). In Remediation, we used a


KeywordsRemediationIntermedialityEssentialismAvant-gardeModernistaestheticsNew-media theory


shorthand when we claimed that one piece remediates another or even thatone media form (computer games) remediates another (narrative film). Wewere not trying to suggest that media are autonomous agents that act oneach other or on other aspects of our mediated culture. Remediation is aprocess that is realized in and through the creative practices of individualproducers, designers, and artists. Sometimes this remediation is consciousand intended; sometimes individual designers may not acknowledge theirdependence on earlier media even to themselves. But in all cases they areengaging in a dialogue with their audience, for it is the audience who willconstruct the meaning of the remediation.

In the remediation of one form by another, there is always a combinationof homage and rivalry. A remediating form pays homage by borrowingrepresentational practices of an older one. At the same time, the newer formis trying to surpass the older one in some way, for the simple reason that itmust justify its claim on our cultural attention. There are several ways inwhich a new form might justify such a claim. For example, it might offeritself as a cheaper or more efficient alternative. However, such technicalimprovements per se do not seem to constitute new media forms; all thealphabetic soup of different wireless protocols (such as GPRS, EDGE,WCDMA, and so on) is still regarded as the same medium for mobiletelephony and data exchange. In order to constitute a new medium or asignificant new form within an existing medium, designers must produce asignificant change in representational practice with the tacit or explicitsuggestion that this change offers an experience that is more compelling,more ‘authentic’, even more ‘real’. If we look at media innovations sincephotography, we see that remediations with significant cultural impactoften claim to provide a more faithful representation of our experience ofnature or human relations.

In this discussion of the real, I am not making my own claim about theontological status of any medium, as Andre Bazin did, for example, in hisessay on photography (Bazin 1980). I wish to focus on culturalconstructions of the real – on the belief that media can achieve a status ofimmediacy. We can call this a desire for transparency, and even in oursophisticated media age this desire remains pervasive. One need only readthe film reviews in popular newspapers to understand the tenacity of anuncritical desire for transparency, when reviewers regularly complain thata character in a film never ‘comes to life’ or praise a film for putting us intouch with the true emotions of the characters.

Remediation is meant above all to describe the competition amongvarious media forms over the construction of the real. This competition isnot the only aspect of the relationships among media, and remediation isnot the only approach to examining such relationships. Remediation can beregarded as part of the larger project of intermediality, a version ofcomparative media studies that is now quite vigorous in Europe,particularly Germany, and in Canada, although not perhaps in the UnitedStates. Intermediality can encompass a broad range of relationships,although scholars who address intermediality have tended to focus onformal relationships and aesthetic effects. (This is in contrast to those whocharacterize their work as the cultural studies of visual media.) Amongmany authors in intermediality, we can name Jürgen E. Müller (1996),

26 Jay David Bolter


Yvonne Spielmann (1998 and 2005), Irina Rajewsky (2002), all of whomhave written books with this title. There is also a journal entitledIntermedialités published by the University of Montreal.

Remediation was never meant to be a purely formal concept, however,for the borrowing of representational practices always has social,ideological, and economic as well as aesthetic dimensions. With The Lordof the Rings, for example, we could explore how Peter Jackson usedvarious filmic techniques to blur the class distinctions sanctioned inTolkien’s original books, or, more generally, how Jackson used thetechniques of blockbuster film-making (and advertising) to broaden thefilm’s appeal to an audience large enough to pay for the enormous costs ofproduction – costs which of course the novels did not entail. These areaspects of remediation to which we alluded, but did not devote enoughspace in our book.

Each filmic technique (e.g. the long take or the jump cut), everyaffordance in a computer interface (e.g. clickable icons or the use of thejoystick) – every significant formal practice embedded in popular mediaartefacts contributes to a cultural construction of the real. And thisconstruction has implications for the production and the consumption ofmedia. Borrowing a technique in a new medium may change theconstruction. So for example, the long take in film was for Bazin anindication of authenticity: as a sign of an auteur, it was a technique thatsimultaneously ensured the realism and the aesthetic quality of film. Itbecame associated with directors who were regarded as artists withimportant vision. And yet in computer games the remediation of thecinematic long take is the relentless first-person point of view that hascome to be identified with a violent genre, known as the so-called ‘firstperson’ shooter. This genre is often regarded as a cultural embarrassmentand is not held to be either realistic or the work of auteurs.

In studying the remediation of the real in various media forms, Grusinand I identified two representational strategies: transparency andhypermediacy. This is obviously not a new idea with us. Many arthistorians and media theorists had presented dichotomies that influencedour thinking. Above all, we were influenced by the standard interpretationsof modernism, by Greenberg and many others, which set up a dichotomybetween the transparency of illusionistic painting and the reflexivepractices (hypermediacy) of modern art. There is also McLuhan’s muchmaligned distinction between hot and cool media (where hot media mightbe thought of as transparent). And there is Benjamin’s distinction betweenauratic and non-auratic art. With transparency and hypermediacy, wewanted to describe in particular the ways in which designers and artistsmask or acknowledge their debt to earlier media forms, as they seek tomobilize their audience’s sense of the real or the authentic.

Scholars who have picked up our dichotomy have shown more interestin the notion of hypermediacy than transparency. Ours is an age ofhypermediacy: with forms ranging from handwriting and print to film and3D games, our diverse media economy welcomes hybrids. Digitaltechnology makes it relatively easy for designers to hybridize differentmedia. Since its formation in the first part of the twentieth century, theavant-garde has usually pursued strategies of hybridity and hypermediacy,



and today even popular media forms (MTV, rock-music concerts, mobile-phone applications) often favour these strategies. Even in this era ofMySpace and YouTube, however, we should not discount the influence ofthe desire for transparency or immediacy (the erasure of the medium).

In recent work, my colleagues at Georgia Tech and I have argued for arelationship between immediacy and Benjamin’s notion of aura (Bolter,MacIntyre, Gandy and Schweitzer 2006). The desire for immediacy canalso be understood as the desire for presence. At least since Derrida,deconstructing our culture’s desire for presence has been a part of the post-structuralist and postmodern project. Theorists in these traditions tend totreat the issue as if it were settled: that we cannot attain presence throughany technology of representation. But critical theory has never had muchinfluence on either the popular imagination or the academic communityoutside of the humanities and some social sciences. The notion of presencelives on, for example, in computer science, among those specializing invirtual reality: Presence (published by MIT Press) is the name of the mostimportant journal devoted to the study of virtual reality (VR) and userperception. Many popular new-media writers still like to imagine aperfected form of VR as the ultimate medium for achieving a transparentform of art and entertainment.

The profound difference over the notion of presence – it is not an activedisagreement because the two sides do not belong to the same discoursecommunity – reminds us of the intellectual diversity among the variousgroups working in digital media. The most important division among thesegroups is the one between theorists and practitioners. Those who designand program applications and games tend to read only the most popular andaccessible media theorists. Those working in the traditional entertainmentworld (television and film) read new-media authors who offer new mediaas a ‘respectful remediation’ of film and television. Meanwhile, digitalartists give us various blends of theory (art and design theory as well ascritical theory) with practice.

There are profound differences even among new-media academics.Specialists in human–computer interaction (HCI), who are now studyingdigital mediation in the workplace and in social life, generally ground theirwork on the literature and techniques of cognitive science or the socialsciences. Communication-studies researchers have their own vast literaturewith both empirical and theoretical approaches, and often examine digitalmedia from the perspective of traditional mass media. Humanists withliterary or art history backgrounds are the ones most likely to bringpostmodern theory into the discussion of new media.

What Grusin and I have identified as the distinction betweentransparency and hypermediacy expresses itself in different ways in each ofthese communities. No author could hope to find a language that wouldspeak to them all.

The many languages of new media I would like to bring The Language of New Media into the discussion,keeping in mind the question: what community does Manovich belong toand which communities does he address? I would characterize his book(not surprisingly) as a study in remediation, because a key argument is that

28 Jay David Bolter


digital media today borrow and extend the representational practices ofavant-garde cinema of the 1920s. In his prologue, Manovich specificallyrelates the key qualities of digital media to those of cinema as practised byDziga Vertov in The Man with a Movie Camera (1929). This is exactly aremediating relationship: avant-garde cinema provides an authenticity thatManovich is seeking to appropriate for digital art. In a remediatingrelationship the new medium not only borrows from the old, but also offerssomething new, and for Manovich the element that distinguishes digitalmedia from avant-garde film (and all other media) is ‘transcoding’. As weshall see, transcoding is the process by which old media forms aretransformed into software.

When I claim that The Language of New Media conceives of digital artin important ways as a remediation of avant-garde film, I am not trying todiminish the scope of this work. With his commanding knowledge of themedia technologies and media theory in the twentieth century, Manovichhas given us the first convincing genealogy of new media. Manovichrealizes that the original avant-garde offered a highly influential definitionof the artist for the twentieth (and now the twenty-first) century: the task ofthe artist is to offer critical alternatives to the representational practices ofmainstream media. In The Language of New Media Manovich shows howthese avant-garde alternatives re-emerge in the interface, the operations,and the code of new media.

Historians of twentieth-century visual art such as Rosalind Krauss(1985) might object to this generalization of the impact and meaning of theavant-garde (and of course this is my formulation here and not necessarilyManovich’s). Hal Foster might argue that Manovich pays too little attentionto the neo-avant-garde such as Andy Warhol or the minimalists such asFrank Serra and Robert Morris (Foster 1996). For Foster, the minimalistsand other artists of the 1960s constitute a second key moment in the historyof the avant-garde’s questioning of established practice; in this period whatwas called into question was the practice of the high modernists with theirabstraction and insistence on the purity and isolation of their art. But TheLanguage of New Media is not trying to be a comprehensive history of theavant-garde in the twentieth century. Much of the art of the neo-avant-gardedoes not bear the same relation to its contemporary media technologies thatthe original avant-garde bore to film. There is one important exception,however: experimental film and video of the 1960s and 1970s, to whichManovich does not perhaps do justice. He mentions Hollis Frampton andStan Brakhage, but he has almost nothing to say about the video art of NamJune Paik, Woody and Steina Vasulka, and others (Spielmann 2005). Yetthese video artists were also exploring a technological innovation. Just asthe 1920s avant-garde was responding to narrative film of its day, videoartists were offering an alternative to broadcast television in theirs.According to Spielmann they were insisting that recorded video was itsown medium with its own representational practices, separate from those oftelevision. In his brilliant genealogy of the screen (Manovich 2001a:95–115), Manovich seems interested in almost every kind of screen (filmscreen, radar screen, computer screen) other than the video or televisionscreen. I suggest that an examination of video art of the mid-twentiethcentury as the third moment between avant-garde film of the 1920s and



digital art today would strengthen Manovich’s case. For example, thetechniques of the video artists could enrich the analysis of operations inChapter 3, as Manovich (2001a: 149–52) suggests in a short section onvideo compositing.

I would like to comment on another question raised by Manovich’sappeal to the avant-garde: the distinction between elite and popular forms,between art and entertainment. In the second half of the twentieth century,the boundary between serious art and entertainment has blurred. Highmodernists, such as Clement Greenberg, one of whose most famous essayswas entitled ‘Avant-Garde and Kitsch’, still believed in the superiority ofelite art over consumer-driven entertainment (Greenberg 1939). Even if theclaim of superiority can no longer be sustained, there is certainly asignificant gap between contemporary (digital and analogue) art that growsout of the avant-garde tradition and the diverse strands of popularentertainment. No one today could fail to see the differences inrepresentational practices and in audience reception between Manovich’sown Soft Cinema project and the traditional cinema of Spielberg or JamesCameron – anymore than anyone in the 1920s would have had difficultyappreciating the differences between Chien Andalou and Chaplin’s TheGold Rush.

Once again, these differences are operative in the digital world, in partbecause of the diverse communities pursuing the making and critiquing ofdigital artefacts. They become clear if we place The Language of NewMedia alongside a book that appeared a few years earlier and is perhapsequally influential today, Janet Murray’s Hamlet on the Holodeck (Murray1997). Although both of these books set out to describe the possibilities ofdigital technology as an expressive medium, they have few intertexts incommon. It is not surprising that as a literary critic Murray would draw onliterary models, where Manovich draws on the visual arts. But most ofMurray’s literary references are to canonical authors (Shakespeare,Dickens). The literary avant-garde does not figure prominently. Herreferences to visual media forms come mostly from popular film (e.g.Casablanca) or mainstream television (Star Trek). Indeed the ‘holodeck’ inher title refers to the VR machine in the television and film series Star Trek– a series that, were he alive today, Greenberg would identify as archetypalkitsch. Murray has no interest in relating digital media to the visual avant-garde. She takes it for granted that film and television are narrative mediaforms, because narrative forms have constituted the popular tradition ineach of those media. Although the premise of her book is that the digitalmedium will be the new cinema, this does not mean, as it does forManovich, that digital technology will bring forth a new avant-gardelanguage that will fragment and reconstitute our conventional narrativestrategies. Instead, for Murray digital technology will allow the viewer toinsert herself seamlessly into traditional Hollywood narrative.

Murray’s view of the digital medium is therefore popular and influentialoutside of the communities of the visual arts and media studies. She offersa vision of smooth and seamless interactive narrative that appeals to theentertainment industry itself, because she implies that those brought up inthe world of mainstream film and television can simply extend theiraesthetic principles into the digital realm. Murray’s case for interactive

30 Jay David Bolter


narrative is particularly convincing for those in Hollywood trying tointroduce user participation into conventional broadcast television (so-called ‘enhanced TV’ or ‘interactive TV’) and those in the computer- andvideo-game industry who make games that repurpose film and sports.

With her view of digital narrative, Murray shows us the alternative toManovich: the mainstream against which Manovich’s avant-garde is inrevolt. It is hard, for example, to imagine a conference in which Murray andManovich would both be asked to keynote. Nevertheless, there is oneimportant area of apparent agreement in their work – on the question ofwhether (new) media have essential qualities.

Essentialism and new media Essentialism remains strong in popular media theory today, where theessentialist strategy is to identify certain key properties that a mediumpossesses – properties that are supposed to emerge from the technologyitself. These properties then serve to justify arguments about how themedium can or should be used. Essentialist arguments receive asympathetic hearing from many in new media, especially those computerspecialists who develop and advance the technology. On the other hand,those working in cultural studies have long argued against essentialism anddeterminism in communications technologies as elsewhere intechnoscience. In the 1970s Raymond Williams offered a well-known andcompelling critique of technological determinism in the work of McLuhan(Williams 1974). A straightforward reading of McLuhan does suggest anessentialist – most famously in his characterization of media as hot andcool. Film is hot, for example, because of the high resolution of theanalogue photographic image; television is cool because the low-resolutionvideo image requires the viewer’s participation to complete the picture(McLuhan 1964).

Murray too is an essentialist. She identifies four ‘essential properties’ ofdigital environments: they are the procedural, the participatory, the spatial,and the encyclopaedic. These properties function in her view bothdescriptively and prescriptively (Murray 1997: 71–90). They describe howdigital artefacts function, and at the same time it is the task of the digitaldesigner to explore these qualities in their applications. For Murray, themandate of the digital age is to develop the latent possibilities of themedium by bringing these qualities to fruition. Manovich is subtler: heseems to me to be pulled in two directions here, probably because he is sowidely read both in modernism and in recent art and critical theory. Theadmirer of the modernist avant-garde is drawn to the notion of essentialqualities to be identified and explored, but the student of postmodern artand theory is concerned to avoid a charge of determinism.

It is useful to compare Manovich’s five ‘principles’ (numericalrepresentation, modularity, automation, variability, transcoding) withMurray’s four essential properties, for the lists are quite different. At leasttwo of Murray’s qualities are really popular cultural metaphors for theexperience offered by digital media. To say that the computer is spatial orencyclopaedic is really to say that these are metaphors that we as a culturehave chosen (or might choose) to characterize our experience withcomputers: they are the metaphors of cyberspace and the digital library.



Manovich’s principles are more abstract, and his focus is on the ways inwhich producers (such as artists) manipulate digital technology as thematerial for their expressive efforts. Manovich rejects ‘participation’ as anessential characteristic: he objects to the notion of interactivity as being toobroad to be useful (Manovich 2001a: 55–61). Murray and Manovich seemto agree on one foundational principle: the coded nature of computerapplications. Murray calls this quality ‘procedurality’, while Manovichrefers to ‘transcoding’. But even here the differences are important.

Manovich’s concept is reflexive in a way that Murray’s is not. With‘transcoding’ he is identifying a process by which not only earlier mediaobjects but perhaps whole media forms are transformed into the newlanguage of computer code. He suggests that the introduction of thecomputer as code marks a break in the history of media:

This perspective [comparing properties and aesthetics of various media] isimportant and I am using it frequently in this book, but it is not sufficient. Itcannot address the most fundamental quality of new media that has no historicalprecedent – programmability. Comparing new media to print, photography, ortelevision will never tell us the whole story […]. To understand the logic of newmedia, we need to turn to computer science. It is there that we may expect to findthe new terms, categories, and operations that characterize media that becameprogrammable. From media studies, we move to something that can be called‘software studies’ – from media theory to software theory. [italics in original]

(Manovich 2001a: 47–48)

From my perspective, this may be the single most important passage in TheLanguage of New Media, because it captures precisely Manovich’s claimfor the remediating power of new digital media. Like other remediations,the digital must ultimately surpass its predecessors by promoting somequality that is unique to it. As we have said, for Manovich that quality isnot interactivity (the most commonly made claim for the digital); it is thefact that all digital artefacts are ultimately software. The code that underliesa digital artefact is what gives it its uniqueness (‘no historical precedent’)and therefore its authenticity. In remediating avant-garde film of the 1920s,digital art not only borrows the representational practices of that period, butalso reimagines them by embodying them in code. Manovich’s own recentproject in the remediation of avant-garde film is called Soft Cinema toacknowledge both its roots and its uniqueness (www.manovich.net/softcinemadomain). Manovich appears to be coming down on one side ofa divide by taking what we can call the ‘code view’.

Code and interfaceManovich’s insight into the relationship of new media and the avant-gardeis perhaps unique among new-media theorists. However, the code view invarious forms is widely shared by computer specialists who actually writeapplications. The rhetoric of the code view sometimes turns up amongdigital poets and artists as well. I would argue, however, that there is analternative construction of the digital, widely held but less clearlyarticulated: an ‘interface view’. If the code is what lies beneath the surfaceof a digital artefact, the interface is the surface that the artefact presents to

32 Jay David Bolter


its viewer or user: the input (what the user types on the keyboard, how theuser manipulates the joystick, and so on) and the output (what the user seeson the screen or in a headset and what she hears through the speakers). Theinterface is all that most users ever know of any application: it constitutesthe digital experience for them. Over the past thirty years the emergence ofmore sophisticated and varied interfaces has been the critical element in thedefining of new digital media forms. The graphical user interface (GUI),developed at Xerox PARC in the 1970s and perfected in the Applecomputers of the 1980s, was the single most important step in our culture’schanging understanding of what the computer could be. (The developmentof the GUI was not predetermined: it was in part the result of the creativedecisions of such developers as Englebart and Kay.) Manovich in factdevotes two chapters to exploring the digital interface that grew out of theGUI together with the operations (menu items, filters, cutting, pasting, andso on) that the digital artist carries on through that interface.

Many, perhaps most, digital artists today (including Manovich himself)are really artists of the interface: that is, their concern is with the experiencethat the application offers to the user, not the code that constitutes theapplication. Every programmer knows that there are many different waysto achieve the same result: that many different sequences of code canproduce the same interface and therefore the same digital experience. Manydigital artists and even application designers and game designers do notprogram themselves, but instead trust their ideas to others to code.Furthermore, most genres of computer games are almost all interface: thealgorithms that drive the play action are usually trivial or at least wellknown. (The great exception would be the graphics algorithms –techniques for making the graphics more responsive and morephotorealistic. Yet computer graphics is a kind of coding that is completelyin the service of the interface.) What matters to them is simply that theprogram should render the design operative – that it should deliver thegraphics, the sounds, and the interactions.

The code and interface views reflect different aesthetic and culturalcriteria. The interface view is almost by definition hybrid, multiple,concerned with effects rather than technological essences. The interfaceview also suggests a focus on the relationship of the viewer (user) with theartefact, or on the triangular relationship of viewer, artefact, and artist. Thecode view in its purest form focuses on the program itself as a texture ofsymbols, and those who favour the code view in this form are drawn to atraditional aesthetics of the artefact. They speak of the beauty, simplicity,and elegance of the code (Bolter and Gromala 2006).

Among new-media enthusiasts and theorists, the code view andessentialism often go together. This seems to be particularly true of thosewho work in the area of ‘interactive narrative’, whose research focuses onnarrative ‘engines’ – that is, on algorithms and data structures that willproduce a dynamic story as output. Some researchers regard the form of theoutput, the visual expression of the story, as secondary. Some suggest thatthe same engine could drive various outputs: a prose version, a screen-based video version, and (ultimately) a virtual reality. The game designerChris Crawford’s massive Storytron project (www.storytron.com) is oneexample of a system in which the engine is vastly more important than the



form of the output. Michael Mateas and Andrew Stern’s interactive drama,Façade, also seems to me to privilege the code over the interface (Mateasand Stern 2005). Façade contains a sophisticated engine, which shapes thedialogue as a series of beats within a larger story arc; what the user sees onthe screen are two animated figures, whose stiff gestures would fool no userinto thinking that she was really sitting in the home of a couple of oldfriends. Façade has been praised as a step on the way to the holodeck, butits visual style is compelling and ironic, precisely because it calls intoquestion the claim of transparency that Façade makes. Digital artiststrained in the traditions of the visual arts, on the other hand, are unlikely toemphasize the code at the expense of the visual, aural, or tactile experiencethat their work offers.

It should be no surprise that my two categories (the code and interfaceviews) are not always neatly observed and cannot capture every aspect ofvaried digital designers and artists. For example, some digital artistsactually make the code part of the interface – part of what the viewer/usersees and experiences. This is true of such digital poets as John Cayley, whocalls some of his work ‘code poetry’ (Engberg 2005; Hayles 2002). Butdigital artefacts that display their code are promoting an aesthetic ofhybridity, not the seamless transparency that belongs for example tointeractive narrativists. They in fact form part of Manovich’s new mediaavant-garde.

The Language of New Media seems to me to be seeking to reconcilewhat I am calling the code and interface views. Throughout the book,Manovich contends that the notion of software supersedes (absorbs,transcodes) the traditional notion of media. This is an argument that he alsomakes explicit in the essay ‘For a Post-media Aesthetics’(www.artmargins.com/content/eview/manovich5.html). In a sense wecould say that software becomes the remediation of media. Manovich’s‘software view’ is more encompassing than the code view, for softwareconsists of both the interface and the code. Nevertheless, by insisting thatthe discourse of software should now replace the traditional discourse ofmedia, Manovich runs the risk of validating a hierarchy that the computer-science community has accepted for decades, in which the code is moreimportant than the interface it generates. It is true, on the other hand, thatnew developments in the field of HCI (human–computer interaction) aregradually eroding that sense of hierarchy even within the computer-sciencecommunity. The code view seems to be growing among new-mediatheorists at a time when it is losing ground in the field of computer sciencefrom which it emerged.

Manovich wants to do justice to the hybridity of contemporary mediayet without rejecting essentialism altogether; he wants to reconcile theessential principle of transcoding with a new cinematic practice that ishybrid and contingent. Chapter 5 (‘The Forms’) lays out the argument fora ‘database logic’ through which new media can offer an alternative tonarrative cinema and literature. For Manovich the database represents anew media aesthetic, founded on the computational dichotomy of data andalgorithm. Database logic applies even to computer games, which oftenonly appear to have a narrative structure:

34 Jay David Bolter


This [the event-driven experience of a computer game] is another example of thegeneral principle of transcoding discussed in the first chapter – the projection ofthe ontology of a computer onto culture itself […] The world is reduced to twokinds of software objects that are complementary to each other – data structuresand algorithms.

(Manovich 2001a: 223)

By Chapter 6, which returns to the comparison of new media and cinema,it has become clear that database cinema is a remediation of traditionalcinema. Manovich wants to underline the historical significance of thismove to a new media form, while avoiding the charge of determinism.Even before listing his principles in Chapter 1, he writes that: ‘Not everynew media object obeys these principles. They should be considered not asabsolute laws but rather as general tendencies of a culture undergoingcomputerization’ (Manovich 2001a: 27).

So unlike Murray, who commits herself firmly to the mainstream mediaaesthetic (transparency) and the mainstream technophilosophy(essentialism), Manovich is aware of an interplay of dichotomies(transparency–hybridity, essentialism–contingency). I think he is seekingto promote this interplay in an atmosphere freed of technologicaldeterminism, as he notes in his culminating distinction between thenarrative and database forms:

Rather than trying to correlate database and narrative forms with modern mediaand information technologies, or deduce them from these technologies, I prefer tothink of them as two competing imaginations, two basic creative impulses, twoessential responses to the world.


I do too. I would also like to suggest that narrative and database as aestheticforms in fact align with the two representational strategies of transparencyand hypermedia. Although there are narrative styles in the twentiethcentury that are highly sophisticated and self-referential, mainstreamnarrative film and the current modest attempts at digital interactivenarrative are usually promoted as transparent forms, which permit theviewer/user to become lost (or ‘immersed’) in the story (or the story world).Avant-garde film as well as postmodern and digital art are almost bydefinition self-aware and in my terms hypermediated. Manovich identifiesthe classic avant-gardist Dziga Vertov and the contemporary film-makerPeter Greenaway both as practitioners of his ‘database cinema’ (Manovich2001a: 237–43). The very fact that the database form can be expressed inmore than one medium shows that Manovich does not want to insist toostrongly on the power of the computer medium to determine aestheticprinciples.

Theory and practiceI would like to close by touching briefly on another question: therelationship of theory and practice in new-media studies. In the past decademany universities in North America and Europe have instituted new-mediaprogrammes, most of which pay at least lip service to the notion that theory



and practice should be closely combined. Because I teach in one suchprogramme, it is fair to ask whether Remediation has succeeded in bringingtogether theory and practice – whether this monograph on media historyand theory has practical value for designers and artists. Although I hopedthis would be the case, the practical impact of Remediation has been small.I hope that some artists and designers have taken some inspiration from abook that seeks to validate the notion of working across media and deniesthe need for purity in digital media forms. But Remediation is analyticalrather than prescriptive, and its argument is that artists and designersborrow from and refashion other media forms all the time, whether they areaware of this process or not. Their audiences receive their work in this spiritas well, consciously or unconsciously comparing the new media artefactsto those in other media that they know. In this sense, there is no need forartists and designers to change their practice. The only further argument Icould make is that a conscious knowledge of the practices of remediationwould help media producers to articulate what they already knowintuitively. It is at least plausible that the concepts of remediation mighthelp in the teaching of media design, and I believe the book may be read insome new-media programmes for this reason.

The same question may be posed for The Language of New Media. It isa book of history and theory, not a practical manual. Manovich has hadmore success in crossing the gulf between theory and practice, above allbecause he is himself a practising artist. The book can be read as ahistorically sophisticated manifesto for the database art that Manovichhimself creates: specifically, for his Soft Cinema project. The fact thatManovich himself can produce art that grows out of his theory makes thetheory itself more convincing.

ReferencesBazin, A. (1980 [1967]), ‘The ontology of the photographic image’, in A. Trachtenberg

(ed.), Classic Essays in Photography, New Haven: Leete’s Islands Books, pp. 237–44.

Bloom, H. (1997), The Anxiety of Influence: A Theory of Poetry, 2nd edn., New York:Oxford University Press.

Bolter, J. D., MacIntyre, B., Gandy, M. and Schweitzer, P. (2006), ‘New media and thepermanent crisis of aura’, Convergence, 12: 1, pp. 21–39.

Bolter, J. D. and Gromala, D. (2006), ‘Transparency and reflectivity: digital art and theaesthetics of interface design’, in P.A. Fishwick (ed.), Aesthetic Computing,Cambridge, MA: MIT Press, pp. 369–82.

Engberg, M. (2005), Stepping into the River: Experiencing John Cayley’s River Island,available from http://www.dichtung-digital.com/2005/2/Engberg/index.htm. (Lastaccessed February 1st. 2007)

Foster, H. (1996), The Return of the Real: The Avant-garde at the End of the Century,Cambridge, MA: MIT Press.

Greenberg, C. (1939), ‘Avant-garde and kitsch’, Partisan Review, 6: 5, pp. 34–49.Hayles, N.K. (2002), Writing Machines, Cambridge, MA: MIT Press.Holtzman, S. (1997), Digital Mosaics: The Aesthetics of Cyberspace, New York: Simon

and Schuster.Krauss, R. (1985), The Originality of the Avant-Garde and Other Modernist Myths,

Cambridge, MA: MIT Press.

36 Jay David Bolter


Manovich, L. (2001a), The Language of New Media, Cambridge, MA: MIT Press.———— (2001b), ‘For a post-media aesthetics’, Art Margins, available from

http://www.artmargins.com/content/eview/manovich5.html.Mateas, M. and Stern, A. (artist) (2005), Façade: A One-Act Interactive Drama.

http://forums.adventuregamers.com/showthread.php?t=2076 (Last accessedFebruary 1st. 2007)

McLuhan, M. (1964), Understanding Media: The Extensions of Man, New York: NewAmerican Library Inc.

Müller, Jürgen, E. (1996), Intermedialität, Formen modener kultureller Kommunikation,Münster: Nordus Publikationen.

Murray, J. (1997), Hamlet on the Holodeck, Cambridge, MA: MIT Press.Rajewsky, I. (2002), Intermedialität, Stuttgart: UTB.Spielmann, Y. (1998), Intermedialität: Das System Peter Greenaway. München:

Wilhelm Fink.———— (2005), Video: Das reflektive Medium, Frankfurt am Main: Suhrkamp Verlag.Williams, R. (1974), Television: Technology and Cultural Form, London: Fontana.





Alan Kay’s universal media machine

Lev Manovich

AbstractWhile new-media theorists have spend considerable efforts in trying tounderstand the relationships between digital media and older physical andelectronic media, the important sources – the writing and projects by IvanSutherland, Douglas Englebart, Ted Nelson, Alan Kay, and other pioneersworking in the 1960s and 1970s – remain largely unexamined. What weretheir reasons for inventing the concepts and techniques that today make itpossible for computers to represent, or ‘remediate’ other media? I suggestthat Kay and others aimed to create a particular kind of new media – ratherthan merely simulating the appearances of old ones. These new media usealready existing representational formats as their building blocks, whileadding many new previously non-existent properties. At the same time, asenvisioned by Kay, these media are expandable – that is, users themselvesshould be able to easily add new properties, as well as to invent new media.Accordingly, Kay calls computers the first ‘metamedium’ whose content is‘a wide range of already-existing and not-yet-invented media’.

Medium: Definition 8. a. A specific kind of artistic technique or means ofexpression as determined by the materials used or the creative methods involved:the medium of lithography. b. The materials used in a specific artistic technique:oils as a medium. (American Heritage Dictionary, 4th edition)

(Houghton Mifflin, 2000)

The best way to predict the future is to invent it.(Alan Kay)

Appearance versus functionAs a result of the adoption of the GUI (graphical user interface) in the1980s, software has replaced many other tools and technologies for thecreative professional and it has also given hundreds of millions of peoplethe ability to create, manipulate, sequence, and share media – but has thisled to the invention of fundamentally new forms of culture? Today,computer scientists along with media companies are busy inventingelectronic books and interactive television; consumers are happilypurchasing (or downloading for free) music albums and feature filmsdistributed in digital form, as well as making photographs and video withtheir digital cameras and cell phones; office workers are reading PDFdocuments that imitate paper. And even at the futuristic edge of digitalculture – inhabited by smart objects and ambient intelligence – traditional


KeywordsSimulationRemediationMetamediumNew-media theoryAlan KayXerox PARC


forms persist: Philips showcases ‘smart’ household mirrors which can holdelectronic notes and videos, while its Director of Research dreams about anormal-looking vase which can hold digital photographs.

In short, the revolution in means of production, distribution, and accessto media has not been accompanied by a similar revolution in the syntaxand semantics of media. It is Alan Kay and his collaborators at PARC (thePalo Alto Research Centre, formerly ‘Xerox PARC’) that we must call totask for making digital computers imitate older media. By systematicallydeveloping easy-to-use GUI-based software to create and edit familiarmedia types, Kay and others appear to have locked the computer into beinga simulation machine for ‘old media’. Technologies developed at PARC,such as the bitmapped colour display used as the main computer screen,laser printing, and the first page description language which eventually ledto Postscript, were conceived to support the computer’s new role as amachine for the simulation of physical media. To put these developmentsin terms of Bolter and Grusin’s very influential book Remediation:Understanding New Media (1999), we can say that GUI-based softwareturned digital computers into what they might call ‘remediation machines’.

Bolter and Grusin define remediation as ‘the representation of onemedium in another’ (Bolter and Grusin 1999). According to their argument,new media always remediate older forms and therefore, we should notexpect that computers would function any differently. This perspectiveemphasizes the continuity between computational media and earlier mediaforms. Rather than being separated by different logics, all media –including computers – follow the same logic of remediation. The onlydifference between computers and other media lies in how and what theyremediate. As Bolter and Grusin put this in the first chapter of their book:‘What is new about digital media lies in their particular strategies forremediating television, film, photography, and painting.’ In another place inthe same chapter, they make an equally strong statement that leaves noambiguity about their position: ‘We will argue that remediation is adefining characteristic of the new digital media.’

If we consider today all the digital media created both by consumers andby professionals – digital photography and video shot with inexpensivecameras and cell phones, the contents of personal blogs and online journals,illustrations created in Photoshop, feature films cut on AVID, etc. – in termsof its appearance, digital media indeed often looks to the casual observerexactly the same way it did before it became digital. Thus, if we limitourselves to looking at the surfaces of media, the remediation argumentaccurately describes much of what goes on with computational media. Butrather than accepting this condition as an inevitable consequence of theuniversal logic of remediation, we should ask why this is the case. In otherwords, if contemporary computational media imitates other media, how didthis become possible? There was definitely nothing in the originaltheoretical formulations of digital computers by Turing or Von Neumannabout computers imitating other media such as books, photography, or film.

The conceptual and technical gap that separates the first room-sizedcomputers – used by the military to calculate the shooting tables of anti-aircraft guns or to crack German communication codes – versus thecontemporary small desktop and laptop computers – used by ordinary

40 Lev Manovich


people to store, edit, and share media – is vast. The contemporary identityof a computer as a media processor took about forty years to emerge, if wecount from 1949 when MIT’s Lincoln Laboratory started to work on itsfirst interactive computers, to 1989 when the first commercial version ofPhotoshop was released. It took generations of brilliant and very creativethinkers to invent the multitude of concepts and techniques that today makeit possible for computers to ‘remediate’ other media so well. What weretheir reasons for doing this? What was their thinking? In short, why didthese people dedicate their careers to inventing the ultimate ‘remediationmachine’?

I cannot consider the thinking of each of the key figures in the historyof media computing in the space of one article. However, we can take acloser look at one place where the identity of a computer as a ‘remediationmachine’ was largely put in place – Alan Kay’s Learning Research Groupat Xerox PARC in operation during the first part of the 1970s.

We can ask two questions: first, what exactly did Kay want to do, andsecond, how did he and his colleagues set about to achieve their aims?1 Thebrief answer – which will be expanded below – is that Kay wanted to turncomputers into a ‘personal dynamic media’ that could be used for learning,discovery, and artistic creation. His group achieved this by systematicallysimulating most existing media within a computer, while simultaneouslyadding many new properties to these media. Kay and his collaborators alsodeveloped a new type of programming language that, at least in theory,would allow users to quickly invent new types of media using the set ofgeneral tools already provided for them. All these tools and simulations ofalready existing media were given a unified user interface designed toactivate multiple mentalities and ways of learning, including thekinaesthetic, the iconic, and the symbolic.

Kay conceived of ‘personal dynamic media’ as a fundamentally newkind of media with a number of historically unprecedented properties, suchas the ability to store all of the user’s information, simulate all types ofmedia within a single machine and, as Kay and Adele Goldberg put it,‘involve the learner in a two-way conversation’ (Kay and Goldberg 2003:399).2 These properties enable new relationships between the user and themedia she may be creating, editing, or viewing on a computer. And this isessential if we want to understand the relationships between computers andearlier media. Briefly put, while visually computational media may closelymimic other media, these media now function in fundamentally differentways.

For instance, consider digital photography that often does in fact imitatethe appearance of traditional photography. For Bolter and Grusin, this is anexample of how digital media ‘remediates’ its predecessors. But rather thanonly paying attention to their appearance, let us think about how digitalphotographs can function. If a digital photograph is turned into a physicalobject in the world – an illustration in a magazine, a poster on the wall, aprint on a t-shirt – it functions in the same ways as its predecessor.3 But ifwe leave the same photograph inside its native computer environment –which may be a laptop, a network storage system, or any computer-enabledmedia device such as a cell phone which allows its user to edit and move itto other devices or the Internet – it can function in ways which, in my view,


1. Kay has expressed hisideas in a few articlesand a large number ofinterviews and publiclectures. Thefollowing have beenmy main primarysources: Alan Kayand Adele Goldberg,‘Personal DynamicMedia’, IEEEComputer, Vol. 10,No. 3 (March 1977).My quotes are fromthe reprint of thisarticle in New MediaReader, eds. NoahWardrip-Fruin andNick Montfort (TheMIT Press, 2003);Alan Kay, ‘The EarlyHistory of Smalltalk’,(HOPL-II/4/93/MA,1993); Alan Kay, ‘APersonal Computerfor Children of AllAges’, Proceedings ofthe ACM NationalConference, Boston,August 1972; AlanKay, Doing withImages MakesSymbols (UniversityVideoCommunications,1987), videotape(available athttp://www.archive.org);Alan Kay, ‘UserInterface: A PersonalView’, p. 193, inBrenda Laurel, ed.,The Art of Human-Computer InterfaceDesign (Reading,Mass.’ Addison-Wesley, 1990), pp.191–207; DavidCanfield Smith et al.,‘Designing the Staruser Interface’, Byte,issue 4 (1982).

2. Since the work ofKay’s group in the1970s, computerscientists, hackers,and designers addedmany other uniqueproperties. Forinstance, we canquickly move mediaaround the Net and


make it radically different from its traditional equivalent. To use a differentterm, we can say that a digital photograph offers its users many moreaffordances that its non-digital predecessor could. For example, a digitalphotograph can be quickly modified in numerous ways, and equallyquickly combined with other images; it can be instantly moved around theworld and shared with other people; and it can be inserted into a textdocument, or an architectural design. Furthermore, we can automatically –by running the appropriate algorithms – improve its contrast, make itsharper, and even in some situations remove blur.

Note that only some of these new properties are specific to a particularmedia – in this case, a digital photograph (i.e. an array of pixels representedas numbers). Many other properties are shared by a larger class of mediaspecies: for instance, at the current stage of digital culture, all types ofmedia files can be attached to an e-mail message. Still others display moregeneral features of the current GUI paradigm (which was developed thirtyyears ago at PARC): for instance, the fast response of the computer to itsuser’s actions assure that there will be, as Kay and Goldberg put it, ‘nodiscernable pause between cause and effect’ (Kay and Goldberg 2003:394). Still other features are enabled by network protocols such as TCP-IPthat allow all kinds of computers and other devices to be connected to thesame network. In summary, we can say that only some of the ‘new DNA’of a digital photograph is due to its particular place of birth, i.e. inside adigital camera. Many of its other features are the result of the currentparadigm of network computing in general.

‘Simulation is the central notion of the Dynabook’While Alan Kay has articulated his ideas in a number of articles and talks,his 1977 article co-authored with one of his main PARC collaborators,computer scientist Adele Goldberg, is a particularly useful resource if wewant to understand contemporary computational media. In this article, Kayand Goldberg describe the vision of the Learning Research Group at PARCin the following way: to create ‘a personal dynamic medium the size of anotebook (the Dynabook) which could be owned by everyone and wouldhave the power to handle virtually all of its owner’s information-relatedneeds’ (Kay and Goldberg 2003: 393).4 Kay and Goldberg ask the readersto imagine that this device ‘had enough power to outrace your senses ofsight and hearing, enough capacity to store for later retrieval thousands ofpage-equivalents of reference materials, poems, letters, recipes, records,drawings, animations, musical scores, waveforms, dynamic simulationsand anything else you would like to remember and change’ (Kay andGoldberg 2003: 394).

In my view, ‘all’ in the first statement is important: it means that theDynabook – or the computational media environment in general, regardlessof the size or form of the device in which it is implemented – shouldsupport the viewing, creating , and editing of all possible media that havetraditionally been used for human expression and communication.Accordingly, while separate programs to create works in different mediawere already in existence, Kay’s group for the first time implemented themall together within a single machine. In other words, Kay’s paradigm wasnot to simply create a new type of computer-based media that would

42 Lev Manovich

share it with millionsof people usingFlickr, YouTube, andother sites.

3. However, considerthe followingexamples of things tocome: ‘Posters inJapan are beingembedded with tagreaders that receivesignals from theuser’s “IC” tag andsend relevantinformation and freeproducts back’(Hoshimo 2005: 25).

4. The emphasis in thisand all followingquotes from thisarticle is mine – L.M.


coexist with other physical media. Rather, the goal was to establish thecomputer as an umbrella, a platform for all already existing expressiveartistic media, which Kay and Goldberg dub the ‘metamedium’. Thisparadigm changes our understanding of what media is. From Lessing’sLaocoon; or, On the Limits of Painting and Poetry (1766) to NelsonGoodman’s Languages of Art (1968), the modern discourse about mediadepends on the assumption that different media have distinct properties andin fact should be understood in opposition to each other. Putting all mediawithin a single computer environment does not necessarily erase alldifferences in what various media can represent and how they are perceived– but it does bring them closer to each other in a number of ways. Some ofthese new connections were already apparent to Kay and his colleagues;others became visible only decades later when the new logic of media setin place at PARC unfolded more fully; some are perhaps still not visible tous today because they have not been given practical realization. Oneobvious example of such connections is the emergence of multimedia as astandard form of communication: web pages, PowerPoint presentations,multimedia artworks, mobile multimedia messages, media blogs, and othercommunication forms which combine media. Another is the rise ofcommon interface conventions and tools which we use in working withdifferent types of media regardless of their origin: for instance, a virtualcamera, a magnifying lens, and, of course the omnipresent copy, cut, andpaste commands.5 Yet another is the ability to map one media into anotherusing appropriate software – images into sound, sound into images,quantitative data into a 3D shape or sound, etc. – used widely today in suchareas as DJ/VJ/live cinema performance and information visualization.This situation is the direct opposite of the modernist media paradigm of theearly twentieth century, which was focused on discovering the uniquelanguage of each artistic medium. All in all, it is as though different mediaare actively trying to reach towards each other, exchanging properties andletting each other borrow their own unique features.

Alan Turing theoretically defined a computer as a machine that cansimulate a very large class of other machines, and it is this simulationability that is largely responsible for the proliferation of computers inmodern society. But as I have already mentioned, neither he nor othertheorists and inventors of digital computers explicitly considered that thissimulation could also include media. It was only Kay and his generationthat extended the idea of simulation to media – thus turning the UniversalTuring Machine into a Universal Media Machine, so to speak. Accordingly,Kay and Goldberg write: ‘In a very real sense, simulation is the centralnotion of the Dynabook’ (Kay and Goldberg 2003: 399). When we usecomputers to simulate some process in the real world – the behaviour of aweather system, the processing of information in the brain, the deformationof a car in a crash – our concern is to correctly model the necessary featuresof this process or system. We want to be able to test how our model wouldbehave in different conditions with different data, and the last thing wewant is for the computer to introduce some new property into the modelthat we ourselves did not specify. In short, when we use computers as ageneral-purpose medium for simulation, we want this medium to becompletely ‘transparent’.

2. Kieslowksi hasreferred to Welles as


5. This elevation of thetechniques ofparticular media to astatus of generalinterface conventionscan be understood asthe further unfoldingof the principlesdeveloped at PARCin the 1970s. First,the PARC teamspecifically wanted tohave a unifiedinterface for all newapplications. Second,they developed theidea of ‘universalcommands’ such as‘move’, ‘copy’, and‘delete’. As describedby the designers ofthe Xerox Starpersonal computerreleased in 1981,‘MOVE is the mostpowerful commandin the system. It isused during textediting to rearrangeletters in a word,words in a sentence,sentences in aparagraph, andparagraphs in adocument. It is usedduring graphicsediting to movepicture elements,such as lines andrectangles, around inan illustration. It isused during formulaediting to movemathematicalstructures, such assummations andintegrals, around inan equation’ (DavidCanfield Smith et al.,‘Designing the Staruser interface’, Byte,4: 1 (1982), pp. 242–82).


But what happens when we simulate different media in a computer? Inthis case, the appearance of new properties may be welcome as they canextend the expressive and communicative potential of these media.Appropriately, when Kay and his colleagues created computer simulationsof existing physical media – i.e. the tools for representing, creating, editing,and viewing these media – they ‘added’ many new properties. For instance,in the case of a book, Kay and Goldberg point out,

It need not be treated as a simulated paper book since this is a new medium withnew properties. A dynamic search may be made for a particular context. The non-sequential nature of the file medium and the use of dynamic manipulation allowsa story to have many accessible points of view.

(Kay and Goldberg 2003: 395)6

Kay and his colleagues also added various other properties to the computersimulation of paper documents. As Kay referred to this in another article,his idea was not to simply imitate paper but rather to create ‘magical paper’(Kay 1999: 199). For instance, the PARC team gave users the ability tomodify the fonts in a document and create new fonts. They alsoimplemented another important idea that was already developed byDouglas Engelbart’s team in the 1960s: the ability to create different viewsof the same structure (I will discuss this in more detail below). And bothEngelbart and Ted Nelson also already ‘added’ something else: the abilityto connect different documents or different parts of the same documentthrough hyperlinking – i.e. what we now know as hypertext andhypermedia. Engelbart’s group also developed the ability for multiple usersto collaborate on the same document. This list goes on and on: e-mail in1965, newsgroups in 1979, the World Wide Web in 1991, and so on.

Each of these new properties has far-reaching consequences. Take‘search’, for instance. Although the ability to search through a page-longtext document does not sound like a very radical innovation, as thedocument gets longer, this ability becomes more and more important. Itbecomes absolutely crucial if we have a very large collection of documents– such as all the web pages available today. Although current search enginesare far from being perfect and new technologies will continue to evolve,imagine how different the culture of the Web would be without them.

Or take the capacity to collaborate on the same document(s) by anumber of users connected to the same network. While it was alreadywidely used by companies in the 1980s and 1990s, it was not until early2000s that the larger public saw the real cultural potential of this ‘addition’to print media. By harvesting the small amounts of labour and expertisecontributed by a large number of volunteers, social software projects –most famously, Wikipedia – created vast and dynamically updatable poolsof knowledge which would be impossible to create in traditional ways. Ina less visible way, every time we do a search on the Web and then click onsome of the results, we also contribute to a knowledge set used byeverybody else – since in deciding the sequence in which to present theresults of a particular search, Google’s algorithms take into account thoseamong the results of previous searches for the same words people foundmost useful.


44 Lev Manovich

6. Emphasis mine –L.M.



Studying the writings and public presentations of the people whoinvented interactive media computing – Sutherland, Engelbart, and Kay –makes it clear that they did not come with new properties of computationalmedia as an afterthought. On the contrary, they knew that they were turningphysical media into new media. In 1968, Engelbart gave his famous demoat the Fall Joint Computer Conference in San Francisco before a fewthousand people that included computer scientists, IBM engineers, peoplefrom other companies involved in computers, and funding officers fromvarious government agencies (Waldrop 2001: 287). Although Engelbarthad ninety minutes, he had a lot to show. Over the few previous years, histeam at the Research Centre for Augmenting Human Intellect hadessentially developed the modern office environment as it exists today (notbe confused with the modern media design environment which wasdeveloped later at PARC). Their computer system included wordprocessing with outlining features, documents connected throughhypertext, online collaboration (i.e. two people at remote locations workingon the same document in real time), online user manuals, an online projectplanning system, and other elements of what is now commonly called the‘computer-supported collaborative work’. This team also developed thekey elements of the modern user interface that were later refined at PARC:the mouse and multiple windows.

Paying attention to the sequence of this demo reveals that whileEngelbart had to make sure that his audience would be able to relate the newcomputer system to what they already knew and could use, his focus was ona completely new set of features available in computer-simulated media.Engelbart devotes the first segment of the demo to word processing, but assoon as he briefly demonstrated text entry, cut, paste, insert, naming, andsaving files – in other words, the set of tools that make a computer into amore versatile typewriter – he then goes on to show at more length thefeatures of his system which no writing medium had before: ‘view control’.7

As Engelbart points out, the new writing medium could switch at the user’swish between many different views of the same information. A text file couldbe sorted in different ways. It could also be organized as a hierarchy with anumber of levels that can be collapsed and expanded – like the outline toolsincluded in modern word processors such as Microsoft Word.

In his demo, Engelbart next shows another example of view control,which today, forty years later, is still not available in popular documentmanagement software. He makes a long ‘to do’ list and organizes it bylocations. He then instructs the computer to display these locations as avisual graph (i.e. a set of points connected by lines). In front of our eyes,representation in one medium changes into another medium – text becomesa graph. But this is not all. The user can control this graph to displaydifferent amounts of information – something that no image in physicalmedia can do. As Engelbart clicks on different points in a graphcorresponding to particular locations, the graph shows the appropriate partof his ‘to do’ list. This ability to interactively change how much and whatinformation an image shows is particularly important in today’sinformation visualization applications.

Next, Engelbart presents ‘a chain of views’ which he preparedbeforehand. He switches between these views using ‘links’, which may


7. The complete videoof DouglasEngelbart’s 1968demo is available athttp://sloan.stanford.edu/MouseSite/1968Demo.html.


look like hyperlinks the way they exist on the Web today – but they actuallyhave a different function. Instead of creating a path between many differentdocuments à la Vannevar Bush’s Memex (often seen as the precursor tomodern hypertext), Engelbart is using links as a method for switchingbetween different views of a single document organized hierarchically. Hebrings up a line of words displayed in the upper part of the screen; when heclicks on these words, more detailed information is displayed in the lowerpart of the screen. This information can, in its turn, contain links to otherviews that show even more detail.

In this way, rather than using links to drift through the textual universeassociatively and ‘horizontally’, we move ‘vertically’ between general andmore detailed information. Appropriately, in Engelbart’s paradigm, we arenot ‘navigating’ – we are ‘switching views’. We can create many differentviews of the same information, and switch between these views in differentways. And this is what Engelbart systematically explains in this first part ofhis demo. He demonstrates that one can change views by issuingcommands, by typing numbers that correspond to different parts of ahierarchy, by clicking on parts of a picture, or on links in the text.

Since new-media theory and criticism emerged in the early 1990s,endless texts have been written about interactivity, hypertext, virtual space,cyberspace, cyborgs, and so on. But I have never seen anyone discuss ‘viewcontrol’. And yet this is one of the most fundamental and radical newtechniques for working with information and media available to us today.‘View control’, i.e. the ability to switch between many different views andkinds of views of the same information, is now implemented in multipleways not only in word processors and e-mail clients, but also in all ‘mediaprocessors’ (i.e. media editing software) such as AutoCAD, Maya, AfterEffects, Final Cut, Photoshop, InDesign, and so on. For instance, in thecase of 3D software, it can usually display the model in at least half a dozendifferent ways: in wireframe, fully rendered, etc. In the case of animationand visual-effects software, since a typical project may contain dozens ofseparate objects, each having dozens of parameters, it is often displayed ina way similar to how outline processors can show text. In other words, theuser can switch between more and less information. One can choose to seeonly those parameters on which the user is working at present. One can alsozoom in and out of the composition. When this is done, parts of thecomposition do not simply get smaller or bigger – they show less or moreinformation automatically. For instance, at a certain scale, the user mayonly see the names of different parameters; but when one zooms into thedisplay, the program may also display the graphs that indicate how theseparameters change over time.

As we can see from the examples analysed above, the aim of theinventors of computational media – Engelbart, Nelson, Kay, and colleagueswith whom they have worked – was not to simply create accuratesimulations of physical media. Instead, in every case, the goal was to create‘a new medium with new properties’ which would allow people tocommunicate, learn, and create in new ways. So while today, the content ofthese new media may often look the same as its predecessors, we shouldnot be fooled by this similarity. The newness lies not in the content but inthe software tools used to create, edit, view, distribute, and share this


46 Lev Manovich



content. Therefore, rather than only looking at the ‘output’ of software-based cultural practices, we need to consider the software itself – since itallows people to work with media in a number of historicallyunprecedented ways. To summarize: while on the level of appearance,computational media indeed often remediate (i.e. re-presents) previousmedia, the software environment in which this media ‘lives’ is verydifferent.

Let me add one more item to the examples discussed above – IvanSutherland’s ‘Sketchpad’ from 1962. Created by Sutherland as a part of hisPh.D. thesis at MIT, Sketchpad deeply influenced all subsequent work incomputational media (including that of Kay) not only because it was thefirst interactive media-authoring program, but also because it made it clearthat computer simulations of physical media can add many exciting newproperties to the media being simulated. Sketchpad was the first softwarethat allowed its users to interactively create and modify line drawings. AsNoah Wardrip-Fruin points out, it ‘moved beyond paper by allowing theuser to work at any of 2000 levels of magnification – enabling the creationof projects that, in physical media, would either be unwieldly large orrequire detailed work at an impractically small size’ (Wardrip-Fruin2003:109). Sketchpad similarly redefined graphical elements of a design asobjects which ‘can be manipulated, constrained, instantiated, representedironically, copied, and recursively operated upon, even recursively merged’(Wardrip-Fruin 2003:109).. For instance, if the designer defines newgraphical elements as instances of a master element and later makes achange to the master, all these instances would also change automatically.

Another new property that perhaps demonstrates most dramatically howcomputer-aided drafting and drawing differed from their physicalcounterparts was Sketchpad’s use of constraints. In Sutherland’s ownwords, ‘The major feature which distinguishes a Sketchpad drawing froma paper and pencil drawing is the user’s ability to specify to Sketchpadmathematical conditions on already drawn parts of his drawing which willbe automatically satisfied by the computer to make the drawing take theexact shape desired’ (Sutherland [1963] 2003). For instance, if a user drewa few lines, and then gave the appropriate command, Sketchpadautomatically moves these lines until they are parallel to each other. If auser gives a different command and selects a particular line, Sketchpadmoves the lines in such a way that they are parallel to each other andperpendicular to the selected line.

Although we have not exhausted the list of new properties thatSutherland built into Sketchpad, it should be clear that this first interactivegraphical editor was not only simulating existing media. Appropriately,Sutherland’s 1963 paper on Sketchpad repeatedly emphasizes the newgraphical capacities of his system, marvelling how it opens new fields of‘graphical manipulation that has never been available before’ (Sutherland[1963] 2003: 123). The very title Sutherland gives to his Ph.D. thesisforegrounds the novelty of his work: Sketchpad: A Man-MachineGraphical Communication System. Rather than conceiving of Sketchpad assimply another medium, Sutherland presents it as something else – acommunication system between two entities: the human and the intelligentmachine. Kay and Goldberg will later also foreground this dimension of



communication by referring to it as ‘a two-way conversation’ and callingthe new ‘metamedium’ ‘active’ (Kay and Goldberg 2003: 394). We can alsothink of Sketchpad as a practical demonstration of J.C. Licklider’s idea ofthe ‘man-machine symbiosis’ applied to image-making and design(Licklider [1960] 2003).

Permanent extendibilityAs we saw, Sutherland, Nelson, Engelbart, Kay, and other pioneers ofcomputational media have added many previously non-existent propertiesto media that they have simulated in a computer. The subsequentgenerations of computer scientists, hackers, and designers have addedmany more properties – but this process is far from finished. And there isno logical or material reason why it will ever be finished.

To add new properties to physical media requires modifying its physicalsubstance. But since computational media exists as software, we can addnew properties or even invent new types of media simply by changingexisting, or writing new, software. Adding plug-ins and extensions, asprogrammers have been doing with Photoshop and Firefox, is another wayto innovate. One can also combine existing software together. For instance,at the moment of this writing in 2007, programmers keep extending thecapacities of mapping media by creating software mash-ups whichcombine the services and data provided by Google Maps, Flickr, Amazon,other sites, and media uploaded by users. In short, ‘new media’ is ‘new’because new properties (i.e. new software techniques) can always be easilyadded to it. Put differently, in industrial, i.e. mass-produced mediatechnologies, ‘hardware’ and ‘software’ were one and the same thing. Forexample, the pages of a book were bound in a particular way that fixed theorder of pages. The reader could change neither the order nor the level ofdetail displayed à la Engelbart’s ‘view control’. Similarly, the filmprojector combined hardware and what we now call ‘media player’software into a single machine. In the same way, the controls built into thetwentieth-century mass-produced camera could not be modified at theuser’s will. And although today the user of a digital camera similarly cannoteasily modify the hardware of her camera, as soon as she transfers thepictures into a computer, she has access to an endless number of controlsand options for modifying her pictures via software.

In the nineteenth and twentieth centuries, there were two types ofsituations when the normally rigidly fixed industrial media was more lessfixed. The first type of situation was when a new media was being firstdeveloped: for instance, the invention of photography in the 1820s–1840s.The second type of situation was when artists would systematicallyexperiment with and ‘open up’ already industrialized media – such as theexperiments with film and video during the 1960s, which came to be called‘Expanded Cinema’.

What used to be separate moments of experimentations with mediaduring the industrial era became the norm in software society. In otherwords, computers have legitimized experimentation with media. Why isthis so?

What differentiates a modern digital computer from any other machine– including industrial media machines for capturing and playing media – is


48 Lev Manovich



the separation of hardware and software. It is because an endless number ofdifferent programs performing different tasks can be written to run on thesame type of machine, this machine – i.e. a digital computer – is used sowidely today. Consequently, the constant invention of new and themodification of existing media software are simply two examples of thisgeneral principle. In other words, experimentation is a default feature ofcomputational media. In its very structure, it is ‘avant-garde’ since it isconstantly being extended and thus redefined.

If in modern culture ‘experimental’ and ‘avant-garde’ are opposed to thenormalized and the stable, this opposition has largely disappeared insoftware culture. And the role of the media avant-garde is performed nolonger by individual artists in their studios, but by a variety of players, fromvery big to very small – from companies such as Microsoft, Adobe, andApple, to independent programmers, hackers, and designers.

But this process of the continual invention of new algorithms does notmove in just any direction. If we look at contemporary media software –CAD, computer drawing and painting, image editing, audio and videoremixing, word processing – we see that most of their fundamentalprinciples were already developed by the generation of Sutherland and Kay.As new techniques continue to be invented, they are layered over thefoundations that were gradually put in place by Sutherland, Engelbart, Kayand others in the 1960s and 1970s.

Of course, we are not dealing here only with the history of ideas. Socialand economic factors – such as the dominance of the media softwaremarket by a handful of companies, or the wide adoption of particular fileformats –– also constrain possible directions of software evolution. Putdifferently, today, software development is an industry and as such it isconstantly balanced between stability and innovation, standardization andexploration of new possibilities. But it is not just any industry. Newprograms can be written and existing programs can be extended andmodified (if the source code is available) by anybody who hasprogramming skills and access to a computer, a programming language,and a compiler. In other words, today, software is fundamentally modifiablein a way that physical industrially produced objects usually are not.Although Turing and Von Neumann already formulated this fundamentalextendibility of software in theory, its contemporary practice – thousandsof people daily involved in extending the capabilities of computationalmedia – is a result of a long historical development. This development tookus from the few early room-sized computers that were not easy toreprogram, to a wide availability of cheap computers and programmingtools decades later. Such democratization of software development was atthe core of Kay’s vision. Kay was particularly concerned with how tostructure programming tools in such a way that would make developmentof media software possible for ordinary users. For instance, at the end ofthe 1977 article I have already been extensively quoting, Kay and Goldbergwrite: ‘We must also provide enough already-written general tools so thata user need not start from scratch for most things she or he may wish to do.’

Comparing the process of continuous media innovation via newsoftware to the history of earlier, pre-computational media reveals a newlogic at work. According to a commonplace idea, when a new medium is



invented, it first closely imitates already existing media before discoveringits own language and aesthetics. Indeed, the Bibles first printed byGutenberg closely imitated the look of handwritten manuscripts; earlyfilms produced in the 1890s and 1900s mimicked the presentational formatof theatre by positioning the actors on an invisible shallow stage and havingthem face the ‘audience’ represented by the fixed camera. Slowly, printedbooks developed a different way of presenting information; similarlycinema also developed its own original concept of narrative space. Throughrepetitive shifts in points of view presented in subsequent shots, theviewers were placed inside this space – thus literally finding themselvesinside the story.

Can this logic apply to the history of computer media? As theorized byTuring and Von Neumann, the computer is a general-purpose simulationmachine. This is its uniqueness and its difference from all other machinesand previous media. This means that the commonplace idea – that a newmedium gradually finds its own language cannot apply to computer media.If this were true, it would go against the very definition of the moderndigital computer. This theoretical argument is supported by practice. Thehistory of computer media so far has not been about arriving at somestandardized language – the way this, for instance, happened with cinema– but rather, it seems to be about the gradual expansion of uses, techniques,and possibilities. Rather than arriving at a particular language, we aregradually discovering that the computer can speak more and morelanguages.

If we are to look more closely at the early history of computer media –for instance, the way we have been looking at Kay’s ideas and work in thistext – we will discover another reason why the idea of a new mediumgradually discovering its own language does not apply to computer media.The systematic practical work on making a computer simulate and extendexisting media (e.g. Sutherland’s Sketchpad, the first interactive wordprocessor developed by Engelbart’s group, etc.) came after computers werealready put to multiple uses – performing different types of calculations,solving mathematical problems, controlling other machines in real time,running mathematical simulations, simulating some aspects of humanintelligence, and so on.8 Therefore, when the generation of Sutherland,Nelson, and Kay started to create ‘new media’, they built it on top of, so tospeak, what computers were already known to be capable of. Consequently,they added new properties into the physical media they were simulatingright away. This can be very clearly seen in the case of Sketchpad.Understanding that one of the roles a computer can play is that of a problemsolver, Sutherland built in a powerful new feature that never before existedin a graphical medium – satisfaction of constraints. To rephrase thisexample in more general terms, we can say that rather than moving from animitation of older media to finding its own language, computational mediawas from the very beginning speaking a new language.

In other words, the pioneers of computational media did not have thegoal of making the computer into a ‘remediation machine’ that wouldsimply represent older media in new ways. Instead, well knowing the newcapabilities provided by digital computers, they set out to createfundamentally new kinds of media for expression and communication.


50 Lev Manovich

8. We should alsomention the work onSAGE by MITLincoln Laboratory,which by the middleof the 1950s, hadalready establishedthe idea of interactivecommunicationbetween a human anda computer via ascreen with agraphical display anda pointing device. Infact, Sutherlanddeveloped Sketchpadon the TX-2, whichwas the new versionof a larger computerMIT had constructedfor SAGE.



These new media would use as their raw ‘content’ the older media, whichalready served humans well for hundreds and thousands of years – writtenlanguage, sound, line drawings and design plans, and continuous toneimages like paintings and photographs. But this does not compromise thenewness of new media. For computational media uses these traditionalhuman media simply as building blocks to create previously unimaginablerepresentational structures, creative and thinking tools, and communicationoptions.

Although Sutherland, Engelbart, Nelson, Kay, and others developedcomputational media on top of already existing developments incomputational theory, programming languages, and computer engineering,it would be incorrect to conceive the history of such influences as onlygoing in one direction – from already existing and more general computingprinciples to particular techniques of computational media. The inventorsof computational media have had to question many, if not most of thealready established ideas about computing. They have defined many newfundamental concepts and techniques of how both software and hardwarework, thus making important contributions to hardware and softwareengineering. A good example is Kay’s development of Smalltalk, which forthe first time systematically established a paradigm of object-orientedprogramming. Kay’s rationale in developing this programming languagewas to give a unified appearance to all applications and the interface of thePARC system and, even more importantly, to enable its users to quicklyprogram their own media tools. Kay cites an interesting example in whichan object-oriented illustration program written in Smalltalk by aparticularly talented 12-year-old girl was only a page long (Kay 1987: v).Subsequently, the object-oriented programming paradigm became verypopular and object-oriented features have been added to most popularlanguages such as C++ and Java.

Looking at the history of computer media and examining the thinking ofits inventors makes it clear that we are dealing with the opposite oftechnological determinism. When Sutherland designed Sketchpad, andNelson conceived hypertext, or Kay programmed a paint program, eachnew property of computer media had to be imagined, implemented, tested,and refined. In other words, these characteristics did not simply come as aninevitable result of a meeting between digital computers and modernmedia. Computational media had to be invented, step-by-step. And it wasinvented by people who were looking for inspiration in modern art,literature, cognitive and educational psychology, and theory of media asmuch as technology. For example, Kay recalls that reading McLuhan’sUnderstanding Media led him to a realization that the computer can be amedium rather than a mere tool (Kay 1990: 192–93).

So far, I have talked about the history of computational media as a seriesof consecutive ‘additions’. However, this history is not only a process ofthe accumulation of ever more options. Although, in general, we have moretechniques at our disposal today than twenty or thirty years ago, it is alsoimportant to remember that many fundamentally new techniques wereconceived but never given commercial implementation, or were poorlyimplemented and did not become popular. Or perhaps they were notmarketed properly. Sometimes the company making the software might go



out of business. At other times, the company that created the softwaremight be purchased by another company that in turn would ‘shelve’ thesoftware so it would not compete with its own products. And so on. Inshort, the reasons why many new techniques have not becomecommonplace are multiple, and are not reducible to a single principle suchas ‘the most easy to use techniques become most popular’.

For instance, one of the ideas developed at PARC was ‘project views’.Each view, according to Kay, ‘holds all the tools and materials for aparticular project and is automatically suspended when you leave it’ (Kay1990: 200). Although currently, in 2007, there are some applications thatimplement this idea, it is not a part of most popular operating systems, thatis, Windows, MAC OS X, and Linux. The same holds true for thecontemporary World Wide Web implementation of hyperlinks. The links onthe Web are static and one-directional. Ted Nelson, who is credited withinventing hypertext around 1964, conceived it from the beginning ashaving a variety of other link types. In fact, when Tim Berners-Leesubmitted his paper about the Web to the ACM Hypertext 1991 conference,his paper was only accepted for a poster session rather than the mainconference program. The reviewers saw his system as being inferior tomany other hypertext systems that were already developed in the academicworld over the previous two decades (Wardrip-Fruin and Montford 2003)..

Computer as metamediumAs we have established, the development of computational media runscontrary to previous media history. But in a certain sense, the idea of a newmedia gradually discovering its own language actually might apply to thehistory of computational media after all. And just as it was the case withprinted books and cinema, this process has taken a few decades. When thefirst computers were built in the middle of the 1940s, they could not beused as media for cultural representation, expression, and communication.Slowly, through the work of Sutherland, Engelbart, Nelson, Papert, andothers in the 1960s, the ideas and techniques were developed which madecomputers into a ‘cultural machine’. One could create and edit text, makedrawings, move around virtual objects, etc. And finally, when Kay and hiscolleagues at PARC systematized and refined these techniques, placingthem under the umbrella of GUI, which made computers accessible tomultitudes, a digital computer was finally, in cultural terms, given its ownlanguage.

Or rather, it became something that no other media has been before. Forwhat has emerged was not yet another medium but, as Kay and Goldberginsist in their article, something qualitatively different and historicallyunprecedented. To mark this difference, they introduce a new term –‘metamedium’.

This metamedium is unique in a number of different ways. One of themwe already discussed in detail – it could represent most other media whileaugmenting them with many new properties. Kay and Goldberg also nameother properties that are equally crucial. The new metamedium is, theyassert, ‘active – it can respond to queries and experiments – so that themessages may involve the learner in a two-way conversation’. For Kay,who was strongly interested in children and learning, this property was


52 Lev Manovich



particularly important since, as he and Goldberg puts it, it ‘has never beenavailable before except through the medium of an individual teacher’ (Kayand Goldberg 2003: 394). Further, as noted above, the new metamediumcan handle ‘virtually all of its owner’s information-related needs’. It canalso ‘serve as a programming and problem solving tool’, and ‘an interactivememory for the storage and manipulation of data’ (Ibid.: 393). But theproperty that is the most important from the point of view of media historyis that the computer metamedium is simultaneously a set of different mediaand a system for generating new media tools and new types of media. Inother words, a computer can be used to create new tools for working in themedia it already provides, as well as to develop new not-yet-inventedmedia.

Using the analogy with print literacy, Kay motivates this property in thefollowing way: ‘The ability to “read” a medium means you can accessmaterials and tools generated by others. The ability to write in a mediummeans you can generate materials and tools for others. You must have bothto be literate’ [original emphasis] (Kay 1990: 193). Accordingly, Kay’s keyeffort at PARC was the development of the Smalltalk programminglanguage. All media-editing applications and GUI itself were written inSmalltalk. This made all the interfaces of all applications consistent,facilitating the quick learning of new programs. Even more importantly,according to Kay’s vision, the Smalltalk language would allow evenbeginning users to write their own tools and define their own media. Inother words, all media-editing applications, which would be provided witha computer, were to serve also as examples, inspiring users to modify themand to write their own applications.

Accordingly, a large part of Kay and Goldberg’s paper is devoted to adescription of software developed by the users of their system: ‘ananimation system programmed by animators’, ‘a drawing and paintingsystem programmed by a child’, ‘a hospital simulation programmed by adecision-theorist’, ‘an audio animation system programmed by musicians’,‘a musical score capture system programmed by a musician’, ‘an electroniccircuit designed by a high school student’. As can be seen from this list thatcorresponds to the sequence of examples in the article, Kay and Goldbergdeliberately juxtapose different types of users – professionals, high-schoolstudents, and children – in order to show that everybody can develop newtools using the Smalltalk programming environment.

This sequence of examples also strategically juxtaposes mediasimulations with other kinds of simulations in order to emphasize that thesimulation of media is only a particular case of computer’s general abilityto simulate all kinds of processes and systems. This juxtaposition ofexamples gives us an interesting way to think about computational media.Just as a scientist may use a simulation to test different conditions and playdifferent what/if scenarios, a designer, a writer, a musician, a film-maker,or an architect working with computer media can quickly ‘test’ differentcreative directions in which the project can be developed, as well as seehow modifications of various ‘parameters’ might affect the project. Thelatter option is particularly easy today, since the interfaces of most media-editing software not only explicitly present these parameters, but alsosimultaneously give the user the controls for their modification. For



54 Lev Manovich

instance, when the formatting palette in Microsoft Word shows the fontused by the currently selected text, it is displayed in a column next to allthe other fonts available. Trying a different font is as easy as scrolling downand selecting the name of a new font.

To give users the ability to write their own programs was a crucial partof Kay’s vision for the new ‘metamedium’ he was inventing at PARC.According to Noah Wardrip-Fruin, Engelbart’s research program wasfocused on a similar goal: ‘Engelbart envisioned users creating tools,sharing tools, and altering the tools of others’ (Wardrip-Fruin and Montfort2003: 232). Unfortunately, when Apple shipped the first Macintosh in1984, which was to become the first commercially successful personalcomputer modelled after PARC system, it did not have an easy-to-useprogramming environment. HyperCard, written for the Macintosh in 1987by Bill Atkinson (who was a PARC alumni) gave users the ability toquickly create certain kinds of applications – but it did not have theversatility and breadth envisioned by Kay. Only recently, as generalcomputer literacy has widened and many scripting languages have becomeavailable – Perl, PHP, Python, ActionScript, Vbscript, JavaScript, etc. –more people have started to create their own tools by writing software. Agood example of a contemporary programming environment – which iscurrently very popular among artists and designers and which, in my view,is close to Kay’s vision – is Processing.9 Built on top of the Javaprogramming language, Processing features a simplified programmingstyle and an extensive library of graphical and media functions. It can beused to develop complex media programs and also to quickly test ideas.Appropriately, the official name for Processing projects is ‘sketches’.10 Inthe words of Processing initiators and main developers Ben Fry and CaseyReas, the language focuses ‘on the “process” of creation rather than endresults’.11 Another popular programming environment that similarlyenables the quick development of media software is MAX/MSP and itssuccessor PD developed by Miller Puckette.

ConclusionThe story I have just related could also be told differently. It is possible toput Sutherland’s work on Sketchpad in the centre of computational mediahistory; or Engelbart and his Research Centre for Augmenting HumanIntellect, which throughout the 1960s, developed hypertext (independentlyof Nelson), the mouse, the window, the word processor, mixedtext/graphics displays, and a number of other ‘firsts’. Or we could shiftfocus to the work of the Architecture Machine Group at MIT, which since1967 was headed by Nicholas Negroponte (and in 1985 became known asThe Media Lab). We also need to recall that by the time Kay’s LearningResearch Group at PARC fleshed out the details of GUI and programmedvarious media editors in Smalltalk (a paint program, an illustrationprogram, an animation program, etc.), artists, film-makers, and architectswere already using computers for more than a decade and a number oflarge-scale exhibitions of computer art were displayed in major museumssuch as the Institute of Contemporary Art, London; the Jewish Museum,New York; and the Los Angeles County Museum of Art. And certainly, interms of advancing techniques for visual representation enabled by


9. http://www.processing.org.

10 http://www.processing.org/reference/environment/.

11 http://www.processing.org/faq/.



computers, other groups of computer scientists had already made importantadvancements. For instance, at the University of Utah, which became themain place for computer-graphics research during the first part of the1970s, scientists were producing 3D computer graphics much superior tothe simple images that could be created on the computers being built atPARC. Next to the University of Utah, a company called Evans andSutherland (headed by the same Ivan Sutherland who was also teaching atUniversity of Utah) was already using 3D graphics for flight simulators –essentially pioneering the type of new media that is now called ‘navigable3D virtual space’.12

The reason I decided to focus on Kay is his theoretical formulations thatplace computers in relation to other media and media history. WhileVannevar Bush, J.C. Licklider, and Douglas Engelbart were primarilyconcerned with augmentation of intellectual and, in particular, scientificwork, Kay was equally interested in computers as ‘a medium of expressionthrough drawing, painting, animating pictures, and composing andgenerating music’ (Wardrip-Fruin and Montfort 2003: 393). Therefore, ifwe really want to understand how and why computers were redefined as acultural media, and how the new computational media is different fromearlier physical media, I think that Kay provides us with the bestperspective. At the end of the 1977 article that served as the basis for ourdiscussion, he and Goldberg summarize their arguments in the phrase,which in my view is the best formulation we have so far of whatcomputational media is artistically and culturally. They call the computer‘a metamedium’ whose content is ‘a wide range of already-existing andnot-yet-invented media’. In another article published in 1984, Kay unfoldsthis definition. By way of conclusion, I would like to quote this longerdefinition, which is as accurate and inspiring today as it was when Kaywrote it:

It [a computer] is a medium that can dynamically simulate the details of any othermedium, including media that cannot exist physically. It is not a tool, though itcan act like many tools. It is the first metamedium, and as such it has degrees offreedom for representation and expression never before encountered and as yetbarely investigated.

[original emphasis] (Kay 1984: 41).

Referenceshttp://www.research.philips.com/newscenter/pictures/display-mirror.html. Accessed 20

January 2005.Bolter, Jay and Grusin, Richard (1999), Remediation: Understanding New Media.

Cambridge, MA: The MIT Press.Canfield Smith, David et al. (1982), ‘Designing the Star user interface’, Byte, 4: 1,

pp. 242–82.Engelbart, Douglas C. (1968), Live public demonstration of the NLS at the Fall Joint

Computer Conference, Convention Center in San Francisco, available online athttp://sloan.stanford.edu/MouseSite/1968Demo.html. Accessed 18 April 2007.

Gassee, Jean-Louis and Rheingold, Howard (1991), ‘The evolution of thinking tools’,in Brenda Laurel (ed.), The Art of Human-Computer Interface Design, Boston, MA:Addison-Wesley Professional.


12. For more on 3Dvirtual navigablespace as a newmedia, or a ‘newcultural form’, seechapter on‘Navigable Space’in The Language ofNew Media.


Hoshimo, Takashi (2005), ‘Bloom time out east’, ME: Mobile Entertainment, 9(November), p. 25, available online from http://www.mobile-ent.biz. Accessed 18April 2007.

Kay, Alan (1972), ‘A personal computer for children of all ages’, Proceedings of theACM National Conference, Boston, August.

Kay, Alan (1984), ‘Computer Software’, Scientific American, 251(3):41-47, September1984.

———— (1987), Doing with Images Makes Symbols, videotape, University VideoCommunications, available from http://www.archive.orgAccessed 18 April 2007.

———— (1990), ‘User interface: A personal view’, in Brenda Laurel (ed.), The Art ofHuman-Computer Interface Design, Boston, MA: Addison-Wesley Professional, pp. 191–207.

———— (1993), ‘The early history of Smalltalk’, (HOPL-II/4/93/MA).http://gagne.homedns.org/~tgagne/contrib/EarlyHistoryST.html Accessed 18 April2007.

Kay, Alan and Goldberg, Adele ([1977] 2003), ‘Personal dynamic media’, in NoahWardrip-Fruin and Nick Montfort (eds), New Media Reader, Cambridge, MA: TheMIT Press, pp. 391-404.

Licklider, J.C. ([1960] 2003), ‘Man–machine symbiosis’, in Noah Wardrip-Fruin andNick Montfort (eds), New Media Reader, Cambridge, MA: The MIT Press.

Manovich, Lev (2001), The Language of New Media, Cambridge, MA: The MIT Press.Processing programming language, available from http://www.processing.org, http://www.processing.org/reference/environment/ and http://processing.org/faq/.

Accessed 20 January 2005.Sutherland, Ivan ([1963] 2003), ‘Sketchpad: a man-machine graphical communication

system’, in Noah Wardrip-Fruin and Nick Montfort (eds), New Media Reader,Cambridge, MA: The MIT Press.

Waldrop, M. Mitchell (2001), The Dream Machine: J.C.R. Licklider and the RevolutionThat Made Computing Personal, London: Viking.

Wardrip-Fruin, Noah and Nick Montfort (eds) (2003), New Media Reader, Cambridge,MA: The MIT Press.


56 Lev Manovich




Convergence by means of globalizedremediation

Arild Fetveit

AbstractProphecies of media convergence have been a key component of recentdiscussions of digitalization. It has been claimed that the manipulability ofdigital data facilitates convergence and allows an erosion of differencesbetween media. This article questions these assumptions by showing howthere are also considerable obstacles against such manipulability, as wellas against the erasure of differences between media. By examiningempirical developments, as well as arguments put forward by theorists likeFriedrich Kittler, Jay Bolter, Lev Manovich and Rosalind Krauss, it ismaintained that what we are seeing is more a proliferations of media thana convergence leading to their unification. In part, this is due to ouraffection for a multiplicity of media. However, one way in which media dobecome similar is by increasingly being remediated on a digital platform.Thereby they become subject to a globalization effect by having theirfunctionalities augmented by basic traits of the computer.

‘[w]hat used to be cinema’s defining characteristics have become just the defaultoptions, with many others available.’

(Lev Manovich, The Language of New Media, 2001: 293)

In the introduction to Grammophone, Film, Typewriter, the German mediatheorist Friedrich Kittler stipulates a convergence, which will eraseprevious differences among media. He writes:

Once movies and music, phone calls and texts reach households via optical fibercables, the formerly distinct media of television, radio, telephone and mailconverge, standardized by transmission frequencies and bit information […]. Thegeneral digitalization of channels and information erases the differences amongindividual media.

(Kittler [1986] 1999: 1)

The gist of Kittler’s prediction of twenty years ago still seems valid inimportant ways. Reports on media convergence have become a dailyfeature in the news. We hear about Yahoo expanding their servicestowards television and mobile phones, about Motorola making a wirelessphone featuring a customized Google search service, about Apple’s newiPhone, which is a combination of mobile phone, video iPod, and Internetdevice, and so on. Media as well as media companies migrate ever more

KeywordsConvergenceRemediationDigitalizationPost-mediumconditionNew mediaMedia aesthetics


seamlessly into neighbouring areas, and telephone companies do thesame.

One of the more salient illustrations of convergence is actually offeredby the mobile phone. Through the years, a number of gadgets have beenoffered as metamedium machines carrying a wide set of media functions,but nothing has been as successful as the mobile telephone. Besides atelephone service, this universal media machine now often offers televisionclips, music, radio, e-mail, web browsing, and computer games. It takesphotographs, records video, and handles the new media SMS and MMS,just as it may offer additional non-media features like an alarm clock,calendar, notebook, calculator, and even penlight. The scope is impressiveconsidering its small size, but many of its functions offer a quality that israther limited, much like the scissors on a Swiss Army knife. Regarding itsphotographic and web-browsing capabilities, for example, it remains moreof a toy than a serious media machine. Thus, on a number of such levels,we need more specialized devices. Still, the mobile telephone compensatesfor its low-fidelity quality in a number of areas by means of versatility andportability.1

The remediation of these media to our mobile telephones makes pickingup our photographic camera to shoot still images or our video camera toshoot moving images, much the same thing.2 When these devices areremediated to our phones as the same physical object, choosing betweenthem now becomes a matter of selecting between two settings on thecamera menu of the telephone. If differences among media are merelybecoming a matter of alternative settings in the same software running onuniversal media machines, are the differences between individual mediathen disappearing? Kittler does not only claim an erasure of the differencesamong individual media. In continuation of the quote above, he alsoprojects the erasure of the very concept of medium:

Inside the computers themselves everything becomes a number: quantity withoutimage, sound, or voice. And once optical fiber networks turn formerly distinctdata flows into a standardized series of digital numbers, any medium can betranslated into any other. With numbers, everything goes. Modulation,transformation, synchronization; delay, storage, transportation; scrambling,scanning, mapping – a total media link on a digital base will erase the veryconcept of medium.

(Kittler [1986] 1999: 1–2)

In the following, my question will be: to what extent and in which wayscould these predictions be said to describe the situation now emerging? Towhat extent is a convergence of media taking place and what form does ittake? Do formerly distinct media converge so that the differences amongthem in fact disappear, and does this imply an erasure of the concept ofmedium? There has in fact been talk of a ‘post-medium condition’ and a‘post-media aesthetics’. These notions have respectively been brought upby the art historian Rosalind Krauss (1999a, 1999b), and by the mediatheorist Lev Manovich (2001b). What do such notions entail, and whatbearing do they have on the convergences addressed here and on the futureof the concept of medium?


58 Arild Fetveit

1. The quality of mobiletelephone cameras issteadily increasing,especially in terms ofstorage andresolution. Lenseswill in most cases notbe allocated muchspace, however, andthis is likely toremain the mostimportant limitationof phone cameras. Ingeneral, producerswill seek to increasethe quality until itreaches a certainlevel of customersatisfaction. In termsof photographs, anew low-fidelityaesthetic partlypromoted by thelimited quality of theearly mobile-phonephotograph haslowered the demandson photographs. Thismight be seen as partof a bigger picture inwhich a low-fidelityaesthetic of mobilityhas gainedprominence, forexample, in televisionspearheaded byreality shows.

2. The concept ofremediation wasintroduced’ by Bolterand Grusin and theyhave kept itsdefinition ratheropen. They describeremediation as ‘therepresentation of onemedium in another’,and argue that‘remediation is adefiningcharacteristic of thenew digital media’(Bolter and Grusin1999: 45). Thisallows remediation tobe about theborrowing of formaltraits, as well as therecirculation ofstories and charactersas in adaptationswhere material from



Before addressing Krauss and Manovich’s notions, I will first confrontthe widespread perception that with digitalization, ‘everything goes’, interms of data manipulation, and second, I will look at how media interact,in terms of competition, symbiosis, and remediation. On this background,I move on to address the concept of medium itself, before concluding witha reassessment of the shape that convergence actually seems to take today.

With digitalization – ‘everything goes’The judgement that digitalization brings unlimited freedom, where‘everything goes’, as Kittler claimed in the 1980s, is still surprisinglycommon, in spite of, among others, Manovich’s demonstrations (2001a:138) of how laborious, ‘time-consuming and difficult’ digital compositing,for example, may be. Such a notion of freedom is, for example, still MarkB.N. Hansen’s bid on the question of what difference the digital makes, inhis book New Philosophy for New Media (2004). His answer is that‘digitalization allows for an almost limitless potential to modify the image,that is, any image’ (Hansen 2004: 31). The qualifier ‘almost’ could indicatesome moderation to this claim. But after Hansen takes Manovich to task for‘correlating new media with earlier media types’, like the cinema and, tosee new media as strongly influenced by earlier media, Hansen goes on tocharacterize new media by their ‘total material fluidity’, because ‘ratherthan being anchored to a specific material support, new media are fullymanipulable, digital data’ (Hansen 2004: 31). Thus, it appears that Hansengrounds his ‘new philosophy for new media’ on an empirically dubious andhyped conception of such media.

In order to correct our perception of what digitalization has brought, itis time to reassess and clarify the view of new media as ‘fullymanipulable’, and focus not only on the manipulability of digital data, butalso on the various limitations to which such a manipulability is subject.The efforts and achievements of the companies producing digital effectsfor cinema give a good indication of how difficult digital data is to mouldand shape. Rendering hair, water, storms, lava from volcanoes, not tomention altering and controlling expressions on the human face, stillrepresent daunting and costly challenges for highly specialized digital-effects companies. Each achievement tends to elicit explicit celebrations:inside the production environment (in journals like AmericanCinematographer and Cinefex), within professional organizations likeSIGGRAPH (devoted to the development of computer graphics), as wellas in public (when marketing new movies featuring spectacular effects).3

Thus, talk of digital media as ‘fully manipulable’ belies the fact that anymeaningful alteration of a photographic image in some way comes upagainst the challenge of drawing. An exploration of to what extent the richset of effects and tools developed for controlling the alteration ofphotographs can aid us in overcoming this challenge is worthy of a studyin its own right.

A second limitation to the manipulability of digital media files is relatedto the fact that institutional players in the field work to limit our freedom tomodify and edit the material we can access via our computers. In order toget a clearer picture of this, it might be useful to consider key aspects wherethe computer differs from earlier media.

2. Kieslowksi hasreferred to Welles as one medium is usedin another. Theconcept, according toBolter and Grusin,also ‘express[es] theway in which onemedium is seen byour culture asreforming orimproving uponanother’ (Bolter andGrusin 1999: 59).This way of definingthe concept alsoallows anydigitalization of amedium to be aremediation of theprevious version ofthe medium. In thefollowing I will drawupon these meaningsfreely, withoutattempts to delimit itsmeaning beyondwhat the flow of theargument will effect.

3. In fact, this meansthat creativity in thenew prolific areabetween live-actioncinema andanimation to a greatextent takes place byway of softwaredevelopment. Thisalso has a bearing onthe ways in whichaesthetic ideas andvisions are spreadtoday. New codemight be written for aspecial project, likeThe Matrix (1999) orLord of the Rings(2001), only toreappear as a set offeatures in editingsoftware, whichsecures viral spreadof an aesthetic effectacross visual culture,though often indiluted versions. Thedynamics of thisfield, in whichprogrammers seek tomake digital datamanipulable inaestheticallyproductive ways, isworthy of a study inits own right.


Medium of display and medium of storage The computer ensures a radical separation between what we might call themedium of display and the medium of storage, initiated by the advent ofmechanical mass production. A traditional medium like painting knew noseparation between the two. Velasquez’s painting Las Meninas (1656) isboth displayed and stored by the singular painting hanging in the Museo delPrado in Madrid. Sculpture is a somewhat more complicated case:sculptures carved in stone, such as Michelangelo’s Statue of David (1504),combine the storage and display function in the original sculpture, like thepainting, in this case to be inspected in the Accademia Gallery in Florence.

With the development of mass-production technologies related to itemslike bronze sculptures, printed books, graphical prints, a separationbetween the storage and display mediums develops, to the extent that castsand presses are stored.4 In the case of photography, the Daguerreotype andthe later Polaroid photograph still combine storage and display functionsmuch as in a painting. But with the positive/negative technique madepublic by Fox Talbot in 1839, a separation occurred, such that the negativebecame the privileged medium of storage for photography, while thepositive print took care of the display function (albeit the use of positiveprints in a storage function – in family albums and elsewhere – should notbe underestimated). In the case of sound, the gramophone record providesanother transitory case. It provides storage in a way that leaves the soundon display in some form, since the sound can be inspected directly on therecord for levels of intensity. This option disappears with the magnetic tape,where the medium of storage can still be held in our hand and inspected forpossible damage, but where the information stored has moved beyond therange of human perception. The computer radicalizes this, especially whenwe have dispensed with floppy disks and the like, and increasingly use ourhard disks for storage. Then we end up first of all seeing the stored worksin terms of icons and file names.

The development sketched above, I believe, has been vital in inspiringcurrent talk of dematerialization. However, what the examples aboveclearly indicate is that any general notion of dematerialization is missingthe point. What is dematerialized in a certain way is our media of storage,in the sense that they are no longer physical objects we can hold in ourhands as much as they are data codes kept in computer files.

This dematerialization and standardization allows data to be stored in acomputer and enables smoother operations of translation between formats,as well as further possibilities for editing and manipulation. When acomputational device also becomes connected to other devices throughinterconnected networks, file transfer becomes easy, and allows an instantmaterialization of various data on a number of connected display unitsacross the globe, notably screens and speakers. This allows for an increaseduse of the computer as a general media player, which in turn provokesproducers to secure a separation between the two features discussed above,that of the display function and what we in terms of the computer might callthe storage and manipulation function, because what we can store in acomputer we also, for the most part, can edit.


60 Arild Fetveit

4. Such technologiespartly dilute the ideaof a unique original,rendering the‘original’ just oneamong many. But nomatter how manythere are, they arestill consideredoriginals. Thus, thedistinction betweenoriginal and copy isanother matter. It isnot based upon massproduction, but uponreproduction of anoriginal by what issomehow regarded asinferior means. Thecopy may, forexample, not becreated from theoriginal cast at thetime of the others,not be signed by theartist, or it mayderive from analternativetechnology, like aphotographicreproduction of an oilpainting, for example(see Benjamin 1973).



Copyright and the wall between editing and displayIn the development of the computer towards a metamedium – with anincreased ability to remediate other media and show us films, televisionprogrammes, newspapers, and journals – the storage and manipulationcapabilities of the computer are often left unengaged or explicitly lockedout. We are often left with the option of displaying media content that wecannot change: to play movies or television programmes, to displaynewspaper or journal articles, and so on. Sometimes we can store mediafiles on our computer; at other times we can neither download, nor storethem. Very often, even if we can store them, editing is cumbersome anddifficult, much more so than the technological possibilities of the computeritself would suggest, and certainly more than notions of ‘totalmanipulability’ indicate. However, a number of software agents offer keysto unlock this wall, at least partly, thus allowing us to copy and share audio-visual and other files. The competence of users also varies ratherdramatically in terms of how skilled they are in finding the right softwareor in hacking their way beyond the limitations of this wall.

A feud is ongoing in this field between copyright holders and encryptionexperts on the one side, and users, software agents, and hackers on theother. The film and music industries, especially, are trying to ensure thattheir material is not distributed for free or re-edited and used in ways thatthey do not approve of. However, they are at the same time increasinglyeager to stimulate user activity in a number of ways, in order to profit fromthe user-driven productivity sometimes talked about under the rubric Web2.0. The strengthening of such an interest represents a major trend in thecurrent entertainment industry. This trend produces a certain slack in termsof how forcefully copyright holders seek to prevent attempts to play withand manipulate their material. Audience activity takes place under thewatchful eye of a producer who is both happy to see people supply theirlabour for free while adding value to his product – but is also nervous,since his lack of control means he cannot dictate exactly how the productwill turn out, or where all the revenue might end up. A major challenge formedia producers is to accommodate, absorb, and utilize this creative vogueempowered by the digital democratization of the means of production.Arvidsson and Sandvik discuss this challenge related to computer games inthe present volume. Henry Jenkins (2003, 2006) has discussed it in termsof audience participation in popular culture. Relating to this, there is also atrend in the art world where generating and reworking social relations areat issue, as discussed by Nicolas Bourriaud (2002) under the term‘relational aesthetics’. In short, ours is not only a time of convergence, butalso of increased democratization of cultural production. In fact, the twoare linked, in that the technological convergence makes it easier to mix,edit, and play with various files, in spite of the limitations I have pointed toabove. But let us now take a closer look at what convergence entails, whatshape it seems to take, and how far it appears to go.

Convergence – competition and symbiosisBolter and Grusin (1999: 47–48) note how ‘television and the World WideWeb are engaged in an unacknowledged competition in which each nowseeks to remediate the other. The competition is economic as well as



aesthetic; it is a struggle to determine whether broadcast television or theInternet will dominate the American and world markets.’ Much like theearlier struggle between television and film, the relationship betweentelevision and the World Wide Web is not only one of struggle, but also oneof tactical cooperation and symbiosis.

An illustration of this can be found in television’s rapid expansion tointegrate mobile telephones and web pages into their programmes. Thisexpansion gives us, not television per se, but a multimedial circuit oftelevision/mobile/Web, which demonstrated its revenue-generatingpotential with Big Brother. This is a set-up that has been adopted innumerous programmes since, in later reality-based programmes like PopStars as well as in a host of other shows, enticing the audience to engage inSMS voting, thereby creating substantial extra revenue. Fictionalprogramming has become multimedial too, through offering websites thatdraw the audience into web-based chats, games, podcasts and ancillaryproducts of various kinds. In most cases this leaves a main medium likeradio, film, or television in the leading position, being supported andexpanded by new points of access and supplementary products.5

But the tendency is not only one of symbiosis between distinct media. Itis also one in which a series of remediations allow established media likenewspapers, photography, radio, film, and television to free themselvespartly from the media technologies in which they were developed andmigrate across any channel open to them. The technological convergencefacilitates these media’s reappearance on the screens of our computers, ourmobile telephones, our PDAs (personal digital assistants), our iPods orwherever they can be displayed. Texts, sounds, and images becomedetached from the media technologies that used to support them, andperhaps even gave birth to them (like in the case of photographic emulsionand the celluloid film strip), and migrate and multiply on technologicalplatforms flattened by digital technology. At the same time, texts as well asmedia seem to excel in exercises of remediation, in which they mirror andcite each other and simulate each other’s characteristic features, as Bolterand Grusin note in terms of television adopting aspects associated with webpages and the reverse. What seems to appear is a double move in which thetechnological convergence supports a tendency toward remediation andrecirculation which also entails elements of aesthetic convergence, but atthe same time, medium specificity also seems to be maintained through therecirculation of specific medial characteristics. This prolific activity ofremediation raises obstacles for any theory of convergence, leading to theend of media, a post-medium situation in which all media converge intoone super medium capable of performing the functionality of everyprevious medium, or at least, capable of satisfying much of our urge formediation.

Now, to get a handle on the complexity and the conflicting tendenciescharacterizing convergence, it is useful to distinguish between the variouslevels on which it takes place.6 First, digitalization itself, of course, rendersmedia texts into data files that, to a substantial degree, are easily combinedand circulated. This is the technical convergence that forms an importantbasis for other kinds of convergence. But to which extent does this situationentail convergence on other levels as well – on the level of cultural forms;


62 Arild Fetveit

5. John Caldwell pointsto high ambitions inthe case of the earlycross-platform workfor the televisionseries Homicide(NBC), but notes thatthis level of ambitionhas become rare(Caldwell 2003: 129).There is a tendencyto use the Internet tosend viewers totelevision and viceversa. Internet sitesalso allow theaudience to engagewith the story worldsin multiple ways,through games,discussion groups,and additions to thefictional world (likeplot expansions andcharacter backgroundstories not used in thedramatic production).They might alsoallow producers totap fan cultures forideas and usefulmaterial. There havealso been attempts tomake the media equalin importance in theunfolding of a story,for example in theuse of radio and theInternet (seeNeumark 2006).

6. Jenkins (2001: 1)argues that, ‘Part ofthe confusion aboutmedia convergencestems from the factthat when people talkabout it, they’reactually describing atleast five processes.’I second Jenkins’scall for distinctionsbetween levels whenaddressingconvergence, andthree of his fivelevels coincide withthe distinctions I haveopted for. InJenkins’s view, theconvergence can be(1) technological, (2)economic, (3) social



on the level of aesthetics; on the level of the physical devices that deliverour media; and on the level of industrial actors?

Clearly, industrial actors are merging on a grand scale, AOL and TimeWarner, Sony and Ericsson, Google and YouTube, to mention just a few, orthey are making exclusive or semi-exclusive partnerships, like Apple withGoogle, Cingular, and others. The more thorny issues to assess are those ofaesthetics and cultural forms, and the level of the physical devices. In termsof the last, it is clear that the computer is aspiring to the condition of ametamedium, a machine which can display and handle all other media insome sense, while the mobile telephone effectively demonstrates thismultimediality with the help of its onboard mini-computer. On thisbackground, industry operators are pondering whether consumers will besatisfied by just carrying a multimedia mobile phone around, as well as adigital music player and a PDA. Or, if we look a few more years ahead, willour present devices be replaced by, or grow into, portable computers the sizeof mobile phones, with virtual keyboards and roll-out screens?7 When itcomes to our homes, will we continue to have radios, musicsystems/players, newspapers, magazines, televisions, and computers, or willwe be satisfied with one or more multimedial super computer(s) taking careof it all? The possible future of a multimedial super computer that canreplace the television and the computer by combining the functionalities ofboth is much a matter of cost and of practicalities. For an expansion of ourpresent televisions to allow us to browse the Internet and to have similarfunctionalities as our computers, they need to be equipped with basiccomputer components, which would incur additional costs. When it comesto our computers, they already handle video and audio streams, so minimaladjustments would seem to be necessary, except for the reading of analoguesignals (which requires an additional tuner), but as the analogue net is aboutto be replaced by digital, this is hardly an issue. This means that socialpreferences may be more important than technical possibilities for assessingthe effects of the convergence. Thus, one of the key questions becomes, willwe want to have our work desk also be the site for aesthetic experience andentertainment, or will we move the device between our workspace and ourliving-room relaxation area according to our needs? Actually, it seems morereasonable to assume that we will have different devices in different places,tailored to the various uses in those places, and that those devices will havea degree of specialization rather than full multifunctionality.

The more complex issue is the convergence of aesthetics and culturalforms respectively. This is only to a limited degree a matter ofdigitalization, as cultural forms intermix and imitate each other throughouthistory. This issue has in fact been the subject of ongoing debates at leastsince Horace pointed to the correspondence between painting and poetry inhis Ars Poetica, condensed in the phrase ‘ut Pictura Poesis’, (as is painting,so is poetry). This observation has provided a major reference point forsuch debates, especially after Gotthold Ephraim Lessing made it a keytarget of criticism in his Laocoön ([1766] 1984), followed up in IrvingBabbitt’s The New Laokoon: An Essay on the Confusion of the Arts (1910),and with substantially greater impact in Clement Greenberg’s ‘Towards aNewer Laocoon’ ([1940] 1985), which may be the first powerfularticulation of Greenberg’s view of modernism.8

2. Kieslowksi hasreferred to Welles as or organic, (4)cultural and (5)global. Hisdescription of theprocesses is crisp andhis idea of ‘social ororganic convergence’original (Jenkins2001: 1). Social ororganic convergenceis a matter of what hecalls consumers’‘multitaskingstrategies fornavigating the newinformationenvironment’, and itoccurs when ‘a highschooler is watchingbaseball on a big-screen television,listening to techno onthe stereo, wordprocessing a paperand writing e-mail tohis friends’ (Jenkins2001: 1). This alsorelates to aneconomicconvergence in whichJenkins locates theexploitation of‘synergies’, and acultural convergencewhere he finds‘transmediastorytelling, thedevelopment ofcontent acrossmultiple channels’(Jenkins 2001: 1).The processes ofconvergence areintertwined andentangled inJenkins’s shortdescription, but Ithink he does well intrying to separatethem out. If a sixthprocess should beadded to Jenkins’sscheme, aestheticconvergence could bea strong candidate.


Greenberg is of special interest in this context, because, as I soon willshow, he develops a conception of modernity that seems surprisinglyinterlinked with Bolter and Grusin’s concept of remediation. Greenbergimplicitly suggests that the convergence of aesthetic forms take place withshifting intensity historically, and that the defining aesthetic of a periodmight first of all be characterized by undoing the effects of such aconvergence. This is actually how he defines modernism. The definitionsprings from a concern with the way in which art forms may compromisetheir own nature in an effort to absorb the effects of, and to imitate thecharacteristics of, other art forms, particularly the dominant art form of theperiod. He says,

There can be, I believe, such a thing as a dominant art form; this was whatliterature had become in Europe by the 17th century. […] Now, when it happensthat a single art is given the dominant role, it becomes the prototype of all art: theothers try to shed their proper characters and imitate its effects.

(Greenberg [1940] 1985)

Greenberg comes to define modernism as a process of undoing the effectsof such imitation, of purging the arts and their media from the impuritycaused by the borrowing and absorptions of effects from other media. Thisprocess, he points out, becomes one of consolidating or entrenching an artform in its own area of competence. In ‘Modernist Painting’, Greenbergwrites,

The essence of modernism lies […] in the use of the characteristic methods of adiscipline to criticize the discipline itself, not in order to subvert it, but in order toentrench it more firmly in its area of competence. […] It quickly emerged that theunique and proper area of competence of each art coincided with all that wasunique in the nature of its medium. The task of self-criticism became to eliminatefrom the specific effects of each art any and every effect that might conceivablybe borrowed from or by the medium of any other art. Thus would each art berendered ‘pure’ […].

(Greenberg [1960] 1993)

Thus, it seems that Greenberg is keenly interested in what we today aremore likely to call ‘remediation’, which may be described as the adoptionor absorption of the effects of one medium in another. In short, and fromthis perspective, it appears that Greenberg is describing modernism asremediation in reverse. The idea of modernism is to undo the effects of aprevious remediation, which has levelled the differences between art forms,because these promiscuous arts thrive in imitating each other’s effects, thatis, their artistic means.

It appears that we find ourselves again in a promiscuous time wheremedia borrow effects from each other, not only because we may still beunder the influence of a postmodern counter-reaction against modernism’spenchant for the purity of media forms, but also because digital technologyencourages the playful recombination of media. This is one of the keyfeatures computer pioneers like Alan Kay sought to develop. The capacityof the computer for sampling and recombining various textual elements, its


64 Arild Fetveit

7. The cost and the sizeof computers are stillshrinking. But as theelectronics of thecomputer getssmaller, the two partsthat must also shrinkare the keyboard andthe screen. A virtualkeyboard, which isprojected on a surface(preferably a table),is already developed,so is a screen thatmay be rolled upwhen not in use.Screen projectioncould also be viable,but has thedisadvantage ofrequiring areasonably whitesurface whereuponthe image may bebeamed. Therefore,chances are that wewill still use laptopslike today’s, onlylighter and withbetter screens,batteries, and biggermemory capacity.Chances are also thatthe divide betweenhigh-fidelity and low-fidelity performancewill become moreinstitutionalized, sothat we have both ahigh-fidelity servicefrom our laptops andstationary computers,as well as a low-fidelity version thatcan fit in our pocket,whether it becameras, browsers, e-mail services andmore. This also callsfor better systems ofintegration betweenthe two kinds ofdevices. Apple mayhave taken animportant step in thisdirection by havingthe iPhone run OS X,the same operatingsystem that runs ontheir computers.

8. The convergence ofaesthetics and



enhanced potential for post-production remixing of previous material, hascontributed to an aesthetic of sampling which we can also see as adding tocultural convergence, where for example the art world and the world ofmedia entertainment come closer. However, in this field of intermix, mediahardly disappear. In fact, the aesthetics of sampling even allow old andoutdated media forms and modes to be reborn into new contexts and tothrive. Yet another reason why media do not disappear so easily is our lovefor them, even when they may seem old and outdated. Their aestheticsseem to live on as they are utilized and recirculated in our digital phonecameras for example, which may have settings for sepia or solarization.Old film stock and the Super 8 format are often found in films and musicvideos. These anachronistic appearances testify to some level of truth inMarshall McLuhan’s statement ([1964] 2001) that ‘the medium is themessage’, in that, at times, the specific medial quality may be of keyimportance to the aesthetics developed. In order to communicate in a richway, we mobilize and use a number of media formats, in part because theyhelp us articulate ourselves in rich and interesting ways.

Another obstacle for a generalized convergence, as envisioned by Kittlerand others, is the lack of standardized formats, which brings us back to thevery basis of a digitally driven convergence. Arriving at a common andpractical format is just the first obstacle here. Agreeing on a standard oftenseems even more difficult, as major economic interests are at stake. Thus,we get the common situation of various forms of war between competingformats. Apple iPods, for example, employ a standard that carries moreinformation than MP3 files, and therefore, sending an iPod music file to aPC-user with an MP3 player may leave the latter without a chance to hearthe music. Similarly, the current battle between the Blu-ray Disc and High-Definition television echoes the war over formats that took place with theestablishment of VHS as the standard video technology in popular use.Computer games continue to confront each other across a number ofcompeting consoles, like Playstation 2 and 3, Xbox, Nintendo Gamecubeand Wii. There is also the silent war going on between telephone companiesand emerging Internet telephone providers that may eventually offermobile services (Apple’s partnership with Cingular might be seen as anattempt to avoid such a possible standoff, by bringing the biggest mobile-telephone company on board rather than risking countermeasures fromtheir side). Industrial strategies set in motion to act against competitors andsecure profit for key players also work in part to halt technicalconvergence. These strategies and behavioural logics also defy any idealoperation of the free market, and instead seek to create and benefit frommarket situations that allow higher prizes than a freely functioning marketwould. This is the context in which we can see European regulators recentclaims that iTunes should be compatible with all MP3 players, so thatmusic bought from the iTunes Store can also play on devices from Apple’scompetitors.

In short, the tendency to produce incompatible standards may bemotivated to some extent by diverging convictions about the benefits of thevarious standards proposed, but it is also driven by the desire to undercutthe workings of a free market and protect major companies from opencompetition. Thus, even on the technological level where digitalization

2. Kieslowksi hasreferred to Welles as cultural forms could,of course, bedescribed from anumber of otherperspectives. Forinstance, aBakhtinianperspective wouldassess voicestravelling across textsand media, which isoften done by way ofthe concept ofintertextuality. AnnaEverett (2003)pursues such aperspective in thedevelopment of herconcept of‘digitextuality’,meant to describe theparticular textualform encouraged bydigital media.


would seem to ensure a common platform, the picture is much morecomplex and riddled with conflicts.

Kittler’s conception of convergence implies a notion of a post-mediumcondition of some sort, an assumption that distinct media will become lessimportant or even disappear, along with the concept of medium itself.Related notions are also put forward by Krauss and Manovich, respectively.

The post-medium condition and post-media aestheticsAfter Kittler, we seem to be confronted with two major attempts toarticulate a ‘post-medium condition’, as Rosalind Krauss (1999) calls it, orthat of a ‘post-media aesthetics’ as it is labelled by Manovich (2001b). Inher article, ‘Reinventing the medium’, Krauss discusses the shifting statusof photography in the art world and how photography in part became a toolfor deconstructing an art practice based on the specificity of the medium,be it the specificity of painting, sculpture, graphic print, or, ironically,photography itself. She argues that photography enters the art world, or, inher words, ‘converges with art’

as a means of both enacting and documenting a fundamental transformationwhereby the specificity of the individual medium is abandoned in favor of apractice focused on what has to be called art-in-general, the generic character ofart independent of a specific, traditional support.

(Krauss 1999b: 293–94)

Thus, photography is implicated in the enactment and the documentation of achange from a situation in which artists work within media (like painting,sculpture, graphical print and so on) to a situation that Krauss refers to as the‘post-medium condition’. This casts photography in a new role, that ofdocumenting various performances or ‘hors-media’ events where art practiceshave migrated beyond and outside established media forms to strategicallyand in ad hoc situations deploy various objects, actions, and events to conveyconceptual ideas. Thus, in this ‘post-medium condition’, which Krauss alsoassociates with postmodernism, practice, according to Krauss, ‘is not definedin relation to a given medium – sculpture – but rather in relation to the logicaloperations on a set of cultural terms, for which any medium – photography,books, lines on walls, mirrors, or sculpture itself – might be used’ (Krauss1999: 289). This move away from the classic media of the academic tradition(which basically is thought of as painting, sculpture and graphic print), andfrom the modernism defined by Greenberg, which explored the mediumspecificity of painting and sculpture in particular, has according to Krauss leftthe artist in a ‘post-medium condition’ to explore ‘art-in-general’.

However, this ‘post-medium condition’ does not so much take away themedia from artists as it refigures their functions and multiplies them.Because now, any medium can be used and almost anything can become amedium; even ice cubes might become media in conceptual ephemeralevents, as well as streams of air hitting the spectator’s head inside thegallery space. But elements of other medial practices could also beimplicated in medial situations which in part may redeem an interest intheir specificity, or in Krauss’s words, ‘reinvent the medium’, or have thesemedia serve ad hoc strategic purposes of different kinds.9


66 Arild Fetveit

9. Krauss herself locatessuch a reinvention ofthe medium in JamesColeman’s doubleportraits, which arephotographic, butalso invoke (orremediate) the medialforms of feature filmsand photo novels. Areinvention of themedium could also beclaimed for GerhardRichter’sphotorealisticpaintings, whichseem to both redeemand re-actualizepainting andphotography at thesame time as they arepositioningthemselves as neitherpainting norphotography. In fact,Richter himselfclaims to be doingphotography, ‘byother means’. Wemust merely learn toaccept thatphotographicemulsion and opticallenses are not integralto photography (seeRichter 1995).



On this background, Krauss’s labelling of a situation in the modern artworld – where artists no longer explore a single medium like painting andsculpture – as a ‘post-medium condition’ is somewhat ironic. The situationis partly defined by the fact that artists work across a range of differentmedia, combining them pragmatically for various purposes and effects. Wecould see this as a move from the singular to the plural, from the mediumartist faithful to the exploration of a specific means of expression, to themedia artist defined by an adulterous pragmatism using whatever mediumor combination of media that seem to work. Thus, ‘the post-mediumcondition’ might as well be labelled ‘the medium condition’, or ‘thecondition of media proliferation’, for this ‘post-medium condition’ is onein which the artist constantly comes to choose certain media among themany possible. Whereas one medium conveniently can be called painting,or sculpture – most of the time without recourse to the concept of medium– a host of different medial options tends to render every one of them a‘medium’. Under such a condition, most art might become ‘media art’, andmost artists ‘media artists’ of some sort.

A further reason why artists become media artists is related to thepolitical and cultural interventionist project of jamming – re-figuring, re-programming – important aspects of the culture articulated in ‘themedia’. And by ‘media’, in this context, I mean major outlets liketelevision, film, newspapers, media that in many ways define our culture,and that artists may feel compelled to comment on. This intervention willoften employ the media formats that are being commented on and it mayrender the art ‘media art’, both in terms of its subject matter and itsmedial means.10

The challenge from Manovich’s notion of a post-media aesthetics isconsiderably greater in that it projects an end to media as well as to theconcept of medium itself. This is first of all because Manovich’s conceptionof a post-media aesthetics is in part based on digitalization. But in hisargument, an internal tension in the concept of medium also comes to thefore, a tension that goes some way towards explaining the persistence of theconcept of medium allegedly faced with annihilation, for instance, the factthat the medium of film seems to have faint problems surviving the deathof the celluloid filmstrip. Manovich starts his dismantling of the concept ofmedium by claiming that:

In the last third of the twentieth century, various cultural and technologicaldevelopments have together rendered meaningless one of the key concepts ofmodern art – that of a medium. However, no new topology of art practice came toreplace media-based typology which divides art into painting, works on paper,sculpture, film, video, and so on.

(Manovich 2001b: 1)

Manovich cites a number of reasons why the concept of the medium, in hiswords, is ‘rendered meaningless’. First, the proliferation of new artisticforms replacing traditional art threaten old typologies of media, sometimeswith arbitrary combinations, sometimes even as dematerialized ‘conceptualart’. The introduction of modern media like video provides yet anotherchallenge in that, as Manovich writes, the


10. Within the art worlditself, however, theterm ‘media art’remains contestedand riddled withuncertainty as to whatit should mean. It hasbeen used in a morespecialized meaningto designate aparticular kind of artwhere the logic of themedium itself isexplored and wherethe medium involvedis preferably digital.From this vantagepoint strategies ofinclusion have beenattempted to integrate‘media art’ (or ‘newmedia art’) into theestablished art world(see Grau 2007).Such a conception ofmedia art has comeup against at leasttwo problems. First,it risks holding onto aGreenbergianconception of art asan interrogation ofthe medium used. Inthe face of art’s socialturn, this risksrendering ‘media art’dated and withoutcontemporarysensibility (seeBourriaud 2002).Second, theinterpretation that allart uses some kind ofmedium and istherefore bydefinition media artrisks a certainredundancy. Thecrisis for the conceptof ‘media art’ hasmanifested itself inthe removal of theconcept of mediumfrom the German artfestival transmediale– international mediaart festival, whichnow is renamedtransmediale –festival for art anddigital culture.


mass medium of television and art medium of video used the same material base[…] and also involved the same condition of perception (television monitor). Theonly justifications of treating them as separate mediums were sociological andeconomic […]. Gradually, this sociological difference in the distributionmechanisms, along with other sociological differences […], became moreimportant criteria in distinguishing between mediums than the distinctions inmaterial used or conditions of perception. In short, sociology and economics tookover aesthetics.

[original emphasis] (Manovich 2001b: 2–3)

In order to strengthen his suit, Manovich does not play the digital card earlyin his discussion. Rather, he attempts to make a strong case for a crisis of themedium existing before digitalization. And he manages to do so to someextent, but even more, his argument comes to lay bare the inherentcomplexity in the concept of medium itself. His account starts off with anassumption that the concept designates the medium’s technological base, orwhat in the language of art history and criticism is usually designated ‘thematerial support’. Eventually, however, Manovich comes to describe whatwe might call a metamorphosis in which the concept of medium comes tobe based as much on other parameters. The parameters Manovich nominatesfor distinguishing between media may actually form part of a richerconception of medium, made more urgent by these acts of disappearanceand of migration away from the initial material supports on which themedium was conceived. Apart from ‘material used’, which otherwise can bereferred to as media technology or material support, he mentions‘distribution mechanisms’, ‘conditions of perception’, and lastly, the lessclearly defined ‘other sociological differences’ (Manovich 2001b: 3).

The key observation here, which may seem paradoxical, is precisely thatthe medium of film survives so effortlessly the death of its medium, in thesense of its technical support.11 Photography as well now seems to thrive onits new digital platform; in fact, it thrives to such an extent that it seemsalmost unreal to think of the repeated death sentences it received during the1990s as its material support became destined for the technical museum.12

In the face of such changes, Manovich notes that ‘despite the obviousinadequacy of the concept of medium to describe contemporary culturaland artistic reality, it persists. It persists through sheer inertia – and alsobecause to put in place a better, more adequate conceptual system is easiersaid than done’ (Manovich 2001b: 4). If the concept persists, it is becausepeople persist in using it. And when the concept of film, for example, seemsto survive even when films are often not shot on film, it is because theconcept of medium is much more complicated than we often realize, andthis is precisely what Manovich comes to articulate in the parametersabove.

Revising the concept of mediumBasically, the concept of medium seems to involve a lot more than thematerial support it is often associated with, as can be seen in a number ofdefinitions, which Bruhn Jensen points out in his article ‘Mixed Media’ inthis volume. If we take as a vantage point the parameters Manovichsuggests, beyond the medium technology (or technical support), we have


68 Arild Fetveit

11. This is one of theconceptual issuesNoël Carrolldiscusses in the threefirst chapters inTheorizing theMoving Image (1996)

12. For photography, theloss of the materialsupport based onchemistry wasespecially powerful,because, as Barthes(1981: 80) notes,photography was notan optical inventionin the wake of theAlbertinianperspective, but achemical invention.Thus, the digitalchange seemed totamper withphotography’s owndefiningcharacteristics.Moreover,photography hadrested comfortablywithin its chemicalground through anumber of othertechnical transitions,and had hardly beensubject to very manyrevolutionary medialchanges, at least, thisI believe is how itwas perceived by thelate 1980s. What waseventuallydemonstrated by thedigitalization ofphotography – that itwas possible to liftthe medium ofphotography awayfrom its medium ofinvention and let it beremediated on adigital platform – hadalready beendemonstrated byother media, like theLP which hadbecome a CD, andcontinues to bedemonstrated,without much publicanxiety, by television.A comparativeproject might well be



‘distribution mechanisms’, ‘conditions of perception’, and the less clearlydefined ‘other sociological differences’. Without aiming for any fullycoherent conception and avoiding internal overlap, we could perhaps add‘sociocultural use’ along with the ‘aesthetic practices’ associated with suchuse.

What complicates the picture, but also makes it more interesting, is thatit seems that the concept of medium, the way that it is currently used, iscapable of migrating between these parameters which make up the conceptof medium, rendering some more important sometimes, and others moreimportant at other times. Thus, the emphasis on the various aspectsdefining a medium may be shifting. At times, the technological aspect ofthe medium seems of central import, while at other times the formal,thematic, and aesthetic aspects, or ‘distribution mechanisms’ and‘sociocultural uses’ come to the fore. W.J.T. Mitchell captures thiscomplexity well, when he fits this difficult term with the following elusivedescription:

An image appears only in some medium or other – in paint, stone, word, ornumbers. But what about media? How do they appear, make themselves manifestand understandable? It is tempting to settle on a rigorously materialist answer tothis question, and to identify the medium as simply the material support in or onwhich the image appears. But this answer seems unsatisfactory on the face of it.A medium is more than the materials of which it is composed. It is, as RaymondWilliams wisely insisted, a material social practice, a set of skills, habits,techniques, tools, codes and conventions.

(Mitchell 2005: 203)

In terms of the position of the concept, Mitchell adds, ‘[t]he concept of amedium […] seems […] to occupy some sort of vague middle groundbetween materials and the things people do with them’ (Mitchell 2005:204). But the vagueness of this ‘ground’ can be clarified, I believe, bystudying how the concept operates in actual cases. Then we might find thatdifferent aspects of the concept will be at play at different times. If we limitourselves to what it is possible to say about the medium of film, we can getan illustration of the multifaceted nature of the concept of ‘medium’, andof how its various aspects can be actualized in almost paradoxical ways. Ithas been noted that the medium of film is dying because film stock is beingreplaced by digital video. In this case, the medium of film is referred to interms of its technical support. If we counter that films survive becausepeople simply love stories told in an audio-visual format, what is referredto is the medium of film as a cultural and aesthetic form. If we say thatfilms will still die because theatres that show films lose their audiences andwill soon have to close, we are conceiving of the medium of film in termsof its primary viewing practice in theatres, more than in terms of thetechnical support and the cultural and aesthetic form.

These complexities relating to the concept of medium have led to recentattempts to clarify and to define the concept. In the article ‘Convergence?I Diverge’, Henry Jenkins proposes to distinguish between media, genresand delivery technologies. He claims: ‘Recorded sound is a medium. Radiodrama is a genre. CDs, MP3 files and eight-track cassettes are delivery

2. Kieslowksi hasreferred to Welles as undertaken here touncover to whatextent various mediahave changed theirtechnical supportsand with whatconsequences andpossible difficulties.


technologies. Genre and delivery technologies come and go, but mediapersist as layers within an ever more complicated information andentertainment system’ (Jenkins 2001). This might seem clarifying. But ifrecorded sound is a medium, and not CDs, what other media are there andis it possible to invent new ones? These questions are implicitly answeredby Jenkins five years later as he lists the following media, ‘spoken words’,‘printed words’, ‘cinema’, ‘theatre’, television’, ‘radio’ (Jenkins 2006: 14).One could try to continue this list by adding ‘book’ and ‘newspaper’. Butthese seem partly superfluous when ‘printed words’ are already mentioned.In fact, ‘book’ and ‘newspaper’ could well be regarded as mere deliverytechnologies for the medium ‘printed words’. Likewise, radio might beregarded not as a medium, but as a delivery technology for the medium‘spoken words’, or perhaps even better, for ‘recorded sound’. In short, theframework proposed by Jenkins is not entirely satisfactory, but it isproductive in provoking us to rethink the concept of medium.

The strategy behind making ‘recorded sound’ a primary example ofwhat a medium is, could be to protect the concept of medium from thecreative turmoil that takes place in the realms of technological and socialinventions on the one side, and cultural and aesthetic inventions on theother.13 But the outcome is not so much to protect the concept as toundermine its relevance. For if it merely designates allegedly stable medialforms like ‘recorded sound’, ‘printed words’, and traditional media likeradio, television, and cinema, it need not be on everybody’s lips, like it istoday.

Whatever the strategy, what is negated by making ‘recorded sound’ aprimary example of what a medium is, is the multiple aspects of theconcept, and its migratory flexibility in which different aspects seem to beemphasized at different times. Any attempt to admit the concept only onedimension, be it technical device, cultural or aesthetic practice, form ofperception, or socio-economic mode of circulation, or medial form like‘recorded sound’, belies this multiplicity.14

The inherent richness and flexibility in the concept of medium is animportant reason why the predictions of Kittler and Manovich – that theconcept will become obsolete in the wake of convergence – may not holdup. If the concept is ‘rendered meaningless’ (Manovich) in one of itsaspects, it may still persist as long as it is meaningful in terms of otheraspects. Another reason, of course, is that media forms do not so muchdisappear as they reappear, through acts of remediation and recirculation.

The conception of ‘medium’ put forward here may also help us toexplain and assess how creativity and renewal take place within a medium.Manovich suggests in his contribution to this volume that new mediaremain perpetually new because they are made up of software that is beingrenewed. This is a suggestive idea, but once we see the many aspects of theconcept of medium, we see that updates to the software first of all renewone aspect of the medium, namely its technical basis. What implicationsthis will have in terms of other aspects of the medium, its cultural andaesthetic forms, or the logic of production and distribution characterizingit, is uncertain. In fact, technical changes (for example, in terms of softwareadditions) need to be accepted, and utilized, so that they have consequenceson other aspects of the medium as well, for them to be truly effective in


70 Arild Fetveit

13. The concept ofdelivery technologiesserves to detach theconcept of mediumfrom the realm oftechnical support.The concept of genreworks to protect theconcept againstshifting aesthetic andcultural practices.

14. From his initialattempt to clarify theconcept, Jenkins hasalso found LisaGitelman’s work onthe concept useful,without having thatinterest amalgamateinto a coherentposition. Gitelmanproposes to ‘definemedia as sociallyrealized structures ofcommunication,where structuresinclude bothtechnological formsand their associatedprotocols, and wherecommunication is acultural practice, aritualized collectionof different people onthe same mental map,sharing and engagingwith popularontologies ofrepresentation’(2006: 7). Sherecognizes, that‘[d]efining media thisway admittedly keepsthings muddy’(2006: 7). In spite ofsaying, somewhatlater that, ‘the“materiality” ofmedia is one of thethings that interestme most’, herdefinition is not clearas to what extenttechnological supportor media materialityis admitted into herdefinition.



contributing to a change of the medium as a whole. This suggests thatcertain technical extensions will hardly be significant, whereas others maylay the basis for powerful reinventions of the medium, through their impacton aspects like aesthetic practices, sociocultural use, distributionmechanisms, and audience practices.

The globalization of software additionsI started this discussion by referring to Kittler’s prediction that ‘[t]hegeneral digitalization of channels and information erases the differencesamong individual media’. And I have used much of the space to raiseobjections against the powerful convergence envisioned by Kittler. I alsosuggested above, however, that the gist of Kittler’s prediction still seemssuccinct. Let me now qualify this beyond the obvious presence of themetamedia, which our computers and our mobile telephones now represent.When assessing the consequences of digitalization, instead of talking aboutthe erasure of differences, and to judge the level of convergence by theamount of differences erased, it may be as productive to talk about theaddition of similar traits, viewing this as a major source fuellingconvergence.

The remediation of most media on a digital platform is about to make allmedia ‘new media’, in the sense that they are all empowered by computertechnologies in one way or another. Hence, they are also more or lesssubject to the addition of new properties, beyond the new affordances theymay already have been granted as part of their digitalization. A basic logicof the computer is therefore coming to redefine both the production and thereception processes related to media, to the extent that computers areinvolved in media production and consumption. Manovich has commentedon this in The Language of New Media where he says

The logic of a computer can be expected to significantly influence the traditionalcultural logic of media; that is, we may expect that the computer layer will affectthe cultural layer. The ways in which the computer models the world, representsdata and allows us to operate on it; the key operations behind all computer programs(such as search, match, sort, and filter); the conventions of HCI […] influence thecultural layer of new media, its organization, its emerging genres, its contents.


In short, with the pervasive presence of computers in the handling ofmedia, the latter have come to be affected by what we can call a‘globalization effect’ in which their new affordances come to be ones whichthe computer can provide to most media, and which feature as standardmenu options in most applications.15 In other words, what we seem to getis a form of convergence by means of a globalized remediation, aremediation where previous media have been remedied by expansions totheir powers that tend to be similar. Simply put, by being more and moresupported by and interwoven with the computer, media now areincreasingly profiting from the enhancements the computer can offer. Asthe computer tends to offer similar enhancements to most media, aconvergence by means of globalized remediation takes place. A couple ofexamples may lay this out in more detail.


15. These are, of course,functions like copy,paste, search, save,but also a host ofother morespecialized optionsthat only some userswill end up using. Anillustration of thepotential in this canbe found by lookingat the plug-ins for theFirefox browser.


In its digital version, the New York Times offers photographic slideshows, film clips and music. The original newspaper, just like any printedmatter, magazine or book, cannot display time-based media, but itsremediated version on the Web can. And it becomes increasingly effectivein doing so as the capacity of digital networks increases. Film providesanother example of computer-augmented affordances. Animated films usedto be able to depict events that were impossible to relay in live-actioncinema. When live-action film is enhanced by digital productiontechnologies, however, its affordances increasingly come to involve thestunning effects of animated films, and these effects can now be offered inphotorealistic quality.16

The increased affordances generated through digitalization may be saidsimply to reflect the characteristics of the computer as a metamedium, itsability to handle most, if not all, previous media, albeit within theparameters of its interface.17 This sets in motion a development that mayeventually make most media capable of displaying most other media. Thistendency is illustrated by the broadcaster BBC which features newspaper-like articles on their websites, complementing their television and radioprogrammes, and newspapers like the New York Times which featurestelevision offerings on their website, complementing their printed matter.As Manovich puts it in his contribution to this volume, ‘it is as thoughdifferent media are actively trying to reach towards each other, exchangingproperties and letting each other borrow their unique features’. However,the ways in which this will pave out, and the actual forms it will take, stillremain to be seen given the high level of dynamism and turmoil in the field.

ConclusionAll in all, what we have before us is a complex picture in whichconvergence is a powerful tendency, continuously counteracted bypowerful counter-tendencies holding back on a full convergence that mightpromise to erase the differences among individual media so as to have usend up with merely one mediatory device. The picture is one in which anumber of obstacles – such as multiple formats, copyright holders’resistance against copying, storage, and manipulation, and not least ourattachment to media aesthetic diversity – continue resisting universalconvergence. At the same time, however, convergence works powerfully tomake files compatible, to have devices communicate with each other, andto endow media with globally shared features. This means that our manymedia do not disappear to be replaced by one, or even just a few.Nevertheless, this is no guarantee that our concept of medium will persist.

As long as we have a multiplicity of media as competing alternatives toconsult, to express ourselves in, or to comment on, we may find the conceptof medium relevant. In the heyday of painting and sculpture in the artacademy, when relatively few media alternatives were around, the need fora concept like ‘medium’ was hardly a matter of urgency. Some of thechanges in the field of media – the occurrences of new means ofcommunication, like SMS, MMS, Skype, Instant Messenger, etc. do notnecessarily urge the use of the concept of medium. We may often becontent talking about them as ‘applications’, or we may just refer to themusing names like ‘Skype’, or ‘Messenger’. In fact, the proliferation of


72 Arild Fetveit

16. This has prepared theground for a vogue ofnew digital-effectsfilms, and for comic-strip heroes enteringthe movie theatres.Commenting on thisdevelopment,Manovich (2001a:295) somewhatprovocatively statesthat ‘cinema can nolonger be clearlydistinguished fromanimation. It is nolonger an indexicalmedia technologybut, rather, asubgenre of painting’.

17. Specific mediamaterialities arethereby lost in thisremediation except tothe extent that theycan be representedwithin the operativemode of thecomputer. It is, forexample, difficult totell that most ofGerhard Richter’spictures are paintingsrather thanphotographs in theirphotographicallymediatedreproduction on acomputer screen.



media in the guise of new applications, which digitalization has brought,may represent a new threat to the concept of ‘medium’, in that we just donot conceive of ‘medium’ as the proper descriptive word for these newmedia. Thus Kittler’s predictions about the erasure of ‘the very concept ofmedium’ may be well worth reassessing in another ten or twenty years. Inthe mean time, we are likely to see more bids on how the concept shouldbe defined.

AcknowledgementsI wish to thank Liv Hausken, Ulrik Ekman, Kiersten Leigh Johnson andFrancesco Lapenta who have read the article in draft form and proposedvaluable adjustments. I am also grateful for the productive interchangesoffered by my colleagues in the Media Aesthetics research project at theUniversity of Oslo, supported by the Norwegian Research Council throughthe programmes KULFO and KIM, and last but not least, I am thankful forgreat discussions with my inspiring students in the Department of Media,Cognition, and Communication, University of Copenhagen, and with mycolleagues in the same department.

ReferencesArvidsson, Adam and Sandvik, Kjetil (2007), Gameplay as Design: Uses of Computer

Players’ Immaterial Labor, London: Intellect Press. Babbitt, Irving (1910), The New Laokoon: An Essay on the Confusion of the Arts,

Boston: Houghton Mifflin Company. Barthes, Roland (1981), Camera Lucida: Reflections on Photography, New York: Hill

and Wang. Benjamin, Walter (1973), ‘The work of art in the age of mechanical reproduction’, in

Illuminations, London: Collins (Fontana).Bolter, Jay and Grusin, Richard (1999), Remediation: Understanding New Media,

Cambridge, MA: MIT Press. Bourriaud, Nicolas (2002), Relational Aesthetics, Paris: Les Presse Du Reel.Caldwell, John (2003), ‘Second-shift aesthetics’, in Anna Everett and John T. Caldwell

(eds), New Media: Theories and Practices of Digitextuality, London and New York:Routledge.

Carroll, Noël (1996), Theorizing the Moving Image, Cambridge: Cambridge UniversityPress.

Everett, Anna (2003), ‘Digitextuality and click theory: Theses on convergence media inthe digital age’, in Anna Everett and John T. Caldwell (eds), New Media: Theoriesand Practices of Digitextuality, London and New York: Routledge.

Gitelman, Lisa (2006) Always Already New: Media, History and the Data of Culture,Cambridge, MA: MIT Press.

Grau, Oliver (2007), MediaArtHistories, Cambridge, MA: MIT Press.Greenberg, Clement ([1940] 1985), ‘Towards a newer Laocoon’, in Francis Frascina

(ed.), Pollock and After: The Critical Debate, London: Paul Chapman.________ ([1960] 1993), ‘Modernist painting’, in John O’Brian (ed.), The Collected

Essays and Criticism, vol. 4, Chicago: University of Chicago Press.Hansen, Mark B.N. (2004), New Philosophy for New Media. Chicago: University of

Chicago Press.Horace ([18 BC] 1989), Epistles Book II and Epistle to the Pisones (‘Art Poetica’), ed.

Niall Rudd, Cambridge: Cambridge University Press.



Jenkins, Henry (2001), ‘Convergence? I diverge’, Technology Review, 1 June, availablefrom http://www.technologyreview.com/Biztech/12434/. Accessed 19 February2007.

———— (2003), ‘Quentin Tarantino’s Star Wars? Digital cinema, media convergence,and participatory culture’, in David Thorburn and Henry Jenkins (eds), Toward anAesthetics of Transition, Cambridge, MA: MIT Press.

———— (2006), Convergence Culture, Cambridge, MA: MIT Press.Kittler, Friedrich A. ([1986] 1999), Grammophone, Film, Typewriter, Stanford, CA:

Stanford University Press.Krauss, Rosalind (1999a), Voyage on the North Sea: Art in the Age of the Post-Medium

Condition, London: Thames & Hudson.———— (1999b), ‘Reinventing the medium’, Critical Inquiry, 25: 2, Chicago:

University of Chicago Press, pp. 289–305. Lessing, Gotthold ([1766] 1984), Laocoön: An Essay on the Limits of Painting and

Poetry, Baltimore: Johns Hopkins University Press.Manovich, Lev (2001a), The Language of New Media, Cambridge, MA: MIT Press.———— (2001b), ‘Post-media aesthetics’, available from http://www.manovich.net/.

Accessed 19 February 2007.McLuhan, Marshall ([1964] 2001), Understanding Media, London and New York:

Routledge.Mitchell, W. J. T. (2005) What Do Pictures Want?: The Lives and Loves of Images.

Chicago: University of Chicago Press.Neumark, Norie (2006), ‘Different spaces, different times: Exploring possibilities for

cross-platform “radio”’, Convergence, 12: 2, pp. 213–24.Richter, Gerhard (1995), The Daily Practice of Painting: Writings 1962–1993,

Cambridge, MA: MIT Press.


74 Arild Fetveit


Northern Lights Film & Media Studies Yearbook 2007

Northern Lights: Film

& M

edia Studies Yearbook | Volume Five

ISSN 1601-829X

Volume 5

intellectwww.intellectbooks.com

Volume Five

intellect Journals | Film

Studies

Northern Lights Volume 5 – 2007

3 Introduction Arild Fetveit and Gitte Bang Stald

7–24 Mixed media: from digital aesthetics towards general communicationtheory

Klaus Bruhn Jensen

25–38 Remediation and the language of new media Jay David Bolter

39–56 Alan Kay’s universal media machine Lev Manovich

57–74 Convergence by means of globalized remediation Arild Fetveit

75–88 The website as unit of analysis? Bolter and Manovich revisited Niels Brügger

89–104 Gameplay as design: uses of computer players’ immaterial labour Adam Arvidsson and Kjetil Sandvik

105–118 On transdiegetic sounds in computer games Kristine Jørgensen

119–140 Power and personality: politicians on the World Wide Web Ib Bondebjerg

141–158 Online debate on digital aesthetics and communication Lev Manovich, Jay David Bolter, Klaus Bruhn Jensen, Arild Fetveit and Gitte

Bang Stald

159 Contributors

9 771601 829000

ISSN 1601-829X0 5

Northern Lights cover.indd 1Northern Lights cover.indd 1 8/22/07 4:51:16 PM8/22/07 4:51:16 PM

Date post:	16-Nov-2014
Category:	Documents
Upload:	intellect-books
View:	1,281 times
Download:	0 times

Northern Lights: Film and Media Studies Yearbook: Volume: 5 | Issue: 1

Documents