| The Anti-Sublime Ideal in New Media |
Along with a Graphical User Interface, a database, navigable space and simulation, dynamic data visualization is one of the genuinely new cultural forms enabled by computing.[1] Of course the fans of Edward Tufte will recall that one can find examples of graphical representation of quantitative data as early as in the eighteenth century but the use of computer medium turns such representations from the exception into the norm. It also makes possible a variety of new visualization techniques and uses for visualisation. With computers we can visualise much larger sets of data, create dynamic visualisations (i.e. animated and interactive), feed in real-time data, base graphical representations of data on its mathematical analysis using a variety of methods from traditional statistics to data mining and map one type of representation into another (images into sounds, sounds into 3D spaces, etc.)
Since Descartes introduced the system for quantifying space in the seventeenth century, graphical representation of functions has been the cornerstone of modern mathematics (for a reminder, use a Mac, start Graphing Calculator and run the demo.) In the last few decades, the use of computers for visualisation enabled the development of a number of new scientific paradigms such as chaos and complexity theories, and artificial life. It also forms the basis of a new field of scientific visualisation. Modern medicine relies on the visualisation of the body and its functioning. Similarly, modern biology depends on DNA and protein visualisation. But while contemporary pure and applied sciences, from mathematics and physics to biology and medicine heavily rely on data visualisation, until recently visualisation in the cultural sphere has been used on a much more limited scale, being confined to 2D graphs and charts in the financial section of a newspaper, or on occasional 3D visualisation on television to illustrate the trajectory of a space station or missile.
I will use the term visualisation for the situations when quantified data which by itself is not visual – the output of meteorological sensors, stock market behaviours, the set of addresses describing the trajectory of a message through a computer network, and so on – is transformed into a visual representation.[2]
The concept of mapping is closely related to visualisation but it makes sense to keep the two separate. By representing all data using the same numerical code, computers make it easy to map one representation into another: grayscale image into 3D surface, a sound wave into an image (think of visualisers in music players such as iTunes), and so on. Visualisation can then be thought of as a particular subset of mapping in which a data set is mapped into an image.
Human culture practically never uses more than four dimensions in its representations because we humans live in 4D space. It is therefore difficult to imagine data in more than these four dimensions: three dimensions of space (X, Y, Z) and time. However, more often than not, the data sets we want to represent have more than four dimensions. In such situations designers and their clients have to choose which dimensions to use and which to omit, and how to map the selected dimensions.
This is the new politics of mapping of computer culture. Who has the power to decide what kind of mapping to use? Which dimensions are selected? What kind of interface is provided for the user? These new questions about data mapping are now as important as more traditional questions about the politics of media representation, by now well rehearsed in cultural criticism (who is represented and how, who is omitted). More precisely, these new questions around the politics of quantified data representation run parallel to the questions about the content of the iconic and narrative media representations. In the later case, we usually deal with the visual images of people, countries and ethnicity, in the former case, the images are abstract 3D animations, 3D charts, graphs, and other types of visual representation used for quantified data.
topBefore moving to the discussion of data visualisation (i.e., mapping of data into the visual domain) in media art, let’s dwell a little longer on the concept of mapping itself. It is possible to think of all representational art as a kind of mapping: taking the wealth of an individual’s and/or a community’s experiences and reducing it to a single image, a narrative or another artistic structure. It is also appropriate (and more interesting) to use the term mapping for describing what new media does to old media. Software allows us to re-map old media objects into new structures – thus turning media into what I call “meta-media.[3] With software, the data can be mapped into another domain – time into 2D space, 2D image into 3D space, sound into 2D image, and so on. In addition, the media object can be manipulated using all standard interface techniques: search, filter, zoom, multiple views, summarise, etc. More complex and unusual mappings are also possible – and the search for such new mappings that allow us to access old media objects in new ways congruent with information interfaces we use in our everyday life – represents one of the most fruitful research directions in new media art.
Let me provide a few examples of meta-media which all involve interfaces new to cinema. For instance, software developed by Steve Mamber (Los Angeles) allows the user to “map” the feature film into a matrix of still images, each image representing a shot from a film. Here time is mapped into space. Another software tool written by Mamber takes shots from the film and reconstructs their architecture as 3D navigable spaces (thus reversing the normal procedure of computer animation). This is where mapping goes from 2D to 3D - from the flat surface of a movie screen to a virtual computer space. The project “Invisible Shape of Things Past” by Art+Com (Berlin) maps historical films of Berlin into new spatial structures that are integrated into a 3D navigable reconstruction of the city.[4] Another groundbreaking mapping project by Art+Com is a virtual opera set whose parameters are interactively controlled by actors during the opera. In this case, positions of the human body are mapped into various parameters of a virtual architecture such as layout, texture, colour, and light. For project designer Joachim Sauter, it was important to preserve the constraints of the traditional opera format – actors fore-grounded by lighting with the set behind them – while carefully adding new dimensions to it.[5] Therefore following the conventions of traditional opera the virtual set appears as a backdrop behind the actors – except now it not a static picture but a dynamic construction that changes throughout the opera.
Note that the mappings in these examples preserve the granularity and the syntactical structure of the old media object, while giving us new ways through which we can navigate, experience its structure, compress and expand our views of the object, and interactively control it. With Mamber’s project, the film still consists of shots that can be played from beginning to end – or we can use the new representation of all the shots in a film as a single interactive 2D image matrix. In the case of “Invisible Shape” we can similarly play the historical film segment from beginning to end – or we can navigate the 3D model of Berlin to see where these films were shot.
This is why I refer to this type of new media as “meta-media.” A meta-media object contains both language and meta-language – both the original media structure (a film, an architectural space, a soundtrack) and the software tools that allow the user to generate descriptions of this structure and to change this structure.
If you think that meta-media is a conservative phenomenon which “betrays” the movement of computer culture to develop its own unique cultural techniques – Artificial Intelligence, Artificial Life, simulation, etc. – you are wrong. Since the late 1960s, modern computing has been grounded in Alan Kay’s concept (influenced by previous groundbreaking work in human computer interface, most importantly Sutherland’s 1962 Sketchpad software) of a computer as a “personal expressive media.” After he arrived to Xerox PARC, Kay directed the development of a word processing program, a music composition program, a paint program, and other tools that redefined the computer as a simulation machine for old media. So while the routine use of computers as media simulators did not become possible until the 1980s, the paradigm itself was already set circa 1970. Gradually, other roles of a modern computer - a machine for computation, real-time control, and network communication – became less visible than its role as “simulation engine” (although the development of the World Wide Web since 1993 obviously made network communication also very important). In summary, the computer’s ability to simulate other media (which means simulating their interfaces and “data formats” such as written text, image, and sound) is not an after-thought – it is the essence of a modern post-1970 computer.
What is crucial to realise is that the computer’s simulation role is as revolutionary as its other roles. Most software tools for media creation and manipulation do not simply simulate old media interfaces – a book page and a table of contents in Acrobat, a pan and a zoom of a virtual camera in Maya, time code count and a razor blade in FCP – but also allow for new type of operations on the media conten. In other words, these tools carry the potential to transform media into meta-media. Re-mapping media data into a new domain is one of the most important among these operations.
The fact that today meta-media – rather than other seemingly “truly” original computer techniques – is in the centre of computer culture is not accidental. The logic of meta-media fits well with other modern key aesthetic paradigms - the remixing of previous cultural content and forms of a given media (most visible in music, architecture, and fashion) and the second type of remixing – that of national cultural traditions now submerged into the medium of globalisation. (In the first approximation, the terms “postmodernism” and “globalisation” can be used as aliases for these two remix paradigms.) Meta-media can therefore be thought-of as the third of these two types of remixing: the remixing between the interfaces of various cultural forms and the new software techniques – in short, the remix between culture and computers.
(If we look at interfaces of media access and manipulation software in this perspective, they begin to look like the work of a radical DJ who mixes operations from the old interfaces of various media with new GUI operations in somewhat erratic and unpredictable ways. My favourite example of such remix is the interface of Adobe Acrobat Reader. It combines (1) the interface from time media software (VCR style arrow buttons); (2) the interface from image manipulation software (a zoom tool); (3) the interface elements closely associated with the printing tradition - although they never existed in print (page icons also controlling the zoom factor); (4) the interfaces that have existed in books (the bookmarks window); (5) the standard elements of GUI such as search, filter, multiple windows).
topMapping one data set into another, or one media into another, is one of the most common operations in computer culture, and it is also common in new media art.[6] Probably the earliest mapping project which received a lot of attention and which lies at the intersection of science and art (because it seems to function well in both contexts) was Natalie Jeremijenko’s “live wire.” Working in Xerox PARC in the early 1990s, Jeremijenko created a functional wire sculpture which reacts in real time to network behaviour: more traffic causes the wire to vibrate more strongly. In the last few years, data mapping has emerged as one of the most important and interesting areas in new media art, attracting the energy of some of the best people in the field. It is not accidental that out of 10 Net Art projects included in 2002 Whitney Biennale, about approximately half presented different kinds of mapping: the visual map of the space of Internet addresses (Jevbratt), 3D navigable model of Earth presenting a range of information about the Earth in multiple layers (Klima), another 3D model illustrating the algorithm used for genome searches (Fry); the diagrams of corporate power relationships in the United States (John On & Futurefarmers). [7]
In order to ground my general observations on data mapping in art in concrete material, I would now like to briefly discuss a few projects by some of the best artists dealing in data visualisation. One of my favourites is John Simon (New York). His work is unique for a number of reasons. First of all, he makes explicit connections in his pieces between the new ideas of new media and various traditions, movements and figures of modern art, in particular Mondrian, Klee, and Sol Levitt. Given that art world and culture at large still largely treat new media as a phenomena in itself which has no connection to the past, Simon’s explicit and systematic explorations of conceptual linkages between new media and modern art are very important. Moreover, while the field of new media art has grown rapidly over the last few years and while artists from all disciplines now routinely use the computer as a tool, there is still only a handful of artists who focus on one of the most fundamental and radical concepts associated with digital computers: that of computation itself (rather than interactivity, network or multimedia). Simon systematically researches how real-time computation can be used to create engaging artworks that are both conceptual and strongly material, offering the viewer rich visual experiences. In his earlier work online piece Every Icon (1998) and his wall-mounted pieces included in the Bitstreams exhibit at the Whitney Museum (2001), Whitney used real-time computation to create artworks that have a starting point in time but no end point; as time progresses, they constantly change. While we can find certain precedents for such artworks in modern art (for instance, kinetic art, early computer art of the 1960s, and conceptual art), Simon pursues a unique strategy of his own: he uses artificial life, cellular automata and other computational techniques to create complex and nuanced images which combine figurative and abstract and explicitly insert themselves within the history of modernist visual research.
If Simon’s images are the result of real-time computation internal to a work itself, those of Lisa Jevbratt (Santa Barbara) often are driven by the Internet data. Jevbratt received her training at CADRE.[8] This program was created by Joel Slayton at San Jose State University who was able to strategically exploit its unique location right in the middle of Silicon Valley to encourage the creation of computer artworks which critically engage with commercial software being created in Silicon Valley for the rest of the world: Internet browsers, search engines, databases, data visualisation tools, etc. With his ex-students, Slayton created a “company” called C5 to further develop critical software tools and environments. Jevbratt is the most well-known artist to emerge from the C5 group. While “software art” has emerged as a new separate category within new media field only about two years ago, Jevbratt, along with other members of the CADRE community, have been working in this category for much longer. In their complexity and functionality, many software projects created at C5 match commercial software, which is still not the case for most new media artists.
In her earlier well-known project 1:1 Jevbratt created a dynamic database containing IP addresses for all the hosts on the World Wide Web, along with five different ways to visualise this information.[9] As the project description by Jevratt points out:
When navigating the web through the database, one experiences a very different web than when navigating it with the "road maps" provided by search engines and portals. Instead of advertisements, pornography, and pictures of people's pets, this web is an abundance of non-accessible information, undeveloped sites, and cryptic messages intended for someone else…The interfaces/visualisations are not maps of the web but are, in some sense, the web. They are super-realistic and yet function in ways images could not function in any other environment or time. They are a new kind of image of the web and they are a new kind of image.
In a 2001 project Mapping the Web Infome Jevbratt continues to work with databases, data gathering and data visualisation tools; and she again focuses on the Web as the most interesting data depository corpus available today.[10] For this project, Jevbratt wrote a special software that enables easy menu-based creation of Web crawlers and visualisation of the collected data (a crawler is a computer program which automatically moves from Web site to Web site collecting data from each). She then invited a number of artists to use this software to create their own crawlers and to visualise the collected data in different ways. This project exemplifies a new functioning of an artist as a designer of software environments that are then made available to others.
Alex Galloway/RSG collective uses a similar approach in his network visualisation project Carnivore (2002). Like Jevbratt, RSG collective created a software system that he opened up for other artists to use. Physically Carnivore is styled like a morph between a non-distinct box for telephone surveillance such as the ones used in GDR, and a modernist sculpture; connected to some point in the network, it intercepts all throughput data. This by itself does not make it art, since a number of commercial software packages perform similar functions. For instance, Etherpeek 4.1 is a LAN analyser that captures packets from attached Ethernet or AirPort networks and uses decoders to break these packets into their component fields. It can decode FTP, HTTP, POP, IMAP, Telnet, Napster, and hundreds of other network protocols. It performs real-time statistical analysis of captured packets and can reconstruct complete e-mail messages from the captured packets. As it is often the case with the artist software (software by CADRE community being an exception), Carnivore only offers a small fraction of the capabilities of its commercial counterparts such as Etherpeek. What it does offer instead is the open architecture that allows other artists to write their own visualisation clients that display the intercepted data in a variety of different ways.
Some of the most talented artists working with the Net have written visualisation clients for Carnivore. The result is a diverse and rich menu of shapes, all driven by network data. Just as in the first decades of the twentieth century modernist artists mapped the visual chaos of the metropolitan experience into simple geometric images, data visualisation artists transform the informational chaos of data packets moving through the network into clear and orderly forms. And if modernism reduced the particular to its Platonic schemas (think of Mondrian, for instance, systematically abstracting the image of a tree in a series of paintings), data visualisation is engaged in a similar reduction as it allows us to see patterns and structures behind the vast and seemingly random data sets. Thus it is possible to think of data visualisation as a new abstraction. But if modernist abstraction was in some sense anti-visual – reducing the diversity of familiar, everyday visual experience to highly minimal and redundant structures (again, Mondrian’s art provides a good example) – data visualisation often employs the opposite strategy: the same data set drives endless variations of images (think of various visualisation plug-ins available for music players such as iTunes.) Thus, data visualisation moves from the concrete to the abstract, and then again to the concrete. The quantitative data is reduced to its patterns and structures that are then exploded into many rich and concrete visual images.
topHaving looked at the particular examples of data visualisation art, we can now draw a few observations and pose a few questions. I often find myself emotionally moved by these projects. Why? Because they carry the promise of rendering the phenomena that are beyond the scale of human senses into something that is within our reach, something visible and tangible? This promise makes data mapping into the exact opposite of the Romantic art concerned with the sublime. In contrast, data visualisation art is concerned with the anti-sublime. If Romantic artists thought of certain phenomena and effects as un-representable, as something which goes beyond the limits of human senses and reason, data visualisation artists target the exact opposite: to map such phenomena into a representation whose scale is comparable to the scales of human perception and cognition. For instance, Jebratt’s 1:1 reduces the cyberspace – usually imagined as vast and maybe even infinite – to a single image that fits within the browser frame. Similarly, the graphical clients for Carnivore transform another invisible and “messy” phenomena – the flow of data packets through the network that belong to different messages and files – into ordered and harmonious geometric figures. The macro and the micro, the infinite and the endless are mapped into manageable visual objects that fit within a single browser frame.
The desire to take what normally falls outside the scale of human senses and to make it visible and manageable aligns data visualisation art with modern science. Its subject matter, i.e. data, puts it within the paradigm of modern art. In the beginning of the twentieth century, art largely abandoned one of its key – if not the key – function: portraying the human being. Instead, most artists turned to other subjects, such as abstraction, industrial objects and materials (Duchamp, minimalists), media images (pop art), the figure of artist herself or himself (performance and video art) – and now data. Of course it can be argued that data art indirectly represents the human being by visualising his or her activities (typically the movements through the Net). Here again I would like to single out the works of Simon who makes explicit references to the tradition of modernist abstraction (one of his works, for instance, refers to Piet Mondrian’s Broadway Boogie-Woogie, 1942-43) – and also includes figurative elements in his compositions, such as outlines of Manhattan Midtown buildings and street traffic. In fact, Simon refers to this piece as a view from his studio window – a type of image that has a well-known history in modern art (for instance, views of Paris by the impressionists).
Another important question worth posing pertains to arbitrary versus motivated choices in mapping. Since computers allow us to easily map any data set into another set, I often wonder why the artist chose this or that mapping when endless other choices were also possible. Even the very best works which use mapping suffer from this fundamental problem. This is the “dark side” of mapping and of computer media in general – its built-in existential angst. By allowing us to map anything into anything else, to construct an infinite number of different interfaces to a media object, to follow infinite trajectories through the object, and so on, computer media simultaneously makes all these choices appear random – unless the artist uses special strategies to motivate his or her choices.
Lets look at one example of this problem. One of the most outstanding architectural buildings of the last decade is the Jewish Museum in Berlin by Daniel Liberskind. The architect put together a map that showed the addresses of Jews living in the neighbourhood of the museum site before World War II. He then connected different points on the map and projected the resulting net onto the faces of the building. The intersections of the net projection and the design became multiple irregular windows. Cutting through the walls and the ceilings at different angles, the windows point to many visual references: narrow eyepiece of a tank; windows of a Medieval cathedral; exploded forms of the cubist/abstract/supremacist paintings of the 1910s-1920s. Just as in the case of Janet Cardiff's audio walks, here the virtual becomes a powerful force that re-shapes the physical. In Jewish Museum, the past literally cuts into the present. Rather than something ephemeral, here data space is materialised, becoming a sort of monumental sculpture.
But there was one problem which I kept thinking about when I visited the still empty museum building in 1999 – the problem of motivation. On one hand, Liberskind's procedure to find the addresses, draw a map and connect all the lines appears very rational, almost the work of scientist. On the other hand, as far as I know, he does not tell us anything about why he projected the net in this way as opposed to any other way. So I find something contradictory in fact that all painstakingly collected and organised data at the end is randomly scattered over the shapes of the building. I think this example illustrates well the basic problem of the whole mapping paradigm. Since usually there are endless ways to map one data set onto another, the particular mapping chosen by the artist often is not motivated, and as a result the work feels arbitrary. We are always told that in good art "form and content form a single whole" and that "content motivates form." Maybe in a "good" work of data art the mapping used must somehow relate to the content and context of data - although I am not sure how this would work in general.
One way to deal with this problem of motivation is not to hide but to foreground the arbitrary nature of the chosen mapping. Rather than always try to be rational, data art can instead make the method out of irrationality.[11] This of course was the key strategy of the twentieth century Surrealists. In the 1960s the late Surrealists – the Situationists – developed a number of methods for their “the dérive” (the drift). The goal of “the dérive” was a kind of spatial “ostranenie” (estrangement): to let the city dweller experience the city in a new way and thus politicise his or her perception of the habitat. One of these methods was to navigate through Paris using a Map of London. This is the kind of poetry and conceptual elegance I find lacking in mapping projects in new media art. Most often these projects are driven by the rational impulse to make sense out of our complex world, the world where many processes and forces are invisible and are out of reach. The typical strategy then is to take some data set – Internet traffic, market indicators, amazon.com book recommendation, or weather – and map it in some way. This strategy echoes not the aesthetics of the Surrealists but a rather different paradigm of the 1920s left avant-garde. The similar impulse to "read off" underlying social relations from the visible reality animated many left artists in the 1920s, including the main hero of my 'The Language of New Media – Dziga Vertov. Vertov' 1929 film A Man With a Movie Camera is a brave attempt at visual epistemology – to reinterpret the often banal and seemingly insignificant images of everyday life as the result of the struggle between old and new.
Important as the data mapping new media projects are, they miss something. While modern art tried to play the role of "data-epistemology," thus entering in completion with science and mass media to explain to us the patterns behind all the data surrounding us, it also always played a more unique role: to show us other realities embedded in our own, to show us the ever-present ambiguity in our perception and experience, to show us what we normally don't notice or don't pay attention to. Traditional "representational” forms - literature, painting, photography, and cinema – played this role very well. For me, the real challenge of data art is not about how to map some abstract and impersonal data into something meaningful and beautiful – economists, graphic designers, and scientists are already doing this quite well. The more interesting and at the end maybe more important challenge is how to represent the personal subjective experience of a person living in a data society. If daily interaction with volumes of data and numerous messages is part of our new “data-subjectivity,” how can we represent this experience in new ways? How new media can represent the ambiguity, the otherness, the multi-dimensionality of our experience, going beyond already familiar and “normalised” modernist techniques of montage, surrealism, absurd, etc.? In short, rather than trying hard to pursue the anti-sublime ideal, data visualisation artists should keep in mind that art has the unique license to portray human subjectivity – including its fundamental new dimension of being “immersed in data.”
Berlin, August 2002
top