Went to a symposium presented by WIRED and the NYPL (link to webcast) that was set up more or less as a confrontation between a man from Google, Lawrence Lessig, and an author and a lawyer representing the publishing industry. Google and others have recently initiated something called Google Book Search, a sort of meta library card catalogue that allows one to search not only via subject, title and author, but also through the contents of a book. This is possible because they have scanned these books and if you look for a subject that is mentioned in a book it will not only tell you a book on that subject but passages in other books, on other subjects, that deal with the subject you are interested in.
What you see in these content searches is a “snippet” — a couple of sentences as a sample of how the subject is mentioned, nothing more. Lessig defends this as “fair use” — the equivalent of quoting a line or 2 of text from a source, which in U.S. law says does not have to be paid for or licensed if only a little bit is quoted. The lawyer disagreed with this, claiming it was somehow not fair use, but he mainly reiterated the buzzwords “they are scanning the WHOLE book” and “they are COPYING the whole book”. This is true, but we who make these requests are not privy to these whole scans and copies, we only see tiny snippets. The lawsuit seems to be an equivalent to the justification for the invasion of Iraq, preemptive action — stopping someone by force because they have the capability of doing something, not because they are or will actually do it. Can I use this justification for attacking corporations and organizations that might abuse my rights in the future?
I was struck that as much information — books, websites, blogs, etc — becomes digitized and immediately catalogued, much of the world remains outside this realm. Images, for example. Sure there are images catalogued on Google and Flickr, and the latter has a more elaborate means of defining an image — as people contribute descriptions to an image, elaborating on its caption, it gets increasingly cross-referenced.
However, I sense that there’s a limit to this. Images, for example, are not just their subject, their author their date. They are personal, traumatic, shocking, dark, light, cheery, artificial, awe-inspiring, fake, large, small, ubiquitous, rare, on and on. The way they strike us is hard to define in words. But that’s where their meaning lies. Sure, words can describe the subject, and that is the way images are usually labeled and catalogued, by subject or “author”. But that’s not how they affect us. That’s not why we like them or fear them or fetishize them.
Images effect us like music; they bypass the language part of the brain that translates things into words, or even into numbers, or even into ones and zeros. They go straight to the emotional centers of our brains — what we sometimes call our hearts — and trigger strong reactions. Words do this to, but via a translator. Only poetry and lyrics, by being the closer to the aural roots of language, by being sound as well as text, manage to sometimes bypass these catalogues and filters.
Thus far, there is no way of adequately defining this stuff — usually words become the default caption, which is pretty much telling only a tiny bit of the story. A picture is worth a thousand words is maybe an understatement. Imagine if it took a thousand words to define every digitized image. Possible to expand the catalogue this much, but who would be the writers? The thing about image and music is that they are, or seem at least to be, so subjective. Your thousand words might not be my thousand words. So not matter how elaborate the Flickr definitions become they only tell a fraction of the story.
I’m no expert, but I’d guess that number and words were initially super useful in early agricultural communities (for describing ownership, taxes, yields), in early social settlements (ditto) and in the attendant military and hunting apparatus — mentally figuring the arc of a projectile or weapon and coordinating how to defend the settlement. As what became civilization became ascendant these ways of thinking took priority, and eventually all of what became called thinking was defined in these terms, in terms of number and word. Our universe became what we could catalogue.
A lot got left out. The mystery and wonder, emotions and relationships… and art and music. Look on many news websites — these latter are defined as enjoyment or entertainment — clearly relegated to something vaguely superfluous. But aren’t they what makes life worth living? Or at least partly so? The “life of the mind” is tons of fun, seriously, but it’s not all there is.
I sense that facial expressions and gesture are in the same category. They are also an extremely complicated visual “language” that we “read” from birth, and we become more fluent as time passes. But the various factorial combinations and gradations of these postures and micro expressions are so vast that cataloguing them is beyond the scope of the world of number and word. And that of the digital world.
There have been numerous attempts to catalogue these facial and body displays, in simple basic ways, and it’s become obvious that not only are there common “words” — an arched eyebrow, a grimace, a smile — but that they also reach across species as well. Apes share some of the same expressions or at least outward manifestations as people. Frogs and fish probably do not. The vast universe that is this other language communicates the emotions and inspires action, hope and desire it is intuitive, and has not really been catalogued.
The digital world may be the climax of the reign of the world viewed as word and number, but it is only touching a part of out world, our lives. I sense that the search engine for gesture, image, sound and expression is a long way off, and may require a kind of “thinking”, if we can even call it thinking, that is so vastly different that what we’ve been doing for 7 thousand plus years, that it may not be possible at all with the tools we’ve developed. I mean the technology and language and mathematics. I don’t mean we can’t discover the key to this other “language”
Our tools we’ve developed, language included, deal with the parts of our brains that have allowed us to achieve a lot, but it is not necessarily the part that most often moves us and motivates us to action, love, hatred, fear, ambition and awe. The part we know and are comfortable with seems to be the part we use to justify those actions, but it is not always the originator.
There are programs that attempt to catalogue songs sonically, based on previously appealing melodic and textural forms and structures, but even these don’t translate the results into words — they compare patterns, waveforms, arcs and frequencies. The program itself makes comparisons, but doesn’t need to name each one. You can look at two paintings or buildings and find the correspondences, but that doesn’t give you the language for naming them, or tell you why one feels scary and one feels comfortable.
Paraphrasing Wittgenstein, my thoughts are limited by my language, might be only half true — there are other languages out there that we use every day, comfortably, fluently and effortlessly, but they are beyond the reach of Google and binary bits, so far.
