Lillian Schwartz Sees in
Four-Dimensions

By Walter Forsberg


Image: Lillian Schwartz at work on a film.  Still from The Artist and the Computer (John K. Ball, 1976).

 

1930: for the first time in history we have publishing of the electromagnetic spectrum by the Westinghouse Company. We realize we were not able to see what is going on in radio waves, or ultra-violet, or infrared waves, with our eyes. We thought, in the old way that is, it was not real if you could not see it. Now, we suddenly discover in 1930, that 99.9% of reality was no longer directly contactable, or apprehensible by the human senses.[1]



I. BACKSTORY

Working at AT&T’s Bell Laboratories as an artist during the late 1960s and early-70s, Lillian Schwartz was among the first American artists to employ computer-coding language to create motion graphics-based film and video art.  As the “pure research” arm of AT&T industries, Bell Labs also served as the maternity ward for other such computer and communications technologies, among them: UNIX (the pioneering operating system software); Ken Knowlton’s early graphics imaging program languages (BEFLIX and EXPLOR); Max Mathews’ early software for creating computer-generated sounds via synthetic digital audio waveforms (MUSIC); and prototypical video phone technology (see: 2001: A Space Odyssey [1968]; Skype).


Schwartz’s tenure at Bell Labs was initially a clandestine one.  In 1968, she met Bell Labs scientist Leon Harmon at the Museum of Modern Art exhibition, The Machine As Seen At the End of the Mechanical Age, in which both had artwork. Harmon subsequently invited her to Bell Labs as a “Resident Visitor,” where she initially worked with Harmon using ASCII code.  She ended up staying over three decades.  As a non-scientist, Schwartz often served in the role of “friendly conversational sounding board” for AT&T researchers and scientists looking for feedback on their ideas.  As an artist, she created a massive body of technology-based work over the course of 34 years.  Her prolific output of moving and still image technology-based art won global acclaim, acquisition by internationally recognized private and public art collections, respect from scientific circles, and an eventual (and egregiously overdue) spot on the AT&T payroll.  By the 1980s, her work had been widely exhibited around the globe; garnered Schwartz an Emmy for “Outstanding Visual Craft;” became part of the permanent collection of the MoMA, among scores of other museums; had won numerous film festival prizes, including the International Award at the Kurzfilmtage Oberhausen; and, Schwartz was made a Fellow of the World Academy of Art and Science, given an honorary doctorate by Kean College, and recruited to teach and lecture at an array of universities and cultural institutions.


Acclaim and honors aside, Schwartz’s oeuvre was equally respected by artists working in both conceptual and computer art realms—a fact highlighted by Carolyn L. Kane’s recent brief history of Schwartz’s career.[2] One such admirer was Salvador Dalí.  After presenting Schwartz with a pen upon which he had urinated for an entire year, Dalí claimed her to be the only person capable of documenting the way light reflected off his crystallized urine, “like diamonds,” imploring her: “It is your job to make a permanent record.”[3]


Computer Animation

Collaborating with the programmer Ken Knowlton, Schwartz brought her background in art and sculpture to Bell Labs projects and expanded on research in the realm of computer-generated graphics.  Schwartz and Knowlton worked closely to generate the individual source images that Schwartz would later transform into sequenced moving pictures, and Schwartz gave Knowlton authorship co-credit for several of her films.[4] Knowlton, writing in 1970, explains:


I expect art to come from artists or artists working closely with programmers—I do not expect much art to come from programmers alone, solely by virtue of their clever gimmicks for doing cute things.  What this means in practical terms, then, is that we need to develop a great deal of collaboration between artists and programmers in order to develop meaningful, understandable, and useful sets of tools and ways of using them.[5]

 

Beginning with Pixillation in 1969-70, Schwartz built on prior work done with text graphics by Knowlton and the filmmaker Stan Vanderbeek in the late-1960s using the BEFLIX graphics language.  (In 1969, when asked by AT&T executives if she could make a promotional film for Bell Labs to improve AT&T’s image on college campuses, the filmic novice Schwartz agreed.  She subsequently taught herself both 16mm filmmaking processes, and how to create images on a computer using Knowlton’s code.)[6] In spite of Knowlton and Vanderbeek’s accomplishments, Schwartz was unsatisfied with BEFLIX’s graphic limitations: “I definitely was.  I never even actually used it!”[7] In an effort to bring non-text-based geometries into her work, Knowlton and Schwartz collaborated on the creation of the EXPLOR language—used for subsequent films such as UFOs (1971), Enigma (1973), and Olympiad (1973).  A “system for computer-generation of still or moving images from explicitly defined patterns, local operations, and randomness,” EXPLOR was capable of generating rectangular arrays of 240 x 340 unit binary black-and-white images (using an IBM 360/50 computer emulating an IBM 7094), which were outputted to microfilm using a Stromberg-Carlson Datagraphics microfilm printing machine.[8] In practical terms, EXPLOR enabled Schwartz to achieve the basic curves, angles, and geometries her mind’s eye demanded.  From these black and white microfilm outputs, Schwartz would have the film lab Multicolor reduce the 35mm microfilmed images to 16mm for the addition of color and the sequencing of action patterns.[9]

Like her acknowledged precursors in the field of what some term “visual music”—the Whitney brothers, John Stehura, et al.—Schwartz’s artistic craft was painstaking.  First, she would draw out her desired image on graph paper, and then deduce how to transpose the “pixel” blocks of the graph paper to the pixels of the computer monitor by means of computer programming, using a punch-card-based process.[10] Once these stills were outputted to 35mm film, Schwartz would sequence variations of these individual still images—first in her mind’s eye, and then in 35mm-to-16mm reduced “action” loops using an optical printing set-up.  Schwartz enlisted the Brooklyn Heights-based optical printer Bruce Cornwell to layer these motion sequence loops over one another, according to a written “score” she devised.[11] As the optical printer photographed the still images into animated sequences, Schwartz and Cornwell would employ vibrant color filters to enhance the black-and-white of binary computer code and to reinforce the layering effect and impact of different motion sequences.  Through her work with Cornwell, Schwartz was able to optically step-print stroboscopic and ecstatically complex motion manipulations to create her finalized films.[12]


Stroboscopy

Beyond the pure technological and artistic achievements of her work's production methodologies and visual aesthetics, Schwartz's films also served as significant investigations into human visual perception.  In his detailed paper, “EXPLOR—A Generator of Images from Explicit Patterns, Local Operations, and Randomness,” Knowlton specifically suggests the potential important role of EXPLOR in creating patterns for “experiments on vision” and simulation of “visual phosphenes”—those weird spots and patterns on one’s closed eyelids.[13] The combination of still images with optical printer-inserted stroboscopy, in works like UFOs and Enigma, also serve to articulate cinematic illusions of motion, laid out as far back as the work of psychologist Max Wertheimer in the 1910s.  In UFOs, for example, the illusion of three atom-like rotating spheres is created despite the fact that at no point do the three shapes appear together in a single frame.

One story Schwartz likes to relate about the stroboscopic effects her films had on persons with ocular afflictions concerns a cross-eyed Whitney Museum administrator, named “Deborah.”  The effects of UFOs’ flicker upon Deborah’s strabismus (crossed-eyes) was so beneficially rectifying that when Schwartz’s film was exhibited at the museum in the early 1970s, Deborah confessed that she would regularly watch the film prior to going on dates to achieve a temporary correction of her crossed-eyes.[14]


Computer Music

To accompany her dazzling array of stroboscopic and ultra-chromatic imagery, Schwartz partnered with musicians who were simultaneously exploring the use of computers and synthesizers, but in the fields of music and composition.  The soundtrack credits of Schwartz's early filmography reads like a litany of computer music heavyweights, among them: Gershon Kingsley (Pixillation [1970]), Emmanuel Ghent (UFOs), Richard Moore (Enigma), Jean-Claude Risset (Mutations [1973]), and Max Mathews (Olympiad and Papillons [1976]).  Mathews, as director of acoustic research at Bell Labs for several decades, played frequent host to all manner of experimental sound artists and musicians as they visited New York.  By hanging around Bell Labs, Schwartz proved an available collaborator.  (It is to Mathews and his Bell Labs colleagues, creators of the first computer-programmed music—a bleep-y jaunt through the 19th century tune, “Daisy Bell”—whom Stanley Kubrick pays tribute to at the end of 2001: A Space Odyssey, as the computer/monster HAL 9000.)


Schwartz’s Later Career at Bell Labs
After creating this body of films in the early and mid-70s, Schwartz continued her artistic pursuits at Bell Labs until the early-2000s in a variety of media, with an even broader spectrum of collaborators, and to far wider international acclaim and recognition.  While the general move to ¾” U-Matic videotape as a moving image production format (see: Schwartz’s Mayan [1975], produced at WNET’s TV Lab using the Paik/Abe video synthesizer, for instance) reflected shifts in technology and the output capacities of the newer and newer computers she worked with, Schwartz continued to use film as a medium for freelance documentaries throughout her career.  Delving into realms of artistic creation and research such as a greater concentration on still image graphics, computer rendering software (including her Emmy award-winning 1984 PSA for the newly-renovated MoMA), and historical image archaeology (notably, her groundbreaking work concerning Leonardo da Vinci’s Mona Lisa), Schwartz continued to feed her curiosity in the commingling of art and technology for decades after her initial filmmaking forays at Bell Labs.

 

II. DISCOVERY
In late-2010, leveraging support from the Orphan Film Project, Colorlab, BB Optics, and New York University’s Moving Image Archiving and Preservation department, several film preservationists and general nerds began work on preserving some of Schwartz’s early films.  Beginning with the little-seen Galaxies (1974), comprised of animated images of outer space provided by NASA scientist Frank Hohl, and following with the classic Schwartz stunner UFOs, this cadre of enthusiasts collaborated with Ohio State University’s Rare Books and Manuscripts department (where Schwartz’s collection is housed) to create new 16mm internegatives, 16mm prints, and high definition video transfers, from original color reversal A-B rolls and full coat magnetic track elements. (Subsequent financial support from the Film Foundation, the National Film Preservation Foundation, and New York Women in Film & Television continues these efforts.)

Recent renewed attention for the films of Schwartz’s early period led her to begin posting several online, both on her website (www.lillian.com) and on YouTube and Vimeo.  In early 2011, as a student assistant was transcoding an old video transfer of one of Schwartz’s films in her Upper East Side studio, Schwartz was working on a more recent body of abstract still images.  These 2-D images, imbued with hidden pixel pigmentations, enable viewers to appreciate the abstractions in an enhanced manner with the use of special ChromaDepth 3-D glasses.  Wearing a pair of such glasses herself, Schwartz glanced across the room at her student’s video monitor and let out a shriek: she realized that her early films could be watched in 3-D.  The event itself was even more surprising, as her films predated ChromaDepth 3-D technology by twenty years.

 

III. CHROMADEPTH 3-D & THE DARK SIDE OF THE MOON
Assigned United States Patent 5,002,364 in March 1991, Frank Lauder and Richard Steenblik’s ChromaDepth technology had a distinct ontological difference from prior 3-D technologies.  Unlike other prevalent systems, images that appear stereoscopic and “3-D” when a viewer wears ChromaDepth glasses can be also viewed, albeit without the effect of 3-D depth-illusion, as simple 2-D images.  Jerry Marks, a stereoscopic video- and printmaker who teaches in the Computer Art MFA program at New York’s School of Visual Arts, characterizes this unique technology as “kind of a ‘weirdo’ 3-D.”[15]

In traditional 3-D processes, source/subject images are “encoded” in some fashion — be it via: a twinned duplication of source images, as in early turn of the century stereoscopy and your childhood ViewMaster™; by way of red and blue color off-set image separation, as in anaglyph 3-D processes; or, through 3-D polarization techniques.  Essentially, it is necessary to perform a process of “encoding” of the source/subject images during the production process—which allow the images to be “decoded” by a viewer’s glasses (simply providing a bookended, inverted reversal of the encoding process). With Lauder and Steenblik’s ChromaDepth, this requirement of an altered source image is eliminated, provided the source image contains pure color. As the glasses are prismatic, differences in pure color within the source image naturally engender a layering of depth. Glasses or no glasses, whatever one looks at is intelligible (unlike, for example, the nauseating blurry haze of playing PS3’s Call of Duty without wearing active shutter glasses). The effect of 3-D layering with ChromaDepth, then, becomes merely a kind of “trippy bonus.”

 

Image: Album cover for The Dark Side of the Moon (Pink Floyd, 1973),
designed by Hipgnosis and George Hardie.

 

Speaking of trippy bonuses, one way to conceive of how ChromaDepth works is to ponder the cover art for Pink Floyd’s 1973 stoner opus, The Dark Side of the Moon (jazz cigarettes and Wizard of Oz: optional).  This well-known album cover stylizes a basic triangular prism refracting white light into the visible light spectrum of color, in a process called “dispersion.” Dispersion breaks up and refractively bends white light’s constituent colors according to each color’s different wavelength.  Remember your high school physics class friend, “Roy G. Biv?”  Well, ChromaDepth knows him too, and uses him to impact your eye’s observation of colors, and your brain’s perception of their spatial relations.

The text of the U.S. patent does the heavy lifting of summarizing ChromaDepth’s effects on human brain perception in scientific terms:

The brain system of the eye perceives depth on the basis of a variety of cues. Accordingly, there do not exist separate and definite planes of depth within the image separated according to color, but rather they are seen to be merely relative depth relationships… Perspective, position and relative size are all important factors in the brain systems’ perception of depth. In the absence of strong clues from the point of view perspective, position and relative size, parallax will tend to dominate.  IF that parallax, by the use of glasses, gives information which is strongly contradicted by perspective, position and size clues, the depth will be perceived in the image in-concert with the non-parallax clues.  Thus, a magazine photo showing black horses on a green field with a blue lake in the background and bluish mountains in the distance would not be expected to appear to have depth when viewed with the prism glasses, however, the effect is such that these objects do appear to have depth.[16]

 

Image: Diagram from Richard A. Steenblik, “Stereoscopic Process and Apparatus Using Different Deviations of Different Colors,”
U. S. Patent 5,002,364, issued March 26, 1991.

 

Above, in a diagram from the U.S. Patent registration, we see the uniformly planar color relations of the source/subject image at the right, dispersed to a depth-illusory-driven altered spatial relationship at center, via the refractive prisms of ChromaDepth glasses.[17] Crucial to this parallax-inspired “tricking” of the human eye is that the colors of the original images be both highly saturated, and uniform in saturation.  The depth-generation from wearing ChromaDepth glasses is at its most profound when these two criteria are met.  The reason for their immense efficacy when used to view many of Schwartz’s early films lies in the fact that Schwartz added colour to her black and white computer imagery via Bruce Cornwell’s optical printer color filters.  Take Olympiad for example: Whereas in 2-D we see three running figures, in red, blue, and green all on the same flat plane, when wearing ChromaDepth 3-D glasses, the red figure appears immensely closer to ourselves as the audience than the green, and blue figures, respectively.

 

Image: Still from Olympiad (Lillian F.  Schwartz, 1973).  Copyright ©1973, 2003 Lillian F. Schwartz.
Courtesy of the Lillian Feldman Schwartz Collection, The Ohio State University Libraries.
 All rights reserved. Published by permission.


To view Olympiad online, please visit:
http://www.lillian.com



IV. IS THIS A NEW INSTANTIATION?

There are all kinds of variations of a single moving image “work,” particularly in the modern age of multi-media, multi-format delivery, and it is astounding how one “thing” might come in a multitude of different versions, or “instantiations.”  There can be format-rooted instantiation differences, such as how a movie on VHS might include color bars that are absent from a 35mm print of the same work.  Other format instantiation differences might occur when blowing up a Super 8 or 16mm film, originally shot and projected at 18 frames per second, to 35mm—a 24 frames per second format (here, there is often a doubling of every third frame to “extend” the work to match the playback speed).  Often, there are “Director’s Cut” instantiation differences (masturbatory, and otherwise), such as Apocalypse Now Redux (1979/2001) or the handful of different versions of Blade Runner (1982).  There can be “colorization” instantiation differences, as witnessed in the controversial altering of black-and-white classics perpetrated to great derision by Turner Broadcasting in the late-1980s. Linguistic instantiations can occur when dialogue in films are dubbed into foreign languages, or even completely re-done in subsequent releases (see: the differences between the 1988 and 2005 English-language dubs of Akira).  Instantiations of a work can vary according to distribution venues, as with televised movies abridged for commercial breaks or in-flight airplane broadcast.  Aspect ratio shifting might even justifiably be considered worthy of instantiation differentiation, as well.

In the late-90s, the International Federation of Library Associations devised the philosophical concept of FRBR (Functional Requirements for Bibliographic Records) to deal with this endless miasma of derivative versioning in books, movies, and artworks.  But, when considering Schwartz’s work, the question arises: does the newly-discovered 3-D-“ness” of her films, when wearing ChromaDepth glasses, qualify as another instantiation—even though nothing physical (duration, or otherwise) about the work has been altered?  Should there be title distinctions between, for example, UFOs (1971) and UFOs [3-D] (2011)?

 

V. WHAT THIS IS ALMOST LIKE, EXCEPT NOT
The phenomenological effect that ChromaDepth perpetrates on Schwartz’s early film work, begs for a Borges short story, one which could immediately draw a poetic parallel to this staggeringly peculiar paradigm.  ChromaDepth glasses seem to be a kind of unidirectional prosthesis that enables depth perception for things that are intensely color-saturated, naturally or otherwise.  Yet, the effect inspires the search for a metaphor: what on Earth is this like?  In the absence of an immediately apparent correct answer, several wrong ones emerge:

  • Secret undersea “World -1” level, at the end of “World 1-2,” in the Nintendo (NES) video game, Super Mario Bros.

Have we unlocked a secret level to Schwartz’s films?  Not quite: inside of such secret levels the original image, here a “level” or “world,” is no longer visible.

  • Night Vision goggles, or “What Predator (1987) Sees”

Do we achieve an enhanced sense of visual perception?  Closer: however, in both instances the original image is significantly distorted.  Shape may remain constant, but color and detail in the original image are transformed.

  • Audio sampling rates of 192 kHz

Do we achieve a higher level of perception, through quality augmentation?  Perhaps, kind of: but the analog-digital conversion is not at play.

  • Opera glasses, or hearing aids

Do we achieve a higher level of perception, through amplification?  Not really: as the source/subject image isn’t exactly “boosted,” per se, which would involve a distortion of the original.

  • That Calvin & Hobbes comic, when Calvin’s dad explains the advent of photographic color

Are we experiencing some kind of trans-historic technological time warp, whereby contemporary reality subsumes all that came before?  Probably not: but, kind of.

 


Click here for comic

 

VI. SEEING IN FOUR DIMENSIONS

The truth is Lillian lucked into the 3-D, and it’s a match made in heaven.[18]
– Jerry Marks


Chorioretinitis

One key to conceiving of the 3-D-“ness” of Schwartz’s films lies in “chorioretinitis” — the degenerative ocular disease that afflicted Schwartz the near-entirety of her adult life.  The Oxford English Dictionary defines it as, “cerebral meningitis affecting the choroid plexus.”[19] Wikipedia breaks it down as, “an inflammation of the choroid (thin pigmented vascular coat of the eye) and retina of the eye. It is also known as choroid retinitis.”[20]

Schwartz speaks of her experience of the disease, thus:

I had the first symptoms in 1954/black spots in my right eye/I was diagnosed within a few weeks when dots became larger/put on cortisone which was to prevent scar tissue/disease was present for five years in early 70s/never determined the cause/at critical point I had no vision in right eye/slowly vision returned but with a scar down the macula like that of a split in the ground, post-earthquake/macula shifted, one side higher than other/irregular gash through it/some small areas around major scarring also scarred/has not changed. There were suggestions of causes but no proof.[21]

Schwartz attributes the disease as being a by-product of the radiation she was
exposed to while working as a United States Navy nurse in Japan at the end of World War II.

Aside from the variant black “spotting” of her vision, the scar tissue caused by the disease had the immediate impact of robbing Schwartz of depth perception in her right eye.  To perceive depth, she became forced to use visual “cues” that indicate the relative positions of things within a spatial field, or area.  For example: if seated across from her at our usual Second Avenue luncheon spot, Schwartz intimated to me that while I might initially appear “as flat as a cut-out,” the conceptual knowledge that artwork on the wall behind me was, in fact, spatially behind me, serves as a cue that permits her a functional sense of depth.[22]  This use of cues to judge depth relations is common for persons afflicted by eye diseases that inhibit stereopsis.[23

In discussing the long-term effects of her chorioretinits to me, Schwartz recalled the impetus behind the production process for her early films: by adding “pure” saturated color to her animated computer graphics, she was able to augment her right eye’s sense of layered depth in her animated loops of still-image computer graphics.  In effect, the lack of depth perception caused by Schwartz’s chorioretinitis affliction of the early-70s somehow, incredibly, anticipated the yet-to-be-invented ChromaDepth technology of the early-90s.  Lillian’s eyes saw into the future.



Image: Walter Forsberg and Lillian Schwartz.  Photo: Dan Streible.  Courtesy of the author.



NOTES

1. Buckminster Fuller, Tunings, phonodisc recording (New York: Tanam Press, 1979)

2. See Carloyn L. Kane, “Digital Art and Experimental Color Systems at Bell Laboratories, 1965-1984: Restoring Interdisciplinary Innovations to Media History,” Leonardo 43, no. 1 (2010): 53-58.

3. See Kane, “Digital Art and Experimental Color Systems,” and Lillian F. Schwartz, LIL, unpublished manuscript, pp. 18-19.

4. Interview with Lillian Schwartz.  December 21, 2010.

5. Ken Knowlton, “Computer Films,” Filmmakers Newsletter 4, No. 2 (1970).  [As reprinted in: Robert Russett and Cecille Starr, Experimental Animation: An Illustrated Anthology (New York: Van Nostrand Reinhold Company, 1976), pp. 194-195.]

6. Interview with Lillian Schwartz. December 21, 2010.

7. Interview with Lillian Schwartz. April 26, 2011.

8. Ken Knowlton, “EXPLOR­—A Generator of Images from Explicit Patterns, Local Operations, and Randomness” (Murray Hill, NJ: Bell Laboratories Publications, 1970), pp. 1 and 20.

9. Interview with Lillian Schwartz. December 21, 2010.

10. The exact computational functions of EXPLOR are, needless to say, complex and beyond the scope of this article. For a detailed explanation see, Knowlton, “EXPLOR­–A Generator of Images.”

11. Interview with Lillian Schwartz. December 21, 2010.

12. The filmmaker, preservationist, and optical printing master Bill Brand recently commented that “the registration of Bruce Cornwell’s reversal print optical work is spectacular.”  Conversation with Bill Brand. April 1, 2011.

13. Knowlton, “EXPLOR—A Generator of Images,” p. 17.

14. Interview with Lillian Schwartz. December 21, 2010.

15. Conversation with Bill Brand, Gerald Marks, and Lillian Schwartz. April 9, 2011.

16. Georgia Tech Research Corp., “Stereoscopic Process and Apparatus Using Different Deviations of Different Colors.”  Patent 5,002,364.  26 March 1991.

17. Ibid., Figure 1.

18. Conversation with Bill Brand, Gerald Marks, and Lillian Schwartz. April 9, 2011.

19. “Chorioretinitis,” The Compact Oxford English Dictionary, second edition (Oxford: Clarendon Press, 1987), p. 252.

20. “Chorioretinitis,” Wikipedia: The Free Encyclopedia, http://en.wikipedia.org/wiki/Chorioretinitis, accessed April 19, 2011.

21. Email correspondence with Lillian Schwartz.  April 19, 2011.

22. Conversation with Bill Brand, Gerald Marks, and Lillian Schwartz.  April 9, 2011.

23. For a terrific consideration of these diseases, see: Oliver Sacks, The Mind’s Eye (New York: Knopf, 2010), pp. 111-143.

 




ABOUT THE AUTHOR

Walter Forsberg was born in Saskatchewan and is currently a Research Fellow at NYU. From 2005-2008 he created films, posters and art projects with Winnipeg's L'Atelier national du Manitoba.




 

 

 

 

 


 

welcome
information
masthead
contact information
guidelines for contributors
subscribe
advertise
store

forthcoming issue
issue #5: blockbuster

current issue
issue #4: exhibition guide

past issues
issue #1: manifest
issue #2: counter-archive
issue #3: new ages


back and forth
interview series
michael robinson
takahiko iimura
anders weberg
jim finn
jacqueline goss
benj gerdes and jennifer hayashida
sam green
oliver laric
thorsten fleisch
jennifer montgomery
stephen connolly
deborah stratman
bill brown
jon rafman
jennifer bolande
evan meaney
sabrina ratté

john lurie
stephen broomer
vanessa renwick

jake barningham

mike stoltz
molly surno