This proposal describes a new kind of video synthesizer which uses the graphic potential of the video medium for a programming language. The instrument, called the Design Device, will perform video synthesis and signal processing unde-control from a mini computer which reads a specially formatted picture language. This programming language will permit artists to prepare ideas away from the studio and provide a permanent record on paper of all useful programs.

Review of Existing Video Synthesizers and Ideas for a New One

To free the video artist from the confines of the real camera-recorded world, it is necessary to develop instruments which generate a television compatible signal from raw electronics. A synthesizer is the paint and palette of the video artist, a device which lets the artist construct spaces from the dictates of imagination.

The first video synthesizers began- to appear almost a decade after the development of complete audio synthesis systems. There are compelling reasons for this delay. The development of a time variant artform is just now being born in the visual arts, centuries after the establishment of a related set of time variant structures in music. Technically, the video synthesizer is more complex than its audio cousin. Video signals cover a frequency spectrum 100 times greater than audio and must be constructed according to a precise timing synchronization which does not exist in the one dimensional audio signal. Consequently, design concepts and instrument components are now coming together for the first time.

There have been two approaches to video synthesizer design, vector graphics and signal intensity. This split is a consequence of the television system itself which uses a one dimensional high frequency signal to describe a two dimensional field of much lower frequency. The systems developed by Steve Rutt and Bill Etra, Computer Image Corp., Vector General, and others operate on the x and y deflection amplifiers of a cathode ray display. The synthesized or processed images coming from these devices are rescanned by a conventional camera for recording on video tape. Digital signal synthesizers, on the other hand, such as those developed by Steven Beck, Dan Sandin, and EMS Ltd., operate on the intensity or-"z" component of the video signal. Their output is made compatible with video standards by processing in a color encoder. The two approaches can be combined in a single device. In fact, Sandin has worked extensively with a computer controlled vector display, and RuttEtra synthesizers are invariably teamed with keyers, colorizers and other signal processors. However, no one has come out with an integrated package that incorporates both approaches. The Design Device will attempt this marriage of design principles.

All the artist designed synthesizers are "modular", that is, specialized devices are linked by patch cords which are manually inserted to complete a complex program. Modular design is essential in video, because it permits parallel and simultaneous processing of high frequency signals. The chief drawback of the general purpose computer in video synthesis is that it performs one operation at a time and cannot keep up with the video clock. As in the development of the audio synthesizer, artists have provided engineers with functional module building blocks which efficiently accomplish commonly needed functions. Modular design also permits a wide range of interconnections depending on the "patch" made between them. For example, a system of only 8 modules, each with a single input and a single output can be patched in over 40,000 different ways.

While modular systems provide both variety and efficiency, they also can present the artist with a confusing welter of two ended wires which makes live performance difficult and leaves him with no permanent record of his patch. The first step in improving this situation came with the introduction of the matrix switching systems of the Arp and EMS synthesizers, adapted for video by Woody and Steina Vasulka. These systems have manually set crosspoints and permit patchfields to be recorded by graphic notation. Going a step further Don Buchla and Bell Labs have developed computer controlled patchfields which are notated with a verbal language. The Design Device will have an electronic switch system, but rather than a verbal language, it will be controlled by a graphic notational system.

All existing video and audio synthesis systems use a building block called the oscillator. The most common technique for generating forms is called additive synthesis in which the output waveforms of oscillators are mixed to form a waveforn. which is the sum of their combined outputs. It is theoretically possible to duplicate any natural waveform by summing many sine waves of different frequency. This approach has led to the construction of synthesis systems with dozens of oscillators. When such systems became untenable because of the large number of signal paths, a device was introduced by Don Buchla which generated a waveform from discretely set increments. This device is now known as a sequencer. It can be used as an oscillator or a controller in voltage controlled systems. Information is loaded into a sequencer manually by setting dozens of potentiometers. Like the patch cord system, it must be set up from scratch every time it is used. There is no convenient way to notate for this device. The general pupose computer can be used as a kind of sequencer since its memory stores lists of numbers, but again the problem of cycle time limits its use to low frequencies. Recent innovations in semiconductor technology, however, have put digital memory within the reach of the video synthesizer. Using modular design techniques, it is possible to build an oscillator module with a programmable

output. In the Design Device there will be two such oscillators. The stored waveform will be loaded from a sampled graph drawn by the artist and scanned by a conventional television camera.

Given that this small memory can serve to store waveforms, a method must be found to clock out its stored information. oscillators such as those found on the RuttEtra or Steven Beck's synthesizer must be synchronized to the rest of the synthesizer in order to produce stable patterns. While it is relatively easy to make a voltage controlled oscillator, it is difficult to maintain synchronization for an analog module. The Design Device uses the video sync generator itself as the clock for its memories. The rate at which the memories are read out is determined by a simple binary counter which is preset to the number of subdivisions it makes of the sync signals. Hence all the patterns produced by the memories are stable both with respect to the video signal and to each other.

The contribution of commercial manufacturers of television equipment must not be overlooked in describing the development of video synthesis. Every news broadcast these days uses chroma key backgrounds, and this technique can prove very handy for the artist. The trouble is that most synthesizers operated on black and white inputs which are synthetically colorized, and the standard chroma keyer must have separate red, green and blue inputs in order to work. The Design Device will have an encoded chroma keyer which will derive the RGB signals from a color tape input, so it will be possible to build generations of imagery by operating on sections of the image encoded in a particular color. This may sound like a major electronic undertaking, but the commercial manufacturers of television sets have put the chroma decoding circuits on three relatively simple integrated circuits. Building an encoded chroma keyer is simply a matter of opening up a color television and using its decoder circuitry. Paik and Abe perfored this operation on a Trinitron at the WNET TV Lab. Unfortunately they did not realize this application of their modification.

Another area in which broadcast television people have made great strides is in time base correction. For the mere sum of $12,000 the broadcaster can replace his $100,000 quad vtr with a time base corrector and a Sony video cassette. This means that an inexpensive vtr can be used as a source in a switcher. For an artist making a multi generational tape, such time base correction is essential. For an artist working on a shoe string budget, $12,000 for a time base corrector is out of the question. The Design Device will have two kinds of "pseudo" time base correction. It has a rescan system which will re enter a black and white image from an external tape source and even permit the luxury of relocating the input image into a new space in the frame. It will also use the cheap part of the big commercial time base correctors which is a digital memory system, only the Design Device will only have one bit of time base correction. This will eliminate the

At the beginning of this discussion of video synthesizers, I called such devices the artist's paint and palette, and no module better fits this description than the colorizer. There are many designs for colorization devices both commercial and unique. In systems which use a camera encoder to generate the final output, the colors are determined by mixing red green and blue components. While pleasantly reminiscent of mixing colored paint, this system is less efficient to use than the colorization made possible by video color parameters: luminance, chrominance and hue. With the latter system it is possible to pass a previously encoded color signal through the synthesizer and recover it unchanged at the other end (through the luminance channel). Now colors can be added by entering signals into the hue and chrominance channels. Where grey scale encoding is used such as in quantization, the single hue parameter can produce rainbow-like effects. The Siegel colorizer uses grey scale encbding to modulate all three parameters simultaneously. Dividing the inputs into three channels gives an increased degree of control over the final output. The Grass Valley colorizer uses the luminance, chrominance, and hue parameters but is not voltage controlled and hence is not dynamic. The Design Device will have a colorizer-sync adder which vary color through four complete rotations of the primaries and their mixes, permitting voltage controlled hue and saturation, and pass a signal unchanged through the luminance port with manually adjustable controls for pedestal and gain.

A Graphic Guitar

The Design Device will be a self contained video synthesizer which can originate graphic patterns and can process prerecorded video signals. It will have provision for interfacing with analog audio synthesis systems and digital computers.

External video sources such as video tape recorders and cameras will be entered into the Design Device through an Input Module (figure 1). There will be four specialized input ports. All inputs will be clamped and sync separated.

Input One is intended for a color signal. The Input Module will clamp the signal and strip sync. Timing signals extracted from the Input One signal will be H drive, V drive, blanking, composite sync, subcarrier and burst flag. These signals will be connected to the Sync Generator for gen lock. There will be a manual adjustment for the phase of the subcarrier. The video portion of the Input One signal will be amplified and distributed to the Input One terminal of the analog matrix described later. There will be a manual adjustment for the chrominance of the distributed signal. The video portion of the Input One signal and all the other external video signals will be distributed through the Design Device as non-composite 0 to + 2 Volts.

Hardwired to Input One will be a Chroma Decoder system nearly identical to ones used on "solid state" television receivers. The RGB outputs of the decoder will be analyzed by comparators with manually adjusted thresholds, and the consequent logic pulse will be distributed into the Design Device matrix.

Input Two is for a black and white camera which is used to rescan a RuttEtra Display. The camera will receive its sync from the Design Device sync generator. Input Three is for any vertical lock vtr. This signal is processed through an analog comparator and is then loaded into a one bit time base corrector. The time base of input Three is matched to Input One within a window of one line and will have horizontal resolution of 256 points. Input Four accepts an external video camera, either black and white or encoded color, which is synched from the Design Device Sync Generator.

The Input Module will drive a video Sync Generator which will synthesize all timing signals not derived from Input One. In the case of a black and white signal, the Sync Generator will provide a free running 3.58 MHz subcarrier and a horizontal drive triggered color burst flag which can be used to synthesize color. In the case of an EIAJ heterodyned color signal, the subcarrier will be derived from the incoming signal. In these cases, the output of the Design Device will not be directly broadcast compatible, but it may be made so by subsequent time base correction. The Sync Generator will also provide a 4.09084 MHz clock pulse ( 260 x H) phase locked to the subcarrier and 15,750 and 60 Hz sawteeth triggered by the horizontal and vertical drives.

The signal processing modules of the Design Device will consist of individually addressable voltage controlled and digital devices with compatible inputs and outputs. Control voltages will have a +5 volt range.

The primary oscillator of the Design Device will be an 256 x 8 bit random access memory (RAM) with an 8 bit digital to analog converter (DAC) as an output buffer. The contents of the memory

will be loaded from a Camera Controller. Waveforms to be loaded into the RAM will be prepared on specially formatted graph paper marked in a rectilinear grid of 256 x 256 lines. The two RAM modules on the Design Device will share one Camera Controller. A switch on the Controller will select which memory is being loaded.

The RAM modules will be read out by a Programmable 8 Bit Counter. The Counter can be advanced by inputting an external trigger. This trigger is subdivided by a number derived from a voltage controlled counter system which essentially is an inexpensive digital to analog converter. Normally the Programmable counter runs from 0 to 255 and then recycles, but provision is made of computer addressing to preset and reset the counter. When implemented subsections of the RAM can be read out selectively.

 

 

8 Bit Programmable Counter Architecture

The Wipe Generator will be built around a triangle wave oscillator which can be varied in frequency and phase from one to ten

 

The inverter circuit consists of a high speed op amp and is intended for reversing polarity of a video signal while maintaining its black level clamp.

The Voltage Controlled Mixer is based on the MC 1445 integrated circuit. It can be used as a two quadrant multiplier, a video mixer, a keyer, a transparent keyer, a pulse width modulator and a voltage controlled amplifier. Detailed construction designs

for this circuit have been made public by Dan Sandin. The 1445 has also been used by Steve Rutt and Carl Geiger in their synthesizer designs.

All video and analog control signals are interconnected in the Design Device through an electronically switched 32 line signal bus. The output of each module can be delegated to four of 32 bus lines. The inputs to each module can be selected from any one of the 32 lines. A block of eight lines is automatically clamped for video. The remaining 24 lines are fed through amplifiers with manual adjustments for bias and gain. outputs can be additively mixed onto a single line without distortion. Any one input line can be shared by all device inputs, but a device can select only one line per input. A section of the signal bus is shown in figure 2.

The switching system is notated graphically, and programs are written by placing marks at the cross points to be closed. The grap,11"is mounted in front of two 32 x 32 CCD cameras, one for the Output selection and one for the Input selection. Exact registration of the program graphic is determined by use of an alignment oscilloscope shared by the two cameras and driven by their onboard DACs. The outputs from the cameras are distributed along the Design Device backplane. Six lines are distributed in an end to end bus and 160 lines ( 32 for the Input multiplexers and 128 for the Output multiplexers) are individually connected from decoders to discrete 8 wide fet multiplexed switches. The address bus and the decoders can be adapted to be operated by computer to take advantage of computer mass storage, precision timing and algorithm programming. While under direct CCD camera control, switching will take 1/30 of a second and will cause one to two bad video frames. Attached are specifications for the CCD camera tobe used. Figure three is a block diagram of the Input line selection system.

The matrix graph appears on the following page
It would be the "fretboard" of the Graphic Guitar
The frets are read graphically by use of a solid state
sensor, The CROMEMCO Image Sensing Module.

The Design Device Colorizer-Sync Adder will have input channels for the chrominance, luminance and hue parameters. The Hue will be voltage controlled through a 1440 phase shift and will make a complete 360 rotation for every two volts applied to the input. Rotation will be based on an eight channel comparator system, an idea suggested by Don Buchla. A block diagram of the hue Phase shifter is shown in figure 4. Chrominance is the control voltage for the amplification of the Hue channel. Luminance will be added to the output of the chrominance amplifier. If an incoming signal already contains a subcarrier, its phase and amplitude can be adjusted at Input One, so that the Colorizer can pass a color signal without distortion. The Colorizer-Sync Adder will also serve as a Proc Amp. Pedestal, Gain, Sync Level and Burst Amplitude will have manual controls.

 

Feasibility

Designing the Design Device is the project of Tom DeWitt under a grant from the National Endowment for the Arts. This $10,000 grant issued in April 1975 will cover his living expenses; it is not intended to purchase components or pay for the services of engineering specialists whose participation is certainly required if the project is to produce more than a heuristic block diagram of the proposed instrument.

The concepts behind the Design Device have received encouraging responses from video artists and engineers. Among those who have expressed a desire to work on the project are Phil Edelstein, an expert in small computer systems and artist-in-residence at WNET's TV Lab; Steve Rutt, designer of the RuttEtra synthesizer and operator of Rutt Electrophysics in New York City; George Kindler, an RPI trained electronics engineer with wide design and construction experience who is also co director of the Electronic Body Arts dance group in Albany; and Roger Kent, a well versed sculptor, calligrapher, graphic designer who has just completed layout work and construction of two large Buchla 500 synthesizers. Moral and some technical support has come from Carl Geiger, Robert Moog and Don Buchla.

Test equipment and working facilities for building the Design Device are located in the Albany area at WMHT TV and the Electronic Music Studio of SUNYA. WMHT, which is sponsoring this grant proposal, is a complete broadcast television station with a color video cassette system recently installed by NYSCA for