[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: [microsound] data -> sound examples



> some point? Who else has traveled the "pure data" route?

In the mid-90s, I worked on a few projects using NetPBM, which allows
for the translation of bitmaps into raw, sequential text files.  It's a
fairly straightforward Unix program that uses pipes to go from format to
format.  At that time I primarily used it for image manipulation, i.e.
pinhole camera photos->bitmaps->10'x10' ASCII images.  At the same time,
I was tooling around with a program for the Amiga call AudioMaster II
(by Aegis), which has a neat feature that allows you to scan the
entirety of the Amiga's memory and play it back as sound.  I created one
track with this technique, using the raw sound to create a feedback
noise piece, aptly named "Digital Noise" (Two Man Psycho Jam, 1995).
For the sake of this conversation, I've posted an mp3 of this up at
http://www.csorg.org/audio.html.

I started investigating this idea again in 1998-99, translating raw text
and bitmaps into sound and vice versa using custom A/D hardware and
software, and NetPBM, Photoshop and SoundForge.  Programs like MetaSynth
and Coagula tend to interpret or map images into "sensible" sound data,
which isn't, IMHO, very much different from color organs in the late
19th and early 20th century.  While the arbitrary assignation of
color/image->sound can be a neat trick, the process isn't unique to a
computer.

What I have been particularly interested in is the idea that all data
inside the computer is essentially the same, and that it just takes
someone to "peel the skin", and peer inside, either with ears or eyes,
or whatever senses we care to translate the switching of 1s and 0s into.
I relate this activity to the descriptions of synesthetics in "The Man
Who Tasted Shapes" by Richard Cytowic.  Synesthetics seem to somehow be
"listening in" on their limbic systems, cutting through the cerebral
cortex for some genetic cross-modal associations.  This cross-modal
association is quite consistent for the individual, although there do
not seem to be any consistencies from synesthete to synesthete.

It appears to me that we pile on logical, interpretive interfaces to the
raw data stream of the computer, drawing loose analogies to the
limbic<->cerebral cortex relationship, as Cytowic describes it.  To me,
the most interesting thing about "data-bending" (as some have dubbed it)
is letting the data speak for itself, trying to listen to the data
stream with as little interference as possible.

> "Data art" has been fashionable in net.art circles for a while now... 
> and Lev Manovich has started writing his "info-aesthetics" project... 
> but nobody much seems to be talking about the aesthetics of 
> data.sound 
> (correct me if I'm wrong)...

I've found some interesting things about shape and form when translating
between images and sound, and from there I can extrapolate some other
ideas about data-bending.  If you're used to working with raw data, it's
a very logical process (but still sometimes difficult to predict).  For
instance, taking a raw, 16-bit, unsigned, 48kHz, -6db sine (or square)
wave and converting it into an image will result in some sort of
dark/light grey striped image (depending on how you format the pixels).
Every two bytes (for 16-bit audio) is read in as a value, sequentially,
creating the peaks and valleys of the sine or square wave.  The louder
the sound, the brighter and darker the values (greater dynamics =
greater value range).  Sines, ramps, and triangle waves produce
gradients, while square waves produce more "posterized" images.  Tighter
striped images and grids tend to produce higher frequency sounds, while
broad stripes result in lower frequencies.  Rotating images 90 degrees
changes sound drastically, as the data is read in sequentially, always
from left to right, top to bottom.  Of course, BMPs and some other image
formats reverse this image reading process, so that may be another area
to explore.

In my work, sounds should be in separate channels to produce color
images (sound1->R, sound2->G, sound3->B).  Interpreting sound->image and
image->sound in this way makes creating moving images quite sticky.
Sound has a much lower data rate than video, so, at 16-bits and 48kHz,
3200 bytes of sound will translate into a 40x40 frame video image (if my
math is correct this late at night).  Let's see:

	1 second of audio = 48000 samples
	48000/30FPS = 1600 pixels (40x40, 16-bit) OR 3200 bytes

Looks alright, I guess.  Of course, reading an image or file from left
to right seems a bit Western-biased and arbitrary as well, so I'm
interested in that kind of thinking, which seems to dive into the core
of how we have developed computers.  We still read in images and sound
sequentially, and sound data is still played back sequentially at the
hardware level, regardless of how many Twerks, Herberts and Prefuses
rearrange it internally.  

Most of my time in the past few years has been spent guiding and
reinterpreting data streams in a variety of ways, trying to figure out
the aesthetics of a pure, sequential data stream, what it will look like
if it starts as sound, what it will sound like if it begins as image.
Most data headers are noisy clicks, as are most data files that are
non-repetitive in nature.  I anticipate that text files are largely
uninteresting because we use so little of the "dynamic range" of bits
and bytes (ASCII typically runs from 32-123, and usually narrower than
that), but I haven't thoroughly explored that.  Of course, repeating
(very) small chunks of data result in all sorts of interesting sounds
that can be fruitful and rewarding both aurally and visually.

I'd be interested in continuing this discussion.  I'd also like to hear
what people think about the notion of "sequential", "left-to-right",
"in-to-out", and how it affects our perceptions regarding this process.

> Thanks,
> 
> Mitchell
> http://www.ce.canberra.edu.au/staff/mitchellwhitelaw


Great topic.

Christopher

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     Christopher Sorg
   Multimedia Artist/Instructor
 The School of the Art Institute of Chicago
  Columbia College Chicago
   http://www.csorg.org
     csorg@xxxxxxxxx
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.693 / Virus Database: 454 - Release Date: 5/31/2004
 


---------------------------------------------------------------------
To unsubscribe, e-mail: microsound-unsubscribe@xxxxxxxxxxxxx
For additional commands, e-mail: microsound-help@xxxxxxxxxxxxx
website: http://www.microsound.org