Re: [microsound] photosynthesis

[Christopher Sorg <csorg@xxxxxxxxxxxx>]

I've done quite a bit of work with this (starting in 1995 with my
Amiga and in 1998 with Photoshop) and I'm pretty far along in developing
some processes and work I'm happy with.  Initially, with the Amiga, you
could use AudiomasterII to scan a disk and play back all the data as
sound.  This resulted in (mostly) some pretty insane digital noise.  

With Photoshop and SoundForge you can import/export raw files, so that
allows you to work with any sort of file format you would like.  I started
with some JPEGs of Kandinsky paintings for an ironic image->sound
translation, at first animating the image with AfterEffects, then using
close-up stills of the paintings and transferring them to sound samples.
Different image formats have different header information describing the
file data, so there is a distictive click at the beginning of each
"sound" image  you import as raw data into SoundForge.  You can avoid this
by saving your Photoshop file as RAW file type.  Also, it's best to use
uncompressed images (TIF or RAW) for a direct 1-1 correlation between
sound and image.

Things I've discovered about this process:

- greyscale images are easier to learn with

- it's easier to start with a simple image or basic sounds (like sine 
waves)

- horizontal images will be very quiet or silent

- angled, abstract shapes produce (typically) more interesting sounds 
(i.e. 30-45 degree black and white stripes create a sine-wave like sound)

- you can calculate the length/sample-rate of your sound by pixels
  (210x210 pixels = 1 sec of 44.1K audio)

- mixing three greyscale sounds would result in an RGB image

- increase the sampling rate (96K) for better video/animation

- go back and forth between sound and image filters.  Posterizing and 
mosaic filters do some interesting EQ'ing and aliasing.  EQing does 
strange things to an image.

Essentially the process is taking a matrix (the X-Y visual grid) and
stretching it out, pixel for pixel as sound.  So if you visuallize this
process, you can get some pretty predictable results.  Also, any sort of
file will work.  Text files are "raw" already, so you could do some neat
things with image->text->audio, run a spell check or grep the file or
something weird like that.  Any file will do.  I'll upload some samples to
the microsound server and post the directory tonight.

Also, I've worked with much of the image->sound software that is out
there, and while I think it's neat, it's much more arbitrary and less
direct than playing with the pure chunks of data.  For instance, one
program translates RGB into channel data (left-right-mono), while up/down
maps to pitch and brightness maps to volume.  This is fine, but it's
really just a reconfiguration of standard musical notation.  Working with
RAW data is a different story; you have to think like the computer, think
about how it is looking at the data.  Understanding file formats,
resolutions and bit-depths, etc...

It's good fun!

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     Christopher Sorg
   Multimedia Artist/Instructor
 The School of the Art Institute of Chicago
   http://csorg.cjb.net
     csorg@xxxxxxxxx

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~