I'm looking to find out more about writing some software that will use
traditional classical instruments to emulate "natural" or "non musical
sounds." The software will perform some type of analyses on an audio
file, I imagine FFT would be used at some point, but the problem with
FFT is that it only tells you what "perfect" or pure sine wave based
frequencies are present in a sound. Besides the flute, not much else
in an orchestra has anything close to a sine wave output. After this
analysis is done, the software will look through a library of sounds
made by traditional instruments. These sounds will include every
noise and playing style every traditional instrument can produce. The
software will then juggle the sounds around at various dynamic levels
in various rhythms and etc until it comes up with the closest
combination to the original sound. Perhaps a car engine sound file
would yield three Double Basses, a flute or two in very quiet
irregular rhythms, and maybe a horn would be involved during gear
changes. I might not have to tell you that Gyorgy Ligeti's
"Atmospheres" and his "Mechanical Music" served as the chief
inspiration for this idea.

Has anybody ever heard of anything like this, or know where I might
start to look for info on this subject? I'm not looking for
programming help, but rather, help with setting up the math. Are
there any scientific communities online that I could point my
questions to? Any books on this type of thing. I've heard Csound
might work for this. I thought Csound was for composing, not for
analyzing existing sound files. I can't seem to come up with the
right keywords to get anything out of Google, but I hoped someone here
might be able to put me on the right path.

(Ryan) wrote in message . com...
I'm looking to find out more about writing some software that will use
traditional classical instruments to emulate "natural" or "non musical
sounds." The software will perform some type of analyses on an audio
file, I imagine FFT would be used at some point, but the problem with
FFT is that it only tells you what "perfect" or pure sine wave based
frequencies are present in a sound. Besides the flute, not much else
in an orchestra has anything close to a sine wave output. After this
analysis is done, the software will look through a library of sounds
made by traditional instruments. These sounds will include every
noise and playing style every traditional instrument can produce. The
software will then juggle the sounds around at various dynamic levels
in various rhythms and etc until it comes up with the closest
combination to the original sound. Perhaps a car engine sound file
would yield three Double Basses, a flute or two in very quiet
irregular rhythms, and maybe a horn would be involved during gear
changes. I might not have to tell you that Gyorgy Ligeti's
"Atmospheres" and his "Mechanical Music" served as the chief
inspiration for this idea.

Has anybody ever heard of anything like this, or know where I might
start to look for info on this subject? I'm not looking for
programming help, but rather, help with setting up the math. Are
there any scientific communities online that I could point my
questions to? Any books on this type of thing. I've heard Csound
might work for this. I thought Csound was for composing, not for
analyzing existing sound files. I can't seem to come up with the
right keywords to get anything out of Google, but I hoped someone here
might be able to put me on the right path.

Ryan,

You might want to visit the Fourier analysis again... my understanding
is that with it, you are not just determining the fundamental
frequency of a sound, but all the other frequencies present in it, as
well. The key is that any given sound IS a collection of sine waves,
at different intensities, with different relationships in *time*.

For example, a square wave is a sine wave at the fundamental, then a
series of harmonics (3rd, 5th, 7th, 9th, etc.) in diminishing
amplitude, in a specific arrangement. Fourier can describe this
arrangement.

What you are attempting to do with sounds reminds me of those posters,
where one large picture (say, of a person) is made up of hundreds of
smaller pictures. The movie poster for "The Truman Show" starring Jim
Carrey comes to mind. Perhaps some of the math or the code from that
system may work for what you are doing.

Regards,

Karl Winkler
Lectrosonics, Inc.
http://www.lectrosonics.com

On Tue, 12 Oct 2004 07:09:24 -0700, Karl Winkler wrote:

snip
What you are attempting to do with sounds reminds me of those posters,
where one large picture (say, of a person) is made up of hundreds of
smaller pictures. The movie poster for "The Truman Show" starring Jim
Carrey comes to mind. Perhaps some of the math or the code from that
system may work for what you are doing.

There is a free program called "Soundmosaic" that does exactly this. It
sorta works.

http://thalassocracy.org/soundmosaic/

(Some people here may appreciate the demo of a George Bush speech
combined with a chimp screaming.)

And "Dissasociated studio" which does the same kind of thing, but within a
single audio file.

http://www.panix.com/~asl2/music/dissoc_studio/

Ryan wrote:
I'm looking to find out more about writing some software that will use
traditional classical instruments to emulate "natural" or "non musical
sounds." The software will perform some type of analyses on an audio
file, I imagine FFT would be used at some point, but the problem with
FFT is that it only tells you what "perfect" or pure sine wave based
frequencies are present in a sound.

No. ANY arbitrary waveform can be decomposed down to sine waves. When you
put the sines back together, you can reconstitute the original wave. This is
the WHOLE POINT of the Fourier series. The time domain and frequency domain
representations of the waveform are equivalent and you can convert from one
to the other and back with impunity.

Besides the flute, not much else
in an orchestra has anything close to a sine wave output. After this
analysis is done, the software will look through a library of sounds
made by traditional instruments. These sounds will include every
noise and playing style every traditional instrument can produce. The
software will then juggle the sounds around at various dynamic levels
in various rhythms and etc until it comes up with the closest
combination to the original sound.

Why use a computer for this anyway? George Gershwin did a perfectly good
job of this by ear.
--scott
--
"C'est un Nagra. C'est suisse, et tres, tres precis."

(Scott Dorsey) wrote in message ...


Hi Scott. How have you been? Heard anymore Sonic Youth of late?

Ryan wrote:
I'm looking to find out more about writing some software that will use
traditional classical instruments to emulate "natural" or "non musical
sounds." The software will perform some type of analyses on an audio
file, I imagine FFT would be used at some point, but the problem with
FFT is that it only tells you what "perfect" or pure sine wave based
frequencies are present in a sound.

No. ANY arbitrary waveform can be decomposed down to sine waves. When you
put the sines back together, you can reconstitute the original wave. This is
the WHOLE POINT of the Fourier series. The time domain and frequency domain
representations of the waveform are equivalent and you can convert from one
to the other and back with impunity.

So what I have to do is perform FFT on each of my sound "samples", the
squeak of a vilon played behind the bridge, a viol's "dry string"
sounds, regular arco, pizzicato, etc, ect, all the ohter instruments,
etc. And then perform an FFT on any given sound file I'm interested
in emulating. After that, what kind of math would be used to sort
through all the samples and figure what goes best where?

Samplitude features an FFT analyses window. It just looks like a
regular EQ anlysis to me. Is it the case that if I take each
frequency as a sine wave and apply it to the given amplitude that I
will have achieved X's sound? Is there anyway to simplify that? Even
the simplest natural sounds have about a 5khz range. Do I have to
create 5000 individual sine waves? The FFT graph only shows frequency
over time, How do I find out about the relationships between the
frequencies as far as timming? For example say a put a sine wave at
2Khz and 1Khz. Obviously the 2Khz occilates twice as fast as the 1
Khz, but beyond that, the starting/ending points (where y=0) might not
sink up. The 2Khz sine may start, say, 300ths of a second after the
1Khz. I don't think info like this can be found out by the FFT
window, can it?

Do I have this right at all, or am I still nopt grasping Fourier
transforms?

Besides the flute, not much else
in an orchestra has anything close to a sine wave output. After this
analysis is done, the software will look through a library of sounds
made by traditional instruments. These sounds will include every
noise and playing style every traditional instrument can produce. The
software will then juggle the sounds around at various dynamic levels
in various rhythms and etc until it comes up with the closest
combination to the original sound.

Why use a computer for this anyway? George Gershwin did a perfectly good
job of this by ear.
--scott

The hope is to use this as a learning tool and eventually stop using
it, not unlike training wheels on a bicycle. I could probably do a
decent job of this in a tonal 4/4 world, but most real life sounds
contain dissonant and microtonal intervals, as well as many
"co-rhythms" that work together to create larger aspects of the sound,
such as pulses, and trigonometric polynomials. Making something that
sounds like a train whistle is one thing. I imagine it would have
been rather difficult for a composer of even Gershwin's skill to
notate out the sound of a babbling brook during a rain storm, with a
distant propeller airplane heard off in the far distance. It would
break my head, not to mention take a considerable amount of time for
me to do this by ear. Whereas with a system of this sort, I could run
twenty analyses and in a day know far more about this type of
orchestration than I would in a month if I did it all in my head. I
would gain a good overall knowledge that I can use as starting points
for future works, I would have a "feel for it". On the other hand,
doing this all by ear until I figure out how to make it work, is like
finding out the details first and only later getting the overall
picture--not the most efficient way of working. Like trying to
complete a jigsaw puzzle with no picture of what the finished puzzle
looks like. Learning the individual interactions between the parts
does not always lead to a good understanding of the whole. Anyway, I
learn best working from the outside in.

Ryan wrote:
(Scott Dorsey) wrote in message ...

Hi Scott. How have you been? Heard anymore Sonic Youth of late?

I'm listening to Toots and the Maytals as I type this...

No. ANY arbitrary waveform can be decomposed down to sine waves. When you
put the sines back together, you can reconstitute the original wave. This is
the WHOLE POINT of the Fourier series. The time domain and frequency domain
representations of the waveform are equivalent and you can convert from one
to the other and back with impunity.

So what I have to do is perform FFT on each of my sound "samples", the
squeak of a vilon played behind the bridge, a viol's "dry string"
sounds, regular arco, pizzicato, etc, ect, all the ohter instruments,
etc. And then perform an FFT on any given sound file I'm interested
in emulating. After that, what kind of math would be used to sort
through all the samples and figure what goes best where?

I'm not sure this will really do what you want, but you can try it. You
could just do a standard correlation coefficient and see how close they
come.

Then again, you could probably just do a correlation coefficient on the
samples themselves. That might be fun to look at.

Samplitude features an FFT analyses window. It just looks like a
regular EQ anlysis to me. Is it the case that if I take each
frequency as a sine wave and apply it to the given amplitude that I
will have achieved X's sound? Is there anyway to simplify that? Even
the simplest natural sounds have about a 5khz range. Do I have to
create 5000 individual sine waves? The FFT graph only shows frequency
over time, How do I find out about the relationships between the
frequencies as far as timming? For example say a put a sine wave at
2Khz and 1Khz. Obviously the 2Khz occilates twice as fast as the 1
Khz, but beyond that, the starting/ending points (where y=0) might not
sink up. The 2Khz sine may start, say, 300ths of a second after the
1Khz. I don't think info like this can be found out by the FFT
window, can it?

No, you probably want a tool like matlab. How many terms you want to
calculate out to depends on how good an approximation you want. I think
that the number of terms that you're going to get is going to be larger
than the number of samples in the original file for most arbitrary sounds.
You can decide to reduce this by bandlimiting the original signal, though.
--scott
--
"C'est un Nagra. C'est suisse, et tres, tres precis."

(Scott Dorsey) wrote in message
I'm listening to Toots and the Maytals as I type this...

Woah. New stuff? I haven't even heard the title before. I'm kinda
weening off the Youth a little bit. Nowadays I'm really into Ligeti.
Have you heard his "Atmospheres" or his "San Fransisco Polyphony", or
his "Continuum (fur Cembalo)"? My goodness! They're must-listens.
He has some of the most revolutionary music I have ever heard. I'm
sure you will understand my wanting for a type of software like this
once you hear these pieces, if you haven't heard them already.

So what I have to do is perform FFT on each of my sound "samples", the
squeak of a vilon played behind the bridge, a viol's "dry string"
sounds, regular arco, pizzicato, etc, ect, all the ohter instruments,
etc. And then perform an FFT on any given sound file I'm interested
in emulating. After that, what kind of math would be used to sort
through all the samples and figure what goes best where?

I'm not sure this will really do what you want, but you can try it. You
could just do a standard correlation coefficient and see how close they
come.

Then again, you could probably just do a correlation coefficient on the
samples themselves. That might be fun to look at.

A correlation for the whole sound file, or a correlation every set
number of seconds, or a type of gui tool to use to set up the sections
you want to emulate. This would be good for Monophonic reduction, but
more math would be involved if you wanted to reduce the sound file to,
say, 3 concurrent, or 13 concurrent instruments, right?

Ideally this software would/could use both of these approaches.

Samplitude features an FFT analyses window. It just looks like a
regular EQ anlysis to me. Is it the case that if I take each
frequency as a sine wave and apply it to the given amplitude that I
will have achieved X's sound? Is there anyway to simplify that? Even
the simplest natural sounds have about a 5khz range. Do I have to
create 5000 individual sine waves? The FFT graph only shows frequency
over time, How do I find out about the relationships between the
frequencies as far as timming? For example say a put a sine wave at
2Khz and 1Khz. Obviously the 2Khz occilates twice as fast as the 1
Khz, but beyond that, the starting/ending points (where y=0) might not
sink up. The 2Khz sine may start, say, 300ths of a second after the
1Khz. I don't think info like this can be found out by the FFT
window, can it?

No, you probably want a tool like matlab. How many terms you want to
calculate out to depends on how good an approximation you want. I think
that the number of terms that you're going to get is going to be larger
than the number of samples in the original file for most arbitrary sounds.
You can decide to reduce this by bandlimiting the original signal, though.
--scott

Terms? As in how many instruments I want to end up with? Or by what
specs I will measure the orignal soundfile? If the later, do you mean
something like bitrate, samplerate, something else? Why would the
number of terms be greater than the samplerate? Is matlab an audio
tool. Probably just a math program right? So I would enter in pcm
info and run the calculations and then use the output to create a pcm
file? Sorry so many questions.

Ryan wrote:

Terms? As in how many instruments I want to end up with? Or by what
specs I will measure the orignal soundfile? If the later, do you mean
something like bitrate, samplerate, something else? Why would the
number of terms be greater than the samplerate? Is matlab an audio
tool. Probably just a math program right? So I would enter in pcm
info and run the calculations and then use the output to create a pcm
file? Sorry so many questions.

Ryan, most of what you are asking about is well beyond the
state of the art, the art being DSP. I would suggest that
you go to comp.dsp and set forth what it is you want to get
more specific feedback about it.


Bob
--

"Things should be described as simply as possible, but no
simpler."

A. Einstein

(Ryan) wrote in message . com...
... I imagine FFT would be used at some point, but the problem with
FFT is that it only tells you what "perfect" or pure sine wave based
frequencies are present in a sound. Besides the flute, not much else
in an orchestra has anything close to a sine wave output....



No, that's the whole point of Fourier Analysis. ALL waveforms no
matter how complex can be de-composed into sine waves of various
amplitudes and phases.

Mark

Symbolic Sound's KYMA does resynthesis in real-time.

Gotta buy the box, though . . .




Kurt Riemann

Hi Ryan-

The sines and cosines that get used to build up a waveform in
Fourier analysis are the "basis functions" of the Fourier
transform. It is possible to decompose signals using many
different types of bases. The Fourier basis (sines and cosines,
harmonically related if the signal is of finite extent) has
some nice mathematical properties that make the decomposition
(and recomposition) simpler, mathematically, than it is with
many other bases. But that simplicity doesn't make the
Fourier basis "right" for all applications.

In your case, you want to use as "basis functions" the signals
played by standard instruments. These are much more complicated
than the sines and cosines in a Fourier basis. Besides the
fact that the sustained waveform from an instrument playing
a note has a non-sinusoidal shape, notes are transient (they
start and stop in time) and also dynamic (their pitch, volume,
and timbre vary in time, e.g., due to
tremolo, vibrato, etc.). Although it is mathematically possible
to represent signals with such dynamic, transient structure via
a Fourier transform, I don't think a Fourier decomposition
is well-suited to your problem.

One approach is to actually take samples of the instruments
you'll use, playing all the notes available, and use them (with
various durations) directly as your basis. This would be the most
accurate approach, but the calculations you'd need to do to find
the expansion coefficients (i.e., the score!) would probably
be extremely difficult computationally, and probably not
well-defined (the basis is likely neither complete nor
orthogonal). You'd be doing something like additive synthesis,
but with a much bigger basis than is usually used! Looking
up some of the math associated with additive synthesis might
provide you with some leads.

A possible option that has the potential to be more computationally
tractible would be to use some kind of wavelet or other
time-scale or time-frequency transform rather
than a Fourier transform. Very roughly speaking, you can
think of such a transform as breaking up a signal into
*localized* pulses, i.e., notes! That is, where a Fourier
transform represents a signal as a sum of "eternal" sines
and cosines of specific frequencies, a time-frequency transform
breaks up the signal into separate parts that are localized both in
frequency *and* time. You might be able to find some way to
project a wavelet or other time-frequency transform of the sound
you are interested in onto the transforms of sounds from the
instruments you have available; this would give you the notes
and volumes needed to most closely match the desired signal.
This won't make any fundamental problems with the incompleteness
or redundancy of your basis (choice of instruments & notes) go
away, but use of such transforms might provide methods of
approximation that make the problem more tractable computationally.

A google search on "wavelets" and "music" will probably get you
started. This wavelet FAQ might also help:

http://www.math.ucdavis.edu/~saito/c...avelet_faq.pdf

Here's a review article on time-frequency analysis of sounds
from musical instruments---your basis functions, so to speak:

http://epubs.siam.org/sam-bin/getfil...cles/38228.pdf

If you want to learn more about Fourier expansions from
a musical point of view, see:

http://ccrma.stanford.edu/~jos/mdft/

Here's a reference that turned up in my own quick googling using
"time scale transform music" that may provide a starting point
for thinking along these lines, if you can find a copy:

Kronland-Martinet R., Grossmann A. "Application of time-frequency and
time-scale methods to the analysis, synthesis and transformation of
natural sounds." in "Representations of Musical Signals", C. Roads,
G. De Poli, A. Picciali Eds, MIT Press, october 1990.

Interlibrary loan may help you here!

A similar search using "time frequency transform music" turned up
"Musical Transformations using the Modification of Time-Frequency
Images" in a 1993 issue of *Computer Music Journal*:

http://mitpress.mit.edu/catalog/item...d=6768&ttype=6

This is just from some quick googling and these are probably not
the best or most recent references that may be relevant. Wavelet
and time-frequency analysis is now very mature and there are
entire textbooks and monographs on these topics. Good
luck with this.

Peace,
Tom Loredo

--

To respond by email, replace "somewhere" with "astro" in the
return address.

Tom Loredo wrote in message ...
Hi Ryan-

The sines and cosines that get used to build up a waveform in
Fourier analysis are the "basis functions" of the Fourier
transform. It is possible to decompose signals using many
different types of bases. The Fourier basis (sines and cosines,
harmonically related if the signal is of finite extent) has
some nice mathematical properties that make the decomposition
(and recomposition) simpler, mathematically, than it is with
many other bases. But that simplicity doesn't make the
Fourier basis "right" for all applications...

Thank you for this copious amount of unsolicited information. It is
already proving useful.

(Ryan) wrote in message . com...
I'm looking to find out more about writing some software that will use
traditional classical instruments to emulate "natural" or "non musical
sounds." The software will perform some type of analyses on an audio
file, I imagine FFT would be used at some point, but the problem with
FFT is that it only tells you what "perfect" or pure sine wave based
frequencies are present in a sound. Besides the flute, not much else
in an orchestra has anything close to a sine wave output. After this
analysis is done, the software will look through a library of sounds
made by traditional instruments. These sounds will include every
noise and playing style every traditional instrument can produce. The
software will then juggle the sounds around at various dynamic levels
in various rhythms and etc until it comes up with the closest
combination to the original sound. Perhaps a car engine sound file
would yield three Double Basses, a flute or two in very quiet
irregular rhythms, and maybe a horn would be involved during gear
changes. I might not have to tell you that Gyorgy Ligeti's
"Atmospheres" and his "Mechanical Music" served as the chief
inspiration for this idea.

Has anybody ever heard of anything like this, or know where I might
start to look for info on this subject? I'm not looking for
programming help, but rather, help with setting up the math. Are
there any scientific communities online that I could point my
questions to? Any books on this type of thing. I've heard Csound
might work for this. I thought Csound was for composing, not for
analyzing existing sound files. I can't seem to come up with the
right keywords to get anything out of Google, but I hoped someone here
might be able to put me on the right path.

I know it's not what you had originally asked, but give a listen to
the first few bars of Mahler's 1st symphony, last movement. Closest
thing I've heard to an orchestra sounding like a jet engine, without
intentionally doing so.

-Karl