speech recognition

The idea is very basic

The outputs(the mic's) are a linear combination of the input sources.
The coefficients give the relative magnitude of each source contributing
to the mic. If a source is further away it will contribute less to the
output. In fact, the coefficient should be 1/distance^2. If the distance
is large 1/distance^2 is small and vice versa.

On 1/21/2012 12:29 AM, Jeffery Tomas wrote:
The idea is very basic

The outputs(the mic's) are a linear combination of the input sources.
The coefficients give the relative magnitude of each source contributing
to the mic. If a source is further away it will contribute less to the
output. In fact, the coefficient should be 1/distance^2. If the distance
is large 1/distance^2 is small and vice versa.

That's true in the far field, but telephone microphones operate in the near
field. One of the effects for sound is that the pressure/velocity ratio is
frequency dependent there.

Jerry

The idea is very basic

The outputs(the mic's) are a linear combination of the input sources. The
coefficients give the relative magnitude of each source contributing to the
mic. If a source is further away it will contribute less to the output. In
fact, the coefficient should be 1/distance^2. If the distance is large
1/distance^2 is small and vice versa.

O_i = sum(1/a_k^2*S_k)

or

O = A*S

where O is the output's(mics), A is the "conversion matrix" and S is the
sources(which is what we want to recover). From basic linear algebra we know
S = A^-1*O.

There are statistical and deterministic methods to determine A since it is
essentially the "transfer function". The important thing is to recognize is:
If A is known and the sources and sinks(mics) are time independent(or at
least slowly changing) then we can recover the sources. The more accurate we
know A the more accurate our decomposition will be.

I'll demonstrate for two sources and sinks but I'll use inverted linear
distance wlog

O1 = a*S1 + b*S2
O2 = c*S1 + d*S2

Inverting gives

S1 = (dO1 - bO2)/(ad - cb)
S2 = -(cO1 - aO2)/(ad - cb)

Now the problem is to determine the coefficients.

This is simple a problem of plugging in values:

Analogy: y = a + bx + cx^2

How do we find a b and c? Well if we know 3 pairs of points we just plug in
and solve the linear system of equations.

Unfortunately we do not know the Source vector. BUT if we "listen" for
common situations we can easily narrow down the search and possibly arrive
at correct coefficients.

Now since we know the outputs we know there ratio. Assume O1/O2 = x and
S1/S2 = y then

y = -(dx - b)/(cx - a)

now this is like the polynomial sampling problem I gave above. We just don't
have our points (x,y) to determine the coefficients.

There are several ways to potentially get y such as negative feedback,
guestimating, kalman filters, monte carlo, etc...

e.g., if we guess at the coefficients we arrive at a certain geometrical
scenario that will either coincide with the real one or be "off"(producing
estimated outputs that do not jive with what we are actually getting... we
can perturb our coefficients until they "wiggle" there way to the correct
values using appropriate adaptive methods).

If we know certain properties of the sources then we can include those in
the algorithms to reduce the complexity/increase the effectiveness of the
algorithms.

For example, by knowing that S1 and S2 are uncorrelated we can determine
that when S2 = 0 then we end up with a 1 source system making it much easier
to solve. e.g., our equation reduces to

S1 = aO1 + bO2

and instead of 4 coefficients we only have to find 2. This results in a
triangular geometry instead of a quadrilateral. Moreover, by symmetric we
can do the same for S2 and we end up with another easy system to solve. This
applies to any number of sources.  With uncorrelated sources we should be
able to treat the system as just one source a time.

For such indepence the coefficients are directly related to the distances
and can be easily calculated from the mic intensities. (a = 1/d1^2 and b > 1/d2^2 but a is the level from O1 and b is the level from O2)

In general though it is not the case that the sources are uncorrelated or
time independent and more advanced tricks or general approximations may have
to be used.

The more I think about it the more it seems like it might be quite accurate
for most signals and setups. One problem though is frequency response as we
are assuming a lossless transmission which is usually not the case and could
cause problems with some algorithmic approaches.

The idea is very basic

The outputs(the mic's) are a linear combination of the input sources. The
coefficients give the relative magnitude of each source contributing to
the mic. If a source is further away it will contribute less to the
output. In fact, the coefficient should be 1/distance^2. If the distance
is large 1/distance^2 is small and vice versa.

O_i = sum(1/a_k^2*S_k)

or

O = A*S

where O is the output's(mics), A is the "conversion matrix" and S is the
sources(which is what we want to recover). From basic linear algebra we
know S = A^-1*O.

There are statistical and deterministic methods to determine A since it
is essentially the "transfer function". The important thing is to
recognize is: If A is known and the sources and sinks(mics) are time
independent(or at least slowly changing) then we can recover the sources.
The more accurate we know A the more accurate our decomposition will be.

I'll demonstrate for two sources and sinks but I'll use inverted linear
distance wlog

O1 = a*S1 + b*S2
O2 = c*S1 + d*S2

Inverting gives

S1 = (dO1 - bO2)/(ad - cb)
S2 = -(cO1 - aO2)/(ad - cb)

Now the problem is to determine the coefficients.

This is simple a problem of plugging in values:

Analogy: y = a + bx + cx^2

How do we find a b and c? Well if we know 3 pairs of points we just plug
in and solve the linear system of equations.

Unfortunately we do not know the Source vector. BUT if we "listen" for
common situations we can easily narrow down the search and possibly
arrive at correct coefficients.

Now since we know the outputs we know there ratio. Assume O1/O2 = x and
S1/S2 = y then

y = -(dx - b)/(cx - a)

now this is like the polynomial sampling problem I gave above. We just
don't have our points (x,y) to determine the coefficients.

There are several ways to potentially get y such as negative feedback,
guestimating, kalman filters, monte carlo, etc...

e.g., if we guess at the coefficients we arrive at a certain geometrical
scenario that will either coincide with the real one or be
"off"(producing estimated outputs that do not jive with what we are
actually getting... we can perturb our coefficients until they "wiggle"
there way to the correct values using appropriate adaptive methods).

If we know certain properties of the sources then we can include those in
the algorithms to reduce the complexity/increase the effectiveness of the
algorithms.

For example, by knowing that S1 and S2 are uncorrelated we can determine
that when S2 = 0 then we end up with a 1 source system making it much
easier to solve. e.g., our equation reduces to

S1 = aO1 + bO2

and instead of 4 coefficients we only have to find 2. This results in a
triangular geometry instead of a quadrilateral. Moreover, by symmetric we
can do the same for S2 and we end up with another easy system to solve.
This applies to any number of sources. With uncorrelated sources we
should be able to treat the system as just one source a time.

For such indepence the coefficients are directly related to the distances
and can be easily calculated from the mic intensities. (a = 1/d1^2 and b
= 1/d2^2 but a is the level from O1 and b is the level from O2)

In general though it is not the case that the sources are uncorrelated or
time independent and more advanced tricks or general approximations may
have to be used.

The more I think about it the more it seems like it might be quite
accurate for most signals and setups. One problem though is frequency
response as we are assuming a lossless transmission which is usually not
the case and could cause problems with some algorithmic approaches.

"Jeffery Tomas" <Jeffery_To...@Gmail.com> wrote:
The idea is very basic

The outputs(the mic's) are a linear combination of the input sources. The
coefficients give the relative magnitude of each source contributing to
the mic. If a source is further away it will contribute less to the
output. In fact, the coefficient should be 1/distance^2. If the distance
is large 1/distance^2 is small and vice versa.

O_i = sum(1/a_k^2*S_k)

or

O = A*S

where O is the output's(mics), A is the "conversion matrix" and S is the
sources(which is what we want to recover). From basic linear algebra we
know S = A^-1*O.

There are statistical and deterministic methods to determine A since it
is essentially the "transfer function". The important thing is to
recognize is: If A is known and the sources and sinks(mics) are time
independent(or at least slowly changing) then we can recover the sources.
The more accurate we know A the more accurate our decomposition will be..

I'll demonstrate for two sources and sinks but I'll use inverted linear
distance wlog

O1 = a*S1 + b*S2
O2 = c*S1 + d*S2

Inverting gives

S1 = (dO1 - bO2)/(ad - cb)
S2 = -(cO1 - aO2)/(ad - cb)

Now the problem is to determine the coefficients.

This is simple a problem of plugging in values:

Analogy: y = a + bx + cx^2

How do we find a b and c? Well if we know 3 pairs of points we just plug
in and solve the linear system of equations.

Unfortunately we do not know the Source vector. BUT if we "listen" for
common situations we can easily narrow down the search and possibly
arrive at correct coefficients.

Now since we know the outputs we know there ratio. Assume O1/O2 = x and
S1/S2 = y then

y = -(dx - b)/(cx - a)

now this is like the polynomial sampling problem I gave above. We just
don't have our points (x,y) to determine the coefficients.

There are several ways to potentially get y such as negative feedback,
guestimating, kalman filters, monte carlo, etc...

e.g., if we guess at the coefficients we arrive at a certain geometrical
scenario that will either coincide with the real one or be
"off"(producing estimated outputs that do not jive with what we are
actually getting... we can perturb our coefficients until they "wiggle"
there way to the correct values using appropriate adaptive methods).

If we know certain properties of the sources then we can include those in
the algorithms to reduce the complexity/increase the effectiveness of the
algorithms.

For example, by knowing that S1 and S2 are uncorrelated we can determine
that when S2 = 0 then we end up with a 1 source system making it much
easier to solve. e.g., our equation reduces to

S1 = aO1 + bO2

and instead of 4 coefficients we only have to find 2. This results in a
triangular geometry instead of a quadrilateral. Moreover, by symmetric we
can do the same for S2 and we end up with another easy system to solve.
This applies to any number of sources.  With uncorrelated sources we
should be able to treat the system as just one source a time.

For such indepence the coefficients are directly related to the distances
and can be easily calculated from the mic intensities. (a = 1/d1^2 and b
= 1/d2^2 but a is the level from O1 and b is the level from O2)

In general though it is not the case that the sources are uncorrelated or
time independent and more advanced tricks or general approximations may
have to be used.

The more I think about it the more it seems like it might be quite
accurate for most signals and setups. One problem though is frequency
response as we are assuming a lossless transmission which is usually not
the case and could cause problems with some algorithmic approaches.

Yes, but none of that has much to do with the how the Blind Source
Separation solutions actually work.  It's fare more interesting than what
you are suggesting above.  It makes none of the assumptions you talk about
above and instead directly calculates all 4 coefficients without knowing
anything about the sources.

I believe what it is doing, is using the fact that when you sum separate
non-correlated signals together, the sum tends to approach a Gaussian power
distribution.  The more uncorrelated signals you sum together, the closer
it becomes to a pure Gaussian distribution.

As such, by measuring how far away from a Gaussian distribution a given
signal is, it can pick the four coefficients which maximize the
non-Gaussian characteristics of the two extracted sources signals.  The two
sources signals will always be less Gaussian than the combined signal.  The
only limitation to the approach is that the signals must be generally
non-correlated, and they must not be Gaussian to start with (which is
generally true for all signals a human would care about to start with).

Hardy points out that in real life acoustic situations the summation is
never just linear.  However, I don't know how relevant that actually is..
If the source signals are arbitrary distorted BEFORE they are mixed, this
sort of approach can still separate the signals (back to their pre-mixed
distorted version).  And likewise, filtering  or distorting  the summed
signal likewise, will not prevent their separation.  So the fact that a
microphone and the recording system is non-linear is not by itself, a
problem, as long as the same transform is effectively applied to each
source _interdependently_.  But, I might be stupid here, because by
definition, a transform which is not linear, might, by definition, imply
the transform is creating dependency.  I'm not sure about that.

If the recording system (and the type of transform Hardy is talking about),
causes cross-modulation of the signals, then the resulting distribution
means the signals become at least partially correlated, which reduces how
well they can be separated.

But, as long as the linear combination dominates over any non-linear
effects, I would think this sort of a approach could still do a good job of
separation, meaning that the separated signals would be far better than the
mixed signals you started with.  And I would think that any audio recording
application that was wroth using, the linear effects would strongly
dominate over the non-linear effects.

--
Curt Welch                                            http://CurtWelch.Com/
c...@kcwc.com                                        http://NewsReader.Com/

RichD wrote:

http://www.sfgate.com/cgi-bin/article.cgi?f=/c/a/2012/01/16/BU8C1MOO20.DTL

He claims he can filter speech from background noise.

I recall discussing this possibility years ago. Someone said, these
filters already exist. They do - they're notch filters! It's close
to brain dead, believing that constitutes 'voice filtering'.

Dr. Watts has been working on this for years, so I was wondering
what techniques he's using, how much is public domain. Anyone
here know anything about the subject, or this product?
Is it neural nets, DSP filters, or what?

--
Rich

A friend of mine, Professor Dana Anderson of the University of Colorado,
Boulder, made a statistics-based digital filter that could separate
different kinds of music mixed together, as well as music from noise.
The demo was really striking--you mix together, say jazz and classical
music from two MP3 players, feed it through the gizmo, and after (iirc)
about 10 seconds of learning, classical comes out of one speaker and
jazz out of the other. Magic stuff--published in IEEE Acoustics around
2006, I think.

Cheers

Phil Hobbs

On Jan 22, 12:39=A0pm, c...@kcwc.com (Curt Welch) wrote:
"Jeffery Tomas" <Jeffery_To...@Gmail.com> wrote:
The idea is very basic

The outputs(the mic's) are a linear combination of the input sources.
T=
he
coefficients give the relative magnitude of each source contributing
to the mic. If a source is further away it will contribute less to
the output. In fact, the coefficient should be 1/distance^2. If the
distanc=
e
is large 1/distance^2 is small and vice versa.

O_i =3D sum(1/a_k^2*S_k)

or

O =3D A*S

where O is the output's(mics), A is the "conversion matrix" and S is
th=
e
sources(which is what we want to recover). From basic linear algebra
we know S =3D A^-1*O.

There are statistical and deterministic methods to determine A since
it is essentially the "transfer function". The important thing is to
recognize is: If A is known and the sources and sinks(mics) are time
independent(or at least slowly changing) then we can recover the
source=
s.
The more accurate we know A the more accurate our decomposition will
be=
.

I'll demonstrate for two sources and sinks but I'll use inverted
linear distance wlog

O1 =3D a*S1 + b*S2
O2 =3D c*S1 + d*S2

Inverting gives

S1 =3D (dO1 - bO2)/(ad - cb)
S2 =3D -(cO1 - aO2)/(ad - cb)

Now the problem is to determine the coefficients.

This is simple a problem of plugging in values:

Analogy: y =3D a + bx + cx^2

How do we find a b and c? Well if we know 3 pairs of points we just
plu=
g
in and solve the linear system of equations.

Unfortunately we do not know the Source vector. BUT if we "listen"
for common situations we can easily narrow down the search and
possibly arrive at correct coefficients.

Now since we know the outputs we know there ratio. Assume O1/O2 =3D x
a=
nd
S1/S2 =3D y then

y =3D -(dx - b)/(cx - a)

now this is like the polynomial sampling problem I gave above. We
just don't have our points (x,y) to determine the coefficients.

There are several ways to potentially get y such as negative
feedback, guestimating, kalman filters, monte carlo, etc...

e.g., if we guess at the coefficients we arrive at a certain
geometrica=
l
scenario that will either coincide with the real one or be
"off"(producing estimated outputs that do not jive with what we are
actually getting... we can perturb our coefficients until they
"wiggle" there way to the correct values using appropriate adaptive
methods).

If we know certain properties of the sources then we can include
those =
in
the algorithms to reduce the complexity/increase the effectiveness of
t=
he
algorithms.

For example, by knowing that S1 and S2 are uncorrelated we can
determin=
e
that when S2 =3D 0 then we end up with a 1 source system making it
much easier to solve. e.g., our equation reduces to

S1 =3D aO1 + bO2

and instead of 4 coefficients we only have to find 2. This results in
a triangular geometry instead of a quadrilateral. Moreover, by
symmetric =
we
can do the same for S2 and we end up with another easy system to
solve. This applies to any number of sources. =A0With uncorrelated
sources we should be able to treat the system as just one source a
time.

For such indepence the coefficients are directly related to the
distanc=
es
and can be easily calculated from the mic intensities. (a =3D 1/d1^2
an=
d b
=3D 1/d2^2 but a is the level from O1 and b is the level from O2)

In general though it is not the case that the sources are
uncorrelated =
or
time independent and more advanced tricks or general approximations
may have to be used.

The more I think about it the more it seems like it might be quite
accurate for most signals and setups. One problem though is frequency
response as we are assuming a lossless transmission which is usually
no=
t
the case and could cause problems with some algorithmic approaches.

Yes, but none of that has much to do with the how the Blind Source
Separation solutions actually work. =A0It's fare more interesting than
wh=
at
you are suggesting above. =A0It makes none of the assumptions you talk
ab=
out
above and instead directly calculates all 4 coefficients without
knowing anything about the sources.

I believe what it is doing, is using the fact that when you sum
separate non-correlated signals together, the sum tends to approach a
Gaussian pow=
er
distribution. =A0The more uncorrelated signals you sum together, the
clos=
er
it becomes to a pure Gaussian distribution.

As such, by measuring how far away from a Gaussian distribution a given
signal is, it can pick the four coefficients which maximize the
non-Gaussian characteristics of the two extracted sources signals.
=A0The=
two
sources signals will always be less Gaussian than the combined signal.
=
=A0The
only limitation to the approach is that the signals must be generally
non-correlated, and they must not be Gaussian to start with (which is
generally true for all signals a human would care about to start with).

Hardy points out that in real life acoustic situations the summation is
never just linear. =A0However, I don't know how relevant that actually
is=
.
If the source signals are arbitrary distorted BEFORE they are mixed,
this sort of approach can still separate the signals (back to their
pre-mixed distorted version). =A0And likewise, filtering =A0or
distorting =A0the su=
mmed
signal likewise, will not prevent their separation. =A0So the fact that
a microphone and the recording system is non-linear is not by itself, a
problem, as long as the same transform is effectively applied to each
source _interdependently_. =A0But, I might be stupid here, because by
definition, a transform which is not linear, might, by definition,
imply the transform is creating dependency. =A0I'm not sure about that.

If the recording system (and the type of transform Hardy is talking
about=
),
causes cross-modulation of the signals, then the resulting distribution
means the signals become at least partially correlated, which reduces
how well they can be separated.

But, as long as the linear combination dominates over any non-linear
effects, I would think this sort of a approach could still do a good
job =
of
separation, meaning that the separated signals would be far better than
t=
he
mixed signals you started with. =A0And I would think that any audio
recor=
ding
application that was wroth using, the linear effects would strongly
dominate over the non-linear effects.

--
Curt Welch =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0
=
=A0 =A0 =A0 =A0 =A0 =A0 =A0http://CurtWelch.Com/
c...@kcwc.com =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0
=A0=
=A0 =A0 =A0 =A0 =A0http://NewsReader.Com/

It makes none of the assumptions you talk about
above and instead directly calculates all 4 coefficients without
knowing
anything about the sources.

Doesn't actually, that is impossible. You see the amplitudes of what
you estimate are not known because the gains are as well.
This is more akin to decorrelation than estimating the unknown matrix.
Remember ICA doesn't give absolute amplitudes, these are scaled later.
Also the order of the estimates are unknown.

Hardy

HardySpicer <gyansor...@gmail.com> wrote:
On Jan 22, 12:39=A0pm, c...@kcwc.com (Curt Welch) wrote:
"Jeffery Tomas" <Jeffery_To...@Gmail.com> wrote:
The idea is very basic

The outputs(the mic's) are a linear combination of the input sources.
T> > he
coefficients give the relative magnitude of each source contributing
to the mic. If a source is further away it will contribute less to
the output. In fact, the coefficient should be 1/distance^2. If the
distanc> > e
is large 1/distance^2 is small and vice versa.

O_i =3D sum(1/a_k^2*S_k)

or

O =3D A*S

where O is the output's(mics), A is the "conversion matrix" and S is
th> > e
sources(which is what we want to recover). From basic linear algebra
we know S =3D A^-1*O.

There are statistical and deterministic methods to determine A since
it is essentially the "transfer function". The important thing is to
recognize is: If A is known and the sources and sinks(mics) are time
independent(or at least slowly changing) then we can recover the
source> > s.
The more accurate we know A the more accurate our decomposition will
be> > .

I'll demonstrate for two sources and sinks but I'll use inverted
linear distance wlog

O1 =3D a*S1 + b*S2
O2 =3D c*S1 + d*S2

Inverting gives

S1 =3D (dO1 - bO2)/(ad - cb)
S2 =3D -(cO1 - aO2)/(ad - cb)

Now the problem is to determine the coefficients.

This is simple a problem of plugging in values:

Analogy: y =3D a + bx + cx^2

How do we find a b and c? Well if we know 3 pairs of points we just
plu> > g
in and solve the linear system of equations.

Unfortunately we do not know the Source vector. BUT if we "listen"
for common situations we can easily narrow down the search and
possibly arrive at correct coefficients.

Now since we know the outputs we know there ratio. Assume O1/O2 =3D x
a> > nd
S1/S2 =3D y then

y =3D -(dx - b)/(cx - a)

now this is like the polynomial sampling problem I gave above. We
just don't have our points (x,y) to determine the coefficients.

There are several ways to potentially get y such as negative
feedback, guestimating, kalman filters, monte carlo, etc...

e.g., if we guess at the coefficients we arrive at a certain
geometrica> > l
scenario that will either coincide with the real one or be
"off"(producing estimated outputs that do not jive with what we are
actually getting... we can perturb our coefficients until they
"wiggle" there way to the correct values using appropriate adaptive
methods).

If we know certain properties of the sources then we can include
those > > in
the algorithms to reduce the complexity/increase the effectiveness of
t> > he
algorithms.

For example, by knowing that S1 and S2 are uncorrelated we can
determin> > e
that when S2 =3D 0 then we end up with a 1 source system making it
much easier to solve. e.g., our equation reduces to

S1 =3D aO1 + bO2

and instead of 4 coefficients we only have to find 2. This results in
a triangular geometry instead of a quadrilateral. Moreover, by
symmetric > > we
can do the same for S2 and we end up with another easy system to
solve. This applies to any number of sources. =A0With uncorrelated
sources we should be able to treat the system as just one source a
time.

For such indepence the coefficients are directly related to the
distanc> > es
and can be easily calculated from the mic intensities. (a =3D 1/d1^2
an> > d b
=3D 1/d2^2 but a is the level from O1 and b is the level from O2)

In general though it is not the case that the sources are
uncorrelated > > or
time independent and more advanced tricks or general approximations
may have to be used.

The more I think about it the more it seems like it might be quite
accurate for most signals and setups. One problem though is frequency
response as we are assuming a lossless transmission which is usually
no> > t
the case and could cause problems with some algorithmic approaches.

Yes, but none of that has much to do with the how the Blind Source
Separation solutions actually work. =A0It's fare more interesting than
wh> > at
you are suggesting above. =A0It makes none of the assumptions you talk
ab> > out
above and instead directly calculates all 4 coefficients without
knowing anything about the sources.

I believe what it is doing, is using the fact that when you sum
separate non-correlated signals together, the sum tends to approach a
Gaussian pow> > er
distribution. =A0The more uncorrelated signals you sum together, the
clos> > er
it becomes to a pure Gaussian distribution.

As such, by measuring how far away from a Gaussian distribution a given
signal is, it can pick the four coefficients which maximize the
non-Gaussian characteristics of the two extracted sources signals.
=A0The> >  two
sources signals will always be less Gaussian than the combined signal..
=A0The
only limitation to the approach is that the signals must be generally
non-correlated, and they must not be Gaussian to start with (which is
generally true for all signals a human would care about to start with).

Hardy points out that in real life acoustic situations the summation is
never just linear. =A0However, I don't know how relevant that actually
is> > .
If the source signals are arbitrary distorted BEFORE they are mixed,
this sort of approach can still separate the signals (back to their
pre-mixed distorted version). =A0And likewise, filtering =A0or
distorting =A0the su> > mmed
signal likewise, will not prevent their separation. =A0So the fact that
a microphone and the recording system is non-linear is not by itself, a
problem, as long as the same transform is effectively applied to each
source _interdependently_. =A0But, I might be stupid here, because by
definition, a transform which is not linear, might, by definition,
imply the transform is creating dependency. =A0I'm not sure about that.

If the recording system (and the type of transform Hardy is talking
about> > ),
causes cross-modulation of the signals, then the resulting distribution
means the signals become at least partially correlated, which reduces
how well they can be separated.

But, as long as the linear combination dominates over any non-linear
effects, I would think this sort of a approach could still do a good
job > > of
separation, meaning that the separated signals would be far better than
t> > he
mixed signals you started with. =A0And I would think that any audio
recor> > ding
application that was wroth using, the linear effects would strongly
dominate over the non-linear effects.

--
Curt Welch =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0
=A0 =A0 =A0 =A0 =A0 =A0 =A0http://CurtWelch.Com/
c...@kcwc.com =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0
=A0> >  =A0 =A0 =A0 =A0 =A0http://NewsReader.Com/

It makes none of the assumptions you talk about
above and instead directly calculates all 4 coefficients without
knowing
anything about the sources.

Doesn't actually, that is impossible. You see the amplitudes of what
you estimate are not known because the gains are as well.
This is more akin to decorrelation than estimating the unknown matrix.
Remember ICA doesn't give absolute amplitudes, these are scaled later.
Also the order of the estimates are unknown.

Ah, Ok, after thinking about it I see why that's true.

With two signals, you have a mixing matrix with 4 constants, which will
both re-scale the input signals, and mix them in different ratios across
the two output signals.  To extract the original signals, you only need to
know the ratio, and it's impossible to extract the starting amplitude,
because different amplitude source signals could produce the same mixed
output signals, just by adjusting the 4 constants.  There's just not enough
information there to know what starting amplitude was.

So ICA gives you in effect, 2 out of the 4 values - which is all you need
to separate the signals, but not enough to know what the original
amplitudes were.



Hardy

--
Curt Welch                                            http://CurtWelch.Com/
c...@kcwc.com                                        http://NewsReader.Com/

HardySpicer <gyansor...@gmail.com> wrote:
On Jan 22, 12:39=A0pm, c...@kcwc.com (Curt Welch) wrote:
"Jeffery Tomas" <Jeffery_To...@Gmail.com> wrote:
The idea is very basic

The outputs(the mic's) are a linear combination of the input sources.
T> > he
coefficients give the relative magnitude of each source contributing
to the mic. If a source is further away it will contribute less to
the output. In fact, the coefficient should be 1/distance^2. If the
distanc> > e
is large 1/distance^2 is small and vice versa.

O_i =3D sum(1/a_k^2*S_k)

or

O =3D A*S

where O is the output's(mics), A is the "conversion matrix" and S is
th> > e
sources(which is what we want to recover). From basic linear algebra
we know S =3D A^-1*O.

There are statistical and deterministic methods to determine A since
it is essentially the "transfer function". The important thing is to
recognize is: If A is known and the sources and sinks(mics) are time
independent(or at least slowly changing) then we can recover the
source> > s.
The more accurate we know A the more accurate our decomposition will
be> > .

I'll demonstrate for two sources and sinks but I'll use inverted
linear distance wlog

O1 =3D a*S1 + b*S2
O2 =3D c*S1 + d*S2

Inverting gives

S1 =3D (dO1 - bO2)/(ad - cb)
S2 =3D -(cO1 - aO2)/(ad - cb)

Now the problem is to determine the coefficients.

This is simple a problem of plugging in values:

Analogy: y =3D a + bx + cx^2

How do we find a b and c? Well if we know 3 pairs of points we just
plu> > g
in and solve the linear system of equations.

Unfortunately we do not know the Source vector. BUT if we "listen"
for common situations we can easily narrow down the search and
possibly arrive at correct coefficients.

Now since we know the outputs we know there ratio. Assume O1/O2 =3D x
a> > nd
S1/S2 =3D y then

y =3D -(dx - b)/(cx - a)

now this is like the polynomial sampling problem I gave above. We
just don't have our points (x,y) to determine the coefficients.

There are several ways to potentially get y such as negative
feedback, guestimating, kalman filters, monte carlo, etc...

e.g., if we guess at the coefficients we arrive at a certain
geometrica> > l
scenario that will either coincide with the real one or be
"off"(producing estimated outputs that do not jive with what we are
actually getting... we can perturb our coefficients until they
"wiggle" there way to the correct values using appropriate adaptive
methods).

If we know certain properties of the sources then we can include
those > > in
the algorithms to reduce the complexity/increase the effectiveness of
t> > he
algorithms.

For example, by knowing that S1 and S2 are uncorrelated we can
determin> > e
that when S2 =3D 0 then we end up with a 1 source system making it
much easier to solve. e.g., our equation reduces to

S1 =3D aO1 + bO2

and instead of 4 coefficients we only have to find 2. This results in
a triangular geometry instead of a quadrilateral. Moreover, by
symmetric > > we
can do the same for S2 and we end up with another easy system to
solve. This applies to any number of sources. =A0With uncorrelated
sources we should be able to treat the system as just one source a
time.

For such indepence the coefficients are directly related to the
distanc> > es
and can be easily calculated from the mic intensities. (a =3D 1/d1^2
an> > d b
=3D 1/d2^2 but a is the level from O1 and b is the level from O2)

In general though it is not the case that the sources are
uncorrelated > > or
time independent and more advanced tricks or general approximations
may have to be used.

The more I think about it the more it seems like it might be quite
accurate for most signals and setups. One problem though is frequency
response as we are assuming a lossless transmission which is usually
no> > t
the case and could cause problems with some algorithmic approaches.

Yes, but none of that has much to do with the how the Blind Source
Separation solutions actually work. =A0It's fare more interesting than
wh> > at
you are suggesting above. =A0It makes none of the assumptions you talk
ab> > out
above and instead directly calculates all 4 coefficients without
knowing anything about the sources.

I believe what it is doing, is using the fact that when you sum
separate non-correlated signals together, the sum tends to approach a
Gaussian pow> > er
distribution. =A0The more uncorrelated signals you sum together, the
clos> > er
it becomes to a pure Gaussian distribution.

As such, by measuring how far away from a Gaussian distribution a given
signal is, it can pick the four coefficients which maximize the
non-Gaussian characteristics of the two extracted sources signals.
=A0The> >  two
sources signals will always be less Gaussian than the combined signal..
=A0The
only limitation to the approach is that the signals must be generally
non-correlated, and they must not be Gaussian to start with (which is
generally true for all signals a human would care about to start with).

Hardy points out that in real life acoustic situations the summation is
never just linear. =A0However, I don't know how relevant that actually
is> > .
If the source signals are arbitrary distorted BEFORE they are mixed,
this sort of approach can still separate the signals (back to their
pre-mixed distorted version). =A0And likewise, filtering =A0or
distorting =A0the su> > mmed
signal likewise, will not prevent their separation. =A0So the fact that
a microphone and the recording system is non-linear is not by itself, a
problem, as long as the same transform is effectively applied to each
source _interdependently_. =A0But, I might be stupid here, because by
definition, a transform which is not linear, might, by definition,
imply the transform is creating dependency. =A0I'm not sure about that.

If the recording system (and the type of transform Hardy is talking
about> > ),
causes cross-modulation of the signals, then the resulting distribution
means the signals become at least partially correlated, which reduces
how well they can be separated.

But, as long as the linear combination dominates over any non-linear
effects, I would think this sort of a approach could still do a good
job > > of
separation, meaning that the separated signals would be far better than
t> > he
mixed signals you started with. =A0And I would think that any audio
recor> > ding
application that was wroth using, the linear effects would strongly
dominate over the non-linear effects.

--
Curt Welch =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0
=A0 =A0 =A0 =A0 =A0 =A0 =A0http://CurtWelch.Com/
c...@kcwc.com =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0
=A0> >  =A0 =A0 =A0 =A0 =A0http://NewsReader.Com/

It makes none of the assumptions you talk about
above and instead directly calculates all 4 coefficients without
knowing
anything about the sources.

Doesn't actually, that is impossible. You see the amplitudes of what
you estimate are not known because the gains are as well.
This is more akin to decorrelation than estimating the unknown matrix.
Remember ICA doesn't give absolute amplitudes, these are scaled later.
Also the order of the estimates are unknown.

Ah, Ok, after thinking about it I see why that's true.

With two signals, you have a mixing matrix with 4 constants, which will
both re-scale the input signals, and mix them in different ratios across
the two output signals.  To extract the original signals, you only need to
know the ratio, and it's impossible to extract the starting amplitude,
because different amplitude source signals could produce the same mixed
output signals, just by adjusting the 4 constants.  There's just not enough
information there to know what starting amplitude was.

So ICA gives you in effect, 2 out of the 4 values - which is all you need
to separate the signals, but not enough to know what the original
amplitudes were.
[...]

On Jan 21, 11:27 pm, c...@kcwc.com (Curt Welch) wrote:

HardySpicer <gyansor...@gmail.com> wrote:
On Jan 22, 12:39=A0pm, c...@kcwc.com (Curt Welch) wrote:
"Jeffery Tomas" <Jeffery_To...@Gmail.com> wrote:
The idea is very basic

The outputs(the mic's) are a linear combination of the input sources.
T> > > he
coefficients give the relative magnitude of each source contributing
to the mic. If a source is further away it will contribute less to
the output. In fact, the coefficient should be 1/distance^2. If the
distanc> > > e
is large 1/distance^2 is small and vice versa.

O_i =3D sum(1/a_k^2*S_k)

or

O =3D A*S

where O is the output's(mics), A is the "conversion matrix" and S is
th> > > e
sources(which is what we want to recover). From basic linear algebra
we know S =3D A^-1*O.

There are statistical and deterministic methods to determine A since
it is essentially the "transfer function". The important thing is to
recognize is: If A is known and the sources and sinks(mics) are time
independent(or at least slowly changing) then we can recover the
source> > > s.
The more accurate we know A the more accurate our decomposition will
be> > > .

I'll demonstrate for two sources and sinks but I'll use inverted
linear distance wlog

O1 =3D a*S1 + b*S2
O2 =3D c*S1 + d*S2

Inverting gives

S1 =3D (dO1 - bO2)/(ad - cb)
S2 =3D -(cO1 - aO2)/(ad - cb)

Now the problem is to determine the coefficients.

This is simple a problem of plugging in values:

Analogy: y =3D a + bx + cx^2

How do we find a b and c? Well if we know 3 pairs of points we just
plu> > > g
in and solve the linear system of equations.

Unfortunately we do not know the Source vector. BUT if we "listen"
for common situations we can easily narrow down the search and
possibly arrive at correct coefficients.

Now since we know the outputs we know there ratio. Assume O1/O2 =3D x
a> > > nd
S1/S2 =3D y then

y =3D -(dx - b)/(cx - a)

now this is like the polynomial sampling problem I gave above. We
just don't have our points (x,y) to determine the coefficients.

There are several ways to potentially get y such as negative
feedback, guestimating, kalman filters, monte carlo, etc...

e.g., if we guess at the coefficients we arrive at a certain
geometrica> > > l
scenario that will either coincide with the real one or be
"off"(producing estimated outputs that do not jive with what we are
actually getting... we can perturb our coefficients until they
"wiggle" there way to the correct values using appropriate adaptive
methods).

If we know certain properties of the sources then we can include
those > > > in
the algorithms to reduce the complexity/increase the effectiveness of
t> > > he
algorithms.

For example, by knowing that S1 and S2 are uncorrelated we can
determin> > > e
that when S2 =3D 0 then we end up with a 1 source system making it
much easier to solve. e.g., our equation reduces to

S1 =3D aO1 + bO2

and instead of 4 coefficients we only have to find 2. This results in
a triangular geometry instead of a quadrilateral. Moreover, by
symmetric > > > we
can do the same for S2 and we end up with another easy system to
solve. This applies to any number of sources. =A0With uncorrelated
sources we should be able to treat the system as just one source a
time.

For such indepence the coefficients are directly related to the
distanc> > > es
and can be easily calculated from the mic intensities. (a =3D 1/d1^2
an> > > d b
=3D 1/d2^2 but a is the level from O1 and b is the level from O2)

In general though it is not the case that the sources are
uncorrelated > > > or
time independent and more advanced tricks or general approximations
may have to be used.

The more I think about it the more it seems like it might be quite
accurate for most signals and setups. One problem though is frequency
response as we are assuming a lossless transmission which is usually
no> > > t
the case and could cause problems with some algorithmic approaches.

Yes, but none of that has much to do with the how the Blind Source
Separation solutions actually work. =A0It's fare more interesting than
wh> > > at
you are suggesting above. =A0It makes none of the assumptions you talk
ab> > > out
above and instead directly calculates all 4 coefficients without
knowing anything about the sources.

I believe what it is doing, is using the fact that when you sum
separate non-correlated signals together, the sum tends to approach a
Gaussian pow> > > er
distribution. =A0The more uncorrelated signals you sum together, the
clos> > > er
it becomes to a pure Gaussian distribution.

As such, by measuring how far away from a Gaussian distribution a given
signal is, it can pick the four coefficients which maximize the
non-Gaussian characteristics of the two extracted sources signals.
=A0The> > >  two
sources signals will always be less Gaussian than the combined signal.
=A0The
only limitation to the approach is that the signals must be generally
non-correlated, and they must not be Gaussian to start with (which is
generally true for all signals a human would care about to start with).

Hardy points out that in real life acoustic situations the summation is
never just linear. =A0However, I don't know how relevant that actually
is> > > .
If the source signals are arbitrary distorted BEFORE they are mixed,
this sort of approach can still separate the signals (back to their
pre-mixed distorted version). =A0And likewise, filtering =A0or
distorting =A0the su> > > mmed
signal likewise, will not prevent their separation. =A0So the fact that
a microphone and the recording system is non-linear is not by itself, a
problem, as long as the same transform is effectively applied to each
source _interdependently_. =A0But, I might be stupid here, because by
definition, a transform which is not linear, might, by definition,
imply the transform is creating dependency. =A0I'm not sure about that.

If the recording system (and the type of transform Hardy is talking
about> > > ),
causes cross-modulation of the signals, then the resulting distribution
means the signals become at least partially correlated, which reduces
how well they can be separated.

But, as long as the linear combination dominates over any non-linear
effects, I would think this sort of a approach could still do a good
job > > > of
separation, meaning that the separated signals would be far better than
t> > > he
mixed signals you started with. =A0And I would think that any audio
recor> > > ding
application that was wroth using, the linear effects would strongly
dominate over the non-linear effects.

--
Curt Welch =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0
=A0 =A0 =A0 =A0 =A0 =A0 =A0http://CurtWelch.Com/
c...@kcwc.com =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0
=A0> > >  =A0 =A0 =A0 =A0 =A0http://NewsReader.Com/

It makes none of the assumptions you talk about
above and instead directly calculates all 4 coefficients without
knowing
anything about the sources.

Doesn't actually, that is impossible. You see the amplitudes of what
you estimate are not known because the gains are as well.
This is more akin to decorrelation than estimating the unknown matrix..
Remember ICA doesn't give absolute amplitudes, these are scaled later..
Also the order of the estimates are unknown.

Ah, Ok, after thinking about it I see why that's true.

With two signals, you have a mixing matrix with 4 constants, which will
both re-scale the input signals, and mix them in different ratios across
the two output signals.  To extract the original signals, you only need to
know the ratio, and it's impossible to extract the starting amplitude,
because different amplitude source signals could produce the same mixed
output signals, just by adjusting the 4 constants.  There's just not enough
information there to know what starting amplitude was.

So ICA gives you in effect, 2 out of the 4 values - which is all you need
to separate the signals, but not enough to know what the original
amplitudes were.

[...]

In multi-antenna radio-communications ,
such as the new 802.11n IEEE Wifi standard,
there's an important concept known as:
"quasi-static fading channel".

All it means, essentially, is that the
"mixing maxtrix" say A, e.g. 2x2 complex-valued
matrix if 2 receive, 2 transmit anttennas,
changes slowly over time.

With sound waves, if the mixing matrix or
mixing coefficients change by little
in 100 milliseconds or 0.1 second, one
might make an analogy to
a quasi-static fading channel, as understood
in radio wave propagation.  Being
quasistatic is better than being a fast-fading
channel for radio waves when the entropy
of the two source ...

read more »

On Jan 23, 5:41 pm, David Bernier <david53...@gmail.com> wrote:

On Jan 21, 11:27 pm, c...@kcwc.com (Curt Welch) wrote:

HardySpicer <gyansor...@gmail.com> wrote:
On Jan 22, 12:39=A0pm, c...@kcwc.com (Curt Welch) wrote:
"Jeffery Tomas" <Jeffery_To...@Gmail.com> wrote:
The idea is very basic

The outputs(the mic's) are a linear combination of the input sources.
T> > > > he
coefficients give the relative magnitude of each source contributing
to the mic. If a source is further away it will contribute less to
the output. In fact, the coefficient should be 1/distance^2. If the
distanc> > > > e
is large 1/distance^2 is small and vice versa.

O_i =3D sum(1/a_k^2*S_k)

or

O =3D A*S

where O is the output's(mics), A is the "conversion matrix" and S is
th> > > > e
sources(which is what we want to recover). From basic linear algebra
we know S =3D A^-1*O.

There are statistical and deterministic methods to determine A since
it is essentially the "transfer function". The important thing is to
recognize is: If A is known and the sources and sinks(mics) are time
independent(or at least slowly changing) then we can recover the
source> > > > s.
The more accurate we know A the more accurate our decomposition will
be> > > > .

I'll demonstrate for two sources and sinks but I'll use inverted
linear distance wlog

O1 =3D a*S1 + b*S2
O2 =3D c*S1 + d*S2

Inverting gives

S1 =3D (dO1 - bO2)/(ad - cb)
S2 =3D -(cO1 - aO2)/(ad - cb)

Now the problem is to determine the coefficients.

This is simple a problem of plugging in values:

Analogy: y =3D a + bx + cx^2

How do we find a b and c? Well if we know 3 pairs of points we just
plu> > > > g
in and solve the linear system of equations.

Unfortunately we do not know the Source vector. BUT if we "listen"
for common situations we can easily narrow down the search and
possibly arrive at correct coefficients.

Now since we know the outputs we know there ratio. Assume O1/O2 =3D x
a> > > > nd
S1/S2 =3D y then

y =3D -(dx - b)/(cx - a)

now this is like the polynomial sampling problem I gave above. We
just don't have our points (x,y) to determine the coefficients.

There are several ways to potentially get y such as negative
feedback, guestimating, kalman filters, monte carlo, etc...

e.g., if we guess at the coefficients we arrive at a certain
geometrica> > > > l
scenario that will either coincide with the real one or be
"off"(producing estimated outputs that do not jive with what we are
actually getting... we can perturb our coefficients until they
"wiggle" there way to the correct values using appropriate adaptive
methods).

If we know certain properties of the sources then we can include
those > > > > in
the algorithms to reduce the complexity/increase the effectiveness of
t> > > > he
algorithms.

For example, by knowing that S1 and S2 are uncorrelated we can
determin> > > > e
that when S2 =3D 0 then we end up with a 1 source system making it
much easier to solve. e.g., our equation reduces to

S1 =3D aO1 + bO2

and instead of 4 coefficients we only have to find 2. This results in
a triangular geometry instead of a quadrilateral. Moreover, by
symmetric > > > > we
can do the same for S2 and we end up with another easy system to
solve. This applies to any number of sources. =A0With uncorrelated
sources we should be able to treat the system as just one source a
time.

For such indepence the coefficients are directly related to the
distanc> > > > es
and can be easily calculated from the mic intensities. (a =3D 1/d1^2
an> > > > d b
=3D 1/d2^2 but a is the level from O1 and b is the level from O2)

In general though it is not the case that the sources are
uncorrelated > > > > or
time independent and more advanced tricks or general approximations
may have to be used.

The more I think about it the more it seems like it might be quite
accurate for most signals and setups. One problem though is frequency
response as we are assuming a lossless transmission which is usually
no> > > > t
the case and could cause problems with some algorithmic approaches.

Yes, but none of that has much to do with the how the Blind Source
Separation solutions actually work. =A0It's fare more interesting than
wh> > > > at
you are suggesting above. =A0It makes none of the assumptions you talk
ab> > > > out
above and instead directly calculates all 4 coefficients without
knowing anything about the sources.

I believe what it is doing, is using the fact that when you sum
separate non-correlated signals together, the sum tends to approach a
Gaussian pow> > > > er
distribution. =A0The more uncorrelated signals you sum together, the
clos> > > > er
it becomes to a pure Gaussian distribution.

As such, by measuring how far away from a Gaussian distribution a given
signal is, it can pick the four coefficients which maximize the
non-Gaussian characteristics of the two extracted sources signals..
=A0The> > > >  two
sources signals will always be less Gaussian than the combined signal.
=A0The
only limitation to the approach is that the signals must be generally
non-correlated, and they must not be Gaussian to start with (which is
generally true for all signals a human would care about to start with).

Hardy points out that in real life acoustic situations the summation is
never just linear. =A0However, I don't know how relevant that actually
is> > > > .
If the source signals are arbitrary distorted BEFORE they are mixed,
this sort of approach can still separate the signals (back to their
pre-mixed distorted version). =A0And likewise, filtering =A0or
distorting =A0the su> > > > mmed
signal likewise, will not prevent their separation. =A0So the fact that
a microphone and the recording system is non-linear is not by itself, a
problem, as long as the same transform is effectively applied to each
source _interdependently_. =A0But, I might be stupid here, because by
definition, a transform which is not linear, might, by definition,
imply the transform is creating dependency. =A0I'm not sure about that.

If the recording system (and the type of transform Hardy is talking
about> > > > ),
causes cross-modulation of the signals, then the resulting distribution
means the signals become at least partially correlated, which reduces
how well they can be separated.

But, as long as the linear combination dominates over any non-linear
effects, I would think this sort of a approach could still do a good
job > > > > of
separation, meaning that the separated signals would be far better than
t> > > > he
mixed signals you started with. =A0And I would think that any audio
recor> > > > ding
application that was wroth using, the linear effects would strongly
dominate over the non-linear effects.

--
Curt Welch =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0
=A0 =A0 =A0 =A0 =A0 =A0 =A0http://CurtWelch.Com/
c...@kcwc.com =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0
=A0> > > >  =A0 =A0 =A0 =A0 =A0http://NewsReader.Com/

It makes none of the assumptions you talk about
above and instead directly calculates all 4 coefficients without
knowing
anything about the sources.

Doesn't actually, that is impossible. You see the amplitudes of what
you estimate are not known because the gains are as well.
This is more akin to decorrelation than estimating the unknown matrix.
Remember ICA doesn't give absolute amplitudes, these are scaled later.
Also the order of the estimates are unknown.

Ah, Ok, after thinking about it I see why that's true.

With two signals, you have a mixing matrix with 4 constants, which will
both re-scale the input signals, and mix them in different ratios across
the two output signals.  To extract the original signals, you only need to
know the ratio, and it's impossible to extract the starting amplitude,
because different amplitude source signals could produce the same mixed
output signals, just by adjusting the 4 constants.  There's just not enough
information there to know what starting amplitude was.

So ICA gives you in effect, 2 out of the 4 values - which is all you need
to separate the signals, but not enough to know what the original
amplitudes were.

[...]

In multi-antenna radio-communications ,
such as the new 802.11n IEEE Wifi standard,
there's an important concept known as:
"quasi-static fading channel".

All it means, essentially, is that the
"mixing maxtrix" say A, e.g. 2x2 complex-valued
matrix if 2 receive, 2 transmit anttennas,
changes slowly over time.

With sound waves, if the mixing matrix or
mixing coefficients change by little
in 100 milliseconds or 0.1 second, one
might make an analogy to
a quasi-static fading channel, as understood
in radio wave propagation.  Being
quasistatic is better than being a fast-fading
channel for radio waves when the entropy
of the two source ...

read more »

But they don't! Of course it depends on the environment but you can
never have just a matrix. It has to be a transfer function matrix
which is time-varying and more than often non-minimum phase. This is
why it isn't easy.
[...]