I was at a friend's house yesterday, and he showed me "Karaoke
Revolution" on the Sony Playstation 2. It is a Karaoke singing game
where you pick a character and a song (mostly popular music like
Britney Spears, Michael Jackson, R.E.M., Phil Collins, etc.). Then the
game goes into "American Idol" mode, you go on a stage with an audience
and, with a USB enabled microphone, you sing along to the tune with the
lyrics scrolling on the bottom of the screen. So far nothing special.

Now here is the catch: The microphone picks up the pitch of your voice,
and you are awarded how closely you match the pitch of the singer who
recorded the song (sound-alike bands are used, presumably due to
licensing costs). The closer you are in timing and pitch to the
original, the higher you score. Pretty cool gimmick, it might help
some people learn how to sing, the feedback on how well you are doing
is very simple and easy to understand ...

It would seem like it would be pretty easy to write a bit of
open-source software that would be capable of extending this to any MP3
with vocals. It would take a bit of work to setup, but I imagine it
would be possible to:

1) Take an MP3 with a vocal track.
2) Strip out the vocal portion (is this possible? I remember seeing ads
years ago for a machine that would remove vocals from recordings, it is
possible to also isolate a vocal track in software?)
3) Store the timing and pitch info for the track in some format,
perhaps encoded in XML.
4) Manually add in the lyrics for the vocals at the appropriate time in
an application. You could click on each vocal segment and hear the
word(s), and then type in the lyric for the meta-MP3 file.
5) Now with the MP3 file and meta-MP3 file, play the above in a karaoke
enabled MP3 player, possibly the same application.

If it was easy enough to do for the average user, I imagine a large
library of meta-MP3 files would quickly appear on the internet ...
Perhaps there could even be a business in selling MP-meta songs ...
Although there would be copyright issues with the lyrics ... The main
limitation of the Sony game is that every time you want to get more
songs, you have to buy the next version (3 exist right now), and it
only might contain one or two songs you really like, since it's mostly
mainstream. What about singing along to David Byrne, Frank Black, or
even Frank Sinatra?

I imagine the format would look something like (just random
brainstorming):

segment
timemark8000ms/timemark
lyric"Ooooh"/lyric
pitch9000hz,60ms/pitch
/segment

-Eric

This has nothing to do with what you wrote...but did anyone see the singer
from American Idol perform the start spangled banner at the
Patriots-Pittsburgh game today?

Man, she was great!

I usually poo-poo American Idol -- but I have to give them credit for one
thing:

a. No one lip syncs, so they actually have to have a voice
b. They advance based on talent, not on looks.

Conceivably, most of the winners of American Idol would be more talented
than a typical Spears, or Timberlake, who are stars not by their singing
talent, but by virtue of looking good on TV.


Clouseau2 wrote:


[...]
5) Now with the MP3 file and meta-MP3 file, play the above in a karaoke
enabled MP3 player, possibly the same application.

Why wouldn't you use some kind of Baysian pattern matching software to
compare one versus the other and get a score ( similar to the way they
evaluate spam.



--
http://texeme.com
Textcasting Technology
Incognito Blog
http://incognito.texeme.com

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Oy! Listen to Clouseau2's latest shpiel!

1) Take an MP3 with a vocal track.
2) Strip out the vocal portion (is this possible? I remember seeing ads
years ago for a machine that would remove vocals from recordings, it is
possible to also isolate a vocal track in software?)

Usually what they do (what plugins for XMMS, WinAMP, etc. exist to do) is
just filter the sound so that the vocal portion is significantly reduced.
Still there, but depending on the track, just less loud than the music.

When I was in college, the Student Activities Board used to occasionally
hire DJs for karaoke nights. When they shelled out the extra bucks for a
good DJ, they got professional equipment with proper music. But eventually
they decided to budget their money and started hiring a guy who just
brought in a laptop with WMP and one of those plugins, plus lyrics encoded
into Powerpoint. It sucked big time. I can't imagine an open-source
version--even in 0.0.1 beta--being any worse.

- --
mark allen adams, jr.
artoodeetoo (at) gmail (dot) com
Internet Explorer: the world's most standards-complaint web browser

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFB9H0qnkcACrrIBtgRAg6dAJ9Gog2LNWzC4V8RaFAY3B ny6csDSgCgxpNf
FRApwwmtweXHPbfwQ21Bhsw=
=qxZ6
-----END PGP SIGNATURE-----


----== Posted via Newsfeeds.Com - Unlimited-Uncensored-Secure Usenet News==----
http://www.newsfeeds.com The #1 Newsgroup Service in the World! 100,000 Newsgroups
---= East/West-Coast Server Farms - Total Privacy via Encryption =---

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Oy! Listen to C.E.O. Gargantua's latest shpiel!
This has nothing to do with what you wrote...but did anyone see the singer
from American Idol perform the start spangled banner at the
Patriots-Pittsburgh game today?

Man, she was great!

Didn't really pay attention. Last time they had the AFCC in the 'Burgh,
they got Donnie Iris for the anthem. I'll take Donnie over *anyone* from
AI any day.

- --
mark allen adams, jr.
artoodeetoo (at) gmail (dot) com
Internet Explorer: the world's most standards-complaint web browser

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFB9H3pnkcACrrIBtgRAiFWAJ9EIDA98hxVXl72EDGDqV j8S6PI4QCfQXJQ
lA8GGdxmEn0Qm5Ni+Aj1wqg=
=GsOU
-----END PGP SIGNATURE-----


----== Posted via Newsfeeds.Com - Unlimited-Uncensored-Secure Usenet News==----
http://www.newsfeeds.com The #1 Newsgroup Service in the World! 100,000 Newsgroups
---= East/West-Coast Server Farms - Total Privacy via Encryption =---

Clouseau2 wrote:
) 2) Strip out the vocal portion (is this possible? I remember seeing ads
) years ago for a machine that would remove vocals from recordings, it is
) possible to also isolate a vocal track in software?)

Usually, vocal tracks are simple to remove, because the vocals tend to be
'center' in a stereo recording, while other instruments are placed in other
'spatial positions'. This removes the vocals quite nicely, but if you
*get* the vocals this way, they'll probably be quite noisy.


SaSW, Willem
--
Disclaimer: I am in no way responsible for any of the statements
made in the above text. For all I know I might be
drugged or something..
No I'm not paranoid. You all think I'm paranoid, don't you !
#EOT

"Clouseau2" wrote in message oups.com...

It would seem like it would be pretty easy to write a bit of
open-source software that would be capable of extending this to any MP3
with vocals.

Yea right. That would not be so easy. Comparing two waves at the same time and
finding pitch with certain approximation matching error. That would take SO much
memory and CPU power. I guess you know how big is an average uncompressed
wave. Depending on scanning frequency and requested accuracy program could
work quite fast on mp3s. But it's very complex code. My friend is Karaoke DJ
and I know he would like this idea instead of those special karaoke CDs.
U)

Clouseau2 wrote:

I was at a friend's house yesterday, and he showed me "Karaoke
Revolution" on the Sony Playstation 2. It is a Karaoke singing game
where you pick a character and a song (mostly popular music like
Britney Spears, Michael Jackson, R.E.M., Phil Collins, etc.). Then the
game goes into "American Idol" mode, you go on a stage with an audience
and, with a USB enabled microphone, you sing along to the tune with the
lyrics scrolling on the bottom of the screen. So far nothing special.

Now here is the catch: The microphone picks up the pitch of your voice,
and you are awarded how closely you match the pitch of the singer who
recorded the song (sound-alike bands are used, presumably due to
licensing costs). The closer you are in timing and pitch to the
original, the higher you score. Pretty cool gimmick, it might help
some people learn how to sing, the feedback on how well you are doing
is very simple and easy to understand ...

It would seem like it would be pretty easy to write a bit of
open-source software that would be capable of extending this to any MP3
with vocals. It would take a bit of work to setup, but I imagine it
would be possible to:

1) Take an MP3 with a vocal track.
2) Strip out the vocal portion (is this possible? I remember seeing ads
years ago for a machine that would remove vocals from recordings, it is
possible to also isolate a vocal track in software?)
3) Store the timing and pitch info for the track in some format,
perhaps encoded in XML.
4) Manually add in the lyrics for the vocals at the appropriate time in
an application. You could click on each vocal segment and hear the
word(s), and then type in the lyric for the meta-MP3 file.
5) Now with the MP3 file and meta-MP3 file, play the above in a karaoke
enabled MP3 player, possibly the same application.

If it was easy enough to do for the average user, I imagine a large
library of meta-MP3 files would quickly appear on the internet ...
Perhaps there could even be a business in selling MP-meta songs ...
Although there would be copyright issues with the lyrics ... The main
limitation of the Sony game is that every time you want to get more
songs, you have to buy the next version (3 exist right now), and it
only might contain one or two songs you really like, since it's mostly
mainstream. What about singing along to David Byrne, Frank Black, or
even Frank Sinatra?

I imagine the format would look something like (just random
brainstorming):

segment
timemark8000ms/timemark
lyric"Ooooh"/lyric
pitch9000hz,60ms/pitch
/segment

-Eric

It would be easier if the music file came in three channels
instead of two.
A pair for stereo audio and one for voice. These are mixed
on playback. (Usually karaoke recordings are not originals
and can easily be manufactured this way with a no brainer 3 channel
MP3 custom encoder / decoder)
Then its no brainer to take lets say a 20 point FFT
on the voice channel with each say 10 levels, and then
matching that in real time with 20 point/10 Level FFT on
the microphone input to get best match estimates. As doing FFT
takes time, some kind of phase delay has to be adjusted
depending on speed of CPU on exactly where in time the match
is made.
With all the crappy singing out there you probably won't need 20/10 FFT,
4/4 may be enough.

C.E.O. Gargantua wrote:
This has nothing to do with what you wrote...but did anyone see the singer
from American Idol perform the start spangled banner at the
Patriots-Pittsburgh game today?

Man, she was great!

She sings pretty good, but, why can't she just sing the song?

It really bugs me whenever they start doing that "diva" thing, and start
dragging out and multi pitching every syllable (think Mariah Cary).
Seems like she is trying to make the song about her, which takes away
from the meaning of the song in my opinion.

Dave Kowalski wrote:

melisma:
a passage of several notes sung to one syllable of text, as in Gregorian
chant
You can hear melisma in just about any Mariah Carey song, or Whitney
Houston, or Backstreet Boys, or Celine Dion.
It drives me crazy-


Yeah, that's it. Drives me crazy too.

I do have to say, though, when Whitney Houston sang the anthem at the
Superbowl several years ago, she did a fabulous job, one of the better
renditions I've heard.

Ufit wrote:
"Clouseau2" wrote in message
oups.com...

It would seem like it would be pretty easy to write a bit of
open-source software that would be capable of extending this to any
MP3
with vocals.

Yea right. That would not be so easy. Comparing two waves at the same
time and
finding pitch with certain approximation matching error. That would
take SO much
memory and CPU power. I guess you know how big is an average
uncompressed
wave. Depending on scanning frequency and requested accuracy program
could
work quite fast on mp3s. But it's very complex code. My friend is
Karaoke DJ
and I know he would like this idea instead of those special karaoke
CDs.
U)

I meant, "this would be pretty easy provided there was a way of
extracting the vocal track".

Dave Kowalski wrote:

melisma:

melisma, malaraia, it's all bad

--
ha

Ufit wrote:

Yea right. That would not be so easy. Comparing two waves at the same time and
finding pitch with certain approximation matching error. That would take SO much
memory and CPU power. I guess you know how big is an average uncompressed
wave. Depending on scanning frequency and requested accuracy program could
work quite fast on mp3s. But it's very complex code. My friend is Karaoke DJ

A diffusion based matching algorithm might work well for spectrogram
matching. See the following paper for more info. If you try to apply
this algorithm for spectrograms, you should, in addition to using
small time-patches incrementally, and multi-resolution in both time
and frequency, first try to match the spectral envelopes, and try
to match the pitch in a later iteration. If you do it holistically,
all-at-once, false pitches will make it rather difficult for the
matching to find its way out of the local minimas, and it is likely
that the result will not be as good.

See:
Hannu Helminen, Jyrki Alakuijala, Katja Pesola, Joakim Laitinen:
Comparison of Local External Force Functions for Non-rigid Registration
of 3D Medical Images. MICCAI (2) 2003: 821-828

This paper is about matching medical images, but it may be tuned to work
with spectrograms. The algorithm is rather simple and robust, and
much faster than a simple local convolution based flexible matching
algorithm.