I’m noting that my posts’ word count is getting lower and lower lately; if this continues I’ll be writing yet another blog mentioning what TV shows I watched and what the cat ate for dinner last night. Except of course that I’d have to get a cat in that case. Or a flying squirrel. Click that link – seriously, click it. I love flying squirrels. The ones I’ve seen for sale here are grey and black, though.

Either way, I hope that my lack of verbosity lately is mostly due to the fact that

1. I’m busy and
2. For some inexplicable reason I find lots and lots of little tidbits to comment on, that don’t seem to combine easily into a single post.

My rant today is on custom songs in Singstar. No, I’m afraid that I haven’t uncovered a neat trick to do the aforementioned; I’m just going to rant about the possibility. First of all: in order to have a Singstar-like program that allows you to load up your favourite song and then automatically determines the singing line from that, you need to exctract the vocals from the song.

Everyone and their sister knows that extracting vocals from a song is no easy feat, and many claim that it’s an unsolvable problem. Once the sounds have been mixed there’s nothing intricate about the sounds that separates vocals in a specific frequency from an instrument playing in the same frequency. Or is there? Humans are very capable of determining what is vocals and what is an instrument, so maybe the answer doesn’t lie in spectrum analysis, but in speech recognition instead. If the speech can be recognized, it might be possible to identify qualities specific to that voice – enough to separate the sounds from the rest of the mix.

Another, simpler, way is to assume that vocals are centered in the mix. Then it’s (basically) a matter of examining the left and right channel to see what parts are the same. Et voi­la – vocal extraction in a much easier fashion. There’s even a DSP plugin for Winamp to do this. The downside is that the results are pretty much…crap. A minor separation can be heard, yes, but we’re definitely not talking about vocal extraction.

But but but, and this is a big butt, this method doesn’t work very well on MP3s due to the encoding, so a better result might be accomplished on better-quality recordings. And hey – what about CDs! Wouldn’t that be an excellent thing for Singstar? To load up your CD in-game and have it process the song you wish to destroy with your screechings? Yes, it would. But I really don’t know how good the vocal extraction would be even on CD sound; and besides that, it wouldn’t work at all for mixes that use stereo panning on the vocals. Not to mention what Sony might have to say about unprotected CDs.

Okay, so now I’ve mentioned all the problems. Time to be a little bit positive. If a combination of speech recognition and stereo vocal extraction could be used to make the vocals more prominent – note that I don’t talk about extracting the vocals totally - then a spectrum analysis could determine the prominent notes. And if the vocals are made more prominent, wouldn’t they be what the analysis uses for determining the notes? And if the song can be divided into suitably discrete parts, couldn’t this technique be used to determine the most probable notes that the singing line ought to consist of? There are methods for determining the BPM, so dividing the song into suitable discrete parts oughtn’t be a major problem.

To quote Young Frankenstein: It… Could… Work!

But of course I don’t know if it’s possible or not, or if the notes would just become muddled averages. It’s an interesting idea nonetheless. If it’s possible, the Singstar research team really should look into stuff like this. It would make things infinitely easier than UltraStar’s non-user-friendly system. (UltraStar: open source PC software that imitates Singstar but allows you to define your own songs. If you have a MIDI file and/or a lot of time and patience.)

Oh, and while I’m on the topic of complaining about the lack of Singstar features I have two other things to mention:

1. There really really really ought to be a way to allow variations. If a person sings something that fits rhytmically and melodically, it ought to give extra credit. Likewise, if the person decides to spice things up by singing a line that isn’t exactly what is defined but works in the current key, it should be fine.
2. I can’t remember the second thing, but I’m sure it was important.

Current best score: Always on My Mind ~9200 on Easy; ~6200 on Hard. I need to practice more.

This entry was posted on Monday, December 18th, 2006 at 12:50 pm and is filed under Games, Software Development. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.