Breaking Eggs And Making Omelettes

Topics On Multimedia Technology and Reverse Engineering


Archives:

Adding C64 SID Music

October 31st, 2012 by Multimedia Mike

I have been working on adding support for SID files — the music format for the Commodore 64 — to the game music website for awhile. I feel a bit out of my element since I’m not that familiar with the C64. But why should I let that slow me down? Allow me to go through the steps I have previously outlined in order to make this happen.



I need to know what picture should represent the system in the search results page. The foregoing picture should be fine, but I’m getting way ahead of myself.

Phase 1 is finding adequate player software. The most venerable contender in this arena is libsidplay, or so I first thought. It turns out that there’s libsidplay (originally hosted at Geocities, apparently, and no longer on the net) and also libsidplay2. Both are kind of old (libsidplay2 was last updated in 2004). I tried to compile libsidplay2 and the C++ didn’t agree with current version of g++.

However, a recent effort named libsidplayfp is carrying on the SID emulation tradition. It works rather well, notwithstanding the fact that compiling the entire library has a habit of apparently hanging the Linux VM where I develop this stuff.

Phase 2 is to develop a testbench app around the playback library. With the help of the libsidplayfp library maintainers, I accomplished this. The testbench app consistently requires about 15% of a single core of a fairly powerful Core i7. So I look forward to recommendations that I port that playback library to pure JavaScript.

Phase 3 is plug into the web player. I haven’t worked on this yet. I’m confident that this will work since phase 2 worked (plus, I have a plan to combine phases 2 and 3).

One interesting issue that has arisen is that proper operation of libsidplayfp requires that 3 C64 ROM files be present (the, ahem, KERNAL, BASIC interpreter, and character generator). While these are copyrighted ROMs, they are easily obtainable on the internet. The goal of my project is to eliminate as much friction as possible for enjoying these old tunes. To that end, I will just bake the ROM files directly into the player.

Phase 4 is collecting a SID song corpus. This is the simplest part of the whole process thanks to the remarkable curation efforts of the High Voltage SID Collection (HVSC). Anyone can download a giant archive of every known SID file. So that’s a done deal.

Or is it? One small issue is that I was hoping that the first iteration of my game music website would focus on, well, game music. There is a lot of music in the HVSC that are original compositions or come from demos. The way that the archive is organized makes it difficult to automatically discern whether a particular SID file comes from a game or not.

Phase 5 is munging the metadata. The good news here is that the files have the metadata built in. The not-so-great news is that there isn’t quite as much as I might like. Each file is tagged with title, author, and publisher/copyright. If there is more than one song in a file, they all have the same metadata. Fortunately, if I can import them all into my game music database, there is an opportunity to add a lot more metadata.

Further, there is no play length metadata for these files. This means I will need to set each to a default length like 2 minutes and do something like I did before in order to automatically determine if any songs terminate sooner.

Oddly, the issue I’m most concerned about is character encoding. This is the first project for which I’m making certain that I understand character encoding since I can’t reasonably get away with assuming that everything is ASCII. So far, based on the random sampling of SID files I have checked, there is a good chance of encountering metadata strings with characters that are not in the lower ASCII set. From what I have observed, these characters map to Unicode code points. So I finally get to learn about manipulating strings in such a way that it preserves the character encoding. At the very least, I need Python to rip the strings out of the binary SID files and make sure the Unicode remains intact while being inserted into an SQLite3 database.

Posted in General | 4 Comments »

Trouble with CoCCA Registry

October 6th, 2012 by Multimedia Mike

I’ve been rather despondent all week. People who see me daily could readily identify this fact. Unfortunately, the exact reason was difficult to adequately explain. The problems that nerds deal with…

When A Domain Expires
As a few people noticed, the multimedia.cx domain and all of it’s subdomains didn’t work this last week. The problem started on Monday, October 1. Whose fault? Well, fundamentally, I neglected to renew the domain name in time. However, I prefer to place the blame on the .cx domain registrar, CoCCA Registry. You see, they have never developed the technology to email a domain holder with a notice that their domain is about to expire or has already expired.

This domain is the only one I have ever held so I don’t have a lot of experience in this matter. I wondered if I was crazy for thinking it would be normal for a registrar to send an email or 2 with status updates about your domain. I get the impression from speaking with others that this is indeed normal. I have 3 different email addresses listed under my account at the registrar– 2 at multimedia.cx and a backup gmail account. I checked spam folders after this incident. Then I remembered that I have never received any email notifications from them (although password reset emails show up, so that part thankfully works). Also, their support emails are black holes.

So, I guess the moral is: be wary of dealing with CoCCA Registry. However, they seem to be the only way to register domains under a wide variety of uncommon country codes.

By Friday, the domain appeared to have been reinstated, even through the status was officially listed as “renewal-pending” according to the web-based management console. Eventually, as cached DNS results started to time out throughout the day, I started seeing subdomains come back. I excitedly used the ‘dig’ command to count down the seconds until gamemusic.multimedia.cx was accessible on the network I was on (the number after the domain name is the time-to-live or ‘TTL’ value):

$ dig +nocmd gamemusic.multimedia.cx +noall +answer
gamemusic.multimedia.cx. 3      IN      A       174.143.152.251
$ dig +nocmd gamemusic.multimedia.cx +noall +answer
gamemusic.multimedia.cx. 2      IN      A       174.143.152.251
$ dig +nocmd gamemusic.multimedia.cx +noall +answer
gamemusic.multimedia.cx. 1      IN      A       174.143.152.251
$ dig +nocmd gamemusic.multimedia.cx +noall +answer
gamemusic.multimedia.cx. 12962  IN      A       207.45.186.114

Finally, today (Saturday), I received a receipt confirming that the domain has been renewed.

8 Years Old
Incidentally, happy eighth birthday to multimedia.cx. It was September, 2004 when I decided to branch out from a simple ISP-based web presence.

People often ask why I went with the .cx TLD. When I decided I wanted a proper domain name 8 years ago, I found that multimedia.X was already taken for just about every TLD value of X. .cx was a notable exception and was distinctive enough (speaking of .X, though, I see that multimedia.xxx is still up for grabs as of this writing; I imagine that would come with a whole other set of problems).

It’s funny that tech nerds often rail against outsourcing too much — email, storage, computing power, web hosting — all to some type of cloud provider under the premise that it could easily be taken away. But this episode teaches me that even having your own domain name is no guarantee of a solid online presence.

Meanwhile, I have taken proactive steps to avert this same situation from arising again:



Barring a lack of automated emails from the registrar, I hope a Google Calendar reminder set up a month ahead of expiration will do the trick.

Posted in General | 7 Comments »