Breaking Eggs And Making Omelettes

Topics On Multimedia Technology and Reverse Engineering


Archives:

Adding AY Files To The Game Music Website

November 30th, 2013 by Multimedia Mike

For the first time since I launched the site in the summer of last year, I finally added support for new systems for my Game Music Appreciation site: A set of chiptune music files which bear the file extension AY. These files come from games that were on the ZX Spectrum and Amstrad CPC computer systems.


ZX Spectrum   Amstrad CPC

Right now, there are over 650 ZX Spectrum games in the site while there are all of 20 Amstrad CPC games. The latter system seems a bit short-changed, but I read that a lot of Amstrad games were straight ports from the Spectrum anyway since the systems possessed assorted similarities. This might help explain the discrepancy.

Technically
The AY corpus has always been low hanging fruit due to the fact that the site already supports the format courtesy of the game-music-emu backend. The thing that blocked me was that I didn’t know much about these systems. I knew that there were 2 systems (and possibly more) that shared the same chiptune format. Apparently, these machines were big in Europe (I was only vaguely aware of them before I started this project).

Both the Spectrum and the Amstrad used Zilog Z-80 CPUs for computing and created music using a General Instruments synthesizer chip designated AY-3-8912, hence the chiptune file extension AY. This has 3 channels similar to the C64 SID chip. Additionally, there’s a fourth channel that game music emu calls “beeper” (and which Wikipedia describes as “one channel with 10 octaves”). Per my listening, it seems similar to the old PC speaker/honker. The metadata for a lot of the songs will specify either (AY) or (Beeper).

Wrangling Metadata
Large collections of AY files are easy to find; as is typical for pure chiptunes, the files are incredibly small.

As usual, the hardest part of the whole process was munging metadata. There seems to be 2 slightly different conventions for AY metadata, likely from 2 different people doing the bulk of the work and releasing the fruits of their labor into the wild. After I recognized the subtle differences between the 2 formats, it was straightforward to craft a tool to perform most of the work, leaving only a minimum of cleanup effort required afterwards.

(As an aside, I think this process is called extract – transform – load, or ETL. Sounds fancy and complicated, yet it’s technically one of the first computer programming tasks I was ever paid to perform.)

Collateral Damage
While pushing this feature, I managed to break the site’s search engine. The search solution I developed was always sketchy (involving compiling a C program as a static binary CGI script and trusting it to run on the server). I will probably need to find a better approach, preferably sooner than later.

Posted in General | 5 Comments »

5 Responses

  1. Kostya Says:

    I still suspect you’ll get to YM3812 or YMF262 chiptune support ;)

  2. Alex Bennee Says:

    So the big hole in my mind is the large corpus of C64 tunes (possibly the best sound chip of any of the 8 bit micros) and anything from the 16 bit micro era (Atari ST and Amiga). Any plans for those?

  3. Multimedia Mike Says:

    @Kostya: Yay! OPL2 & OPL3 support.

    @Alex Bennee: I have been wanting to add support for C64 SID tunes for a long time. A few things consistently block me, though: For one, the player code (libsidplayfp) is an inscrutable C++ behemoth. I sometimes make some headway into getting it incorporated but then a new version comes out and changes things. The second problem is in staging the files. Specifically, discerning the difference between games and demos, since I would really only like to index the former. Unfortunately, the HVSC categorized songs first and foremost by composer and not by game/demo.

  4. Tomer Gabel Says:

    I’m not sure what server you’re running, but setting up an Elasticsearch-based search engine for your site should be dead-simple. If you need any help, let me know.

  5. Multimedia Mike Says:

    @Tomer: Are you referring to Amazon’s Cloudsearch? I’ve looked into that, but it seems like you have to run a search instance at a cost of around $72/month. That’s not very attractive, especially when I only have around 5500 “documents” that I need to index and be able to search through right now, coupled with extremely light search volume.

    Meanwhile, I was able to repair my existing search solution, but I am exploring more modern and stable options, while trying to keep costs minimal.