{"id":60,"date":"2005-07-25T18:19:13","date_gmt":"2005-07-26T00:19:13","guid":{"rendered":"\/?p=60"},"modified":"2006-05-27T09:17:48","modified_gmt":"2006-05-27T16:17:48","slug":"more-nes-video-coding-ideas","status":"publish","type":"post","link":"https:\/\/multimedia.cx\/eggs\/more-nes-video-coding-ideas\/","title":{"rendered":"More NES Video Coding Ideas"},"content":{"rendered":"

[a continuation of Custom NES Video Codec<\/a>]<\/em><\/p>\n

Data compression is all about analyzing the nominal characteristics of a particular type of data and understanding how that data can best be coded using the least amount of information. Let’s look at some of the characteristics of NES video:<\/p>\n

<\/p>\n\n\n\n\n
Fun NES Screenshots:<\/th>\n<\/tr>\n
\"Battletoads<\/td>\n\"Blaster<\/td>\n<\/tr>\n
Battletoads<\/em><\/td>\nBlaster Master<\/em><\/td>\n<\/tr>\n<\/table>\n

<\/center><\/p>\n

<\/p>\n

<\/p>\n\n\n\n
\"The<\/td>\n\"Shadowgate<\/td>\n<\/tr>\n
The Guardian Legend<\/em><\/td>\nShadowgate<\/em><\/td>\n<\/tr>\n<\/table>\n

<\/center><\/p>\n

The data is primarily composed of a background made up of 8×8 tiles that is overlaid with a series of 8×8 sprites (or 8×16 sprites which are technically just 2 stacked 8×8 tiles). The sprites may be moving over a background that stays in the same place from frame to frame. Or, the background might scroll horizontally, vertically, or both from frame to frame.<\/p>\n

To illustrate using the above screenshots as examples, Shadowgate<\/em> is a game in which the entire frame does not change very often. There is no background scrolling. In Blaster Master<\/em>, the background scrolls in every direction and the status is represented as sprites on the left side of the screen. The Guardian Legend<\/em>, when it does scroll, only scrolls in either the vertical or horizontal directions but never both at one time. It also uses a split screen for a status area; only the top 3\/4 of the screen scrolls. Battletoads<\/em> will be a severe aberration to any underlying assumptions that this codec makes. It achieves graphical effects that were considered very sophisticated on the NES. The screenshot above depicts a level in which there was a horizontally-split status screen and a vertical scrolling split.<\/p>\n

If it becomes necessary to encode a block losslessly, it should often be possible to encode in using 4 bits vs. 5 bits. There is a good probability that a given 8×8 block will either be a background tile block or a sprite block. When this is the case, that block can only be rendered with one of the 16-color palettes. Thus, encode each pixel with only 4 bits (with an extra bit or two in front of the block to indicate encoding mode). It may be a little complicated to differentiate between a background\/sprite\/mixed block during encoding, though.<\/p>\n

The driving force behind this video coding brainstorm is that a particular frame either does not change very much from one frame to the next. Either that, or the frame essentially remains the same from one frame to the next while shifting (scrolling) some small number of pixels either horizontally, vertically, or both.<\/p>\n

Some games use trickier graphics such as split-screen status bars (like in the Battletoads<\/em> and The Guardian Legend<\/em> screenshots above). Some may have an inordinate amount of sprite action. So it may be useful to define a slice that is 8 pixels tall. When encoding each row, take into account the various directions it could have moved. Also, notice whether it has moved at all. Analyze current frame on a per-slice basis and figure out which ones have scrolled.<\/p>\n

This codec is meant to be lossless. Thus, it is not necessarily appropriate to compare it to a lossy, constrained bit-rate codec such as MPEG-4. Still, it would be interesting to see how such a codec stacks up against MPEG-4 (and, more recently, H.264), which is what the movies are encoded with. I wrote a little tool to analyze exactly how many bytes are being used to encode the video on various videos taken from the Time Attack<\/a> site. Here are some statistics (all stats assume 60 frames\/second):<\/p>\n