Description of the Apple Graphics (SMC) Codec by Mike Melanson (mike at multimedia.cx) v1.1: March 13, 2003 ======================================================================= NOTE: The information in this document is now maintained in Wiki format at: http://wiki.multimedia.cx/index.php?title=Apple_SMC ======================================================================= Copyright (c) 2002-2003 Mike Melanson Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License". Contents -------- * Introduction * Basics of SMC Data and Decoding * SMC Opcodes * Appendix A: Understanding The 8-Color Encoding Mode * References * Changelog * GNU Free Documentation License Introduction ------------ The Apple Graphics, a.k.a. SMC, codec is sometimes used to compress frames in Apple Quicktime files. It is a computationally simple video codec that operates on 4x4 blocks of 8-bit palettized pixel values. It may as well be called the copy-and-paste codec since it has a number of codes which allow it to copy blocks that have already been rendered in the current frame. The SMC codec is identified in Quicktime files by the codec FOURCC "smc " (note the space, ASCII 0x20, needed to complete the four-character code). Ostensibly, the codec was named after its developer, Sean M. Callahan. Basics of SMC Data and Decoding ------------------------------- All multi-byte values (there's really just 1) are stored in big-endian format. An SMC decoder renders 4x4 pixel blocks from left to right, top to bottom. The source data are 8-bit palettized values. The palette for an SMC-encoded Quicktime file are transported within the stsd atom of the video header. An SMC decoder maintains 3 circular arrays of 256 elements each. Each element contains 2, 4 or 8 colors which are used in decoding 2-, 4-, and 8-color blocks: * color pair array for 2-color encoded blocks +---------+---------+ pair 0 | color 0 | color 1 | +---------+---------+ pair 1 | color 0 | color 1 | +---------+---------+ | .. .. | +---------+---------+ pair 255 | color 0 | color 1 | +---------+---------+ * color quad array for 4-color encoded blocks +---------+---------+---------+---------+ quad 0 | color 0 | color 1 | color 2 | color 3 | +---------+---------+---------+---------+ quad 1 | color 0 | color 1 | color 2 | color 3 | +---------+---------+---------+---------+ | .. .. .. .. | +---------+---------+---------+---------+ quad 255 | color 0 | color 1 | color 2 | color 3 | +---------+---------+---------+---------+ * color octet array for 8-color encoded blocks +---------+---------+---------+---------+ octet 0 | color 0 | color 1 | .. | color 7 | +---------+---------+---------+---------+ octet 1 | color 0 | color 1 | .. | color 7 | +---------+---------+---------+---------+ | .. .. .. .. | +---------+---------+---------+---------+ octet 255 | color 0 | color 1 | .. | color 7 | +---------+---------+---------+---------+ Because the arrays are circular, if an array exceeds 256 elements, new elements are stored starting from the beginning of the array. All 3 arrays are reset before decoding a new frame. The first byte of a chunk is probably a flags byte. XAnim's comments seem to imply that this byte is always 0xe1. I've also observed a 0x80 byte in this byte. In either case, the meaning of the byte is unknown. Bytes 2-4 are the length of the chunk. This value should match the value transported in the Quicktime file's video chunk length atom. Then, the bytestream in the chunk is traversed. The decoder renders blocks based on the upper nibble of command opcodes. SMC Opcodes ----------- After the first 4 bytes, an SMC chunk is a stream of opcodes and associated data. The opcode is the upper nibble of the next byte in the stream. The meaning of each opcode is detailed next. 0x00 or 0x10: Skip Blocks ------------------------- These 2 opcodes instruct the decoder to skip blocks in the output frame. If there are 4x4 pixel blocks in the previous frame that are the same as in the current frame, there's no point in rendering them again, so these opcodes allow the decoder to skip blocks. These opcodes skip n blocks, where n is 1 plus the lower nibble of the opcode byte if the opcode is 0x00, or 1 plus the next byte in the stream if the opcode is 0x10. 0x20 or 0x30: Repeat Last Block ------------------------------- These 2 opcodes instruct the decoder to repeat the last block over the next n blocks, where n is 1 plus the lower nibble of the opcode byte if the opcode is 0x20, or 1 plus the next byte in the stream if the opcode is 0x30. If the first block to be rendered is the first block on a line, repeat the last block from the previous line. 0x40 or 0x50: Repeat Previous 2 Blocks -------------------------------------- These 2 opcodes instruct the decoder to repeat the last pair blocks over the next n pairs of blocks, where n is 1 plus the lower nibble of the opcode byte if the opcode is 0x40, or 1 plus the next byte in the stream if the opcode is 0x50. If the first block to be rendered is the first block on a lines, repeat the last 2 blocks from the previous line. Similarly, if the first block to be rendered is the second block on a line, the first repeated block will be the last block from the previous line, while the second repeated block will be the first block from the current line. 0x60 or 0x70: 1-Color Encoding ------------------------------ These 2 opcodes instruct the decoder to paint the next n blocks the same color. The number of blocks is 1 plus the lower nibble of the opcode byte if the opcode is 0x60, or 1 plus the next byte in the stream if the opcode is 0x70. The color to paint the block is the palette entry indexed by the next byte in the stream. 0x80 or 0x90: 2-Color Encoding ------------------------------ These 2 codes both specify a pair of colors with which to paint the next n blocks. The number of blocks to paint is equal to 1 plus the lower nibble of the opcode. The color pair to be used to paint the blocks depends on the opcode. If the opcode is 0x80, the next 2 bytes in the stream specify the palette indices to be used. The 2 colors are stored in the next available entry in the color pair circular array. If the opcode is 0x90, then the next byte in the stream specifies an index into the color pair circular array containing the color pair to be used. For each block to be rendered, there are 2 bytes (called a and b in this example) in the stream that are arrays of flags which specify which of the 2 colors in the selected pair will paint which pixels in the block. The flags are arranged as follows: a7 a6 a5 a4 a3 a2 a1 a0 b7 b6 b5 b4 b3 b2 b1 b0 For example, it bit 3 of byte b is 1, then the pixel in the lower left corner of the block is set to color 1 from the selected pair, otherwise it is set to color 0. 0xA0 or 0xB0: 4-Color Encoding ------------------------------ These 2 codes both specify a quad of colors with which to paint the next n blocks. The number of blocks to paint is equal to 1 plus the lower nibble of the opcode. The color quad to be used to paint the blocks depends on the opcode. If the opcode is 0xA0, the next 4 bytes in the stream specify the palette indices to be used. The 4 colors are stored in the next available entry in the color quad circular array. If the opcode is 0xB0, then the next byte in the stream specifies an index into the color quad circular array containing the color quad to be used. For each block to be rendered, there are 4 bytes (called a, b, c and d in this example) in the stream that are arrays of flags which specify which of the 4 colors in the selected pair will paint which pixels in the block. The flags are arranged as follows: a76 a54 a32 a10 b76 b54 b32 b10 c76 c54 c32 c10 d76 d54 d32 d10 For example, if the lower 2 bits of byte a are 11 (3 decimal), then color 3 of the selected color quad is placed in the upper right corner of the block. 0xC0 or 0xD0: 8-Color Encoding ------------------------------ These 2 codes both specify an octet of colors with which to paint the next n blocks. The number of blocks to paint is equal to 1 plus the lower nibble of the opcode. The color octet to be used to paint the blocks depends on the opcode. If the opcode is 0xC0, the next 8 bytes in the stream specify the palette indices to be used. The 8 colors are stored in the next available entry in the color octet circular array. If the opcode is 0xD0, then the next byte in the stream specifies an index into the color octet circular array containing the color octet to be used. For each block to be rendered, there are 6 bytes in the stream that will comprise arrays of flags that specify which of the 8 colors in the selected pair will paint which pixels in the block. It's best to assemble the next 6 bytes into 2 24-bit numbers stored in 2 32-bit variables (a and b in this example). The next 6 bytes in the stream are rearranged in the most unusual manner. For 6 bytes: [01 23 45 67 89 AB] broken down into a vector of 12 nibbles: [n0n1 n2n3 n4n5 n6n7 n8n9 nAnB] this is the arrangement of color flags: flags_a = n0n1 n2n4 n5n6 flags_b = n8n9 nAn3 n7nB If you have trouble believing this arrangement, you can verify it for yourself by running the sample code in Appendix A that was taken from the XAnim source code. The flags are arranged as follows: a23-21 a20-18 a17-15 a14-12 a11-9 a8-6 a5-3 a2-0 b23-21 b20-18 b17-15 b14-12 b11-9 b8-6 b5-3 b2-0 For example, bits 23-21 of a32 will yield the index into the selected color octet to specify the color of the first pixel of the block. If bits 23-21 of a32 are 101 (5 decimal), color 5 from the selected octet is placed in the first pixel of the block. 0xE0: 16-Color Encoding ----------------------- For n blocks, where n is 1 plus the lower nibble of the command opcode (no exception here), copy the blocks directly into the output image. Each block is comprised of 16 bytes and each byte is a palette entry. For bytes [0, 1, .., 15] in the bytestream, the block is laid out as 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 0xF0: Unknown Opcode -------------------- The XAnim source code does not necessarily list this as an invalid opcode, just an unknown one. Appendix A: Understanding The 8-Color Encoding Mode --------------------------------------------------- This program demonstrates the mangling of the flags for the 8-color encoding. The indented portion of this program (with all the mbits variable assignments) was taken directly from the XAnim source code. To separate out the code and run it in this test application was the only way I was able to discern the exact arrangement of the flags. It does not make any sense to me, but it does the right thing. This is the output of the program when compiled and run: mbits0 = 00012456, mbits1 = 0089A37B Therefore, for 6 bytes, broken down into 12 nibbles [n0, n1..nA, nB], the flag arrangement is: flags_a = n0n1 n2n4 n5n6 flags_b = n8n9 nAn3 n7nB -------8<------- #include int main() { unsigned int t, mbits0, mbits1; unsigned char bytes[] = {0x01, 0x23, 0x45, 0x67, 0x89, 0xAB}; unsigned char *dptr = bytes; /* from XAnim: */ t = (*dptr++) << 8; t |= *dptr++; mbits0 = (t & 0xfff0) << 8; mbits1 = (t & 0x000f) << 8; t = (*dptr++) << 8; t |= *dptr++; mbits0 |= (t & 0xfff0) >> 4; mbits1 |= (t & 0x000f) << 4; t = (*dptr++) << 8; t |= *dptr++; mbits1 |= (t & 0xfff0) << 8; mbits1 |= (t & 0x000f); printf ("mbits0 = %08X, mbits1 = %08X\n", mbits0, mbits1); return 0; } -------8<------- References ---------- XAnim http://xanim.polter.net/ ChangeLog --------- v1.1: March 13, 2003 - licensed under GNU Free Documentation License - minor cosmetic changes v1.0: February 10, 2002 - added Appendix with source code from XAnim that shows the 8-color flag arrangement - document is now believed to be correct, which is to say that a proper decoder has been successfully implemented based on this information (which is not to claim that there are no more typos or other mistakes) v0.2: January 21, 2002 - added trivia about "SMC" name origin - added note about resetting color arrays - fixed color flags in 8-color encoding - still not believed to be completely correct v0.1: January 19, 2002 - initial release, but not believed to be completely correct yet GNU Free Documentation License ------------------------------ see http://www.gnu.org/licenses/fdl.html