{"id":507,"date":"2008-01-23T22:48:38","date_gmt":"2008-01-24T06:48:38","guid":{"rendered":"http:\/\/multimedia.cx\/eggs\/its-the-little-things\/"},"modified":"2008-01-24T21:51:01","modified_gmt":"2008-01-25T05:51:01","slug":"its-the-little-things","status":"publish","type":"post","link":"https:\/\/multimedia.cx\/eggs\/its-the-little-things\/","title":{"rendered":"It’s The Little Things"},"content":{"rendered":"

There will never be a shortage of things to do on the new FATE Server<\/a>. I didn’t succeed in much tonight but I did modify one minor detail: Rather than reporting the build record times as absolute UTC timestamps, the system now reports that a build record was logged, e.g., 8h45m ago. I think it’s a little more useful. It even correctly reports that the last x86_32 for gcc 2.95.3 occurred 8 days ago (at the time of this writing), which leads me to the next item…<\/p>\n

The problem with the 2.95.3, as mentioned in a previous post, is that the build produces an extraordinary number of warnings, upwards of 900K. Don’t worry about the storage implications because A) I have tons of space; and B) I am storing build stdout\/stderr text compressed using MySQL’s COMPRESS() and UNCOMPRESS() functions. When the client script tries to send over the 900K of text data, something goes bad (server hangs and eventually times out). <\/p>\n

So, why not compress the data on the client side before sending it to the server? I’ll tell you why– because MySQL’s UNCOMPRESS() function doesn’t like the data if it is compressed by either Python’s internal zlib module<\/a> or if it’s compressed by command line gzip. Other solutions? Perhaps having MySQL installed on the client machine so that the script can process the data through MySQL locally before feeding it to the server?<\/p>\n

A little googling turns up this recent blog post<\/a> discussing the matter. It seems that MySQL prepends standard zlib data with a 30-bit “decompressed-length” field (4 bytes with 2 bits masked). When I am more awake, I may try to add those bytes manually to the compressed string. <\/p>\n

Manipulating binary data in very high level languages always frightens me.<\/p>\n

Update:<\/strong> If you would like to see a solution to this problem in Python, see Clientside MySQL Compression<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

There will never be a shortage of things to do on the new FATE Server. I didn’t succeed in much tonight but I did modify one minor detail: Rather than reporting the build record times as absolute UTC timestamps, the system now reports that a build record was logged, e.g., 8h45m ago. I think it’s […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[101],"tags":[],"class_list":["post-507","post","type-post","status-publish","format-standard","hentry","category-fate-server"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t