Monthly Archives: January 2009

This Is What I Was Trying To Avoid

I checked my website bandwidth overview tonight. So far for the month of January, the bandwidth served from the main multimedia.cx domain is actually much higher than the bandwidth served by my gaming blog, which never happens (lots more pictures over there). I dug a little deeper into the details and found this:


Googlebot causing bandwidth havoc

So who is 66.249.67.1? Why, none other than crawl-66-249-67-1.googlebot.com. Why has it taken such an interest in my site? Oh, little pages like this:

66.249.67.1 – – [01/Jan/2009:00:58:01 -0500] “GET /fate/index.php?stderr=41851 HTTP/1.1” 200 69107 “-” “Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)”

66.249.67.1 – – [01/Jan/2009:00:58:06 -0500] “GET /fate/index.php?build_record=43652 HTTP/1.1” 200 3297 “-” “Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)”

You see, I thought I had administered my FATE web database responsibly by adding the appropriate robots exclusion file at http://fate.multimedia.cx/robots.txt by simply disallowing crawlers at this point. I completely neglected that http://multimedia.cx/fate/ is a perfectly valid route into the site.

Lesson learned.

About That 32-bit Chroot

Pursuant to my earlier frustration with building and running 32-bit binaries on a 64-bit Linux installation, I have returned to the chroot suggestions set forth in the comments section for that post. I found the DebootstrapChroot HOWTO on the Ubuntu Wiki which seems to be a fairly authoritative solution. Except that it didn’t work right for the many times I was trying to set it up some weeks ago.

I finally got the 32-bit chroot to work tonight. Thus, I am working to migrate the 8 x86_32 configurations over from the VMware machine. All of the gcc versions work when transplanted directly (2.95.3, though, is only happy living in the same path where it originally resided). I am rebuilding a new version of gcc-svn for x86_32 (no reason to migrate an old version when I am constantly updating from gcc SVN anyway). I could probably migrate the Intel C compiler wholesale, but it would probably be better to take this opportunity to finally upgrade from .15 to .17 in the 10.1 series, at least until Carl Eugen Hoyos gives the all-clear to upgrade to the later series (last I heard, it can’t handle the FFmpeg source).

For my future reference, as well as for the benefit of other confused Ubuntu users, I am documenting how I managed to set up the 32-bit chroot environment. I started with the instructions at https://wiki.ubuntu.com/DebootstrapChroot and found them to be mostly accurate but in the wrong order in some places. Mostly, it had to do with the mount points and when to activate them. The original Wiki describes chrooting as root, performing a bunch of apt-get package maintenance (section: “Setting up your chroot with debootstrap”), and only sometime later (section: “Getting stuff (…) working automagically”) setting up special mount points. Then later on (section: “Setting up a dchroot (non-root) environment”), the document recommends replacing the earlier mount points with a new set. Following those instructions in that order always left me with a confused and corrupted chroot setup. I solved the problem by setting up the second set of mount points before performing the initial package maintenance (and never using the first set).

Also, I found it very useful and bandwidth-saving to make a backup copy (‘cp -a /var/chroot/intrepid /var/chroot/fresh.intrepid’) after the cursory debootstrap command. Just in case, you know, it doesn’t work out quite right the first, second, … tenth time, you won’t have to wipe the chroot directory and download all the packages again from scratch.