Short bio: Computer Scientist, FOSS supporter (read more)
Tux Machines (TM)-specific
Once upon a time there was DSL (Damn Small Linux) which provided a full workable system in a 50 MB image. I used to have it on a small tertiary partition for repair and rescue tasks on my main systems. It was never or at least very rarely needed as I recall, but I liked the pre-set radio stations in XMMS and the backgrounds and conky config in DSL, so I ended up running it more and more over Zenwalk 2.6, which at the time was using a very half-baked early XFCE 4.4, if only to listen to internet radio on a geeky minimalistic looking box.
DSL is no more and has been superceded by TinyCore, which is only 10 MB in size. If that is too minimal for you there is another option, SliTaz GNU/Linux. It is a small distribution based in Switzerland that at exactly 30 MB in size sits somewhere in the middle and comes in French and English by default. You choose your language after booting. It´s been around for a while, I had a look at their 1.0 release in 2008 and was impressed. It is mainly intended as a live system but can be installed to hard drive. For being this small it includes a load of functionality, the Lighttpd web server for instance which makes it perfect for loading from USB stick or CD and running a website from memory.