Just got it today :-). The Arduino is a great learning platform for anyone interested in the embedded world. The entire platform is open (hardware schematics, bootloader, lots of examples, etc.).
Comes with a programming language that abstracts out lot of the bit-level stuff… but for learning purposes I’d recommend just using straight C – i.e. to get right down to the bare metal. You’ll probably end up reading two-thirds of the 350 page ATmega168 specs. Atmel’s AVR docs are really well done, with assembly language and bit-level C-language code snippets. Ubuntu has a ready-made GNU C toolchain for the AVR which works quiet well. There is also an Eclipse plugin.