From Ubuntu software center I think the packages to look for are apache2, mysql-server, there is a php module for apache, phpmyadmin if you want to use that. Unpack Moodle in /var/www/moodle and it should resolve without additional virtual hosts entry.
If you want it on a bootable USB you can do that but you would have to set bios on any machine you wanted to use it on to boot from USB so its not that portable, you could not use it on a PC in a library for example. There is some info on this page:
https://wiki.ubuntu.com/LiveUsbPendrivePersistent
Probably a few other guides out there and maybe better ways to do it.
I just bought an SSD for an old netbook I had stopped using after it was crippled by Windows 7 updates.
I put Linux Mint on it with LAMP, Moodle, Aptana, runs ok, boots in 40 seconds, would be faster with Lubuntu but I wanted a nicer interface. You can pick something like that up fairly cheaply on Ebay if you want something on the move, screen size is a bit of an issue though.