My actual frontpage looks like this:
As you can see, the categories are listened as a list.
What I need, is to change this theme for the categories to be showing as boxes, like on this site:
Which file should I alter?
Actually you don't need to do anything with code at all... well... Try this... Go to the Front Page FAQ and look at the suggestions there. You can also use a Grid layout for the same effect, or even add in a gallery if you want to get really cute.