The easiest thing to do would be to copy the entire site from the server, along with the contents of the database. Then, import the database on your own machine, and edit all the site.ini.append.php files under settings/siteaccess/ to match the database information for your machine.
If you want to know anyway, the important folders are probably var (it's a good idea to clear the cache before you download this one), settings/override, settings/siteaccess, design/, and perhaps extension/ if you have any extensions enabled on your site. Doing it in bits like this though is more likely to go wrong, so just download everything - it'll be far simpler.
If you have to copy between the prod and the dev and the otherway around on a regular basis (eg once you've updated your local design and want to put in in production), rsync comes very handy.
X+
P.S. If I'm right, it only works on modern operating systems (proselytism intended ;)