Your data directory can be anwhere that is reachable by your web server. So it could be on a removable memory device or on the network. To let the system know where the data is you need to modify the config.php file.
The purpose and configuration of the three components in this type of web application, the web server, the database and the moodle PHP code is not obvious. Fortunatly most of the moodle configuration sits in that one file and after some study it is not overly complex.
Marcus