Hi,
If you use submodules, all you need is a repo for your plugin only and you can separate the Moodle codebase. Generally, I have a local clone of the Moodle repo and I clone single branches into my webserver development environment, then I create my plugin as a submodule with an upstream on Github. That way I only am only using my plugin code and occasionally I will update the Moodle code without affecting my plugin development.