We keep a separate git branch with all plugin (or even several of them), then separate git branch(es) with modifications to core and to plugins.
Every time you upgrade moodle (via git), you create a new production branch, pull the upgraded moodle code into it and then pull all your other branches on top of it.
This all requires good knowledge of git, of course.
Another thing I can recommend you is to think about how you can avoid core and plugin modifications. For example, if you are affected by some bug in core and keep re-applying a fix after each upgrade - why not submit the fix itself to moodle core? Maybe it's not even a functional bug but missing
CSS classes that don't allow you to do styling you want. Or moodle uses the same language string for things that are different in your language - fix it in core and you'll help many other people. See
https://docs.moodle.org/dev/Contributing_to_Moodle
Same thing with the plugins - send a pull request to the plugin developer.
Or maybe there is already a way to achieve your goal without hacking? Ask on forums, on plugin comments. Moodle codebase is huge, there are so many settings, one person can't possibly know about everything but the more people you ask the more options you'll find.