How smart is it to do some custom modifications
It depends on how those changes are made and what are your skills in development and patches management. In general:
- If you do not need to customize, then do not customize.
- If you need a custom feature, try to achieve it by a plugin without changing the Moodle core.
- If you need to change the Moodle core, do it wisely: minimise changes, keep it well isolated, maintain a custom git branch, do not alter core DB tables and APIs.
It is important to realize that any such change is not a one-time task. It is not a thing you do once and use it forever. You will have to look after every such change. Re-apply it on the next upgrade, re-test it, deal with eventual conflicts with upstream changes and so on.