I second the comments made by both Anthony and Michael, and add my own. You mention having difficulties "when you try out a new feature", or "when you try to get an existing feature to work properly". I assume that not every existing feature has let you down, so could you be more specific and let us know which ones have worked, and which ones haven't? Also, which "new features" are you talking about? Official Moodle features, or 3rd party plugins and modules?
How much are you ready to invest into Moodle? If you're willing to spend some $ you can follow Michael's suggestion and have a Moodle partner host and maintain your Moodle installation. They will take care of those pesky issues that have prevented you from enjoying Moodle in the past. Since they're familiar with the common problems users face, they can save you "duplicated frustration" so that you can focus on the pedagogy and leave the tech side to them.
Comparisons and advocacy
A Stable Example of Moodle Somewhere?
This discussion has been locked so you can no longer reply to it.