A context is just a fancy name for part of Moodle site.
It might be the whole Moodle site. That is the relevant context for administrator-y thinks like adding users or changing the configuration.
Or it might be a course category, or a course, or an individual activity. So, as you can see, the categories contain each other. A course will be inside one or more categories, and the whole site; and it will contain activities. Another way of saying the same thing is that the contexts form a tree structure.
If you like Object-Oriented Programming, then another way to thing about this is to think of the Context as the base class, which is responsible for being part of the tree and managing permission checking; and then the different parts of Moodle like CourseCategory, Course, Activity, and so on, are subclasses. (And to carry the anology further, different types of activity, like ForumActivity, QuizActivity, and so on, are subclasses of the general Activity.)
Of course, Moodle does not use the PHP class/object syntax for these things, so in the PHP code, they don't look like objects, but in a higher level, vague sense, you can choose to think of it like that.