Yes that is correct; it shows the average mark of all the students in the course for each activity to the all students.
If you don't want the students to see the average and you were just trying to get the average for the teacher to see, go to My preferences instead and activate the desired average settings in the Special Rows section.
Happy 
oodling!