I have not found a way to keep points from appearing when/if the student accesses the individual assignment area; irrespective if the settings for "Grades" is NO and from the Gradebook, the assignment is toggled to invisible.
An alternative (with some limitations) is to do what you are doing BUT, rather than entering points(scores) in the assignment window, enter them in an external spreadsheet - this also will allow you to return to assignments and revise/re-evaluate (comments AND points) UNTIL you are ready to release [allow students to see "comments" and "points" assigned].
When ready to finalize, return to the "allow quick grading" window and enter the points earned from the spreadsheet to each student and make visible both the "Grades" setting AND the "Assignment" from the "Gradebook" - now when students access "Grades" or "Assignment" area, they will see both points and comments.
If you UNCHECK "Send notification emails" your other objective will be achieved during the actual grading/comment phase. IF you follow the above alternative, you can check the "notification" box AFTER entering all scores and then students will receive the usual email alert.