Changing the continue button to something like 'Close Preview' might help.
Although, I still think some sort of warning might be warranted here. I think a pop up when the Guest clicks on the preview icon would be ideal. Something like "This is only a preview, any changes made in this form will not be saved"
Or, having the Guest preview capability set in the Feedback settings with the default being off. Then those that really need this capability can turn it on accordingly.
Thank-you for your consideration!