I am just experimenting with this app and trying to use it as a helpdesk feature.
You need to adjust the permissions. The app is loaded as a course activity so open within the course and under 'Dialogue Administration' choose permissions.
Change the permissions Receive, who can be the recipient when opening a conversationmod/dialogue:receive to include / exclude the groups you want. For example if you exclude 'students' then they can not send to another student.
But then you have to decide who they can send a Dialogue to. When you are actually in the app the user has to select a recipient from the drop down box. I am using this as a helpdesk option. I have created a user called 'HelpDesk' with management role. Within the description area of the app I ask users to type 'Helpdesk' in the 'Open with People search potential box'
This app needs to sit within a course. I have created a Helpdesk course and under course admin set groups to separate. I think this means they can not see other peoples posts but am still checking this.
The Dialogue App fires an email to the 'Helpdesk' with a 'view in Moodle' and the 'Helpdesk' reply fires an email to the user with a 'view in moodle' link. This is a very good feature if you are trying to keep all communication out of an email inbox. It is good for history of conversation and support advice and also GDPR - all communication with that user can be deleted with the user account if needed. It is very useful for support.