Reasonably easy using CSS and if you want different icons for different links, you can see a working example at https://practicelearning.info
Here’s an extract of the CSS from that site:
.dropdown-menu>li>a[title="ScOPTbox Resources"]:before{content:"\f15b"; font-family: FontAwesome; width: 1.2em; display: inline-block; text-align: center; margin-right: 0.5em;}
.dropdown-menu>li>a[title="Reading Reviews"]:before{content:"\f02d"; font-family: FontAwesome; width: 1.2em; display: inline-block; text-align: center; margin-right: 0.5em;}
.dropdown-menu>li>a[title="Links"]:before{content:"\f0c1"; font-family: FontAwesome; width: 1.2em; display: inline-block; text-align: center; margin-right: 0.5em;}
Actually, looking over that now, this CSS is actually quite messy. You could probably define the font-family, width, display, text-align and margin for the whole navbar and then just define the content for each link. But this is old and it works and I don't have to time to redo the whole site at the moment 😕