You don't have to run Selenium in a headless environment. If you've got a GUI session running, you can just run that java -jar line in a shell and it will work, it will just display the browser window and show it completing the steps.
If you do want to run it headlessly, you could try the recent work to support headless chrome (this actually bypasses Selenium entirely, which is nice):
https://tracker.moodle.org/browse/MDL-58948
Or if you want to do it with firefox, it's something like this:
- Install a headless X server like Xvfb (exactly how differs according to your OS, something like apt-get install xvfb)
- Start an instance of the headless X server to create a virtual display, e.g Xvfb :1 -screen 0 1600x1200x32
- In a new shell, set the display environment variable to the display you just created. In the example above, I think it will be export DISPLAY=0:1
- In the same shell, run the java -jar line. This will start selenium within the virtual display.
- Run behat, pointing it at the selenium sever (probably localhost:4444). The javascript tests should run, but you wont see them.