Unfortunately, you can't purge a particular CSS file from the cache. The files are combined into one and cached together, so there's no mechanism to purge the CSS for a particular plugin.
I have two strategies when working with CSS that I've found quicker than Theme Designer Mode. Firstly, you can manually purge caches (either through /admin/purgecaches.php or /admin/cli/purge_caches.php) then reload the page, which means you one have the extra wait when you've done something rather than on every page.
Secondly, on the Style Editor tab of
Firefox's Developer Tools, there's an option to import a file (looks like a box with an upward arrow). If you import the file you want to edit, the rules will be applied to the current page, overriding the same rules in previously loaded files. You can then make changes in Firefox's editor and see them reflected live. When you're done, save the file, purge caches and reload to check the final result.