That certainly would have been true in the past. With modern PHP accelerators, I am not sure if that rule of thumb is still relevant.
... Googles ...
https://nikic.github.io/2012/01/09/Disproving-the-Single-Quotes-Performance-Myth.html
http://www.phptherightway.com/pages/The-Basics.html
Those both pre-date PHP 7. It seems that string interpolation was faster if you were inserting many values into a string. Concatenation faster if you are only inserting a few. In either case the difference is small. Mostly favour the more readable code, which is probably a matter of opinion.