As an update on this, Tim is now looking at another way to solve this problem without adding this functionality to format_text, which Damyon suggested. The new approach is to turn off all filters in the call to format_text (which you can already do), then apply the filters afterward. This will still involve a (smaller) core change as there is currently no easy way to apply all the filters in a context except x, but it'll be at a lower level in the API.
Damyon also pointed out that it wasn't very clear what our use case was for this requirement. To explain that, basically the 'URL to link' filter is great because it means people e.g. students can just type in a link in any forum post and it will work, so we'd like to turn it on. However, we have a system (a custom module) for staff to create formal study material using an XML-based authoring approach. In this system staff have full control over what is, and what isn't, linked. For example, if they are writing a course about web programming, they might refer to www.example.org in some text - in this case they do not wish that to become a link.
While staff could potentially mark that particular piece of text as not being linked, we have approximately 60,000 existing instances of this and we don't want to go through all of these either adding custom filter configuration (and presumably then having to make our system also automatically do this for each one in future) or adding in nolink markers around everything that looks like a link, etc.
We do want to apply other filters such as to turn video links into embedded players, because our system is already expecting that filter to be in use.
In the past we also had a similar requirement to turn off the emoticon filter within this same module, because that was applying unintended emoticons.
--sam