Since you're on a roll, adding linking for files should be easy ;)
Instead of having the SCM viewer url point to the changeset, just
have it point to the root location. E.g.

http://fisheye.example.com/changelog/svn/

You can now figure out what to add depending on where you are.
E.g.

http://fisheye.example.com/changelog/svn/?cs=${revision} for the
revision

http://fisheye.example.com/changelog/svn/${filename}?r=${revision}

to go directly to the file.

This would work in fisheye anyway ;) I'm sure there are rules you could
discern for the other SCM viewer tools too.

--
What might be useful here is to have a couple of predefined configurations for fisheye and viewcvs etc where the user only needs to specify the base url - the changelist and file urls are automatically configured (based on the contents of an external properties file - so that it can be tweaked as needed).

Then, if there is an scm viewer that does not have built in support, we allow the user to specify all three pieces of information.
a) base url.
b) changelist url with appropriate variables.
c) file url with appropriate variables.