Honestly, I've never questioned it. The upside seems that yes, it could make refactoring easier. Downside, though, is not knowing where certain tests live. The command+shift+T shortcut to open the test file corresponding to a production code file is very useful to me, too.
Thanks, I'll have a think about it