Today I’ve set up a private and public SSH key on one server, and I copied the public key to our CVS server. However, my login names were different on those machines, and the keys didn’t help.
Then my colleague advised to create a file under ~/.ssh called config, with an entry like this:
Host happycvs.happycompany.com
User peterk
And suddenly SSH connected fine!