runners.ssh does force you to enter a password (no public key support)
The registration wizard does force you to enter a password for the runners.ssh which is probably not the right way to configure ssh access.
The wizard should accept an empty password (explaining that this would try to use the public key for authentication).
