Deploy key will not allow access to a public project unless it is specifically added

This doesn't make sense. If the project is public - Anonymous/deploy key access via ssh should allow access to clone as well. I shouldn't have to explicitly add the deploy key to the project.