Merge branch 'patch-28' into 'master'

Update robots.txt (#51167)

See merge request gitlab-org/gitlab-ce!21585

public/robots.txt

@@ -21,6 +21,8 @@ Disallow: /groups/new
 Disallow: /groups/*/edit
 Disallow: /users
 Disallow: /help
+# Only specifically allow the Sign In page to avoid very ugly search results
+Allow: /users/sign_in
 # Global snippets
 User-Agent: *