Multibyte character downcasing problem (LDAP)
- Zendesk ticket: https://gitlab.zendesk.com/agent/tickets/9470
- Dev link: None
- GitLab version: 8.0.4
Expected behavior
Users with multibyte characters should be able to sign in via LDAP.
Observed behavior
An LDAP user with multibyte characters is unable to sign in via LDAP. All available information is below. The user receives error "Could not authorize you from Ldapmain because "Undefined method `provider' for nil:nilclass""
Debug logging:
Started POST "/users/auth/ldapmain/callback" for 194.19.95.162 at 2015-10-06 09:47:49 +0000
Processing by OmniauthCallbacksController#ldapmain as HTML
Parameters: {"utf8"=>"✓", "authenticity_token"=>"[FILTERED]", "username"=>"nooyvsur", "password"=>"[FILTERED]"}
User Load (0.9ms) SELECT "users".* FROM "users" WHERE "users"."authentication_token" = 'O7p7nk8yrmyXLKzSBPPnscFbZMmIzMX6c7bCroTayf4=' ORDER BY "users"."created_at" DESC, "users"."id" DESC LIMIT 1
ApplicationSetting Load (0.6ms) SELECT "application_settings".* FROM "application_settings" ORDER BY "application_settings"."id" DESC LIMIT 1
(0.1ms) BEGIN
(0.1ms) ROLLBACK
Identity Load (0.9ms) SELECT "identities".* FROM "identities" WHERE "identities"."provider" = 'ldapmain' AND (lower(extern_uid) = 'cn=Øyvind surname,ou=example,dc=domain,dc=local') ORDER BY "identities"."created_at" ASC, "identities"."id" ASC LIMIT 1
User Load (0.7ms) SELECT "users".* FROM "users" WHERE "users"."email" = 'Oyvind.surname@example.com' ORDER BY "users"."created_at" DESC, "users"."id" DESC LIMIT 1
Namespace Load (0.8ms) SELECT "namespaces".* FROM "namespaces" WHERE (lower(path) = 'nooyvsur' OR lower(name) = 'nooyvsur') ORDER BY "namespaces"."created_at" DESC, "namespaces"."id" DESC LIMIT 1
Namespace Load (0.5ms) SELECT "namespaces".* FROM "namespaces" WHERE (lower(path) = 'nooyvsur1' OR lower(name) = 'nooyvsur1') ORDER BY "namespaces"."created_at" DESC, "namespaces"."id" DESC LIMIT 1
(0.2ms) BEGIN
User Exists (0.5ms) SELECT 1 AS one FROM "users" WHERE "users"."email" = 'oyvind.surname@example.com' LIMIT 1
Identity Exists (0.3ms) SELECT 1 AS one FROM "identities" WHERE ("identities"."extern_uid" = 'CN=Øyvind Surname,DC=domain,DC=local' AND "identities"."provider" = 'ldapmain') LIMIT 1
Customer tested downcasing of the multibyte character in the rails console:
"Running in a ruby-console (gitlab-ctl console), you can see that the ruby downcase method does not work; this is probably what is used on the right side of the first query ("Identity Load"):"
irb(main):040:0> "ABC\u00D8"
=> "ABCØ"
irb(main):041:0> "ABC\u00D8".downcase
=> "abcØ"
However, inside the database, all is dandy:
gitlabhq_production=# select U&'ABC\00D8' from users limit 1;
?column?
----------
ABCØ
(1 row)
gitlabhq_production=# select lower (U&'ABC\00D8') from users limit 1;
lower
-------
abcø
(1 row)
Customer reported finding the following cases where downcase
is used in a query. This may cause problems in this case.
./app/models/concerns/issuable.rb: where("LOWER(title) like :query", query: "%#{query.downcase}%")
./app/models/concerns/issuable.rb: where("LOWER(title) like :query OR LOWER(description) like :query", query: "%#{query.downcase}%")
./app/models/user.rb: where(conditions).where(["lower(username) = :value OR lower(email) = :value", { value: login.downcase }]).first
./app/models/user.rb: where("lower(name) LIKE :query OR lower(email) LIKE :query OR lower(username) LIKE :query", query: "%#{query.downcase}%")
./lib/gitlab/o_auth/user.rb: if user = Gitlab::LDAP::User.find_by_uid_and_provider(ldap_person.dn.downcase, ldap_person.provider)
Next steps
gitlab-foss!1472 (merged) may solve a portion of this issue. Does the above fix all or part of this issue?
@dblessing will try to reproduce by adding a similar user to OpenLDAP. If reproducible, I will post the LDIF here so others can test.
When can we deliver a fix? The user who's account this affects is unable to login.