Merge branch '7-9-4' into '7-9-stable'

Changes for 7.9.4

Includes next fixes:

- Security: Fix project import URL regex to prevent arbitary local repos from being imported
- Fixed issue where only 25 commits would load in file listings
- Fix LDAP identities  after config update

See merge request !1769

CHANGELOG

 Please view this file on the master branch, on stable branches it's out of date.
+v 7.10.0 (unreleased)
+  - Fix project import URL regex to prevent arbitary local repos from being imported.
+  - Fix bug where Wiki pages that included a '/' were no longer accessible (Stan Hu)
+  - Fix bug where error messages from Dropzone would not be displayed on the issues page (Stan Hu)
+  - Add ability to configure Reply-To address in gitlab.yml (Stan Hu)
+  - Fix broken side-by-side diff view on merge request page (Stan Hu)
+  - Set Application controller default URL options to ensure all url_for calls are consistent (Stan Hu)
+  - Allow HTML tags in Markdown input
+  - Fix code unfold not working on Compare commits page (Stan Hu)
+  - Fix dots in Wiki slugs causing errors (Stan Hu)
+  - Make maximum attachment size configurable via Application Settings (Stan Hu)
+  - Update poltergeist to version 1.6.0 to support PhantomJS 2.0 (Zeger-Jan van de Weg)
+  - Fix cross references when usernames, milestones, or project names contain underscores (Stan Hu)
+  - Disable reference creation for comments surrounded by code/preformatted blocks (Stan Hu)
+  - Reduce Rack Attack false positives causing 403 errors during HTTP authentication (Stan Hu)
+  - enable line wrapping per default and remove the checkbox to toggle it (Hannes Rosenögger)
+  - extend the commit calendar to show the actual commits made on a date (Hannes Rosenögger)
+  - Fix a link in the patch update guide
+  - Add a service to support external wikis (Hannes Rosenögger)
+  - Omit the "email patches" link and fix plain diff view for merge commits
+  - List new commits for newly pushed branch in activity view.
+  - Add sidetiq gem dependency to match EE
+  - Add changelog, license and contribution guide links to project tab bar.
+  - Improve diff UI
+  - Fix alignment of navbar toggle button (Cody Mize)
+  - Fix checkbox rendering for nested task lists
+  - Identical look of selectboxes in UI
+  - Upgrade the gitlab_git gem to version 7.1.3
+  - Move "Import existing repository by URL" option to button.
+  - Improve error message when save profile has error.
+  - Passing the name of pushed ref to CI service (requires GitLab CI 7.9+)
+  - Add location field to user profile
+  - Fix print view for markdown files and wiki pages
+  - Fix errors when deleting old backups
+  - Improve GitLab performance when working with git repositories
+  - Add tag message and last commit to tag hook (Kamil Trzciński)
+  - Restrict permissions on backup files
+  - Improve oauth accounts UI in profile page
+  - Add ability to unlink connected accounts
+  - Replace commits calendar with faster contribution calendar that includes issues and merge requests
+  - Add inifinite scroll to user page activity
+  - Don't include system notes in issue/MR comment count.
+  - Don't mark merge request as updated when merge status relative to target branch changes.
+  - Link note avatar to user.
+  - Make Git-over-SSH errors more descriptive.
+  - Fix EmailsOnPush.
+  - Refactor issue filtering
+  - AJAX selectbox for issue assignee and author filters
+  - Fix issue with missing options in issue filtering dropdown if selected one
+  - Prevent holding Control-Enter or Command-Enter from posting comment multiple times.
+  - Prevent note form from being cleared when submitting failed.
+  - Improve file icons rendering on tree (Sullivan Sénéchal)
+  - API: Add pagination to project events
+  - Get issue links in notification mail to work again.
+  - Don't show commit comment button when user is not signed in.
+  - Fix admin user projects lists.
+  - Don't leak private group existence by redirecting from namespace controller to group controller.
+  - Ability to skip some items from backup (database, respositories or uploads)
+  - Fix "Hello @username." references not working by no longer allowing usernames to end in period.
+  - Archive repositories in background worker.
+  - Import GitHub, Bitbucket or GitLab.com projects owned by authenticated user into current namespace.
+  - Project labels are now available over the API under the "tag_list" field (Cristian Medina) 
+  - Fixed link paths for HTTP and SSH on the admin project view (Jeremy Maziarz)
+  - Fix and improve help rendering (Sullivan Sénéchal)
+  - Fix final line in EmailsOnPush email diff being rendered as error.
+
+v 7.9.4
+  - Security: Fix project import URL regex to prevent arbitary local repos from being imported
+  - Fixed issue where only 25 commits would load in file listings
+  - Fix LDAP identities  after config update
+
+v 7.9.3
+  - Contains no changes
+  - Add icons to Add dropdown items.
+
+v 7.9.2
+  - Contains no changes
+
 v 7.9.1
   - Include missing events and fix save functionality in admin service template settings form (Stan Hu)
   - Fix "Import projects from" button to show the correct instructions (Stan Hu)

app/models/identity.rb

@@ -15,4 +15,5 @@ class Identity < ActiveRecord::Base
   belongs_to :user
   validates :extern_uid, allow_blank: true, uniqueness: { scope: :provider }
+  validates :user_id, uniqueness: { scope: :provider }
 end

app/models/project.rb

@@ -136,7 +136,7 @@ class Project < ActiveRecord::Base
   validates_uniqueness_of :name, scope: :namespace_id
   validates_uniqueness_of :path, scope: :namespace_id
   validates :import_url,
-    format: { with: URI::regexp(%w(ssh git http https)), message: 'should be a valid url' },
+    format: { with: /\A#{URI.regexp(%w(ssh git http https))}\z/, message: 'should be a valid url' },
     if: :import?
   validates :star_count, numericality: { greater_than_or_equal_to: 0 }
   validate :check_limit, on: :create

app/views/projects/refs/logs_tree.js.haml

@@ -15,5 +15,5 @@
     if(current_url == log_url) {
       // Load 10 more commit log for each file in tree
       // if we still on the same page
-      ajaxGet('#{logs_file_namespace_project_ref_path(@project.namespace, @project, @ref, @path || '/', offset: (@offset +  @limit))}');
+      ajaxGet('#{logs_file_namespace_project_ref_path(@project.namespace, @project, @ref, @path || '', offset: (@offset +  @limit))}');
     }

config/gitlab.yml.example

@@ -128,6 +128,15 @@ production: &base
   ldap:
     enabled: false
     servers:
+      ##########################################################################
+      #
+      # Since GitLab 7.4, LDAP servers get ID's (below the ID is 'main'). GitLab
+      # Enterprise Edition now supports connecting to multiple LDAP servers.
+      #
+      # If you are updating from the old (pre-7.4) syntax, you MUST give your
+      # old server the ID 'main'.
+      #
+      ##########################################################################
       main: # 'main' is the GitLab 'provider ID' of this LDAP server
         ## label
         #

config/initializers/1_settings.rb

@@ -64,10 +64,11 @@ Settings.ldap['enabled'] = false if Settings.ldap['enabled'].nil?
 # backwards compatibility, we only have one host
 if Settings.ldap['enabled'] || Rails.env.test?
   if Settings.ldap['host'].present?
+    # We detected old LDAP configuration syntax. Update the config to make it
+    # look like it was entered with the new syntax.
     server = Settings.ldap.except('sync_time')
-    server['provider_name'] = 'ldap'
     Settings.ldap['servers'] = {
-      'ldap' => server
+      'main' => server
     }
   end
@@ -80,6 +81,7 @@ if Settings.ldap['enabled'] || Rails.env.test?
   end
 end
+
 Settings['omniauth'] ||= Settingslogic.new({})
 Settings.omniauth['enabled']      = false if Settings.omniauth['enabled'].nil?
 Settings.omniauth['providers']  ||= []

db/migrate/20150411000035_fix_identities.rb

+class FixIdentities < ActiveRecord::Migration
+  def up
+    # Up until now, legacy 'ldap' references in the database were charitably
+    # interpreted to point to the first LDAP server specified in the GitLab
+    # configuration. So if the database said 'provider: ldap' but the first
+    # LDAP server was called 'ldapmain', then we would try to interpret
+    # 'provider: ldap' as if it said 'provider: ldapmain'. This migration (and
+    # accompanying changes in the GitLab LDAP code) get rid of this complicated
+    # behavior. Any database references to 'provider: ldap' get rewritten to
+    # whatever the code would have interpreted it as, i.e. as a reference to
+    # the first LDAP server specified in gitlab.yml / gitlab.rb.
+    new_provider = if Gitlab.config.ldap.enabled
+                     first_ldap_server = Gitlab.config.ldap.servers.values.first
+                     first_ldap_server['provider_name']
+                   else
+                     'ldapmain'
+                   end
+
+    # Delete duplicate identities
+    execute "DELETE FROM identities WHERE provider = 'ldap' AND user_id IN (SELECT user_id FROM identities WHERE provider = '#{new_provider}')"
+
+    # Update legacy identities
+    execute "UPDATE identities SET provider = '#{new_provider}' WHERE provider = 'ldap';"
+
+    if table_exists?('ldap_group_links')
+      execute "UPDATE ldap_group_links SET provider = '#{new_provider}' WHERE provider IS NULL OR provider = 'ldap';"
+    end
+  end
+
+  def down
+  end
+end

db/schema.rb

@@ -11,7 +11,7 @@
 #
 # It's strongly recommended that you check this file into your version control system.
-ActiveRecord::Schema.define(version: 20150313012111) do
+ActiveRecord::Schema.define(version: 20150411000035) do
   # These are extensions that must be enabled in order to support this database
   enable_extension "plpgsql"

lib/gitlab/ldap/config.rb

@@ -27,8 +27,6 @@ module Gitlab
       def initialize(provider)
         if self.class.valid_provider?(provider)
           @provider = provider
-        elsif provider == 'ldap'
-          @provider = self.class.providers.first
         else
           self.class.invalid_provider(provider)
         end

lib/gitlab/ldap/user.rb

@@ -13,7 +13,7 @@ module Gitlab
         def find_by_uid_and_provider(uid, provider)
           # LDAP distinguished name is case-insensitive
           identity = ::Identity.
-            where(provider: [provider, :ldap]).
+            where(provider: provider).
             where('lower(extern_uid) = ?', uid.downcase).last
           identity && identity.user
         end

spec/lib/gitlab/ldap/config_spec.rb

@@ -16,19 +16,5 @@ describe Gitlab::LDAP::Config do
     it "raises an error if a unknow provider is used" do
       expect{ Gitlab::LDAP::Config.new 'unknown' }.to raise_error
     end
-
-    context "if 'ldap' is the provider name" do
-      let(:provider) { 'ldap' }
-
-      context "and 'ldap' is not in defined as a provider" do
-        before { Gitlab::LDAP::Config.stub(providers: %w{ldapmain}) }
-
-        it "uses the first provider" do
-          # Fetch the provider_name attribute from 'options' so that we know
-          # that the 'options' Hash is not empty/nil.
-          expect(config.options['provider_name']).to eq('ldapmain')
-        end
-      end
-    end
   end
 end