Merge remote-tracking branch 'origin/master' into ci-and-ce-sitting-in-a-tree-k-i-s-s-i-n-g

CHANGELOG

 Please view this file on the master branch, on stable branches it's out of date.
 v 8.0.0 (unreleased)
+  - Fix broken sort in merge request API (Stan Hu)
   - Bump rouge to 1.10.1 to remove warning noise and fix other syntax highlighting bugs (Stan Hu)
   - Gracefully handle errors in syntax highlighting by leaving the block unformatted (Stan Hu)
   - Add "replace" and "upload" functionalities to allow user replace existing file and upload new file into current repository
@@ -48,6 +49,7 @@ v 8.0.0 (unreleased)
   - Fix highlighting of deleted lines in diffs.
   - Added service API endpoint to retrieve service parameters (Petheő Bence)
   - Add FogBugz project import (Jared Szechy)
+  - Sort users autocomplete lists by user (Allister Antosik)
 v 7.14.3
   - No changes

Gemfile

@@ -167,7 +167,7 @@ gem "slack-notifier", "~> 1.0.0"
 gem 'asana', '~> 0.0.6'
 # FogBugz integration
-gem 'ruby-fogbugz', '~> 0.2.0'
+gem 'ruby-fogbugz', '~> 0.2.1'
 # d3
 gem 'd3_rails', '~> 3.5.5'

Gemfile.lock

@@ -596,7 +596,7 @@ GEM
       powerpack (~> 0.0.6)
       rainbow (>= 1.99.1, < 3.0)
       ruby-progressbar (~> 1.4)
-    ruby-fogbugz (0.2.0)
+    ruby-fogbugz (0.2.1)
       crack (~> 0.4)
     ruby-progressbar (1.7.5)
     ruby-saml (1.0.0)
@@ -886,8 +886,8 @@ DEPENDENCIES
   rerun (~> 0.10.0)
   rqrcode-rails3 (~> 0.1.7)
   rspec-rails (~> 3.3.0)
-  rubocop (~> 0.28.0)
-  ruby-fogbugz (~> 0.2.0)
+  rubocop (= 0.28.0)
+  ruby-fogbugz (~> 0.2.1)
   sanitize (~> 2.0)
   sass-rails (~> 4.0.5)
   sdoc (~> 0.3.20)

app/controllers/autocomplete_controller.rb

@@ -32,6 +32,7 @@ class AutocompleteController < ApplicationController
     @users ||= User.none
     @users = @users.search(params[:search]) if params[:search].present?
     @users = @users.active
+    @users = @users.reorder(:name)
     @users = @users.page(params[:page]).per(PER_PAGE)
     unless params[:search].present?

doc/reply_by_email/README.md

@@ -2,11 +2,48 @@
 GitLab can be set up to allow users to comment on issues and merge requests by replying to notification emails.
-In order to do this, you need access to an IMAP-enabled email account, with a provider or server that supports [email sub-addressing](https://en.wikipedia.org/wiki/Email_address#Sub-addressing). Sub-addressing is a feature where any email to `user+some_arbitrary_tag@example.com` will end up in the mailbox for `user@example.com`, and is supported by providers such as Gmail, Yahoo! Mail, Outlook.com and iCloud, as well as the [Postfix](http://www.postfix.org/) mail server which you can run on-premises.
+## Get a mailbox
+
+Reply by email requires an IMAP-enabled email account, with a provider or server that supports [email sub-addressing](https://en.wikipedia.org/wiki/Email_address#Sub-addressing). Sub-addressing is a feature where any email to `user+some_arbitrary_tag@example.com` will end up in the mailbox for `user@example.com`, and is supported by providers such as Gmail, Yahoo! Mail, Outlook.com and iCloud, as well as the Postfix mail server which you can run on-premises.
+
+If you want to use Gmail with Reply by email, make sure you have [IMAP access enabled](https://support.google.com/mail/troubleshooter/1668960?hl=en#ts=1665018) and [allow less secure apps to access the account](https://support.google.com/accounts/answer/6010255).
+
+To set up a basic Postfix mail server with IMAP access on Ubuntu, follow [these instructions](./postfix.md).
 ## Set it up
-In this example, we'll use the Gmail address `gitlab-replies@gmail.com`. If you're actually using Gmail with Reply by email, make sure you have [IMAP access enabled](https://support.google.com/mail/troubleshooter/1668960?hl=en#ts=1665018) and [allow less secure apps to access the account](https://support.google.com/accounts/answer/6010255).
+In this example, we'll use the Gmail address `gitlab-replies@gmail.com`.
+
+### Omnibus package installations
+
+1. Find the `reply_by_email` section in `/etc/gitlab/gitlab.rb`, enable the feature, enter the email address including a placeholder for the `reply_key` and fill in the details for your specific IMAP server and email account:
+
+    ```ruby
+    gitlab_rails['reply_by_email_enabled'] = true
+    gitlab_rails['reply_by_email_address'] = "gitlab-replies+%{reply_key}@gmail.com"
+    gitlab_rails['reply_by_email_host'] = "imap.gmail.com" # IMAP server host
+    gitlab_rails['reply_by_email_port'] = 993 # IMAP server port
+    gitlab_rails['reply_by_email_ssl'] = true # Whether the IMAP server uses SSL
+    gitlab_rails['reply_by_email_email'] = "gitlab-replies@gmail.com"  # Email account username. Usually the full email address.
+    gitlab_rails['reply_by_email_password'] = "password" # Email account password
+    gitlab_rails['reply_by_email_mailbox_name'] = "inbox" # The name of the mailbox where incoming mail will end up. Usually "inbox".
+    ```
+
+    As mentioned, the part after `+` in the address is ignored, and any email sent here will end up in the mailbox for `gitlab-replies@gmail.com`.
+
+1. Reconfigure GitLab for the changes to take effect:
+
+    ```sh
+    sudo gitlab-ctl reconfigure
+    ```
+
+1. Verify that everything is configured correctly:
+
+    ```sh
+    sudo gitlab-rake gitlab:reply_by_email:check
+    ```
+
+1. Reply by email should now be working.
 ### Installations from source
@@ -21,17 +58,17 @@ In this example, we'll use the Gmail address `gitlab-replies@gmail.com`. If you'
     ```sh
     sudo editor config/gitlab.yml
     ```
-    
+
     ```yaml
     reply_by_email:
       enabled: true
       address: "gitlab-replies+%{reply_key}@gmail.com"
     ```
-    As mentioned, the part after `+` is ignored, and this will end up in the mailbox for `gitlab-replies@gmail.com`.
-2. Find `config/mail_room.yml.example` and copy it to `config/mail_room.yml`:
-    
+    As mentioned, the part after `+` in the address is ignored, and any email sent here will end up in the mailbox for `gitlab-replies@gmail.com`.
+
+2. Copy `config/mail_room.yml.example` to `config/mail_room.yml`:
     ```sh
     sudo cp config/mail_room.yml.example config/mail_room.yml
     ```
@@ -72,47 +109,33 @@ In this example, we'll use the Gmail address `gitlab-replies@gmail.com`. If you'
           :worker: EmailReceiverWorker
     ```
-4.  Find `lib/support/init.d/gitlab.default.example` and copy it to `/etc/default/gitlab`:
-    
+5. Edit the init script configuration at `/etc/default/gitlab` to enable `mail_room`:
     ```sh
-    sudo cp lib/support/init.d/gitlab.default.example /etc/default/gitlab
-    ```
-
-5. Edit `/etc/default/gitlab` to enable `mail_room`:
-
-    ```sh
-    sudo editor /etc/default/gitlab
-    ```
-    
-    ```sh
-    mail_room_enabled=true
+    sudo mkdir -p /etc/default
+    echo 'mail_room_enabled=true' | sudo tee -a /etc/default/gitlab
     ```
 6. Restart GitLab:
-    
+
     ```sh
     sudo service gitlab restart
     ```
-7. Check if everything is configured correctly:
+7. Verify that everything is configured correctly:
     ```sh
-    sudo bundle exec rake gitlab:reply_by_email:check RAILS_ENV=production
+    sudo -u git -H bundle exec rake gitlab:reply_by_email:check RAILS_ENV=production
     ```
 8. Reply by email should now be working.
-### Omnibus package installations
-
-TODO
-
 ### Development
 1. Go to the GitLab installation directory.
 1. Find the `reply_by_email` section in `config/gitlab.yml`, enable the feature and enter the email address including a placeholder for the `reply_key`:
-    
+
     ```yaml
     reply_by_email:
       enabled: true
@@ -121,8 +144,8 @@ TODO
     As mentioned, the part after `+` is ignored, and this will end up in the mailbox for `gitlab-replies@gmail.com`.
-2. Find `config/mail_room.yml.example` and copy it to `config/mail_room.yml`:
-    
+2. Copy `config/mail_room.yml.example` to `config/mail_room.yml`:
+
     ```sh
     sudo cp config/mail_room.yml.example config/mail_room.yml
     ```
@@ -166,12 +189,12 @@ TODO
     ```
 6. Restart GitLab:
-    
+
     ```sh
     bundle exec foreman start
     ```
-7. Check if everything is configured correctly:
+7. Verify that everything is configured correctly:
     ```sh
     bundle exec rake gitlab:reply_by_email:check RAILS_ENV=development

doc/reply_by_email/postfix.md

+# Set up Postfix for Reply by email
+
+This document will take you through the steps of setting up a basic Postfix mail server with IMAP authentication on Ubuntu, to be used with Reply by email.
+
+The instructions make the assumption that you will be using the email address `replies@gitlab.example.com`, that is, username `replies` on host `gitlab.example.com`. Don't forget to change it to your actual host when executing the example code snippets.
+
+## Configure your server firewall
+
+1. Open up port 25 on your server so that people can send email into the server over SMTP.
+2. If the mail server is different from the server running GitLab, open up port 143 on your server so that GitLab can read email from the server over IMAP.
+
+## Install packages
+
+1. Install the `postfix` package if it is not installed already:
+
+    ```sh
+    sudo apt-get install postfix
+    ```
+
+    When asked about the environment, select 'Internet Site'. When asked to confirm the hostname, make sure it matches `gitlab.example.com`.
+
+1. Install the `mailutils` package.
+
+    ```sh
+    sudo apt-get install mailutils
+    ```
+
+## Create user
+
+1. Create a user for replies.
+
+    ```sh
+    sudo useradd -m -s /bin/bash replies
+    ```
+
+1. Set a password for this user.
+
+    ```sh
+    sudo passwd replies
+    ```
+
+    Be sure not to forget this, you'll need it later.
+
+## Test the out-of-the-box setup
+
+1. Connect to the local SMTP server:
+    
+    ```sh
+    telnet localhost 25
+    ```
+
+    You should see a prompt like this:
+
+    ```sh
+    Trying 127.0.0.1...
+    Connected to localhost.
+    Escape character is '^]'.
+    220 gitlab.example.com ESMTP Postfix (Ubuntu)
+    ```
+
+    If you get a `Connection refused` error instead, verify that `postfix` is running:
+
+    ```sh
+    sudo postfix status
+    ```
+
+    If it is not, start it:
+
+    ```sh
+    sudo postfix start
+    ```
+
+1. Send the new `replies` user a dummy email to test SMTP, by entering the following into the SMTP prompt:
+    
+    ```
+    ehlo localhost
+    mail from: root@localhost
+    rcpt to: replies@localhost
+    data
+    Subject: Re: Some issue
+
+    Sounds good!
+    .
+    quit
+    ```
+
+    (Note: The `.` is a literal period on its own line)
+
+1. Check if the `replies` user received the email:
+    
+    ```sh
+    su - replies
+    mail
+    ```
+
+    You should see output like this:
+
+    ```
+    "/var/mail/replies": 1 message 1 unread
+    >U   1 root@localhost                           59/2842  Re: Some issue
+    ```
+
+    Quit the mail app:
+
+    ```sh
+    q
+    ```
+
+1. Log out of the `replies` account and go back to being `root`:
+
+    ```sh
+    logout
+    ```
+
+## Configure Postfix to use Maildir-style mailboxes
+
+Courier, which we will install later to add IMAP authentication, requires mailboxes to have the Maildir format, rather than mbox.
+
+1. Configure Postfix to use Maildir-style mailboxes:
+    
+    ```sh
+    sudo postconf -e "home_mailbox = Maildir/"
+    ```
+
+1. Restart Postfix:
+    
+    ```sh
+    sudo /etc/init.d/postfix restart
+    ```
+
+1. Test the new setup:
+    
+    1. Follow steps 1 and 2 of _[Test the out-of-the-box setup](#test-the-out-of-the-box-setup)_.
+    2. Check if the `replies` user received the email:
+    
+        ```sh
+        su - replies
+        MAIL=/home/replies/Maildir
+        mail
+        ```
+
+        You should see output like this:
+
+        ```
+        "/home/replies/Maildir": 1 message 1 unread
+        >U   1 root@localhost                           59/2842  Re: Some issue
+        ```
+
+        Quit the mail app:
+
+        ```sh
+        q
+        ```
+
+1. Log out of the `replies` account and go back to being `root`:
+
+    ```sh
+    logout
+    ```
+
+## Install the Courier IMAP server
+    
+1. Install the `courier-imap` package:
+
+    ```sh
+    sudo apt-get install courier-imap
+    ```
+
+## Configure Postfix to receive email from the internet
+
+1. Let Postfix know about the domains that it should consider local:
+    
+    ```sh
+    sudo postconf -e "mydestination = gitlab.example.com, localhost.localdomain, localhost"
+    ```
+
+1. Let Postfix know about the IPs that it should consider part of the LAN:
+    
+    We'll assume `192.168.1.0/24` is your local LAN. You can safely skip this step if you don't have other machines in the same local network.
+    
+    ```sh
+    sudo postconf -e "mynetworks = 127.0.0.0/8, 192.168.1.0/24"
+    ```
+
+1. Configure Postfix to receive mail on all interfaces, which includes the internet:
+    
+    ```sh
+    sudo postconf -e "inet_interfaces = all"
+    ```
+
+1. Configure Postfix to use the `+` delimiter for sub-addressing:
+    
+    ```sh
+    sudo postconf -e "recipient_delimiter = +"
+    ```
+
+1. Restart Postfix:
+    
+    ```sh
+    sudo service postfix restart
+    ```
+
+## Test the final setup
+
+1. Test SMTP under the new setup:
+    
+    1. Connect to the SMTP server:
+        
+        ```sh
+        telnet gitlab.example.com 25
+        ```
+
+        You should see a prompt like this:
+
+        ```sh
+        Trying 123.123.123.123...
+        Connected to gitlab.example.com.
+        Escape character is '^]'.
+        220 gitlab.example.com ESMTP Postfix (Ubuntu)
+        ```
+
+        If you get a `Connection refused` error instead, make sure your firewall is setup to allow inbound traffic on port 25.
+
+    1. Send the `replies` user a dummy email to test SMTP, by entering the following into the SMTP prompt:
+        
+        ```
+        ehlo gitlab.example.com
+        mail from: root@gitlab.example.com
+        rcpt to: replies@gitlab.example.com
+        data
+        Subject: Re: Some issue
+
+        Sounds good!
+        .
+        quit
+        ```
+
+        (Note: The `.` is a literal period on its own line)
+
+    1. Check if the `replies` user received the email:
+    
+        ```sh
+        su - replies
+        MAIL=/home/replies/Maildir
+        mail
+        ```
+
+        You should see output like this:
+
+        ```
+        "/home/replies/Maildir": 1 message 1 unread
+        >U   1 root@gitlab.example.com                           59/2842  Re: Some issue
+        ```
+
+        Quit the mail app:
+
+        ```sh
+        q
+        ```
+
+    1. Log out of the `replies` account and go back to being `root`:
+
+        ```sh
+        logout
+        ```
+
+1. Test IMAP under the new setup:
+    
+    1. Connect to the IMAP server:
+        
+        ```sh
+        telnet gitlab.example.com 143
+        ```
+
+        You should see a prompt like this:
+
+        ```sh
+        Trying 123.123.123.123...
+        Connected to mail.example.gitlab.com.
+        Escape character is '^]'.
+        - OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION] Courier-IMAP ready. Copyright 1998-2011 Double Precision, Inc.  See COPYING for distribution information.
+        ```
+
+    1. Sign in as the `replies` user to test IMAP, by entering the following into the IMAP prompt:
+
+        ```
+        a login replies PASSWORD
+        ```
+
+        Replace PASSWORD with the password you set on the `replies` user earlier.
+
+        You should see output like this:
+
+        ```
+        a OK LOGIN Ok.
+        ```
+
+    1. Disconnect from the IMAP server:
+
+        ```sh
+        a logout
+        ```
+
+## Done!
+
+If all the tests were successfull, Postfix is all set up and ready to receive email! Continue with the [Reply by email](./README.md) guide to configure GitLab.
+
+---------
+
+_This document was adapted from https://help.ubuntu.com/community/PostfixBasicSetupHowto, by contributors to the Ubuntu documentation wiki._

lib/api/merge_requests.rb

@@ -55,7 +55,7 @@ module API
           else merge_requests
           end
-        merge_requests.reorder(issuable_order_by => issuable_sort)
+        merge_requests = merge_requests.reorder(issuable_order_by => issuable_sort)
         present paginate(merge_requests), with: Entities::MergeRequest
       end

spec/requests/api/merge_requests_spec.rb

@@ -2,11 +2,12 @@ require "spec_helper"
 describe API::API, api: true  do
   include ApiHelpers
+  let(:base_time) { Time.now }
   let(:user) { create(:user) }
   let!(:project) {create(:project, creator_id: user.id, namespace: user.namespace) }
-  let!(:merge_request) { create(:merge_request, :simple, author: user, assignee: user, source_project: project, target_project: project, title: "Test") }
-  let!(:merge_request_closed) { create(:merge_request, state: "closed", author: user, assignee: user, source_project: project, target_project: project, title: "Closed test") }
-  let!(:merge_request_merged) { create(:merge_request, state: "merged", author: user, assignee: user, source_project: project, target_project: project, title: "Merged test") }
+  let!(:merge_request) { create(:merge_request, :simple, author: user, assignee: user, source_project: project, target_project: project, title: "Test", created_at: base_time) }
+  let!(:merge_request_closed) { create(:merge_request, state: "closed", author: user, assignee: user, source_project: project, target_project: project, title: "Closed test", created_at: base_time + 1.seconds) }
+  let!(:merge_request_merged) { create(:merge_request, state: "merged", author: user, assignee: user, source_project: project, target_project: project, title: "Merged test", created_at: base_time + 2.seconds) }
   let!(:note) { create(:note_on_merge_request, author: user, project: project, noteable: merge_request, note: "a comment on a MR") }
   let!(:note2) { create(:note_on_merge_request, author: user, project: project, noteable: merge_request, note: "another comment on a MR") }
@@ -74,8 +75,8 @@ describe API::API, api: true  do
           expect(response.status).to eq(200)
           expect(json_response).to be_an Array
           expect(json_response.length).to eq(3)
-          expect(json_response.last['id']).to eq(@mr_earlier.id)
-          expect(json_response.first['id']).to eq(@mr_later.id)
+          response_dates = json_response.map{ |merge_request| merge_request['created_at'] }
+          expect(response_dates).to eq(response_dates.sort)
         end
         it "should return an array of merge_requests in descending order" do
@@ -83,8 +84,8 @@ describe API::API, api: true  do
           expect(response.status).to eq(200)
           expect(json_response).to be_an Array
           expect(json_response.length).to eq(3)
-          expect(json_response.first['id']).to eq(@mr_later.id)
-          expect(json_response.last['id']).to eq(@mr_earlier.id)
+          response_dates = json_response.map{ |merge_request| merge_request['created_at'] }
+          expect(response_dates).to eq(response_dates.sort.reverse)
         end
         it "should return an array of merge_requests ordered by updated_at" do
@@ -92,17 +93,17 @@ describe API::API, api: true  do
           expect(response.status).to eq(200)
           expect(json_response).to be_an Array
           expect(json_response.length).to eq(3)
-          expect(json_response.last['id']).to eq(@mr_earlier.id)
-          expect(json_response.first['id']).to eq(@mr_later.id)
+          response_dates = json_response.map{ |merge_request| merge_request['updated_at'] }
+          expect(response_dates).to eq(response_dates.sort.reverse)
         end
         it "should return an array of merge_requests ordered by created_at" do
-          get api("/projects/#{project.id}/merge_requests?sort=created_at", user)
+          get api("/projects/#{project.id}/merge_requests?order_by=created_at&sort=asc", user)
           expect(response.status).to eq(200)
           expect(json_response).to be_an Array
           expect(json_response.length).to eq(3)
-          expect(json_response.last['id']).to eq(@mr_earlier.id)
-          expect(json_response.first['id']).to eq(@mr_later.id)
+          response_dates = json_response.map{ |merge_request| merge_request['created_at'] }
+          expect(response_dates).to eq(response_dates.sort)
         end
       end
     end