Resolve "Ref's interface is inconsistent with Rugged's API"

This branch is based off !130 so tests can pass successfully, so that will need to be merged first and this rebased from master afterwards. This will give a better syntax to https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6532

Closes #30

See merge request !131

!103 introduced an optimization where a Refs target would point to the
dereferenced object, which saved us a lot operations, but became
inconsistent with Rugged's API, where target would always reference the
immediate object pointed at. These changes add a new property,
dereference_target to be consistent with Rugged but still be able to get
the target efficiently.

Revert to previous diff pruning behavior

Before 10.6.9, if a rugged diff met the conditions for being too-large
(which obiously meant it was also collapsable), too-large would take
precedence. This reverts the behavior back to that, since gitlab-ce
depends on this

Closes #31

See merge request !132

Before 10.6.9, if a rugged diff met the conditions for being too-large
(which obiously meant it was also collapsable), too-large would take
precedence. This reverts the behavior back to that, since gitlab-ce
depends on this

Fix specs after change to gitlab-test to account for new branch

Also, when verifying the last commit on master search for the branch by name, instead of counting on it being the last branch, to be more resilient to future changes on gitlab-test

Closes #29

See merge request !130

Ignore invalid encoding when parsing attributes

See https://gitlab.com/gitlab-org/gitlab_git/issues/28 for the discussion leading up to these changes.

See merge request !129

This prevents us from raising a hard error when parsing Git attribute
files that are encoded using an invalid/unsupported encoding. This in
turn means a GitLab user won't be presented with an HTTP 500 error.

Fixes #28

Optimize diff creation from Rugged::Patch

Our diff threshold for pruning is fairly small, but we used to iterate over *each* line, even of a 100mb patch, just to check these small thresholds. Now we stop when any of the limits is met.

This should fix the issue reported in https://gitlab.com/gitlab-org/gitlab-ce/issues/2529

See merge request !128

Add bundle-audit gem and CI task

Also correct `source` in Gemfile to use HTTPS.

See merge request !127

Our diff threshold for pruning is fairly small, but we used to iterate
over *each* line, even of a 100mb patch, just to check these small
thresholds. Now we stop when any of the limits is met.

Also correct `source` in Gemfile to use HTTPS.

Fix invalid `be_valid_commit` matcher

This matcher could have theoretically evaluated to the following:

```ruby
true
false
true
true
```

...and never would have caused a failure. Indeed, it should have been
failing for quite a while.

See merge request !126

This matcher could have theoretically evaluated to the following:

```ruby
true
false
true
true
```

...and never would have caused a failure. Indeed, it should have been
failing for quite a while.

Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

Straight diffs support

Based on https://gitlab.com/gitlab-org/gitlab_git/merge_requests/41. 

All credentials to @ben.boeckel 

See merge request !125

Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

Signed-off-by: Dmitriy Zaporozhets <dmitriy.zaporozhets@gmail.com>

Currently, only `git diff A...B` is supported. Add a `straight` argument
for direct diff and commit results.

Fix broken repo error raised after requesting a commit

Prevents a 500 error on screen after `Rugged::RepositoryError: Could not find repository from '/var/opt/gitlab/git-data/repositories/user/project.git'`, which could happen when checking for the last commit in the list of projects (dashboard).

Related https://gitlab.com/gitlab-org/gitlab-ce/issues/20501

See merge request !124

Also added relevant spec with broken repo.

This adds Git attribute parser support for file paths that don't contain
any attributes.

gives same permissions to file even if the file was renamed

Used to fix https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/5979

See merge request !118

Mark blobs as binary whenever this is known

This removes the need for using `Linguist::BlobHelper#binary?` whenever the binary status is already known, in turn reducing loading times of https://gitlab.com/nrclark/dummy_project/commit/81ebdea5df2fb42e59257cb3eaad671a5c53ca36 (by about 2-ish seconds locally).

See merge request !123

Previously we would rely on Linguist::BlobHelper to determine if a blob
was binary or not. Since Rugged knows if a blob is binary we can instead
just inherit this information and fall back to BlobHelper if the binary
flag wasn't set explicitly.

When testing this with https://gitlab.com/nrclark/dummy_project/commit/81ebdea5df2fb42e59257cb3eaad671a5c53ca36
it reduces loading times (locally) by around 2 seconds.

Improve handling of large diffs

This MR adjusts the way checking for large diffs takes place. Prior to this MR the procedure was basically as follows:

1. Iterate over every diff in a collection
2. Just load the entire diff into memory, why not
3. Check if the resulting content _including_ any diff markers/meta data exceed a threshold
4. Prune or collapse the diff

This MR changes things around so the procedure is instead as follows:

1. Iterate over every diff in a collection
2. Check if the data modified (excluding diff markers) is larger than a threshold
3. If this is not the case, proceed as usual. if this _is_ the case we'll prune/collapse the diff

See merge request !122

Prior to this commit the DiffCollection class was responsible for
checking if a diff had to be collapsed or was too large to be displayed
altogether.

This commit changes both DiffCollection and Diff so that Diff itself
checks if its too large or has to be collapsed. These checks happen when
the Diff is being initialised. The patch size is based on the size of
every line in every hunk of the diff, instead of relying on the diff as
a string including diff markers.

DiffCollection still has an extra check to collapse diffs when it has
iterated over too many files. Since this is unrelated to the actual
sizes this has been kept as-is.

For binary files no pruning takes place as the diffs for these files are
not displayed. In the past the size of a diff was reported based on the
diff's size (including metadata). If we were to use the actual file's
size a diff would be marked as being too large and in the case of an
image would never be displayed.

When the "binary" option is set to true the "diff" option is to be set
to false automatically.

Git attributes can be defined using an absolute path. This commit fixes
the matching procedure so that attributes and paths with leading slashes
are supported properly.

Previously using Gitlab::Git::Attributes when $GIT_DIR/info/attributes
didn't exist would result in a runtime error. This commit fixes this so
an empty Hash is produced instead.