Global code search

We discussed it with @sytses and @dzaporozhets a few months ago but I couldn't find an issue so I created it. It can be implemented using ES only.

/cc @JobV @balameb

---

Edit:

UI changes

• A new tab in the Search results view after searching through /.
• Display the complete path of the file, as shown below (quick wireframe ahead):

Added ~113221 label

Added feature proposal label

Added ~139512 label

+1 for this implementation. Our company heavily relying on gitlab infrastructure and it would be really nice to be able to search through multiple projects

@JobV are we able to schedule this feature?

This was requested by https://na34.salesforce.com/0066100000AFOMC

I think this makes sense to schedule @JobV

Added ~313021 label

It's about time that we start working on this. What is the weight @vsizov?

I updated the weight.

Since global search is based on ElasticSearch it will be an EE only feature, right? This is a bummer. I'd say that global search is an essential feature of a SCM system.

We are in the process of searching all of our repos by hand, and have an EE install. I would love this feature, but I am unsure about the integration with Elastic Search... WOuld the searching be farmed out to the elastic search server,m and presented inside of GITLAB? AT the end of the day, the speed isn't important to our use case - we would be satisfied with a batch based search...

@akaphenom we are in the same situation and recently have integrated Gitlab EE with Elasitcsearch (ELS). As far as we saw, every search query now is performed by ELS directly (farmed out to the elastic search server) and presented inside Gitlab. But further, we also discovered by doing some manual queries on the ELS index, that commits, file contents, etc. of all repositories are already stored in ELS. So as far as I understood (please correct me if I am wrong), all the data needed for global codes search is already in ELS, there is "just" missing the API, UI, etc. for global code search in Gitlab. As a temporary workaround I was initially thinking to create a simple external tool that queries Elasticsearch directly for achieving global code search, but of course, I would prefer to avoid that. @vsizov Do you already have a rough idea when this feature will be implemented? (weeks, months, years)?

@speer Yes, I think it will be in 8.12. And yes, it's a matter of UI changes mostly.

@vsizov can we change this milestone 8.12 if it is being planned for 8.12?

@JobV ^^^

@vsizov if you're confident you can work on this for 8.12 (cc @DouweM), I'm happy to do so.

BTW, apart from UI changes we need to make parent/child relations between commits/blobs and project. It's not that expensive, but it'll require some time because everything is placed outside of application in the gem gitlab-elastisearch-git and we need to do it configurable enough.

Milestone changed to %8.12

@vsizov You seemed pretty confident about this last time we spoke, so I set the milestone. If anything changes, report back in this issue!

Added ~481018 label

Reassigned to @vsizov

@vsizov @DouweM is UX needed here?

@jschatz1 I think, No

Updated the issue with the changes with need to make in the product.

@vsizov few questions about UI changes

A new tab in the Search results view after searching through /.

I am not sure I see new tab on screenshot

Display the complete path of the file, as shown below

We should visually separate project name from file path here. Maybe ask UX designer for help?

@dzaporozhets The new tab is simply the Code tab which didn't appear before when you were searching globally.

For the second point, I agree that UX could help figure this out. @jschatz1 ?

Yes, I will ask UI team to help when I'm done.

Yeah, ES is buggy. It's already two bugs I got while working with ES. Now I get this https://github.com/elastic/elasticsearch/issues/14999

Upgrade to 2.4.0 has helped but lots of time wasted because ES search query was crashing silently. The HTTP response was 200, only ES logs showed the error.

UPDATE: We will need to inform our users that ES should be updated.

The new tab is simply the Code tab which didn't appear before when you were searching globally.

@regisF @vsizov thanks. I was confused by screenshot because it has selected project in dropdown which means search is going into the project. And project search already has Code tab. I would expect it to look like

mentioned in merge request !699 (merged)

https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/699

Status changed to closed by merge request !699 (merged)

Mentioned in commit 574d75cb