Support of importing Android git repositories managed by Android repo

When we have the Nested Groups available, we'll be available to support Android Open Source Project (AOSP) repositories at URLs like below:

https://android.googlesource.com/device/asus/deb/

https://android.googlesource.com/device/asus/flo/

Could we have the import tool support importing git repositories with above nested groups?

Alternatively could we support to import all the required git repositories based on the Android repo manifest file, for example, the default one at https://android.googlesource.com/platform/manifest ?

References: https://source.android.com/source/developing.html https://gerrit.googlesource.com/git-repo/+/master/docs/manifest-format.txt

/cc @xyzzy

added ~874211 feature proposal labels

/cc: @smcgivern

I think the existing import tool belongs more to Platform than Discussion, so I'll let @DouweM take a look at this.

First thoughts: I didn't even know we had this! Is it something we want to continue supporting? (If people use it frequently, then I guess we do need to do so!)

The way it works now conflicts with https://gitlab.com/gitlab-org/gitlab-ce/issues/28283, which we can fix, but that depends on the answer to my question 🙂

added ~480950 import labels

@xiaogang_gitlab Are you suggesting to have the import tool support importing nested groups? That's not directly related to Gerrit/AOSP as far as I can see. If it currently doesn't, that sounds like a bug we should fix. /cc @dzaporozhets

I think it would also be interesting to build an actual Gerrit importer tool in the GitLab UI that understands that manifest format and imports all the projects into a specific group, but that's obviously more complex :)

@DouweM Yes for the import tool to import from the repo storage into the Gitlab and present the multi-layer paths in the way as Google like https://android.googlesource.com. Since it remains the same structure, there's almost no need for end users to update their script.

Yes it would be even better to have the capability to parse the manifest files.

At current state import from URL does not support URL parsing with automatic creating of groups hierarchy. I think we can make it work with manifest file. Its not necessary related to nested groups as it should be able to work with flat and deep nested structure. I believe main benefit here is providing one file to GitLab and have GitLab creating all necessary groups, nested groups and projects. /cc @DouweM

@dzaporozhets The request as written seems to be about the existing https://docs.gitlab.com/ee/raketasks/import.html tool, which looks at the dirs in repositories and automatically creates groups/projects for them. I don't think that currently works with nested groups.

added subgroups label

@DouweM I can confirm that it does not import existing repositories with nested subdirectories. We are running a trial right now. Here is the message I sent to our trial contact regarding this:

Importing an existing project with nested directories using gitlab-rake gitlab:import:repos does not create subgroups or move imported repositories into subgroups even if the entire group structure was created in the GUI before the import. I tried importing both with no existing groups and also importing after I manually created nested groups to match our structure. In both cases the imports failed with the message " * Failed trying to create group top_group/subgroup"

@DougM-Saltillo Thank you for commenting.

The request as written seems to be about the existing https://docs.gitlab.com/ee/raketasks/import.html tool, which looks at the dirs in repositories and automatically creates groups/projects for them. I don't think that currently works with nested groups

@DouweM ok then we should implement/fix it. Probably should be a separate issue describing exact problem

Has there been any development on this?

moved to gitlab#8546

Hey! 👋

GitLab is moving all development for both GitLab Community Edition and Enterprise Edition into a single codebase. The current gitlab-ce repository will become a read-only mirror, without any proprietary code. All development is moved to the current gitlab-ee repository, which we will rename to just gitlab in the coming weeks. As part of this migration, issues will be moved to the current gitlab-ee project.

If you have any questions about all of this, please ask them in our dedicated FAQ issue.

https://gitlab.com/gitlab-org/gitlab-ee/issues/13855

You can also find more information here:

• https://gitlab.com/gitlab-org/gitlab-ee/issues/13304
• https://about.gitlab.com/2019/08/23/a-single-codebase-for-gitlab-community-and-enterprise-edition/

Frequently Asked Questions

For an up to date list of questions and answers, please take a look at https://gitlab.com/gitlab-org/gitlab-ee/issues/13855

What will we do with the repository names?

https://gitlab.com/gitlab-org/gitlab-ee/ will become https://gitlab.com/gitlab-org/gitlab, and https://gitlab.com/gitlab-org/gitlab-ce will become https://gitlab.com/gitlab-org/gitlab-foss.

Why rename gitlab-ce to gitlab-foss?

Using "gitlab" and "gitlab-ce" would be confusing, so we decided to rename gitlab-ce to gitlab-foss to make the purpose of this FOSS repository more clear

I created a merge requests for CE, and this got closed. What do I need to do?

You will need to create a merge request at https://gitlab.com/gitlab-org/gitlab-ee/. This link will eventually redirect to https://gitlab.com/gitlab-org/gitlab.

How does the licensing work in this new setup?

Everything in the ee/ directory is proprietary. Everything else is free and open source software. If your merge request does not change anything in the ee/ directory, the process of contributing changes is the same as when using the gitlab-ce repository.

Will you accept merge requests on the gitlab-ce/gitlab-foss project after it has been renamed?

No. Merge requests submitted to this project will be closed automatically.

Will I still be able to view old issues and merge requests in gitlab-ce/gitlab-foss?

Yes.

How will this affect users of GitLab CE using Omnibus?

No changes will be necessary, as the packages built remain the same.

How will this affect users of GitLab CE that build from source?

Once the project has been renamed, you will need to change your Git remotes to use this new URL. GitLab will take care of redirecting Git operations so there is no hard deadline, but we recommend doing this as soon as the projects have been renamed.

Where can I see a timeline of the remaining steps?

https://gitlab.com/gitlab-org/gitlab-ee/issues/13304

Will community contributions submitted to the new "gitlab" repository be available in the new gitlab-foss repository?

Yes, these changes will be synced to gitlab-foss automatically, several times per day.

locked this issue