When importing gitlab thinks no files exist

Created by: joeandaverde

After importing a non-empty project GitLab is suggesting for me to add new files to the repo:

Here are the steps I performed:

vagrant@gitlab:/home/git$ sudo su git
git@gitlab:~$ cd repositories/
git@gitlab:~/repositories$ ls
joseph.andaverde
git@gitlab:~/repositories$ git clone --bare http://ssi-vcs/engineering/illuminate/clinical.demo/
Cloning into bare repository 'clinical.demo.git'...
git@gitlab:~/repositories$ ls
clinical.demo.git  joseph.andaverde
git@gitlab:~/repositories$ cd ../gitlab/
git@gitlab:~/gitlab$ bundle exec rake gitlab:import:repos RAILS_ENV=production
Processing clinical.demo.git
 * Created clinical.demo (clinical.demo.git)
Done!
git@gitlab:~/gitlab$

Then when visiting the newly created project I'm presented with this:

Created by: Razer6

Maybe related to #3693 (closed)?

By Administrator on 2013-05-13T19:23:37 (imported from GitLab project)

By Administrator on 2013-05-13T19:23:37 (imported from GitLab)

Created by: scross01

When doing an import gitlab doesn't seem to like repositories with a period in the name, the period gets changed to a dash, so clinical.demo becomes clinical-demo

What worked for me was to replace the new repository directory created by the import with the original import directory, something along the lines of:

cd repositories
rm -Rf joseph.andaverde/clinical-demo.git
mv clinical.demo.git joseph.andaverde/clinical-demo.git

Similar issue also occurs if the imported repository uses capital letters, which get changed to lowercase by gitlab import.

By Administrator on 2013-05-13T19:45:15 (imported from GitLab project)

By Administrator on 2013-05-13T19:45:15 (imported from GitLab)

Created by: joeandaverde

I cloned the repo with the period in the name to a repo without a period (demo.git instead of clinical.demo.git).

After importing this the problem persisted. I removed the imported repo (joseph.andaverde/demo.git) and replaced it with the direct clone I made in repositories/. This resolved the issue now the files are showing up.

The problem seems to be with how the files that are copied to the group ("joseph.andaverde" in this case) directory.

By Administrator on 2013-05-13T20:01:13 (imported from GitLab project)

By Administrator on 2013-05-13T20:01:13 (imported from GitLab)

Created by: joeandaverde

It's worth noting that repos that do not end in .git don't get imported.

By Administrator on 2013-05-13T20:01:56 (imported from GitLab project)

By Administrator on 2013-05-13T20:01:56 (imported from GitLab)

Created by: joeandaverde

I think I've found the main cause of the problem.

When performing the steps above (cloning at command line to the repositories directory) and then executing the bundle rake here are the steps the rake task takes.

Step 1: Creates a new project WITHOUT an import url (import.rake -> project_params). This import url must be an http url (it should actually be the file system path to the cloned directory in this case) https://github.com/gitlabhq/gitlabhq/blob/master/lib/tasks/gitlab/import.rake#L40-#L46

project_params = {
  name: path,
}

Step 2: The project is considered not valid in create_context.rb right before trying to do shell.import_repository so it never tries to import the repo from the directory. If this was able to import from the directory the files would be present. https://github.com/gitlabhq/gitlabhq/blob/master/app/contexts/projects/create_context.rb#L49-#L58

# Import project from cloneable resource
if @project.valid? && @project.import_url.present?
  shell = Gitlab::Shell.new
  if shell.import_repository(@project.path_with_namespace, @project.import_url)
    # We should create satellite for imported repo
    @project.satellite.create unless @project.satellite.exists?
    true
  else
    @project.errors.add(:import_url, 'cannot clone repo')
  end
end

Step 3: The project observer fires asynchronously and creates a bare project under the first admin namespace https://github.com/gitlabhq/gitlabhq/blob/master/app/observers/project_observer.rb#L4-#L8

def after_create(project)
  unless project.forked?
    GitlabShellWorker.perform_async(
        :add_repository,
        project.path_with_namespace
    )

    log_info("#{project.owner.name} created a new project \"#{project.name_with_namespace}\"")
  end
end

The third step above is the reason why the repository is empty when importing from the file system.

By Administrator on 2013-05-15T04:30:26 (imported from GitLab project)

By Administrator on 2013-05-15T04:30:26 (imported from GitLab)

Created by: bbodenmiller

Does this issue still exist on the latest master? Thanks for the issue report. Please reformat your issue to conform to the issue tracker guidelines found in our contributing guidelines.

By Administrator on 2013-06-15T08:26:46 (imported from GitLab project)

By Administrator on 2013-06-15T08:26:46 (imported from GitLab)

Created by: petergus

I'm running on branch 5.4 and there was some change in the code to reflect 2e8db0df however it is still not working for me.

By Administrator on 2013-08-12T17:05:40 (imported from GitLab project)

By Administrator on 2013-08-12T17:05:40 (imported from GitLab)

Created by: fleuryc

+1

By Administrator on 2013-09-25T13:45:20 (imported from GitLab project)

By Administrator on 2013-09-25T13:45:20 (imported from GitLab)

Created by: jvanbaarsen

@joeandaverde Can you confirm if this issue still exists?

By Administrator on 2014-01-09T10:24:51 (imported from GitLab project)

By Administrator on 2014-01-09T10:24:51 (imported from GitLab)

Created by: jvanbaarsen

It's been at least 2 weeks (and a new release) since we heard from you. I'm closing this issue but if you still experience this problem, please open a new issue (but also reference the old issue(s)). Make sure to also include the necessary debugging information conforming to the issue tracker guidelines found in our contributing guidelines. /cc @razer6

By Administrator on 2014-03-02T16:26:16 (imported from GitLab project)

By Administrator on 2014-03-02T16:26:16 (imported from GitLab)