Images cannot show when projects' path was changed

GitLab 7.8

When projects' path (or namespace) was changed, the previous attachments, such as images cannot be shown.

Error info:

Started GET "/uploads/devops/common/3c818558de/design_and_dev.PNG" for 127.0.0.1 at 2015-03-30 14:30:28 
+0800

Processing by Projects::UploadsController#show as HTML
  Parameters: {"id"=>"devops/common", "secret"=>"3c818558de", "filename"=>"design_and_dev.PNG"}Completed 500 Internal Server Error in 35ms

NoMethodError (undefined method `repository' for nil:NilClass):
  app/controllers/application_controller.rb:123:in `repository'
  lib/gitlab/middleware/static.rb:7:in `call'

I guess it is related to GitLab 7.8 feature that introduced "Fix access control and protection against XSS for note attachments and other uploads."

No. This is probably another bug. Is devops/common the new project name?

edit: let me rephrase that. It is probably a side effect of that change, but the root cause is different.
I assume the links don't get updated accordingly after the rename.
I'll test it when I have a bit time this week.

devops/common is the new project name.

the process is like this:

1. current gitlab version is 7.7
2. change the project path to devops/common
3. then upgrade gitlab to 7.8
4. then the bug shows

/cc @DouweM Can you take a look at this?
I'm not sure if we fixed it already in 7.9.
I'm a bit busy at the moment, so I can't do an analysis right now. :(

Nope, definitely not fixed and definitely a bug. Thanks a lot for reporting this @hutusi.

For uploads, the project path is used in three places:

• The actual project
• The uploads/:namespace/:project folder containing the files.
• The full https://gitlab.com/:namespace/:project/uploads/... URLs within comments

If the path is changed on the actual project, the files aren't moved and the URLs in the comments aren't edited.

We could fix the last one by not including the full URL in comments, and instead just uploads/:secret/:filename, and leave it to the Markdown parser to compute the absolute path, like it already does when referencing files from the repository. This way it'll always use the up-to-date project path.

For moving the upload files, we'd have to add this to the procedures for moving a project or namespace.

@haynes Do you have some other ideas or does this sound good to you?

@DouweM That sounds good to me :)

mentioned in issue #1539 (closed)

@DouweM I just saw your branch related to moving a project and remembered this issue.
Do you know if it still exists?

@haynes Yep :( Haven't had time to look into it yet. Maybe I'm underestimating its urgency.

@DouweM hmmm it'll get urgent when a big customer tries to move his project.
I guess it would be ok not to have it in 7.13 as long as there is an emergency plan for such a situation.
But I think we shouldn't wait longer than 7.14 to fix this.

@haynes All right.

@JobV When do you propose to schedule this? It's not completely trivial to fix (see my comment)

@DouweM we don't have the time with the large amount of issues in 7.13 and the absence of some devs during 7.14: I propose 7.15.

mentioned in issue #1947 (closed)

@JobV All right. I've cross-posted this internally to make scheduling easier: https://dev.gitlab.org/gitlab/gitlabhq/issues/2487

Milestone changed to 7.15

mentioned in issue #1969 (closed)

mentioned in issue #1969 (closed)

mentioned in issue #2435 (moved)

mentioned in issue #2184 (closed)

mentioned in issue #1539 (closed)

mentioned in issue #2184 (closed)

As @sque has pointed out this issue has come up many times recently. Please make this a priority to fix sooner than later.

@DouweM did anything change for this issue?

@JobV Nothing has changed, still an issue.

Milestone changed to 8.1

Reassigned to @vsizov

https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/1521

mentioned in merge request !1521 (merged)

mentioned in merge request !1521 (merged)

Status changed to closed by commit 0b9273a2

mentioned in commit 0b9273a2

The merge request worked for new attachments added as the namespace is no longer added when attachments are uploaded e.g. what used to be (SITE_URL/namespace/uploads/SECRET/FILE_NAME) is now just (/uploads/SECRET/FILE_NAME) However, the fix does not correct previously added attachment path issues in issue/merge request descriptions or issue/merge request comments.

This worked for me to update the postgres DB. Make sure to backup first if anyone uses it. NO warranty This is also assuming that your uploads/attachments all were referenced to the same project that the issue/comments were posted to and that you didn't upload to one project and then paste the link in an another project as they would lose that context.

Connect to the db

sudo -u gitlab-psql /opt/gitlab/embedded/bin/psql -h /var/opt/gitlab/postgresql -d gitlabhq_production

Update the data to remove absolute paths to attachments

UPDATE issues SET description = regexp_replace(description, '\(http.*?(/uploads/.*?)\)', '(\1)', 'g'), created_at=created_at, updated_at=updated_at, deleted_at=deleted_at;

UPDATE merge_requests SET description=regexp_replace(description, '\(http.*?(/uploads/.*?)\)', '(\1)', 'g'), created_at=created_at, updated_at=updated_at, deleted_at=deleted_at, locked_at=locked_at;

UPDATE notes SET note=regexp_replace(note, '\(http.*?(/uploads/.*?)\)', '(\1)', 'g'), created_at=created_at, updated_at=updated_at;

Thanks @paulvisco That helped

@mipmip Oh you might hate me. I forgot to update this after I found an issue with any timestamps in the tables getting updated to the current time. I hope you worked from a backup.

I updated the above UPDATE statements now to include the timestamp fields.

Because I screwed it up on mine and had already starting adding new issues, etc I couldn't roll back to the backup. So I restored the database to a new instance, dumps those three tables. Then I created issues2, merge_requests2, and notes2 in the original system and imported the dumped data from the backups in.

Then I restored the timestamps from there. Something like... and then repeated for the other 2 tables.

UPDATE issues i, issues2 i2 SET i.created_at=i2.created_at, i.updated_at=i2.updated_at, i.deleted=i2.deleted WHERE i.id = i2.id;

Hope that helps.

I am using GitLab 9.2.0 27cff7db. We are still seeing this issue.

We migrated projects into Subgroups, so perhaps the migration into a subgroup is failing for attachments and uploaded images / files?

Should I open a new issue?

mentioned in issue #33703 (moved)

Since this may be new, I opened #33703 (moved).

mentioned in issue gitlab#10442