Import JIRA project issues into GitLab project issues
Description
- Import issues from a JIRA project into a GitLab project.
- This feature is only available (as a button, or whatever UI), once you have already set up our existing JIRA project service integration. So that way, we don't have to mess with any weird project set up design / tech things in this issue. The assumption is that the integration is already there and we are building on our existing base.
- So for a qualified GitLab project, anytime you can go into some menu, click a button (or some UI), and then it just dumps a copy of all the JIRA issues into GitLab issues. Again it is stateless and not idempotent.
- There is some transactional process to copy all issues from the JIRA project and create new corresponding issues into the GitLab project.
- If the GitLab project has any existing issues, they are unchanged in this process.
- If there is any error, everything is rolled back, so that you cannot have partial copies after the process.
- The process is not idempotent. So if you run the process twice immediately, you will have two copies of the JIRA issues in the GitLab project.
- There is no state maintained. After you have done the import, both JIRA and GitLab are not aware that it was done. Certainly there is no linkage or synching between issues in the two systems.
- For each JIRA issue in the JIRA project, create a corresponding issue in the GitLab project, with these mappings:
- JIRA issue summary -> GitLab issue title
- JIRA issue description -> GitLab issue description
- Simplified text handling/conversion
- GitLab issue author := GitLab user who activates this import process.
- All other fields and attributes in the GitLab issue are set to as if it was created in the GitLab web UI with nothing beyond the above selected. E.g. the issue is in open state and there are no labels associated.
Original proposal
As a user, I want to import existing issues from JIRA to GitLab so that I can switch from JIRA to GitLab as my ticketing system. The import would include:
- issue title
- issue comments
- current issue state (if not all states through the issue's history)
- attachements
- current assignee (if not all assignees through the issue's history)
Activity
@csilver Thanks for the report! I've edited the description to fall in line with the feature proposal template in our contribution guidelines
Related issue: https://gitlab.com/gitlab-org/gitlab-ce/issues/23530
@JobV I think it would be good to prioritize this.
We successfully switched from Jira to GitLab Issues (EE) yesterday (Yay!), so we thought we'd share this tip until this is more properly solved in GitLab.
We used https://gist.github.com/florisb/1266d3584dbfdbd2a8a55d31e2518edd as a basis and with some trivial adjustments to work with Python 3 as well as some project specific adjustments we said goodbye to Jira for this particular project.
It successfully handled
- Create issue
- Title
- Description
- Reporter
- Comments with correct author
- Attachments
- Manual username mapping for reporter and comment author (we had different usernames on Jira compared to GitLab..)
- Custom query to only extract the issues we wanted to move
It did not handle
- Issue type to GitLab Issue Label
- Long titles (GitLabs title limit is less than Jira's)
- Jira format to markdown (you'll need to search/replace stuff like {noformat)}
- Assignee
- Issue links
- Watchers
- Voters
- Components
- Labels
- Resolution
- Status
- Case where issue already existed (due to multiple runs, crashes, ..)
- Missing username in GitLab (we used a dummy GitLab user for that purpose along with a note about who the original reporter/author was)
, but still for our purpose with this project that was acceptable since we only cared about moving unresolved issues and we could quite easily label the issues afterwards based on search filters.
Obviously not a one-size-fits-all, but we're still happy with an acceptable and rather effortless switch!
This has been on my mind for a while. @victorwu can you spec it out and see if we can schedule it?
@mydigitalself : Heads up. I'll be taking a look at this and digging a bit. I'll ping you later and we can talk about any platform/integration ideas you might have.
@smcgivern : Could you comment on the technical feasibility of this at this point? Since we already have JIRA integration, I hope this wouldn't be totally brand new from scratch work?
I tried to scope it to a very minimal MVP. Do take a look and comment if anything can be further simplified.
Pending some work on my part to understand users between the two system for copying author + assignee. In the worse case, we could just default the GitLab issue author to be one dedicated one (whoever activated the import).
@dbalexandre what do you think of the description? Would it be possible to have an 'importer' like the GH and Bb ones work on an existing project, and do what the description describes?
@adamniedzielski wdyt about the idea in general? I think our JIRA project service, when correctly configured, already has the credentials needed for this.
As far as I understand, all our importers operate before a project was created. To reuse our JIRA project service we have to have a project persisted in the database. We can solve this in two different ways:
- only allow to import issues from JIRA once, when project is created and require configuring JIRA service as a step in this process
- find a reasonable place in our project settings for "Import issues from JIRA" button and allow to do that multiple times
I think that I'm more in favour of option 2. because it allows you to import code from GitHub and issues from JIRA (imports are non-exclusive).
I didn't look into JIRA API at all. It should be possible considering the operations that we do now, but @felipe_artur knows better.
The proposal has a very limited scope - I like it
One side note. When working with our users using JIRA integration I noticed that often they have multiple GitLab projects connected to one JIRA project (think: one product, but frontend and backend are in two separate repositories). I don't know if we can do anything better with that, but maybe somebody has an idea.
find a reasonable place in our project settings for "Import issues from JIRA" button and allow to do that multiple times
I think this would ideally be able to use the existing JIRA service, but then that's a bit weird: if you import, you don't want the service active any more, because you've copied the issues through to GitLab
When working with our users using JIRA integration I noticed that often they have multiple GitLab projects connected to one JIRA project (think: one product, but frontend and backend are in two separate repositories).
Great point! I also don't have a particularly strong suggestion here.
@smcgivern The description sounds good
I think that I'm more in favour of option 2. because it allows you to import code from GitHub and issues from JIRA (imports are non-exclusive).
I liked the second option that @adamniedzielski suggested. We should make the importer idempotent, so users can continuously import issues from Jira before moving to GitLab definitively.
Thank you @smcgivern @dbalexandre @adamniedzielski .
I'd like to scope this first iteration to be as small as possible, even if at first it appears "unusable". One strategy is to get that really really small scope as this issue, and then we can create separate issues to address the obvious problems. Scheduling-wise, we can then make this first issue as gitlab-ee
992791 in a milestone, and the follow-ups as gitlab-ee992792 for that very same milestone. So if we could just merge the gitlab-ee~992791 into master first, then the rest will be sweet delicious icing.I really like the concept you folks mentioned with:
find a reasonable place in our project settings for "Import issues from JIRA" button and allow to do that multiple times
So how about something like this:
- This feature is only available (as a button, or whatever UI), once you have already set up our existing JIRA project service integration. So that way, we don't have to mess with any weird project set up design / tech things in this issue. The assumption is that the integration is already there and we are building on our existing base. I love this.
- So for a qualified GitLab project, anytime you can go into some menu, click a button, and then it just dumps a copy of all the JIRA issues into GitLab issues. Again it is stateless and not idempotent.
- I think the transactional requirement still holds. But I'm open to simplifying that even further if you folks think it's not worth it.
So quick improvements that we could make gitlab-ee~992792 :
- Solve the problem of copying the user fields, i.e. the assignee and author fields.
- Solve the problem of mapping the crazy configurable JIRA issue statues into GitLab issue states (open, closed) and labels, etc.
- Solve other fields.
- Solve the problem of GitLab projects that are not already configured and talking to a JIRA project.
- Solve the idempotency problem. Introduce some state concept.
- Provide some holistic solution building on the above mechanisms to effectively migrate JIRA issues to GitLab issues. Support intermediary states, deprecation periods, etc. etc. so that you can have a smooth transition over time versus a big bang approach.
@victorwu thanks!
it just dumps a copy of all the JIRA issues into GitLab issues
Given that we aren't mapping state, assignee, author, etc. - is this just title and description?
Solve the problem of mapping the crazy configurable JIRA issue statues into GitLab issue states (open, closed) and labels, etc.
From what I remember of JIRA, it had a very sensible approach to things like this (because even different projects in the same JIRA instance can have different workflows), where it would present you with a list of the states on each 'side' of the transition, and ask you to map them. That does introduce some state, or mean a wizard-type approach, though.
@oyvindhorneland @Martinus can you comment here?
@oyvindhorneland it seems from https://gitlab.com/gitlab-org/gitlab-ee/issues/1871#note_25012385 that this very limited first iteration would have done almost everything that you needed, although not everything that was possible. (The main miss being user mappings.) Is that accurate?
Given that we aren't mapping state, assignee, author, etc. - is this just title and description?
Yes, just title and description, and also I assume the GitLab issue author needs to be filled in since I assume it's a required field, or GitLab might behave weird if we don't include it? So that can be just the person who clicked the import button.
@victorwu yes, that was my assumption - there would be an author, but it wouldn't be the right author.
Great thanks @smcgivern . Let's let this stew until 9.2 earliest.
@oyvindhorneland it seems from https://gitlab.com/gitlab-org/gitlab-ee/issues/1871#note_25012385 that this very limited first iteration would have done almost everything that you needed, although not everything that was possible. (The main miss being user mappings.) Is that accurate?
@smcgivern This very limited first iteration would, compared to the one I mentioned, lack some features I needed:
- User mappings
- Comments
- Attachments
- Ability to limit exported issues to a Jira Query
- Customizing generated title and description
User mappings
A hoperfully rather easy feature here would be to look if the Jira username was found in GitLab and only use the "default user" if not. I would expect that this would work for many.
Advanced user mapping support where username has changed (like in our scenario) is not an MVP thing, but allowing a user defined script to lookup this mapping would be flexible and nice down the road .
While using the current user as the default user on issues is probably fine as a first (not sure that's really all that "viable", though), support for a "we did not find the user" dummy user, or a message detailing the original user, would be nice.
For our import I extracted original display name and username and added that on top of both description, comments and attachment-info as a comment. So even if the user was not present/mapped in GitLab the information was still there about the original author.
Comments
In my opinion quite essential to handle this during import. Comments will often contain key information. Easy to retrieve using the Jira API, see example in my original comment.
Attachments
Also quite essential. Attachments relevant to the issue must be imported or at least linked to. Easy to retrieve using the Jira API, see example in my original comment.
Ability to limit exported issues to a Jira Query
Jira will probably often contain thousands of issues depending on the project. In our case we had several thousands of closed issues and significantly less open issues that we actually needed to transfer.
While also moving closed issues could be interesting, allowing a Jira query expression instead of just fetching all the issues would likely not increase complexity much and it would also allow to only move or test move some issues to see how things work.
Customizing generated title and description
Probably not a MVP thing, but the Jira title and description does not contain the jira issue key (PROJECT_KEY-). This is useful information since this key might have been registered or communicated already. You can change the title and description, but the key will stay the same.
The key might also make it easier to handle duplicates during import.
I modified the title and description and added the Jira issue key there to allow the users to search for the issues if they already knew the Jira issue key.
- Title: ISSUE_KEY - TITLE
- Description:
https://JIRA_HOST/browse/ISSUE_KEY Issue Type (from Jira): Bug (example) Priority (from Jira): Major (example) Reporter: NAMEThanks @oyvindhorneland!
@adamniedzielski Yes it is!
We should make the importer idempotent, so users can continuously import issues from Jira before moving to GitLab definitively.
Somehow cannot change milestone for this issue.
Moving to https://gitlab.com/gitlab-org/gitlab-ee/issues/1939
@Martinus : If an issue is in the EE project, it will be released in EE. And we have additional labels for gitlab-ee
1228034 and gitlab-ee333913 .If it is in the CE project, it will be released in CE.
On occasion we move issues from one project to another, reflecting our changing decisions. The current location of an issue (which project it is in) reflects the current decision.
So for this issue, it would be in gitlab-ee~1228034 if we were to schedule and work on it.
Edited by Victor WuI see another related need when migrating issues from JIRA/Fisheye to Gitlab. This is about mentioning issues in project's commit message : In JIRA, you can specify a commit message like "might fix #PJ-1234" and then the commit reference appear in JIRA's issue details.
You can imagine that kind of information is very valuable.
An approach would be to provide a way to create a gitlab issue with a specified ID. Then search/replace in the git repo for all commit messages s/#PJ-1234/#1234 would be enough.
What do you think ?
PS: I'm writing this as a gitlab community edition user ;)
mentioned in merge request gitlab-com/www-gitlab-com!6938 (merged)
mentioned in merge request gitlab-com/www-gitlab-com!7213 (merged)
mentioned in issue #1444 (closed)
We couldn't wait and wrote (yet another) CLI for that : https://gitlab.com/smallstack/jira2gitlab
In comparison to other solutions this supports (custom-)field mapping which was a requirement for us!
