[meta] API v4

gitlab-org/gitlab-ce#17541 - Move /licenses api endpoint to /templates/licenses

https://gitlab.com/gitlab-org/gitlab-ce/issues/11810 - Remove deprecated merge_request endpoints in favour of merge_requests endpoints

https://gitlab.com/gitlab-org/gitlab-ce/issues/16834 - Update API endpoints for raw files

https://gitlab.com/gitlab-org/gitlab-ce/issues/14156 - Remove deprecated expires_at from ProjectSnippet API

mentioned in merge request !5392 (closed)

https://gitlab.com/gitlab-org/gitlab-ce/issues/14596 - Post should be put for block/unblock

https://gitlab.com/gitlab-org/gitlab-ce/issues/8099 - Paginate all API endpoints that return an array

https://gitlab.com/gitlab-org/gitlab-ce/issues/1746 - Group members endpoint is not paginated.

https://gitlab.com/gitlab-org/gitlab-ce/issues/20236 - Tags endpoint is not paginated

#20248 (closed) - No pagination on labels endpoint

!2397 - Use swagger for generated docs on the API

@DouweM Im pinging you here as we talked about this yesterday a bit.

mentioned in merge request !5469 (merged)

#20429 (closed) - Make all API DELETE endpoints return 204 and respect the If-Unmodified-Since header in GitLab 9.0 (we have @stanhu's approval on this one).

#20569 (closed) - Remove keys endpoints in favour of deploy_keys

#20584 (closed) - DELETE deploy_key doesn't use Entities::SSHKey

#20728 (moved) - Improve consistency in the way we retrieve project & group in API endpoints

#20732 (closed) - Always return 409 on POST /:sources/:id/members when member already exist

mentioned in merge request !5717 (merged)

gitlab-org/gitlab-ce#14492 - API endpoint /projects/fork/:id should move to /projects/:id/fork

mentioned in issue #11811 (closed)

cc @dzaporozhets as I closed #11811 (closed) to remove duplication.

@dzaporozhets Should we label this issue as promised too?

@zj thanks. Yes label this one too so we don't miss it at any chance

Added promised label

Can we deprecate the /session endpoint? cc @patricio

Also suggest that we enforce a standard way to add the namespace/project in each endpoint (e.g. gitlab-org/gitlab-ce instead of project ID 123456) so that we don't rely on encoding%2F everywhere.

#22007 (closed): In the current API, there are two possible ways of searching for projects:

GET /projects?search=hello (added in 2015 by 8689ce1e)
GET /projects/search/hello (added in 2013 by 4cfbfbb8)

I think the first endpoint is sufficient so we should probably schedule the removal of the latter for 9.0.

mentioned in issue #22042 (closed)

gitlab-org/gitlab-ce#22132, make params consistent when creating branches.

iid and id are confusing, it would be great if we could remove the universal id entirely :) I'll create an issue for that later (unless someone beats me to it).

Actually, talking about ids, I believe there's a fundamental design flaw in this API implementation as a RESTful interface. The issue is that the returned contents are indexed by ids instead of URIs of resources.

Basically, as an API implementer I don't care that the gitlab-org/gitlab-ce project has an id of 42, what I care is to identify it in a unique, understandable and universal way. So the unique ID of this project, as an API implementer is https://gitlab.com/gitlab-org/gitlab-ce.

So working on an utility, when I lookup a merge request (for example), or a project snippet, I have to first query the API for what's the id of a given project, to then be able to access the subresource by its id. It's always an unnecessary extra step and conversion that's not useful.

Basically, if I want to access a project, I should do it through /projects/:namespace/:repository, following the exact same rationale as the web UI!

To follow up on my example, I could get requests in one step by doing /projects/:namespace/:repository/requests for example.

my 2 cents

@guyzmo I agree. Having to know the project's actual numeric ID, which isn't used anywhere in the web UI, is unexpected at best.

@guyzmo @rspeicher actually you can use namespace/project-name as the ID in the API, it just wasn't documented until recently (thanks for fixing that @axil!).

The only problem might be that it requires the slash in namespace/project-name to be encoded, e.g. /api/v3/projects/gitlab-org%2Fgitlab-ce/issues

Also suggest that we enforce a standard way to add the namespace/project in each endpoint (e.g. gitlab-org/gitlab-ce instead of project ID 123456) so that we don't rely on encoding %2F everywhere.

I really like @stanhu's suggestion for this :)

Only problem will be with nested groups, since they'd be something like namespace/subnamespace/subnamespace/project-name, maybe even more depending on how many layers we allow.

And if we make nested groups EE-only, the API code would have to differ – possibly significantly – between the two editions. Or we would just have some pointless code in CE and if there are any bugs in the implementation we'd have to modify CE and wait for a merge into EE, I guess?

gitlab-org/gitlab-ee#1030: Remove Project Git Hooks in EE, they've been deprecated and renamed Push Rules.

We really need a better changelog for the API.

@zj @dzaporozhets maybe we should have a breaking change label so we don't miss anything for 9.0?

well, about the nested projects thing…

I think keeping a canonical <namespace>/<repository> scheme helps a lot as a standard way to keep some sort of conceptual interoperability between the git services (gitlab, github, bitbucket, launchpad, gogs…). So for namespace support, I could imagine a couple of ways to make them:

keep that canonical scheme, either by having a project "rooted" with a global namespace, but that can be included within some "metaprojects".
transform the path separator into some other character (that would be not allowed within a namespace name), like : or \…

for example, I'm developing a tool that handles several git repository services from command line, and ATM I'm focusing on providing the same set of features across the services, and one of the hypothesis I'm considering is that a repository slug follows the <namespace>/<repository> scheme!

@guyzmo The problem is that we need to retain support for Git URLs, so I'm not sure : would work, also not sure if Git does anything special with \, or if that'd cause any unforeseen problems. Plus it just doesn't look very good :P

yup, I totally agree the paths like that are quite hackish (except if - or _ would have been reserved, maybe a ., which would build nice looking pathes…).

Though anyway, I really think that having a canonical repo slug would be the most way!

maybe we should have a breaking change label so we don't miss anything for 9.0?

@connorshea hm good idea. please create one

I wouldn't worry too much about including IDs in URLs. The important point for a RESTish API is that the client should not be constructing URLs, they should be included as opaque references in API results. That's a lot of change though.

https://gitlab.com/gitlab-org/gitlab-ce/issues/22413 HTTP 409 is not appropriate for a permissions error

Added breaking change label

breaking change has now been added to relevant issues, please add the label if I missed any.

Mentioned in issue #22600 (closed)

I've promoted URL opacity to an issue - #22600 (closed) - as it's come up in at least one customer request

Mentioned in issue #22700 (moved)

Mentioned in issue #22693 (moved)

#22818 (closed) - API: Remove /licenses, /licenses/:key, /gitignores, /gitignores/:name, /gitlab_ci_ymls, and /gitlab_ci_ymls/:name

https://gitlab.com/gitlab-org/gitlab-ce/issues/23061 - API: consolidate project lists

https://gitlab.com/gitlab-org/gitlab-ce/issues/23104 - API: remove public param for projects

Added ~122770 label

Pagination for SystemHooks endpoint: gitlab-org/gitlab-ce#23302

#23335 (closed) - When a new SystemHook creation fails, we should return 400, not 404

gitlab-org/gitlab-ce#23468 - Remove POST /projects/:id/triggers/build in favour of POST /projects/:id/pipelines

#24016 (closed): POST /:id/hooks shouldn't return a 404

#23924 (moved): List deploy keys for non-admin users through API

Mentioned in issue #22113 (closed)

We should think about adding few scopes too, this is related: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/7615#note_18919851

@brodock Unless it changes behaviour we should do it already. This list is getting long, and so far the API has never had a lot of scheduled items for a release. So if we can do something now, we should.

We can start adding specific purpose scopes for more granular control, and to keep the backward compatibility we keep the current scope as "allow mostly everything", and deprecate it in a future release.

What do you think? @zj @dzaporozhets @rspeicher @rymai @DouweM

Some ideas: https://developer.github.com/v3/oauth/#scopes https://developer.atlassian.com/bitbucket/concepts/bitbucket-rest-scopes.html https://confluence.atlassian.com/bitbucket/oauth-on-bitbucket-cloud-238027431.html#OAuthonBitbucketCloud-Scopes

@brodock Did you see #20492 (closed), #13951 (closed), #20191 (closed), #18369 (closed)?

sorry @DouweM, my search fu only landed here :)

gitlab-org/gitlab-ce#25183 - Remove filter for IID on projects/:id/merge_request (same for issues)

https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6661#note_20053161 Update to Grape 0.19.0 and make all DELETE endpoints return a 204.

https://gitlab.com/gitlab-org/gitlab-ce/issues/26315 - Use the IssueFinder to filter by labels in the API

mentioned in merge request !8600 (closed)

I created a branch 9-0-api-changes to early start breaking API changes.

mentioned in merge request !8736 (merged)

mentioned in issue #27154 (closed)

mentioned in issue #27142 (closed)

@smcgivern So, some assumptions based on discussions around v4 so far:

We'll have a v3 and v4 namespace on 9.0
Nothing will change on v3 so far (it will be removed on 10.0)
v4 will "inherit" v3, so any call to a v4 endpoint without changes will fallback to v3
Just a reminder, following this idea we'll end up with duplicated endpoints in some cases (#27142 (closed))
The other way around will also occur, a v3 call for a endpoint that don't exist on v3, but exists on v4 will "fall up" to v4

Am I missing something? @to1ne @razer6

I've created a new issue around the new namespace #27180 (closed)

Edit: Added the ref to v4 namespace issue.

@oswaldo that's pretty much right. I think it's easiest if lib/api is v4 by default, and we make exceptions for v3, so that the entirety of the v4 API is in one place. That way v4 doesn't have to inherit, and v3 can fall up.

I don't have a suggestion on the duplicated endpoints - maybe for renames, they should not 'fall up'? @to1ne wdyt?

@smcgivern @oswaldo Yes, I also prefer defaulting to v4. The great things about this approach: it will be easy to the delete the v3 in 10.0 and when we do, we won't lose the history.

So I think we first need a MR that fixes #27180 (closed). After that, all other MR's with breaking change should include a (partial) copy of the original implementation for v3.

added Deliverable ~480950 labels

changed title from [Meta] List API endpoints to change in 9.0 to API v4

assigned to @oswaldo

changed title from API v4 to [meta] API v4

changed milestone to %8.17

mentioned in merge request !8793 (merged)

#27532 (closed) - Remove projects/:id/milestones?iid=x filter on V4

#27585 (closed) - Move /ci/api/v1 to /api/v4/ci

@stanhu @DouweM @marin let's get rid of Git Annex with 9.0 as well: https://gitlab.com/gitlab-org/gitlab-ee/issues/1648

mentioned in issue #27532 (closed)

changed milestone to %9.0

mentioned in merge request !3328 (closed)

gitlab-ce#28017 - API: Separate CE params to avoid wrong conflict resolutions

#28137 (closed) - API: Pass labels as array instead of comma-seperated list

Dropping of the theme_id property: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/9199/diffs#note_23336330

mentioned in merge request !9199 (merged)

I've listed all the current open issues in the description. I've categorized them on weight. If you like to contribute, feel free to pick one from the list you like.

@DouweM @smcgivern @rymai @razer6 Please have a look at categorization of the open issues. Sure let me know if you don't agree. I also could use some opinions in the In discussion issues.

https://gitlab.com/gitlab-org/gitlab-ce/issues/28286 - Remove duplicated endpoint for deleting a deploy key

mentioned in issue #25688 (moved)

https://gitlab.com/gitlab-org/gitlab-ce/issues/28327 - Make the subscription API more RESTful

https://gitlab.com/gitlab-org/gitlab-ce/issues/28328 - Make the starring API more RESTful

#22018 (closed) - list of merge requests assigned to the milestone, currently in review: !9298 (merged)

mentioned in issue #28405 (closed)

mentioned in merge request !9375 (merged)

We will support (bugs and security fixes) API v3 at least until GitLab 9.3 (this is reflected in the 8.17 blog post). Then we'll decide what to do next.

I'm concerned that we'd be doing a disservice to our community and customers by removing it entirely too early.

Full conversation here: https://gitlab.slack.com/archives/f_9-0-breaking-change/p1487712778000791

Modified the issue description:

-In 9.3 or later if there are good technical reasons:
+In 9.4 or later if there are good technical reasons:

Environment should use ProjectBasic entity: https://gitlab.com/gitlab-org/gitlab-ce/issues/28598

I created this issue for making pagination limits configurable: #28675 (moved)

mentioned in issue #22374 (closed)

mentioned in merge request !9723 (merged)

I created !9723 (merged) to make some general documentation changes. Feel free to push changes.

@to1ne @oswaldo Before we can close this, we need a blurb added to the 9.0 release blogpost!

added Next Patch Release and removed Deliverable labels

changed milestone to %9.1

@DouweM I think we can close it now, right?

@oswaldo Yes! Awesome work everyone :)

closed

Would like to double-check the status of the removal of V3. On https://docs.gitlab.com/ce/api/v3_to_v4.html we explicitly say that API V3 will be removed in GitLab 9.5, to be released on August 22, 2017. But here we mention this condition:

In 9.4 or later if there are good technical reasons

As this may confuse our users, I think this should be raised :)

cc @to1ne @DouweM @smcgivern

Toon and Douwe are both off this week!

It looks like @dblessing added this line to match the blog post, which says:

We will continue to support v3 of the API until August 2017 and so we encourage you to make any necessary changes to applications that use the v3 API.

That doesn't technically say we will remove it in 9.5. I guess we could just update v3_to_v4 to clarify that?

Note that in 9.3, the v3 API was already changed due to a sharing with v4 and then v4 changed, inadvertently changing v3 in an incompatible way.

Is there an issue for stabilizing and making sure that backwards compat is kept with at least v4? We've noticed many unannounced changes in API objects and hook bits before… I know of #21468 (moved) for webhooks (kinda), but is there one for the API?

That doesn't technically say we will remove it in 9.5. I guess we could just update v3_to_v4 to clarify that?

@smcgivern Sounds good to me!

mentioned in merge request !13074 (merged)

https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/13074

[meta] API v4

API version change

Open issues

Tough nuts

Quick wins

In between

Optional

Ongoing

Done

Current doc with V3 to V4 API changes

Designs

Child items ...

Activity

Admin message

Admin message

[meta] API v4

API version change

Open issues

Tough nuts

Quick wins

In between

Optional

Ongoing

Done

Current doc with V3 to V4 API changes

Activity