Provide GitLab CI YAML linting tool as standalone native app

I've been searching for it today, but haven't found a native app (in brief, tool) version of the GitLab CI YAML (in brief, the YAML) file linter as available over the web. This is a pressing requirement, since it allows one to use the tool from e.g. repos' pre-commit configs (e.g., via Yelp pre-commit) and thereby actively prevent commits that result in invalid YAML.

We might want to create an API endpoint on GitLab which return statuscode 200, or 400 (or something). Would that suffice?

@ayufan had another idea I forgot, could you enlighten us what you think is best?

@zj: a remote API would certainly be more costly to maintain for GitLab.com than a tool with a command line interface, wouldn't it? (Offering such an API is of course independent of offering a CLI tool, so that doesn't mean GitLab.com shouldn't host it, though.) Also, Git is a distributed VCS and working on your repo offline is a primary workflow. If a pre-commit hook would have to perform network transfers (in fact, uniquely, among the current Yelp pre-commit hooks) that would slow down and single-point-of-failure-ize the whole precommit workflow.

Offering such a tool attains feature parity with Travis CI, who offer a YAML linter as Ruby gem. I think that's a good thing in this case.

The upside of an endpoint would be that given your remote, and a fixed path, you'd be able to tell if your CI YAML works for that exact version. You don't want to create a tool which does like: gitlab ci lint --version 9.0.5 or anything right?

I definitely see a use case here, though. :)

Also, moving this to GitLab CE as this is a template project, not where this feature is implemented, although it'd be great if we could test contributions. 💯

Moved from gitlab-ci-yml#2 (closed)

Added ~19173 feature proposal labels

This would be quite easy to gemify our current GitLab CI Config code, and provide CLI tool to validate .gitlab-ci.yml.

Does it make sense from product perspective @markpundsack?

@grzesiek What do you think about the versioning problem? Or is there no problem?

@zj, strictly speaking there's both remote API version and the GitLab CI version at work then. It's hard to see how remote API dependency would be easier to work with than CLI tool dependency.

@zj Probably releasing gem with the same version that GitLab itself would solve that, but I'm not sure if we should do that. From technical perspective - this is doable.

@grzesiek: that's what I thought at first, but actually @zj's idea of having a CLI param to specify the GitLab CI version will work better. Forcing the same version (without another way to control validation, e.g. schema version) for both the tool and GitLab CI breaks semantic versioning, since the tool has an independent feature and patch life cycle vs. GitLab CI itself.

@sander_maijers Good point about semantic versioning. Anyway, we can consider it, but I don't think it is a high priority issue now.

Changed title: Provide GitLab CI YAM linting tool as standalone native app → Provide GitLab CI YAML linting tool as standalone native app

@grzesiek I'd be in favor of a gem, as you suggested, but minus all the versioning stuff. Users will have to make sure they update their gem's periodically, or run older versions using existing techniques. The best scenario would be if the CLI and GitLab both used the exact same linter, factored out as a gem, perhaps. I do NOT want to worry about keeping linting code in sync, which is why hitting an API was originally favored.

mentioned in issue #20739 (moved)

Mentioned in issue #24702 (closed)

Also requested in https://gitlab.com/gitlab-org/gitlab-ce/issues/24702

The best scenario would be if the CLI and GitLab both used the exact same linter, factored out as a gem, perhaps.

It is possible to do that with the new CI configuration that lies entirely in the separate module Gitlab::Ci::Config, but we recently decided to merge Gitlab::Git (also factored out as a gem) into GitLab (https://gitlab.com/gitlab-org/gitlab-ce/issues/24374) to make it easier to introduce change, so we need to think twice before we decide to extract Ci::Config into separate gem.

We have an endpoint now, you should use that IMO. Checkout the docs. Good thing about this is, from your git remote you can check what the host of your code is, and check against that version of GitLab.

mentioned in issue #20785 (moved)

perhaps atom and sublime packages?

Lets start with feedback in the editor. The response time for the API endpoint is sub 50ms, so that should be a great start!

mentioned in issue #26907 (moved)

Today I just made a quick bash script that validate the .gitlab-ci.yml using the dedicated Gitlab API : https://gitlab.com/orobardet/gitlab-ci-linter It can be used as a simple git pre-commit hook, and is compatible with self hosted instance of Gitlab (autodetection of the API URL based on the remote 'origin' url).

Not the best script ever, but it does the job, and may be usefull for some other peoples, so I shared it.

moved to gitlab#5687

Hey! 👋

GitLab is moving all development for both GitLab Community Edition and Enterprise Edition into a single codebase. The current gitlab-ce repository will become a read-only mirror, without any proprietary code. All development is moved to the current gitlab-ee repository, which we will rename to just gitlab in the coming weeks. As part of this migration, issues will be moved to the current gitlab-ee project.

If you have any questions about all of this, please ask them in our dedicated FAQ issue.

https://gitlab.com/gitlab-org/gitlab-ee/issues/13855

You can also find more information here:

• https://gitlab.com/gitlab-org/gitlab-ee/issues/13304
• https://about.gitlab.com/2019/08/23/a-single-codebase-for-gitlab-community-and-enterprise-edition/

Frequently Asked Questions

For an up to date list of questions and answers, please take a look at https://gitlab.com/gitlab-org/gitlab-ee/issues/13855

What will we do with the repository names?

https://gitlab.com/gitlab-org/gitlab-ee/ will become https://gitlab.com/gitlab-org/gitlab, and https://gitlab.com/gitlab-org/gitlab-ce will become https://gitlab.com/gitlab-org/gitlab-foss.

Why rename gitlab-ce to gitlab-foss?

Using "gitlab" and "gitlab-ce" would be confusing, so we decided to rename gitlab-ce to gitlab-foss to make the purpose of this FOSS repository more clear

I created a merge requests for CE, and this got closed. What do I need to do?

You will need to create a merge request at https://gitlab.com/gitlab-org/gitlab-ee/. This link will eventually redirect to https://gitlab.com/gitlab-org/gitlab.

How does the licensing work in this new setup?

Everything in the ee/ directory is proprietary. Everything else is free and open source software. If your merge request does not change anything in the ee/ directory, the process of contributing changes is the same as when using the gitlab-ce repository.

Will you accept merge requests on the gitlab-ce/gitlab-foss project after it has been renamed?

No. Merge requests submitted to this project will be closed automatically.

Will I still be able to view old issues and merge requests in gitlab-ce/gitlab-foss?

Yes.

How will this affect users of GitLab CE using Omnibus?

No changes will be necessary, as the packages built remain the same.

How will this affect users of GitLab CE that build from source?

Once the project has been renamed, you will need to change your Git remotes to use this new URL. GitLab will take care of redirecting Git operations so there is no hard deadline, but we recommend doing this as soon as the projects have been renamed.

Where can I see a timeline of the remaining steps?

https://gitlab.com/gitlab-org/gitlab-ee/issues/13304

Will community contributions submitted to the new "gitlab" repository be available in the new gitlab-foss repository?

Yes, these changes will be synced to gitlab-foss automatically, several times per day.

locked this issue