Skip to content

WIP Add support for testing the cookbook before building the package.

Marin Jankovski requested to merge test_kitchen into master

Adding support for testing the internal cookbook validity before building the package.

Reason(Pros):

  • While almost all changes are tested before accepting them, it is extremely time consuming to test cookbook changes on all platforms. When chasing deadline this gets especially problematic.
  • It has happened in the past that we release a package with broken internal cookbook.
  • With the introduction of package server built packages are pushed to everyone as soon as they are built so even catching errors as soon as the package is built causes issues for end users.

Cons:

  • Adds another step to already time consuming build
  • Depending on yet another set of external resources(test-kitchen, kitchen-docker, docker)

Resolving the cons:

  • For now, only enabled on branch builds which means that tagged packages build is not slower
  • While depending on external resources can be time consuming, it is also time consuming fixing the issues after they are released to the ether

What has been also tried:

Instead of going down the test-kitchen route and testing only the cookbook, I've tried testing the whole package after it was built.

Pros of this approach:

  • Testing the whole package installation, not only the cookbook

Cons of this approach

  • Tests would have been executed at the very end of the build process so catching errors is again time consuming as it would take 2 long build processes to confirm if the package is working
  • Still depends on external sources, mostly docker
  • The installation itself was taking much(much) longer so there was no time justification for this approach anymore

Left to do

If this gets introduced as is, all omnibus builders need to have support for docker. This support will also need to enable non-sudo docker access which is always controversial.

Fixes #852 (closed)

Merge request reports