How to better help non-Ruby/Rails developers get work done with GDK

GDK was originally developed to help experienced Ruby on Rails developers get set up quickly so they can get work done in the GitLab source code. It is now 2 years later (994ec536 dates from 2014-08-06) and practice shows we also want other people to be productive in GDK:

• front-end specialists
• first time contributors trying to fix a small bug
• ...

I think we can do a better job getting people started, but I am not quite sure how. I am also not sure if enough people know how to ask for help with GDK, if we are responding to them in the right way, etc.

I am less concerned about experienced Ruby/Rails developers because they can solve more GDK problems on their own, but they should also be encouraged to raise issues / help improve things.

cc @winniehell @lbennett @stanhu

Let me start off with emphasizing how awesome this development kit is. I think I never got started with a project of that size with so few knowledge about it so fast (I think it was less than two hours until I got my own code running locally). You are doing a great job on this!

Next, let me emphasize that this issue is even aweseomer!

Now for some things that are not perfect yet.

• Add instructions on how / where I can find test user for my local running instance. For me the easiest was to setup the LDAP (which was amazingly straight forward). That should probably not be the case.
• Use only one working directory. I'm running make update in the development kit but I need to run rake test in gitlab-ce. Maybe just adding make test would be sufficient. That should probably also include linting (everything that needs to pass before I can contribute).
• Merge request linting: Things like checking locally if there are conflicts with master and if an entry to the CHANGELOG has been added would be a great help. Maybe that can be part of make test.

I am also not sure if enough people know how to ask for help with GDK

A dedicated place for how-tos would be great! Sometimes questions that pop up are very trivial (e.g. how do I lint Ruby?) but they pop up for many new contributors. Having a note in the README where to ask those questions would be enough.

Generally I've heard nothing but praise from people trying to get up and running with the GDK.

One thing that might help speed things up is having a Docker container that already has everything you need already, so you don't need to install Ruby yourself.

so you don't need to install Ruby yourself

@stanhu But you need to install Docker?

@winniehell Good point. :) But it may still be easier to install Docker + a GDK container that already has everything ready than to run. The installation directions involve other dependencies needed to build GitLab (e.g. phantomjs).

@stanhu Don't get me wrong—Docker images are great. But having everything to run also means it is more involved to replace any component (like trying out a different Ruby version for example).

We already have an issue to clean up the README https://gitlab.com/gitlab-org/gitlab-development-kit/issues/126 ; after we do that it will be much easier to add a 'how to' section.

Running more commands, like GitLab tests, from the GDK root: this does not make sense to me. It just happens to be that GitLab is spread over multiple projects (gitlab-ce, gitlab-shell, gitlab_git, gitlab-workhorse etc.). Each of these has stand-alone tests.

Avoiding Ruby issues with Docker: see #131, #135. I think this has potential but it is a non-trivial amount of work and I am concerned about adoption. It only makes sense to me if it is really easy to use. That means Docker itself has to be mature enough (the fact they offer native OS X virtualization now is a good sign to me) and that we (I) have to work hard to make GDK fit with Docker.

@jacobvosmaer-gitlab Sounds great, seems like you don't need my feedback, because you covered already everything. Sorry, I didn't have a look at the other issues before commenting.

@winniehell sorry if I sounded dismissive! That was not my intention. :)

I want there to be a 'how to' section in the GDK documentation and your feedback is very welcome.

I also forgot to respond to your third point:

Merge request linting: Things like checking locally if there are conflicts with master and if an entry to the CHANGELOG has been added would be a great help. Maybe that can be part of make test.

Both of these points are specific to the gitlab-ce project and therefore not quite in scope for GDK. I suggest raising them in the 'core' slack channel or creating a gitlab-ce issue for them.

@elidevco I understand what you mean. BTW I think http://docs.gitlab.com/ce/development/README.html might be useful for you.

I love what Bundler does when stuff crashes:

https://github.com/bundler/bundler/blob/2a138ed736b2210a21588db44b567dbdea6f7a18/lib/bundler/friendly_errors.rb#L53

Maybe we could do something like that? Not sure how hard it'd be.

@winniehell merge request linting is essentially Danger, we really need to get support for GitLab in that! Unfortunately the gitlab gem is what's blocking us right now :(

Hi guys, First of all, I really like the GDK and the ease to get it running Second, is there a (open) gdk slack channel? (googled for it, but all I found was integration between gitlab and slack)

Maybe as an improvement suggestion. Sometimes I walk into problems when switching branches with still running processes, bound ports. So it would be great to have a clean target in the Makefile that kills all processes

BTW the documentation page is also great. Did not know about that. Thanks @jacobvosmaer-gitlab

Is there a help command that comes with the GDK? I just realized I always have to come to the README if I want to find out what commands exist, would be nice to have a command that just outputs all the available options (e.g. make, make update, make clean-config all, ./run, support/set-gitlab-upstream, support/enable-postgres-extensions, etc.).

@connorshea thanks for the link to Bundler's friendly errors. That would be something to build into gitlab-ce if we want it. GDK cannot see into Ruby exceptions.

@hwdegroot interesting idea to have a 'GDK slack channel'. I am not 100% sure how to call it and how it would work out technically (does Slack allow self-sign-up?) cc @stanhu

Re having a make command to kill running processes: I am not keen on this because of the risk it kills other processes that do not belong to GDK. The nice thing about foreman (./run) is that it is clear what processes belong to it. BTW we just made some new improvements to make foreman shut down GDK services more reliably. (https://gitlab.com/gitlab-org/gitlab-development-kit/commit/7be8c0e40f6ebe06dcd36e7ad9a7116a2d3fc130)

@connorshea no we don't have a help command at the moment. I agree it would be nice to have a short list of the available commands in GDK.

@connorshea what about cat HELP (see 03e364d2)

@jacobvosmaer-gitlab feels kind of hacky, but it does work! Thanks :)

@jacobvosmaer-gitlab here is how you can make a public Slack organization: https://github.com/rauchg/slackin

I'm a smart guy but I haven't used Rails in seven years, then I fell head over heels in love with Gitlab, I love it and want to have its babies, so of course now I have ideas of things I want to write and add to it, and so I started with the GDK, and I think I can offer some insights. Two weeks ago I was reading a book on Rails. Last weekend I got gitlab GDK dev environments up on four different already supported OSs just so I could walk through and watch a working GDK sample. Then I got it working on OpenSUSE which had some hurdles but which was still ridiculously easy considering how green I am as a Ruby Rails dev.

I have now gotten GDK up on Ubuntu, Debian, OS X, SuSE, and Centos. I love the GDK I think it's fabulous.

It would be friendly to check the path for things and check their versions.

If you run gdk install and redis-server is not found, or is a bad version, or git is not found or is old, or postgres is not found or is old, it would be super friendly to tell users about it.

It would be super friendly to provide some curl scripts to get known required development and tool packages onto each common target (linux distros, and os x, via homebrew)

So if I was a frontend CSS designer and I wanted my own little gitlab, and I've never heard of rvm let alone run it, then I think I would like some help getting that all on there. It seems to me that on Linux and OS X the current consensus of rubyists is that you should use rvm to manage your ruby installs, but as a new person to rvm I found it requiring a login shell for rvm use to succeed to be user-hostile to people who just log into Ubuntu or Suse and use KDE Plasma and don't know how to change their Konsole to use /bin/bash -l. So a pre-rdk curl script could be all kinds of helpful to this kind of user, and to me.

After rvm install ruby ... and so on, lots of people might appreciate it if we went on to check if the things necessary to bootstrap rails are going to succeed, and if not, point the user at a README page to get out of that pit. When we exit this stage, we have rvm, ruby, rails, nokogiri, header and library issues, XCode command line tool issues, and any number of other things out of our way.

Any failures during gdk install should place a marker file on the disk

I was a bit tripped up by what to do if gdk init works (it always did for me) and gdk install doesn't complete. After a while I figured it's best to wipe whatever folder gdk init created, and start again from gdk init. Failure cases I hit include, redis-server is installed but not in the path, headers and libs are missing or won't install even when they're present because gem install nokogiri or gem install charlock_holmes was failing.

I think if the GDK readme was organized per distro, and included my suggested bootstrap shell scripts, it would be possible to automate the "get my platform ready for gdk" to a high degree, for those non-experts who desire it.

If I ran gdk update but the gdk install failed, the marker file on disk would tell the user "this gitlab environment failed gdk install, you need to correct those issues and try gdk install again".

I could draw a flowchart if that helps visualize the pits of GDK fail, and how to prompt users to get out of them.

gdk update should update gitlab-shell

On several vms in the last while it seems I have to manually cd gitlab-shell and then git pull, I sort of wish that gitlab-shell would be fetched automatically, if it's out of date. Not sure how you would detect that but gdk update fails for me like this recently:

WARNING: This version of GitLab depends on gitlab-shell 3.6.6, but you're running 3.6.3. Please update gitlab-shell.

It's super friendly that you get this warning. It would be even more friendly if gdk update knew when a gitlab-shell fetch was necessary, and knew that "git pull upstream master" will be needed by some users to fetch gitlab and gitlab-shell, and "git pull origin master" will be needed by others depending on whether or not they have their own forks.

What if Expert Developers may also appreciate a standardized environment?

But I disagree that only non-experts may desire it. My mantra is automate-all-the-things because developers who can do things consistently and in an automated fashion will appreciate enormous Pits of Success and the relative absence of Pits of Failure just as much as non-experts.

Next time one of the acknowledged Ruby Experts moves to a new laptop, if they run into a pit of failure, and they then contribute their learnings in terms of a merge request to the GDK, that adds a bit of error checking and a friendly error message, the time they save may be their own, six months from now, or any other contributor's time, expert or not. If you've been happily hacking for six months inside vagrant or docker, and decide to run something outside of those, and on your main OS image, you might want to have some easy single command as easy as vagrant up too.

I opened and then closed a ticket to suggest putting bootstrapping of rvm into gdk. Now I see how silly that is. But the underlying wish is to get a blank factory install {Ubuntu,SUSE,Redhat,Centos,Arch,Debian} VM to become a working gitlab development image with a single command. Even expert ruby developers probably get tired of dealing with what interesting things maintainers of packages like nokogiri and other packages that have done, and what your last apt-get upgrade on your Ubuntu platform has done, and dealing with include file combinations and library link failures. libicu C++ mangled names are output and you have to determine how to get this gem onto the platform where you're having a particular issue you need to delve into and solve, and you're stuck getting around gem install problems. Detecting and solving these things can be better than google-it-try-it-rinse-repeat.

Mentioned in issue #135

cc: @jacobvosmaer-gitlab ^

@warren.postma This is awesome feedback, thank you so much for the passion

Some more feedback in the form of questions:

1. If I run the tests in the main gitlab project with rake tests, but I have the FULL gitlab up (gdk run, not gdk run db), ought I to expect as many as 67 failures out of 13022 examples? Wicked amount of tests kids. So impressed.

2. How can I correlate the results of rake test to the gitlab-ce online test results since all the tests there are split up? I certainly don't see any error counts for the same git rev that total 67, so I suppose I am seeing a mix of test failures that are due to me running on OpenSuSE LEAP.

A snippet showing my failed tests list for git rev a98ad03ba18da0b1534f36dafafa9a1c644d0bf1: https://gitlab.com/warren.postma/gitlab-ce/snippets/29573

I'd love to know what the pass/fail count for that rev is from any of the official devs, if they were to do:

~/gitlab/gitlab-development-kit/gitlab> bin/rake test

Look like there is an experimental Vagrantfile, but I think there's a lot we can do to improve this. I've started a branch and am working to:

• provide a docker-compose file(s)
• update the Vagrant box to use the docker provider instead of provisioner (will allow for autodetection of native docker)
• eliminate some of the quirkiness with password-based rsync (if it will even be needed after the first two steps)

Thanks @LongLiveCHIEF As a reference I've used the following Dockerfile personally https://gitlab.com/MrChrisW/gitlab-docker/blob/master/gdk/Dockerfile

@MrChrisW @LongLiveCHIEF I heard @grzesiek is running in docker too. So you might want to work together/ create snippets.

@warren.postma Thanks for in-depth analysis about what may be missing in GDK. I agree that we should improve things, but the vision you described resembles a fully-fledged environment with all-the-things automated, which is quite difficult to achieve as it requires a lot of work. I agree that we would benefit from having more automation in GDK, but we have to automate things wisely, as we do not want to limit developers in their work and we want to keep GDK simple.

A lot of great insights are present in your description, we should certainly open a few separate issues and move discussion there. We work in highly agile environment, so we usually need separate issues for each suggestion, where we will discuss things and decide whether we should move things forward in that direction or not. We do not have unlimited resources, so we need to choose minimal viable changes wisely.

If you take a look at product handbook, which is also relevant for GDK, you will see that we agree with concepts like convention over configuration, but since GDK is a simple product for complex needs, changes that bring more automation are not that easy to introduce and require thoughtful steps. We have a bunch of issues like that in issue tracker already.

Please don't hesitate to open an issue when you have some great ideas how to improve things. GDK is also an open source project, so all merge requests are welcome as well!

I have now been using GDK for a month and my list of things to change in GDK is smaller because I have a bit more of a clue about what's going on, but I will probably send some merge requests for things like "Check if X is in path and print friendly message and stop before anything else happens in gdk install".

One thing I still don't know about is whether bundle exec rake gitlab:test should actually pass with zero errors on every developer workstation without additional setup than whatever the GDK does? Anybody know?

@warren.postma It believe that all tests should pass, unfortunately we are not able to run gitlab:test to check that, because at this point it may take a few days to finish without our heavily parallelized CI environment

On my 8 core Xeon workstation it takes 137 minutes. Does that probably mean that it's dying without running large swathes of the tests?

https://gitlab.com/warren.postma/gitlab-ce/snippets/29573

Finished in 137 minutes 58 seconds (files took 21.68 seconds to load)
13022 examples, 67 failures, 11 pending
Failed examples:
...

@warren.postma I am not sure if that runtime indicates skipped tests. @grzesiek when you say it would take days, I wonder if you base that on adding up the duration of individual parallel CI fragments. I think each CI job has quite a bit of overhead, and some of them get retried several times to overcome transient failures.

I will give a local test run a try on my work machine tonight.

@jacobvosmaer-gitlab On master we have 85 builds. something like 30 minutes on average (this comes to my mind just after looking at timings, I didn't do any calculations). That means it may take up to 40 hours to process pipeline on master. Regular pipeline in MRs is a half od that (or less), so let's say 15 hours, CI overheard (additional CI specific jobs) may be something like 30%.

That test run I was trying fell on its face within 15 minutes with some Spinach failure :(

Running the full test suite locally has not been a part of my workflow for a long time. I usually work with just a couple of spec files that I expect to be relevant, and then I push to gitlab.com once in a while to trigger a full build and see what other specs I need to fix.

As someone who just brought up GDK on OpenSUSE and had no idea where the system works on opensuse and where it doesn't, one possible read of my result is that the tests work great and there are 78 regressions on OpenSUSE.

I would like it if all the bundle exec rake gitlab:tests pass on at least ONE operating system (say Ubuntu), and I knew where the Green grass of happiness ends and the Red grass of brokenness really is. The feeling for me is one of blindness. I have no idea.

Does that seem worthwhile as a hope/expectation? The tests run on the cloud on one container, and don't necessarily work on developer machines? I guess that's what you get from the success of getting everyone to write tests. Now you have so many tests, you can't run them (they take too long).

It's all kind of anxiety inducing for me, as a new person to the platform.

Maybe something NEW developers would enjoy would be a bundle exec rake gitlab:sanity target that runs a collection of really great and really useful tests that we can use to see that the basic platform is up and working.

So what you want to know is if you installed everything correctly? I think rake gitlab:test is not a good choice for that.

We used to recommend rake gitlab:check but I don't know how well that works for development (it was made to catch problems in production installations). But the whole idea of running a rake task might not be quite sufficient anymore because that won't go through workhorse.

@grzesiek weren't you involved with developing fully integrated tests for the packages? Anything we can learn from that?

@jacobvosmaer-gitlab @warren.postma We are currently considering bringing something like rake gitlab:test:integration or Settings -> Integration tests to GitLab, through GitLab QA, see gitlab-qa#25 (closed) for more details.

A note of explanation: GitLab QA project is integration tests suite for GitLab that is a sanity check using entirely click-driven test scenarios with a headless browser under the hood.

@grzesiek would those tests be appropriate for post-installation "is it working" tests in production or development? Or would they be more for our internal process?

@jacobvosmaer-gitlab This tests are meant to be run against every instance that is complete, and has all moving parts installed. It is designed to run against production instance, but should also work against development one.

These are back-box, entirely click-driven test scenarios, so there is no need to access database etc. We have some flexibility here, I was thinking about releasing a Gem that is capable of running integration test against every instance, see gitlab-qa#4 (closed).