Remove the documentation in Help section from the Rails app

The Help section has caused a lot of headaches, and I have been a big proponent of removing it from the Rails app due to the constraints it introduces.

Problems with /help

• Holds back docs.gitlab.com from being made great again.
• Designers must consider it when implementing/updating the interface.
• Tons of problems with routes in Rails (/help/help, broken routing to the Getting Started with Pipelines page, etc.).
• Constrains the structural design and updating of the documentation.
• Slows the technical writing team down.
• Part of the reason docs.gitlab.com has such a terrible build system. (we switched to a proper static site generator since and deploying to Pages)
• Slower than it should be because it’s part of the main Rails app.
• Any "advanced features", e.g. a table of contents, need to be implemented in our static site as well as Rails. This is a massive pain for making the Help/Documentation easier to use.

You know the reason nobody has an m. site anymore? It doubled the work they had to do to implement a feature, they were essentially reimplementing it entirely for mobile. This is what we're doing here, and it sucks.

Considerations for removing it

I'm not suggesting we just rip out the Help section, the idea would be that it be replaced with some very basic information, links to docs.gitlab.com, and that's it.

In order to ensure that the docs will be in-sync with the on-premise version users are working with, the documentation site either needs to have a versioning system (based on the state of the GitLab repository's tags presumably), or we have to ship a static site alongside every GitLab instance to serve as their documentation. It cannot continue to be embedded in the Rails app, as this is what's causing the problems I discussed above.

We need to:

• Have some way to make sure users are viewing the correct version of the documentation for their instance.
• Ensure that, if we use docs.gitlab.com, enterprise customers are okay with that.
• Replace Help links from the Search suggestions and help text with their new counterparts.
• Make sure replacing this doesn't prevent the Docs team from adding new documentation on new features, it has to essentially be fully replaced in a single release.

Added help page in GitLab FOSS label

When we rip it out, can we also make the help available for different releases? Something like

• https://help.gitlab.io/ or https://help.gitlab.io/latest
• https://help.gitlab.io/v8.9
• https://help.gitlab.io/v8.8 ...

That would make it easier to compare how features have changed over time.

@SeanPackham this is pretty convincing stuff from Connor. I care most about having the docs in the same repo. If we can't render them in the application that is acceptable. Programming without internet is very hard anyway.

@dzaporozhets what do you think?

One consideration is that switching to Kramdown will make things a lot easier already, so maybe do that first https://gitlab.com/gitlab-org/gitlab-ce/issues/18552

Thank you for this @sytses I agree keeping it as part of the app makes sense and then we can tweak the documentation authoring process: swapping to Kramdown, improving CI builds, creating documentation authoring tools etc.

@SeanPackham, do you mean keeping docs as part of the repo, or the app? I am in favor of removing it from the app, but of course keeping the source in the repo. Although, as mentioned, there are many prerequisites.

@winniehell Yes, we will need versioned docs if we do this. It should stay at https://docs.gitlab.com, but it can still be deployed via GitLab Pages on GitLab.com.

I really like where this is going. Thanks for the discussion

@dblessing sorry I meant keeping it part of the repo/project. Thank you for checking.

I care most about having the docs in the same repo

@sytses same for me. So I am ok with @connorshea proposal:

I'm not suggesting we just rip out the Help section, the idea would be that it be replaced with some very basic information, links to docs.gitlab.com, and that's it.

Mentioned in issue gitlab-foss#22617 (closed)

Ooo, this issue has gained some traction :)

I would really love to see us have a single source of truth instead of maintaining both /help and docs.gitlab.com. I discussed with @SeanPackham and our notes (appending on top of @connorshea ) are:

---

Thoughts:

• Separate it from the Rails app
• Separate the docs static site tech (Middleman) from something that has to work with Rails
• Embed it in the app under /help using an iframe
• Static docs will be built as part of the app startup process and served via rails asset engine (might have the most pushback on this).
• Alternatively we commit the built docs, this has the added benefit of being immediately readable by someone downloading the repo, navigating to docs and opening index.html
• We can use fancy things like yaml frontmatter that can be used by the static site generator.

Challenges:

• GitLab docs will be eventually be built using a static site generator (SSG), but where will the source code live? If we are to continue serving all docs under doc/, will that dir also contain what an SSG contains (Gemfile, config.yml, static assets etc.)?
• Not known impact to users/customers

Todos:

• (easy) We link to the internal docs in various places. Find all help_page_path occurrences and replace with the new doc links.

@axil fyi we dont allow render of iframe in GitLab from different host/domain.

Embed it in the app under /help using an iframe

Please, let's not do this. It will be ugly, multiple nav, etc. Let's remove it from the app and then if we can provide an easy way for someone to deploy an internal doc site, that's fantastic. Let's make that a secondary goal, though so we don't delay.

2 cents: I think removing it and unifying all documentation improves dependability and discoverability.

Great reference for versioned docs: https://www.sidefx.com/docs/

More versioned docs:

• https://atom.io/docs
• https://help.github.com/
• http://facebook.github.io/react-native/versions.html

@axil we can change help_page_path to help_page_link_helper or something like that.

def help_page_link_helper(path)
   help_page_path("#{path}")
end

And then have things like:

= link_to "(?)", help_page_link_helper("integration/bitbucket")

Once we're ready to change over to using docs.gitlab.com, we just change the helper method to point to the docs domain, and we're good.

@axil I'm also opposed to the use of an iframe, that's less-than-ideal from a security standpoint and would be somewhat finicky. I'd rather we link to docs.gitlab.com or (ideally) have a Jekyll or Middleman site inside the Rails app.

Also, it should be part of the build process to create that, the final site shouldn't be included in the repository since that'd cause too much repo bloat.

Another SSG worth looking into is Nanoc, it seems to be the one best suited for creating versioned documentation.

I also think we should give a fair shake to Read the Docs (self-hosted by us, so we can use it on our domain).

Ok, the iframe thing was just a thought, so if the consensus says no, we don't use it :) cc @SeanPackham

@connorshea I'm very much inclined to use nanoc (I met its author in Prague :)), and yes, versioning the docs would be awesome!

for nanoc. I have a POC hanging out somewhere from a long time ago. It works well and is really flexible as far as allowing us to keep most of the current structure we have and without requiring us to change any markdown file names (like middleman would).

Mentioned in issue gitlab-foss#22701 (closed)

Mentioned in issue gitlab-foss#20068 (closed)

Mentioned in issue gitlab-foss#23476 (closed)

Mentioned in merge request gitlab-foss!7142 (merged)

@connorshea: You suggested in the issue body that we replace the Help section with very basic info that links to the docs.gitlab.com. Would it make sense to just have 'Help' link directly to docs.gitlab.com instead?

@awhildy the Help section also has keyboard shortcuts, a link to buying a support subscription, the version and version check, etc. I think we should leave the page.

Mentioned in issue gitlab-foss#17658 (closed)

Mentioned in issue gitlab-com/gitlab-docs#44 (closed)

Mentioned in issue gitlab-com/gitlab-docs#53

Mentioned in issue gitlab-com/gitlab-docs#48

Another compelling reason to switch to maintaining solely the docs portal is the ability to do stuff like automatically generating index pages for relevant documentation.

For example, see how GH does this in https://help.github.com/articles/managing-user-account-settings/, where they include the relevant docs with a line describing what each is about. I tried to do something similar with our MRs here https://docs.gitlab.com/ce/user/project/merge_requests.html, but all done manually. @connorshea is pretty sure that that GH used Jekyll's collections feature. We could do the same with nanoc, but we are held back by /help.

changed title from Remove Help section from the Rails app to Remove the documentation in Help section from the Rails app

I briefly talked about this with a customer+ in GitLab FOSS. They did not see a major reason that the online versioned docs would not be sufficient for them. Servers are often firewalled off from the world, but rarely are workstations, so a user clicking a help link that goes to docs.gitlab.com would be fine - especially if that link went directly to the correct version.

Some status updates on this issue:

• We've built a new GitLab Docs site using Nanoc: https://docs.gitlab.com / https://gitlab.com/gitlab-com/gitlab-docs
• There's a WIP MR for said site that adds versioning: https://gitlab.com/gitlab-com/gitlab-docs/merge_requests/5
• There are currently some blockers for the versioning, they're expanded upon in the MR.

If we can get versioning in, that'll be the biggest step toward removing (most of) the Help section.

mentioned in issue gitlab-foss#29429 (moved)

One more reason to focus entirely on the docs portal is that we could avoid things like splitting Omnibus and source docs in areas that are hard to follow:

• Pages admin docs
  • Omnibus
  • Source
• Redis HA
  • Omnibus
  • Source

And now we need to split the Geo docs because they are very hard to follow https://gitlab.com/gitlab-org/gitlab-ee/issues/1908.

---

If we used only the docs portal we could do something like what the Atom flight manual does and have both info (Omnibus + source) that could be toggled with a button.

For example, there is info on installing on all three platforms and instructions change with a click of a button:

• Rendered page http://flight-manual.atom.io/getting-started/sections/installing-atom
• Source markdown https://github.com/atom/flight-manual.atom.io/blob/master/content/getting-started/sections/installing-atom.md

Another consideration for removing the docs under help is to prevent someone from knowing the GitLab version. It might not be that difficult to deduct GitLab's version from comparing the docs that appear on /help with what we have in the repo. We show the help page on every instance whether you are logged in or not.

Here are my notes after investigating how we can proceed with this move. Essentially, Docker has solved the multiple versions issue with a way which I believe we can also use. They leverage the power of Docker images, DockerHub and GitHub Pages. We can do all three.

---

Advantages of keeping only the static site generator

• We can create redirect pages without relying on the real document being available all the time in the master branch (https://gitlab.com/gitlab-com/gitlab-docs/issues/96). We can have a list of redirects in the gitlab-docs project and create the HTML redirects each time the site is built. Advantage: flexibility, no ghost documents residing inside /doc.
• Using kramdown as our Markdown engine which gives us more flexibility.
• Extending the HTML pipeline with https://github.com/gjtorikian/extended-markdown-filter which will give us the power to do things like https://github.com/gjtorikian/extended-markdown-filter/#admonition-blocks

Solving the multiple versions puzzle

1. Multiple versions the Docker way (TODO add explanation how they did it)
2. We support up to 3 versions in the docs portal including the master one, so we have: master, stable, stable-1, stable-2. All previous versions got to the archives project.
3. Create gitlab-docs-archive which will have all old precompiled html versions saved in docker containers in our registry.
4. We update this once a month, we don't care how much time it will take to deploy

We assume /help is killed and we link to the docs portal from inline links. With that said, how are we gonna link the inline GitLab docs to the correct docs portal. We now have https://docs.gitlab.com. One possible sollution would be to make the docs URL configurable in gitlab.yml. That way it's only a matter of setting a default with each release and sysadmins can always take our docs portal build it on-premise and change the URL to point to theirs. The default URL would be pointing to docs.gitlab.com.

Notes

• Never link to docs on master, master is the development branch and things change/get deleted all the time. Prefer to link the stable branches instead or use the Permalink button.
• Use index.md for index files instead of README.md. Because of this, we now use a hack to symlink README.html to index.html, which led to https://gitlab.com/gitlab-org/gitlab-ee/commit/779d212ee70fdd5c961cc4cc43722695f45ebd86 This would be solved by https://gitlab.com/gitlab-org/gitlab-ce/issues/18933.

Random thoughts

• Make gitab-docs versioned now that all products follow the same versions? What are the advantages/disadvantages?
• The raketask doesn't have to build different versions, just the latest stable one. We build a docker image each month that contains the previous months' builds plus the new version. See how Docker does it.

mentioned in issue gitlab-com/gitlab-docs#97 (closed)

changed milestone to %Backlog in gitlab-foss

@JobV fya

@axil I don't understand the following part:

• We support up to 3 versions in the docs portal including the master one, so we have: master, stable, stable-1, stable-2. All previous versions got to the archives project.

• Create gitlab-docs-archive which will have all old precompiled html versions saved in docker containers in our registry.
• We update this once a month, we don't care how much time it will take to deploy

What happens with the docker images of the older versions?

mentioned in issue gitlab-foss#32038 (closed)

I know this was sort of mentioned above:

Servers are often firewalled off from the world, but rarely are workstations, so a user clicking a help link that goes to docs.gitlab.com would be fine - especially if that link went directly to the correct version.

... but some environments do not allow user workstations to access the internet. A solution to this would be to allow GitLab server admins to modify the URL that /help leads to and allow admins to host the gitlab-com/gitlab-docs static site on their own servers.

@JobV any outstanding objections to remove /help? cc @axil

A solution to this would be to allow GitLab server admins to modify the URL that /help leads to and allow admins to host the gitlab-com/gitlab-docs static site on their own servers.

@rabbitfang my thoughts exactly :) In my comment https://gitlab.com/gitlab-org/gitlab-ce/issues/18739#note_26376603, I write:

We assume /help is killed and we link to the docs portal from inline links. With that said, how are we gonna link the inline GitLab docs to the correct docs portal. We now have https://docs.gitlab.com. One possible sollution would be to make the docs URL configurable in gitlab.yml. That way it's only a matter of setting a default with each release and sysadmins can always take our docs portal build it on-premise and change the URL to point to theirs. The default URL would be pointing to docs.gitlab.com.

@SeanPackham @axil let's create an issue for that idea. Do we think that we should do that at the same time? I think not and am in support of removing /help.

mentioned in issue gitlab-foss#32514 (moved)

@JobV I created https://gitlab.com/gitlab-org/gitlab-ce/issues/32514. I think we should do this at the same time yes. But first we need to figure out versioning in our docs portal.

mentioned in issue gitlab-foss#34069 (closed)

mentioned in issue gitlab-foss#34005 (moved)

mentioned in merge request gitlab-com/gitlab-docs!97

mentioned in issue gitlab-foss#34673 (closed)

changed milestone to %10.0 in gitlab-foss

Tentatively scheduling for 9.6.

Removing docs from the app will reduce the Omnibus package size by 38M :)

marked this issue as related to gitlab-foss#22617 (closed)

marked this issue as related to gitlab-foss#34005 (moved)

marked this issue as related to gitlab-com/gitlab-docs#53

moved from gitlab-foss#18739 (moved)