Add support for LDAP

This can be done only after #11 (closed)

mentioned in issue gitlab-org/omnibus-gitlab#1884 (closed)

This is targetted for PCF Tile v1.1.x with GitLab EE 8.15.x at this time. GitLab 8.16.x will bring more features, and we can't continue to delay on the customer ask for LDAP until that time.

@MrChrisW I think I heard that you did some custom work on LDAP for PCF for a customer. Is this true, any details you could provide?

@marin Unfortunately it's a rumor

assigned to @WarheadsSE

@WarheadsSE Please write down the challenges you've encountered so far.

At this point, allow to note the things I have found:

• The LDAP configuration is a YAML blob
• We'll have to add all the fields, marking them all optional, in PCF's tile-generator & interface.
• Marking them all optional is our only method at this time, as LDAP is not a hard requirement
• There is no way to have a field required only when another field is filled
• We'd be generating a YAML blob inside a YAML blob. This alone, before going deeper into the stack, could be even more complicated
• Testing the implementation starting with BOSH will require functional access to an existing configured LDAP server from the CF/BOSH infrastructure.

I'll need to find some documentation on the development-accessible LDAP server for testing this, but this is on my priority list for this week.

I've done part of the work here to add the fields individually, knowing how messy it was going to get. Once I had part of this in place, it struck me that it was not a scalable solution to have all the fields, because that would limit the number of LDAP/AD servers available to the customers.

@marin @guidowb @dhigham Does it make sense to anyone to include the field as a large text blob? If that is even possible in Pivotal's OpsManager.

Can you can plug in some processing after receiving the text blob? If yes, we can make sure that what received is valid.

That will be my next question. Since we'll have to sort that part out. We do have a rake task for that, but I don't know if we'll be able to do the processing at OpsManager.

Matt Cholick from Pivotal has stated that multi-line text input (for YAML) may function when using the text, as it is present as an HTML textarea in the form.

Apparently, a second option is to use collection type, and individual fields, which somewhat removes the need for the blob. There are minor things we can do for verification via constraints, however they seem to be not very robust in capabilities.

Using collections will allow for multiple servers to be defined. Using selectors, we can handle the "these fields are not optional when LDAP is enabled" scenario. However, using a collection inside a selector looks too complicated to trust at the moment since even Pivotal is not sure they will work combined.

I believe that for now, using a single selector, with individual named properties is a better option for getting functionality in place.

We're currently seeing 500 errors from the OpsManager server when attempting to use a selector with a collection. We can get around this with a YAML blob in a textarea however, buildteam really doesn't want to do this because to the sheer probability that the input is bad, malformed, or outright screwy is so high.

We're going to continue working towards a method to use a collection for ldap_servers, and a boolean for enable/disable. At this point, however, we may not be able to put the constraints we want into place. Wherein the collection is required if ldap_enable: true, and hidden/optional if ldap_enable: false

After a call with @dhigham, it is confirmed from OpsManager devs that there is absolutely no support for nesting a mix of collection and selector. We'll have to sort something out, and I am waiting to hear back from @dhigham on when we'll get to talk to the OpsManager team in some manner, in order how to make our case work. There was mention of "embedding another form inside the selector", but we need a greater description of what and how that would work.

Okay, after having a call with PCF OpsManger team, here are the findings:

• You can not use generated properties like selector and collection inside each other, there is no functional recursion.
• When the value of a selector is changed, and the tile settings are saved, any setting no longer applicable is discarded. This rules out GitLab using a selector at this time.
• Our best bet is to use:
  • Just a collection and enable/disable based on presence
  • A boolean, collection and logic that states that if enabled, but nothing in the collection, then disregard the enable.

The changes are done but during testing, LDAP connection was not working.

We suspected it is due to ports not being open on the AWS side for port 389. This was added today and @WarheadsSE can continue debugging the new issue that popped up after this change.

@WarheadsSE will write down more details of the current challenge once there is more info and push the WIP code.

Tested this (manually) in the GCP deployment, and it worked.