WIP: GitLab.com migration to Azure

The GitLab.com migration to Azure blog post.

Closes gitlab-org/blog-posts#2

I think the flow of the article can be improved.

I think we should mention that we're grateful for the free credits and the great support we have gotten from Azure.

@AshleyS can you review?

@sytses I put the article in a Q&A form answering questions that most developers would ask. I feel that a prose format would bombard the reader with a lot of information all at once and Q&A would ease the reading since facts are presented in point form, making it easier to read.

I am working on rectifying the post from the suggestions above, I am already done with the grammatical errors. Waiting for @jnijhof input.

Added 2 commits:

• 899b504d - Grammer corrections, unproven content redaction and flow improvement
• 60ea84bb - Technical Details correction and language correction from suggested edits

We need to add more details, especially about the premium storage https://gitlab.com/gitlab-com/operations/issues/9#note_2617405

@jacobvosmaer can you add details and push a new commit?

I don't think this post should go live before @jacobvosmaer had a chance to add more detail about premium storage

Title changed from GitLab.com migration to Azure to WIP: GitLab.com migration to Azure

I have to agree with myself here http://twitter.com/gitlabceohere/status/664547348161261568, in the end we own our availability. I made the following mistakes:

1. We didn't allow enough time to study the documentation
2. We didn't interview other companies about their experiences
3. We didn't have our setup reviewed by Azure experts
4. We didn't set up the support account before moving
5. We didn't run a staging environment

The gist of the article should be that we want to give people moving to Azure a heads up, but I'm 100% responsible for the downtime by insisting on moving before properly preparing.

Well never the less, having two portals, not able to create premium storage account via the portal but API only and a lot of functionalities missing in their ruby sdk doesn't help either.

And what should we say about the premium storage? The only benefit is when using the premium storage account you have 5000 IOPS per disk instead of 500 per disk. The total request rate limit for the storage account is the same namely 20000, so even when we already had premium storage we were hitting this limit anyway.

@jnijhof Thanks for the context.

@kelvinmutuma Can adapt the article based on my and Jeroen's comments?

Seems like we're not the only people struggling https://twitter.com/daraghward/status/665200555254530048

@jnijhof @sytses Is there anything we can currently do about the total request limit on Azure? Since if we are already hitting the max, does this not present a cause for alarm for our users?

Added 1 commit:

• d9699ecc - Include a conclusion and remove line breaks below headers when answer is in prose

@jnijhof I also don't understand why we have 2 portals instead on 1. Could you please clarify why this is the case?

Waiting for Jeroen's clarification so I can submit a hopefully closing commit.

Plus I can also make change to the last line of my new commit which currently reads

We are however dedicated to ensuring that GitLab.com is fast and stable and are apologetic for the recent troubles our users have had to go through.

A better way to put it would be:

We are however dedicated to ensuring that GitLab.com is fast and stable and are apologetic for the uptime troubles our users have had to endure since our migration

Is there anything we can currently do about the total request limit on Azure? Since if we are already hitting the max, does this not present a cause for alarm for our users?

Yes that's why we decided to migrate our virtual machines to multiple storage accounts which we already did.

I also don't understand why we have 2 portals instead on 1. Could you please clarify why this is the case?

The first portal is the "old" one which we actually use the for the most part and instead of upgrading Azure created a whole new portal next to this "old" one with some new features.

@sytses is this MR still waiting for my input?

@kelvinmutuma ping me once everything is solved and I'll edit for grammar and add a thank you.

@AshleyS I just got the input from Jeroen today which is what I was waiting for. Lemme finish up after the team call so that we can hopefully close this today.

Added 1 commit:

• a077c314 - Include Azure Premium Storage Details

@AshleyS You can now review and add a thank you

Added 1 commit:

• 019ea92e - updates to azure blog post

@sytses will you give a final review before I accept the MR? I changed the date to today.

For examples -> For example

Added 1 commit:

• 182f13c5 - updating example

Added 1 commit:

• 69947149 - wording updates

Reassigned to @nearlythere

@nearlythere to merge next week once final approval from @sytses - perhaps Wednesday blog post.

I think I made a mistake asking for this blog post. We should fix our problems and move on instead of pointing out problems on a platform that gave us free hosting and is trying really hard to help us. Sorry @kelvinmutuma, @AshleyS and others for the work.

Status changed to closed

@sytses Would it help if we reword to eliminate finger pointing? If we can post something about our recent troubles it will help users to understand and not be so upset. I think users will appreciate information about what we're doing to improve, too - eliminating SPOF, evaluating long term, planning for growth, etc. It's sort of a long-running post-mortem. Thanks for considering this.

@dblessing That would be nice. But I have a hard time seeing the content of that blog post not being about a lot of things we hit in Azure.

@luisrudge we currently have better support and our infrastructure become far more stable then ever been since we moved everything to PLRS storage.