Extend IP blocking mechanism

Closes #9092 (closed)

This will be an extension for changes done in !2515 (merged)

Changes in Application settings

before

after

Changes in Admin navigation

before

after - visible only when 'IP blocking' is enabled

IP blacklist management

IP whitelist management

DNS Blacklists management

DNS Whitelists management

Thanks for creating a WIP @tmaczukin

WIP Documentation in https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/2459

Added 59 commits:

• f3c9fd4b...1d5f96cf - 52 commits from branch master
• 0f57c966 - Create class for checking IP against DNSBLs
• 15f5c845 - Add dnsbl_check configuration in gitlab.yml
• 2d50b6ff - Move config loading to DNSBLCheck#create_from_config
• 71951d81 - Add check against DNSBL in RegistrationsController.create
• e1667b1e - Fix typo - 'threshold' instead of 'treshold'
• 124651c4 - Reorder 'describe' blocks in dnsbl_check_spec.rb
• 0e131f1c - Move 'dnsbl_check enabled' setting to ApplicationSetting model

Added 4 commits:

• 29efd4f9 - Change 'alias' to 'alias_method' in dnsbl_check_spec
• 55a2b77f - Add models for IP black/white listing and DNS black/white lists configuration
• 011a71cf - Modify registration dnsbl check - use database data instead of gitlab.yml
• 15da4a33 - Add IP Whitelist/Blacklist management

Added spam fighting label

Added 3 commits:

• cf6b9722 - Add DNS white/black lists management
• b107acbe - Views refactorisation
• 8e69f56c - Modify IP check step in registrations_controller and DNSXLCheck library

Added 70 commits:

• 8e69f56c...a382ad99 - 54 commits from branch master
• bb53b647 - Create class for checking IP against DNSBLs
• 897dbd2f - Add dnsbl_check configuration in gitlab.yml
• 11deb14d - Move config loading to DNSBLCheck#create_from_config
• 8e3ca426 - Add check against DNSBL in RegistrationsController.create
• 9f68e6eb - Fix typo - 'threshold' instead of 'treshold'
• 18bb22b4 - Reorder 'describe' blocks in dnsbl_check_spec.rb
• 48e2b2b6 - Move 'dnsbl_check enabled' setting to ApplicationSetting model
• f1f24425 - Change 'alias' to 'alias_method' in dnsbl_check_spec
• 3f670a20 - Add models for IP black/white listing and DNS black/white lists configuration
• 93384e86 - Modify registration dnsbl check - use database data instead of gitlab.yml
• efd5c95c - Add IP Whitelist/Blacklist management
• 898463b3 - Add DNS white/black lists management
• 3d444313 - Views refactorisation
• 7cdd11e1 - Modify IP check step in registrations_controller and DNSXLCheck library
• eddc6d97 - Remove unimplemented specs and factories
• 6fd05c1b - Update CHANGELOG

Title changed from [WIP] Add check against DNSBL at signup to Add check against DNSBL at signup

@rspeicher Please review the changes :)

It seems there's a ton of duplication in the various controllers, and I'm wondering if we'd benefit from a base controller they could inherit from.

@dzaporozhets Please weigh in. This is bigger than I anticipated and I don't think I'd be comfortable squeezing this into 8.4 at the last second.

On second thought maybe it's best to split this into two distinct MRs:

1. Auto-blocking against RBLs, immediate in 8.4.
2. Custom blacklisting/whitelisting for 8.5.

@rspeicher I think, that this is a good idea. I'll split this MR into two new:

1. One with RBL check against hard-coded lists. That MR will contain only basic settings (enabled/disabled and thresholds).
2. Second with the other stuff (local blacklist/whitelist and DNS lists configuration).

In that case we will have also time to remove code duplication from controllers. I'm aware of this duplicates, and I was already thinking about creating some base controller. But at the end I've decided, that - as for now - more important is to finish the implementation, and that the refactorization could be done after release.

Added 1 commit:

• 92025670 - Add some modifications

Why is Blacklist/Whitelist singular for IP, but plural for DNS? I'm not sure I understand what these DNS lists are :)

For DNS lists: https://en.wikipedia.org/wiki/DNSBL

IP blacklist/whitelist is GitLab internal list of IPs marked as blacklisted or whitelisted.

DNS blacklists/whitelists - is a list of DNSBL servers used for blacklisting/whitelisting

Thats why I use plural for DNS: this is a list of lists :). And for IP: list of adresses.

Title changed from Add check against DNSBL at signup to [WIP] Add check against DNSBL at signup

@DouweM @yorickpeterse: Thank's for Your comments! I'm currently splitting this MR into two other, but I'll take them into account.

Title changed from [WIP] Add check against DNSBL at signup to [WIP] Extend IP blocking mechanism

mentioned in merge request !2515 (merged)

Milestone changed to 8.5

Added 3 commits:

• 20869fd5 - Add IP blocking against DNSBL at sign-up
• ac9ec4fb - View refactorisation
• aa690d22 - Fix 'use_threshold' usage inside of DNSXLCheck

Added 1 commit:

• 1b9ce6e2 - Fix migration name

mentioned in commit 1553c560

mentioned in commit 6a5cd3ca

Added 128 commits:

• 1b9ce6e2...a4ff270d - 127 commits from branch master
• bf64752c - Merge branch 'master' into feature/check-against-rbl

Added 2 commits:

• 3817ff7b - Move abstraction to base controllers in Admin::IpBlocking::
• 98e4977a - Fix rendering 'active' element in 'Ip Blocking' navigation