Raise errors when "# noqa" lines don't have any issues

Please describe how you installed Flake8

On Ubuntu Artful:

$ apt install flake8

Please provide the exact, unmodified output of flake8 --bug-report

{
  "dependencies": [
    {
      "dependency": "setuptools",
      "version": "36.0.1"
    }
  ],
  "platform": {
    "python_implementation": "CPython",
    "python_version": "3.5.3+",
    "system": "Linux"
  },
  "plugins": [
    {
      "plugin": "mccabe",
      "version": "0.6.1"
    },
    {
      "plugin": "pycodestyle",
      "version": "2.3.1"
    },
    {
      "plugin": "pyflakes",
      "version": "1.5.0"
    }
  ],
  "version": "3.2.1"
}

Please describe the problem or feature

I would like to be able to instruct flake8 to error out when a "# noqa" comment doesn't cause an error to be ignored. This would help us keep our codebase as flake'd as possible, because we'd be forced to remove linting ignores as they are fixed or become irrelevant.

As a specific example, we have code which currently causes an F401 because the imports are only used in Python typing comments. A reductive case:

from typing import Optional  # noqa: F401

def foo():
    # type: () -> Optional[int]
    ...

Once #118 has been addressed, this line should no longer be raising an F401, so I would like to be forced to remove the noqa comment. (This would then mean, after the removal, that we would correctly catch the addition of unused typing imports.)

This makes good sense. I'm not sure when #118 will be addressed but this shouldn't be blocked on this.

changed milestone to %3.x.0

added feature:accepted priority:medium labels

@sigmavirus24 I had a cursory look around the code to work out how to do this, and I couldn't really figure out where it would fit in. Do you have a high-level idea of how this would be implemented that I could have a stab at?