It's created when Runner with shell executor is used

This is a cause of problems for users. Although we already made fixing
it easy (gdk reconfigure) it was still hard for users to detect /
understand what was wrong.

Thanks Stan!