workaround for buggy lexers
It seems that some lexers can be buggy when used together with git blame.
This results in a 500 error when a user tries to open the blame page in some cases.
The workaround is pretty simple. Not only fall back to plaintext when the file content can't be guessed, but also when the lexer fails.
This closes #1521 (closed) and #1646 (closed)
/cc @DouweM @JobV It would be awesome to have this in a patch release for 7.11 or even in the release. But that's probably to late.
Merge request reports
Activity
mentioned in issue #1646 (closed)
Added 1 commit:
- dfce7b62 - workaround for buggy lexers
mentioned in commit 9fe8c74d
Thanks @haynes!
I agree the fallback should be to plain text, but I don't think we should close out #1521 (closed) yet. The issue is that line-by-line syntax highlighting confuses Rugments/rouge, and the right behavior either involves making the formatter/lexer work this way or running the entire file through in one pass.
This MR will make the resulting file look a little funny: because each line is formatted independently, some lines will be highlighted, while others will not.
@stanhu I wasn't even aware formatting is done per-line, that's interesting. I'll get this in 7.11, we can look at a proper solution for 7.12. I'll reopen the issue.
mentioned in commit 98f12863
9 9 10 10 begin 11 11 lexer = Rugments::Lexer.guess(filename: blob_name, source: blob_content) 12 rescue Rugments::Lexer::AmbiguousGuess 12 result = formatter.format(lexer.lex(blob_content)).html_safe 13 rescue 
The previous rescue arugment only took into account when Rugments couldn't guess a file.
But lexer.lex can fail as well even with the correct syntax highlighting detected.
This would result in errors like this for example:RuntimeError - empty stack!: rugments (1.0.0.beta6) lib/rugments/regex_lexer.rb:227:in `state' rugments (1.0.0.beta6) lib/rugments/regex_lexer.rb:268:in `stream_tokens' rugments (1.0.0.beta6) lib/rugments/lexer.rb:376:in `lex' rugments (1.0.0.beta6) lib/rugments/formatters/html.rb:72:in `process_tokens' rugments (1.0.0.beta6) lib/rugments/formatters/html.rb:42:in `format_untableized' rugments (1.0.0.beta6) lib/rugments/formatters/html.rb:35:in `format' app/helpers/blob_helper.rb:16:in `highlight' app/views/projects/blame/show.html.haml:39:in `block (2 levels) in _app_views_projects_blame_show_html_haml__4561567618660177775_70233404832100' app/views/projects/blame/show.html.haml:38:in `block in _app_views_projects_blame_show_html_haml__4561567618660177775_70233404832100' gitlab_git (7.1.2) lib/gitlab_git/blame.rb:20:in `block in each' gitlab_git (7.1.2) lib/gitlab_git/blame.rb:14:in `each' app/views/projects/blame/show.html.haml:14:in `_app_views_projects_blame_show_html_haml__4561567618660177775_70233404832100' actionview (4.1.9) lib/action_view/template.rb:145:in `block in render' activesupport (4.1.9) lib/active_support/notifications.rb:161:in `instrument' actionview (4.1.9) lib/action_view/template.rb:339:in `instrument' actionview (4.1.9) lib/action_view/template.rb:143:in `render' rack-mini-profiler (0.9.0) lib/mini_profiler/profiling_methods.rb:108:in `block in profile_method' actionview (4.1.9) lib/action_view/renderer/template_renderer.rb:55:in `block (2 levels) in render_template' actionview (4.1.9) lib/action_view/renderer/abstract_renderer.rb:38:in `block in instrument' activesupport (4.1.9) lib/active_support/notifications.rb:159:in `block in instrument' activesupport (4.1.9) lib/active_support/notifications/instrumenter.rb:20:in `instrument' activesupport (4.1.9) lib/active_support/notifications.rb:159:in `instrument' actionview (4.1.9) lib/action_view/renderer/abstract_renderer.rb:38:in `instrument' actionview (4.1.9) lib/action_view/renderer/template_renderer.rb:54:in `block in render_template' actionview (4.1.9) lib/action_view/renderer/template_renderer.rb:62:in `render_with_layout' actionview (4.1.9) lib/action_view/renderer/template_renderer.rb:53:in `render_template' actionview (4.1.9) lib/action_view/renderer/template_renderer.rb:17:in `render' actionview (4.1.9) lib/action_view/renderer/renderer.rb:42:in `render_template' actionview (4.1.9) lib/action_view/renderer/renderer.rb:23:in `render' actionview (4.1.9) lib/action_view/rendering.rb:99:in `_render_template' actionpack (4.1.9) lib/action_controller/metal/streaming.rb:217:in `_render_template' actionview (4.1.9) lib/action_view/rendering.rb:82:in `render_to_body' actionpack (4.1.9) lib/action_controller/metal/rendering.rb:32:in `render_to_body' actionpack (4.1.9) lib/action_controller/metal/renderers.rb:32:in `render_to_body' actionpack (4.1.9) lib/abstract_controller/rendering.rb:25:in `render' actionpack (4.1.9) lib/action_controller/metal/rendering.rb:16:in `render' actionpack (4.1.9) lib/action_controller/metal/instrumentation.rb:41:in `block (2 levels) in render' activesupport (4.1.9) lib/active_support/core_ext/benchmark.rb:12:in `block in ms' /home/hannes/.rubies/ruby-2.1.5/lib/ruby/2.1.0/benchmark.rb:294:in `realtime' activesupport (4.1.9) lib/active_support/core_ext/benchmark.rb:12:in `ms' actionpack (4.1.9) lib/action_controller/metal/instrumentation.rb:41:in `block in render' actionpack (4.1.9) lib/action_controller/metal/instrumentation.rb:84:in `cleanup_view_runtime' activerecord (4.1.9) lib/active_record/railties/controller_runtime.rb:25:in `cleanup_view_runtime' actionpack (4.1.9) lib/action_controller/metal/instrumentation.rb:40:in `render' actionpack (4.1.9) lib/action_controller/metal/implicit_render.rb:10:in `default_render' actionpack (4.1.9) lib/action_controller/metal/implicit_render.rb:5:in `send_action' actionpack (4.1.9) lib/abstract_controller/base.rb:189:in `process_action' actionpack (4.1.9) lib/action_controller/metal/rendering.rb:10:in `process_action' actionpack (4.1.9) lib/abstract_controller/callbacks.rb:20:in `block in process_action' activesupport (4.1.9) lib/active_support/callbacks.rb:113:in `call' activesupport (4.1.9) lib/active_support/callbacks.rb:166:in `block in halting' activesupport (4.1.9) lib/active_support/callbacks.rb:166:in `block in halting' activesupport (4.1.9) lib/active_support/callbacks.rb:166:in `block in halting' activesupport (4.1.9) lib/active_support/callbacks.rb:166:in `block in halting' activesupport (4.1.9) lib/active_support/callbacks.rb:166:in `block in halting' activesupport (4.1.9) lib/active_support/callbacks.rb:229:in `block in halting' activesupport (4.1.9) lib/active_support/callbacks.rb:149:in `block in halting_and_conditional' activesupport (4.1.9) lib/active_support/callbacks.rb:149:in `block in halting_and_conditional' activesupport (4.1.9) lib/active_support/callbacks.rb:166:in `block in halting' activesupport (4.1.9) lib/active_support/callbacks.rb:166:in `block in halting' activesupport (4.1.9) lib/active_support/callbacks.rb:166:in `block in halting' activesupport (4.1.9) lib/active_support/callbacks.rb:166:in `block in halting' activesupport (4.1.9) lib/active_support/callbacks.rb:166:in `block in halting' activesupport (4.1.9) lib/active_support/callbacks.rb:166:in `block in halting' activesupport (4.1.9) lib/active_support/callbacks.rb:166:in `block in halting' activesupport (4.1.9) lib/active_support/callbacks.rb:229:in `block in halting' activesupport (4.1.9) lib/active_support/callbacks.rb:166:in `block in halting' activesupport (4.1.9) lib/active_support/callbacks.rb:166:in `block in halting' activesupport (4.1.9) lib/active_support/callbacks.rb:166:in `block in halting' activesupport (4.1.9) lib/active_support/callbacks.rb:86:in `run_callbacks' actionpack (4.1.9) lib/abstract_controller/callbacks.rb:19:in `process_action' actionpack (4.1.9) lib/action_controller/metal/rescue.rb:29:in `process_action' actionpack (4.1.9) lib/action_controller/metal/instrumentation.rb:31:in `block in process_action' activesupport (4.1.9) lib/active_support/notifications.rb:159:in `block in instrument' activesupport (4.1.9) lib/active_support/notifications/instrumenter.rb:20:in `instrument' activesupport (4.1.9) lib/active_support/notifications.rb:159:in `instrument' actionpack (4.1.9) lib/action_controller/metal/instrumentation.rb:30:in `process_action' actionpack (4.1.9) lib/action_controller/metal/params_wrapper.rb:250:in `process_action' activerecord (4.1.9) lib/active_record/railties/controller_runtime.rb:18:in `process_action' actionpack (4.1.9) lib/abstract_controller/base.rb:136:in `process' actionview (4.1.9) lib/action_view/rendering.rb:30:in `process' rack-mini-profiler (0.9.0) lib/mini_profiler/profiling_methods.rb:108:in `block in profile_method' actionpack (4.1.9) lib/action_controller/metal.rb:196:in `dispatch' actionpack (4.1.9) lib/action_controller/metal/rack_delegation.rb:13:in `dispatch' actionpack (4.1.9) lib/action_controller/metal.rb:232:in `block in action' actionpack (4.1.9) lib/action_dispatch/routing/route_set.rb:82:in `dispatch' actionpack (4.1.9) lib/action_dispatch/routing/route_set.rb:50:in `call' actionpack (4.1.9) lib/action_dispatch/journey/router.rb:73:in `block in call' actionpack (4.1.9) lib/action_dispatch/journey/router.rb:59:in `call' actionpack (4.1.9) lib/action_dispatch/routing/route_set.rb:685:in `call' request_store (1.0.5) lib/request_store/middleware.rb:9:in `call' rack-cors (0.2.9) lib/rack/cors.rb:54:in `call' rack-attack (4.2.0) lib/rack/attack.rb:104:in `call' warden (1.2.3) lib/warden/manager.rb:35:in `block in call' warden (1.2.3) lib/warden/manager.rb:34:in `call' rack (1.5.2) lib/rack/etag.rb:23:in `call' rack (1.5.2) lib/rack/conditionalget.rb:25:in `call' rack (1.5.2) lib/rack/head.rb:11:in `call' actionpack (4.1.9) lib/action_dispatch/middleware/params_parser.rb:27:in `call' actionpack (4.1.9) lib/action_dispatch/middleware/flash.rb:254:in `call' rack (1.5.2) lib/rack/session/abstract/id.rb:225:in `context' rack (1.5.2) lib/rack/session/abstract/id.rb:220:in `call' actionpack (4.1.9) lib/action_dispatch/middleware/cookies.rb:562:in `call' activerecord (4.1.9) lib/active_record/query_cache.rb:36:in `call' activerecord (4.1.9) lib/active_record/connection_adapters/abstract/connection_pool.rb:621:in `call' actionpack (4.1.9) lib/action_dispatch/middleware/callbacks.rb:29:in `block in call' activesupport (4.1.9) lib/active_support/callbacks.rb:82:in `run_callbacks' actionpack (4.1.9) lib/action_dispatch/middleware/callbacks.rb:27:in `call' actionpack (4.1.9) lib/action_dispatch/middleware/reloader.rb:73:in `call' actionpack (4.1.9) lib/action_dispatch/middleware/remote_ip.rb:76:in `call' better_errors (1.0.1) lib/better_errors/middleware.rb:84:in `protected_app_call' better_errors (1.0.1) lib/better_errors/middleware.rb:79:in `better_errors_call' better_errors (1.0.1) lib/better_errors/middleware.rb:56:in `call' actionpack (4.1.9) lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call' actionpack (4.1.9) lib/action_dispatch/middleware/show_exceptions.rb:30:in `call' railties (4.1.9) lib/rails/rack/logger.rb:38:in `call_app' railties (4.1.9) lib/rails/rack/logger.rb:20:in `block in call' activesupport (4.1.9) lib/active_support/tagged_logging.rb:68:in `block in tagged' activesupport (4.1.9) lib/active_support/tagged_logging.rb:26:in `tagged' activesupport (4.1.9) lib/active_support/tagged_logging.rb:68:in `tagged' railties (4.1.9) lib/rails/rack/logger.rb:20:in `call' quiet_assets (1.0.2) lib/quiet_assets.rb:18:in `call_with_quiet_assets' actionpack (4.1.9) lib/action_dispatch/middleware/request_id.rb:21:in `call' rack (1.5.2) lib/rack/methodoverride.rb:21:in `call' rack (1.5.2) lib/rack/runtime.rb:17:in `call' rack (1.5.2) lib/rack/lock.rb:17:in `call' actionpack (4.1.9) lib/action_dispatch/middleware/static.rb:84:in `call' lib/gitlab/middleware/static.rb:9:in `call' rack (1.5.2) lib/rack/sendfile.rb:112:in `call' rack-mini-profiler (0.9.0) lib/mini_profiler/profiler.rb:298:in `call' railties (4.1.9) lib/rails/engine.rb:514:in `call' railties (4.1.9) lib/rails/application.rb:144:in `call' railties (4.1.9) lib/rails/railtie.rb:194:in `method_missing' rack (1.5.2) lib/rack/builder.rb:138:in `call' rack (1.5.2) lib/rack/urlmap.rb:65:in `block in call' rack (1.5.2) lib/rack/urlmap.rb:50:in `call' unicorn (4.6.3) lib/unicorn/http_server.rb:552:in `process_client' unicorn-worker-killer (0.4.2) lib/unicorn/worker_killer.rb:51:in `process_client' unicorn (4.6.3) lib/unicorn/http_server.rb:632:in `worker_loop' unicorn (4.6.3) lib/unicorn/http_server.rb:500:in `spawn_missing_workers' unicorn (4.6.3) lib/unicorn/http_server.rb:511:in `maintain_worker_count' unicorn (4.6.3) lib/unicorn/http_server.rb:277:in `join' unicorn (4.6.3) bin/unicorn_rails:209:in `<top (required)>' () home/hannes/.gem/ruby/2.1.5/bin/unicorn_rails:23:in `<main>'I don't know in what other ways the lexer can fail, so I decided to rescue from everything here and fall back to the plaintext lexer if there is an error.
It's a workaround until we can properly fix how we handle lexers.
In the meantime this at least prevents 500 errors.See the two linked bugs linked in the MR for more infos.
Please register or sign in to reply