v3: Flickering to iTerm2 background color when scrolling inside Emacs.
By Brian.J.... on November 01, 2014 22:04 (imported from Google Code)
I have git bisected and found out that 02bffb5d is the commit where this behavior first occurs.
Here's a short video clip showing two builds of iTerm2, parent commit of 02bffb5d on the left, 02bffb5d on the right, and the scrolling behavior in Emacs.
https://www.youtube.com/watch?v=fAVN251zyYU&feature=youtu.be
iTerm2 background color: rgb(50,50,50)
Emacs background color: rgb(63,63,63)
Activity
Comment 2 by Brian.J.... on November 09, 2014 04:24
I can't reproduce it anymore on 02bffb5d. I fortunately saved the old iTerm.app showing the problem and was able to diff the newly generated iTerm.app without the problem. Besides the binary being different, Info.plist was also different with a new value for key BuildMachineOSBuild. Then I remembered that I updated from OS X Yosemite Public Beta 6 to 10.10.1 prerelease. So something might have changed with Xcode.
I then tested master and I definitely can see it still on master. I will git bisect again....
-
Comment 3 by Brian.J.... on November 09, 2014 04:56
Ok I got it to reproduce on 02bffb5d.
I'm not sure the exact requirements to trigger the bug, but resizing vertically or pressing Command to show underlines seem to help trigger it.Yes, commenting the body of removeUnderline got rid of the flickering problem.
-
Comment 4 by gnach... on November 09, 2014 06:38
The question then is why removeUnderline is getting called when all you're doing is scrolling around in emacs. Can you add this statement to the body of removeUnderline to see why it gets called when scrolling:
NSLog(@"%@", [NSThread callStackSymbols]);
Then scroll around in emacs to repro the flicker and see what call stacks are responsible. I agree that resizing and pressing Cmd will do it, but I don't believe that's what I saw in the video--looked like you were just holding down ^V.
-
Comment 5 by Brian.J.... on November 09, 2014 16:46
It seems [PTYTextView mouseEntered] is spammed when the mouse cursor is inside iTerm.
http://youtu.be/DILYdGbA31U
You can see the time stamp update constantly. -
Comment 6 by gnach... on November 10, 2014 01:18
Commit 82b63842 should fix this; it only removes the underline if one is present.
Status: Fixed
-
Comment 8 by Brian.J.... on November 10, 2014 04:46
Sorry to be the bearer of bad news, but the flickering persists on master.
I did another git bisect with removeUnderline commented out for each commit.
git bisect start
# bad: [82b63842] Don't remove an underline unless there is an underline.
git bisect bad 82b63842
# good: [02bffb5d] Underline clickable URLs when cmd is held down
git bisect good 02bffb5d
# good: [02bffb5d] Underline clickable URLs when cmd is held down
git bisect good 02bffb5d
# bad: [e12620a7] Add support for saved search keyboard shortcuts and, correspondingly, temporary searches in the FindViewController.
git bisect bad e12620a7
# bad: [e12620a7] Add support for saved search keyboard shortcuts and, correspondingly, temporary searches in the FindViewController.
git bisect bad e12620a7
# good: [a6bc6e5d] Merge branch 'master' of https://github.com/gnachman/iTerm2
git bisect good a6bc6e5d
# bad: [00c406ac] Finish converting VT100Terminal's public APIs into properties
git bisect bad 00c406ac
# good: [2c4d3cf3] Merge branch 'master' of https://github.com/gnachman/iTerm2
git bisect good 2c4d3cf3
# bad: [145e6143] Merge branch 'master' of https://github.com/gnachman/iTerm2
git bisect bad 145e6143
# bad: [32694f91] Fix size of about box
git bisect bad 32694f91
# bad: [0ff995ef] Change how redraw timer is fired to be more consistently paced during cursor movement
git bisect bad 0ff995ef
# skip: [403d5c08] Fix merge conflicts
git bisect skip 403d5c08
# good: [2051d8a1] Clean up PreferencePanel.h somewhat and make changes to a divorced session's profile caused by a change to the original profile be reflected in the Edit Current Session preference panel
git bisect good 2051d8a1
# good: [18663b68] Fix how divorced sessions update their profiles by finding changes in the shared profile or session profile and updating overridden fields appropriately.
git bisect good 18663b68
# good: [18663b68] Fix how divorced sessions update their profiles by finding changes in the shared profile or session profile and updating overridden fields appropriately.
git bisect good 18663b68
# good: [045fc523] Fix bug where iTermSearchField would capture arrow key presses even when it wasn't key. Became an issue first with the command history tool. Fixes bug 2865.
git bisect good 045fc523
# first bad commit: [0ff995ef] Change how redraw timer is fired to be more consistently paced during cursor movement -
Comment 9 by Brian.J.... on November 10, 2014 05:16
To help you reproduce this issue I tried to recreate with "emacs -q -nsl", which loads Emacs without any configuration.
I loaded the "misterioso" color theme, which I think is bundled with Emacs, but any color theme that has a sufficiently different background color from iTerm should work.
Then scroll down holding or . (redrawing nearly all or all of the screen) doesn't result in a flicker. It seems it only flickers when redrawing around half the screen, so wait for the cursor to reach the bottom and the next or should result in a flicker. -
Comment 11 by gnach... on November 10, 2014 06:02
I'm not an emacs user, so I'm not quite sure how to load color themes. I tried M-x load-theme and entered misterioso when prompted but I don't have it, and I can't see how to list the themes I do have.
I suspect that what you're hitting is a timing issue where the speed we redraw at doesn't quite mesh with emacs's update rate. It repeatedly clears and redraws the screen, and we get unlucky sometimes and redraw our view between emacs clearing and drawing the screen. Ideally we'd only draw our view when emacs had just finished drawing the latest page of data, but there's no way to know that this is what's going on. I think that the fact that it used to work was luck, basically, and I expect you'd have different results on a faster or slower machine, over ssh, a different key repeat rate, or if your CPU utilization was different. If I'm right that this is the issue, tweaking the timing might fix it for you (for now) but would probably cause this issue for another group of users.
That being said, the commit you found has two independent changes:
-
The "super fast" update interval was removed. We used to redraw the screen almost immediately after a very small bit of data was received on the tty. In this change, we now wait .1 seconds to redraw if we get less than 1k of data. This was a performance optimization since .1 seconds is not perceptible, and the number of bytes of data read isn't a great predictor of whether we're in a state where we should draw less to conserve CPU for data processing vs screen drawing.
-
The timer for redrawing counts time elapsed since the last draw against the time desired until the next draw. This helps keep the time between draws consistent. This change is limited to the area around line 3005 in PTYSession.m.
I'm curious which change is responsible for what you're seeing.
-
-
Comment 13 by Brian.J.... on November 10, 2014 06:20
- The timer for redrawing counts time elapsed since the last draw against the time desired until the next draw. This helps keep the time between draws consistent. This change is limited to the area around line 3005 in PTYSession.m.
Reverting the hunks for this change fixes the problem.
-
Comment 15 by gnach... on November 11, 2014 06:52
Hm, the fact that it's negative is interesting. I can't see how that would actually happen unless time is not monotonic, which I've heard of but not on Mac OS. If you increase 0 to .017 in the expression MAX(0, timeout - timeSinceLastUpdate) does that help?
-
Comment 16 by Brian.J.... on November 11, 2014 23:02
I don't think it breaks monotonicity.
Here's a screenshot of NSLog, the first column being timeout and the 2nd column being timeSinceLastUpdate. When it switches timeout from 0.5 to 0.0333333, there's a good chance the time since the last timeout is > 0.0333333. The .017 helps a little, but the best result is when the timer is set for timeout.What seems to be happening is that Emacs will output some VT100 codes for scrolling and maybe clearing the screen where the new text would go, then at a later time output the new text that would be in the cleared region, instead of batching everything together.
-
Comment 17 by gnach... on November 13, 2014 23:38
Maybe the right thing to do is to allow a bit of extra time after the screen is cleared? I don't know, the problem is that when you're holding down C-v that the clear the screen-draw the screen cycle repeats really fast, and there's no way to pick a timing that will be correct in every case.
I guess the "real" right answer is to add double-buffering escape codes to VT100. I wonder if the emacs folks would go for that :)
-
@naseer-ahmed Hm, really shouldn't (I was trying to find this issue to close it, actually). If you have Prefs>Advanced>Terminal: Avoid repainting the screen when the cursor is hidden turned on then you'd only see flicker on very slow ssh connections. Can you make a session log (Shell>Log>Start, do some scrolling, Shell>Log>Stop) and attach it?
Attached
-
@naseer-ahmed What you're seeing must be different than what this flag was trying to fix. It looks like emacs scrolls by atomically moving all the lines up/down by one row, hiding the cursor, drawing the exposed line, and showing the cursor. Could you make a screen cap with quicktime so I can see what you're experiencing? The log looks OK to me when I replay it, but there can be subtle timing issues here.
yes - that's right. I tried switching both color schemes to zenburn, but there is still a problem since the two color schemes don't seem to match. This issue only happens in emacs (I'm using spacemacs) - vim with a light background is fine. I'm still not sure if the blame here lies in emacs or in iTerm2 - it does seem to work fine in Terminal.app
-
I do get the occasional bit of background color showing through in terminal. Their refresh rate drops a lot while repeatedly cat'ing that file for some reason. The issue is that emacs shows the cursor briefly between scrolling and drawing the newline revealed line of text. Here's a typical sequence:
^[[?25h^[[?12;25h^[]50;CursorShape=0^G^[[1;39r^[[1;1H^[[1M^[[1;41r^[[39;1H^[[?25l- Show the cursor
- Start blinking the cursor & show the cursor
- Set the cursor shape (this is an iTerm2-specific extension you or spacemacs added)
- Set the scrolling region to rows 1-39
- Move the cursor to (1,1)
- Delete one line (scrolling those below up)
- Set the scrolling region to rows 1-41
- Move the cursor to (y=39,x=1)
- Hide the cursor
It's dumb to show the cursor when all emacs is doing is scrolling a bunch of lines up; they don't want the cursor shown then. This confuses the Avoid repainting while the cursor is hidden trick, which is an unfortunate side effect of this wart.
Note to self: I think I can work around it by extending the algorithm to keep the double-buffered grid around for a very short amount of time after the cursor is shown.
Reassigned to @gnachman
Awesome - thanks for looking at it @gnachman
-
I looked at this some more and I don't see a way to make emacs scroll nicely when it's drawing its background color in this inherently broken way. I've moved the preference into prefs>profiles>session so you have more granular control over when it's on, since the benefit it confers is real for most users. IMO emacs has a bug in the way it draws and it should be improved.
Fixed in 5c79a19e which was used to fix #3773 (closed)
