Performance issues with Tmux on nightly iTerm2-3_0_20160720-nightly

Thanks for filing an issue! Please answer the questions below so I can help you.

• iTerm2 version: Build 3.0.20160720-nightly
• OS version: 10.11.3
• Attach ~/Library/Preferences/com.googlecode.iterm2.plist here (drag-drop from finder into this window) (I'm executing the nightly build directly from the Downloads folder so I don't know if the ~/Library/Preferences/com.googlecode.iterm2.plist belongs to the nightly build or to the other version I have installed. Please advise)
• Attach a debug log, if possible. Instructions at https://iterm2.com/debuglog
• Are you reporting a performance issue or a hang? Yes. Sample Sample_of_iTerm2.txt

Detailed steps to reproduce the problem:

1. Launch iterm2
2. Do a git log (git log og --graph --pretty=format:'%Cred%h%Creset - %C(bold blue)%an%Creset %Cgreen(%cr) %C(yellow)%d%Creset %s ' --abbrev-commit --date=relative)

What happened: The output lags as if tmux/iterm2 couldn't keep with the log. Also the CPU usage for iterm2 spikes close to 50%

What should have happened: The output should be smooth as in the stable version of iterm2 Build 3.0.4

Been getting this, too. Any screen updates in tmux panes have a noticeable delay. I have to stick with 3.0.20160713 for tmux to perform adequately.

You should get some improvement by turning on Prefs > Advanced > Use adaptive framerate.

Milestone changed to %Ship 3.0

Milestone changed to %Feature Complete 3.1

Just tried with 20160725. I'm not noticing any improvement using adaptive framerate. Here's what it looks like: https://share.esdf.io/amRvw4bK1z/

This is what it looks like with 20160713 (adaptive framerate is off): https://share.esdf.io/VLVtyVdYxN/

If it matters, I'm using Neovim with termguicolors enabled, and the full window is 293x95. The videos are Neovim running in tmux with 3 vertically split panes to the right with no text in them besides the shell prompt. This is over local SSH to an Ubuntu server.

I know the new way of drawing is slower than the old way. I'm working on a solution. It's a tricky problem.

FWIW the new way gives you ligatures and will fix all kinds of problems where non-ascii text was rendered incorrectly.

It's worth something, but I can't enjoy it ATM

It seems strange that the performance is drastically affected in tmux. The performance is reasonable using tmux integration, but it's something I can't get used to without the ^b prefix.

@tweekmonster Was that neovim shot in tmux? I didn't see a status bar and I can't get vim to let me drag pane dividers while in tmux.

@madtrick Is the performance issue you see when navigating around in less with up/down arrows?

Yeah, it's Neovim. Didn't realize you couldn't drag the status bars in Vim

I only used it as a example since it was easy to visually convey the problem. The performance is about the same if you scroll in a tmux pane or there's fast output printing. It's made worse if there's a lot of colored text.

I've spent quite a bit of time working on this, but no big wins so far. For future reference:

Draw text in multiple threads

Since text rendering is where we spend most of our time, I tried rendering text in a concurrent dispatch queue. Unfortunately the overhead of compositing it back into the graphics context canceled out the win from getting parallel rendering. I first tried CGContextDrawImage, which was horrible, and then CGLayer, which was fast but creating the layers was slow enough to exactly cancel out the performance benefit. This effort is in the parallelDrawing branch.

Create attributed strings and CTLineRef in multiple threads

The next-slowest part is creating attributed strings and core text lines. It's not that slow, though. Moving it into a concurrent queue didn't provide a measurable improvement. The overhead of shuffling between queues equaled the benefit from parallelism. This effort is in the parallelBuilder branch.

Use CALayers

Giving PTYTextView a layer kills performance, presumably because the layer is enormous. Giving SessionView a layer actually helps by about 10%! But it breaks transparency. So the next step is to make the layer present only when there's no transparency.

Next steps

What I haven't figured out yet is where all the time goes. If drawRect takes 0.3 seconds then the time between calls is like 0.46 seconds. Instruments shows 75% of main thread time is spent in drawRect but only 83.3% of main thread time is in my own code. I guess the rest of the time is copying pixels among internal buffers, which I have no control over. If I could just get a little more work done between -drawRect calls it would be lovely.

mentioned in issue #4957 (closed)

mentioned in issue #4964 (closed)

Any progress on this issue? Would love to see this resolved.

I just made a few commits that should help. Please try tomorrow's nightly build. You'll get the best performance bump if you use a font that does not have ligatures (so avoid Hasklig, PragmataPro, and OperatorMono).

Note to self: I still need to put a layer on SessionView when there is no transparency.

@gnachman any idea when the issue that affects fonts with ligatures will be solved? You are doing a great job BTW

@ahmedelgabri There's still some improvement with ligatures, but not as much as without them. The problem is to get ligatures I have to render with CoreText. It is several times slower than rendering text with Core Graphics. The optimization I made was to use Core Graphics for letters and numbers, which don't usually get ligatures, and Core Text for symbols and non-ASCII text. The ball's in Apple's court as far as I can tell. Terminal.app uses a dynamic refresh rate like iTerm2's adaptive refresh rate to work around the issues with CoreText's slowness.

Note to self: layers make a bigger difference on my iMac than what I'd mentioned earlier. Here are some stats:

With a layer:

2016-11-12 14:04:41.253024 iTerm2[9294:22041215]       Total drawRect: µ=18.341ms σ=13.008ms (95% CI ≅ 5.333ms–31.349ms) 𝚺=2732.78ms N=149 avg. events=1
2016-11-12 14:04:41.253061 iTerm2[9294:22041215]    Construct BG runs: µ=0.662ms σ=0.473ms (95% CI ≅ 0.188ms–1.135ms) 𝚺=98.58ms N=149 avg. events=80
2016-11-12 14:04:41.253088 iTerm2[9294:22041215]              Draw BG: µ=4.927ms σ=5.309ms (95% CI ≅ 0.000ms–10.236ms) 𝚺=734.11ms N=149 avg. events=1
2016-11-12 14:04:41.253113 iTerm2[9294:22041215]         Construction: µ=7.889ms σ=5.407ms (95% CI ≅ 2.482ms–13.296ms) 𝚺=1175.43ms N=149 avg. events=408
2016-11-12 14:04:41.253138 iTerm2[9294:22041215]   Build attr strings: µ=0.105ms σ=0.072ms (95% CI ≅ 0.033ms–0.176ms) 𝚺=15.58ms N=149 avg. events=492
2016-11-12 14:04:41.253162 iTerm2[9294:22041215]        Compute Attrs: µ=1.544ms σ=1.092ms (95% CI ≅ 0.452ms–2.637ms) 𝚺=230.09ms N=149 avg. events=29984
2016-11-12 14:04:41.253185 iTerm2[9294:22041215]              Segment: µ=0.138ms σ=0.096ms (95% CI ≅ 0.042ms–0.234ms) 𝚺=20.62ms N=149 avg. events=575
2016-11-12 14:04:41.253296 iTerm2[9294:22041215]             Advances: µ=0.648ms σ=0.462ms (95% CI ≅ 0.186ms–1.110ms) 𝚺=96.54ms N=149 avg. events=29902
2016-11-12 14:04:41.253322 iTerm2[9294:22041215]        Combine Attrs: µ=0.534ms σ=0.366ms (95% CI ≅ 0.169ms–0.900ms) 𝚺=79.63ms N=149 avg. events=575
2016-11-12 14:04:41.253346 iTerm2[9294:22041215]       Update Builder: µ=1.543ms σ=1.084ms (95% CI ≅ 0.458ms–2.627ms) 𝚺=229.87ms N=149 avg. events=29902
2016-11-12 14:04:41.253369 iTerm2[9294:22041215]              Drawing: µ=4.541ms σ=3.225ms (95% CI ≅ 1.316ms–7.766ms) 𝚺=676.61ms N=149 avg. events=408
2016-11-12 14:04:41.253394 iTerm2[9294:22041215]        Between calls: µ=15.297ms σ=17.232ms (95% CI ≅ 0.000ms–32.529ms) 𝚺=2279.23ms N=149 avg. events=1

Without a layer

2016-11-12 14:05:39.486680 iTerm2[9581:22045096]       Total drawRect: µ=25.297ms σ=10.356ms (95% CI ≅ 14.942ms–35.653ms) 𝚺=2529.74ms N=100 avg. events=1
2016-11-12 14:05:39.486711 iTerm2[9581:22045096]    Construct BG runs: µ=0.845ms σ=0.348ms (95% CI ≅ 0.497ms–1.193ms) 𝚺=84.50ms N=100 avg. events=101
2016-11-12 14:05:39.486737 iTerm2[9581:22045096]              Draw BG: µ=5.929ms σ=3.024ms (95% CI ≅ 2.905ms–8.954ms) 𝚺=592.95ms N=100 avg. events=1
2016-11-12 14:05:39.486762 iTerm2[9581:22045096]         Construction: µ=10.760ms σ=4.113ms (95% CI ≅ 6.647ms–14.873ms) 𝚺=1075.99ms N=100 avg. events=540
2016-11-12 14:05:39.486787 iTerm2[9581:22045096]   Build attr strings: µ=0.151ms σ=0.056ms (95% CI ≅ 0.095ms–0.207ms) 𝚺=15.11ms N=100 avg. events=663
2016-11-12 14:05:39.486811 iTerm2[9581:22045096]        Compute Attrs: µ=2.125ms σ=0.833ms (95% CI ≅ 1.292ms–2.958ms) 𝚺=212.50ms N=100 avg. events=40972
2016-11-12 14:05:39.486835 iTerm2[9581:22045096]              Segment: µ=0.208ms σ=0.079ms (95% CI ≅ 0.129ms–0.286ms) 𝚺=20.77ms N=100 avg. events=765
2016-11-12 14:05:39.486952 iTerm2[9581:22045096]             Advances: µ=0.854ms σ=0.336ms (95% CI ≅ 0.517ms–1.190ms) 𝚺=85.35ms N=100 avg. events=40870
2016-11-12 14:05:39.486990 iTerm2[9581:22045096]        Combine Attrs: µ=0.781ms σ=0.295ms (95% CI ≅ 0.487ms–1.076ms) 𝚺=78.14ms N=100 avg. events=765
2016-11-12 14:05:39.487020 iTerm2[9581:22045096]       Update Builder: µ=2.101ms σ=0.812ms (95% CI ≅ 1.289ms–2.913ms) 𝚺=210.06ms N=100 avg. events=40870
2016-11-12 14:05:39.487046 iTerm2[9581:22045096]              Drawing: µ=7.393ms σ=3.105ms (95% CI ≅ 4.287ms–10.498ms) 𝚺=739.26ms N=100 avg. events=540
2016-11-12 14:05:39.487073 iTerm2[9581:22045096]        Between calls: µ=25.069ms σ=9.951ms (95% CI ≅ 15.118ms–35.020ms) 𝚺=2506.91ms N=100 avg. events=1

Everyone please test the latest nightly build and let me know if you get weird rendering issues or missing ligatures. If you do, please attach your $HOME/Library/Preferences/com.googlecode.iterm2.plist to this issue with a screenshot.

@gnachman Just installed it & it's much better than before even with a ligatures enabled font, using Fira Code in my case, not as smooth as normal fonts but way better than before & I can live with that. I'll keep any eye on any rendering issues or missing ligatures.

I take that back about the ligatures enabled font still very sluggish, rest is really good

@ahmedelgabri I'd like to reproduce the sluggishness you report:

1. What kind of hardware are you on?
2. Can you provide a session log? Open a new tab, do Session>Log>Start, record yourself working for a while, Session>Log>Stop, and attach the log file here.
3. The font with sluggishness is Fira Code, I presume?

@gnachman

1. MacBook Pro (Retina, 15-inch, Mid 2015) running macOS sierra 10.12.1
2. I couldn't get the latest nightly build working at all, It just kept crashing, will try again later today hopefully. I tried:
   • with a clean com.googlecode.iterm2.plist instead of my own
   • tried installing it directly from the website instead of homebrew cask.
   • removing iterm2 files completely using brew cask zap iterm2-nightly or brew cask zap iterm2 or using App Cleaner when I installed it through the website.
3. Yeah the one I tried was Fira Code.

@gnachman this is what is currently happening with me when I try the nightly build. The terminal in the background is hyper just to remove confusion.

Sorry about the crashing issue, it has been resolved. Please try again with a newer nightly build.

@gnachman that's a session log using FiraCode fira-code

One thing to note that I'm usually inside a tmux session too.

@ahmedelgabri Can you test this build please:

http://iterm2.com/adhocbuilds/iTerm2-3_0_20161204_211223-adhoc.zip

I have a theory that there were unnecessary redraws when the cursor's visibility changed. The cursor will probably do funky things in this build, but I'm just interested in knowing if it helps with the performance issues you see.

@gnachman Been using that build that you sent nearly the whole day & it's definitely better than before. Still not as fast as when not using a font with ligatures. Older versions were not even usable for a couple of mins.

I will continue using it tomorrow too & see.

@gnachman been using that build for two days now, definitely a huge improvement.

@ahmedelgabri Thanks. The cursor fix is in commit 0fcf051a.

closed