Fix a bug where coords and content would not be in 1:1 correspondence when an url action is found and hard newlines are ignored (set by an advanced pref, which is not the default setting). Remove the corresponding coords as well as the newlines from the string. Issue 5929.

Fix a bug where the newline policy kiTermTextExtractorNullPolicyFromLastToEnd would include a leading newline after removing text prior to a null. This caused URL actions to detect a region that began one cell early. That null policy is only used in URL action detection.

Issue 5929

Prevent text extractor's line cache from outliving the vcall to performBlockWithLineCache, which caused a test to access freed memory

Add an optimization to the text extractor to cache the current line so repeated calls to characterAt: on the same line are faster. This speeds up double-click on long lines.

Change how the length limit in rangeForWordAt: works. The word is now truncated instead of returning nothing at all when it is too long. This is necessary since the limit was lowered from infinity in various places, such as semantic history.

Use binary search when trying to find indexes in sorted array. This speeds up finding word boundaries.

Limit the number of characters that are searched for a word to 100k (was unlimited) and also don't search for the bounds of a wrapped line farther than they could possibly extend when a limit is present. This helps but does not completely fix some hangs when there is a very long line in a narrow window. Issue 5855.

Also fix the menu opener to not select the last item.

Update OSC 8 to require parameters prior to the URL. A difference in parameters prevents hover-underlining together.

The new syntax is:

OSC 8 ; params ; url ST

params are (for now) space delimited ASCII and may not contain a ;.
url is a URL. I'm more accepting that I have to be and will take UTF-8 encoded URLs that aren't properly percent-escaped.

OSC 8 ; url ST Anchor text OSC 8 ; ST

If state restoration gives us an inconsistent state then CharToStr can return nil, which makes appendString: raise an exception. A colleague ran into this issue causing repeated crashes when restoring a tmux session.

Revert "Improve performance of enumerating characters by adding enumeration APIs to LineBlock, LineBuffer, and VT100Screen. Converting ascii screen_char_t to NSString was also optimized."

This reverts commit de3f4276.

Improve performance of enumerating characters by adding enumeration APIs to LineBlock, LineBuffer, and VT100Screen. Converting ascii screen_char_t to NSString was also optimized.

Fix incorrect tab filler orphan removal and tests that incorrectly assumed tab fillers followed a tab character. They precede it. Issue 5075

Change how orphan tab fillers are found in contentInRange:. An out-of-bounds lookup in eolBlock: was removed (tabFillerOrphansOnRow:line+1, when line is the last line). This should speed up the method by not creating lots of index sets and searching the line twice (except when there are lots of tab filler orphans, which is rare).

Extend contentInRange:... to support truncating the head of its result instead of always truncating the tail when cappedAtSize>=0. This fixes a bug where prefixes for semantic history got their tails truncated when ignoring hard newlines (or if the session is over 2000 cells wide). Issue 4902.

Trim trailing newline and adjacent whitespace when selecting the output of the last command. Issue 4975

Treat trailing spaces as nulls even when a windowed selection is not right aligned. Fixes problem where such lines get joined improperly at the cost of introducing the occiasonal bogus newline.

Improve performance when doing word selection. Reduce the longest word findable by smart selection in word selection mode to 1000 bytes. Do not try to do word selection in smart selection when an action is required (this greatly speeds up pressing cmd while hovering over a long line). Optimize how word strings are found when building up a string by enumerating backwards from the cursor position.

Fix a bug where -[iTermTextExtractor rangeForWord:] would crash when dealing with surrogate pairs and composed characters because it incorrectly assumed cells were 1:1 with characters in strings.

Fix bug where alt-click to move cursor on command line didn't handle double-width characters properly.

Add Select Current Command. Also adds rangeByTrimmingWhitespaceFromRange because tcsh will usually have whitespace due to its prompt limitation, so this is ncessary.

Fix a bug where double-width characters' DWC_RIGHT component was incorrectly included in string to which smart selection regexes were applied, essentially breaking smart selection for double-width characters. Modernize the smart selection code some and add a unit test for this bug. Some more details in issue 4325.

Improve double click selection: Fix bug where a double-click on the right half of a double-width character would not have an effect. Fix a bug where consecutive double-width characters would not get  selected on double-click. Fix a bug where Chinese words composed of more than one ideograph didn't select the full word on double click (issue 4325). Add a first test for iTermTextExtractor.

Use DEPRECATED_ATTRIBUTE to mark functions deprecated in the comments.

Make getting attributed strings from PTYTextView and iTermTextExtractor support the same features as non-attributed strings and remove some duplicated code.

Fix the broken PTYTextViewTest tests.

Add basic unit tests for getting selected (attributed) text from PTYTextView.

Fixes issue 3535.