Start using and suggesting standardized Tree-sitter highlights captures

[uncenter]

Check for existing issues

• Completed

Describe the feature

Zed currently uses non-standard captures in its builtin extensions/languages and themes, which has led to community language and theme extensions doing so as well. This is detrimental, as the non-standard subset of keywords Zed uses limit the precision of Tree-sitter highlighting. (This is distinctly different from #9656, where the injection captures are different in a manner that is actually breaking for Zed.) Standardized keywords can be found from https://neovim.io/doc/user/treesitter.html#treesitter-highlight-groups and some from https://docs.helix-editor.com/themes.html#scopes.

1. Zed's documented list of captures - https://zed.dev/docs/extensions/languages#syntax-highlighting - should be updated/expanded, or refer to other sources (such as the ones I linked previously).
2. Zed's builtin themes (this repository) should add support for more detailed captures.
3. Zed's builtin languages (in both crates/languages/ and in extensions/) should copy over queries from https://github.com/nvim-treesitter/nvim-treesitter/tree/master/queries where possible, which are - in most cases - significantly better than what exists so far in Zed.
4. Some sort of notice should be provided to theme authors that they should support these new/different captures now being used.
   • I've done some analysis with a tool I've built for analyzing Zed extensions' support by themes and usage by languages of certain captures, and unfortunately literally only 3 extensions support captures like keyword.type.

For the most part, 1) and 3) can be achieved somewhat easily. The docs update is small, and there aren't too many language extensions to begin with in this repository. However 2) and 4), theme support, is trickier - we don't want syntax highlighting regressions (which could occur by improving the captures without theme support), only improvements. I don't know where to begin updating Zed's builtin themes, and I'm not sure there exists a good way to announce changes like this to extension authors.

Some anecdotal? evidence of this:

• Allow keyword to have different colors #22138

Examples of improvements via adopting nvim-treesitter queries:

• Improve diff syntax highlighting queries #21740

Zed Version and System Specs

N/A

If applicable, add mockups / screenshots to help present your vision of the feature

No response

[jpike88]

@notpeter I'm not sure why this got changed to discussion and left to collect dust... it's a major shortcoming of what's supposed to be an IDE, good syntax/semantic highlighting is critical for properly usability/accessibility/ergonomics. As a product manager I'd consider things like these to at the top of the list of priorities, they're just as important as adding more bells and whistles.

If zed's using its own in-built captures that are inferior to the already existing ones written offered by the different tree sitter languages, why?

[notpeter]

This hasn't been moved to a discussion to gather dust, but because in it's current state it's not scoped to makes it actionable. It needs more definition before it can be even be considered, let alone prioritized.

The characterization of Neovim's captures as "standardized" and Zed's as "non-standard" is (I believe) is arbitrary. Both implementations support features which are not present in upstream, so both are "non-standard".

I am not a tree-sitter expert, but my understanding it NeoVim queries may use lua expressions -- this is very much Neovim specific and in order to support them Zed would to either ship with an embedded Lua interpreter or implement the subset of Lua necessary to support those captures.

In theory, I am (personally) in favor of a convergence of Neovim/Zed tree-sitter queries to reduce the requirement for duplicated knowledge and effort, but it is by no means a trivial effort. It would require someone with a deep knowledge tree-sitter, Zed and Neovim -- and would require enumerating all of the divergences, implement features so both Zed and Neovim both support the converged "standardized" set and then have a migration path of thousands (or possibly tens of thousands) of existing tree-sitter queries for hundreds of languages. And it assumes any discovered conflicts (Zed / NeoVim want a particular implementation because it better aligns with the existing datastructures and needs of their editor) can be resolved.

@jpike88 I think it's easy to say "I'd consider things like these to at the top of the list of priorities" when it's unscoped and you're not offering to do the work. Please don't tag zed staff members, it's notification spam and is quick way to demonstrate that you don't value our time. Don't do it.

That said, if anyone wants to dig in, I would love to see incremental work towards convergence (on both sides).

[colemcanelly]

And it assumes any discovered conflicts (Zed / NeoVim want a particular implementation because it better aligns with the existing datastructures and needs of their editor) can be resolved.

From light research and reading this thread, this seems like this is the crux of what is missing before any of these changes are actionable. @uncenter, would you happen to know what the current datastructures / needs of Zed look like, and where to find more in the code? I would imagine the tree sitter captures are crucial for navigation, repl/runnables, in addition to highlights, so we would first need to isolate and document where the captures are currently used before we can make any of these improvements to highlighting.

After we have that information, we could make a translation table from old captures → new captures, and plan out the code changes that need to take place in Zed, in addition to your steps 1-4.

I would be happy to get the ball rolling on some of the "incremental work" so that this can be a reality in Zed, I just don't have context for what Zed uses the results of the tree sitter queries for.

[uncenter]

I think the easiest, most impactful improvement would be to adopt a more complete set of captures/scopes for Tree-sitter queries. You could go with Helix's, or Neovim's. I come across this issue often - today, catppuccin/zed#107 - where having better, more detailed queries means themes and languages can have better experiences. Is that actionable enough?

[notpeter]

Asking for Zed to adopt a more complete set of captures/scopes is not actionable (for me to implement at least).

Can you link to Helix or Neovim's captures/scopes?

[uncenter]

For sure, sorry-

Helix: https://docs.helix-editor.com/themes.html#syntax-highlighting
Neovim: https://github.com/nvim-treesitter/nvim-treesitter/blob/main/CONTRIBUTING.md#highlights (same list also at https://neovim.io/doc/user/treesitter.html#treesitter-highlight-groups)

Neovim itself did a semi-adoption of Helix-style captures a few years back, might be helpful for context nvim-treesitter/nvim-treesitter#4799.

[daivinhtran]

I could see why this is a large scope of work and potentially mandates cross-collaboration with other editors if done right.

In the meantime, I fixed the highlighting for addition/deletion in #38384.