feat(WIP): python parser #21
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
1. Since clangd does not support semanticTokens/range method, use semanticTokens/full + filtering to emulate. 2. Since the concept of package and module does not apply to C/C++, treat the whole repo as a single package/module.
Custom pylsp is based on [python-lsp-server](https://github.com/python-lsp/python-lsp-server), and plus the following pull requests: 1. semanticTokens/full: python-lsp/python-lsp-server#645 2. typeDefinition: python-lsp/python-lsp-server#533 Maybe also 3. implementation: python-lsp/python-lsp-server#644
Parses and generates json. Tons of ad hoc decisions.
Python LSP does not print `def` or `(` as semtokens, but we need their position to determine which semantic tokens are in parameters or return values. So add a manual parsing based on `sym.Text`.
…mbols For example, a class `Foo` has a method `bar`. When trying to infer the symbol from a location within `bar`, we follow getSymbolByToken -> getSymbolByLocation -> filterEntitySymbols. The `getSymbolByLocation` presents two candidates `Foo` and `bar`. We should accept the `bar` because it is most specific. Existing Rust implementation avoids the problem because `Foo` will be an impl symbol, which is not an entity symbol. However in Python, `Foo` is a class and thus has to be an entity symbol.
Hoblovski
force-pushed
the
feat/python-parser
branch
from
42d5d6a to
537a2ed
Compare
|
一些输出的例子 testdata/pysimpleobj：单文件，简单的 python class testdata/pythonsingle：单文件，比较复杂 testdata/pythonsimple：多文件多模块，外部依赖，+OOP method |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
实现 python 的 parser，基于修改后的 python-lsp-server
（暂时先不合）
修改后的 python-lsp-server 在 https://github.com/Hoblovski/python-lsp-server/tree/abc 。
安装直接
python -m pip install -e .，然后检查pylsp是否在 PATH 里。Python 的一个问题是，相当多代码没有类型标注，导致函数依赖不太明确。
例如
fn foo(x: Foobar)明显有个依赖 foo -> Foobar，但是def foo(x)就不行，除非写成def foo(x: Foobar)。一个办法是用类型推断工具自动加一些类型标注，但是效果有限而且疑似有些复杂了。
另外一个问题是效率比较低，我没有修改 collect 的逻辑，导致 parse 一个 astropy 会特别特别慢（大概几个小时都做不完）。
现在在 perf 代码调优……
最后关于实现
class Foo {\n def foobar... }Check the PR title.
(Optional) Translate the PR title into Chinese.
(Optional) More detailed description for this PR(en: English/zh: Chinese).
en:
zh(optional):
(Optional) Which issue(s) this PR fixes:
(optional) The PR that updates user documentation: