Skip to content

add Method_TextDocumentSemanticTokensFullDelta#4073

Merged
fendor merged 28 commits into
masterfrom
4072-textdocumentsemantictokensfulldelta
Feb 21, 2024
Merged

add Method_TextDocumentSemanticTokensFullDelta#4073
fendor merged 28 commits into
masterfrom
4072-textdocumentsemantictokensfulldelta

Conversation

@soulomoon

@soulomoon soulomoon commented Feb 13, 2024

Copy link
Copy Markdown
Collaborator

Mainly the following things have been done:

  1. add semanticTokensCache and semanticTokensId to shake extras.
  2. implement Method_TextDocumentSemanticTokensFullDelta
  3. add a few test to test Method_TextDocumentSemanticTokensFullDelta
  4. remove persistentGetSemanticTokensRule since it is of no use since we do not use useWithStaleFastMT for semantic tokens rule.

@soulomoon soulomoon linked an issue Feb 13, 2024 that may be closed by this pull request
@soulomoon
soulomoon marked this pull request as ready for review February 13, 2024 16:01
@michaelpj

Copy link
Copy Markdown
Collaborator

I'm on holiday but I'll take a look next week!

@fendor fendor left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good to me, I mostly have minor suggestions.

While I do approve, let's wait until michael also had time to look at this :)

Comment thread plugins/hls-semantic-tokens-plugin/src/Ide/Plugin/SemanticTokens/Internal.hs Outdated
Comment thread plugins/hls-semantic-tokens-plugin/test/SemanticTokensTest.hs Outdated
Comment thread plugins/hls-semantic-tokens-plugin/test/SemanticTokensTest.hs Outdated
Comment thread plugins/hls-semantic-tokens-plugin/test/SemanticTokensTest.hs Outdated
Comment thread plugins/hls-semantic-tokens-plugin/test/SemanticTokensTest.hs Outdated
Comment thread ghcide/src/Development/IDE/Core/Shake.hs Outdated
Comment thread plugins/hls-semantic-tokens-plugin/src/Ide/Plugin/SemanticTokens/Internal.hs Outdated
Comment thread plugins/hls-semantic-tokens-plugin/src/Ide/Plugin/SemanticTokens/Internal.hs Outdated
Comment thread plugins/hls-semantic-tokens-plugin/src/Ide/Plugin/SemanticTokens/Internal.hs Outdated
soulomoon and others added 6 commits February 15, 2024 03:08
Co-authored-by: fendor <fendor@users.noreply.github.com>
…ns/Internal.hs

Co-authored-by: fendor <fendor@users.noreply.github.com>
@soulomoon

soulomoon commented Feb 14, 2024

Copy link
Copy Markdown
Collaborator Author

Changes look good to me, I mostly have minor suggestions.

Thanx for the reviews @fendor . Fixed most of them.

While I do approve, let's wait until michael also had time to look at this :)

Sure.

@michaelpj michaelpj left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly worried about adding lots of stuff to ShakeExtras if we don't have to.

Comment thread plugins/hls-semantic-tokens-plugin/src/Ide/Plugin/SemanticTokens/Internal.hs Outdated
,publishedDiagnostics :: STM.Map NormalizedUri [Diagnostic]
-- ^ This represents the set of diagnostics that we have published.
-- Due to debouncing not every change might get published.
,semanticTokensCache:: STM.Map NormalizedUri SemanticTokens

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these could probably be variables local to the plugin? I think we only need to put thing in here if they're really shared across multiple rules etc.

So the function that creates the plugin would run in IO or similar and create the TVars, which then get passed into the handlers.

@soulomoon soulomoon Feb 19, 2024

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree that putting it in shakeExtras is suboptimal, but I am also not sure about local state to plugin too.
Since no other plugins have their own plugin state. It would mean we need to change idePlugins :: Recorder (WithPriority Log) -> IdePlugins IdeState to idePlugins :: Recorder (WithPriority Log) -> IO (IdePlugins IdeState).
And open a door to introduce state to plugins breaking the current centralized state management in the hls build system. 🤔

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think plugin-local state is inherently problematic. But I admit that it's not totally clear to me what stuff should be in ShakeExtras 🤔 thoughts @fendor ?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added a comment to the shakeExtra stating it might not be ideal to store the varaibles there, but we do not find a better place to store it yet.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not have any specific thoughts, if the variable could be easily plugin local, I think I would prefer that.

Comment thread plugins/hls-semantic-tokens-plugin/src/Ide/Plugin/SemanticTokens/Internal.hs Outdated
Comment thread plugins/hls-semantic-tokens-plugin/src/Ide/Plugin/SemanticTokens/Internal.hs Outdated
let hsFinder = idSemantic getTyThingMap (hieKindFunMasksKind hieKind) refMap
return $ computeRangeHsSemanticTokenTypeList hsFinder virtualFile ast

-- | Persistent rule to ensure that semantic tokens doesn't block on startup

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why drop this?

@soulomoon soulomoon Feb 19, 2024

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For semantic tokens,
lastValueIO return [](case with persistent key) or Nothing(case without persistent key).
Between faking a stale value [] and mark the semantic tokens rules' value actually Failed in shake state.
The former would hide the failure in both user of the rule and in the shake caching result(I don't think it is wise, we want to know if we fail in this case).

Also we are not using IdeAction with useWithStaleFastMT(Which look directly into the caching and demand instance response) for semantic tokens as in other IdeAction that requires instance response(e.g. hover, this fix startup of hover when the rule first compute without any stale value in the cach).

The persistent rule is rather useless for us and might shadow some actual failures.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, so does this mean we have to wait quite a while to get responses when we start up?

@soulomoon soulomoon Feb 19, 2024

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, as we have been. Persistent rule won't help us here(Since it is not ideAction, the persistent rule serves only as a fall back when the actual computation of the rule failed and not already having a cache) .

@soulomoon

soulomoon commented Feb 19, 2024

Copy link
Copy Markdown
Collaborator Author

Thanx for the reviews, welcom back from vacation! @michaelpj , I fixed some of them, and some might need discussion.

@michaelpj michaelpj left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wz1000 still interested in your feedback on the ShakeExtras vs plugin-local state vars question, but for now we're going to leave it as it is with a note.

,hiddenDiagnostics :: STMDiagnosticStore
,publishedDiagnostics :: STM.Map NormalizedUri [Diagnostic]

-- storing semantic tokens for each file in shakeExtras might

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this comment is in the wrong place, it's interrupting the haddock for the field above

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe put it in a Note or something

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. Did it

@fendor fendor left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving, LGTM :)

@fendor
fendor merged commit 24b40ca into master Feb 21, 2024
soulomoon added a commit to soulomoon/haskell-language-server that referenced this pull request Feb 23, 2024
* add Method_TextDocumentSemanticTokensFullDelta

* remove persistentGetSemanticTokensRule

* add doc about semanticTokensCache location

* add Note [Semantic Tokens Cache Location]

---------

Co-authored-by: fendor <fendor@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

textDocument/semanticTokens/full/delta

3 participants