Skip to content

Review masking and add traces when things don't cancel timely#2768

Merged
pepeiborra merged 14 commits into
masterfrom
fix-excessive-masking
Mar 11, 2022
Merged

Review masking and add traces when things don't cancel timely#2768
pepeiborra merged 14 commits into
masterfrom
fix-excessive-masking

Conversation

@pepeiborra

Copy link
Copy Markdown
Collaborator

No description provided.

@pepeiborra

pepeiborra commented Mar 9, 2022

Copy link
Copy Markdown
Collaborator Author

Who reviews changes made by the code owner? I need a volunteer once again, thanks @wz1000 @michaelpj @eddiemundo

Comment thread hls-graph/src/Development/IDE/Graph/Internal/Database.hs Outdated

@wz1000 wz1000 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable.

@michaelpj michaelpj left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the first time I've looked at the AIO stuff. It makes me a little nervous: this stuff is fiendishly difficult, can we definitely not get away with an existing library?

-- Masked to ensure that the previous values are flushed together with the map update
-- update the map
mask_ $ join $ atomicallyNamed "modifyFileExists" $ do
join $ mask_ $ atomicallyNamed "modifyFileExists" $ do

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change doesn't make sense to me. I think this just makes the mask apply to the atomic execution of the STM block, which is already atomic. Previously it ensured that we would perform the effects of the returned action as well without being interrupted. But now I think it doesn't do that, and indeed maybe does nothing?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It still prevents the STM transaction from being interrupted by an edit, which is important to accurately track file dirtiness.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we can end up not calling recordDirtyKeys, since that happens in the IO action afterwards. Is that important? Maybe the comment could say.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The IO is just logging, recordDirtyKeys mutates the collection inside the STM transaction. That said I appreciate how this is confusing and perhaps the mask_ was better at the top level for clarity

Comment thread ghcide/src/Development/IDE/Core/Shake.hs Outdated
-> [Key] -- ^ Previous direct dependencies of Id
-> HashSet Key -- ^ Current direct dependencies of Id
-> IO ()
-- mask to ensure that all the reverse dependencies are updated

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one I wonder if we could avoid by pushing the scope of the STM transaction out a bit?

@pepeiborra pepeiborra Mar 10, 2022

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

STM transactions can still be interrupted by exceptions, so the question is how badly do we want to record the reverse deps. The answer is very badly - failure to do so will lead to unsoundness, specially if this is the first build.

EDIT: oh, correcting myself, since we don't mark the build as done until after the reverse keys have updated, I don't think that the mask is needed at all.

unless (null asyncs) $ do
let loop = forever $ do
sleep 10
traceM "cleanupAsync: waiting for asyncs to finish"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really want to use Debug.Trace here?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you are seeing this trace, then most likely hls-graph is broken.

Plus, I didn't want to thread a logger around....

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're in IO, you could just print to stderr. It's a small difference, but I do think not having Debug.Trace in production code is good.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a fair point

Comment thread hls-graph/src/Development/IDE/Graph/Internal/Database.hs Outdated
Comment thread hls-graph/src/Development/IDE/Graph/Internal/Database.hs Outdated
Comment thread hls-graph/src/Development/IDE/Graph/Internal/Database.hs Outdated
@pepeiborra

Copy link
Copy Markdown
Collaborator Author

This is the first time I've looked at the AIO stuff. It makes me a little nervous: this stuff is fiendishly difficult, can we definitely not get away with an existing library?

AIO is just a reader monad to keep track of the asyncs in flight, right? Surely you mean something else

@michaelpj

Copy link
Copy Markdown
Collaborator

I guess I mean the functions operating on AIO. Perhaps they would exist in an equally gnarly form without AIO, I don't know. I trust your ability to write them, I don't trust my ability to review whether the masking is sensible :D

@pepeiborra

Copy link
Copy Markdown
Collaborator Author

AIO is just a thin wrapper around IO/Async. The gnarliness existed before AIO, it's evil magic contributed by @ndmitchell who, in turn, blames @simonmar.

@pepeiborra
pepeiborra merged commit fbbf76b into master Mar 11, 2022
@pepeiborra
pepeiborra deleted the fix-excessive-masking branch March 11, 2022 11:37
July541 pushed a commit to July541/haskell-language-server that referenced this pull request Mar 30, 2022
…l#2768)

* Review masking and add traces when things don't cancel timely

* fixup

* use sleep consistently

* redundant imports

* hlints

* fix 9.2 build

* Fix 9.2 build for real

* remove unnecessary polymorphism

* Avoid spawning loop async unnecessrily

* flush asyncs ref

* Add comments and apply @michaelpj suggestions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants