Skip to content

CI: caching: fix early termination expression check & cabal.project replacement#2520

Merged
mergify[bot] merged 9 commits into
haskell:masterfrom
Anton-Latukha:2021-12-21-ci-fxs
Dec 22, 2021
Merged

CI: caching: fix early termination expression check & cabal.project replacement#2520
mergify[bot] merged 9 commits into
haskell:masterfrom
Anton-Latukha:2021-12-21-ci-fxs

Conversation

@Anton-Latukha

@Anton-Latukha Anton-Latukha commented Dec 21, 2021

Copy link
Copy Markdown
Collaborator

Details are in the commit messages.

  1. We not noticed the first specific of the CI, because the early cutoff behaved as it should. It was respected to return 'false' to terminate early, but that check always returned 'false'. When working in GHC 9.2 PR could not force the caching build to start, because check that I fix - always skipped the steps (always returned False). New check is done in the way how it is done in other if: ... != 'true' statements in the workflow.

  2. The second specific is both Ubuntu-specific & maybe GitHub specific. Some OSes run alias cp='cp -i' & when I tried to rm manually - cabal.project appeared to be protected from simple rm by setinning (most probably) of a "sticky bit" on $HOME directory. I found that-out by working in GHC 9.2 PR (where index-state is actually different between .project files & noticed that CI cache key was with the same inferred index-state in all jobs). Now rm -f forced & that allows removing file with a sticky bit. log reports the situation of files changes there: https://github.com/haskell/haskell-language-server/runs/4597911492?check_suite_focus=true#step:14:460

@Anton-Latukha

Anton-Latukha commented Dec 21, 2021

Copy link
Copy Markdown
Collaborator Author

Ok. Now lets actually check the early termination & after that the new key (should go into full run).

@Anton-Latukha

Copy link
Copy Markdown
Collaborator Author

In the
https://github.com/haskell/haskell-language-server/runs/4596861600?check_suite_focus=true
of
haskell#2503

Noted that:

I rm cabal.project. The sources of cabal-ghc921.project contain:
index-state: 2021-12-18T00:00:07Z
cp cabal-ghc921.project cabal.project
Then from cabal.project CI retrieves finderprint 2021-11-29T08:11:08Z.
Seems like GitHub uses alias cp=cp -i - preventing UNIX-like default copying over behaviour.
& rm cabal.project does not remove the file.
cabal.project seems to be protected from deletion.
@Anton-Latukha

Anton-Latukha commented Dec 21, 2021

Copy link
Copy Markdown
Collaborator Author

I advanced the index-state.

Which changes the key. So the old key of cache & new id for cache of the workflow does not align & so old storage is loaded, but the full build&cache process starts & then saves into the new key.

Resulting into the proper long caching run: https://github.com/haskell/haskell-language-server/runs/4599160496?check_suite_focus=true

@Anton-Latukha

Copy link
Copy Markdown
Collaborator Author

Ok, now I need to wait until this run completes & then drop the modifications. & have a 1 clean run on {test,bench}, it is honest to do at least because I changed test workflow.

@Anton-Latukha

Anton-Latukha commented Dec 21, 2021

Copy link
Copy Markdown
Collaborator Author

Ok, now run is complete https://github.com/haskell/haskell-language-server/pull/2520/checks.

By dropping the WIP commit - disabling caching & enabling {test,bench}.

@Anton-Latukha

Copy link
Copy Markdown
Collaborator Author

@Anton-Latukha
Anton-Latukha marked this pull request as ready for review December 21, 2021 21:32
@Anton-Latukha

Anton-Latukha commented Dec 21, 2021

Copy link
Copy Markdown
Collaborator Author

Seems I checked everything about early termination this time.

& seem that cabal.project switcharoo now works.

I hope we solved the main problems there & hope that soon we would exhaust hidden from us bugs in the CI to become sure in configuration robustness.

@jneira jneira left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, good catches, thanks

Comment thread .github/workflows/caching.yml Outdated
Comment thread .github/workflows/caching.yml Outdated
@jneira jneira added the merge me Label to trigger pull request merge label Dec 22, 2021
@mergify
mergify Bot merged commit e6c1fca into haskell:master Dec 22, 2021
@Anton-Latukha

Anton-Latukha commented Dec 22, 2021

Copy link
Copy Markdown
Collaborator Author

Would note, that now because according .project are loaded & tested under - CI pipeline would ship according context & so uncover some related to that effects/bugs inside pull request processes.

@Anton-Latukha
Anton-Latukha deleted the 2021-12-21-ci-fxs branch December 22, 2021 13:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge me Label to trigger pull request merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants