Description
I've been doing some work that has a fairly big dependency stack (100-200 packages) and as such takes a few minutes to warm up. Because of this non-Revise-able code is extremely painful, so I'd like to take another look at timholy/Revise.jl#18. This has some fairly long history of course, but to avoid getting buried in the discussion, I figured a new issue for my specific proposal would be best.
One solution that was previously discussed (in #22721) was to have backedges for bindings, but this was rejected as too expensive. My proposal aims to basically do this, but shift the cost of backedge tracking to the redefinition stage where it is more acceptable.
In particular, my proposal is:
- Make each module's binding table partitioned by world ages. Introducing a new binding does not raise the world age, but bindings do record the world age where they were introduced.
- Change binding lookup to only match bindings that are visible to the current world age
- When wanting to redefine a binding (e.g. to replace a struct definition by an updated struct definition of the same name), we do a GC-like walk of all methods in the system and look at their source. If they contain a GlobalRef for the binding being replaced, we truncate the world bounds of any associated method instances (in effect having the GlobalRef serve as "forward edges" that get re-validated by the redefinition). This operation would increase the world age.
- We add an array for explicit forward edges from non-syntactic resolutions of bindings during inference. This can basically happen for two reasons:
- Something like
getfield(Main, :foo)which will get resolved via the getfield_tfunc. - Generated functions
- Something like
If this proposal seems reasonable, I'd hope to nerdsnipe @timholy into taking charge :)