Artifact hashes wrong on MacOS when depot resides on a FAT32 drive

[IanButterworth]

In testing performance related to slow disk access, I moved the depot onto a FAT32 formatted USB pen drive and noticed that the artifact hashes are repeatably different.

This is probably highly unlikely to happen in the wild, but I thought I'd report.

julia> versioninfo()
Julia Version 1.7.0-DEV.185
Commit 770e228c61 (2020-12-29 00:32 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin19.5.0)
  CPU: Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.0 (ORCJIT, skylake)
Environment:
  JULIA_NUM_THREADS = 6
  JULIA_DEPOT_PATH = /Volumes/pendrive/.julia

(@v1.7) pkg> add libpng_jll
   Resolving package versions...
   Installed libpng_jll ─ v1.6.37+6
 Downloading artifact: libpng
┌ Error: Tree Hash Mismatch!
│   Expected git-tree-sha1:   fb2695d9f2ba598ca2213b0768aac906ebcd1000
│   Calculated git-tree-sha1: 0e2015d6296e3f1119adcc39a621077dfe5ce50f
  Downloaded artifact: libpng

(@v1.7) pkg> st
Status `/Volumes/pendrive/.julia/environments/v1.7/Project.toml`
→ [b53b4c65] libpng_jll v1.6.37+6
Info packages marked with → not downloaded, use `instantiate` to download

(@v1.7) pkg> instantiate
 Downloading artifact: libpng
┌ Error: Tree Hash Mismatch!
│   Expected git-tree-sha1:   fb2695d9f2ba598ca2213b0768aac906ebcd1000
│   Calculated git-tree-sha1: 0e2015d6296e3f1119adcc39a621077dfe5ce50f
  Downloaded artifact: libpng
 Downloading artifact: libpng
┌ Error: Tree Hash Mismatch!
│   Expected git-tree-sha1:   fb2695d9f2ba598ca2213b0768aac906ebcd1000
│   Calculated git-tree-sha1: 0e2015d6296e3f1119adcc39a621077dfe5ce50f
  Downloaded artifact: libpng

julia> using libpng_jll
ERROR: InitError: Artifact "libpng" was not installed correctly. Try `using Pkg; Pkg.instantiate()` to re-install all missing resources.
Stacktrace:

[StefanKarpinski]

There's no end to the ways file systems can be broken. The only solution is to stop hashing on the file system and switch to verifying tarball hashes. I've been meaning to do this, but haven't gotten to it yet.

[IanButterworth]

There seems to be an additional bug, as the artifact is being installed with the calculated hash (0e2015d6296e3f1119adcc39a621077dfe5ce50f) rather than the expected hash, which makes the artifact unfindable

This should be a soft error, right? The artifact should still be loadable

[fredrikekre]

There seems to be an additional bug, as the artifact is being installed with the calculated hash (0e2015d6296e3f1119adcc39a621077dfe5ce50f) rather than the expected hash

Yes, #1885 was only a half fix since the artifact is not placed in the correct location heh. Has been known for quite some time, just never go around to fix it since hash computation has been pretty reliable for most people with "regular system" setups.

[StefanKarpinski]

@fredrikekre, since you seem to understand the issue, would you be willing to make a fix? Otherwise we should at least open a specific issue for that problem with a description of what needs to be fixed.

[fredrikekre]

Can you try with #2322 @ianshmean ?

[IanButterworth]

Great, that fixes it for the MacOS/FAT32 setup. Soft error, still loads

(jl_yU2xQX) pkg> activate --temp
  Activating new environment at `/var/folders/_6/1yf6sj0950vcg4t91m9ltb5w0000gn/T/jl_hMlQO9/Project.toml`

(jl_hMlQO9) pkg> add libpng_jll
   Resolving package versions...
   Installed libpng_jll ── v1.6.37+6
   Installed JLLWrappers ─ v1.1.4
 Downloading artifact: libpng
┌ Error: Tree Hash Mismatch!
│   Expected git-tree-sha1:   fb2695d9f2ba598ca2213b0768aac906ebcd1000
│   Calculated git-tree-sha1: 0e2015d6296e3f1119adcc39a621077dfe5ce50f
│ Moving artifact to the expected location.
  Downloaded artifact: libpng
Updating `/private/var/folders/_6/1yf6sj0950vcg4t91m9ltb5w0000gn/T/jl_hMlQO9/Project.toml`
  [b53b4c65] + libpng_jll v1.6.37+6
Updating `/private/var/folders/_6/1yf6sj0950vcg4t91m9ltb5w0000gn/T/jl_hMlQO9/Manifest.toml`
  [692b3bcd] + JLLWrappers v1.1.4
  [b53b4c65] + libpng_jll v1.6.37+6
  [0dad84c5] + ArgTools
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [f43a241f] + Downloads
  [b77e0a4c] + InteractiveUtils
  [b27032c2] + LibCURL
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [ca575930] + NetworkOptions
  [44cfe95a] + Pkg
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [fa267f1f] + TOML
  [a4e569a6] + Tar
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [deac9b47] + LibCURL_jll
  [29816b5a] + LibSSH2_jll
  Progress [========================================>]  5/5
5 dependencies successfully precompiled in 2 seconds

julia> using libpng_jll

julia>