Ruby Arena object should use xrealloc/xfree so Ruby is aware of allocated memory

[haberman]

Once the upb arena can report the number of bytes allocated (https://github.com/protocolbuffers/upb/issues/528) we should make the Ruby Arena object report bytes allocated via the dfree member of Arena_type.

This will hopefully convince Ruby to run the GC more often and prevent the growing memory usage we have seen reported in #9467.

[fowles]

We can test this theory simply by lying to it and reporting large numbers and seeing what happens

[zhangskz]

After trying Matt's suggestion to report arbitrary large number (also adding print statement to sanity check) showed that implementing dsize (note: initial issue has typo dfree -> dsize) doesn't help since dsize never gets called by GC.

However, Josh found this lead which seems much more promising: ffi/ffi#753
Adding a ruby-specific upb_alloc using xmalloc/xrealloc/xfree in place of upb_global_alloc for Arena_alloc (by using upb_Arena_Init instead of upb_Arena_New) should allow Ruby to be aware of the memory being allocated.

[zhangskz]

Looks like this worked. Will send out fix shortly.

Results using the script from #9467 with local source (gem "google-protobuf", :path => "~/protobuf/ruby/")

Encoded payload length: 1888895
Memory before test loop:    rss 43,976    vsz 1,168,080
Starting test loop...
1..200:  3.6 ms/rep    rss 134,880    vsz 1,260,180
201..400:  2.7 ms/rep    rss 134,880    vsz 1,260,180
401..600:  2.7 ms/rep    rss 134,880    vsz 1,260,180
601..800:  2.7 ms/rep    rss 134,880    vsz 1,260,180
801..1000:  2.9 ms/rep    rss 81,368    vsz 1,204,764
1001..1200:  2.9 ms/rep    rss 90,888    vsz 1,215,924
1201..1400:  2.9 ms/rep    rss 88,280    vsz 1,214,104
1401..1600:  2.9 ms/rep    rss 67,688    vsz 1,191,084
1601..1800:  3.0 ms/rep    rss 82,732    vsz 1,213,476
1801..2000:  3.0 ms/rep    rss 90,028    vsz 1,215,060
2001..2200:  3.0 ms/rep    rss 90,736    vsz 1,218,912
2201..2400:  2.9 ms/rep    rss 92,932    vsz 1,218,156
2401..2600:  3.0 ms/rep    rss 89,500    vsz 1,214,700
2601..2800:  3.0 ms/rep    rss 91,276    vsz 1,218,156
2801..3000:  3.6 ms/rep    rss 92,596    vsz 1,218,372
3001..3200:  4.0 ms/rep    rss 92,764    vsz 1,218,084
3201..3400:  3.3 ms/rep    rss 92,668    vsz 1,218,156
3401..3600:  3.5 ms/rep    rss 89,500    vsz 1,214,700
3601..3800:  3.9 ms/rep    rss 89,416    vsz 1,215,240
3801..4000:  4.1 ms/rep    rss 120,416    vsz 1,250,892
4001..4200:  3.2 ms/rep    rss 120,416    vsz 1,250,892
4201..4400:  2.7 ms/rep    rss 120,416    vsz 1,250,892
4401..4600:  2.7 ms/rep    rss 120,416    vsz 1,250,892
4601..4800:  2.7 ms/rep    rss 120,416    vsz 1,250,892
4801..5000:  2.7 ms/rep    rss 120,416    vsz 1,250,892
Total test time: 16.0 seconds
Memory after final GC:    rss 120,416    vsz 1,250,892

[haberman]

Fixed by: #9586