[fix][broker] Unregister non-static metrics collectors registered in Prometheus default registry

[lhotari]

Motivation

For upgrading Mockito to 5.17, it's necessary that already shutdown Pulsar broker test instances don't get referenced later. This is due to the way how Mockito mocks are cleaned up after each test. In newer Mockito versions, an exception will be thrown if such accesses happen.
Most metric collectors are static in Pulsar, but there are 2 instances in the broker where the collector dynamically gets the value. These should be unregistered at shutdown.

Modifications

• add unregister logic to ObserverGauge class
• unregister pendingTopicLoadRequests and pendingLookupRequests at shutdown

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 74.18%. Comparing base (bbc6224) to head (d34afbc).
Report is 1073 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #24257      +/-   ##
============================================
+ Coverage     73.57%   74.18%   +0.61%     
+ Complexity    32624    32202     -422     
============================================
  Files          1877     1866      -11     
  Lines        139502   144978    +5476     
  Branches      15299    16561    +1262     
============================================
+ Hits         102638   107552    +4914     
- Misses        28908    28922      +14     
- Partials       7956     8504     +548     

Flag	Coverage Δ
inttests	26.72% <45.45%> (+2.14%)	⬆️
systests	23.27% <45.45%> (-1.06%)	⬇️
unittests	73.65% <100.00%> (+0.81%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...rg/apache/pulsar/broker/service/BrokerService.java	83.86% <100.00%> (+3.08%)	⬆️
...broker/stats/prometheus/metrics/ObserverGauge.java	100.00% <100.00%> (ø)

... and 1080 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.