[improve][build] Allow building and running tests on JDK 24 and upcoming JDK 25 LTS

[lhotari]

Motivation

As a preparation for JDK 25 LTS version, Pulsar build should allow running with JDK 24 and with JDK 25 once it becomes available so that we can start fixing the different compatibility issues.

One of the already detected issues is that the Security Manager has been removed in JDK 24 with "JEP 486: Permanently Disable the Security Manager". This breaks at least tiered storage since it uses Hadoop libraries which depend on Security Manager (issue: HDFS-17778). There are also failing tests in Pulsar IO RabbitMQ connector which use Qpid libraries which require Security Manager features.

Modifications

• change enforcer plugin config to allow building with JDK 24 and JDK 25 in addition to JDK 17 and JDK 24.
• add required JVM arguments for JDK 24+
• fix previous issue where Pulsar CI Flaky tests aren't built and run with JDK 21 by default
• Add JDK 24 and JDK 25 choices for triggered Pulsar CI and Pulsar CI Flaky GitHub Actions workflows
• Run Shade integration tests with JDK 24 too
• Fix issue in building the Docker image for JDK 24

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

[dao-jun]

Goooood job！

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 74.15%. Comparing base (bbc6224) to head (d5e7121).
Report is 1085 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #24268      +/-   ##
============================================
+ Coverage     73.57%   74.15%   +0.57%     
+ Complexity    32624    32565      -59     
============================================
  Files          1877     1866      -11     
  Lines        139502   145040    +5538     
  Branches      15299    16577    +1278     
============================================
+ Hits         102638   107552    +4914     
- Misses        28908    28940      +32     
- Partials       7956     8548     +592     

Flag	Coverage Δ
inttests	26.72% <ø> (+2.13%)	⬆️
systests	23.28% <ø> (-1.04%)	⬇️
unittests	73.64% <ø> (+0.80%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 1083 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.