Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-17611

Remove ZK from Kafka 4.0

    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 4.0.0
    • None
    • None

    Description

      Remove ZK from Kafka 4.0

      Attachments

        Issue Links

          is blocked by

          Improvement - An improvement or enhancement to an existing feature or task. KAFKA-17916 Convert Kafka Connect system tests to use KRaft

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Test - A new unit, integration or system test. KAFKA-14572 Migrate EmbeddedKafkaCluster used by Streams integration tests from EmbeddedZookeeper to KRaft

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Task - A task that needs to be done. KAFKA-14569 Migrate EmbeddedKafkaCluster used by Connect integration tests from EmbeddedZookeeper to KRaft

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Task - A task that needs to be done. KAFKA-17609 Convert Kafka Streams system tests to use KRaft

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Task - A task that needs to be done. KAFKA-17915 Convert Kafka Client system tests to use KRaft

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Task - A task that needs to be done. KAFKA-17925 Convert Kafka Client integration tests to use KRaft

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved
          is related to

          Task - A task that needs to be done. KAFKA-18364 migrating from zk to kraft document

          • Major - Major loss of function.
          • Resolved
          1.
          		 Remove some tests that only apply to ZK mode or migration Sub-task Resolved Colin McCabe
          2.
          		 Remove ZK migration code Sub-task Resolved Colin McCabe
          3.
          		 Remove AclAuthorizer Sub-task Resolved PoAn Yang
          4.
          		 Remove KafkaServer references in tests Sub-task Resolved Mickael Maison
          5.
          		 Remove KafkaServer Sub-task Resolved Mickael Maison
          6.
          		 Remove zk type and instance from ClusterTest Sub-task Resolved PoAn Yang
          7.
          		 Remove the E2E uses of accessing ACLs from zk Sub-task Resolved TengYao Chi
          8.
          		 Remove ZK from ducktape in 4.0 Sub-task Resolved Kevin Wu
          9.
          		 Remove BrokerRegistrationRequestTest and remove EmbeddedZookeeper from RaftClusterInvocationContext Sub-task Resolved TengYao Chi
          10.
          		 Remove kafka.security.authorizer.AclAuthorizer from AclCommand Sub-task Resolved Mickael Maison
          11.
          		 Remove ZK from Kafka READMEs in 4.0 Sub-task Resolved Ming-Yen Chung
          12.
          		 Remove KafkaServer references in tools tests Sub-task Resolved TengYao Chi
          13.
          		 Remove KafkaServer references in streams tests Sub-task Resolved Mickael Maison
          14.
          		 Convert MultipleListenersWithSameSecurityProtocolBaseTest to KRaft Sub-task Resolved Colin McCabe
          15.
          		 Set default test quorum to kraft Sub-task Resolved Colin McCabe
          16.
          		 Remove zk from BaseConsumerTest Sub-task Resolved PoAn Yang
          17.
          		 remove junit-platform.properties Sub-task Resolved Chia-Ping Tsai
          18.
          		 Remove zookeeper_migration_test.py Sub-task Resolved Ming-Yen Chung
          19.
          		 Remove ZooKeeper from jmh-benchmarks Sub-task Resolved Mickael Maison
          20.
          		 Remove --zookeeper option and logic from ConfigCommand Sub-task Resolved Cheng-Yan Wang
          21.
          		 Default all JUnit tests to KRaft Sub-task Resolved Colin McCabe
          22.
          		 Remove zk-only test: `FetchRequestTestDowngrade` Sub-task Resolved Chia-Chuan Yu
          23.
          		 Remove new_consumer from E2E Sub-task Resolved PoAn Yang
          24.
          		 Remove assorted ZK-related files Sub-task Resolved Colin McCabe
          25.
          		 Use version 3.8 to run the ZooKeeper service for end-to-end tests Sub-task Resolved PoAn Yang
          26.
          		 Remove ZooKeeper from the docs Sub-task Resolved Mickael Maison
          27.
          		 Update the Scram configuration section for KRaft Sub-task Resolved PoAn Yang
          28.
          		 Remove `isZkMigrationTest` and related code Sub-task Resolved Jhen-Yung Hsu
          29.
          		 Remove isKRaftTest from the kraft-only tests Sub-task Resolved Jhen-Yung Hsu
          30.
          		 add kraft support to testReplicaPlacementAllServers and testReplicaPlacementPartialServers Sub-task Resolved Chu Cheng Li
          31.
          		 Remove zk migration from RaftManagerTest, BrokerLifecycleManagerTest, KafkaConfigTest, and ReplicaManagerTest Sub-task Resolved Chia-Chuan Yu
          32.
          		 Remove zk migration from code base Sub-task Resolved TengYao Chi
          33.
          		 Remove ZooKeeperMainWithTlsSupportForKafka Sub-task Resolved Jhen-Yung Hsu
          34.
          		 tests/kafkatest/tests/core/upgrade_test.py needs to be re-added as KRaft Sub-task Resolved David Jacot
          35.
          		 Delete ZkSecurityMigrator Sub-task Resolved Mickael Maison
          36.
          		 Convert ConsumerRebootstrapTest to KRaft Sub-task Resolved 黃竣陽
          37.
          		 Convert CustomQuotaCallbackTest to KRaft Sub-task Resolved Kevin Wu
          38.
          		 Disable CustomQuotaCallbackTest and remove `isKRaftTest` Sub-task Resolved 黃竣陽
          39.
          		 Move configs out of "kraft" directory Sub-task Resolved TengYao Chi
          40.
          		 Remove zookeeper.connect from RemoteLogManagerTest Sub-task Resolved 黃竣陽
          41.
          		 Update process.roles to required configuration Sub-task Resolved PoAn Yang
          42.
          		 Remove zkBroker from RequestListenerType Sub-task Resolved 黃竣陽
          43.
          		 Remove KafkaConfig GroupType config check and warn log Sub-task Resolved 黃竣陽
          44.
          		 Remove zk config `control.plane.listener.name` Sub-task Resolved TaiJuWu
          45.
          		 Remove zookeeper configurations Sub-task Resolved Colin McCabe
          46.
          		 Remove PasswordEncoderConfigs Sub-task Resolved Chia-Ping Tsai
          47.
          		 Clean up Server config documents which contained zookeeper Sub-task Resolved 黃竣陽
          48.
          		 Remove zookeeper.connect in Test Sub-task Resolved 黃竣陽
          49.
          		 Remove KafkaConfig.interBrokerProtocolVersion Sub-task Resolved 黃竣陽
          50.
          		 Remove ZkConfigManager Sub-task Resolved kangning.li
          51.
          		 Remove zookeeper config in config_property.py Sub-task Resolved 黃竣陽
          52.
          		 Remove ZkMetadataCache Sub-task Resolved PoAn Yang
          53.
          		 Remove EncryptingPasswordEncoder Sub-task Resolved Mingdao Yang
          54.
          		 Remove raftSupport in KafkaApisTest Sub-task Resolved Unassigned
          55.
          		 Change KafkaConfigTest to use kraft config Sub-task Resolved Unassigned
          56.
          		 Remove reserved.broker.max.id and broker.id.generation.enable Sub-task Resolved PoAn Yang
          57.
          		 Remove ZkAlterPartitionManager Sub-task Resolved 黃竣陽
          58.
          		 Remove UpdateMetadata and LeaderAndIsr RPC Sub-task Resolved TaiJuWu
          59.
          		 Remove ZooKeeper from KafkaApis Sub-task Resolved TaiJuWu
          60.
          		 Remove ZooKeeper logic from DynamicBrokerConfig Sub-task Resolved PoAn Yang
          61.
          		 Remove controlled.shutdown.max.retries and controlled.shutdown.retry.backoff.ms Sub-task Resolved Jhen-Yung Hsu
          62.
          		 Remove ZkProducerIdManager Sub-task Resolved 黃竣陽
          63.
          		 Remove EmbeddedZookeeper Sub-task Resolved TengYao Chi
          64.
          		 Remove AdminZkClient Sub-task Resolved TengYao Chi
          65.
          		 Remove KRaftRegistrationResult Sub-task Resolved TengYao Chi
          66.
          		 Remove ZkAdminManager, DelayedCreatePartitions, CreatePartitionsMetadata, ZkConfigRepository, DelayedDeleteTopics Sub-task Resolved 黃竣陽
          67.
          		 Remove ZkBrokerEpochManager Sub-task Resolved 黃竣陽
          68.
          		 Remove ZkData and related unused references Sub-task Resolved PoAn Yang
          69.
          		 Remove OffsetTrackingListener Sub-task Resolved TengYao Chi
          70.
          		 Remove FinalizedFeatureChangeListener Sub-task Resolved TengYao Chi
          71.
          		 Remove ZooKeeperClient Sub-task Resolved 黃竣陽
          72.
          		 Remove ZkFinalizedFeatureCache and StateChangeFailedException Sub-task Resolved 黃竣陽
          73.
          		 Remove ZkNodeChangeNotificationListener Sub-task Resolved 黃竣陽
          74.
          		 Remove KafkaController Sub-task Resolved PoAn Yang
          75.
          		 Remove unused code from AutoTopicCreationManager Sub-task Resolved Logan Zhu
          76.
          		 Remove zookeeper dependencies in build.gradle Sub-task Resolved Colin McCabe
          77.
          		 Remove ZkFourLetterWords Sub-task Resolved TengYao Chi
          78.
          		 Remove MetadataCacheControllerNodeProvider Sub-task Resolved xuanzhang gong
          79.
          		 Remove MetadataSupport Sub-task Resolved 黃竣陽
          80.
          		 Remove DelayedDeleteTopics Sub-task Resolved 黃竣陽
          81.
          		 Remove zkBroker listener Sub-task Resolved 黃竣陽
          82.
          		 Update log4j2.yaml Sub-task Resolved David Jacot
          83.
          		 Remove ReplicaManager#stopReplicas Sub-task Resolved David Jacot
          84.
          		 Remove ReplicaManager#electLeaders Sub-task Resolved Chu Cheng Li
          85.
          		 Remove zkClient & maybeUpdateMetadataCache from ReplicaManager Sub-task Resolved Chu Cheng Li
          86.
          		 Clean up LogConfig Sub-task Resolved Mingdao Yang
          87.
          		 Remove kafka.controller.Election Sub-task Resolved TengYao Chi
          88.
          		 Remove TopicDeletionManager Sub-task Resolved TengYao Chi
          89.
          		 Remove PartitionStateMachine Sub-task Resolved TengYao Chi
          90.
          		 Remove ControllerEventManager Sub-task Resolved TengYao Chi
          91.
          		 Remove ControllerState Sub-task Resolved TengYao Chi
          92.
          		 Remove ControllerContext Sub-task Resolved TengYao Chi
          93.
          		 Remove ControllerChannelContext Sub-task Resolved TengYao Chi
          94.
          		 Remove ControllerChannelManager Sub-task Resolved TengYao Chi
          95.
          		 Remove ReplicaStateMachine Sub-task Resolved TengYao Chi
          96.
          		 Remove DelegationTokenManagerZk Sub-task Resolved 黃竣陽
          97.
          		 Remove RackAwareMode Sub-task Resolved 黃竣陽
          98.
          		 Remove Json.scala , cleanup AclEntry.scala Sub-task Resolved 黃竣陽
          99.
          		 Remove ZooKeeper logic from JaasUtils Sub-task Resolved Mickael Maison
          100.
          		 Cleanup NodeApiVersions zkMigrationEnabled field Sub-task Resolved 黃竣陽
          101.
          		 Remove ZooKeeperInternals Sub-task Resolved 黃竣陽
          102.
          		 Remove ZooKeeperClientException Sub-task Resolved 黃竣陽
          103.
          		 Clean Partition.scala zookeeper logic Sub-task Resolved 黃竣陽
          104.
          		 Remove KafkaConfig zookeeper related logic Sub-task Resolved 黃竣陽
          105.
          		 sharePartitionManager and clientMetricsManager should be required to KafkaAPIs Sub-task Resolved Apoorv Mittal
          106.
          		 Remove UpdataMetadataRequest in KafkaApisTest Sub-task Resolved TaiJuWu
          107.
          		 Cleanup AlterPartitionManager Sub-task Resolved 黃竣陽
          108.
          		 Remove Zookeeper logic from LogConfig Sub-task Resolved 黃竣陽
          109.
          		 Remove Zookeeper logic from LogManager Sub-task Resolved 黃竣陽
          110.
          		 Cleanup KRaftConfig documentation Sub-task Resolved TengYao Chi
          111.
          		 Remove unnecessary version check from KafkaApisTest Sub-task Resolved Ming-Yen Chung
          112.
          		 Update javadoc and comment of ConfigType Sub-task Resolved TengYao Chi
          113.
          		 Cleanup the doc of SocketServerConfigs#ADVERTISED_LISTENERS_CONFIG Sub-task Resolved TengYao Chi
          114.
          		 Remove JaasModule#zkDigestModule, JaasTestUtils#zkSections and its usage Sub-task Resolved TengYao Chi
          115.
          		 Cleanup FinalizedFeatures Sub-task Resolved TengYao Chi
          116.
          		 Cleanup SaslSetup Sub-task Resolved TengYao Chi
          117.
          		 Remove UpdateMetadataRequest in MetadataCacheTest Sub-task Resolved TaiJuWu
          118.
          		 Cleanup RequestHandlerHelper Sub-task Resolved 黃竣陽
          119.
          		 Remove StateChangeFailedException Sub-task Resolved 黃竣陽
          120.
          		 Remove TopicKey.scala Sub-task Resolved 黃竣陽
          121.
          		 Cleanup GroupMetadataManager Sub-task Resolved TengYao Chi
          122.
          		 Cleanup DelegationTokenManager Sub-task Resolved TengYao Chi
          123.
          		 Remove QuotaConfgHandler Sub-task Resolved TengYao Chi
          124.
          		 Cleanup ReplicaManager Sub-task Resolved TengYao Chi
          125.
          		 Remove AuthorizerUtils#sessionToRequestContext Sub-task Resolved TengYao Chi
          126.
          		 Remove ZkCachedControllerId In MetadataCache Sub-task Resolved 黃竣陽
          127.
          		 Cleanup BrokerLifecycleManager Sub-task Resolved 黃竣陽
          128.
          		 Remove ControllerMetadataMetrics zk related Metrics Sub-task Resolved 黃竣陽
          129.
          		 Remove optional ForwardingManager in ApiVersionManager Sub-task Resolved Ming-Yen Chung
          130.
          		 Remove UnifiedLog#legacyFetchOffsetsBefore Sub-task Resolved TengYao Chi
          131.
          		 Clean ReplicaManagerBuilder Sub-task Resolved TengYao Chi
          132.
          		 Remove ZkConfigs Sub-task Resolved Ming-Yen Chung
          133.
          		 Update Benchmark system tests Sub-task Resolved PoAn Yang
          134.
          		 Update ConsoleConsumerTest system test Sub-task Resolved TengYao Chi
          135.
          		 Update TestVerifiableProducer system test Sub-task Resolved PoAn Yang
          136.
          		 Fix test_bounce.py sys test Sub-task Resolved Mingdao Yang
          137.
          		 document the removal of `inter.broker.protocol.version` Sub-task Resolved Nick Guo
          138.
          		 Remove leader.imbalance.per.broker.percentage Sub-task Resolved 黃竣陽

          Activity

            People

              cmccabe Colin McCabe
              cmccabe Colin McCabe
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: