logEndOffset could be lost due to log cleaning

It's possible for the log cleaner to remove all records in the suffix of the log. If the partition is then reassigned, the new replica won't be able to see the true logEndOffset since there is no record batch associated with it. If this replica becomes the leader, it will assign an already used offset to a newly produced record, which is incorrect.

It's relatively rare to trigger this issue since the active segment is never cleaned and typically is not empty. However, the following is one possibility.

1. records with offset 100-110 are produced and fully replicated to all ISR. All those records are delete records for certain keys.
2. record with offset 111 is produced. It forces the roll of a new segment in broker b1 and is added to the log. The record is not committed and is later truncated from the log, leaving an empty active segment in this log. b1 at some point becomes the leader.
3. log cleaner kicks in and removes records 100-110.
4. The partition is reassigned to another broker b2. b2 replicates all records from b1 up to offset 100 and marks its logEndOffset at 100. Since there is no record to replicate after offset 100 in b1, b2's logEndOffset stays at 100 and b2 can join the ISR.
5. b2 becomes the leader and assign offset 100 to a new record.