[FLINK-36558] Fix column metadata parsing compatibility with MySQL 8.0.17 / 8.0.18 #3647
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This closes FLINK-36558.
It is known that MySQL 8.0.17 and 8.0.18 writes Array-typed key index TableMap data into Binlog differently.
The "extra byte" issue has been fixed since MySQL 8.0.19. Since MySQL takes Binlog format as an internal implementation detail[1] and doesn't guarantee its backwards compatibility, MySQL CDC has to deal with the compatibility problem itself.
The actual code change is limited to
TableMapEventDataDeserializer.java
where the encoded "column metadata bytes size" is used to determine how many bytes should be sent toreadMetadata
function for parsing. Any unused bytes will be ignored and will not affect parsing the following fields.[1] https://bugs.mysql.com/bug.php?id=105545