cordinator(ticdc): Fix Puller Resolved TS Lag Calculation and Deprecate current_ts Field in Stats

[wlwilliamx]

What problem does this PR solve?

Issue Number: close #11561

What is changed and how it works?

Summary:

This PR addresses a critical issue in the Puller Resolved TS Lag metric, where the lag was incorrectly displayed as -55 years. The root cause of the problem was the improper initialization of the CurrentTs field in the Stats structure within table.pb.go. This led to a situation where CurrentTs remained zero while ResolvedTs was correctly populated, resulting in a negative lag calculation that erroneously reflected a time difference of 55 years (from 1970 to the current year).

Detailed Explanation:

• Issue Background:

  • The Puller Resolved TS Lag was calculated as the difference between CurrentTs and ResolvedTs from the Stats structure.
  • However, the NewReplicationSet function did not correctly initialize CurrentTs, leading to incorrect lag values when metrics were collected.

• Design Intent of current_ts:

  • The current_ts field was initially designed to record the time at the processor side, which would then be used by the coordinator for lag calculation. This design was meant to avoid including the transmission delay and processing time on the coordinator side, as these could introduce inaccuracies in the lag calculation.
  • However, this potential inaccuracy is minimal since the communication between the processor and coordinator happens within the TiCDC cluster, where network delays are typically in the millisecond range. Given that the lag metric is reported in seconds, any millisecond-level discrepancy is considered acceptable.
  • Deprecating the current_ts field may introduce a slight inaccuracy in the lag calculation, as the "current time" will now be fetched directly by the coordinator rather than from the processor. This could result in a minor error due to transmission delays and processing time. However, this inaccuracy is negligible for practical purposes, as the delay is typically within milliseconds and does not significantly impact the metric, which is measured in seconds.
  • Additionally, since this field was part of the protobuf (pb) structure, it added extra overhead to every network transmission by carrying an additional value. It also introduced more complexity into the code, requiring maintenance of this field, which was part of the issue that led to the incorrect metric calculations in this case.

• Root Cause:

  • The use of CurrentTs from the Stats structure was flawed because this timestamp was not reliably up-to-date.
  • The metric calculation requires the "current time," which should always be fetched directly from PD to ensure accuracy, rather than relying on a potentially outdated CurrentTs stored in the Stats structure.

• Solution Implemented:

  • This PR updates the metric calculation to fetch the latest "current time" directly from PD whenever calculating the lag.
  • All references to the CurrentTs field within the codebase, particularly in metric calculations, have been updated to reflect this change.

• Deprecation of current_ts Field:

  • The current_ts field in the Stats structure, defined in cdc/processor/tablepb/table.proto, has been marked as deprecated.
  • The Stats structure is used for data transmission between the Coordinator and Processor, where current_ts was unnecessarily included in the state for each Table Span.
  • This PR removes all dependencies on current_ts within the project, ensuring that the Coordinator and Processor no longer rely on this outdated field.

Impact:

This PR ensures accurate lag calculation in the Puller Resolved TS Lag metric and removes the outdated current_ts field, improving both performance and maintainability by reducing unnecessary transmission overhead and field maintenance. While this change may introduce a slight inaccuracy in the lag calculation due to the use of coordinator-side timestamps, this impact is minimal and well within acceptable thresholds.

Check List

Tests

• Unit test
• Integration test

Questions

Will it cause performance regression or break compatibility?

No. The deprecation of the current_ts field in the Stats structure is backward-compatible because the field is not actively used elsewhere in the project beyond the affected metrics. The existing field remains in the protocol buffer definition but is marked as deprecated, so it won’t break compatibility with older versions that still use the field.

Do you need to update user documentation, design documentation or monitoring documentation?

No.

Release note

None

[wlwilliamx]

/cc @3AceShowHand @asddongmen

[codecov]

Codecov Report

Attention: Patch coverage is 27.27273% with 8 lines in your changes missing coverage. Please review.

Project coverage is 57.4003%. Comparing base (25676cf) to head (2519fa6).
Report is 4 commits behind head on master.

Additional details and impacted files

Components	Coverage Δ
cdc	61.0503% <27.2727%> (-0.1953%)	⬇️
dm	51.0463% <ø> (-0.0093%)	⬇️
engine	63.3950% <ø> (ø)

Flag	Coverage Δ
unit	57.4003% <27.2727%> (-0.0907%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@@               Coverage Diff                @@
##             master     #11624        +/-   ##
================================================
- Coverage   57.4909%   57.4003%   -0.0907%     
================================================
  Files           851        852         +1     
  Lines        126486     126846       +360     
================================================
+ Hits          72718      72810        +92     
- Misses        48377      48638       +261     
- Partials       5391       5398         +7     

[3AceShowHand]

can we remove this field?

[wlwilliamx]

Currently, we cannot directly remove the current_ts field from the protobuf definition. Although this message is used solely for communication within the TiCDC cluster, there is a risk of incompatibility during rolling upgrades. In such scenarios, different versions of TiCDC (coordinator and processor) may coexist, and removing the current_ts field could lead to communication errors between nodes running different versions.

However, the field is being deprecated and is no longer used for any critical functionality. Once we are confident that it is not in use across any versions, we can plan to safely remove it in a future release without breaking compatibility.

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: 3AceShowHand

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [3AceShowHand]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2024-09-25 07:25:20.941338528 +0000 UTC m=+1637190.681762466: ☑️ agreed by 3AceShowHand.

[wlwilliamx]

/retest

[wlwilliamx]

/retest

[wlwilliamx]

/retest

[wlwilliamx]

/test cdc-integration-storage-test

[wlwilliamx]

/test cdc-integration-mysql-test