Validate rpc response consistency

[kyscott18]

Description

Before further processing, we should validate that all referenced data is present. All blockHashs and transactionHashs should have a corresponding block and transaction. This should happen in both sync-historical and sync-realtime.

Related

Several users have reported that transaction is undefined in their ponder indexing functions. After further investigation this is a result of the database being corrupted, with a log.transactionHash referencing a transaction that doesn't exist

[khaidarkairbek]

What should be the proper handling if the log references the transaction that is not in the block? Should it throw an error?

[kyscott18]

What should be the proper handling if the log references the transaction that is not in the block? Should it throw an error?

Great question - I think the sync-historical logic should log an error containing exactly what is inconsistent and then fail. sync-realtime should log a similar warning, and then enter this retry loop that is already used to handle some rpc-related errors.

ponder/packages/core/src/sync-realtime/index.ts

Lines 644 to 654 in 8790396

	} catch (_error) {
	if (isKilled) return;
	const error = _error as Error;
	error.stack = undefined;
	args.common.logger.warn({
	service: "realtime",
	msg: `Failed to process '${args.network.name}' block ${hexToNumber(block.number)}`,
	error,
	});

The reason for the different behavior in sync-historical vs sync-realtime is that json rpc providers are generally less reliable (because of load balancing or reorg handilng logic or whatever reason) when the requested data is closer to the tip of the chain but eventually able to serve the correct data after retrying. Farther back in the historical range the providers are more reliable but errors are persistent.

[khaidarkairbek]

I see, can i take over this issue btw?

[kyscott18]

Yes, that would be great. I think you should start with the sync-realtime, cause it should be more straightforward than the sync-historical, and make a draft pr when you finish that part.

[kyscott18]

@khaidarkairbek Here's another spot in the same function we would want to update that does some rpc response validation and throws and error:

ponder/packages/core/src/sync-realtime/index.ts

Lines 391 to 395 in 6d45c52

	if (block.logsBloom !== zeroLogsBloom && logs.length === 0) {
	throw new Error(
	`Detected invalid '${args.network.name}' eth_getLogs response.`,
	);
	}