Hydra backlog

[dzhelezov]

Overview

This issue covers future directions and nice-to-haves for Hydra v2, which refers to Hydra post-Babylon release (Hydra v0) and post-refactoring (Hydra v1).

Grading

Impact: 1 to 5

• 1 -> nice to have but no subsential improvement over the feature set / long-term maintanance
• 5 -> will enable genuinely new use-case/onboard users

Complexity: 1 to 5

• 1 -> Will fit at most in a week-long sprint, fully clear how to implement
• 5 -> Requires substantial "grooming" to estimate costs and figure out implementation details, likely > 1 month

Features to consider

Indexing

• Add Prometheus metrics Complexity: 1 Impact: 2
• Event and block subscriptions Indexer API: subscriptions #25 Complexity: 2 Impact: 1
• Index transactions, blocks, and meta information Index transactions #119 Complexity: 2 Impact: 3
• Handle runtime upgrades Complexity: 4 Impact: 3

Processing

• Manifest file Define mappings in a manifest file #5 Complexity: 2 Impact: 3
• Events filtering Arbitrary filters for fetching events from the indexers #105 Complexity: 2 Impact: 2
• Type-safe mappings Type safe mappings in query node #13 Complexity: 4 Impact: 4
• Processing middleware (= plugin-based pipelining) Abstract mappings #61 Complexity: 5 Impact: 5
• Pipelining plugins (such as decoding sudo calls, sub-calls (Decode extrinsic args for sub call #6), batch calls): Complexity: 3 Impact: 4 <-- depends on Design pipeling for the processor #110
• Block arrival hooks Add block arrival processing hooks #14 Complexity: 3 Impact: 1
• Transactions support Add transactions support for the processor #120 Complexity: 4 Impact: 4

CLI/Shema generation

• @field type WIP: Field type spec #64 Complexity: 2 Impact: 1
• Revisit Full-text-search queries (multiple language support) Complexity: 2 Impact: 2
• Independent schema generation (Separate tool for generating model types and classes from the input schema #108) Complexity: 3 Impact: 2

Maintenance and tech debt

• Improve test coverage to at least 80% for the core components: hydra-indexer, hydra-processor, hydra-cli Complexity: 2 Impact: 2
• More (integration) tests for invalid schemas (Add more tests for invalid schemas #15) Complexity: 2 Impact: 3

DevOps

• Setup a delivery pipeline, currently all the releases (to NPM and Dockerhub) are done manually in an ad-hoc fashion Complexity: 3 Impact: 2
• Graphana dashboards for the processor and indexer metrics (Create reference Graphana for Indexer and Processor #107) Complexity: 2 Impact: 1
• Deploy prometheus server + graphana + alerts (Improve observability: integrate Prometheus or any other metrics collector #3) Complexity: 2 Impact: 3