Diffing streams

[jbothma]

It would be nice to be able to diff two streams and see

• which fields differ between the same entity in the two streams
• which entities only occur in one or the other stream

This needs to ignore differences in the ordering, of entities in the stream, property values in an entity

It would also be nice if it could

• ignore differences in timestamps, perhaps optionally
• perhaps ignore user-defined properties and attributes

This, along with the ability to extract one or more enitities using a whitelist of IDs would make it much easier to pull out a sample of specific entities, and inspect differences.

My use case is when working on code producing a stream, and wanting to see the differences between a known version and a new version.

[pudo]

Some fraction of this is implemented here, but it's so bad we never really advertised it :/

https://github.com/opensanctions/opensanctions/tree/main/contrib/delta