Bluesky: look into optimizing API calls

[snarfed]

Wow! Bluesky only has 26 users, ~1% of all active users that we poll, but it's already the solid majority of all outbound HTTP requests! We should eventually look into optimizing those. First step would be to get the like/repost cache working, based on counts, and only fetch those when the count increases...but I'm not sure the Bluesky API gives us those counts? cc @JoelOtter

[JoelOtter]

No it does! I just was unaware of how the caching works to begin with. Happy to have a look :)

[JoelOtter]

Actually it looks like Bluesky is already using the cache for replies, reposts and likes?

[snarfed]

You're right, granary reads and writes it, and it gets passed through from the Bluesky.last_activities_cache_json datastore property here:

bridgy/tasks.py

Lines 132 to 144 in 8c809d2

	cache = util.CacheDict()
	if source.last_activities_cache_json:
	cache.update(json_loads(source.last_activities_cache_json))
	# search for links first so that the user's activities and responses
	# override them if they overlap
	links = source.search_for_links()
	# this user's own activities (and user mentions)
	resp = source.get_activities_response(
	fetch_replies=True, fetch_likes=True, fetch_shares=True,
	fetch_mentions=True, count=30, etag=source.last_activities_etag,
	min_id=source.last_activity_id, cache=cache)

...and yet it's not working. Hmm. 😐 Maybe first check if any of the Bluesky entities in the datastore actually have anything stored in that property? https://console.cloud.google.com/datastore/entities/query?project=brid-gy

[JoelOtter]

OH

[JoelOtter]

This might actually be (part of?) the cause of #1592

[snarfed]

Huh, you're probably right! Great catch.

[snarfed]

Looks like it worked! When there's nothing new, a Bluesky poll now only does a single API call, down from as many as 91 before. 😆

[snarfed]

(@JoelOtter you should hopefully be able to see the dashboard too, feel free to poke around: https://console.cloud.google.com/monitoring/dashboards/builder/0e3a0a00-ce6f-4f3e-a907-4ddabd3288a6;duration=PT24H?project=brid-gy&dashboardBuilderState=%257B%2522editModeEnabled%2522%3Afalse%257D )