Missing imagery for 2022 in Sentinel-2-c1-l2a

[kurtjoy]

Hey, When I hit the new S2 stac collection for AOIs in New Zealand, it returns only a limited number of available granules for 2019 and 2022.
For instance using:

from pystac_client import Client
catalog = Client.open('https://earth-search.aws.element84.com/v1')
query = catalog.search(bbox = (176.314459,-38.743711,176.348385,-38.718918), collections = ['sentinel-2-c1-l2a'])
items = list(query.get_items()))

returns

---

Year: 2018 Images: 1
2018-12-27T22:16:40.354000000 to 2018-12-27T22:16:40.354000000

Year: 2019 Images: 10
2019-11-12T22:16:42.685000000 to 2019-12-27T22:16:38.212000000

Year: 2020 Images: 68
2020-01-01T22:16:38.674000000 to 2020-12-31T22:16:43.773000000

Year: 2021 Images: 72
2021-01-05T22:16:42.586000000 to 2021-12-31T22:16:39.597000000

Year: 2022 Images: 7
2022-10-02T22:16:50.813000000 to 2022-12-31T22:16:42.595000000

Year: 2023 Images: 70
2023-01-05T22:16:41.478000000 to 2023-12-31T22:16:44.564000000

Year: 2024 Images: 4
2024-01-05T22:16:39.902000000 to 2024-01-20T22:16:42.934000000

Cheers!

Kurt

[matthewhanson]

Hello @kurtjoy ,

We made a poor assumption - when we started this reprocessing a few weeks ago we mistakenly thought that ESA was done with the reprocessing of Collection-1. However the reprocessing is still going on, with at least half of 2022, and 2019 and earlier still to do, which amounts to a bit less than half of the entire archive. As they process this data it will be added to our new Collection, but we do not have an estimate of their timeline.

We'll send out an update to the Earth Search mailing list this week, once we have finished processing the back inventory and take a closer look at the temporal distribution of new scenes coming in on a daily basis.

[kurtjoy]

Hey @matthewhanson not a problem at all! Thanks for your great work on this stuff!

[aleny91]

Hi there,

I have a question about the upcoming changes. Given that the new collection doesn't encompass the entire archive, is there still an intention to stop getting new scenes for the two existing collections on February 1, 2024?

Thank you for the great work!
Alessia

[matthewhanson]

Hi @aleny91 , No, for now we will continue running the old pipeline and won't deprecate it until we find out more about the expected timing of Collection-1.

[yerzhan7orazayev]

Hi @matthewhanson

When one can expect the availability of all the past data in v1?

[matthewhanson]

@yerzhan7orazayev It appears that ESA has updated their completion date at https://sentinels.copernicus.eu/web/sentinel/technical-guides/sentinel-2-msi/copernicus-sentinel-2-collection-1-availability-status#:~:text=The%20Sentinel%2D2%20Collection%201,2024%2Fbeginning%20of%20Q2%202024.

Which now says end of Q1 2024 or start of Q2. If accurate we can expect the processing to be done in the next 4-8 weeks, but we'll see in the next few weeks if they seem to be on schedule.

[jamescw19]

Looks like that page has been updated again, with no mention of dates any more. I assume that means ESA has completed processing, and the missing imagery should all be available?

[kurtjoy]

@jamescw19 Unfortunately I suspect not. I'm still getting holes when I check (Checking daily)

[jamescw19]

@kurtjoy I can now see the Copernicus page lists Q2 2024 as the completion date for processing, so it should all be available. Are you still seeing missing data?

[keithpeck]

These are the counts by year for sentinel-2-c1-l2a vs sentinel-2-l2a from https://earth-search.aws.element84.com/v1/search

year	sentinel-c1-l2a	sentinel-2-l2a
2016	13	14672
2017	24560	1283241
2018	1329398	4237038
2019	554620	4432227
2020	4013130	7733597
2021	4054501	6479424
2022	229699	4121669
2023	4247054	3718786
2024	2363469	2229386