Skip to content

Commit

Permalink
Prevent data loss from backfill calls
Browse files Browse the repository at this point in the history
  • Loading branch information
kelvinou01 committed Aug 21, 2023
1 parent d2d954b commit de43a5f
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions src/extract.py
Original file line number Diff line number Diff line change
Expand Up @@ -102,10 +102,10 @@ def parse_and_check_date(input_date: str) -> Date:
date = datetime.strptime(input_date, "%d/%m/%Y").date()

today = datetime.now().date()
more_than_a_month_ago = (today - date).days > 30
if more_than_a_month_ago:
more_than_ten_days_ago = (today - date).days > 10
if more_than_ten_days_ago:
raise ValueError(
"Use extract_backfill.py to extract posts made more than a month ago.",
"Cannot extract for dates made more than 10 days ago. Data loss may occur.",
)

return date
Expand Down

0 comments on commit de43a5f

Please sign in to comment.