- Added logic to determine if a feed is "new" or requires an "update" based on database checks.
- Integrated new `payload_type` field in `TransitFeedSyncPayload` to track the status of each feed ("new" or "update").
- Implemented checks for `external_id` in `public.externalid` table and corresponding `feed_url` in `public.feed` table.
- Filtered out feeds with HTTP status codes 404 and 500, as well as those from Japan and France.
- Added methods for fetching and extracting data from the TransitLand API, combining operator and feed data to prepare payloads.

…nal ID checks

- Added logic to determine if a feed is "new" or requires an "update" based on database checks.
- Integrated new `payload_type` field in `TransitFeedSyncPayload` to track the status of each feed ("new" or "update").
- Implemented checks for `external_id` in `public.externalid` table and corresponding `feed_url` in `public.feed` table.
- Filtered out feeds with HTTP status codes 404 and 500, as well as those from Japan and France.
- Added methods for fetching and extracting data from the TransitLand API, combining operator and feed data to prepare payloads.

…nal ID checks

- Added logic to determine if a feed is "new" or requires an "update" based on database checks.
- Integrated new `payload_type` field in `TransitFeedSyncPayload` to track the status of each feed ("new" or "update").
- Implemented checks for `external_id` in `public.externalid` table and corresponding `feed_url` in `public.feed` table.
- Filtered out feeds with HTTP status codes 404 and 500, as well as those from Japan and France.
- Added methods for fetching and extracting data from the TransitLand API, combining operator and feed data to prepare payloads.

- Added logic to determine if a feed is "new" or requires an "update" based on database checks.
- Integrated new `payload_type` field in `TransitFeedSyncPayload` to track the status of each feed ("new" or "update").
- Implemented checks for `external_id` in `public.externalid` table and corresponding `feed_url` in `public.feed` table.
- Filtered out feeds with HTTP status codes 404 and 500, as well as those from Japan and France.
- Added methods for fetching and extracting data from the TransitLand API, combining operator and feed data to prepare payloads.

…stop_id

2. adjusted payload to reflect external_id update

Incorporated URL status check and merged feed sync data using pandas DataFrames for enhanced processing. Refactored process_sync() function to include filtering, grouping, and data extraction improvements, leading to more efficient and accurate feed sync operations.

… feat/transitland-feed-sync

 Unit tests to verify the functionality of TransitFeedSyncProcessor, covering data retrieval, rate limit handling, synchronization processing, URL status checks, data extraction, and merging/filtering of data.

 Unit tests to verify the functionality of TransitFeedSyncProcessor, covering data retrieval, rate limit handling, synchronization processing, URL status checks, data extraction, and merging/filtering of data.

Standardized comments and docstrings for better clarity and consistency. Improved code readability by cleaning up unnecessary lines and aligning comment styles.

Ensured that all modified files include a newline at the end

Updated README.md

…e readability and maintain consistency.

Added 'source' attribute to payload and implemented new logic for determining feed updates or new entries based on 'external_id' and 'source'. Refactored test cases to align with these changes.

Extended the timeout for the `requests.head` call from 10 to 25 seconds to accommodate slower server responses.

Added 'source' parameter in README.md and reformat function arguments in main.py and test_feed_sync.py for better readability. Ensure consistent indentation and correct minor formatting issues.

Added 'source' parameter in README.md and reformat function arguments in main.py and test_feed_sync.py for better readability. Ensure consistent indentation and correct minor formatting issues.

Added the 'source' field to the payload dictionary in the README.md example and made minor whitespace adjustments in the main source and test files.

Co-authored-by: David Gamez <[email protected]>