TAP_SPREADSHEETS_ANYWHERE_TABLES environment variable is not seen by the tap

[ian-lewis-d]

I have multiple data sources in S3 that have the same format and the same (>20) tables but delivered from separate S3 buckets. I am running from Meltano.

There are several possible solutions:

1. create a separate meltano environment for each dataset (separate s3 buckets)
   Cumbersome, bulky and very inflexible. Error prone.

2. parameterise the s3_bucket and s3_prefix settings inside meltano.yml (for each table) using environment variables
   I have tried this and it just doesn't work. May be related to this which will (I hope) fix the issue feat: Expand environment variables in array setting values meltano/meltano#8268

3. create a different JSON config file for each dataset and load into TAP_SPREADSHEETS_ANYWHERE_TABLES
   I have attempted to load the JSON tables array into TAP_SPREADSHEETS_ANYWHERE_TABLES but it only works when the string is stored in .env and does not work with an exported variable.

I also tried setting TAP_SPREADSHEETS_ANYWHERE_TABLES to the filename of the tables config but it only expects a JSON string.

Are there any other solutions to dealing with multiple environments, with this tap, that I could explore?