Email subscription scraper

[Mearnest]

A really super awesome feature would be a scraper that generates meeting events from meeting information in emails. These emails would come from email subscriptions to different board meetings and what not.

Unfortunately, the information is likely to be inside a word or pdf document.

The motivation for this is to be able to keep the website up to date.

[Mearnest]

This can be in any programming language (even the ones below), it just needs to feed the postgres database Elixir reads from.

(insert-into postgres (scrape (email-attach word)) (scrape (email-attach pdf)))

[aaronarduino]

I'd be interested in making this. Would we use a email service like mailgun?

[aaronarduino]

Or maybe we could use something like https://context.io/docs/lite?

[Mearnest]

Whatever works best. Keep in mind that the resulting data has to be inserted into the meeting_types, meeting_dates, and meeting_extras tables in postgres, unless generating json is easier, or providing Elixir with an API.

[Mearnest]

You pretty much have total freedom to use any tech here. It just needs to be run on the same server, unless Elixir is talking to an api running somewhere else.

[aaronarduino]

I've been working on this lately, see link for progress: https://github.com/aaronarduino/public-meetings/tree/proto-email-scraper

[Mearnest]

Looking good so far!