Marskafka is a Kafka data processing pipeline that uses deep learning with Kafka to colorize & tweet images from NASA’s Mars Rover API
- Images from our Mars Rovers are consumed from NASA's API and are filtered between two topics:
queue.bwimg
for greyscale imagesstream.colorimg
for RGB images
- An instance of the deoldify deep learning network listens to the greyscale queue, colorizes the images, and forwards them to the colored stream
- Both topics will them merge at nabokov– a twitter bot that takes 3 images from the current epoch (daily) and tweets them to this account
I built this to learn about distributed computing, publish/subscribe models, and stream processing
Directory | Description |
---|---|
pushkin | HTTP service that consumes Mars API & pipes to kafka |
tolstoy | Deep learning service |
nabokov | Twitter Bot |