df16-sf-kafka-demo

Demo code for the Dreamforce 2016 session Analyzing Salesforce Data with Heroku, Kafka, and Connect.

The demo consists of the following parts:

1. A script that continuously posts to Salesforce Chatter.
2. Apex triggers that send Salesforce Chatter data to a Heroku app using an asynchronous HTTP callout.
3. A producer (the Heroku app), which writes a subset of the data to Apache Kafka on Heroku.
4. A consumer that reads from Kafka and performs sentiment analysis.

Heroku setup

This creates the app that receives REST requests from Salesforce and writes to Kafka.

1. heroku apps:create MY-APP-NAME
2. heroku addons:create heroku-kafka:beta-standard-0
3. heroku plugins:install heroku-kafka
4. heroku kafka:wait
5. heroku kafka:topics:create chatter
6. Deploy this repo to your app.

Salesforce setup

1. Add the Apex class and triggers from the apex directory to your Salesforce org.
2. Update the endpoint URL in HerokuPoster.cls to your Heroku app's URL.
3. Add your Heroku app's URL to the Remote Site Settings in Salesforce.

Posting Chatter data continuously

See the instructions at https://github.com/alouie-sfdc/chatter-data-pump

Running the Kafka consumer locally

This will use the open source TextBlob package to perform sentiment analysis on the Chatter data.

1. pip install -r requirements.txt
2. python consumer.py
3. CTRL-C to quit