Firehose Quickstart

Summary

• -how to get new relic api key
• -how to get nr account id

Pre-reqs

• Linux host
• Access to internet
• Docker running

Installation

Download the ktranslate image:

docker pull kentik/ktranslate:v2

Send netflow/sflow data with:

docker run \
 -e NEW_RELIC_API_KEY=NRII-$NR_TOKEN \
 -v `pwd`/snmp-base.yaml:/snmp-base.yaml \
 --net=host \
 --name ktranslate \
 kentik/ktranslate:v2 \
  -nr_account_id $NR_ACCOUNT_ID \
  -metrics=jchf \
  -snmp=/snmp-base.yaml \
  -tee_logs=true \
  -nf.source=netflow5 \
  -nf.port=9995 \
  -flow_only=true \
  nr1.flow

Sample Output:

2021-07-06T21:51:19.581 ktranslate [Info] flow Netflow listener running on 0.0.0.0:9995 for format netflow5 and a batch size of 10000
2021-07-06T21:51:19.581 ktranslate [Info] flow Netflow listener sending fields Type,TimeReceived,SequenceNum,SamplingRate,SamplerAddress,TimeFlowStart,TimeFlowEnd,Bytes,Packets,SrcAddr,DstAddr,Etype,Proto,SrcPort,DstPort,InIf,OutIf,SrcMac,DstMac,SrcVlan,DstVlan,VlanId,IngressVrfID,EgressVrfID,IPTos,ForwardingStatus,IPTTL,TCPFlags,IcmpType,IcmpCode,IPv6FlowLabel,FragmentId,FragmentOffset,BiFlowDirection,SrcAS,DstAS,NextHop,NextHopAS,SrcNet,DstNet,HasMPLS,MPLSCount,MPLS1TTL,MPLS1Label,MPLS2TTL,MPLS2Label,MPLS3TTL,MPLS3Label,MPLSLastTTL,MPLSLastLabel
2021-07-06T21:51:19.581 ktranslate [Info] KTranslate System running with format new_relic, compression gzip, max flows: 10000, sample rate 1000:1 after 100

Optional: Pass in a MaxMind GeoIP DB to the --geo flag. For example: -geo /etc/country/GeoLite2-Country.mmdb

---

Send VPC logs to New Relic with:

docker run \
 -e NEW_RELIC_API_KEY=NRII-$NR_TOKEN \
 -p 8082:8082 \
 kentik/ktranslate:v2 \
  -nr_account_id $NR_ACCOUNT_ID \
  -metrics=jchf \
  -vpc=aws \
  -sqs_name=sqs-queue-name \
  -tee_logs=true \
  nr1.vpc