Skip to content

teragrep/lsh_01

 
 

Repository files navigation

lsh_01

lsh_01 logstash-http-input to syslog bridge

Features

Listens to HTTP requests and sends them forward using RELP

Documentation

Example configuration can be found here.

By default, the configuration file is expected to be in etc/config.properties. That path can be changed with properties.file system property.

Properties can be overwritten on demand by setting them with property.name=value.

Currently supported configurations:

Property Default Value Notes

server.listenAddress

127.0.0.1

Address the server will listen to

server.listenPort

8080

Port the server will listen to

server.threads

1

How many threads the server will use

server.maxPendingRequests

128

How many pending requests are allowed

server.maxContentLength

262144

How big requests are allowed in bytes

relp.target

127.0.0.1

RELP server address

relp.port

601

RELP server port

relp.reconnectInterval

10000

How long to wait before reconnecting in milliseconds

relp.rebindRequestAmount

1000000

How many requests from a RELP connection until rebinding it

relp.rebindEnabled

false

Sets whether rebinding RELP connections is enabled

healthcheck.enabled

true

Sets if an internal healthcheck endpoint is enabled.

healthcheck.url

/healthcheck

An internal healthcheck endpoint that will always reply 200 ok regardless of security settings. Accessing this url won’t generate any events.

security.authRequired

true

Sets whether Basic HTTP Authorization headers are required. Username for lookups will be empty string '' if set to false.

credentials.file

etc/credentials.json

A json file with array of identity:credential mappings

lookups.hostname.file

etc/hostname.json

Path to username-to-hostname lookup table

lookups.appname.file

etc/appname.json

Path to username-to-appname lookup table

payload.splitType

none

Sets how to split incoming messages. Supported split types are 'regex' and 'json_array'. Use 'none' for no splitting.

payload.splitType.regex.pattern

\n (newline)

A regex based on which incoming requests will be split into multiple outgoing messages

prometheus.port

1234

Port used by the server that provides DropWizard metrics

Lookup tables

By default, the appname and hostname lookup tables are expected to be in etc/appname.json and etc/hostname.json respectively.

Empty string '' will be used for looking up values when authentication is not required, otherwise it will use one extracted from Authorization header.

The index is the username and value is what it should map the values to. The value of 'nomatch' will be used if lookup fails.

Limitations

  • Developed on and requires Java 11

  • Doesn’t support SSL

How to use

Run maven on Java 11:

mvn clean package

How to run in a container

Mount configurations to /config/ and run.

Expected files are:

  • /config/config.properties

  • /config/credentials.json

  • /config/hostname.json

  • /config/appname.json

  • Optional: Custom log4j2.xml can be provided by placing it in /config/log4j2.xml, otherwise default settings will be used.

For example:

docker run -ti -v /path/to/config:/config ghcr.io/teragrep/lsh_01:latest

How to run and use in development

Run maven on Java 11:

mvn clean package

Run the .jar file (example with default properties):

java -Dproperties.file=etc/config.properties -Dcredentials.file=etc/credentials.json -Dlookups.hostname.file=etc/hostname.json -Dlookups.appname.file=etc/appname.json -Dlog4j2.configurationFile=file:rpm/src/main/resources/log4j2.xml -jar target/lsh_01-jar-with-dependencies.jar

A RELP-server has to be in place. A server and documentation for setting it up can be found in rlp_07. Make sure that you set relp.port in configuration.properties to the same port that you are using with the RELP-server.

Example for sending messages from command line:

curl localhost:8080 -d "foo foo"

Contributing

You can involve yourself with our project by opening an issue or submitting a pull request.

Contribution requirements:

  1. All changes must be accompanied by a new or changed test. If you think testing is not required in your pull request, include a sufficient explanation as why you think so.

  2. Security checks must pass

  3. Pull requests must align with the principles and values of extreme programming.

  4. Pull requests must follow the principles of Object Thinking and Elegant Objects (EO).

Read more in our Contributing Guideline.

Contributor License Agreement

Contributors must sign Teragrep Contributor License Agreement before a pull request is accepted to organization’s repositories.

You need to submit the CLA only once. After submitting the CLA you can contribute to all Teragrep’s repositories.