PiCo: High-Performance Data-Analytics Pipelines in C++

PiCo (Pipeline Composition) is an open-source C++11 header-only DSL for high-performance data analytics, featuring low latency, high throughput, and minimal memory footprint on multi-core platforms.

Take a look to the word-count code to see how easy is writing a PiCo pipeline!

Report bugs + get help

https://github.com/alpha-unito/pico/issues/new

Build and run tests

The following steps require cmake >=3.1 as build system.

Get PiCo:

git clone https://github.com/alpha-unito/pico.git pico

The current implementation is based on FastFlow as runtime system. Get it and a link it:

git clone --single-branch -b fastflow3 https://github.com/fastflow/fastflow.git /path/to/fastflow
ln -s /path/to/fastflow/ff pico/

⛑️ A better solution for including FastFlow as dependency is under development!

Build and run the tests: (and examples)

cd pico
mkdir build && cd build
cmake .. -DPICO_ENABLE_UNIT_TEST=ON
cmake --build .
ctest

Use PiCo in your code

Good news! PiCo is header-only, you do not need to build/link any library to use it in your code. Just include PiCo headers at the beginning of your source file:

#include "pico/pico.hpp"

and use good ol' compiler flags to include PiCo (and FastFlow) when compiling your app:

git clone https://github.com/alpha-unito/pico.git /path/to/pico
git clone --single-branch -b fastflow3 https://github.com/fastflow/fastflow.git /path/to/fastflow
g++ -I/path/to/pico/include -I/path/to/fastflow app.cc

⛑️ A modern CMake-based solution for linking PiCo (with its dependencies) is under development!

Examples

The examples folder contains some proof-of-concept applications, showing the PiCo user experience:

• word-count: PiCo pipelines 101 + visualizing application graphs
• stock-market: batch vs stream pipelines
• page-rank: iterative pipelines

PiCo Team

Maurizio Drocco [email protected] (maintainer)
Claudia Misale [email protected] (creator + co-maintainer)
Alberto Riccardo Martinelli [email protected] (co-maintainer)

Contributors

Marco Aldinucci [email protected] (boss)
Massimo Torquati [email protected] (FastFlow maintainer)
Guy Tremblay [email protected] (DSL wizard)

How to cite PiCo

C. Misale, M. Drocco, G. Tremblay, A. R. Martinelli, and M. Aldinucci, "PiCo: High-Performance Data Analytics Pipelines in Modern C++," Future Generation Computer Systems, Volume 87, 2018.