Skip to content

astrolabsoftware/fink-broker

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Quality Gate Status Maintainability Rating Sentinel PEP8 codecov Documentation Status

What is Fink?

Fink is an alert broker, that is a layer between astronomical alert issuers and the scientific community analysing the alert data. It exposes services to help the scientists to efficiently analyse the alert data from telescopes and surveys. Among several, it collects and stores alert data, enriches them with information from other surveys and catalogues or user-defined added values such as machine-learning classification scores, and redistributes the most promising events for further analyses, including follow-up observations.

Fink's main scientific objective is to optimize the scientific impact of the Rubin Observatory alert data stream. We do not limit ourselves to a specific area, but instead our ambition is to study the transient and variable sky as a whole, from Solar system objects to galactic and extragalactic science. In practice thanks to the Zwicky Transient Facility alert stream, we are active since 2019 on Solar system objects, young stellar objects, microlensing, supernovae, kilonovae, gamma-ray bursts, active galactic nuclei, and even anomaly detection. On the technological side, Fink aims at providing a robust infrastructure and state-of-the-art streaming services to Rubin scientists, to seamlessly enable user-defined science cases in a big data context.

How Fink works?

The current Fink platform works in four steps. First, alerts from multiple streams are continuously ingested and stored on disk (Apache Spark Structured Streaming). Second, alerts satisfying the quality cuts defined by the broker team are processed by a set of science modules. These science modules -- currently a dozen, spanning solar system objects to galactic and extra-galactic science -- are independent processing units developed by the community of users, and deployed in the Fink platform. They can work on a single input alert stream, or combine several streams together. These science modules enrich the initial alert packets using several techniques such as cross-match with external catalogs of astronomical objects, or classification using machine or deep learning based algorithms. All added-values are made public for the benefit of everyone. Third, enriched alert packets are filtered based on their content, and the most promising events are redistributed to the scientific community in real-time (Apache Kafka). The filtering is again community-driven, and users design and deploy filters to receive tailored information in real-time. Finally, all enriched alert packets are stored in a database (Apache HBase) for permanent access and for further analyses.

The Fink galaxy

Fink is made of several blocks that interconnect to provide all services:

All are open source, and we thank all contributors!

Useful links

Outside academia

Fink has participated to a number of events with the private sector, in particular:

  • 06/24: Fink was featured at the Kubecon Europe 2024.
  • 05/20: Fink has been selected to the Google Summer of Code 2020 (CERN-HSF org)! Congratulations to saucam who will work on the project this year. Focus on graph database with JanusGraph
  • 10/19: Fink was featured at the Spark+AI Summit Europe 2019.
  • 05/19: Fink has been selected to the Google Summer of Code 2019 (CERN-HSF org)! Congratulations to abhishekchauhn who will work on the project this year. Focus on alert redistribution with Apache Kafka