Skip to content

Latest commit

 

History

History
99 lines (83 loc) · 8.37 KB

api-docs.md

File metadata and controls

99 lines (83 loc) · 8.37 KB
Raw

API Specification

Packages :

kafkaconnect.operator.io/v1alpha1

Resource Types:

KafkaConnect

Field Description
apiVersion
string		 kafkaconnect.operator.io/v1alpha1
kind
string		 KafkaConnect
metadata
Kubernetes meta/v1.ObjectMeta		 Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
KafkaConnectSpec		 Refer to KafkaConnectSpec below
status
KafkaConnectStatus		 (optional)
Refer to KafkaConnectStatus below

KafkaConnectSpec

Field Description
connectors
KafkaConnectSpecConnectors		 (optional)
this field define the different connector run in this kafka connect cluster and ratio between connecto task and pod number for the autoscaler
ingressSpec
KafkaConnectSpecIngressSpec		 (optional)
this spec define how to create an ingress object to expose kafkaconnect rest api, check below section for more detail. The ingress object will not be created if this spec is absent.
kafkaConnectRestAPIPort
int32		 the kafka connect rest api port, operator will create service on this port and connect to this port to read/update connectors' config and status
podSpec
Kubernetes core/v1.PodSpec		 Refer to the Kubernetes API documentation for the fields of the podSpec field. The Kafka Connect Operator will create kubernetes deployment object base on the podSpec here
scaleStabilizationSec
int64		 (optional)
the duration in sec between each scale action, default 300

KafkaConnectSpecConnectors

Field Description
connectorConfigs
KafkaConnectSpecConnectorsConnectorConfigs array		 this field define each connector in this cluster, their task max and whether we should expose the lag as a metric or not.
initPodReplicas
int32		 (optional)
Default: 1
the initial Kubernetes Deployement replicas number. If taskPerPod > 0, the replicas number will be auto-scaled base on the total number of kafka connector tasks.
kafkaBrokers
string		 Kafka brokers, seprated by ,
keepUnknownConnectors
bool		 (optional)
Default: false
If false, it will delete all connector not defined in connectorConfigs during each reconcil action
taskPerPod
int32		 (optional)
Default: 1
The number of the kafka connector task per pod. This number will be used to auto-scale deployment's replicas base on the total number of kafka connector tasks in this cluster. If taskPerPod<=0, deployment's replicas will not be auto-scaled and initPodReplicas will be used in kubernetes deployment replicas

KafkaConnectSpecConnectorsConnectorConfigs

Field Description
name
string		 the name of the connector
url
string		 the link of a connector config
configMap
ConfigMap		 the configmap which contain the connector config, if both url and configMap exist, url will be used
taskMax
int32		 (optional)
the tasks.max in the connector config. If not defined, during the init step, operator will fill this field by the value in the connector config. The value in this field will overwrite the value in the connector config and it will be updated by auto-scaler
exposeLagMetric
boolean		 (optional)
default: false
If the operator need to expose the connector's total lag in a customer metric or not. If true a metric with name ${connectorName}-lag will be exposed for more example, check the autoscaler example

KafkaConnectSpecConnectorsConnectorConfigsConfigMap

Field Description
name
string		 the name of the k8s configmap
item
string		 the key of the data

KafkaConnectSpecIngressSpec

Field Description
parentDomain
string		 the parent domain from what the ingress will be created
style
enum
[pathStyle, domainStyle]		 (optional)
default: pathStyle
this field define how the ingress will be created. If style = pathStyle, ingress will be created as ${parentDomain}/${metadata.name} Else if style = domainStyle, ingress will be created as ${metadata.name}.${parentDomain}

KafkaConnectStatus

Field Description
connectorStatus
KafkaConnectStatusConnectorStatus array		 status of each connector in connectorConfigs field
lastScaleTime
date-time		 the date-time of last auto-scale operation
podNb
int32		 the current replicas number of the generated kubernetes deployment
updating
bool		 if this kafkaconnect object is being updated, due to object creation or autoscale

KafkaConnectStatusConnectorStatus

Field Description
name
string		 The name defined connectorConfigs
error
string		 The error message related to the connector config read or update via rest api
taskNb
int32		 The current task number of this kafka connector

KafkaConnectAutoScaler

Field Description
apiVersion
string		 kafkaconnect.operator.io/v1alpha1
kind
string		 KafkaConnectAutoScaler
metadata
Kubernetes meta/v1.ObjectMeta		 Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
KafkaConnectAutoScalerSpec		 Refer to KafkaConnectAutoScalerSpec below
status
Kubernetes autoscaling/v2beta2 HorizontalPodAutoscalerStatus		 (optional)
Refer to Kubernetes autoscaling/v2beta2 HorizontalPodAutoscalerStatus

KafkaConnectAutoScalerSpec

Field Description
kcScaleTargetRef
KafkaConnectorReference		 The field refer to a kafka connector in a specific kafka connect object
minTasks
int32		 (optional)
default: 1
this is the lower limit for the number of tasks to which the autoscaler can scale down.
maxTasks
int32		 this is the upper limit for the number of tasks to which the autoscaler can scale up.
metrics
Kubernetes autoscaling/v2beta2 MetricSpec array		 Refer to Kubernetes autoscaling/v2beta2 MetricSpec

KafkaConnectorReference

Field Description
name
string		 kafka connect object name
apiVersion
string		 kafkaconnect.operator.io/v1alpha1
kafkaConnectorName
string		 kafka connector name