Add new chart for ScalarDB Analytics with PostgreSQL #242
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| # Patterns to ignore when building packages. | ||
| # This supports shell glob matching, relative path matching, and | ||
| # negation (prefixed with !). Only one pattern per line. | ||
| .DS_Store | ||
| # Common VCS dirs | ||
| .git/ | ||
| .gitignore | ||
| .bzr/ | ||
| .bzrignore | ||
| .hg/ | ||
| .hgignore | ||
| .svn/ | ||
| # Common backup files | ||
| *.swp | ||
| *.bak | ||
| *.tmp | ||
| *.orig | ||
| *~ | ||
| # Various IDEs | ||
| .project | ||
| .idea/ | ||
| *.tmproj | ||
| .vscode/ |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| apiVersion: v2 | ||
| name: scalardb-analytics-postgresql | ||
| description: ScalarDB Analytics with PostgreSQL | ||
| type: application | ||
| version: 1.0.0-SNAPSHOT | ||
| appVersion: 3.10.2 | ||
| deprecated: false | ||
| icon: https://scalar-labs.com/wp-content/themes/scalar/assets/img/logo_scalar.svg | ||
| keywords: | ||
| - scalardb | ||
| - scalardb-analytics | ||
| - analytics | ||
| home: https://scalar-labs.com/ | ||
| sources: | ||
| - https://github.com/scalar-labs/scalardb-analytics-postgresql | ||
| maintainers: | ||
| - name: Takanori Yokoyama | ||
| email: [email protected] | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| # scalardb-analytics-postgresql | ||
|
There was a problem hiding this comment. This file is automatically generated based on the There was a problem hiding this comment. Just curious. How is this file generated? There was a problem hiding this comment. You can generate this file by using Also, in our repository, you can run the |
||
|
|
||
| ScalarDB Analytics with PostgreSQL | ||
| Current chart version is `1.0.0-SNAPSHOT` | ||
|
|
||
| ## Values | ||
|
|
||
| | Key | Type | Default | Description | | ||
| |-----|------|---------|-------------| | ||
| | fullnameOverride | string | `""` | String to fully override scalardb-analytics-postgresql.fullname template | | ||
| | nameOverride | string | `""` | String to partially override scalardb-analytics-postgresql.fullname template (will maintain the release name) | | ||
| | scalardbAnalyticsPostgreSQL.affinity | object | `{}` | The affinity/anti-affinity feature, greatly expands the types of constraints you can express. | | ||
| | scalardbAnalyticsPostgreSQL.databaseProperties | string | The minimum template of database.properties is set by default. | The database.properties to access the underlying databases that Schema Importer will load schemas into PostgreSQL. | | ||
| | scalardbAnalyticsPostgreSQL.extraVolumeMounts | list | `[]` | Defines additional volume mounts. | | ||
| | scalardbAnalyticsPostgreSQL.extraVolumes | list | `[]` | Defines additional volumes. If you want to mount a volume for PGDATA, you can mount extra volumes. | | ||
| | scalardbAnalyticsPostgreSQL.image.pullPolicy | string | `"IfNotPresent"` | Specify a image pulling policy. | | ||
| | scalardbAnalyticsPostgreSQL.image.repository | string | `"ghcr.io/scalar-labs/scalardb-analytics-postgresql"` | Docker image repository of ScalarDB Analytics with PostgreSQL. | | ||
| | scalardbAnalyticsPostgreSQL.image.tag | string | `""` | Override the image tag whose default is the chart appVersion | | ||
| | scalardbAnalyticsPostgreSQL.imagePullSecrets | list | `[{"name":"reg-docker-secrets"}]` | Optionally specify an array of imagePullSecrets. Secrets must be manually created in the namespace. | | ||
| | scalardbAnalyticsPostgreSQL.nodeSelector | object | `{}` | nodeSelector is form of node selection constraint. | | ||
| | scalardbAnalyticsPostgreSQL.podAnnotations | object | `{}` | Pod annotations for the scalardb-analytics-postgresql deployment. | | ||
| | scalardbAnalyticsPostgreSQL.podSecurityContext | object | `{"fsGroup":201,"seccompProfile":{"type":"RuntimeDefault"}}` | PodSecurityContext holds pod-level security attributes and common container settings. | | ||
| | scalardbAnalyticsPostgreSQL.podSecurityContext.fsGroup | int | `201` | To work ScalarDB Analytics with PostgreSQL properly, you must set "201" to "podSecurityContext.fsGroup". | | ||
| | scalardbAnalyticsPostgreSQL.postgresql.databaseName | string | `"scalardb"` | The database name that you create in PostgreSQL. Schema Importer create some object such a view of ScalarDB Analytics with PostgreSQL in this database. | | ||
| | scalardbAnalyticsPostgreSQL.postgresql.secretName | string | `"scalardb-analytics-postgresql-superuser-password"` | The secret resource name that includes superuser password for PostgreSQL. | | ||
| | scalardbAnalyticsPostgreSQL.replicaCount | int | `3` | Default values for number of replicas. | | ||
| | scalardbAnalyticsPostgreSQL.resources | object | `{}` | Resources allowed to the pod. | | ||
| | scalardbAnalyticsPostgreSQL.secretName | string | `""` | Secret name that includes sensitive data such as credentials. Each secret key is passed to Pod as environment variables using envFrom. | | ||
| | scalardbAnalyticsPostgreSQL.securityContext | object | `{"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"runAsNonRoot":true,"runAsUser":999}` | Setting security context at the pod applies those settings to all containers in the pod. | | ||
| | scalardbAnalyticsPostgreSQL.securityContext.allowPrivilegeEscalation | bool | `false` | AllowPrivilegeEscalation controls whether a process can gain more privileges than its parent process. | | ||
| | scalardbAnalyticsPostgreSQL.securityContext.capabilities | object | `{"drop":["ALL"]}` | Capabilities (specifically, Linux capabilities), are used for permission management in Linux. Some capabilities are enabled by default. | | ||
| | scalardbAnalyticsPostgreSQL.securityContext.runAsNonRoot | bool | `true` | Containers should be run as a non-root user with the minimum required permissions (principle of least privilege). | | ||
| | scalardbAnalyticsPostgreSQL.securityContext.runAsUser | int | `999` | The PostgreSQL official image use the "postgres (UID=999)" user by default. | | ||
| | scalardbAnalyticsPostgreSQL.service.annotations | object | `{}` | Service annotations, e.g: prometheus, etc. | | ||
| | scalardbAnalyticsPostgreSQL.service.ports.postgresql.port | int | `5432` | PostgreSQL public port | | ||
| | scalardbAnalyticsPostgreSQL.service.ports.postgresql.protocol | string | `"TCP"` | PostgreSQL protocol | | ||
| | scalardbAnalyticsPostgreSQL.service.ports.postgresql.targetPort | int | `5432` | PostgreSQL k8s internal port | | ||
| | scalardbAnalyticsPostgreSQL.service.type | string | `"ClusterIP"` | service types in kubernetes | | ||
| | scalardbAnalyticsPostgreSQL.serviceAccount.automountServiceAccountToken | bool | `false` | Specify to mount a service account token or not. | | ||
| | scalardbAnalyticsPostgreSQL.serviceAccount.serviceAccountName | string | `""` | Name of the existing service account resource. | | ||
| | scalardbAnalyticsPostgreSQL.strategy.rollingUpdate.maxSurge | string | `"25%"` | The number of pods that can be created above the desired amount of pods during an update | | ||
| | scalardbAnalyticsPostgreSQL.strategy.rollingUpdate.maxUnavailable | string | `"25%"` | The number of pods that can be unavailable during the update process | | ||
| | scalardbAnalyticsPostgreSQL.strategy.type | string | `"RollingUpdate"` | New pods are added gradually, and old pods are terminated gradually, e.g: Recreate or RollingUpdate | | ||
| | scalardbAnalyticsPostgreSQL.tolerations | list | `[]` | Tolerations are applied to pods, and allow (but do not require) the pods to schedule onto nodes with matching taints. | | ||
| | schemaImporter.entrypointShell.maxRetryCount | int | `10` | Maximum retry count of Schema Importer in entrypoint.sh. | | ||
| | schemaImporter.entrypointShell.retryInterval | int | `3` | Retry interval of Schema Importer in entrypoint.sh. | | ||
| | schemaImporter.image.pullPolicy | string | `"IfNotPresent"` | Specify a image pulling policy. | | ||
| | schemaImporter.image.repository | string | `"ghcr.io/scalar-labs/scalardb-analytics-postgresql-schema-importer"` | Docker image repository of Schema Importer. | | ||
| | schemaImporter.image.tag | string | `""` | Override the image tag whose default is the chart appVersion | | ||
| | schemaImporter.namespaces | list | `[]` | | | ||
| | schemaImporter.securityContext | object | `{"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"runAsNonRoot":true}` | Setting security context at the pod applies those settings to all containers in the pod. | | ||
| | schemaImporter.securityContext.allowPrivilegeEscalation | bool | `false` | AllowPrivilegeEscalation controls whether a process can gain more privileges than its parent process. | | ||
| | schemaImporter.securityContext.capabilities | object | `{"drop":["ALL"]}` | Capabilities (specifically, Linux capabilities), are used for permission management in Linux. Some capabilities are enabled by default. | | ||
| | schemaImporter.securityContext.runAsNonRoot | bool | `true` | Containers should be run as a non-root user with the minimum required permissions (principle of least privilege). | | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| {{ template "chart.header" . }} | ||
|
|
||
| {{ template "chart.description" . }} | ||
| Current chart version is `{{ template "chart.version" . }}` | ||
|
|
||
| {{ template "chart.requirementsSection" . }} | ||
|
|
||
| {{ template "chart.valuesSection" . }} |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| scalardbAnalyticsPostgreSQL: | ||
| databaseProperties: | | ||
| scalar.db.storage=jdbc | ||
| scalar.db.contact_points=jdbc:postgresql://postgresql.default.svc.cluster.local:5432/postgres | ||
| scalar.db.username=postgres | ||
| scalar.db.password=postgres | ||
|
|
||
| schemaImporter: | ||
| namespaces: | ||
| - ct | ||
|
Comment on lines
+1
to
+10
There was a problem hiding this comment. This file is a custom values file for the testing in the CI. |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| {{/* vim: set filetype=mustache: */}} | ||
| {{/* | ||
| Expand the name of the chart. | ||
| */}} | ||
| {{- define "scalardb-analytics-postgresql.name" -}} | ||
| {{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }} | ||
| {{- end }} | ||
|
|
||
| {{/* | ||
| Create a default fully qualified app name. | ||
| We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). | ||
| If release name contains chart name it will be used as a full name. | ||
| */}} | ||
| {{- define "scalardb-analytics-postgresql.fullname" -}} | ||
| {{- if .Values.fullnameOverride }} | ||
| {{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }} | ||
| {{- else }} | ||
| {{- $name := default .Chart.Name .Values.nameOverride }} | ||
| {{- if contains $name .Release.Name }} | ||
| {{- .Release.Name | trunc 63 | trimSuffix "-" }} | ||
| {{- else }} | ||
| {{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }} | ||
| {{- end }} | ||
| {{- end }} | ||
| {{- end }} | ||
|
|
||
| {{/* | ||
| Create chart name and version as used by the chart label. | ||
| */}} | ||
| {{- define "scalardb-analytics-postgresql.chart" -}} | ||
| {{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }} | ||
| {{- end }} | ||
|
|
||
| {{/* | ||
| Common labels | ||
| */}} | ||
| {{- define "scalardb-analytics-postgresql.labels" -}} | ||
| helm.sh/chart: {{ include "scalardb-analytics-postgresql.chart" . }} | ||
| {{ include "scalardb-analytics-postgresql.selectorLabels" . }} | ||
| {{- if .Chart.AppVersion }} | ||
| app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} | ||
| {{- end }} | ||
| app.kubernetes.io/managed-by: {{ .Release.Service }} | ||
| {{- end }} | ||
|
|
||
| {{/* | ||
| Selector labels | ||
| */}} | ||
| {{- define "scalardb-analytics-postgresql.selectorLabels" -}} | ||
| app.kubernetes.io/name: {{ include "scalardb-analytics-postgresql.name" . }} | ||
| app.kubernetes.io/instance: {{ .Release.Name }} | ||
| app.kubernetes.io/app: scalardb-analytics-postgresql | ||
| {{- end }} | ||
|
|
||
| {{/* | ||
| Create the name of the service account to use | ||
| */}} | ||
| {{- define "scalardb-analytics-postgresql.serviceAccountName" -}} | ||
| {{- if .Values.scalardbAnalyticsPostgreSQL.serviceAccount.serviceAccountName }} | ||
| {{- .Values.scalardbAnalyticsPostgreSQL.serviceAccount.serviceAccountName }} | ||
| {{- else }} | ||
| {{- print (include "scalardb-analytics-postgresql.fullname" .) "-sa" | trunc 63 | trimSuffix "-" }} | ||
| {{- end }} | ||
| {{- end }} |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: {{ include "scalardb-analytics-postgresql.fullname" . }}-database-properties | ||
| namespace: {{ .Release.Namespace }} | ||
| data: | ||
| # Create a database.properties file which is config file of ScalarDB Analytics with PostgreSQL. | ||
| database.properties: | ||
| {{- toYaml .Values.scalardbAnalyticsPostgreSQL.databaseProperties | nindent 4 }} | ||
| --- | ||
| apiVersion: v1 | ||
| kind: ConfigMap | ||
| metadata: | ||
| name: {{ include "scalardb-analytics-postgresql.fullname" . }}-entrypoint-shell | ||
| namespace: {{ .Release.Namespace }} | ||
| data: | ||
| entrypoint.sh: | | ||
| #!/bin/bash | ||
|
Comment on lines
+17
to
+18
There was a problem hiding this comment. At this time, I create the The main purpose of this shell is to implement the We need this file on the helm chart side to run the existing stable versions (v3.10) images. However, I will create this There was a problem hiding this comment. So, this runs Schema Importer every time the pod starts? There was a problem hiding this comment. Yes. That's right. For example, we run Schema Importer in the following cases:
There was a problem hiding this comment. Thank you for the response. I have one more question: why is it necessary to include the Schema Importer in the pod? I'm considering the possibility of running the Schema Importer separately, outside of the pod. There was a problem hiding this comment. @brfrn169 In conclusion, I want to deploy ScalarDB Analytics with PostgreSQL as a For example, I want to avoid manually running the Schema Importer in the following cases.
ChallengesScalarDB Schema Loader and ScalarDL Schema Loader create database schemas (i.e., create some tables or objects) on the backend database side. In other words, those objects are persisted by the backend database. So, basically, ScalarDB/ScalarDL has no state in themselves. However, Schema Importer creates some objects (e.g., foreign servers, extensions, and views) on the ScalarDB Analytics with PostgreSQL side. In other words, strictly, ScalarDB Analytics with PostgreSQL has states. It's a So, if the pod is restarted for some reason, the loaded objects are lost. In this case, we have to re-run Schema Importer to re-load all objects on the PostgreSQL. As well as the pods restart, we have to run the Schema Importer if we scale out (add a new pod) to create some objects in the new pod (in the new PostgreSQL). This is because each pod (each PostgreSQL) has objects (e.g., views) respectively. Solution 1 (make it
|
||
|
|
||
| MAX_RETRY_COUNT=${SCHEMA_IMPORTER_MAX_RETRY_COUNT} | ||
| RETRY_INTERVAL=${SCHEMA_IMPORTER_RETRY_INTERVAL} | ||
| COUNT=0 | ||
|
|
||
| # A ScalarDB Analytics with PostgreSQL container and a Schema Importer | ||
| # container start at the same time in a pod. And, it takes a few seconds | ||
| # to start PostgreSQL. So, first, we wait ${RETRY_INTERVAL} seconds | ||
| # before we run Schema Importer. | ||
| echo "Sleep ${RETRY_INTERVAL} seconds to wait for PostgreSQL start." | ||
|
|
||
| while [[ ${COUNT} -lt ${MAX_RETRY_COUNT} ]] | ||
| do | ||
| sleep ${RETRY_INTERVAL} | ||
| echo "Retry count: ${COUNT}" | ||
| COUNT=$((COUNT + 1)) | ||
| "$@" | ||
| if [[ $? -ne 0 ]]; then | ||
| echo "INFO: Schema Importer failed. Will retry after ${RETRY_INTERVAL} seconds." | ||
| else | ||
| # The Schema Importer container will sleep infinitely. This is | ||
| # because Kubernetes will restart a pod if the sidecar container | ||
| # exits. In other words, if we exit the Schema Importer container, | ||
| # it causes CrashLoopBackOff. | ||
| echo "INFO: Schema Importer succeeded. This container sleeps infinitely." | ||
| sleep inf | ||
| fi | ||
| done | ||
|
|
||
| echo "ERROR: Schema Importer failed ${MAX_RETRY_COUNT} times. Please check your configuration." >&2 | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At this time, ScalarDB Analytics with PostgreSQL does not release the
SNAPSHOTversion. So, I set the latest stable version.However, we are working on releasing the
SNAPSHOTversion on the ScalarDB Analytics with PostgreSQL side. In the future, we will set theSNAPSHOTversion here.