Skip to content

This repository demonstrates some of the basic features of the latest release of Red Hat Data Grid 8 and how to deploy a RHDG cluster on OCP and RHEL

License

Notifications You must be signed in to change notification settings

romerobu/rhdg8-server

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

63 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Red Hat Data Grid 8 server

1. Introduction

Red Hat Data Grid is an in-memory, distributed, NoSQL datastore solution. Your applications can access, process, and analyze data at in-memory speed to deliver a superior user experience.

Red Hat Data Grid provides value as a standard architectural component in application infrastructures for a variety of real-world scenarios and use cases:

  • Data caching and transient data storage.

  • Primary data store.

  • Low latency compute grid.

1.1. Features and benefits

To support modern data management requirements with rapid data processing, elastic scalability, and high availability, Red Hat Data Grid offers:

  • NoSQL data store. Provides simple, flexible storage for a variety of data without the constraints of a fixed data model.

  • Apache Spark and Hadoop integration. Offers full support as an in-memory data store for Apache Spark and Hadoop, with support for Spark resilient distributed datasets (RDDs) and Discretized Streams (Dstreams), as well as Hadoop I/O format.

  • Rich querying. Provides easy search for objects using values and ranges, without the need for key-based lookups or an object’s exact location.

  • Polyglot client and access protocol support. Offers read/write capabilities that let applications written in multiple programming languages easily access and share data. Applications can access the data grid remotely, using REST, or Hot Rod—for Java™, C++, and .NET.

  • Distributed parallel execution. Quickly process large volumes of data and support long-running compute applications using simplified Map-Reduce parallel operations.

  • Flexible persistence. Increase the lifespan of information in the memory for improved durability through support for both shared nothing and shared database—RDBMS or NoSQL—architectures.

  • Comprehensive security. Authentication, role-based authorization, and access control are integrated with existing security and identity structures to give only trusted users, services, and applications access to the data grid.

  • Cross-datacenter replication. Replicate applications across datacenters and achieve high availability to meet service-level agreement (SLA) requirements for data within and across datacenters.

  • Rolling upgrades. Upgrade your cluster without downtime for continuous, uninterrupted operations for remote users and applications.

1.2. RHDG Operator

The Data Grid Operator provides operational intelligence and reduces management complexity for deploying Data Grid on OpenShift, automatically upgrading clusters when new image versions become available.

In order to upgrade Data Grid clusters, the operator checks the version of the image of each Data Grid node. If the operator determines that a new version of the image is available, it gracefully shuts down all nodes, applies the new image, and restarts the nodes.

On OpenShift, the Operator Lifecycle Manager (OLM) enables upgrades for Data Grid Operator.

2. Deploying RHDG on RHEL

This section explains how to configure, run, and monitor Data Grid servers. The binaries are available on the RH downloads website.

2.1. Running RHDG locally in standalone mode

This is the basic installation option, valid for basic exploration and development purposes. Follow these steps to run the server in this mode:

Unzip the server
cd ~/Downloads
unzip redhat-datagrid-8.1.0-server.zip
Create the developer user
./redhat-datagrid-8.1.0-server/bin/cli.sh user create developer -p developer
Launch the server
./redhat-datagrid-8.1.0-server/bin/server.sh
16:33:08,457 INFO  (main) [BOOT] JVM OpenJDK 64-Bit Server VM Red Hat, Inc. 11.0.9+11
[...]
16:33:13,327 INFO  (main) [org.infinispan.SERVER] ISPN080004: Protocol SINGLE_PORT listening on 127.0.0.1:11222
16:33:13,327 INFO  (main) [org.infinispan.SERVER] ISPN080034: Server 'hat-42208' listening on http://127.0.0.1:11222
16:33:13,327 INFO  (main) [org.infinispan.SERVER] ISPN080001: Red Hat Data Grid Server 8.1.0.GA started in 4806ms

2.2. Running RHDG locally in clustering mode

Unzip the server as in the previous example, locate the folder named server, duplicate it and then use one with each server instance.

Unzip the server
cd ~/Downloads
unzip redhat-datagrid-8.1.0-server.zip
Duplicate the server config
cp redhat-datagrid-8.1.0-server/server redhat-datagrid-8.1.0-server/server-01
cp redhat-datagrid-8.1.0-server/server redhat-datagrid-8.1.0-server/server-02
Create the developer user in both instances
./redhat-datagrid-8.1.0-server/bin/cli.sh user create developer -p developer --server-root=server-01
./redhat-datagrid-8.1.0-server/bin/cli.sh user create developer -p developer --server-root=server-02
Launch both server instances
./redhat-datagrid-8.1.0-server/bin/server.sh --node-name=node-01 --server-root=redhat-datagrid-8.1.0-server/server-01 --port-offset=0
./redhat-datagrid-8.1.0-server/bin/server.sh --node-name=node-02 --server-root=redhat-datagrid-8.1.0-server/server-02 --port-offset=100

After running both commands, you will see in both terminals similar logs to the ones shown below:

[...]
20:19:29,614 INFO  (main) [org.infinispan.SERVER] ISPN080034: Server 'node-01' listening on http://127.0.0.1:11222
20:19:29,614 INFO  (main) [org.infinispan.SERVER] ISPN080001: Red Hat Data Grid Server 8.1.0.GA started in 5585ms
20:19:36,637 INFO  (jgroups-8,node-01) [org.infinispan.CLUSTER] ISPN000094: Received new cluster view for channel cluster: [node-01|1] (2) [node-01, node-02]
20:19:36,647 INFO  (jgroups-8,node-01) [org.infinispan.CLUSTER] ISPN100000: Node node-02 joined the cluster
20:19:37,365 INFO  (jgroups-5,node-01) [org.infinispan.CLUSTER] [Context=org.infinispan.CLIENT_SERVER_TX_TABLE]ISPN100002: Starting rebalance with members [node-01, node-02], phase READ_OLD_WRITE_ALL, topology id 2
[...]
20:19:38,463 INFO  (jgroups-5,node-01) [org.infinispan.CLUSTER] [Context=___hotRodTopologyCache_hotrod]ISPN100010: Finished rebalance with members [node-01, node-02], topology id 5

3. Deploying RHDG on OCP using the Operator

An Operator is a method of packaging, deploying and managing a Kubernetes-native application. A Kubernetes-native application is an application that is both deployed on Kubernetes and managed using the Kubernetes APIs and kubectl tooling.

Install Data Grid Operator into a OpenShift namespace to create and manage Data Grid clusters.

3.1. Deploying the RHDG operator

Create subscriptions to Data Grid Operator on OpenShift so you can install different Data Grid versions and receive automatic updates.

To deploy the RHDG operator, you will need to create three different objects:

  • Two Openshift projects that will contain the operator and the objects of the RHDG cluster.

  • An OperatorGroup, which provides multitenant configuration to OLM-installed Operators. An Operator group selects target namespaces in which to generate required RBAC access for its member Operators. As we are not deploying our operator in the default namespace (openshift-operators), we will need to create one to set the namespaces where the Data Grid operator will be able to create and monitorize clusters.

ℹ️
The OperatorGroup resource allows to configure four possible namespace-scopes for the operator. Please check Annex: Configure the scope of your operator before executing the commands of this section.
  • A Subscription, which represents an intention to install an Operator. It is the custom resource that relates an Operator to a CatalogSource. Subscriptions describe which channel of an Operator package to subscribe to, and whether to perform updates automatically or manually.

I have created an OCP template to quickly deploy this operator. Just execute the following command have it up and running on your cluster.

Bear in mind that you will need cluster-admin permissions to deploy an operator, as it is necessary to create cluster-wide CRDs (Custom Resource Definitions).
oc process -f templates/rhdg-01-operator.yaml | oc apply -f -

This template provides two parameters to modify the project where the operator and the cluster is installed. It is possible to deploy both on the same project or in different projects. By default, values are:

  • OPERATOR_NAMESPACE = rhdg8-operator

  • CLUSTER_NAMESPACE = rhdg8

Modify them just passing arguments to the template:

oc process -f templates/rhdg-01-operator.yaml -p OPERATOR_NAMESPACE="other-namespace" -p CLUSTER_NAMESPACE="another-namespace" | oc apply -f -

It is also possible to install the operator from the web console. For more information, please check the official documentation.

3.2. Deploying a RHDG cluster

Data Grid Operator lets you create, configure, and manage Data Grid clusters. Data Grid Operator adds a new Custom Resource (CR) of type Infinispan that lets you handle Data Grid clusters as complex units on OpenShift.

Data Grid Operator watches for Infinispan Custom Resources (CR) that you use to instantiate and configure Data Grid clusters and manage OpenShift resources, such as StatefulSets and Services. In this way, the Infinispan CR is your primary interface to Data Grid on OpenShift.

I have created an OCP template to quickly deploy a basic RHDG cluster with 3 replicas. Execute the following command have it up and running on your cluster.

oc process -f templates/rhdg-02-cluster.yaml | oc apply -f -

This template provides two parameters to modify the project where the cluster is installed and the name of the cluster to deploy. The cluster namespace should be the same as in the previous step. By default, values are:

  • CLUSTER_NAMESPACE = rhdg8

  • CLUSTER_NAME = rhdg

Modify them just passing arguments to the template:

oc process -f templates/rhdg-02-cluster.yaml -p CLUSTER_NAMESPACE="another-namespace" -p CLUSTER_NAME="my-cluster" | oc apply -f -

3.3. Creating RHDG caches using the Operator CRD

⚠️
Creating caches with Data Grid Operator is available as a technology preview. If you want to create caches in a production environment, I encourage you to use the REST or Java alternatives.

Data Grid stores entries into caches, which can be created using several methods: REST, CLI, programmatically using the Java Client or using the Cache CRD. In this section, we will explore how to create caches using the Operator.

ℹ️
For other ways of creating caches, please check this other Git repository with information about the Data Grid client.

To create caches with Data Grid Operator, you use Cache CRs to add caches from templates or XML configuration. Bear in mind the following contrains:

  • You can create a single cache for each Cache CR.

  • If you edit caches in the OpenShift Web Console, changes do not take effect on the Data Grid cluster. You must delete the CR and create it again with the new configuration.

  • Deleting Cache CRs in the OpenShift Web Console, does not remove caches from Data Grid clusters. You must delete caches through the console or CLI.

I have created an OCP template to quickly set up two caches on the RHDG cluster:

  • operator-cache-01: Based on an xml configuration.

  • operator-cache-02: Based on an already defined templated.

In order to apply this template, just execute the following command:

oc process -f templates/rhdg-03-caches.yaml | oc apply -f -

This template provides two parameters to modify the project where the cluster is installed and the name of the cluster to deploy. The cluster namespace should be the same as in the previous step. By default, values are:

  • CLUSTER_NAMESPACE = rhdg8

  • CLUSTER_NAME = rhdg

Modify them just passing arguments to the template:

oc process -f templates/rhdg-03-caches.yaml -p CLUSTER_NAMESPACE="another-namespace" -p CLUSTER_NAME="my-cluster" | oc apply -f -

Interact with the newly created caches with he following commands:

# Set your variables. These are default:
CLUSTER_NAMESPACE="rhdg8"
CLUSTER_NAME="rhdg"
RHDG_URL=$(oc get route ${CLUSTER_NAME}-external -n ${CLUSTER_NAMESPACE} -o template='https://{{.spec.host}}')

# Check all the caches on your cluster
curl -X GET -k -u developer:developer -H "Content-Type: application/json" ${RHDG_URL}/rest/v2/caches | jq

# Check information about an specific cache
curl -X GET -k -u developer:developer -H "Content-Type: application/json" ${RHDG_URL}/rest/v2/caches/${CACHE_NAME} | jq

# Delete a cache
curl -X DELETE -k -u developer:developer ${RHDG_URL}/rest/v2/caches/${CACHE_NAME}

For more information about how to create caches using the CRD, please check the official documentation.

3.4. Monitoring RHDG with Prometheus

Data Grid exposes a metrics endpoint that provides statistics and events to Prometheus.

After installing OpenShift Container Platform 4.6, cluster administrators can optionally enable monitoring for user-defined projects. By using this feature, cluster administrators, developers, and other users can specify how services and pods are monitored in their own projects. You can then query metrics, review dashboards, and manage alerting rules and silences for your own projects in the OpenShift Container Platform web console. We are going to take advantage of this feature.

⚠️
Enabling monitoring for user-defined projects

Monitoring of user-defined projects is not enabled by default. To enable it, you need to modify a Configmap of the openshift-monitoring. You need permissions to create and modify Configmaps in this project. You only have to execute this command once per namespace. Please, do not execute it before checking if this was done before, you can override work from your colleagues

oc apply -f templates/rhdg-02-ocp-user-workload-monitoring.yaml

After executing the command above, you will see some pods in the following namespace:

oc get pods -n openshift-user-workload-monitoring

I have created an OCP template to quickly configure metrics monitorization of a RHDG cluster. Execute the following command:

oc process -f templates/rhdg-04-monitoring.yaml | oc apply -f -

This template provides two parameters to modify the project where the cluster was installed and the name of the cluster itself. By default, values are:

  • CLUSTER_NAMESPACE = rhdg8

  • CLUSTER_NAME = rhdg

Modify them just passing arguments to the template:

oc process -f templates/rhdg-04-monitoring.yaml -p CLUSTER_NAMESPACE="another-namespace" -p CLUSTER_NAME="my-cluster" | oc apply -f -

For more information, access the Openshift documentation for the monitoring stack and the RHDG documenation to configure monitoring for RHDG 8 on OCP.

4. Monitoring RHDG with Grafana

A typical OpenShift monitoring stack includes Prometheus for monitoring both systems and services, and Grafana for analyzing and visualizing metrics.

Administrators are often looking to write custom queries and create custom dashboards in Grafana. However, Grafana instances provided with the monitoring stack (and its dashboards) are read-only. To solve this problem, we can use the community-powered Grafana operator provided by OperatorHub.

To deploy the community-powered Grafana operator on OCP 4.6 just follow these steps:

4.1. Deploy the Grafana operator

oc process -f templates/grafana-01-operator.yaml | oc apply -f -

4.2. Create a Grafana instance

Now, we will create a Grafana instance using the operator:

oc process -f templates/grafana-02-instance.yaml | oc apply -f -

4.3. Create a Grafana data source

Now, we will create a Grafana data source:

PROJECT=grafana

oc adm policy add-cluster-role-to-user cluster-monitoring-view -z grafana-serviceaccount -n ${PROJECT}
BEARER_TOKEN=$(oc serviceaccounts get-token grafana-serviceaccount -n ${PROJECT})
oc process -f templates/grafana-03-datasource.yaml -p BEARER_TOKEN=${BEARER_TOKEN} | oc apply -f -

4.4. Create a Grafana dashboard

Now, we will create a Grafana dashboard:

DASHBOARD_NAME="grafana-dashboard-rhdg8"
# Create a configMap containing the Dashboard
oc create configmap $DASHBOARD_NAME --from-file=dashboard=grafana/$DASHBOARD_NAME.json -n $PROJECT
# Create a Dashboard object that automatically updates Grafana
oc process -f templates/grafana-04-dashboard.yaml -p DASHBOARD_NAME=$DASHBOARD_NAME | oc apply -f -

4.5. Access Grafana as admin

After accessing Grafana using the OCP SSO, you may log in as admin. Retrieve the credentials from the secret using the following commands:

oc get secret grafana-admin-credentials -n $PROJECT -o jsonpath='{.data.GF_SECURITY_ADMIN_USER}' | base64 --decode
oc get secret grafana-admin-credentials -n $PROJECT -o jsonpath='{.data.GF_SECURITY_ADMIN_PASSWORD}' | base64 --decode

For more information, access the Grafana main documentation or the Grafana operator documentation.

Annex: Configure the scope of your operator

An Operator group, defined by the OperatorGroup resource, provides multitenant configuration to OLM-installed Operators. An Operator group selects target namespaces in which to generate required RBAC access for its member Operators.

If you want to modify the default behavior of the template provided in this repository, modify lines 26 to 33 of this template.

1) AllNamespaces: The Operator can be a member of an Operator group that selects all namespaces (target namespace set is the empty string ""). This configuration allows us to create DG clusters in every namespace of the cluster:

- apiVersion: operators.coreos.com/v1
  kind: OperatorGroup
  metadata:
    name: datagrid
    namespace: ${OPERATOR_NAMESPACE}
  spec: {}

2) MultiNamespace: The Operator can be a member of an Operator group that selects more than one namespace. Choose this option if you want to have several operators that manage RHDG clusters. For example, if you want to have a different operator per Business Unit managing several Openshift projects:

- apiVersion: operators.coreos.com/v1
  kind: OperatorGroup
  metadata:
    name: datagrid
    namespace: ${OPERATOR_NAMESPACE}
  spec:
    targetNamespaces:
      - ${CLUSTER_NAMESPACE-1}
      - ${CLUSTER_NAMESPACE-2}

3) SingleNamespace: The Operator can be a member of an Operator group that selects one namespace. This is useful if we want every application (Each OCP namespace) to be able to configure and deploy their own DG clusters:

- apiVersion: operators.coreos.com/v1
  kind: OperatorGroup
  metadata:
    name: datagrid
    namespace: ${OPERATOR_NAMESPACE}
  spec:
    targetNamespaces:
      - ${CLUSTER_NAMESPACE}

For more information, check the Openshift documentation about Operator Groups and the official documentation to install DG on Openshift.

Annex: Stern - Tail logs from multiple pods

In some situations, you will need to monitor logs from several pods of the same application and maybe you want to check to which pod did the request arrived. Stern allows you to tail multiple pods on Kubernetes and multiple containers within the pod. Each result is color coded for quicker debugging.

First, you will need to install it on your machine. After that, log in to your cluster and monitoring the previous deployment is as simple as executing the following command:

stern --namespace=$CLUSTER_NAMESPACE -l clusterName=$CLUSTER_NAME

The previous command will show all the logs from all the pods from a namespace that contain a given label.

There are many filters and configuration options. Check the documentation for a full list of them

Annex: Deploying the Infinispan operator

The same configuration rules from the previous chapter apply.

oc process -f templates/infinispan-01-operator.yaml -p OPERATOR_NAMESPACE="infinispan-operator" -p CLUSTER_NAMESPACE="infinispan" | oc apply -f -

oc process -f templates/infinispan-02-cluster.yaml -p CLUSTER_NAMESPACE="infinispan" -p CLUSTER_NAME="infinispan" | oc apply -f -

It is also possible to install the operator from the web console. For more information, please check the official documentation.

Annex: Deploying RHDG 8 without operator

⚠️
Bear in mind that this template is for testing purposes and that the installation made with it will not benefit from any kind of Red Hat support.

If you need to test any configuration that the operator does not provide, it is possible to deploy the RHDG cluster manually creating all the Openshift resources that the Operator manages automagically.

By default, the template deploys a RHDG cluster named rhdg in the project rhdg8-operatorless. Deploy your cluster performing the following command:

oc process -f templates/rhdg-99-operatorless.yaml | oc apply -f -

Due to the limitation of OCP templates, if you want to change the cluster name and the namespace, you will have to edit lines #103, and #107 of the file templates/rhdg-99-operatorless.yaml. This is the pattern that you have to follow [Substitute the ENV_VARS with the correct values]:

  data:
    infinispan.yaml: |
      clusterName: ${CLUSTER_NAME}
      jgroups:
        transport: tcp
        dnsPing:
          query: ${CLUSTER_NAME}-ping.${CLUSTER_NAMESPACE}.svc.cluster.local
        diagnostics: false

After modifying the yaml, process the template with the correct params:

oc process -f templates/rhdg-99-operatorless.yaml -p CLUSTER_NAMESPACE="another-namespace" -p CLUSTER_NAME="my-cluster" | oc apply -f -

About

This repository demonstrates some of the basic features of the latest release of Red Hat Data Grid 8 and how to deploy a RHDG cluster on OCP and RHEL

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Shell 100.0%