Use OCI Borneo in Github Actions Pipeline

[adam-sav]

I have a question about OCI borneo when trying to run pytest in a github actions pipeline. Specifically, I have a python FastAPI instance that is running uvicorn inside of a docker container. When this container is running in our development or production environment, we use a private-public key pair for setting up our NoSQL handle. However, our deployment pipeline needs to run pytest, and the pipeline intentionally does not load the private key into the environment variables. I would like to know a way to connect to borneo using environment variables and not config files so that our github pipeline can run pytest without needing the private key.

[gmfeinberg]

Hi. Good question. In the OCI environment borneo uses the OCI Python SDK for authentication (creation of a signer). I don't think that SDK has a way to perform API key-based auth via environment variables (as is possible with Instance Principal and Resource Principal auth). You might check with that SDK or group, or (probably the easiest) implement your own environment variables that you use to fill in the appropriate parameters to borneo's SignatureProvider interface

[adam-sav]

Ah yes, let me specify. We use the environment variables to fill in the arguments for a SignatureProvider(). We don't just use them directly.

[dario-vega]

Hi @adam-sav

Your question is simpler than I understood. Can you try this?

def get_connection(region, comp):
    print("Connecting to the Oracle NoSQL Cloud Service")
    provider = SignatureProvider(
      tenant_id=os.getenv('OCI_TENANCY')
      ,user_id=os.getenv('OCI_USER')
      ,fingerprint=os.getenv('OCI_FINGERPRINT')
      ,private_key=os.getenv('OCI_KEY_CONTENT')
    )
    config = NoSQLHandleConfig(region, provider)
    config.set_default_compartment(comp)
    return(NoSQLHandle(config))

The private_key parameter of SignatureProvider is the path to the private key or the private key content.

In my local env, I have a file called h.pem with the private key, so to have the content, I am executing it.

export OCI_KEY_CONTENT=$(cat $HOME/.oci/h.pem)

In GitHub Actions, you can use a secret or similar.

    env:
      OCI_USER: ${{ secrets.OCI_USER }}
      OCI_TENANCY: ${{ secrets.OCI_TENANCY }}
      OCI_FINGERPRINT: ${{ secrets.OCI_FINGERPRINT }}
      OCI_KEY_CONTENT: ${{ secrets.OCI_KEY_CONTENT }}
      OCI_REGION: ${{ secrets.OCI_REGION }}

You need the content of the private key to connect using the API key.

Note: also set the parameter passphrase if needed

[adam-sav]

Alright, cool. That's the way I have it right now, but I wasn't sure if there was anything else to it or a better way to go about it. Thank you.

[dario-vega]

As we discussed, I only use API keys when it is the only possibility. This is the case when using GitHub Actions. You have other types of auth -

• instance_obo_user
• instance_principal
• resource_principal
• security_token
• OKE Workload Identity

It depends on the services used to deploy your application (VM, OKE, Container Instance, Laptop, ...)

[dario-vega]

@adam-sav let us know if you need more information about the authentication methods

[dario-vega]

@adam-sav as discussed above, You have other types of authentication when connecting from an OKE cluster, and here you have more details for 2 of them. If you need more detailed information on how to configure or how to do the connection, we can provide an example

Authentication Method: Instance Principals

Instance Principals is an Oracle Cloud Infrastructure Identity and Access Management (IAM) capability that lets you make service calls from an instance. With instance principals, you don't need to configure user credentials for the services running on your compute instances or rotate the credentials. Instances themselves are now a principal type in IAM. Each compute instance has its own identity and is authenticated by adding certificates to the instance. These certificates are automatically created, assigned to instances, and rotated.

Using instance principals authentication, you can authorize an instance to make API calls on Oracle Cloud Infrastructure services. After you set up the required resources and policies, an application running on an instance can call Oracle Cloud Infrastructure public services, removing the need to configure user credentials or a configuration file. Instance principal authentication can be used from an instance where you don't want to store a configuration file.

Authentication Method: OKE Workload Identity

You can only use workload identities to grant access to OCI resources when using enhanced clusters. See Enhanced Clusters.

In Kubernetes, a workload is an application running on a Kubernetes cluster. A workload can be one application component running inside a single pod, or several application components running inside a set of pods that work together—all the pods in the workload run in the same namespace.

To grant all the pods in workload access to Kubernetes resources, you can specify that every pod in the workload is to use the same Kubernetes service account. You can then grant permission to the Kubernetes cluster role to the service account. The service account binds the pods in the workload to cluster role permissions and grants them access to Kubernetes resources.

https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contenggrantingworkloadaccesstoresources.htm

[dario-vega]

@adam-sav

It's just an idea. Did you try to configure GitHub self-hosted runners?

GitHub Actions self-hosted Runners can be associated with a GitHub repository, an organization, or an enterprise.

There exist examples deploying GitHub Actions self-hosted Runners in OCI. The self-hosted Runner can run on Virtual Machines or Container, enabling you to run container-based workflows.

I didn't test it, but if your workflows run in OCI using GitHub self-hosted runners instead of GitHub-hosted runners, you can probably use instance or resource principal for the authentication.

[dario-vega]

@adam-sav

Additionally, The Oracle NoSQL Database Cloud Simulator simulates the cloud service and lets you write and test applications locally without accessing the Oracle NoSQL Database Cloud Service. After building, debugging, and testing your application with the Oracle NoSQL Database Cloud Simulator, you can move your application to Oracle NoSQL Database Cloud Service for production.

You can use test containers. Define your test dependencies as code, and then simply run your tests, and containers will be created and then deleted. But in this case, you are using local resources instead of OCI resources.

[adam-sav]

Yeah, we have github self-hosted runners working for our system. We also got it working in our pipeline with direct sends to Oracle. It works just fine now. Thank you!

…

On Mon, May 6, 2024 at 9:05 AM Javier Dario VEGA BAEZ < ***@***.***> wrote: @adam-sav <https://github.com/adam-sav> Additionally, The *Oracle NoSQL Database Cloud Simulator* simulates the cloud service and lets you write and test applications locally without accessing the Oracle NoSQL Database Cloud Service. After building, debugging, and testing your application with the Oracle NoSQL Database Cloud Simulator, you can move your application to Oracle NoSQL Database Cloud Service for production. You can use test containers. Define your test dependencies as code, and then simply run your tests, and containers will be created and then deleted. But in this case, you are using local resources instead of OCI resources. — Reply to this email directly, view it on GitHub <#164 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A5FBZYN653G7NQR2M3LIC6TZA6E2TAVCNFSM6AAAAABE32S7GKVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TGMRZHAZTA> . You are receiving this because you were mentioned.Message ID: ***@***.***>