#1356 implements OAuth2 Client Credentials Flow

[willi-mueller]

Description

This PR implements OAuth2 Client Credentials Flow with access token refresh for server-to-server authorization.

Related Issues

• Resolves Implement OAuth2 Client Credentials Flow #1356

Additional Context

Example usage:

import dlt

from rest_api import RESTAPIConfig, rest_api_source
from dlt.sources.helpers.rest_client.auth import OAuth2ClientCredentialsFlow
from dlt.sources.helpers.rest_client.paginators import JSONResponseCursorPaginator

class OAuth2Zoom(OAuth2ClientCredentials):
            """OAuth 2.0 as required by e.g. Zoom Video Communications, Inc."""

            def build_access_token_request(self) -> Dict[str, Any]:
                authentication: str = b64encode(
                    f"{self.client_id}:{self.client_secret}".encode()
                ).decode()
                return {
                    "headers": {
                        "Authorization": f"Basic {authentication}",
                        "Content-Type": "application/x-www-form-urlencoded",
                    },
                    "data": {
                        "grant_type": "account_credentials",
                        **self.access_token_request_data,
                    },
                }

zoom_auth = OAuth2ClientCredentialsHTTPBasic(
            access_token_url=dlt.secrets["sources.zoom.access_token_url"]
            client_id=dlt.secrets["sources.zoom.client_id"]
            client_secret=dlt.secrets["sources.zoom.client_secret"],
            access_token_request_data={
                "account_id": dlt.secrets["sources.zoom.account_id"],
            },
        )

config: RESTAPIConfig = {
    "client": {
        "base_url": "https://api.zoom.us/v2",
        "auth": zoom_auth,
        "paginator": JSONResponseCursorPaginator(
            cursor_path="response.next_page_token",
            cursor_param="next_page_token",
        ),
    },
    "resources": [
        "users",
        {
            "name": "meetings",
            "endpoint": {
                "path": "users/{user_id}/meetings",
                "params": {
                    "user_id": {
                        "type": "resolve",
                        "resource": "users",
                        "field": "id",
                    }
                },
            },
        },
    ],
}

zoom_source = rest_api_source(config)
pipeline = dlt.pipeline(pipeline_name="zoom_test", destination="duckdb", progress="log")
load_info = pipeline.run(zoom_source)

print(load_info)

[netlify]

✅ Deploy Preview for dlt-hub-docs canceled.

Name	Link
🔨 Latest commit	8da3672
🔍 Latest deploy log	https://app.netlify.com/sites/dlt-hub-docs/deploys/66758b73d097170008956fbb

[willi-mueller]

Questions:

1. Am I using @configspeccorrectly? Do I have to implement the __init__ in this way?
2. make lint has many errors unrelated to this PR when invoked on my PC. How can I test if it lints properly?
3. How to implement a test case? I have a running pipeline pasted into this PR but I'm unsure where to implement it in the dlt core. Can you help me?

Thank you!

[willi-mueller]

I'm doing more research right now on which OAuth sources would be covered by this implementation. So far, I have tested only with Zoom. I know that LinkedIn and Hivebrite would not work.

[willi-mueller]

for the record: one user in slack reported that he used this OAuth2 implementation successfully for Hivebrite

https://hivebrite.com/documentation/api/admin#oauth-post

https://dlthub-community.slack.com/archives/C04DQA7JJN6/p1716812907076409?thread_ts=1715003601.550219&cid=C04DQA7JJN6

[burnash]

Hey @willi-mueller, thanks again for the PR. Looks good in general. I'll help you with the tests, let me know if it's ok if I push to your branch.

[willi-mueller]

Hey @willi-mueller, thanks again for the PR. Looks good in general. I'll help you with the tests, let me know if it's ok if I push to your branch.

Yes, please go ahead and refactor and add tests, I'll learn from that <3
The branch is yours.

[burnash]

@willi-mueller the CI is failing tests/sources/helpers/rest_client/test_client.py::TestRESTClient::test_oauth2_client_credentials_flow_wrong_client_id
Could you please take a look?

[burnash]

This looks great!

[willi-mueller]

This looks great!

Wonderful, thank you for the thorough checks and coaching. Could you please merge the PR? I am not authorized by the branch protection rule.