Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: endpoint to get all files #21

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

feat: endpoint to get all files #21

wants to merge 1 commit into from

Conversation

psankhe28
Copy link
Collaborator

@psankhe28 psankhe28 commented Sep 11, 2024

Description

Checklist

  • My code follows the contributing guidelines
    of this project, including, in particular, with regard to any style guidelines
  • The title of my PR complies with the
    Conventional Commits specification; in particular, it clearly
    indicates that a change is a breaking change
  • I acknowledge that all my commits will be squashed into a single commit,
    using the PR title as the commit message
  • I have performed a self-review of my own code
  • I have commented my code in hard-to-understand areas
  • I have updated the user-facing documentation to describe any new or
    changed behavior
  • I have added type annotations for all function/class/method interfaces
    or updated existing ones (only for Python, TypeScript, etc.)
  • I have provided appropriate documentation
    (Google-style Python docstrings) for all
    packages/modules/functions/classes/methods or updated existing ones
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature
    works
  • New and existing unit tests pass locally with my changes
  • I have not reduced the existing code coverage

Comments

Summary by Sourcery

Add a new API endpoint to list all files in the MinIO storage, including the implementation of a corresponding controller function to handle the requests and return the file list.

New Features:

  • Introduce a new API endpoint '/list_files' to retrieve a list of all files stored in the MinIO storage.

Enhancements:

  • Add a new function 'list_files' in the controller to handle requests to the '/list_files' endpoint, utilizing the MinIO client to fetch file names.

Signed-off-by: Pratiksha Sankhe <[email protected]>
Copy link
Contributor

sourcery-ai bot commented Sep 11, 2024

Reviewer's Guide by Sourcery

This pull request implements a new endpoint to retrieve a list of all files in the MinIO storage. The changes include updating the API specification and adding a new controller function to handle the endpoint.

File-Level Changes

Change Details Files
Add new API endpoint for listing files
  • Define new GET endpoint '/list_files'
  • Specify response schema as an array of strings
  • Include error responses for 400 and 500 status codes
cloud_storage_handler/api/specs/specs.yaml
Implement controller function for listing files
  • Create new 'list_files' function
  • Use MinIO client to retrieve objects from 'files' bucket
  • Return list of file names as JSON response
  • Handle S3Error and return appropriate error message
cloud_storage_handler/api/elixircloud/csh/controllers.py

Tips
  • Trigger a new Sourcery review by commenting @sourcery-ai review on the pull request.
  • Continue your discussion with Sourcery by replying directly to review comments.
  • You can change your review settings at any time by accessing your dashboard:
    • Enable or disable the Sourcery-generated pull request summary or reviewer's guide;
    • Change the review language;
  • You can always contact us if you have any questions or feedback.

@psankhe28 psankhe28 marked this pull request as draft September 11, 2024 17:45
Copy link
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @psankhe28 - I've reviewed your changes - here's some feedback:

Overall Comments:

  • Consider implementing pagination for the file list to handle large numbers of files more efficiently.
  • The error handling could be more granular. Instead of catching all S3Errors and returning a 500 status code, consider handling different types of S3Errors separately and returning appropriate status codes.
Here's what I looked at during the review
  • 🟡 General issues: 2 issues found
  • 🟢 Security: all looks good
  • 🟢 Testing: all looks good
  • 🟢 Complexity: all looks good
  • 🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment to tell me if it was helpful.


def list_files():
"""Endpoint to list all files in the MinIO bucket."""
try:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Implement more granular error handling.

Consider catching more specific exceptions in addition to S3Error. This could help in providing more accurate error messages to the client.

    try:
        minio_client = get_minio_client()
        objects = minio_client.list_objects("files")
    except minio.error.S3Error as s3_err:
        raise HTTPException(status_code=500, detail=f"S3 error: {str(s3_err)}")
    except minio.error.MinioException as minio_err:
        raise HTTPException(status_code=500, detail=f"MinIO error: {str(minio_err)}")
    except Exception as e:
        raise HTTPException(status_code=500, detail=f"Unexpected error: {str(e)}")

"""Endpoint to list all files in the MinIO bucket."""
try:
minio_client = get_minio_client()
objects = minio_client.list_objects("files")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Avoid hardcoding the bucket name.

Instead of hardcoding "files" as the bucket name, consider making it a configurable parameter. This will make the code more flexible and easier to maintain.

MINIO_BUCKET_NAME = os.getenv('MINIO_BUCKET_NAME', 'files')

objects = minio_client.list_objects(MINIO_BUCKET_NAME)

Copy link

codecov bot commented Sep 11, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (4f7c103) to head (11827ee).

Additional details and impacted files
@@            Coverage Diff            @@
##              main       #21   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            3         3           
  Lines           23        23           
=========================================
  Hits            23        23           
Flag Coverage Δ
test_integration 100.00% <ø> (ø)
test_unit 100.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement "GET /list_files" operation
1 participant