GSOC 2023: Developing Shareable Albums for Niffler Datasets Discussion

[yuvraj-wale]

Hey,

I've been working on my proposal for the shareable albums feature and wanted to briefly discuss my approach with you. I would really appreciate your thoughts and feedback on the implementation plan.

The proposed approach for integrating the shareable album feature into the existing workflow involves developing a new module, which will handle the following tasks:

• Accept the extracted DICOM images folder as input, with its path specified in the config.json file.
• Connect to the metadata store (e.g., MongoDB) to load metadata and filter DICOM images based on the metadata to identify a relevant subset for researchers.
• Copy the identified subset to an output subset folder provided by the user.
• Allow users to directly use a pre-existing custom subset as input, optionally skipping the subset identification process.
• Authenticate the user with their Kheops instance.
• Create an album and add the identified subset to it using the Kheops API, then generate a shareable link for the album.
• Provide additional album handling functions, such as viewing a list of albums, generating shareable links for pre-existing albums, deleting an album, adding a study to an album, and more.

Upon successful integration, the end user will be able to:

• Generate shareable albums from retrieved DICOM images.
• Generate shareable albums from pre-identified query-based subsets.
• Utilize album management and manipulation functions.

The project's goals include:

• Implementing metadata querying and subset identification functionality.
• Adding album creation and subset uploading features using the KHEOPS API.
• Introducing additional album functions, such as viewing a list of created albums, adding subsets to existing albums, deleting albums, etc.
• Conducting comprehensive testing and providing complete documentation for the module.

Thank You.
@anbhimi @pradeeban @haritrivedi @judywawira

[pradeeban]

Implementing metadata querying and subset identification functionality.

Check with the meta-extraction module code to see whether code can be borrowed (ideally, we should refactor that module. But for the purpose of GSoC, we could just copy-paste code from there to begin with). It deals with Mongo and DICOM metadata stored in Mongo.

app-layer module also deals with mongo and the DICOM metadata. But it is in Java.

To ensure this integration to be smooth, I recommend a new Niffler module (similar to cold-extraction, meta-extraction, png-extraction, ...) in Niffler.

Which programming language do you plan to use, to implement this module? Most Niffler modules are developed in Python (except for app-layer, which is developed in Java).

[yuvraj-wale]

I was planning to develop the "shareable albums" module in Python and will definitely check out the meta-extraction code to see if there's any reusable code. I just wanted to confirm, are you suggesting that I create a separate module specifically for the "Implementing metadata querying and subset identification functionality"? If so, that would result in two modules: 1) subset-identification module and 2) shareable-albums module. My initial thought was to implement all the features in one comprehensive module called "NifflerShare." I'd appreciate your thoughts and input on the approach.

[pradeeban]

One module should be sufficient.

You should aim to name it as something without "Niffler" (since that is redundant) and without CAPS (to follow the current naming convention). "albums" or "shareable-albums" as a single module (with all your features) is cool.

[pradeeban]

Overall, I like your vision. Please make sure to submit your project proposal to GSoC on time before the deadline.

[yuvraj-wale]

Thank you for your feedback! I'll make sure to submit my project proposal to GSoC before the deadline.