Update Image Extraction Microservices Description

[darshad-github]

SMI Readme - IsIdentifiable and ExtractImages are not displayed in the visualisation.

[rkm]

Standard anonymisation extraction pipeline

The current extraction services look like:

• ExtractImages: Reads UIDs from a CSV file. Sends
  • ExtractionRequestMessage to CohortExtractor and
  • ExtractionRequestInfoMessage to CohortPackager
• CohortExtractor: Matches DICOM UIDs in the ExtractionRequestMessage to the physical file location(s). Sends
  • ExtractFileMessage to CTPAnonymiser and
  • ExtractFileCollectionInfoMessage to CohortPackager
• CTPAnonymiser: Microservice wrapper for CTP. Anonymises images specified in the input ExtractFileMessage and copies the output to specified directory. Sends
  • ExtractedFileStatusMessage to IsIdentifiable when anonymisation was successful, and to CohortPackager when the anonymisation fails
• IsIdentifiable: Checks for PII in the file specified by ExtractedFileStatusMessage and sends
  • ExtractedFileVerificationMessage to CohortPackager
• CohortPackager: Receives ExtractionRequestInfoMessages, ExtractFileCollectionInfoMessages, ExtractedFileStatusMessage, and ExtractedFileVerificationMessages, and stores them in MongoDB. Once it detects all messages for an extraction have been received, it generates a set of extraction reports and marks the job as completed.

Identifiable extraction pipeline

We also have the ability to run an "identifiable" (non anonymised) extraction, which looks similar to above except:

• FileCopier is used in place of of CTPAnonymiser. This sends
  • ExtractedFileStatusMessage to CohortPackager
• IsIdentifiable is not part of the pipeline

Both extractions use RabbitMQ routing keys to direct messages to the appropriate services.

ToDo

Steps to fix this:

• Update/replace the diagram to include the missing services as above
  • I can't find the original version of the current diagram, so let's create a new one using Draw.io.
• Update the table with the information above