[WIP] Split Queue handling and DB writing to address #1216

[akmohtashami]

To address #1216, this PR splits the ES into two separate services:

1- QueueService: This service is solely responsible for maintaining the queue. It assigns tasks to the worker. Once the results are ready it receives them and assigns it to an ES to write to database. Once done, QueueService asks ES to also create next step operations and push them to the queue. In this way, there is no time consuming process being done in the QueueService itself which allows it to always be available. The only exception currently is finding missing operations and invalidating a submission. But these happen less regularly.

2- EvaluationService: This service handles the time consuming tasks needed for maintaining the queue (e.g. writing to the database and computing new operations). The current implementation also allows for multiple ES to be running. I am considering renaming this to QueueWorkerService.

The PR is currently WIP because there are some tasks left:

• Making sure dequeuing an operation fails when it is being written by ES (there is no way to cancel this so we can only fail)
• Some final clean ups

But I wanted to see if there are any comments in advance before finalizing.

[wil93]

I am considering renaming this to QueueWorkerService.

Maybe even just QueueWorker? 🙂