Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: init concurrent rpc for rabbit broker #1605

Open
wants to merge 2 commits into
base: feat/rmq-update
Choose a base branch
from

Conversation

sfran96
Copy link
Contributor

@sfran96 sfran96 commented Jul 18, 2024

Description

Refactoring of the _RPCCallback class to RPCManager with support for an RPC consumer per channel. Took some inspiration from what was discussed in #1318 and made adaptations to what's explained in the RabbitMQ direct reply-to documentation.

Closes #1318

Type of change

Please delete options that are not relevant.

  • Documentation (typos, code examples, or any documentation updates)
  • Bug fix (a non-breaking change that resolves an issue)
  • New feature (a non-breaking change that adds functionality)
  • Breaking change (a fix or feature that would disrupt existing functionality)
  • This change requires a documentation update

Checklist

  • My code adheres to the style guidelines of this project (scripts/lint.sh shows no errors)
  • I have conducted a self-review of my own code
  • I have made the necessary changes to the documentation
  • My changes do not generate any new warnings
  • I have added tests to validate the effectiveness of my fix or the functionality of my new feature
  • Both new and existing unit tests pass successfully on my local environment by running scripts/test-cov.sh
  • I have ensured that static analysis tests are passing by running scripts/static-analysis.sh
  • I have included code examples to illustrate the modifications

@Lancetnik
Copy link
Member

Sorry, but I don't like correlation_id mapping in the services. It is a bad way for huge RMQ cluster (too many messages we should filter). I see this feature implementation via Channel Pool usage and acquiring a new channel from the pool for each request.

@sfran96
Copy link
Contributor Author

sfran96 commented Jul 19, 2024

I understand your point, I've updated it to use the CM you added in your branch, but what I encounter is then issues when using RPC and the consumer and the publisher are in the same machine as the channel pool is shared among the two and the publisher hoards them all before the consumers can get a chance at acquiring one.

I don't know what the best solution would be here, I've read that you usually have a channel for producing and another for consuming, but then does this mean having two channel pools?

I've also tried not using the scope within the RPC callback (therefore releasing the channel back to the pool ASAP) plus a lock per channel, but then what occurs is that the channels do not scale very well with the number of concurrent RPC requests, unless the number of requests is ridiculously higher than the channel pool, so concurrency is higher but not highest.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants