How to handle io_uring_enter returning <to_submit with IOSQE_IO_LINK

[Hawk777]

The documentation for IOSQE_IO_LINK has this to say:

Chains are not supported across submission boundaries. Even if the last SQE in a submission has this flag set, it will still terminate the current chain.

But as an application author, I don’t get to control submission boundaries! Even if I have two SQEs free in the ring, and I fill them both and pass 2 as the to_submit value on io_uring_enter, io_uring_enter itself could decide to return 1 (the man page doesn’t say it won’t, and looking at the kernel source code, io_uring_enter delegates to io_submit_sqes, which could break out part way through its loop if memory allocation fails in io_alloc_req or if io_submit_sqe returns false—AFAICT even IORING_SETUP_SUBMIT_ALL won’t prevent a short submission in the event of allocation failure). So how am I supposed to reliably use IOSQE_IO_LINK if io_uring_enter could once in a while just decide to arbitrarily break my batch in half?

This is not quite the same as this question. That question was about handling running out of SQEs in the ring, which the application can handle by just not doing that (don’t start a chain if you can’t finish it). This question is about putting an entire chain in the ring, and then the kernel deciding to break up the batch.

[cmazakas]

What I do as an application author is, I have this function:

inline void
reserve_sqes( io_uring* ring, unsigned n ) {
  auto r = io_uring_sq_space_left( ring );
  if ( r < n ) {
    io_uring_submit( ring );
  }
}

.
This way, I can ensure that my ring has enough space in the SQ before I go ahead and create the chain.

[Hawk777]

Thanks, but I think you misunderstood. I’m not concerned about space in the ring. I’m concerned about the fact that io_uring_enter doesn’t guarantee to submit all the SQEs. So I could have a 128-entry ring, completely empty, I add 2 SQEs to it, I call io_uring_enter, and it returns 1, because for whatever reason (e.g. memory pressure) the kernel decided to only actually consume the first SQE.

[cmazakas]

Ha ha, I'm sorry, I completely misread your post.

This is a good question. I'm actually not sure but I would assume the implementation isn't hostile. Or at least, I yolo use io_uring_submit(ring) and haven't run into any issues with my SQE chains.

In this case, I might try adding a simple assert() statement to make sure that the number of consumed SQEs matches my expectations but who knows.

[andyg24]

In the unlikely event that a short submission occurs, couldn't you just wait for submitted requests to complete before submitting the rest?

[Hawk777]

I could do that. Or perhaps carry on but add IOSQE_IO_DRAIN to the first SQE in the next submission. Those would hurt performance, but I guess if a short submission is expected to be very unlikely, it’s probably fine. Just seems like a bit of an odd situation.

[isilence]

I could do that. Or perhaps carry on but add IOSQE_IO_DRAIN to the first SQE in the next submission. Those would hurt performance, but I guess if a short submission is expected to be very unlikely, it’s probably fine. Just seems like a bit of an odd situation.

I advise against IOSQE_IO_DRAIN. It's pretty hard to implement zero cost for hot path, so that it's likely in the future just the fact that you used it once with a ring will disable some optimisations for that ring or the overall system. E.g. with a static_key

[isilence]

@Hawk777, also take a look at IORING_SETUP_SUBMIT_ALL. The only way it'd return early is if it's OOM and the kernel can't allocate a request.

[isilence]

Without SQ poll, this integer has acquire/release semantics so it should always be correct to the kernel in that case, but with SQ poll it has a relaxed memory ordering so in that case you will get inconsistent behavior unless you wait long enough after the submit call before placing a new SQ entry.

@arajni3, It's vise versa, acquire/release with SQPOLL and relaxed otherwise. It's also fine submitting new SQEs right after, liburing transparently implements all the necessary synchronisation. Please don't mislead people.

[Hawk777]

@Hawk777, also take a look at IORING_SETUP_SUBMIT_ALL. The only way it'd return early is if it's OOM and the kernel can't allocate a request.

Ah, I guess if the only early return reason really is actual total OOM and expected to be nonrecoverable, that would be OK? The way the man page was written, I read it as “don’t stop on errors” (e.g. invalid opcode, invalid parameter, stuff like that); I was considering that the kernel might still decide “for fairness with other applications/avoiding using too much memory/whatever reason I feel like, I only want to take 100 SQEs right now and leave the other ones for later” and that would not be considered an error (IORING_SETUP_SUBMIT_ALL’s man page documentation doesn’t say it will always submit all the SQEs, it only says it won’t stop for errors, at least in the version I have).

I advise against IOSQE_IO_DRAIN. It's pretty hard to implement zero cost for hot path, so that it's likely in the future just the fact that you used it once with a ring will disable some optimisations for that ring or the overall system. E.g. with a static_key

Good to know! That sounds like something that might be worth documenting in the man page. I suppose the alternative is to just implement exactly the same thing in userspace, i.e. just don’t submit any more ops for the affected chain until every preceding op has CQEd.

[isilence]

@Hawk777, also take a look at IORING_SETUP_SUBMIT_ALL. The only way it'd return early is if it's OOM and the kernel can't allocate a request.

Ah, I guess if the only early return reason really is actual total OOM and expected to be nonrecoverable, that would be OK?

Right, and if happens it'll likely OOM-kill your process anyway at how it is now. If there is enough interest, it can be improved to "we'll deliver all CQEs even if an early request allocation fail occurs as long as there are enough free CQ entries".

There is actually a second reason it will have to stop, that is if the userspace screws up the SQ (the SQ index array in particular), but the userspace is in total control of it and liburing hides all details.

The way the man page was written, I read it as “don’t stop on errors” (e.g. invalid opcode, invalid parameter, stuff like that); I was considering that the kernel might still decide “for fairness with other applications/avoiding using too much memory/whatever reason I feel like, I only want to take 100 SQEs right now and leave the other ones for later” and that would not be considered an error (IORING_SETUP_SUBMIT_ALL’s man page documentation doesn’t say it will always submit all the SQEs, it only says it won’t stop for errors, at least in the version I have).

Let's say it's "don’t stop even on errors". It always try to submit all SQEs, the difference is what happens when something fails.

I advise against IOSQE_IO_DRAIN. It's pretty hard to implement zero cost for hot path, so that it's likely in the future just the fact that you used it once with a ring will disable some optimisations for that ring or the overall system. E.g. with a static_key

Good to know! That sounds like something that might be worth documenting in the man page.

It'd be a good addition indeed.

[Hawk777]

Well, if the only reason the kernel would stop taking SQs early is either catastrophic (OOM) or user error, as long as that is guaranteed, I suppose it’s not really a big deal then. It’s just, without any such documented guarantee, I didn’t know whether I could rely on that.