Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SegRwLock : a segment RwLock Type #424

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

pingzhaozz
Copy link

Add a new RwLock type as SegRwLock which separate the cache line for Read state and Write state in RwLock to solve the cache-line false sharing problem in RwLock in multi threads scenarios.

Since the 'Data' protected by RwLock shares the same cache-line with lock state. When lock state changes the whole cache-line need update which is not necessary for 'Data' in Read lock, and cause unnecessary cache-line update. SegRwLock put Read state in one cache-line and Write state and 'Data' in the other cache-line, to avoid the false sharing between Read state and 'Data'.

Signed-off-by: Ping Zhao  <[email protected]>
Signed-off-by: Lin Yang   <[email protected]>
Signed-off-by: Ping Zhao  <[email protected]>
Signed-off-by: Lin Yang   <[email protected]>
@Amanieu
Copy link
Owner

Amanieu commented Jan 29, 2024

It's not clear to me what use case this is trying to solve. The most common use case for RwLock is to have many readers but few writers. Your implementation does not help with this case since all the readers are still thrashing read_state.

Have you considered a different implementation using a sharded RwLock? The basic idea is to have an array of [CachePadded<RawRwLock>; N] and assign readers to one of the locks depending on its thread ID. This is optimized for read performance: writers will need to acquire a write lock on all the locks in the array before being able to grant write access.

@pingzhaozz
Copy link
Author

pingzhaozz commented Jan 30, 2024

Original RwLock memory layout likes this:

pub struct RwLock<T: ?Sized> {
    raw: RawRwLock,
    data: UnsafeCell<T>,
}

raw and data share the same cache-line. In multi read locks, raw change will cause false sharing with data since the data doesn't change in read, only the read state changes. SegRwLock separate read cache-line and write + data cache-line, which memory layout looks like:

pub struct SegRwLock<T: ?Sized> {
    read: ReadLock,
    padded: <CachePadded>
    write: WriteLock,
    data: UnsafeCell<T>,
}

One matter need to care is that repr(align(n)) doesn't work inside the struct. So manually added the pad.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants