can we acquire blocking lock while holding a spinlock


(Muhammed Zaki Muhammed Husain Bakshi) #1

I am maintaining a spinlock for safeguarding access to coremap. But now if I try to acquire a lock for pte while holding this spinlock I am violating a KASSERT in wchan_sleep() which requires me to hold only one spinlock.
How would I solve this problem? One solution I can think of is to create a blocking lock for coremap once coremap initialisation is done, then use this blocking lock instead of spinlock from that point onward, but is this the correct way to go?
Please Help!
Thank you.


(Blake Wallace Hurlburt) #2

I think your solution would work. However, there may be a larger problem ahead of you: you cannot hold a spinlock while performing any blocking operation - including acquiring a lock, but also including memcpy and disk I/O. You may need to implement synchronization over PTEs in a non-blocking way.


(Muhammed Zaki Muhammed Husain Bakshi) #3

Thanks. This is the same problem I am having with spinlock for coremap, it fails in VOP_READ/VOP_WRITE and/or lock_acquire(). I am hoping if I replaced my spinlock with a blocking lock it would go away. Apart from the solution I mentioned are there any other solution to this?
Also how can one implement synchronization in a non-blocking way?


(Ashish Tyagi) #4

Hey Blake, could you elaborate on why this is true? And is this only true for OS161 or any system in general?


(Blake Wallace Hurlburt) #5

This is definitely true for OS161. I am not sure about other systems, but I would assume so, at least for any BSD-like system since this is what OS161 is based on. Consider the implications of acquiring a spinlock and performing a blocking operation.

When you acquire a spinlock, interrupts on the current CPU are disabled until you drop the spinlock. To perform a blocking operation like attempting to acquire a lock, it may be necessary to sleep, and to sleep requires a context switch - an event which is handled by causing an interrupt. Therefore it is impossible to perform a blocking operation while holding a spinlock or in any other situation where interrupts are disabled, particularly within interrupt contexts.


(Blake Wallace Hurlburt) #6

Remember that the two possible ways of implementing a synchronization primitive are by blocking (sleeping) until the resource is available, or spinning until the resource is available. Since blocking is probably not possible, you may need to use synchronization devices that involve spinning rather than blocking.


(Ashish Tyagi) #7

Thanks! That’s very informative.


(Richard Salvatore Hanulewicz) #8

Without reading the other responses, this is my immediate response to this topic:


(Geoffrey Challen) #9

OS/161 prevents you from holding a spinlock when going to sleep. There is no reason why this wouldn’t work in practice, but it would cause terrible performance—hence it is prohibited.

The performance problem is caused by the fact that waiters will spin waiting for the lock. Assuming the critical section is short, that’s OK. But any operation that causes me to sleep is going to be very, very, very long. Having another thread holding an active core and spinning waiting for an I/O operation today is a huge waste of resources.

As a final performance concern, some of the low-level instructions required to acquire a spinlock require expensive memory operations due to processor memory caches. So not only is spinning bad because it wastes a core, but the instructions that are being executed can generate a huge amount of load on the memory bus and in the cache coherence protocol. So again: not good.


(Carl Nuessle) #10

Another $.02 –

The problem is not holding both a spinlock and blocklock at the same time. Rather, it is the order in which they are acquired. As discussed above, you can’t (shouldn’t) risk blocking–e.g. trying to acquire a block lock–while already holding a spinlock. So, do it the other way around: first grab all block locks, then all spinlocks. As always with multiple locks, to prevent deadlock, all code paths should acquire locks in the same order.

So – if you’re already holding a spinlock and need block lock(s), you’ll need to drop the spinlock first before reacquiring it. Caution that, in the meantime, the state of the shared data protected by the spinlock may have changed. You’ll need to re-check any assumptions. Another approach that works is to leave breadcrumbs in the data as a signal to other threads during the period you do not hold the spinlock.


(Ashish Tyagi) #11

+1 for the breadcrumbs idea!