Skip to content
  • Paul E. McKenney's avatar
    rcu: Inform RCU of irq_exit() activity · 8762705a
    Paul E. McKenney authored
    This is a port to TINY_RCU of Peter Zijlstra's commit #ec433f0c
    
    
    
    The rcu_read_unlock_special() function relies on in_irq() to exclude
    scheduler activity from interrupt level.  This fails because exit_irq()
    can invoke the scheduler after clearing the preempt_count() bits that
    in_irq() uses to determine that it is at interrupt level.  This situation
    can result in failures as follows:
    
         $task			IRQ		SoftIRQ
    
         rcu_read_lock()
    
         /* do stuff */
    
         <preempt> |= UNLOCK_BLOCKED
    
         rcu_read_unlock()
           --t->rcu_read_lock_nesting
    
        			irq_enter();
        			/* do stuff, don't use RCU */
        			irq_exit();
        			  sub_preempt_count(IRQ_EXIT_OFFSET);
        			  invoke_softirq()
    
        					ttwu();
        					  spin_lock_irq(&pi->lock)
        					  rcu_read_lock();
        					  /* do stuff */
        					  rcu_read_unlock();
        					    rcu_read_unlock_special()
        					      rcu_report_exp_rnp()
        					        ttwu()
        					          spin_lock_irq(&pi->lock) /* deadlock */
    
           rcu_read_unlock_special(t);
    
    This can be triggered 'easily' because invoke_softirq() immediately does
    a ttwu() of ksoftirqd/# instead of doing the in-place softirq stuff first,
    but even without that the above happens.
    
    Cure this by also excluding softirqs from the rcu_read_unlock_special()
    handler and ensuring the force_irqthreads ksoftirqd/# wakeup is done
    from full softirq context.
    
    It is also necessary to delay the ->rcu_read_lock_nesting decrement until
    after rcu_read_unlock_special().  This delay is handled by the commit
    "Protect __rcu_read_unlock() against scheduler-using irq handlers".
    
    Signed-off-by: default avatarPeter Zijlstra <a.p.zijlstra@chello.nl>
    Signed-off-by: default avatarPaul E. McKenney <paul.mckenney@linaro.org>
    Signed-off-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
    8762705a