1. 06 Aug, 2012 1 commit
  2. 28 Feb, 2012 1 commit
  3. 24 Feb, 2012 1 commit
    • Ingo Molnar's avatar
      static keys: Introduce 'struct static_key', static_key_true()/false() and... · c5905afb
      Ingo Molnar authored
      static keys: Introduce 'struct static_key', static_key_true()/false() and static_key_slow_[inc|dec]()
      
      So here's a boot tested patch on top of Jason's series that does
      all the cleanups I talked about and turns jump labels into a
      more intuitive to use facility. It should also address the
      various misconceptions and confusions that surround jump labels.
      
      Typical usage scenarios:
      
              #include <linux/static_key.h>
      
              struct static_key key = STATIC_KEY_INIT_TRUE;
      
              if (static_key_false(&key))
                      do unlikely code
              else
                      do likely code
      
      Or:
      
              if (static_key_true(&key))
                      do likely code
              else
                      do unlikely code
      
      The static key is modified via:
      
              static_key_slow_inc(&key);
              ...
              static_key_slow_dec(&key);
      
      The 'slow' prefix makes it abundantly clear that this is an
      expensive operation.
      
      I've updated all in-kernel code to use this everywhere. Note
      that I (intentionally) have not pushed through the rename
      blindly through to the lowest levels: the actual jump-label
      patching arch facility should be named like that, so we want to
      decouple jump labels from the static-key facility a bit.
      
      On non-jump-label enabled architectures static keys default to
      likely()/unlikely() branches.
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      Acked-by: default avatarJason Baron <jbaron@redhat.com>
      Acked-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      Cc: a.p.zijlstra@chello.nl
      Cc: mathieu.desnoyers@efficios.com
      Cc: davem@davemloft.net
      Cc: ddaney.cavm@gmail.com
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Link: http://lkml.kernel.org/r/20120222085809.GA26397@elte.huSigned-off-by: default avatarIngo Molnar <mingo@elte.hu>
      c5905afb
  4. 22 Feb, 2012 2 commits
  5. 27 Dec, 2011 1 commit
  6. 06 Dec, 2011 3 commits
  7. 05 Dec, 2011 1 commit
  8. 07 Nov, 2011 1 commit
  9. 25 Oct, 2011 3 commits
  10. 29 Jun, 2011 1 commit
  11. 25 May, 2011 1 commit
  12. 04 Apr, 2011 1 commit
    • Jason Baron's avatar
      jump label: Introduce static_branch() interface · d430d3d7
      Jason Baron authored
      Introduce:
      
      static __always_inline bool static_branch(struct jump_label_key *key);
      
      instead of the old JUMP_LABEL(key, label) macro.
      
      In this way, jump labels become really easy to use:
      
      Define:
      
              struct jump_label_key jump_key;
      
      Can be used as:
      
              if (static_branch(&jump_key))
                      do unlikely code
      
      enable/disale via:
      
              jump_label_inc(&jump_key);
              jump_label_dec(&jump_key);
      
      that's it!
      
      For the jump labels disabled case, the static_branch() becomes an
      atomic_read(), and jump_label_inc()/dec() are simply atomic_inc(),
      atomic_dec() operations. We show testing results for this change below.
      
      Thanks to H. Peter Anvin for suggesting the 'static_branch()' construct.
      
      Since we now require a 'struct jump_label_key *key', we can store a pointer into
      the jump table addresses. In this way, we can enable/disable jump labels, in
      basically constant time. This change allows us to completely remove the previous
      hashtable scheme. Thanks to Peter Zijlstra for this re-write.
      
      Testing:
      
      I ran a series of 'tbench 20' runs 5 times (with reboots) for 3
      configurations, where tracepoints were disabled.
      
      jump label configured in
      avg: 815.6
      
      jump label *not* configured in (using atomic reads)
      avg: 800.1
      
      jump label *not* configured in (regular reads)
      avg: 803.4
      Signed-off-by: default avatarPeter Zijlstra <a.p.zijlstra@chello.nl>
      LKML-Reference: <20110316212947.GA8792@redhat.com>
      Signed-off-by: default avatarJason Baron <jbaron@redhat.com>
      Suggested-by: default avatarH. Peter Anvin <hpa@linux.intel.com>
      Tested-by: default avatarDavid Daney <ddaney@caviumnetworks.com>
      Acked-by: default avatarRalf Baechle <ralf@linux-mips.org>
      Acked-by: default avatarDavid S. Miller <davem@davemloft.net>
      Acked-by: default avatarMathieu Desnoyers <mathieu.desnoyers@efficios.com>
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      d430d3d7
  13. 29 Oct, 2010 1 commit
  14. 28 Oct, 2010 2 commits
    • Jason Baron's avatar
      jump label: Fix deadlock b/w jump_label_mutex vs. text_mutex · 91bad2f8
      Jason Baron authored
      register_kprobe() downs the 'text_mutex' and then calls
      jump_label_text_reserved(), which downs the 'jump_label_mutex'.
      However, the jump label code takes those mutexes in the reverse
      order.
      
      Fix by requiring the caller of jump_label_text_reserved() to do
      the jump label locking via the newly added: jump_label_lock(),
      jump_label_unlock(). Currently, kprobes is the only user
      of jump_label_text_reserved().
      Reported-by: default avatarIngo Molnar <mingo@elte.hu>
      Acked-by: default avatarMasami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
      Signed-off-by: default avatarJason Baron <jbaron@redhat.com>
      LKML-Reference: <759032c48d5e30c27f0bba003d09bffa8e9f28bb.1285965957.git.jbaron@redhat.com>
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      91bad2f8
    • Jason Baron's avatar
      jump label: Fix module __init section race · b842f8fa
      Jason Baron authored
      Jump label uses is_module_text_address() to ensure that the module
      __init sections are valid before updating them. However, between the
      check for a valid module __init section and the subsequent jump
      label update, the module's __init section could be freed out from under
      us.
      
      We fix this potential race by adding a notifier callback to the
      MODULE_STATE_LIVE state. This notifier is called *after* the __init
      section has been run but before it is going to be freed. In the
      callback, the jump label code zeros the key value for any __init jump
      code within the module, and we add a check for a non-zero key value when
      we update jump labels. In this way we require no additional data
      structures.
      
      Thanks to Mathieu Desnoyers for pointing out this race condition.
      Reported-by: default avatarMathieu Desnoyers <mathieu.desnoyers@efficios.com>
      Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
      Signed-off-by: default avatarJason Baron <jbaron@redhat.com>
      LKML-Reference: <c6f037b7598777668025ceedd9294212fd95fa34.1285965957.git.jbaron@redhat.com>
      
      [ Renamed remove_module_init() to remove_jump_label_module_init()
        as suggested by Masami Hiramatsu. ]
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      b842f8fa
  15. 22 Sep, 2010 2 commits