• Linus Torvalds's avatar
    Fix TLB gather virtual address range invalidation corner cases · 8e220cfd
    Linus Torvalds authored
    commit 2b047252d087be7f2ba088b4933cd904f92e6fce upstream.
    Ben Tebulin reported:
     "Since v3.7.2 on two independent machines a very specific Git
      repository fails in 9/10 cases on git-fsck due to an SHA1/memory
      failures.  This only occurs on a very specific repository and can be
      reproduced stably on two independent laptops.  Git mailing list ran
      out of ideas and for me this looks like some very exotic kernel issue"
    and bisected the failure to the backport of commit 53a59fc6 ("mm:
    limit mmu_gather batching to fix soft lockups on !CONFIG_PREEMPT").
    That commit itself is not actually buggy, but what it does is to make it
    much more likely to hit the partial TLB invalidation case, since it
    introduces a new case in tlb_next_batch() that previously only ever
    happened when running out of memory.
    The real bug is that the TLB gather virtual memory range setup is subtly
    buggered.  It was introduced in commit 597e1c35 ("mm/mmu_gather:
    enable tlb flush range in generic mmu_gather"), and the range handling
    was already fixed at least once in commit e6c495a96ce0 ("mm: fix the TLB
    range flushed when __tlb_remove_page() runs out of slots"), but that fix
    was not complete.
    The problem with the TLB gather virtual address range is that it isn't
    set up by the initial tlb_gather_mmu() initialization (which didn't get
    the TLB range information), but it is set up ad-hoc later by the
    functions that actually flush the TLB.  And so any such case that forgot
    to update the TLB range entries would potentially miss TLB invalidates.
    Rather than try to figure out exactly which particular ad-hoc range
    setup was missing (I personally suspect it's the hugetlb case in
    zap_huge_pmd(), which didn't have the same logic as zap_pte_range()
    did), this patch just gets rid of the problem at the source: make the
    TLB range information available to tlb_gather_mmu(), and initialize it
    when initializing all the other tlb gather fields.
    This makes the patch larger, but conceptually much simpler.  And the end
    result is much more understandable; even if you want to play games with
    partial ranges when invalidating the TLB contents in chunks, now the
    range information is always there, and anybody who doesn't want to
    bother with it won't introduce subtle bugs.
    Ben verified that this fixes his problem.
    Reported-bisected-and-tested-by: default avatarBen Tebulin <tebulin@googlemail.com>
    Build-testing-by: default avatarStephen Rothwell <sfr@canb.auug.org.au>
    Build-testing-by: default avatarRichard Weinberger <richard.weinberger@gmail.com>
    Reviewed-by: default avatarMichal Hocko <mhocko@suse.cz>
    Acked-by: default avatarPeter Zijlstra <peterz@infradead.org>
    Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
    Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
tlb.h 5.53 KB