1. 29 Apr, 2013 1 commit
  2. 28 Feb, 2013 1 commit
    • Sasha Levin's avatar
      hlist: drop the node parameter from iterators · b67bfe0d
      Sasha Levin authored
      I'm not sure why, but the hlist for each entry iterators were conceived
      
              list_for_each_entry(pos, head, member)
      
      The hlist ones were greedy and wanted an extra parameter:
      
              hlist_for_each_entry(tpos, pos, head, member)
      
      Why did they need an extra pos parameter? I'm not quite sure. Not only
      they don't really need it, it also prevents the iterator from looking
      exactly like the list iterator, which is unfortunate.
      
      Besides the semantic patch, there was some manual work required:
      
       - Fix up the actual hlist iterators in linux/list.h
       - Fix up the declaration of other iterators based on the hlist ones.
       - A very small amount of places were using the 'node' parameter, this
       was modified to use 'obj->member' instead.
       - Coccinelle didn't handle the hlist_for_each_entry_safe iterator
       properly, so those had to be fixed up manually.
      
      The semantic patch which is mostly the work of Peter Senna Tschudin is here:
      
      @@
      iterator name hlist_for_each_entry, hlist_for_each_entry_continue, hlist_for_each_entry_from, hlist_for_each_entry_rcu, hlist_for_each_entry_rcu_bh, hlist_for_each_entry_continue_rcu_bh, for_each_busy_worker, ax25_uid_for_each, ax25_for_each, inet_bind_bucket_for_each, sctp_for_each_hentry, sk_for_each, sk_for_each_rcu, sk_for_each_from, sk_for_each_safe, sk_for_each_bound, hlist_for_each_entry_safe, hlist_for_each_entry_continue_rcu, nr_neigh_for_each, nr_neigh_for_each_safe, nr_node_for_each, nr_node_for_each_safe, for_each_gfn_indirect_valid_sp, for_each_gfn_sp, for_each_host;
      
      type T;
      expression a,c,d,e;
      identifier b;
      statement S;
      @@
      
      -T b;
          <+... when != b
      (
      hlist_for_each_entry(a,
      - b,
      c, d) S
      |
      hlist_for_each_entry_continue(a,
      - b,
      c) S
      |
      hlist_for_each_entry_from(a,
      - b,
      c) S
      |
      hlist_for_each_entry_rcu(a,
      - b,
      c, d) S
      |
      hlist_for_each_entry_rcu_bh(a,
      - b,
      c, d) S
      |
      hlist_for_each_entry_continue_rcu_bh(a,
      - b,
      c) S
      |
      for_each_busy_worker(a, c,
      - b,
      d) S
      |
      ax25_uid_for_each(a,
      - b,
      c) S
      |
      ax25_for_each(a,
      - b,
      c) S
      |
      inet_bind_bucket_for_each(a,
      - b,
      c) S
      |
      sctp_for_each_hentry(a,
      - b,
      c) S
      |
      sk_for_each(a,
      - b,
      c) S
      |
      sk_for_each_rcu(a,
      - b,
      c) S
      |
      sk_for_each_from
      -(a, b)
      +(a)
      S
      + sk_for_each_from(a) S
      |
      sk_for_each_safe(a,
      - b,
      c, d) S
      |
      sk_for_each_bound(a,
      - b,
      c) S
      |
      hlist_for_each_entry_safe(a,
      - b,
      c, d, e) S
      |
      hlist_for_each_entry_continue_rcu(a,
      - b,
      c) S
      |
      nr_neigh_for_each(a,
      - b,
      c) S
      |
      nr_neigh_for_each_safe(a,
      - b,
      c, d) S
      |
      nr_node_for_each(a,
      - b,
      c) S
      |
      nr_node_for_each_safe(a,
      - b,
      c, d) S
      |
      - for_each_gfn_sp(a, c, d, b) S
      + for_each_gfn_sp(a, c, d) S
      |
      - for_each_gfn_indirect_valid_sp(a, c, d, b) S
      + for_each_gfn_indirect_valid_sp(a, c, d) S
      |
      for_each_host(a,
      - b,
      c) S
      |
      for_each_host_safe(a,
      - b,
      c, d) S
      |
      for_each_mesh_entry(a,
      - b,
      c, d) S
      )
          ...+>
      
      [akpm@linux-foundation.org: drop bogus change from net/ipv4/raw.c]
      [akpm@linux-foundation.org: drop bogus hunk from net/ipv6/raw.c]
      [akpm@linux-foundation.org: checkpatch fixes]
      [akpm@linux-foundation.org: fix warnings]
      [akpm@linux-foudnation.org: redo intrusive kvm changes]
      Tested-by: default avatarPeter Senna Tschudin <peter.senna@gmail.com>
      Acked-by: default avatarPaul E. McKenney <paulmck@linux.vnet.ibm.com>
      Signed-off-by: default avatarSasha Levin <sasha.levin@oracle.com>
      Cc: Wu Fengguang <fengguang.wu@intel.com>
      Cc: Marcelo Tosatti <mtosatti@redhat.com>
      Cc: Gleb Natapov <gleb@redhat.com>
      Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      b67bfe0d
  3. 20 Dec, 2012 17 commits
    • David Howells's avatar
      FS-Cache: Clear remaining page count on retrieval cancellation · 91c7fbbf
      David Howells authored
      Provide fscache_cancel_op() with a pointer to a function it should invoke under
      lock if it cancels an operation.
      
      Use this to clear the remaining page count upon cancellation of a pending
      retrieval operation so that fscache_release_retrieval_op() doesn't get an
      assertion failure (see below).  This can happen when a signal occurs, say from
      CTRL-C being pressed during data retrieval.
      
      FS-Cache: Assertion failed
      3 == 0 is false
      ------------[ cut here ]------------
      kernel BUG at fs/fscache/page.c:237!
      invalid opcode: 0000 [#641] SMP
      Modules linked in: cachefiles(F) nfsv4(F) nfsv3(F) nfsv2(F) nfs(F) fscache(F) auth_rpcgss(F) nfs_acl(F) lockd(F) sunrpc(F)
      CPU 0
      Pid: 6075, comm: slurp-q Tainted: GF     D      3.7.0-rc8-fsdevel+ #411                  /DG965RY
      RIP: 0010:[<ffffffffa007f328>]  [<ffffffffa007f328>] fscache_release_retrieval_op+0x75/0xff [fscache]
      RSP: 0000:ffff88001c6d7988  EFLAGS: 00010296
      RAX: 000000000000000f RBX: ffff880014cdfe00 RCX: ffffffff6c102000
      RDX: ffffffff8102d1ad RSI: ffffffff6c102000 RDI: ffffffff8102d1d6
      RBP: ffff88001c6d7998 R08: 0000000000000002 R09: 0000000000000000
      R10: 0000000000000000 R11: 0000000000000000 R12: 00000000fffffe00
      R13: ffff88001c6d7ab4 R14: ffff88001a8638a0 R15: ffff88001552b190
      FS:  00007f877aaf0700(0000) GS:ffff88003bc00000(0000) knlGS:0000000000000000
      CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
      CR2: 00007fff11378fd2 CR3: 000000001c6c6000 CR4: 00000000000007f0
      DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
      DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
      Process slurp-q (pid: 6075, threadinfo ffff88001c6d6000, task ffff88001c6c4080)
      Stack:
       ffffffffa007ec07 ffff880014cdfe00 ffff88001c6d79c8 ffffffffa007db4d
       ffffffffa007ec07 ffff880014cdfe00 00000000fffffe00 ffff88001c6d7ab4
       ffff88001c6d7a38 ffffffffa008116d 0000000000000000 ffff88001c6c4080
      Call Trace:
       [<ffffffffa007ec07>] ? fscache_cancel_op+0x194/0x1cf [fscache]
       [<ffffffffa007db4d>] fscache_put_operation+0x135/0x2ed [fscache]
       [<ffffffffa007ec07>] ? fscache_cancel_op+0x194/0x1cf [fscache]
       [<ffffffffa008116d>] __fscache_read_or_alloc_pages+0x413/0x4bc [fscache]
       [<ffffffff810ac8ae>] ? __alloc_pages_nodemask+0x195/0x75c
       [<ffffffffa00aab0f>] __nfs_readpages_from_fscache+0x86/0x13d [nfs]
       [<ffffffffa00a5fe0>] nfs_readpages+0x186/0x1bd [nfs]
       [<ffffffff810d23c8>] ? alloc_pages_current+0xc7/0xe4
       [<ffffffff810a68b5>] ? __page_cache_alloc+0x84/0x91
       [<ffffffff810af912>] ? __do_page_cache_readahead+0xa6/0x2e0
       [<ffffffff810afaa3>] __do_page_cache_readahead+0x237/0x2e0
       [<ffffffff810af912>] ? __do_page_cache_readahead+0xa6/0x2e0
       [<ffffffff810afe3e>] ra_submit+0x1c/0x20
       [<ffffffff810b019b>] ondemand_readahead+0x359/0x382
       [<ffffffff810b0279>] page_cache_sync_readahead+0x38/0x3a
       [<ffffffff810a77b5>] generic_file_aio_read+0x26b/0x637
       [<ffffffffa00f1852>] ? nfs_mark_delegation_referenced+0xb/0xb [nfsv4]
       [<ffffffffa009cc85>] nfs_file_read+0xaa/0xcf [nfs]
       [<ffffffff810db5b3>] do_sync_read+0x91/0xd1
       [<ffffffff810dbb8b>] vfs_read+0x9b/0x144
       [<ffffffff810dbc78>] sys_read+0x44/0x75
       [<ffffffff81422892>] system_call_fastpath+0x16/0x1b
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      91c7fbbf
    • David Howells's avatar
      FS-Cache: Mark cancellation of in-progress operation · 1f372dff
      David Howells authored
      Mark as cancelled an operation that is in progress rather than pending at the
      time it is cancelled, and call fscache_complete_op() to cancel an operation so
      that blocked ops can be started.
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      1f372dff
    • David Howells's avatar
      FS-Cache: One of the write operation paths doesn't set the object state · 7ef001e9
      David Howells authored
      In fscache_write_op(), if the object is determined to have become inactive or
      to have lost its cookie, we don't move the operation state from in-progress,
      and so an assertion in fscache_put_operation() fails with an assertion (see
      below).
      
      Instrumenting fscache_op_work_func() indicates that it called
      fscache_write_op() before calling fscache_put_operation() - where the assertion
      failed.  The assertion at line 433 indicates that the operation state is
      IN_PROGRESS rather than being COMPLETE or CANCELLED.
      
      Instrumenting fscache_write_op() showed that it was being called on an object
      that had had its cookie removed and that this was due to relinquishment of the
      cookie by the netfs.  At this point fscache no longer has access to the pages
      of netfs data that were requested to be written, and so simply cancelling the
      operation is the thing to do.
      
      FS-Cache: Assertion failed
      3 == 5 is false
      ------------[ cut here ]------------
      kernel BUG at fs/fscache/operation.c:433!
      invalid opcode: 0000 [#1] SMP
      Modules linked in: cachefiles(F) nfsv4(F) nfsv3(F) nfsv2(F) nfs(F) fscache(F) auth_rpcgss(F) nfs_acl(F) lockd(F) sunrpc(F)
      CPU 0
      Pid: 1035, comm: kworker/u:3 Tainted: GF            3.7.0-rc8-fsdevel+ #411                  /DG965RY
      RIP: 0010:[<ffffffffa007db22>]  [<ffffffffa007db22>] fscache_put_operation+0x11a/0x2ed [fscache]
      RSP: 0018:ffff88003e32bcf8  EFLAGS: 00010296
      RAX: 000000000000000f RBX: ffff88001818eb78 RCX: ffffffff6c102000
      RDX: ffffffff8102d1ad RSI: ffffffff6c102000 RDI: ffffffff8102d1d6
      RBP: ffff88003e32bd18 R08: 0000000000000002 R09: 0000000000000000
      R10: 0000000000000000 R11: 0000000000000000 R12: ffffffffa00811da
      R13: 0000000000000001 R14: 0000000100625d26 R15: 0000000000000000
      FS:  0000000000000000(0000) GS:ffff88003bc00000(0000) knlGS:0000000000000000
      CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
      CR2: 00007fff7dd31c68 CR3: 000000003d730000 CR4: 00000000000007f0
      DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
      DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
      Process kworker/u:3 (pid: 1035, threadinfo ffff88003e32a000, task ffff88003bb38080)
      Stack:
       ffffffff8102d1ad ffff88001818eb78 ffffffffa00811da 0000000000000001
       ffff88003e32bd48 ffffffffa007f0ad ffff88001818eb78 ffffffff819583c0
       ffff88003df24e00 ffff88003882c3e0 ffff88003e32bde8 ffffffff81042de0
      Call Trace:
       [<ffffffff8102d1ad>] ? vprintk_emit+0x3c6/0x41a
       [<ffffffffa00811da>] ? __fscache_read_or_alloc_pages+0x4bc/0x4bc [fscache]
       [<ffffffffa007f0ad>] fscache_op_work_func+0xec/0x123 [fscache]
       [<ffffffff81042de0>] process_one_work+0x21c/0x3b0
       [<ffffffff81042d82>] ? process_one_work+0x1be/0x3b0
       [<ffffffffa007efc1>] ? fscache_operation_gc+0x23e/0x23e [fscache]
       [<ffffffff8104332e>] worker_thread+0x202/0x2df
       [<ffffffff8104312c>] ? rescuer_thread+0x18e/0x18e
       [<ffffffff81047c1c>] kthread+0xd0/0xd8
       [<ffffffff81421bfa>] ? _raw_spin_unlock_irq+0x29/0x3e
       [<ffffffff81047b4c>] ? __init_kthread_worker+0x55/0x55
       [<ffffffff814227ec>] ret_from_fork+0x7c/0xb0
       [<ffffffff81047b4c>] ? __init_kthread_worker+0x55/0x55
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      7ef001e9
    • David Howells's avatar
      FS-Cache: Fix signal handling during waits · 9c04caa8
      David Howells authored
      wait_on_bit() with TASK_INTERRUPTIBLE returns 1 rather than a negative error
      code, so change what we check for.  This means that the signal handling in
      fscache_wait_for_retrieval_activation()  should now work properly.
      
      Without this, the following bug can be seen if CTRL-C is pressed during
      fscache read operation:
      
      FS-Cache: Assertion failed
      2 == 3 is false
      ------------[ cut here ]------------
      kernel BUG at fs/fscache/page.c:347!
      invalid opcode: 0000 [#1] SMP
      Modules linked in: cachefiles(F) nfsv4(F) nfsv3(F) nfsv2(F) nfs(F) fscache(F) auth_rpcgss(F) nfs_acl(F) lockd(F) sunrpc(F)
      CPU 1
      Pid: 15006, comm: slurp-q Tainted: GF            3.7.0-rc8-fsdevel+ #411                  /DG965RY
      RIP: 0010:[<ffffffffa007fcb4>]  [<ffffffffa007fcb4>] fscache_wait_for_retrieval_activation+0x167/0x177 [fscache]
      RSP: 0018:ffff88002a4c39a8  EFLAGS: 00010292
      RAX: 000000000000001a RBX: ffff88002d3dc158 RCX: 0000000000008685
      RDX: ffffffff8102ccd6 RSI: 0000000000000001 RDI: ffffffff8102d1d6
      RBP: ffff88002a4c39c8 R08: 0000000000000002 R09: 0000000000000000
      R10: ffffffff8163afa0 R11: ffff88003bd11900 R12: ffffffffa00868c8
      R13: ffff880028306458 R14: ffff88002d3dc1b0 R15: ffff88001372e538
      FS:  00007f17426a0700(0000) GS:ffff88003bd00000(0000) knlGS:0000000000000000
      CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
      CR2: 00007f1742494a44 CR3: 0000000031bd7000 CR4: 00000000000007e0
      DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
      DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
      Process slurp-q (pid: 15006, threadinfo ffff88002a4c2000, task ffff880023de3040)
      Stack:
       ffff88002d3dc158 ffff88001372e538 ffff88002a4c3ab4 ffff8800283064e0
       ffff88002a4c3a38 ffffffffa0080f6d 0000000000000000 ffff880023de3040
       ffff88002a4c3ac8 ffffffff810ac8ae ffff880028306458 ffff88002a4c3bc8
      Call Trace:
       [<ffffffffa0080f6d>] __fscache_read_or_alloc_pages+0x24f/0x4bc [fscache]
       [<ffffffff810ac8ae>] ? __alloc_pages_nodemask+0x195/0x75c
       [<ffffffffa00aab0f>] __nfs_readpages_from_fscache+0x86/0x13d [nfs]
       [<ffffffffa00a5fe0>] nfs_readpages+0x186/0x1bd [nfs]
       [<ffffffff810d23c8>] ? alloc_pages_current+0xc7/0xe4
       [<ffffffff810a68b5>] ? __page_cache_alloc+0x84/0x91
       [<ffffffff810af912>] ? __do_page_cache_readahead+0xa6/0x2e0
       [<ffffffff810afaa3>] __do_page_cache_readahead+0x237/0x2e0
       [<ffffffff810af912>] ? __do_page_cache_readahead+0xa6/0x2e0
       [<ffffffff810afe3e>] ra_submit+0x1c/0x20
       [<ffffffff810b019b>] ondemand_readahead+0x359/0x382
       [<ffffffff810b0279>] page_cache_sync_readahead+0x38/0x3a
       [<ffffffff810a77b5>] generic_file_aio_read+0x26b/0x637
       [<ffffffffa00f1852>] ? nfs_mark_delegation_referenced+0xb/0xb [nfsv4]
       [<ffffffffa009cc85>] nfs_file_read+0xaa/0xcf [nfs]
       [<ffffffff810db5b3>] do_sync_read+0x91/0xd1
       [<ffffffff810dbb8b>] vfs_read+0x9b/0x144
       [<ffffffff810dbc78>] sys_read+0x44/0x75
       [<ffffffff81422892>] system_call_fastpath+0x16/0x1b
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      9c04caa8
    • David Howells's avatar
      FS-Cache: Add transition to handle invalidate immediately after lookup · 96969521
      David Howells authored
      Add a missing transition to the FS-Cache object state machine to handle an
      invalidation event occuring between the back end completing the object lookup
      by calling fscache_obtained_object() (which moves to state OBJECT_AVAILABLE)
      and the backend returning to fscache_lookup_object() and thence to
      fscache_object_state_machine() which then does a goto lookup_transit to handle
      the transition - but lookup_transit doesn't handle EV_INVALIDATE.
      
      Without this, the following BUG can be logged:
      
      	FS-Cache: Unsupported event 2 [5/f7] in state OBJECT_AVAILABLE
      	------------[ cut here ]------------
      	kernel BUG at fs/fscache/object.c:357!
      
      Where event 2 is EV_INVALIDATE.
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      96969521
    • David Howells's avatar
      NFS: nfs_migrate_page() does not wait for FS-Cache to finish with a page · 8c209ce7
      David Howells authored
      nfs_migrate_page() does not wait for FS-Cache to finish with a page, probably
      leading to the following bad-page-state:
      
       BUG: Bad page state in process python-bin  pfn:17d39b
       page:ffffea00053649e8 flags:004000000000100c count:0 mapcount:0 mapping:(null)
      index:38686 (Tainted: G    B      ---------------- )
       Pid: 31053, comm: python-bin Tainted: G    B      ----------------
      2.6.32-71.24.1.el6.x86_64 #1
       Call Trace:
       [<ffffffff8111bfe7>] bad_page+0x107/0x160
       [<ffffffff8111ee69>] free_hot_cold_page+0x1c9/0x220
       [<ffffffff8111ef19>] __pagevec_free+0x59/0xb0
       [<ffffffff8104b988>] ? flush_tlb_others_ipi+0x128/0x130
       [<ffffffff8112230c>] release_pages+0x21c/0x250
       [<ffffffff8115b92a>] ? remove_migration_pte+0x28a/0x2b0
       [<ffffffff8115f3f8>] ? mem_cgroup_get_reclaim_stat_from_page+0x18/0x70
       [<ffffffff81122687>] ____pagevec_lru_add+0x167/0x180
       [<ffffffff811226f8>] __lru_cache_add+0x58/0x70
       [<ffffffff81122731>] lru_cache_add_lru+0x21/0x40
       [<ffffffff81123f49>] putback_lru_page+0x69/0x100
       [<ffffffff8115c0bd>] migrate_pages+0x13d/0x5d0
       [<ffffffff81122687>] ? ____pagevec_lru_add+0x167/0x180
       [<ffffffff81152ab0>] ? compaction_alloc+0x0/0x370
       [<ffffffff8115255c>] compact_zone+0x4cc/0x600
       [<ffffffff8111cfac>] ? get_page_from_freelist+0x15c/0x820
       [<ffffffff810672f4>] ? check_preempt_wakeup+0x1c4/0x3c0
       [<ffffffff8115290e>] compact_zone_order+0x7e/0xb0
       [<ffffffff81152a49>] try_to_compact_pages+0x109/0x170
       [<ffffffff8111e94d>] __alloc_pages_nodemask+0x5ed/0x850
       [<ffffffff814c9136>] ? thread_return+0x4e/0x778
       [<ffffffff81150d43>] alloc_pages_vma+0x93/0x150
       [<ffffffff81167ea5>] do_huge_pmd_anonymous_page+0x135/0x340
       [<ffffffff814cb6f6>] ? rwsem_down_read_failed+0x26/0x30
       [<ffffffff81136755>] handle_mm_fault+0x245/0x2b0
       [<ffffffff814ce383>] do_page_fault+0x123/0x3a0
       [<ffffffff814cbdf5>] page_fault+0x25/0x30
      
      nfs_migrate_page() calls nfs_fscache_release_page() which doesn't actually wait
      - even if __GFP_WAIT is set.  The reason that doesn't wait is that
      fscache_maybe_release_page() might deadlock the allocator as the work threads
      writing to the cache may all end up sleeping on memory allocation.
      
      However, I wonder if that is actually a problem.  There are a number of things
      I can do to deal with this:
      
       (1) Make nfs_migrate_page() wait.
      
       (2) Make fscache_maybe_release_page() honour the __GFP_WAIT flag.
      
       (3) Set a timeout around the wait.
      
       (4) Make nfs_migrate_page() return an error if the page is still busy.
      
      For the moment, I'll select (2) and (4).
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      Acked-by: default avatarJeff Layton <jlayton@redhat.com>
      8c209ce7
    • David Howells's avatar
      FS-Cache: Exclusive op submission can BUG if there's been an I/O error · 8d76349d
      David Howells authored
      The function to submit an exclusive op (fscache_submit_exclusive_op()) can BUG
      if there's been an I/O error because it may see the parent cache object in an
      unexpected state.  It should only BUG if there hasn't been an I/O error.
      
      In this case the problem was produced by remounting the cache partition to be
      R/O.  The EROFS state was detected and the cache was aborted, but not
      everything handled the aborting correctly.
      
      SysRq : Emergency Remount R/O
      EXT4-fs (sda6): re-mounted. Opts: (null)
      Emergency Remount complete
      CacheFiles: I/O Error: Failed to update xattr with error -30
      FS-Cache: Cache cachefiles stopped due to I/O error
      ------------[ cut here ]------------
      kernel BUG at fs/fscache/operation.c:128!
      invalid opcode: 0000 [#1] SMP 
      CPU 0 
      Modules linked in: cachefiles nfs fscache auth_rpcgss nfs_acl lockd sunrpc
      
      Pid: 6612, comm: kworker/u:2 Not tainted 3.1.0-rc8-fsdevel+ #1093                  /DG965RY
      RIP: 0010:[<ffffffffa00739c0>]  [<ffffffffa00739c0>] fscache_submit_exclusive_op+0x2ad/0x2c2 [fscache]
      RSP: 0018:ffff880000853d40  EFLAGS: 00010206
      RAX: ffff880038ac72a8 RBX: ffff8800181f2260 RCX: ffffffff81f2b2b0
      RDX: 0000000000000001 RSI: ffffffff8179a478 RDI: ffff8800181f2280
      RBP: ffff880000853d60 R08: 0000000000000002 R09: 0000000000000000
      R10: 0000000000000001 R11: 0000000000000001 R12: ffff880038ac7268
      R13: ffff8800181f2280 R14: ffff88003a359190 R15: 000000010122b162
      FS:  0000000000000000(0000) GS:ffff88003bc00000(0000) knlGS:0000000000000000
      CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
      CR2: 00000034cc4a77f0 CR3: 0000000010e96000 CR4: 00000000000006f0
      DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
      DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
      Process kworker/u:2 (pid: 6612, threadinfo ffff880000852000, task ffff880014c3c040)
      Stack:
       ffff8800181f2260 ffff8800181f2310 ffff880038ac7268 ffff8800181f2260
       ffff880000853dc0 ffffffffa0072375 ffff880037ecfe00 ffff88003a359198
       ffff880000853dc0 0000000000000246 0000000000000000 ffff88000a91d308
      Call Trace:
       [<ffffffffa0072375>] fscache_object_work_func+0x792/0xe65 [fscache]
       [<ffffffff81047e44>] process_one_work+0x1eb/0x37f
       [<ffffffff81047de6>] ? process_one_work+0x18d/0x37f
       [<ffffffffa0071be3>] ? fscache_enqueue_dependents+0xd8/0xd8 [fscache]
       [<ffffffff810482e4>] worker_thread+0x15a/0x21a
       [<ffffffff8104818a>] ? rescuer_thread+0x188/0x188
       [<ffffffff8104bf96>] kthread+0x7f/0x87
       [<ffffffff813ad6f4>] kernel_thread_helper+0x4/0x10
       [<ffffffff81026b98>] ? finish_task_switch+0x45/0xc0
       [<ffffffff813abd1d>] ? retint_restore_args+0xe/0xe
       [<ffffffff8104bf17>] ? __init_kthread_worker+0x53/0x53
       [<ffffffff813ad6f0>] ? gs_change+0xb/0xb
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      8d76349d
    • David Howells's avatar
      FS-Cache: Limit the number of I/O error reports for a cache · 75bc4113
      David Howells authored
      Limit the number of I/O error reports for a cache to 1 to prevent massive
      amounts of noise.  After the first I/O error the cache is taken off line
      automatically, so must be restarted to resume caching.
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      75bc4113
    • David Howells's avatar
      FS-Cache: Don't mask off the object event mask when printing it · c2d35bfe
      David Howells authored
      Don't mask off the object event mask when printing it.  That way it can be seen
      if threre are bits set that shouldn't be.
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      c2d35bfe
    • David Howells's avatar
      FS-Cache: Initialise the object event mask with the calculated mask · 03acc4be
      David Howells authored
      Initialise the object event mask with the calculated mask rather than unmasking
      undefined events also.
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      03acc4be
    • David Howells's avatar
      CacheFiles: Add missing retrieval completions · b4cf1e08
      David Howells authored
      CacheFiles is missing some calls to fscache_retrieval_complete() in the error
      handling/collision paths of its reader functions.
      
      This can be seen by the following assertion tripping in fscache_put_operation()
      whereby the operation being destroyed is still in the in-progress state and has
      not been cancelled or completed:
      
      FS-Cache: Assertion failed
      3 == 5 is false
      ------------[ cut here ]------------
      kernel BUG at fs/fscache/operation.c:408!
      invalid opcode: 0000 [#1] SMP
      CPU 2
      Modules linked in: xfs ioatdma dca loop joydev evdev
      psmouse dcdbas pcspkr serio_raw i5000_edac edac_core i5k_amb shpchp
      pci_hotplug sg sr_mod]
      
      Pid: 8062, comm: httpd Not tainted 3.1.0-rc8 #1 Dell Inc. PowerEdge 1950/0DT097
      RIP: 0010:[<ffffffff81197b24>]  [<ffffffff81197b24>] fscache_put_operation+0x304/0x330
      RSP: 0018:ffff880062f739d8  EFLAGS: 00010296
      RAX: 0000000000000025 RBX: ffff8800c5122e84 RCX: ffffffff81ddf040
      RDX: 00000000ffffffff RSI: 0000000000000082 RDI: ffffffff81ddef30
      RBP: ffff880062f739f8 R08: 0000000000000005 R09: 0000000000000000
      R10: 0000000000000000 R11: 0000000000000003 R12: ffff8800c5122e40
      R13: ffff880037a2cd20 R14: ffff880087c7a058 R15: ffff880087c7a000
      FS:  00007f63dcf636e0(0000) GS:ffff88022fc80000(0000) knlGS:0000000000000000
      CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
      CR2: 00007f0c0a91f000 CR3: 0000000062ec2000 CR4: 00000000000006e0
      DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
      DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
      Process httpd (pid: 8062, threadinfo ffff880062f72000, task ffff880087e58000)
      Stack:
       ffff880062f73bf8 0000000000000000 ffff880062f73bf8 ffff880037a2cd20
       ffff880062f73a68 ffffffff8119aa7e ffff88006540e000 ffff880062f73ad4
       ffff88008e9a4308 ffff880037a2cd20 ffff880062f73a48 ffff8800c5122e40
      Call Trace:
       [<ffffffff8119aa7e>] __fscache_read_or_alloc_pages+0x1fe/0x530
       [<ffffffff81250780>] __nfs_readpages_from_fscache+0x70/0x1c0
       [<ffffffff8123142a>] nfs_readpages+0xca/0x1e0
       [<ffffffff815f3c06>] ? rpc_do_put_task+0x36/0x50
       [<ffffffff8122755b>] ? alloc_nfs_open_context+0x4b/0x110
       [<ffffffff815ecd1a>] ? rpc_call_sync+0x5a/0x70
       [<ffffffff810e7e9a>] __do_page_cache_readahead+0x1ca/0x270
       [<ffffffff810e7f61>] ra_submit+0x21/0x30
       [<ffffffff810e818d>] ondemand_readahead+0x11d/0x250
       [<ffffffff810e83b6>] page_cache_sync_readahead+0x36/0x60
       [<ffffffff810dffa4>] generic_file_aio_read+0x454/0x770
       [<ffffffff81224ce1>] nfs_file_read+0xe1/0x130
       [<ffffffff81121bd9>] do_sync_read+0xd9/0x120
       [<ffffffff8114088f>] ? mntput+0x1f/0x40
       [<ffffffff811238cb>] ? fput+0x1cb/0x260
       [<ffffffff81122938>] vfs_read+0xc8/0x180
       [<ffffffff81122af5>] sys_read+0x55/0x90
      Reported-by: default avatarMark Moseley <moseleymark@gmail.com>
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      b4cf1e08
    • David Howells's avatar
      FS-Cache: Provide proper invalidation · ef778e7a
      David Howells authored
      Provide a proper invalidation method rather than relying on the netfs retiring
      the cookie it has and getting a new one.  The problem with this is that isn't
      easy for the netfs to make sure that it has completed/cancelled all its
      outstanding storage and retrieval operations on the cookie it is retiring.
      
      Instead, have the cache provide an invalidation method that will cancel or wait
      for all currently outstanding operations before invalidating the cache, and
      will cause new operations to queue up behind that.  Whilst invalidation is in
      progress, some requests will be rejected until the cache can stack a barrier on
      the operation queue to cause new operations to be deferred behind it.
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      ef778e7a
    • David Howells's avatar
      FS-Cache: Fix operation state management and accounting · 9f10523f
      David Howells authored
      Fix the state management of internal fscache operations and the accounting of
      what operations are in what states.
      
      This is done by:
      
       (1) Give struct fscache_operation a enum variable that directly represents the
           state it's currently in, rather than spreading this knowledge over a bunch
           of flags, who's processing the operation at the moment and whether it is
           queued or not.
      
           This makes it easier to write assertions to check the state at various
           points and to prevent invalid state transitions.
      
       (2) Add an 'operation complete' state and supply a function to indicate the
           completion of an operation (fscache_op_complete()) and make things call
           it.  The final call to fscache_put_operation() can then check that an op
           in the appropriate state (complete or cancelled).
      
       (3) Adjust the use of object->n_ops, ->n_in_progress, ->n_exclusive to better
           govern the state of an object:
      
      	(a) The ->n_ops is now the number of extant operations on the object
      	    and is now decremented by fscache_put_operation() only.
      
      	(b) The ->n_in_progress is simply the number of objects that have been
      	    taken off of the object's pending queue for the purposes of being
      	    run.  This is decremented by fscache_op_complete() only.
      
      	(c) The ->n_exclusive is the number of exclusive ops that have been
      	    submitted and queued or are in progress.  It is decremented by
      	    fscache_op_complete() and by fscache_cancel_op().
      
           fscache_put_operation() and fscache_operation_gc() now no longer try to
           clean up ->n_exclusive and ->n_in_progress.  That was leading to double
           decrements against fscache_cancel_op().
      
           fscache_cancel_op() now no longer decrements ->n_ops.  That was leading to
           double decrements against fscache_put_operation().
      
           fscache_submit_exclusive_op() now decides whether it has to queue an op
           based on ->n_in_progress being > 0 rather than ->n_ops > 0 as the latter
           will persist in being true even after all preceding operations have been
           cancelled or completed.  Furthermore, if an object is active and there are
           runnable ops against it, there must be at least one op running.
      
       (4) Add a remaining-pages counter (n_pages) to struct fscache_retrieval and
           provide a function to record completion of the pages as they complete.
      
           When n_pages reaches 0, the operation is deemed to be complete and
           fscache_op_complete() is called.
      
           Add calls to fscache_retrieval_complete() anywhere we've finished with a
           page we've been given to read or allocate for.  This includes places where
           we just return pages to the netfs for reading from the server and where
           accessing the cache fails and we discard the proposed netfs page.
      
      The bugs in the unfixed state management manifest themselves as oopses like the
      following where the operation completion gets out of sync with return of the
      cookie by the netfs.  This is possible because the cache unlocks and returns
      all the netfs pages before recording its completion - which means that there's
      nothing to stop the netfs discarding them and returning the cookie.
      
      
      FS-Cache: Cookie 'NFS.fh' still has outstanding reads
      ------------[ cut here ]------------
      kernel BUG at fs/fscache/cookie.c:519!
      invalid opcode: 0000 [#1] SMP
      CPU 1
      Modules linked in: cachefiles nfs fscache auth_rpcgss nfs_acl lockd sunrpc
      
      Pid: 400, comm: kswapd0 Not tainted 3.1.0-rc7-fsdevel+ #1090                  /DG965RY
      RIP: 0010:[<ffffffffa007050a>]  [<ffffffffa007050a>] __fscache_relinquish_cookie+0x170/0x343 [fscache]
      RSP: 0018:ffff8800368cfb00  EFLAGS: 00010282
      RAX: 000000000000003c RBX: ffff880023cc8790 RCX: 0000000000000000
      RDX: 0000000000002f2e RSI: 0000000000000001 RDI: ffffffff813ab86c
      RBP: ffff8800368cfb50 R08: 0000000000000002 R09: 0000000000000000
      R10: ffff88003a1b7890 R11: ffff88001df6e488 R12: ffff880023d8ed98
      R13: ffff880023cc8798 R14: 0000000000000004 R15: ffff88003b8bf370
      FS:  0000000000000000(0000) GS:ffff88003bd00000(0000) knlGS:0000000000000000
      CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
      CR2: 00000000008ba008 CR3: 0000000023d93000 CR4: 00000000000006e0
      DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
      DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
      Process kswapd0 (pid: 400, threadinfo ffff8800368ce000, task ffff88003b8bf040)
      Stack:
       ffff88003b8bf040 ffff88001df6e528 ffff88001df6e528 ffffffffa00b46b0
       ffff88003b8bf040 ffff88001df6e488 ffff88001df6e620 ffffffffa00b46b0
       ffff88001ebd04c8 0000000000000004 ffff8800368cfb70 ffffffffa00b2c91
      Call Trace:
       [<ffffffffa00b2c91>] nfs_fscache_release_inode_cookie+0x3b/0x47 [nfs]
       [<ffffffffa008f25f>] nfs_clear_inode+0x3c/0x41 [nfs]
       [<ffffffffa0090df1>] nfs4_evict_inode+0x2f/0x33 [nfs]
       [<ffffffff810d8d47>] evict+0xa1/0x15c
       [<ffffffff810d8e2e>] dispose_list+0x2c/0x38
       [<ffffffff810d9ebd>] prune_icache_sb+0x28c/0x29b
       [<ffffffff810c56b7>] prune_super+0xd5/0x140
       [<ffffffff8109b615>] shrink_slab+0x102/0x1ab
       [<ffffffff8109d690>] balance_pgdat+0x2f2/0x595
       [<ffffffff8103e009>] ? process_timeout+0xb/0xb
       [<ffffffff8109dba3>] kswapd+0x270/0x289
       [<ffffffff8104c5ea>] ? __init_waitqueue_head+0x46/0x46
       [<ffffffff8109d933>] ? balance_pgdat+0x595/0x595
       [<ffffffff8104bf7a>] kthread+0x7f/0x87
       [<ffffffff813ad6b4>] kernel_thread_helper+0x4/0x10
       [<ffffffff81026b98>] ? finish_task_switch+0x45/0xc0
       [<ffffffff813abcdd>] ? retint_restore_args+0xe/0xe
       [<ffffffff8104befb>] ? __init_kthread_worker+0x53/0x53
       [<ffffffff813ad6b0>] ? gs_change+0xb/0xb
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      9f10523f
    • David Howells's avatar
      FS-Cache: Make cookie relinquishment wait for outstanding reads · ef46ed88
      David Howells authored
      Make fscache_relinquish_cookie() log a warning and wait if there are any
      outstanding reads left on the cookie it was given.
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      ef46ed88
    • David Howells's avatar
      FS-Cache: Check that there are no read ops when cookie relinquished · 0f972b56
      David Howells authored
      Check that the netfs isn't trying to relinquish a cookie that still has read
      operations in progress upon it.  If there are, then give log a warning and BUG.
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      0f972b56
    • David Howells's avatar
      CacheFiles: Downgrade the requirements passed to the allocator · 5f4f9f4a
      David Howells authored
      Downgrade the requirements passed to the allocator in the gfp flags parameter.
      FS-Cache/CacheFiles can handle OOM conditions simply by aborting the attempt to
      store an object or a page in the cache.
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      5f4f9f4a
    • David Howells's avatar
      CacheFiles: Fix the marking of cached pages · c4d6d8db
      David Howells authored
      Under some circumstances CacheFiles defers the marking of pages with PG_fscache
      so that it can take advantage of pagevecs to reduce the number of calls to
      fscache_mark_pages_cached() and the netfs's hook to keep track of this.
      
      There are, however, two problems with this:
      
       (1) It can lead to the PG_fscache mark being applied _after_ the page is set
           PG_uptodate and unlocked (by the call to fscache_end_io()).
      
       (2) CacheFiles's ref on the page is dropped immediately following
           fscache_end_io() - and so may not still be held when the mark is applied.
           This can lead to the page being passed back to the allocator before the
           mark is applied.
      
      Fix this by, where appropriate, marking the page before calling
      fscache_end_io() and releasing the page.  This means that we can't take
      advantage of pagevecs and have to make a separate call for each page to the
      marking routines.
      
      The symptoms of this are Bad Page state errors cropping up under memory
      pressure, for example:
      
      BUG: Bad page state in process tar  pfn:002da
      page:ffffea0000009fb0 count:0 mapcount:0 mapping:          (null) index:0x1447
      page flags: 0x1000(private_2)
      Pid: 4574, comm: tar Tainted: G        W   3.1.0-rc4-fsdevel+ #1064
      Call Trace:
       [<ffffffff8109583c>] ? dump_page+0xb9/0xbe
       [<ffffffff81095916>] bad_page+0xd5/0xea
       [<ffffffff81095d82>] get_page_from_freelist+0x35b/0x46a
       [<ffffffff810961f3>] __alloc_pages_nodemask+0x362/0x662
       [<ffffffff810989da>] __do_page_cache_readahead+0x13a/0x267
       [<ffffffff81098942>] ? __do_page_cache_readahead+0xa2/0x267
       [<ffffffff81098d7b>] ra_submit+0x1c/0x20
       [<ffffffff8109900a>] ondemand_readahead+0x28b/0x29a
       [<ffffffff81098ee2>] ? ondemand_readahead+0x163/0x29a
       [<ffffffff810990ce>] page_cache_sync_readahead+0x38/0x3a
       [<ffffffff81091d8a>] generic_file_aio_read+0x2ab/0x67e
       [<ffffffffa008cfbe>] nfs_file_read+0xa4/0xc9 [nfs]
       [<ffffffff810c22c4>] do_sync_read+0xba/0xfa
       [<ffffffff81177a47>] ? security_file_permission+0x7b/0x84
       [<ffffffff810c25dd>] ? rw_verify_area+0xab/0xc8
       [<ffffffff810c29a4>] vfs_read+0xaa/0x13a
       [<ffffffff810c2a79>] sys_read+0x45/0x6c
       [<ffffffff813ac37b>] system_call_fastpath+0x16/0x1b
      
      As can be seen, PG_private_2 (== PG_fscache) is set in the page flags.
      
      Instrumenting fscache_mark_pages_cached() to verify whether page->mapping was
      set appropriately showed that sometimes it wasn't.  This led to the discovery
      that sometimes the page has apparently been reclaimed by the time the marker
      got to see it.
      Reported-by: default avatarM. Stevens <m@tippett.com>
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      Reviewed-by: default avatarJeff Layton <jlayton@redhat.com>
      c4d6d8db
  4. 21 Jul, 2011 1 commit
  5. 07 Jul, 2011 1 commit
    • David Howells's avatar
      FS-Cache: Add a helper to bulk uncache pages on an inode · c902ce1b
      David Howells authored
      Add an FS-Cache helper to bulk uncache pages on an inode.  This will
      only work for the circumstance where the pages in the cache correspond
      1:1 with the pages attached to an inode's page cache.
      
      This is required for CIFS and NFS: When disabling inode cookie, we were
      returning the cookie and setting cifsi->fscache to NULL but failed to
      invalidate any previously mapped pages.  This resulted in "Bad page
      state" errors and manifested in other kind of errors when running
      fsstress.  Fix it by uncaching mapped pages when we disable the inode
      cookie.
      
      This patch should fix the following oops and "Bad page state" errors
      seen during fsstress testing.
      
        ------------[ cut here ]------------
        kernel BUG at fs/cachefiles/namei.c:201!
        invalid opcode: 0000 [#1] SMP
        Pid: 5, comm: kworker/u:0 Not tainted 2.6.38.7-30.fc15.x86_64 #1 Bochs Bochs
        RIP: 0010: cachefiles_walk_to_object+0x436/0x745 [cachefiles]
        RSP: 0018:ffff88002ce6dd00  EFLAGS: 00010282
        RAX: ffff88002ef165f0 RBX: ffff88001811f500 RCX: 0000000000000000
        RDX: 0000000000000000 RSI: 0000000000000100 RDI: 0000000000000282
        RBP: ffff88002ce6dda0 R08: 0000000000000100 R09: ffffffff81b3a300
        R10: 0000ffff00066c0a R11: 0000000000000003 R12: ffff88002ae54840
        R13: ffff88002ae54840 R14: ffff880029c29c00 R15: ffff88001811f4b0
        FS:  00007f394dd32720(0000) GS:ffff88002ef00000(0000) knlGS:0000000000000000
        CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
        CR2: 00007fffcb62ddf8 CR3: 000000001825f000 CR4: 00000000000006e0
        DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
        DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
        Process kworker/u:0 (pid: 5, threadinfo ffff88002ce6c000, task ffff88002ce55cc0)
        Stack:
         0000000000000246 ffff88002ce55cc0 ffff88002ce6dd58 ffff88001815dc00
         ffff8800185246c0 ffff88001811f618 ffff880029c29d18 ffff88001811f380
         ffff88002ce6dd50 ffffffff814757e4 ffff88002ce6dda0 ffffffff8106ac56
        Call Trace:
         cachefiles_lookup_object+0x78/0xd4 [cachefiles]
         fscache_lookup_object+0x131/0x16d [fscache]
         fscache_object_work_func+0x1bc/0x669 [fscache]
         process_one_work+0x186/0x298
         worker_thread+0xda/0x15d
         kthread+0x84/0x8c
         kernel_thread_helper+0x4/0x10
        RIP  cachefiles_walk_to_object+0x436/0x745 [cachefiles]
        ---[ end trace 1d481c9af1804caa ]---
      
      I tested the uncaching by the following means:
      
       (1) Create a big file on my NFS server (104857600 bytes).
      
       (2) Read the file into the cache with md5sum on the NFS client.  Look in
           /proc/fs/fscache/stats:
      
      	Pages  : mrk=25601 unc=0
      
       (3) Open the file for read/write ("bash 5<>/warthog/bigfile").  Look in proc
           again:
      
      	Pages  : mrk=25601 unc=25601
      Reported-by: default avatarJeff Layton <jlayton@redhat.com>
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      Reviewed-and-Tested-by: default avatarSuresh Jayaraman <sjayaraman@suse.de>
      cc: stable@kernel.org
      Signed-off-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      c902ce1b
  6. 25 May, 2011 1 commit
  7. 14 Jan, 2011 1 commit
  8. 12 Aug, 2010 1 commit
  9. 24 Jul, 2010 1 commit
  10. 22 Jul, 2010 3 commits
    • Tejun Heo's avatar
      fscache: drop references to slow-work · d098adfb
      Tejun Heo authored
      fscache no longer uses slow-work.  Drop references to it.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Acked-by: default avatarDavid Howells <dhowells@redhat.com>
      d098adfb
    • Tejun Heo's avatar
      fscache: convert operation to use workqueue instead of slow-work · 8af7c124
      Tejun Heo authored
      Make fscache operation to use only workqueue instead of combination of
      workqueue and slow-work.  FSCACHE_OP_SLOW is dropped and
      FSCACHE_OP_FAST is renamed to FSCACHE_OP_ASYNC and uses newly added
      fscache_op_wq workqueue to execute op->processor().
      fscache_operation_init_slow() is dropped and fscache_operation_init()
      now takes @processor argument directly.
      
      * Unbound workqueue is used.
      
      * fscache_retrieval_work() is no longer necessary as OP_ASYNC now does
        the equivalent thing.
      
      * sysctl fscache.operation_max_active added to control concurrency.
        The default value is nr_cpus clamped between 2 and
        WQ_UNBOUND_MAX_ACTIVE.
      
      * debugfs support is dropped for now.  Tracing API based debug
        facility is planned to be added.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Acked-by: default avatarDavid Howells <dhowells@redhat.com>
      8af7c124
    • Tejun Heo's avatar
      fscache: convert object to use workqueue instead of slow-work · 8b8edefa
      Tejun Heo authored
      Make fscache object state transition callbacks use workqueue instead
      of slow-work.  New dedicated unbound CPU workqueue fscache_object_wq
      is created.  get/put callbacks are renamed and modified to take
      @object and called directly from the enqueue wrapper and the work
      function.  While at it, make all open coded instances of get/put to
      use fscache_get/put_object().
      
      * Unbound workqueue is used.
      
      * work_busy() output is printed instead of slow-work flags in object
        debugging outputs.  They mean basically the same thing bit-for-bit.
      
      * sysctl fscache.object_max_active added to control concurrency.  The
        default value is nr_cpus clamped between 4 and
        WQ_UNBOUND_MAX_ACTIVE.
      
      * slow_work_sleep_till_thread_needed() is replaced with fscache
        private implementation fscache_object_sleep_till_congested() which
        waits on fscache_object_wq congestion.
      
      * debugfs support is dropped for now.  Tracing API based debug
        facility is planned to be added.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Acked-by: default avatarDavid Howells <dhowells@redhat.com>
      8b8edefa
  11. 01 Jun, 2010 1 commit
  12. 26 May, 2010 1 commit
  13. 07 Apr, 2010 1 commit
  14. 30 Mar, 2010 1 commit
    • Tejun Heo's avatar
      include cleanup: Update gfp.h and slab.h includes to prepare for breaking... · 5a0e3ad6
      Tejun Heo authored
      include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
      
      percpu.h is included by sched.h and module.h and thus ends up being
      included when building most .c files.  percpu.h includes slab.h which
      in turn includes gfp.h making everything defined by the two files
      universally available and complicating inclusion dependencies.
      
      percpu.h -> slab.h dependency is about to be removed.  Prepare for
      this change by updating users of gfp and slab facilities include those
      headers directly instead of assuming availability.  As this conversion
      needs to touch large number of source files, the following script is
      used as the basis of conversion.
      
        http://userweb.kernel.org/~tj/misc/slabh-sweep.py
      
      The script does the followings.
      
      * Scan files for gfp and slab usages and update includes such that
        only the necessary includes are there.  ie. if only gfp is used,
        gfp.h, if slab is used, slab.h.
      
      * When the script inserts a new include, it looks at the include
        blocks and try to put the new include such that its order conforms
        to its surrounding.  It's put in the include block which contains
        core kernel includes, in the same order that the rest are ordered -
        alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
        doesn't seem to be any matching order.
      
      * If the script can't find a place to put a new include (mostly
        because the file doesn't have fitting include block), it prints out
        an error message indicating which .h file needs to be added to the
        file.
      
      The conversion was done in the following steps.
      
      1. The initial automatic conversion of all .c files updated slightly
         over 4000 files, deleting around 700 includes and adding ~480 gfp.h
         and ~3000 slab.h inclusions.  The script emitted errors for ~400
         files.
      
      2. Each error was manually checked.  Some didn't need the inclusion,
         some needed manual addition while adding it to implementation .h or
         embedding .c file was more appropriate for others.  This step added
         inclusions to around 150 files.
      
      3. The script was run again and the output was compared to the edits
         from #2 to make sure no file was left behind.
      
      4. Several build tests were done and a couple of problems were fixed.
         e.g. lib/decompress_*.c used malloc/free() wrappers around slab
         APIs requiring slab.h to be added manually.
      
      5. The script was run on all .h files but without automatically
         editing them as sprinkling gfp.h and slab.h inclusions around .h
         files could easily lead to inclusion dependency hell.  Most gfp.h
         inclusion directives were ignored as stuff from gfp.h was usually
         wildly available and often used in preprocessor macros.  Each
         slab.h inclusion directive was examined and added manually as
         necessary.
      
      6. percpu.h was updated not to include slab.h.
      
      7. Build test were done on the following configurations and failures
         were fixed.  CONFIG_GCOV_KERNEL was turned off for all tests (as my
         distributed build env didn't work with gcov compiles) and a few
         more options had to be turned off depending on archs to make things
         build (like ipr on powerpc/64 which failed due to missing writeq).
      
         * x86 and x86_64 UP and SMP allmodconfig and a custom test config.
         * powerpc and powerpc64 SMP allmodconfig
         * sparc and sparc64 SMP allmodconfig
         * ia64 SMP allmodconfig
         * s390 SMP allmodconfig
         * alpha SMP allmodconfig
         * um on x86_64 SMP allmodconfig
      
      8. percpu.h modifications were reverted so that it could be applied as
         a separate patch and serve as bisection point.
      
      Given the fact that I had only a couple of failures from tests on step
      6, I'm fairly confident about the coverage of this conversion patch.
      If there is a breakage, it's likely to be something in one of the arch
      headers which should be easily discoverable easily on most builds of
      the specific arch.
      Signed-off-by: default avatarTejun Heo <tj@kernel.org>
      Guess-its-ok-by: default avatarChristoph Lameter <cl@linux-foundation.org>
      Cc: Ingo Molnar <mingo@redhat.com>
      Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
      5a0e3ad6
  15. 29 Mar, 2010 1 commit
  16. 24 Mar, 2010 1 commit
  17. 08 Mar, 2010 1 commit
  18. 16 Dec, 2009 1 commit
  19. 20 Nov, 2009 1 commit
    • David Howells's avatar
      FS-Cache: Provide nop fscache_stat_d() if CONFIG_FSCACHE_STATS=n · 4fa9f4ed
      David Howells authored
      Provide nop fscache_stat_d() macro if CONFIG_FSCACHE_STATS=n lest errors like
      the following occur:
      
      	fs/fscache/cache.c: In function 'fscache_withdraw_cache':
      	fs/fscache/cache.c:386: error: implicit declaration of function 'fscache_stat_d'
      	fs/fscache/cache.c:386: error: 'fscache_n_cop_sync_cache' undeclared (first use in this function)
      	fs/fscache/cache.c:386: error: (Each undeclared identifier is reported only once
      	fs/fscache/cache.c:386: error: for each function it appears in.)
      	fs/fscache/cache.c:392: error: 'fscache_n_cop_dissociate_pages' undeclared (first use in this function)
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      4fa9f4ed
  20. 19 Nov, 2009 3 commits
    • David Howells's avatar
      CacheFiles: Catch an overly long wait for an old active object · fee096de
      David Howells authored
      Catch an overly long wait for an old, dying active object when we want to
      replace it with a new one.  The probability is that all the slow-work threads
      are hogged, and the delete can't get a look in.
      
      What we do instead is:
      
       (1) if there's nothing in the slow work queue, we sleep until either the dying
           object has finished dying or there is something in the slow work queue
           behind which we can queue our object.
      
       (2) if there is something in the slow work queue, we return ETIMEDOUT to
           fscache_lookup_object(), which then puts us back on the slow work queue,
           presumably behind the deletion that we're blocked by.  We are then
           deferred for a while until we work our way back through the queue -
           without blocking a slow-work thread unnecessarily.
      
      A backtrace similar to the following may appear in the log without this patch:
      
      	INFO: task kslowd004:5711 blocked for more than 120 seconds.
      	"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
      	kslowd004     D 0000000000000000     0  5711      2 0x00000080
      	 ffff88000340bb80 0000000000000046 ffff88002550d000 0000000000000000
      	 ffff88002550d000 0000000000000007 ffff88000340bfd8 ffff88002550d2a8
      	 000000000000ddf0 00000000000118c0 00000000000118c0 ffff88002550d2a8
      	Call Trace:
      	 [<ffffffff81058e21>] ? trace_hardirqs_on+0xd/0xf
      	 [<ffffffffa011c4d8>] ? cachefiles_wait_bit+0x0/0xd [cachefiles]
      	 [<ffffffffa011c4e1>] cachefiles_wait_bit+0x9/0xd [cachefiles]
      	 [<ffffffff81353153>] __wait_on_bit+0x43/0x76
      	 [<ffffffff8111ae39>] ? ext3_xattr_get+0x1ec/0x270
      	 [<ffffffff813531ef>] out_of_line_wait_on_bit+0x69/0x74
      	 [<ffffffffa011c4d8>] ? cachefiles_wait_bit+0x0/0xd [cachefiles]
      	 [<ffffffff8104c125>] ? wake_bit_function+0x0/0x2e
      	 [<ffffffffa011bc79>] cachefiles_mark_object_active+0x203/0x23b [cachefiles]
      	 [<ffffffffa011c209>] cachefiles_walk_to_object+0x558/0x827 [cachefiles]
      	 [<ffffffffa011a429>] cachefiles_lookup_object+0xac/0x12a [cachefiles]
      	 [<ffffffffa00aa1e9>] fscache_lookup_object+0x1c7/0x214 [fscache]
      	 [<ffffffffa00aafc5>] fscache_object_state_machine+0xa5/0x52d [fscache]
      	 [<ffffffffa00ab4ac>] fscache_object_slow_work_execute+0x5f/0xa0 [fscache]
      	 [<ffffffff81082093>] slow_work_execute+0x18f/0x2d1
      	 [<ffffffff8108239a>] slow_work_thread+0x1c5/0x308
      	 [<ffffffff8104c0f1>] ? autoremove_wake_function+0x0/0x34
      	 [<ffffffff810821d5>] ? slow_work_thread+0x0/0x308
      	 [<ffffffff8104be91>] kthread+0x7a/0x82
      	 [<ffffffff8100beda>] child_rip+0xa/0x20
      	 [<ffffffff8100b87c>] ? restore_args+0x0/0x30
      	 [<ffffffff8104be17>] ? kthread+0x0/0x82
      	 [<ffffffff8100bed0>] ? child_rip+0x0/0x20
      	1 lock held by kslowd004/5711:
      	 #0:  (&sb->s_type->i_mutex_key#7/1){+.+.+.}, at: [<ffffffffa011be64>] cachefiles_walk_to_object+0x1b3/0x827 [cachefiles]
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      fee096de
    • David Howells's avatar
      FS-Cache: Actually requeue an object when requested · 868411be
      David Howells authored
      FS-Cache objects have an FSCACHE_OBJECT_EV_REQUEUE event that can theoretically
      be raised to ask the state machine to requeue the object for further processing
      before the work function returns to the slow-work facility.
      
      However, fscache_object_work_execute() was clearing that bit before checking
      the event mask to see whether the object has any pending events that require it
      to be requeued immediately.
      
      Instead, the bit should be cleared after the check and enqueue.
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      868411be
    • David Howells's avatar
      FS-Cache: Start processing an object's operations on that object's death · 60d543ca
      David Howells authored
      Start processing an object's operations when that object moves into the DYING
      state as the object cannot be destroyed until all its outstanding operations
      have completed.
      
      Furthermore, make sure that read and allocation operations handle being woken
      up on a dead object.  Such events are recorded in the Allocs.abt and
      Retrvls.abt statistics as viewable through /proc/fs/fscache/stats.
      
      The code for waiting for object activation for the read and allocation
      operations is also extracted into its own function as it is much the same in
      all cases, differing only in the stats incremented.
      Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
      60d543ca