Forum Linux.noyau Problème NFS quota projects

Posté par  . Licence CC By‑SA.
Étiquettes :
0
4
avr.
2014

Bonjour

Je rencontre actuellement un problème sur un système de fichiers formaté en xfs et exporté en nfs v3.
Sur ce système de fichiers il a été décidé de positionner des quotas project ( /etc/projid & /etc/projects )

Le volume est monté sur le serveur et exporter en nfs

mount ==> /dev/mapper/vghome-lvhome on /home type xfs (rw,prjquota)
exportfs ==> /home xx.xx.xx.xx/21(rw,async,no_subtree_check,no_root_squash)

Le problème qui survient c'est que lorsqu'un utilisateur dépasse le quota sur un répertoire, le système de fichier côté client est indisponible , côté serveur le fs est accessible normallement.

IL faut que j'augmente le quota du répertoire pour que le fs soit denouveau disponible

j'ai vu que sur mon server et sur mes client le process [rpciod] est unique contrairement à d'autre serveur et même client ou ce process est multiple ( 8 ou 16 ).

Comment augmenter ce process ?

Avez vous déjà rencontré ce type de problème ?


Apr 3 06:41:47 server kernel: [2384793.913916] nfsd: page allocation failure: order:1, mode:0x20
Apr 3 06:41:47 server kernel: [2384793.913921] Pid: 2981, comm: nfsd Tainted: G O 3.2.0-0.bpo.4-amd64 #1 Debian 3.2.51-1~c7+1
Apr 3 06:41:47 server kernel: [2384793.913923] Call Trace:
Apr 3 06:41:47 server kernel: [2384793.913925] [] ? warn_alloc_failed+0x10a/0x11d
Apr 3 06:41:47 server kernel: [2384793.913935] [] ? __alloc_pages_nodemask+0x715/0x7a4
Apr 3 06:41:47 server kernel: [2384793.913941] [] ? kmem_getpages+0x4b/0x107
Apr 3 06:41:47 server kernel: [2384793.913944] [] ? fallback_alloc+0x165/0x200
Apr 3 06:41:47 server kernel: [2384793.913947] [] ? kmem_cache_alloc_node_trace+0x97/0x115
Apr 3 06:41:47 server kernel: [2384793.913951] [] ? __alloc_skb+0x70/0x136
Apr 3 06:41:47 server kernel: [2384793.913953] [] ? dev_alloc_skb+0x16/0x2d
Apr 3 06:41:47 server kernel: [2384793.913962] [] ? ipoib_alloc_rx_skb+0x57/0xc9 [ib_ipoib]
Apr 3 06:41:47 server kernel: [2384793.913967] [] ? ipoib_ib_handle_rx_wc+0x1d9/0x63c [ib_ipoib]
Apr 3 06:41:47 server kernel: [2384793.913976] [] ? mlx4_cq_completion+0x6a/0x8c [mlx4_core]
Apr 3 06:41:47 server kernel: [2384793.913980] [] ? ipoib_poll+0x8f/0x11b [ib_ipoib]
Apr 3 06:41:47 server kernel: [2384793.913984] [] ? __wake_up+0x35/0x46
Apr 3 06:41:47 server kernel: [2384793.913987] [] ? net_rx_action+0xa8/0x207
Apr 3 06:41:47 server kernel: [2384793.913991] [] ? iov_iter_fault_in_readable+0xe/0x52
Apr 3 06:41:47 server kernel: [2384793.913995] [] ? __do_softirq+0xc4/0x1a0
Apr 3 06:41:47 server kernel: [2384793.913999] [] ? handle_irq_event_percpu+0x163/0x181
Apr 3 06:41:47 server kernel: [2384793.914003] [] ? call_softirq+0x1c/0x30
Apr 3 06:41:47 server kernel: [2384793.914008] [] ? do_softirq+0x3f/0x79
Apr 3 06:41:47 server kernel: [2384793.914010] [] ? irq_exit+0x44/0xb4
Apr 3 06:41:47 server kernel: [2384793.914013] [] ? do_IRQ+0x94/0xaa
Apr 3 06:41:47 server kernel: [2384793.914017] [] ? common_interrupt+0x6e/0x6e
Apr 3 06:41:47 server kernel: [2384793.914018] [] ? iov_iter_fault_in_readable+0xe/0x52
Apr 3 06:41:47 server kernel: [2384793.914023] [] ? generic_file_buffered_write+0xdc/0x28e
Apr 3 06:41:47 server kernel: [2384793.914035] [] ? xfs_file_buffered_aio_write+0x10b/0x15d [xfs]
Apr 3 06:41:47 server kernel: [2384793.914042] [] ? xfs_file_aio_write+0x18b/0x226 [xfs]
Apr 3 06:41:47 server kernel: [2384793.914050] [] ? xfs_iget+0x50d/0x582 [xfs]
Apr 3 06:41:47 server kernel: [2384793.914058] [] ? xfs_file_fsync+0x234/0x234 [xfs]
Apr 3 06:41:47 server kernel: [2384793.914061] [] ? do_sync_readv_writev+0xa3/0xde
Apr 3 06:41:47 server kernel: [2384793.914067] [] ? fh_compose+0x33f/0x33f [nfsd]
Apr 3 06:41:47 server kernel: [2384793.914070] [] ? rw_copy_check_uvector+0x52/0xfa
Apr 3 06:41:47 server kernel: [2384793.914072] [] ? __kmalloc+0x104/0x116
Apr 3 06:41:47 server kernel: [2384793.914074] [] ? copy_from_user+0x18/0x30
Apr 3 06:41:47 server kernel: [2384793.914079] [] ? security_file_permission+0x18/0x2d
Apr 3 06:41:47 server kernel: [2384793.914081] [] ? do_readv_writev+0xaf/0x125
Apr 3 06:41:47 server kernel: [2384793.914087] [] ? nfsd_setuser+0x199/0x1de [nfsd]
Apr 3 06:41:47 server kernel: [2384793.914091] [] ? nfsd_setuser_and_check_port+0x76/0x92 [nfsd]
Apr 3 06:41:47 server kernel: [2384793.914097] [] ? nfsd_vfs_write+0xc9/0x2b5 [nfsd]
Apr 3 06:41:47 server kernel: [2384793.914101] [] ? nfsd_open+0xfd/0x14f [nfsd]
Apr 3 06:41:47 server kernel: [2384793.914106] [] ? nfsd_write+0xbb/0xdc [nfsd]
Apr 3 06:41:47 server kernel: [2384793.914112] [] ? nfsd3_proc_write+0xcf/0xea [nfsd]
Apr 3 06:41:47 server kernel: [2384793.914116] [] ? nfsd_dispatch+0xd9/0x1bb [nfsd]
Apr 3 06:41:47 server kernel: [2384793.914123] [] ? svc_process_common+0x2d0/0x4ca [sunrpc]
Apr 3 06:41:47 server kernel: [2384793.914126] [] ? try_to_wake_up+0x191/0x191
Apr 3 06:41:47 server kernel: [2384793.914131] [] ? nfsd_svc+0x17f/0x17f [nfsd]
Apr 3 06:41:47 server kernel: [2384793.914136] [] ? svc_process+0x10d/0x12c [sunrpc]
Apr 3 06:41:47 server kernel: [2384793.914140] [] ? nfsd+0xe3/0x127 [nfsd]
Apr 3 06:41:47 server kernel: [2384793.914144] [] ? kthread+0x7a/0x82
Apr 3 06:41:47 server kernel: [2384793.914146] [] ? kernel_thread_helper+0x4/0x10
Apr 3 06:41:47 server kernel: [2384793.914149] [] ? kthread_worker_fn+0x147/0x147
Apr 3 06:41:47 server kernel: [2384793.914152] [] ? gs_change+0x13/0x13

Merci d'avance

  • # précisions

    Posté par  (site web personnel) . Évalué à 2.

    le système de fichier côté client est indisponible 
    

    Pour le même utilisateur, d'autres ?

    Que donne la commande qui renseigne le quota dispo ?

    Système - Réseau - Sécurité Open Source

    • [^] # Re: précisions

      Posté par  . Évalué à 1.

      Bonjour

      Oui exact, le système de fichiers est indisponible pour tous les utilisateurs (y compris root).

  • # Mes 2 cts

    Posté par  . Évalué à 2.

    Bonjour, si je comprend bien la call-trace elle indique un échec d'allocation de page mémoire dans le kernel suite à une interruption de réception de frame ethernet.
    Sur ce lien un problème similaire se contournait en doublant (ou quadruplant) la valeur sysctl vm.min_free_kbytes
    Peut être que cela résoudra aussi ton problème ?

    • [^] # Re: Mes 2 cts

      Posté par  . Évalué à 1.

      Bonjour

      Merci pour votre réponse,
      Pour moi cette valeur est à vm.min_free_kbytes = 67584
      par contre je ne sais pas si c'est déjà élevé ou non, en comparaison sur d'autre serveur exportant du nfs je suis largement en dessous ( vm.min_free_kbytes = 17248 )

      Alors j'ai déjà réussit à diminuer le phénomène ( d'indispo mais pas assez ls /home répond arès 2min :) ) en augmentant le nombre de process nfsd de 128 à 512 mais .
      J'ai essayé de passer à 1024 mais pas plus d'effet.

      J'ai réellement l'impression que ce process [rpciod] est lié au problème
      et je ne sais pas du tout comment augmenter son nombre.

      root 1441 2 0 07:58 ? 00:00:00 [rpciod]

      • [^] # Re: Mes 2 cts

        Posté par  . Évalué à 1.

        Désolé mais je ne suis pas un gourou de nfs.
        Si le "nombre" dont tu parle est le PID du process par contre je peut t'assurer qu'il s'agit d'un nombre permettant uniquement d'identifier dans le système le process en question.
        Le PID d'un programme ne change pas la manière dont le système autorise ou non les allocations ou la priorité d'exécution.
        La première ligne de ton post
        Apr 3 06:41:47 server kernel: [2384793.913916] nfsd: page allocation failure: order:1, mode:0x20
        Montre bien que l'erreur est indiqué par nfsd.
        Peut être qu'en activant les options de débbug de nfsd et rpcd tu aurais des traces indiquant l'origine du probleme.
        Est ce que par hasard ton réseau utiliserait des frames avec un MTU autre que 1500 du sytle 9000 ? Peut tu essayer de repasser en 1500 ?

        • [^] # Re: Mes 2 cts

          Posté par  . Évalué à 1.

          Ah exact j'y avais pas pensé par contre c'est de l'ib donc

          aujourd'hui j'ai un MTU à MTU:2044 , je vais le passer à MTU:65520 pour voir .
          Merci pour la piste , je vais également activer les mode debug , je te tiens au courant

  • # Suivi - Problème NFS quota projects

    Posté par  . Évalué à 1.

    En cherchant un peu cela implique de changer le mode de 'datagram' actuellement à 'connected'
    le problème c'est que c'est un cluster avec pas mal de noeuds et il faudrait le changer sur toutes les cartes IB
    J'ai activer le mode debug sur nfsd nfs et rpc , pour de nouveau call trace mais pas de nouveau depassement de quota.
    Je tenterai de reproduire le problème demain matin afin de voir qu'elle erreur je peux retrouver

    Je vous tiens au courant

  • # Toujours le problème mais pas les meme message d'erreur

    Posté par  . Évalué à 1.

    J'ai reproduit le problème qui survient lors du dépassement du quota hard et
    je n'est plus les même message d'erreur

    Apr 10 07:08:46 server kernel: [255844.565640] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.565645] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.565685] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.565703] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.565745] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.565810] RPC: Want update, refage=1800, age=954
    Apr 10 07:08:46 server kernel: [255844.565815] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.565831] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.565859] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.565903] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.565909] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.565950] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.565974] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.566015] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.566097] RPC: Want update, refage=1800, age=954
    Apr 10 07:08:46 server kernel: [255844.566107] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.566125] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.566157] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.566209] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.566210] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.566244] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.566267] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.566304] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.566365] RPC: Want update, refage=1800, age=954
    Apr 10 07:08:46 server kernel: [255844.566370] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.566384] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.566409] RPC: Want update, refage=1800, age=904
    Apr 10 07:08:46 server kernel: [255844.571522] rpc-srv/tcp: nfsd: got error -32 when sending 140 bytes - shutting down socket
    Apr 10 07:08:55 server kernel: [255853.954616] RPC: Want update, refage=1800, age=921
    Apr 10 07:08:58 server kernel: [255857.363244] nfsd: page allocation failure: order:1, mode:0x20

    Apr 10 07:08:08 server kernel: [255806.807201] INFO: task nfsd:2616 blocked for more than 120 seconds.
    Apr 10 07:08:08 server kernel: [255806.814454] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
    Apr 10 07:08:08 server kernel: [255806.823395] nfsd D ffff88045a5f0ab0 0 2616 2 0x00000000
    Apr 10 07:08:08 server kernel: [255806.823401] ffff88045a5f0ab0 0000000000000046 0000000000000000 ffff88044041b590
    Apr 10 07:08:08 server kernel: [255806.823405] 0000000000013740 ffff88046d379fd8 ffff88046d379fd8 0000000000013740
    Apr 10 07:08:08 server kernel: [255806.823408] ffff88045a5f0ab0 0000000000013740 0000000000013740 ffff88046d378010
    Apr 10 07:08:08 server kernel: [255806.823411] Call Trace:
    Apr 10 07:08:08 server kernel: [255806.823424] [] ? ipoib_start_xmit+0x683/0x710 [ib_ipoib]
    Apr 10 07:08:08 server kernel: [255806.823430] [] ? __mutex_lock_common+0x10c/0x172
    Apr 10 07:08:08 server kernel: [255806.823434] [] ? tcp_try_rmem_schedule+0x45/0x29a
    Apr 10 07:08:08 server kernel: [255806.823437] [] ? mutex_lock+0x1a/0x2c
    Apr 10 07:08:08 server kernel: [255806.823448] [] ? xfs_rw_ilock+0x1e/0x2e [xfs]
    Apr 10 07:08:08 server kernel: [255806.823456] [] ? xfs_file_buffered_aio_write+0x5c/0x15d [xfs]
    Apr 10 07:08:08 server kernel: [255806.823459] [] ? tcp_rcv_established+0x662/0x69e
    Apr 10 07:08:08 server kernel: [255806.823466] [] ? xfs_file_aio_write+0x18b/0x226 [xfs]
    Apr 10 07:08:08 server kernel: [255806.823475] [] ? xfs_iget+0x50d/0x582 [xfs]
    Apr 10 07:08:08 server kernel: [255806.823482] [] ? xfs_file_fsync+0x234/0x234 [xfs]
    Apr 10 07:08:08 server kernel: [255806.823486] [] ? do_sync_readv_writev+0xa3/0xde
    Apr 10 07:08:08 server kernel: [255806.823492] [] ? fh_compose+0x33f/0x33f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.823494] [] ? rw_copy_check_uvector+0x52/0xfa
    Apr 10 07:08:08 server kernel: [255806.823497] [] ? __kmalloc+0x104/0x116
    Apr 10 07:08:08 server kernel: [255806.823500] [] ? copy_from_user+0x18/0x30
    Apr 10 07:08:08 server kernel: [255806.823503] [] ? security_file_permission+0x18/0x2d
    Apr 10 07:08:08 server kernel: [255806.823506] [] ? do_readv_writev+0xaf/0x125
    Apr 10 07:08:08 server kernel: [255806.823511] [] ? nfsd_setuser+0x199/0x1de [nfsd]
    Apr 10 07:08:08 server kernel: [255806.823516] [] ? nfsd_setuser_and_check_port+0x76/0x92 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.823521] [] ? nfsd_vfs_write+0xc9/0x2b5 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.823526] [] ? nfsd_open+0xfd/0x14f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.823530] [] ? nfsd_write+0xbb/0xdc [nfsd]
    Apr 10 07:08:08 server kernel: [255806.823536] [] ? nfsd3_proc_write+0xcf/0xea [nfsd]
    Apr 10 07:08:08 server kernel: [255806.823540] [] ? nfsd_dispatch+0xd9/0x1bb [nfsd]
    Apr 10 07:08:08 server kernel: [255806.823547] [] ? svc_process_common+0x2d0/0x4ca [sunrpc]
    Apr 10 07:08:08 server kernel: [255806.823551] [] ? try_to_wake_up+0x191/0x191
    Apr 10 07:08:08 server kernel: [255806.823555] [] ? nfsd_svc+0x17f/0x17f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.823560] [] ? svc_process+0x10d/0x12c [sunrpc]
    Apr 10 07:08:08 server kernel: [255806.823565] [] ? nfsd+0xe3/0x127 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.823568] [] ? kthread+0x7a/0x82
    Apr 10 07:08:08 server kernel: [255806.823571] [] ? kernel_thread_helper+0x4/0x10
    Apr 10 07:08:08 server kernel: [255806.823574] [] ? kthread_worker_fn+0x147/0x147
    Apr 10 07:08:08 server kernel: [255806.823577] [] ? gs_change+0x13/0x13
    Apr 10 07:08:08 server kernel: [255806.823579] INFO: task nfsd:2617 blocked for more than 120 seconds.
    Apr 10 07:08:08 server kernel: [255806.830718] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
    Apr 10 07:08:08 server kernel: [255806.839659] nfsd D ffff88045a5f1890 0 2617 2 0x00000000
    Apr 10 07:08:08 server kernel: [255806.839664] ffff88045a5f1890 0000000000000046 0702422800000000 ffffffff8160d020
    Apr 10 07:08:08 server kernel: [255806.839667] 0000000000013740 ffff88046e7e5fd8 ffff88046e7e5fd8 0000000000013740
    Apr 10 07:08:08 server kernel: [255806.839671] ffff88045a5f1890 0000000000013740 0000000000013740 ffff88046e7e4010
    Apr 10 07:08:08 server kernel: [255806.839674] Call Trace:
    Apr 10 07:08:08 server kernel: [255806.839680] [] ? __mutex_lock_common+0x10c/0x172
    Apr 10 07:08:08 server kernel: [255806.839683] [] ? mutex_lock+0x1a/0x2c
    Apr 10 07:08:08 server kernel: [255806.839690] [] ? ipoib_start_xmit+0x683/0x710 [ib_ipoib]
    Apr 10 07:08:08 server kernel: [255806.839701] [] ? xfs_rw_ilock+0x1e/0x2e [xfs]
    Apr 10 07:08:08 server kernel: [255806.839708] [] ? xfs_file_buffered_aio_write+0x5c/0x15d [xfs]
    Apr 10 07:08:08 server kernel: [255806.839712] [] ? read_tsc+0x5/0x16
    Apr 10 07:08:08 server kernel: [255806.839719] [] ? xfs_file_aio_write+0x18b/0x226 [xfs]
    Apr 10 07:08:08 server kernel: [255806.839727] [] ? xfs_iget+0x50d/0x582 [xfs]
    Apr 10 07:08:08 server kernel: [255806.839734] [] ? xfs_file_fsync+0x234/0x234 [xfs]
    Apr 10 07:08:08 server kernel: [255806.839738] [] ? do_sync_readv_writev+0xa3/0xde
    Apr 10 07:08:08 server kernel: [255806.839744] [] ? fh_compose+0x33f/0x33f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.839747] [] ? rw_copy_check_uvector+0x52/0xfa
    Apr 10 07:08:08 server kernel: [255806.839749] [] ? __kmalloc+0x104/0x116
    Apr 10 07:08:08 server kernel: [255806.839752] [] ? copy_from_user+0x18/0x30
    Apr 10 07:08:08 server kernel: [255806.839755] [] ? security_file_permission+0x18/0x2d
    Apr 10 07:08:08 server kernel: [255806.839758] [] ? do_readv_writev+0xaf/0x125
    Apr 10 07:08:08 server kernel: [255806.839763] [] ? nfsd_setuser+0x199/0x1de [nfsd]
    Apr 10 07:08:08 server kernel: [255806.839768] [] ? nfsd_setuser_and_check_port+0x76/0x92 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.839772] [] ? nfsd_vfs_write+0xc9/0x2b5 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.839777] [] ? nfsd_open+0xfd/0x14f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.839782] [] ? nfsd_write+0xbb/0xdc [nfsd]
    Apr 10 07:08:08 server kernel: [255806.839787] [] ? nfsd3_proc_write+0xcf/0xea [nfsd]
    Apr 10 07:08:08 server kernel: [255806.839791] [] ? nfsd_dispatch+0xd9/0x1bb [nfsd]
    Apr 10 07:08:08 server kernel: [255806.839798] [] ? svc_process_common+0x2d0/0x4ca [sunrpc]
    Apr 10 07:08:08 server kernel: [255806.839802] [] ? try_to_wake_up+0x191/0x191
    Apr 10 07:08:08 server kernel: [255806.839806] [] ? nfsd_svc+0x17f/0x17f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.839811] [] ? svc_process+0x10d/0x12c [sunrpc]
    Apr 10 07:08:08 server kernel: [255806.839815] [] ? nfsd+0xe3/0x127 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.839818] [] ? kthread+0x7a/0x82
    Apr 10 07:08:08 server kernel: [255806.839821] [] ? kernel_thread_helper+0x4/0x10
    Apr 10 07:08:08 server kernel: [255806.839824] [] ? kthread_worker_fn+0x147/0x147
    Apr 10 07:08:08 server kernel: [255806.839827] [] ? gs_change+0x13/0x13
    Apr 10 07:08:08 server kernel: [255806.839829] INFO: task nfsd:2618 blocked for more than 120 seconds.
    Apr 10 07:08:08 server kernel: [255806.846966] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
    Apr 10 07:08:08 server kernel: [255806.855930] nfsd D ffff88045a544af0 0 2618 2 0x00000000
    Apr 10 07:08:08 server kernel: [255806.855935] ffff88045a544af0 0000000000000046 0000000000000000 ffffffff8160d020
    Apr 10 07:08:08 server kernel: [255806.855938] 0000000000013740 ffff88046dc4bfd8 ffff88046dc4bfd8 0000000000013740
    Apr 10 07:08:08 server kernel: [255806.855941] ffff88045a544af0 0000000000013740 0000000000013740 ffff88046dc4a010
    Apr 10 07:08:08 server kernel: [255806.855945] Call Trace:
    Apr 10 07:08:08 server kernel: [255806.855951] [] ? __mutex_lock_common+0x10c/0x172
    Apr 10 07:08:08 server kernel: [255806.855953] [] ? mutex_lock+0x1a/0x2c
    Apr 10 07:08:08 server kernel: [255806.855957] [] ? sch_direct_xmit+0x8a/0x138
    Apr 10 07:08:08 server kernel: [255806.855968] [] ? xfs_rw_ilock+0x1e/0x2e [xfs]
    Apr 10 07:08:08 server kernel: [255806.855975] [] ? xfs_file_buffered_aio_write+0x5c/0x15d [xfs]
    Apr 10 07:08:08 server kernel: [255806.855985] [] ? dev_queue_xmit+0x4eb/0x4fd
    Apr 10 07:08:08 server kernel: [255806.855992] [] ? xfs_file_aio_write+0x18b/0x226 [xfs]
    Apr 10 07:08:08 server kernel: [255806.856000] [] ? xfs_iget+0x50d/0x582 [xfs]
    Apr 10 07:08:08 server kernel: [255806.856007] [] ? xfs_file_fsync+0x234/0x234 [xfs]
    Apr 10 07:08:08 server kernel: [255806.856011] [] ? do_sync_readv_writev+0xa3/0xde
    Apr 10 07:08:08 server kernel: [255806.856017] [] ? fh_compose+0x33f/0x33f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.856020] [] ? rw_copy_check_uvector+0x52/0xfa
    Apr 10 07:08:08 server kernel: [255806.856023] [] ? __kmalloc+0x104/0x116
    Apr 10 07:08:08 server kernel: [255806.856025] [] ? copy_from_user+0x18/0x30
    Apr 10 07:08:08 server kernel: [255806.856029] [] ? security_file_permission+0x18/0x2d
    Apr 10 07:08:08 server kernel: [255806.856032] [] ? do_readv_writev+0xaf/0x125
    Apr 10 07:08:08 server kernel: [255806.856037] [] ? nfsd_setuser+0x199/0x1de [nfsd]
    Apr 10 07:08:08 server kernel: [255806.856041] [] ? nfsd_setuser_and_check_port+0x76/0x92 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.856046] [] ? nfsd_vfs_write+0xc9/0x2b5 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.856051] [] ? nfsd_open+0xfd/0x14f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.856056] [] ? nfsd_write+0xbb/0xdc [nfsd]
    Apr 10 07:08:08 server kernel: [255806.856061] [] ? nfsd3_proc_write+0xcf/0xea [nfsd]
    Apr 10 07:08:08 server kernel: [255806.856066] [] ? nfsd_dispatch+0xd9/0x1bb [nfsd]
    Apr 10 07:08:08 server kernel: [255806.856072] [] ? svc_process_common+0x2d0/0x4ca [sunrpc]
    Apr 10 07:08:08 server kernel: [255806.856076] [] ? try_to_wake_up+0x191/0x191
    Apr 10 07:08:08 server kernel: [255806.856081] [] ? nfsd_svc+0x17f/0x17f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.856085] [] ? svc_process+0x10d/0x12c [sunrpc]
    Apr 10 07:08:08 server kernel: [255806.856090] [] ? nfsd+0xe3/0x127 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.856093] [] ? kthread+0x7a/0x82
    Apr 10 07:08:08 server kernel: [255806.856096] [] ? kernel_thread_helper+0x4/0x10
    Apr 10 07:08:08 server kernel: [255806.856099] [] ? kthread_worker_fn+0x147/0x147
    Apr 10 07:08:08 server kernel: [255806.856102] [] ? gs_change+0x13/0x13
    Apr 10 07:08:08 server kernel: [255806.856104] INFO: task nfsd:2619 blocked for more than 120 seconds.
    Apr 10 07:08:08 server kernel: [255806.863233] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
    Apr 10 07:08:08 server kernel: [255806.872185] nfsd D ffff88046d47a9b0 0 2619 2 0x00000000
    Apr 10 07:08:08 server kernel: [255806.872190] ffff88046d47a9b0 0000000000000046 01508c1600000000 ffff88047da62f60
    Apr 10 07:08:08 server kernel: [255806.872193] 0000000000013740 ffff88046ddbffd8 ffff88046ddbffd8 0000000000013740
    Apr 10 07:08:08 server kernel: [255806.872196] ffff88046d47a9b0 0000000000013740 0000000000013740 ffff88046ddbe010
    Apr 10 07:08:08 server kernel: [255806.872200] Call Trace:
    Apr 10 07:08:08 server kernel: [255806.872206] [] ? __mutex_lock_common+0x10c/0x172
    Apr 10 07:08:08 server kernel: [255806.872208] [] ? mutex_lock+0x1a/0x2c
    Apr 10 07:08:08 server kernel: [255806.872220] [] ? xfs_rw_ilock+0x1e/0x2e [xfs]
    Apr 10 07:08:08 server kernel: [255806.872227] [] ? xfs_file_buffered_aio_write+0x5c/0x15d [xfs]
    Apr 10 07:08:08 server kernel: [255806.872231] [] ? tcp_rcv_established+0x565/0x69e
    Apr 10 07:08:08 server kernel: [255806.872239] [] ? xfs_file_aio_write+0x18b/0x226 [xfs]
    Apr 10 07:08:08 server kernel: [255806.872246] [] ? xfs_iget+0x50d/0x582 [xfs]
    Apr 10 07:08:08 server kernel: [255806.872254] [] ? xfs_file_fsync+0x234/0x234 [xfs]
    Apr 10 07:08:08 server kernel: [255806.872257] [] ? do_sync_readv_writev+0xa3/0xde
    Apr 10 07:08:08 server kernel: [255806.872263] [] ? fh_compose+0x33f/0x33f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.872266] [] ? rw_copy_check_uvector+0x52/0xfa
    Apr 10 07:08:08 server kernel: [255806.872268] [] ? __kmalloc+0x104/0x116
    Apr 10 07:08:08 server kernel: [255806.872271] [] ? copy_from_user+0x18/0x30
    Apr 10 07:08:08 server kernel: [255806.872274] [] ? security_file_permission+0x18/0x2d
    Apr 10 07:08:08 server kernel: [255806.872277] [] ? do_readv_writev+0xaf/0x125
    Apr 10 07:08:08 server kernel: [255806.872282] [] ? nfsd_setuser+0x199/0x1de [nfsd]
    Apr 10 07:08:08 server kernel: [255806.872286] [] ? nfsd_setuser_and_check_port+0x76/0x92 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.872291] [] ? nfsd_vfs_write+0xc9/0x2b5 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.872296] [] ? nfsd_open+0xfd/0x14f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.872300] [] ? nfsd_write+0xbb/0xdc [nfsd]
    Apr 10 07:08:08 server kernel: [255806.872305] [] ? nfsd3_proc_write+0xcf/0xea [nfsd]
    Apr 10 07:08:08 server kernel: [255806.872310] [] ? nfsd_dispatch+0xd9/0x1bb [nfsd]
    Apr 10 07:08:08 server kernel: [255806.872316] [] ? svc_process_common+0x2d0/0x4ca [sunrpc]
    Apr 10 07:08:08 server kernel: [255806.872320] [] ? try_to_wake_up+0x191/0x191
    Apr 10 07:08:08 server kernel: [255806.872324] [] ? nfsd_svc+0x17f/0x17f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.872329] [] ? svc_process+0x10d/0x12c [sunrpc]
    Apr 10 07:08:08 server kernel: [255806.872333] [] ? nfsd+0xe3/0x127 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.872337] [] ? kthread+0x7a/0x82
    Apr 10 07:08:08 server kernel: [255806.872340] [] ? kernel_thread_helper+0x4/0x10
    Apr 10 07:08:08 server kernel: [255806.872343] [] ? kthread_worker_fn+0x147/0x147
    Apr 10 07:08:08 server kernel: [255806.872345] [] ? gs_change+0x13/0x13
    Apr 10 07:08:08 server kernel: [255806.872347] INFO: task nfsd:2620 blocked for more than 120 seconds.
    Apr 10 07:08:08 server kernel: [255806.879476] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
    Apr 10 07:08:08 server kernel: [255806.888427] nfsd D ffff88046d47a2c0 0 2620 2 0x00000000
    Apr 10 07:08:08 server kernel: [255806.888431] ffff88046d47a2c0 0000000000000046 07026ee600000000 ffff88047da55610
    Apr 10 07:08:08 server kernel: [255806.888435] 0000000000013740 ffff88046d3f3fd8 ffff88046d3f3fd8 0000000000013740
    Apr 10 07:08:08 server kernel: [255806.888438] ffff88046d47a2c0 0000000000013740 0000000000013740 ffff88046d3f2010
    Apr 10 07:08:08 server kernel: [255806.888441] Call Trace:
    Apr 10 07:08:08 server kernel: [255806.888447] [] ? __mutex_lock_common+0x10c/0x172
    Apr 10 07:08:08 server kernel: [255806.888450] [] ? mutex_lock+0x1a/0x2c
    Apr 10 07:08:08 server kernel: [255806.888457] [] ? ipoib_start_xmit+0x683/0x710 [ib_ipoib]
    Apr 10 07:08:08 server kernel: [255806.888467] [] ? xfs_rw_ilock+0x1e/0x2e [xfs]
    Apr 10 07:08:08 server kernel: [255806.888475] [] ? xfs_file_buffered_aio_write+0x5c/0x15d [xfs]
    Apr 10 07:08:08 server kernel: [255806.888482] [] ? xfs_file_aio_write+0x18b/0x226 [xfs]
    Apr 10 07:08:08 server kernel: [255806.888490] [] ? xfs_iget+0x50d/0x582 [xfs]
    Apr 10 07:08:08 server kernel: [255806.888497] [] ? xfs_file_fsync+0x234/0x234 [xfs]
    Apr 10 07:08:08 server kernel: [255806.888501] [] ? do_sync_readv_writev+0xa3/0xde
    Apr 10 07:08:08 server kernel: [255806.888507] [] ? fh_compose+0x33f/0x33f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.888509] [] ? rw_copy_check_uvector+0x52/0xfa
    Apr 10 07:08:08 server kernel: [255806.888512] [] ? __kmalloc+0x104/0x116
    Apr 10 07:08:08 server kernel: [255806.888514] [] ? copy_from_user+0x18/0x30
    Apr 10 07:08:08 server kernel: [255806.888518] [] ? security_file_permission+0x18/0x2d
    Apr 10 07:08:08 server kernel: [255806.888521] [] ? do_readv_writev+0xaf/0x125
    Apr 10 07:08:08 server kernel: [255806.888526] [] ? nfsd_setuser+0x199/0x1de [nfsd]
    Apr 10 07:08:08 server kernel: [255806.888530] [] ? nfsd_setuser_and_check_port+0x76/0x92 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.888535] [] ? nfsd_vfs_write+0xc9/0x2b5 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.888540] [] ? nfsd_open+0xfd/0x14f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.888544] [] ? nfsd_write+0xbb/0xdc [nfsd]
    Apr 10 07:08:08 server kernel: [255806.888549] [] ? nfsd3_proc_write+0xcf/0xea [nfsd]
    Apr 10 07:08:08 server kernel: [255806.888554] [] ? nfsd_dispatch+0xd9/0x1bb [nfsd]
    Apr 10 07:08:08 server kernel: [255806.888560] [] ? svc_process_common+0x2d0/0x4ca [sunrpc]
    Apr 10 07:08:08 server kernel: [255806.888564] [] ? try_to_wake_up+0x191/0x191
    Apr 10 07:08:08 server kernel: [255806.888568] [] ? nfsd_svc+0x17f/0x17f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.888573] [] ? svc_process+0x10d/0x12c [sunrpc]
    Apr 10 07:08:08 server kernel: [255806.888577] [] ? nfsd+0xe3/0x127 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.888580] [] ? kthread+0x7a/0x82
    Apr 10 07:08:08 server kernel: [255806.888583] [] ? kernel_thread_helper+0x4/0x10
    Apr 10 07:08:08 server kernel: [255806.888586] [] ? kthread_worker_fn+0x147/0x147
    Apr 10 07:08:08 server kernel: [255806.888589] [] ? gs_change+0x13/0x13
    Apr 10 07:08:08 server kernel: [255806.888591] INFO: task nfsd:2622 blocked for more than 120 seconds.
    Apr 10 07:08:08 server kernel: [255806.895721] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
    Apr 10 07:08:08 server kernel: [255806.904667] nfsd D ffff88046d722140 0 2622 2 0x00000000
    Apr 10 07:08:08 server kernel: [255806.904672] ffff88046d722140 0000000000000046 0000000000000000 ffffffff8160d020
    Apr 10 07:08:08 server kernel: [255806.904675] 0000000000013740 ffff88045a7abfd8 ffff88045a7abfd8 0000000000013740
    Apr 10 07:08:08 server kernel: [255806.904678] ffff88046d722140 0000000000013740 0000000000013740 ffff88045a7aa010
    Apr 10 07:08:08 server kernel: [255806.904681] Call Trace:
    Apr 10 07:08:08 server kernel: [255806.904687] [] ? __mutex_lock_common+0x10c/0x172
    Apr 10 07:08:08 server kernel: [255806.904690] [] ? mutex_lock+0x1a/0x2c
    Apr 10 07:08:08 server kernel: [255806.904701] [] ? xfs_rw_ilock+0x1e/0x2e [xfs]
    Apr 10 07:08:08 server kernel: [255806.904708] [] ? xfs_file_buffered_aio_write+0x5c/0x15d [xfs]
    Apr 10 07:08:08 server kernel: [255806.904713] [] ? tcp_rcv_established+0x565/0x69e
    Apr 10 07:08:08 server kernel: [255806.904720] [] ? xfs_file_aio_write+0x18b/0x226 [xfs]
    Apr 10 07:08:08 server kernel: [255806.904728] [] ? xfs_iget+0x50d/0x582 [xfs]
    Apr 10 07:08:08 server kernel: [255806.904734] [] ? fh_compose+0x33f/0x33f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.904742] [] ? xfs_file_fsync+0x234/0x234 [xfs]
    Apr 10 07:08:08 server kernel: [255806.904745] [] ? do_sync_readv_writev+0xa3/0xde
    Apr 10 07:08:08 server kernel: [255806.904749] [] ? fh_compose+0x33f/0x33f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.904752] [] ? rw_copy_check_uvector+0x52/0xfa
    Apr 10 07:08:08 server kernel: [255806.904755] [] ? __kmalloc+0x104/0x116
    Apr 10 07:08:08 server kernel: [255806.904757] [] ? copy_from_user+0x18/0x30
    Apr 10 07:08:08 server kernel: [255806.904760] [] ? security_file_permission+0x18/0x2d
    Apr 10 07:08:08 server kernel: [255806.904763] [] ? do_readv_writev+0xaf/0x125
    Apr 10 07:08:08 server kernel: [255806.904768] [] ? nfsd_setuser+0x199/0x1de [nfsd]
    Apr 10 07:08:08 server kernel: [255806.904772] [] ? nfsd_setuser_and_check_port+0x76/0x92 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.904777] [] ? nfsd_vfs_write+0xc9/0x2b5 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.904782] [] ? nfsd_open+0xfd/0x14f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.904786] [] ? nfsd_write+0xbb/0xdc [nfsd]
    Apr 10 07:08:08 server kernel: [255806.904791] [] ? nfsd3_proc_write+0xcf/0xea [nfsd]
    Apr 10 07:08:08 server kernel: [255806.904796] [] ? nfsd_dispatch+0xd9/0x1bb [nfsd]
    Apr 10 07:08:08 server kernel: [255806.904802] [] ? svc_process_common+0x2d0/0x4ca [sunrpc]
    Apr 10 07:08:08 server kernel: [255806.904806] [] ? try_to_wake_up+0x191/0x191
    Apr 10 07:08:08 server kernel: [255806.904810] [] ? nfsd_svc+0x17f/0x17f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.904814] [] ? svc_process+0x10d/0x12c [sunrpc]
    Apr 10 07:08:08 server kernel: [255806.904819] [] ? nfsd+0xe3/0x127 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.904822] [] ? kthread+0x7a/0x82
    Apr 10 07:08:08 server kernel: [255806.904825] [] ? kernel_thread_helper+0x4/0x10
    Apr 10 07:08:08 server kernel: [255806.904828] [] ? kthread_worker_fn+0x147/0x147
    Apr 10 07:08:08 server kernel: [255806.904830] [] ? gs_change+0x13/0x13
    Apr 10 07:08:08 server kernel: [255806.904832] INFO: task nfsd:2623 blocked for more than 120 seconds.
    Apr 10 07:08:08 server kernel: [255806.911962] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
    Apr 10 07:08:08 server kernel: [255806.920912] nfsd D ffff88046d722830 0 2623 2 0x00000000
    Apr 10 07:08:08 server kernel: [255806.920917] ffff88046d722830 0000000000000046 01508b4a00000000 ffffffff8160d020
    Apr 10 07:08:08 server kernel: [255806.920920] 0000000000013740 ffff8804594c5fd8 ffff8804594c5fd8 0000000000013740
    Apr 10 07:08:08 server kernel: [255806.920923] ffff88046d722830 0000000000013740 0000000000013740 ffff8804594c4010
    Apr 10 07:08:08 server kernel: [255806.920926] Call Trace:
    Apr 10 07:08:08 server kernel: [255806.920933] [] ? __mutex_lock_common+0x10c/0x172
    Apr 10 07:08:08 server kernel: [255806.920935] [] ? mutex_lock+0x1a/0x2c
    Apr 10 07:08:08 server kernel: [255806.920942] [] ? ipoib_start_xmit+0x683/0x710 [ib_ipoib]
    Apr 10 07:08:08 server kernel: [255806.920953] [] ? xfs_rw_ilock+0x1e/0x2e [xfs]
    Apr 10 07:08:08 server kernel: [255806.920960] [] ? xfs_file_buffered_aio_write+0x5c/0x15d [xfs]
    Apr 10 07:08:08 server kernel: [255806.920967] [] ? xfs_file_aio_write+0x18b/0x226 [xfs]
    Apr 10 07:08:08 server kernel: [255806.920975] [] ? xfs_iget+0x50d/0x582 [xfs]
    Apr 10 07:08:08 server kernel: [255806.920982] [] ? xfs_file_fsync+0x234/0x234 [xfs]
    Apr 10 07:08:08 server kernel: [255806.920986] [] ? do_sync_readv_writev+0xa3/0xde
    Apr 10 07:08:08 server kernel: [255806.920992] [] ? fh_compose+0x33f/0x33f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.920994] [] ? rw_copy_check_uvector+0x52/0xfa
    Apr 10 07:08:08 server kernel: [255806.920997] [] ? __kmalloc+0x104/0x116
    Apr 10 07:08:08 server kernel: [255806.920999] [] ? copy_from_user+0x18/0x30
    Apr 10 07:08:08 server kernel: [255806.921004] [] ? security_file_permission+0x18/0x2d
    Apr 10 07:08:08 server kernel: [255806.921006] [] ? do_readv_writev+0xaf/0x125
    Apr 10 07:08:08 server kernel: [255806.921011] [] ? nfsd_setuser+0x199/0x1de [nfsd]
    Apr 10 07:08:08 server kernel: [255806.921016] [] ? nfsd_setuser_and_check_port+0x76/0x92 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.921021] [] ? nfsd_vfs_write+0xc9/0x2b5 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.921025] [] ? nfsd_open+0xfd/0x14f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.921030] [] ? nfsd_write+0xbb/0xdc [nfsd]
    Apr 10 07:08:08 server kernel: [255806.921035] [] ? nfsd3_proc_write+0xcf/0xea [nfsd]
    Apr 10 07:08:08 server kernel: [255806.921039] [] ? nfsd_dispatch+0xd9/0x1bb [nfsd]
    Apr 10 07:08:08 server kernel: [255806.921046] [] ? svc_process_common+0x2d0/0x4ca [sunrpc]
    Apr 10 07:08:08 server kernel: [255806.921049] [] ? try_to_wake_up+0x191/0x191
    Apr 10 07:08:08 server kernel: [255806.921054] [] ? nfsd_svc+0x17f/0x17f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.921058] [] ? svc_process+0x10d/0x12c [sunrpc]
    Apr 10 07:08:08 server kernel: [255806.921062] [] ? nfsd+0xe3/0x127 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.921066] [] ? kthread+0x7a/0x82
    Apr 10 07:08:08 server kernel: [255806.921069] [] ? kernel_thread_helper+0x4/0x10
    Apr 10 07:08:08 server kernel: [255806.921071] [] ? kthread_worker_fn+0x147/0x147
    Apr 10 07:08:08 server kernel: [255806.921074] [] ? gs_change+0x13/0x13
    Apr 10 07:08:08 server kernel: [255806.921076] INFO: task nfsd:2625 blocked for more than 120 seconds.
    Apr 10 07:08:08 server kernel: [255806.928205] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
    Apr 10 07:08:08 server kernel: [255806.937141] nfsd D ffff88046da82180 0 2625 2 0x00000000
    Apr 10 07:08:08 server kernel: [255806.937145] ffff88046da82180 0000000000000046 ffff8804628ac000 ffff88047da62f60
    Apr 10 07:08:08 server kernel: [255806.937148] 0000000000013740 ffff8804597bdfd8 ffff8804597bdfd8 0000000000013740
    Apr 10 07:08:08 server kernel: [255806.937151] ffff88046da82180 0000000000013740 0000000000013740 ffff8804597bc010
    Apr 10 07:08:08 server kernel: [255806.937154] Call Trace:
    Apr 10 07:08:08 server kernel: [255806.937161] [] ? __mutex_lock_common+0x10c/0x172
    Apr 10 07:08:08 server kernel: [255806.937164] [] ? mutex_lock+0x1a/0x2c
    Apr 10 07:08:08 server kernel: [255806.937175] [] ? xfs_rw_ilock+0x1e/0x2e [xfs]
    Apr 10 07:08:08 server kernel: [255806.937182] [] ? xfs_file_buffered_aio_write+0x5c/0x15d [xfs]
    Apr 10 07:08:08 server kernel: [255806.937187] [] ? tcp_rcv_established+0x565/0x69e
    Apr 10 07:08:08 server kernel: [255806.937194] [] ? xfs_file_aio_write+0x18b/0x226 [xfs]
    Apr 10 07:08:08 server kernel: [255806.937202] [] ? xfs_iget+0x50d/0x582 [xfs]
    Apr 10 07:08:08 server kernel: [255806.937209] [] ? xfs_file_fsync+0x234/0x234 [xfs]
    Apr 10 07:08:08 server kernel: [255806.937213] [] ? do_sync_readv_writev+0xa3/0xde
    Apr 10 07:08:08 server kernel: [255806.937219] [] ? fh_compose+0x33f/0x33f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.937222] [] ? rw_copy_check_uvector+0x52/0xfa
    Apr 10 07:08:08 server kernel: [255806.937224] [] ? __kmalloc+0x104/0x116
    Apr 10 07:08:08 server kernel: [255806.937226] [] ? copy_from_user+0x18/0x30
    Apr 10 07:08:08 server kernel: [255806.937230] [] ? security_file_permission+0x18/0x2d
    Apr 10 07:08:08 server kernel: [255806.937233] [] ? do_readv_writev+0xaf/0x125
    Apr 10 07:08:08 server kernel: [255806.937237] [] ? nfsd_setuser+0x199/0x1de [nfsd]
    Apr 10 07:08:08 server kernel: [255806.937242] [] ? nfsd_setuser_and_check_port+0x76/0x92 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.937247] [] ? nfsd_vfs_write+0xc9/0x2b5 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.937251] [] ? nfsd_open+0xfd/0x14f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.937256] [] ? nfsd_write+0xbb/0xdc [nfsd]
    Apr 10 07:08:08 server kernel: [255806.937261] [] ? nfsd3_proc_write+0xcf/0xea [nfsd]
    Apr 10 07:08:08 server kernel: [255806.937265] [] ? nfsd_dispatch+0xd9/0x1bb [nfsd]
    Apr 10 07:08:08 server kernel: [255806.937272] [] ? svc_process_common+0x2d0/0x4ca [sunrpc]
    Apr 10 07:08:08 server kernel: [255806.937276] [] ? try_to_wake_up+0x191/0x191
    Apr 10 07:08:08 server kernel: [255806.937280] [] ? nfsd_svc+0x17f/0x17f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.937284] [] ? svc_process+0x10d/0x12c [sunrpc]
    Apr 10 07:08:08 server kernel: [255806.937288] [] ? nfsd+0xe3/0x127 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.937292] [] ? kthread+0x7a/0x82
    Apr 10 07:08:08 server kernel: [255806.937295] [] ? kernel_thread_helper+0x4/0x10
    Apr 10 07:08:08 server kernel: [255806.937298] [] ? kthread_worker_fn+0x147/0x147
    Apr 10 07:08:08 server kernel: [255806.937300] [] ? gs_change+0x13/0x13
    Apr 10 07:08:08 server kernel: [255806.937302] INFO: task nfsd:2626 blocked for more than 120 seconds.
    Apr 10 07:08:08 server kernel: [255806.944433] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
    Apr 10 07:08:08 server kernel: [255806.953371] nfsd D ffff88046da83650 0 2626 2 0x00000000
    Apr 10 07:08:08 server kernel: [255806.953375] ffff88046da83650 0000000000000046 014f950400000000 ffffffff8160d020
    Apr 10 07:08:08 server kernel: [255806.953379] 0000000000013740 ffff8804548d7fd8 ffff8804548d7fd8 0000000000013740
    Apr 10 07:08:08 server kernel: [255806.953382] ffff88046da83650 0000000000013740 0000000000013740 ffff8804548d6010
    Apr 10 07:08:08 server kernel: [255806.953385] Call Trace:
    Apr 10 07:08:08 server kernel: [255806.953392] [] ? __mutex_lock_common+0x10c/0x172
    Apr 10 07:08:08 server kernel: [255806.953395] [] ? mutex_lock+0x1a/0x2c
    Apr 10 07:08:08 server kernel: [255806.953401] [] ? ipoib_start_xmit+0x683/0x710 [ib_ipoib]
    Apr 10 07:08:08 server kernel: [255806.953412] [] ? xfs_rw_ilock+0x1e/0x2e [xfs]
    Apr 10 07:08:08 server kernel: [255806.953419] [] ? xfs_file_buffered_aio_write+0x5c/0x15d [xfs]
    Apr 10 07:08:08 server kernel: [255806.953426] [] ? xfs_file_aio_write+0x18b/0x226 [xfs]
    Apr 10 07:08:08 server kernel: [255806.953434] [] ? xfs_iget+0x50d/0x582 [xfs]
    Apr 10 07:08:08 server kernel: [255806.953441] [] ? xfs_file_fsync+0x234/0x234 [xfs]
    Apr 10 07:08:08 server kernel: [255806.953445] [] ? do_sync_readv_writev+0xa3/0xde
    Apr 10 07:08:08 server kernel: [255806.953450] [] ? fh_compose+0x33f/0x33f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.953453] [] ? rw_copy_check_uvector+0x52/0xfa
    Apr 10 07:08:08 server kernel: [255806.953456] [] ? __kmalloc+0x104/0x116
    Apr 10 07:08:08 server kernel: [255806.953458] [] ? copy_from_user+0x18/0x30
    Apr 10 07:08:08 server kernel: [255806.953462] [] ? security_file_permission+0x18/0x2d
    Apr 10 07:08:08 server kernel: [255806.953465] [] ? do_readv_writev+0xaf/0x125
    Apr 10 07:08:08 server kernel: [255806.953470] [] ? nfsd_setuser+0x199/0x1de [nfsd]
    Apr 10 07:08:08 server kernel: [255806.953474] [] ? nfsd_setuser_and_check_port+0x76/0x92 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.953479] [] ? nfsd_vfs_write+0xc9/0x2b5 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.953484] [] ? nfsd_open+0xfd/0x14f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.953488] [] ? nfsd_write+0xbb/0xdc [nfsd]
    Apr 10 07:08:08 server kernel: [255806.953493] [] ? nfsd3_proc_write+0xcf/0xea [nfsd]
    Apr 10 07:08:08 server kernel: [255806.953498] [] ? nfsd_dispatch+0xd9/0x1bb [nfsd]
    Apr 10 07:08:08 server kernel: [255806.953504] [] ? svc_process_common+0x2d0/0x4ca [sunrpc]
    Apr 10 07:08:08 server kernel: [255806.953507] [] ? try_to_wake_up+0x191/0x191
    Apr 10 07:08:08 server kernel: [255806.953511] [] ? nfsd_svc+0x17f/0x17f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.953516] [] ? svc_process+0x10d/0x12c [sunrpc]
    Apr 10 07:08:08 server kernel: [255806.953520] [] ? nfsd+0xe3/0x127 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.953523] [] ? kthread+0x7a/0x82
    Apr 10 07:08:08 server kernel: [255806.953527] [] ? kernel_thread_helper+0x4/0x10
    Apr 10 07:08:08 server kernel: [255806.953529] [] ? kthread_worker_fn+0x147/0x147
    Apr 10 07:08:08 server kernel: [255806.953532] [] ? gs_change+0x13/0x13
    Apr 10 07:08:08 server kernel: [255806.953534] INFO: task nfsd:2627 blocked for more than 120 seconds.
    Apr 10 07:08:08 server kernel: [255806.960668] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
    Apr 10 07:08:08 server kernel: [255806.969608] nfsd D ffff88046c1a0ea0 0 2627 2 0x00000000
    Apr 10 07:08:08 server kernel: [255806.969612] ffff88046c1a0ea0 0000000000000046 01508b5000000000 ffff88047da62f60
    Apr 10 07:08:08 server kernel: [255806.969616] 0000000000013740 ffff8804549d3fd8 ffff8804549d3fd8 0000000000013740
    Apr 10 07:08:08 server kernel: [255806.969619] ffff88046c1a0ea0 0000000000013740 0000000000013740 ffff8804549d2010
    Apr 10 07:08:08 server kernel: [255806.969622] Call Trace:
    Apr 10 07:08:08 server kernel: [255806.969628] [] ? __mutex_lock_common+0x10c/0x172
    Apr 10 07:08:08 server kernel: [255806.969631] [] ? mutex_lock+0x1a/0x2c
    Apr 10 07:08:08 server kernel: [255806.969637] [] ? ipoib_start_xmit+0x683/0x710 [ib_ipoib]
    Apr 10 07:08:08 server kernel: [255806.969648] [] ? xfs_rw_ilock+0x1e/0x2e [xfs]
    Apr 10 07:08:08 server kernel: [255806.969655] [] ? xfs_file_buffered_aio_write+0x5c/0x15d [xfs]
    Apr 10 07:08:08 server kernel: [255806.969663] [] ? xfs_file_aio_write+0x18b/0x226 [xfs]
    Apr 10 07:08:08 server kernel: [255806.969670] [] ? xfs_iget+0x50d/0x582 [xfs]
    Apr 10 07:08:08 server kernel: [255806.969678] [] ? xfs_file_fsync+0x234/0x234 [xfs]
    Apr 10 07:08:08 server kernel: [255806.969682] [] ? do_sync_readv_writev+0xa3/0xde
    Apr 10 07:08:08 server kernel: [255806.969687] [] ? fh_compose+0x33f/0x33f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.969690] [] ? rw_copy_check_uvector+0x52/0xfa
    Apr 10 07:08:08 server kernel: [255806.969693] [] ? __kmalloc+0x104/0x116
    Apr 10 07:08:08 server kernel: [255806.969695] [] ? copy_from_user+0x18/0x30
    Apr 10 07:08:08 server kernel: [255806.969699] [] ? security_file_permission+0x18/0x2d
    Apr 10 07:08:08 server kernel: [255806.969701] [] ? do_readv_writev+0xaf/0x125
    Apr 10 07:08:08 server kernel: [255806.969706] [] ? nfsd_setuser+0x199/0x1de [nfsd]
    Apr 10 07:08:08 server kernel: [255806.969710] [] ? nfsd_setuser_and_check_port+0x76/0x92 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.969715] [] ? nfsd_vfs_write+0xc9/0x2b5 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.969720] [] ? nfsd_open+0xfd/0x14f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.969724] [] ? nfsd_write+0xbb/0xdc [nfsd]
    Apr 10 07:08:08 server kernel: [255806.969729] [] ? nfsd3_proc_write+0xcf/0xea [nfsd]
    Apr 10 07:08:08 server kernel: [255806.969734] [] ? nfsd_dispatch+0xd9/0x1bb [nfsd]
    Apr 10 07:08:08 server kernel: [255806.969739] [] ? svc_process_common+0x2d0/0x4ca [sunrpc]
    Apr 10 07:08:08 server kernel: [255806.969743] [] ? try_to_wake_up+0x191/0x191
    Apr 10 07:08:08 server kernel: [255806.969747] [] ? nfsd_svc+0x17f/0x17f [nfsd]
    Apr 10 07:08:08 server kernel: [255806.969752] [] ? svc_process+0x10d/0x12c [sunrpc]
    Apr 10 07:08:08 server kernel: [255806.969756] [] ? nfsd+0xe3/0x127 [nfsd]
    Apr 10 07:08:08 server kernel: [255806.969759] [] ? kthread+0x7a/0x82
    Apr 10 07:08:08 server kernel: [255806.969762] [] ? kernel_thread_helper+0x4/0x10
    Apr 10 07:08:08 server kernel: [255806.969765] [] ? kthread_worker_fn+0x147/0x147
    Apr 10 07:08:08 server kernel: [255806.969768] [] ? gs_change+0x13/0x13

Suivre le flux des commentaires

Note : les commentaires appartiennent à celles et ceux qui les ont postés. Nous n’en sommes pas responsables.