i'm running some code and whilst it hasn't at any point exited or given my a visible error message, it seems to be stalling for large periods of time, and then picks up again, and then stalls, so i ran dmesg and got the following:
mount: server appcs.ra.phy.cam.ac.uk not responding, timed out
TempoNest[951]: segfault at 7fffea89cd50 ip 0000000000417a04 sp 00007fffea89cd50 error 6 in TempoNest[400000+a9000]
TempoNest[827]: segfault at 41 ip 000000000040ee2e sp 00007fff2b55fff0 error 4 in TempoNest[400000+a9000]
TempoNest[850] general protection ip:40ee2e sp:7fffc92e9910 error:0 in TempoNest[400000+a9000]
TempoNest[872]: segfault at 41 ip 000000000040ee2e sp 00007fff7c34fcf0 error 4 in TempoNest[400000+a9000]
INFO: task TempoNest:6147 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
TempoNest D 0000000000000001 0 6147 5924 0x00000080
ffff88010de03c78 0000000000000086 0000000000000000 ffffffffa0799e20
ffff88010de03be8 ffffffff81012a69 ffff88010de03c28 ffffffff8109b669
ffff8802496e45f8 ffff88010de03fd8 000000000000fb88 ffff8802496e45f8
Call Trace:
[<ffffffff81012a69>] ? read_tsc+0x9/0x20
[<ffffffff8109b669>] ? ktime_get_ts+0xa9/0xe0
[<ffffffff8109b669>] ? ktime_get_ts+0xa9/0xe0
[<ffffffff8110ffe0>] ? sync_page+0x0/0x50
[<ffffffff814ea7a3>] io_schedule+0x73/0xc0
[<ffffffff8111001d>] sync_page+0x3d/0x50
[<ffffffff814eb15f>] __wait_on_bit+0x5f/0x90
[<ffffffff81110253>] wait_on_page_bit+0x73/0x80
[<ffffffff81090c20>] ? wake_bit_function+0x0/0x50
[<ffffffff81126665>] ? pagevec_lookup_tag+0x25/0x40
[<ffffffff811106cb>] wait_on_page_writeback_range+0xfb/0x190
[<ffffffff81110898>] filemap_write_and_wait_range+0x78/0x90
[<ffffffff811a50be>] vfs_fsync_range+0x7e/0xe0
[<ffffffff811a518d>] vfs_fsync+0x1d/0x20
[<ffffffffa075f590>] nfs_file_flush+0x70/0xa0 [nfs]
[<ffffffff81173abc>] filp_close+0x3c/0x90
[<ffffffff81173bb5>] sys_close+0xa5/0x100
[<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
TempoNest[1582]: segfault at 7ffff058d1f0 ip 000000000042b54d sp 00007ffff058d1f0 error 6 in TempoNest[400000+ca000]
TempoNest[21147]: segfault at 0 ip 0000000000411b5a sp 00007fff61aa2f90 error 4 in TempoNest[400000+ca000]
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
TempoNest[6648] general protection ip:370f29d54d sp:7fff32979770 error:0 in libstdc++.so.6.0.13[370f200000+e8000]
TempoNest[6669] general protection ip:370f29d54d sp:7fff7bd3c8b0 error:0 in libstdc++.so.6.0.13[370f200000+e8000]
TempoNest[7787] general protection ip:370f29d54d sp:7fff94517280 error:0 in libstdc++.so.6.0.13[370f200000+e8000]
TempoNest[7817] general protection ip:370f29d54d sp:7fff3bc06550 error:0 in libstdc++.so.6.0.13[370f200000+e8000]
TempoNest[7842]: segfault at 4000 ip 00000037082480ac sp 00007fff53684760 error 4 in libc-2.12.so[3708200000+18a000]
TempoNest[7861] general protection ip:370f29d54d sp:7fff418fa420 error:0 in libstdc++.so.6.0.13[370f200000+e8000]
TempoNest[7971] general protection ip:370f29d54d sp:7fff76550640 error:0 in libstdc++.so.6.0.13[370f200000+e8000]
TempoNest[7990] general protection ip:370f29d54d sp:7fffde5f2d60 error:0 in libstdc++.so.6.0.13[370f200000+e8000]
TempoNest[9004] general protection ip:370f29d54d sp:7fff4bfae000 error:0 in libstdc++.so.6.0.13[370f200000+e8000]
TempoNest[9025] general protection ip:370f29d54d sp:7fffb34f3c10 error:0 in libstdc++.so.6.0.13[370f200000+e8000]
TempoNest[9041] general protection ip:370f29d54d sp:7fff86a320b0 error:0 in libstdc++.so.6.0.13[370f200000+e8000]
TempoNest[9059] general protection ip:370f29d54d sp:7fff913794a0 error:0 in libstdc++.so.6.0.13[370f200000+e8000]
TempoNest[9074] general protection ip:370f29d54d sp:7ffff942db20 error:0 in libstdc++.so.6.0.13[370f200000+e8000]
TempoNest[9189] general protection ip:370f29d54d sp:7fffac340900 error:0 in libstdc++.so.6.0.13[370f200000+e8000]
TempoNest[9208] general protection ip:370f29d54d sp:7fffc51ba760 error:0 in libstdc++.so.6.0.13[370f200000+e8000]
TempoNest[10238] general protection ip:370f29d54d sp:7fff2aeef770 error:0 in libstdc++.so.6.0.13[370f200000+e8000]
TempoNest[10293] general protection ip:370f29d25b sp:7fffadc78430 error:0 in libstdc++.so.6.0.13[370f200000+e8000]
TempoNest[12364] general protection ip:370f29d54d sp:7fffd1c6b480 error:0 in libstdc++.so.6.0.13[370f200000+e8000]
TempoNest[12424]: segfault at 0 ip 000000000040bbbf sp 00007fff673b4710 error 4 in TempoNest[400000+ce000]
TempoNest[14531]: segfault at 0 ip 000000000040bbbc sp 00007fff8011c6d0 error 4 in TempoNest[400000+ce000]
TempoNest[15636] trap invalid opcode ip:40bd9f sp:7fff24c35210 error:0 in TempoNest[400000+cd000]
TempoNest[15654]: segfault at 0 ip 000000000040bbcd sp 00007fff60986920 error 4 in TempoNest[400000+ce000]
TempoNest[15673]: segfault at 0 ip 000000000040bbca sp 00007fff76439760 error 4 in TempoNest[400000+ce000]
TempoNest[7766] trap invalid opcode ip:40be74 sp:7fff998a57c0 error:0 in TempoNest[400000+cd000]
TempoNest[14083]: segfault at 0 ip 000000000040bc30 sp 00007fff32bdce00 error 6 in TempoNest[400000+ce000]
TempoNest[14107]: segfault at 0 ip 000000000040bc48 sp 00007fffe1fbfa40 error 6 in TempoNest[400000+ce000]
TempoNest[14135]: segfault at 72 ip 00000037082480ac sp 00007fff7caaec70 error 4 in libc-2.12.so[3708200000+18a000]
TempoNest[15883]: segfault at e70000000a ip 00002b388fcfcee0 sp 00007fff472acab0 error 4 in libtempo2.so.0.0.0[2b388fcec000+7b000]
TempoNest[15897]: segfault at 8 ip 00002aff15244ee0 sp 00007fffaee3bfd0 error 4 in libtempo2.so.0.0.0[2aff15234000+7b000]
TempoNest[16894]: segfault at 8 ip 00002afc90140ee0 sp 00007fff5d47cfe0 error 4 in libtempo2.so.0.0.0[2afc90130000+7b000]
TempoNest[18342]: segfault at 0 ip 00000000004186c3 sp 00007fff2cfd31f0 error 4 in TempoNest[400000+cf000]
TempoNest[23959]: segfault at 8 ip 00002ada3c54fee0 sp 00007ffffdf05420 error 4 in libtempo2.so.0.0.0[2ada3c53f000+7b000]
TempoNest[19067] general protection ip:4098a5 sp:7fff31e19308 error:0 in TempoNest[400000+cf000]
TempoNest[28229]: segfault at 0 ip 0000000000406410 sp 00007fff4c7fe9b0 error 6 in TempoNest[400000+b7000]
TempoNest[29797]: segfault at 0 ip 0000000000418ebb sp 00007fff4da33850 error 4 in TempoNest[400000+cf000]
TempoNest[26501] trap invalid opcode ip:4123ca sp:7fff06c29610 error:0 in TempoNest[400000+cd000]
TempoNest[26565] trap invalid opcode ip:4123ca sp:7fff23f6a2e0 error:0 in TempoNest[400000+cd000]
TempoNest[2642]: segfault at 21 ip 000000000040994c sp 00007fffac976ee8 error 6 in TempoNest[400000+d0000]
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
TempoNest[17172]: segfault at 0 ip 0000000000419020 sp 00007fffba1f9b20 error 4 in TempoNest[400000+d0000]
TempoNest[17309]: segfault at 0 ip 0000000000419020 sp 00007ffffe7870d0 error 4 in TempoNest[400000+d0000]
TempoNest[18318]: segfault at 0 ip 0000000000419020 sp 00007fff174261f0 error 4 in TempoNest[400000+d0000]
TempoNest[18359]: segfault at 0 ip 0000000000419020 sp 00007fffd55998f0 error 4 in TempoNest[400000+d0000]
TempoNest[18417]: segfault at 0 ip 0000000000412f6f sp 00007fff900293c0 error 4 in TempoNest[400000+d0000]
TempoNest[18441]: segfault at 0 ip 0000000000412f6f sp 00007fffb043bbd0 error 4 in TempoNest[400000+d0000]
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
TempoNest[11774] general protection ip:4096b6 sp:7fff890e1b58 error:0 in TempoNest[400000+d0000]
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
TempoNest[15230]: segfault at 0 ip 000000000041b9c0 sp 00007fffd99d9950 error 4 in TempoNest[400000+d0000]
TempoNest[15271]: segfault at 0 ip 000000000041b9c0 sp 00007fff46611c40 error 4 in TempoNest[400000+d0000]
TempoNest[15317]: segfault at 60 ip 00000000004123a6 sp 00007fff5754c7e0 error 4 in TempoNest[400000+d0000]
TempoNest[16464]: segfault at 0 ip 0000000000411908 sp 00007fff1d5d1f70 error 4 in TempoNest[400000+d0000]
TempoNest[16586]: segfault at 0 ip 000000000041190c sp 00007fff08a3d240 error 4 in TempoNest[400000+d0000]
TempoNest[17604]: segfault at 0 ip 0000000000411914 sp 00007ffff7aa7b80 error 4 in TempoNest[400000+d0000]
TempoNest[17759]: segfault at 0 ip 0000000000411914 sp 00007fffd7aca430 error 4 in TempoNest[400000+d0000]
TempoNest[17806]: segfault at 0 ip 0000000000411914 sp 00007fff33473d30 error 4 in TempoNest[400000+d0000]
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
TempoNest[31640]: segfault at 8 ip 00002b315e6cbee0 sp 00007fff34292c30 error 4 in libtempo2.so.0.0.0[2b315e6bb000+7b000]
TempoNest[2498] general protection ip:40b512 sp:7fff19384038 error:0 in TempoNest[400000+eb000]
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
TempoNest[30073]: segfault at 210 ip 00000000004137e4 sp 00007fffcd2dd4f0 error 4 in TempoNest[400000+eb000]
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
NFS: v4 server apgpu04 does not accept raw uid/gids. Reenabling the idmapper.
ib_qib IB0:1 Turning LOS on
ib_qib IB0:1 re-enabled QDR adaptation ibclt 2
ib_qib IB0:1 Turning LOS off
main[7785]: segfault at 0 ip 000000000041ed7d sp 00007fff7f0bd940 error 4 in main[400000+8c000]
main[7842]: segfault at 0 ip 00000000004205b0 sp 00007fff8ea0ba40 error 4 in main[400000+8c000]
ib_qib IB0:1 Turning LOS on
ib_qib IB0:1 re-enabled QDR adaptation ibclt 2
ib_qib IB0:1 Turning LOS off
TempoNest[4997] general protection ip:414bed sp:7fff4f50f7b0 error:0 in TempoNest[400000+d7000]
TempoNest[6094] general protection ip:414bed sp:7fffe53d53f0 error:0 in TempoNest[400000+d7000]
TempoNest[6106] general protection ip:414bed sp:7fff519a7930 error:0 in TempoNest[400000+d7000]
TempoNest[11610] general protection ip:414bed sp:7fff5e48fc60 error:0 in TempoNest[400000+d7000]
TempoNest[11666] general protection ip:414bed sp:7fff960209a0 error:0 in TempoNest[400000+d7000]
TempoNest[12664] general protection ip:414bed sp:7fff23c978a0 error:0 in TempoNest[400000+d7000]
TempoNest[12738] general protection ip:414bed sp:7fff25cbc7e0 error:0 in TempoNest[400000+d7000]
TempoNest[12774] general protection ip:414bed sp:7ffff12816e0 error:0 in TempoNest[400000+d7000]
TempoNest[13757] general protection ip:414c3d sp:7fff46c2e8e0 error:0 in TempoNest[400000+d7000]
TempoNest[16043] general protection ip:415e0d sp:7ffff3df74a0 error:0 in TempoNest[400000+d7000]
TempoNest[16081] general protection ip:415e0d sp:7fff1a31a650 error:0 in TempoNest[400000+d7000]
TempoNest[16111] general protection ip:4157b1 sp:7fff78bdb960 error:0 in TempoNest[400000+d6000]
TempoNest[16206] general protection ip:41535f sp:7fff78e06ef0 error:0 in TempoNest[400000+d6000]
TempoNest[17285] general protection ip:4152fc sp:7fffde4b0ce0 error:0 in TempoNest[400000+d6000]
TempoNest[6563]: segfault at 40 ip 000000000040b7a9 sp 00007fff46f20cb8 error 6 in TempoNest[400000+dd000]
TempoNest[9782] general protection ip:40b7b0 sp:7fff1b0fe8d8 error:0 in TempoNest[400000+dd000]
Anyway, i'm not sure what to do with this, or how to find out where this is happening, because the program never exits.. can anyone tell me how to debug this?
Thanks
