vServer unresponsive

Turbokeks · Screenshot from 2015-09-14 11-58-23.png

Hey,

seit Samstag Nacht macht mein Server komische Sachen. Nach einem Reset ging alles wieder gut bis heute Mittag. Der Server wirkt unresponsive, antwortet dann nach ner Zeit doch auf Anfragen.Ram/CPU ist nicht ausgelastet. Ich vermute, dass es irgendein Problem mit der Festplatte ist, liege ich da richtig? Hat wer sonstige Ideen?

Im Syslog fand ich folgendes:

Zitat

Sep 13 03:45:58 server1 kernel: [383540.321203] INFO: rcu_sched self-detected stall on CPU
Sep 13 03:45:58 server1 kernel: [383595.308008] INFO: rcu_sched detected stalls on CPUs/tasks: { 1} (detected by 0, t=19655 jiffies, g=10331907, c=10331906, q=914)
Sep 13 03:45:58 server1 kernel: [383595.308024] sending NMI to all CPUs:
Sep 13 03:45:58 server1 kernel: [383595.308064] NMI backtrace for cpu 0
Sep 13 03:45:58 server1 kernel: [383595.308070] CPU: 0 PID: 27970 Comm: kworker/0:1 Tainted: G C 3.16.0-4-amd64 #1 Debian 3.16.7-ckt11-1+deb8u3
Sep 13 03:45:58 server1 kernel: [383595.308071] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 0.5.1 01/01/2011
Sep 13 03:45:58 server1 kernel: [383595.308098] Workqueue: events cache_reap
Sep 13 03:45:58 server1 kernel: [383595.308099] task: ffff88007358f3f0 ti: ffff880170134000 task.ti: ffff880170134000
Sep 13 03:45:58 server1 kernel: [383595.308100] RIP: 0010:[<ffffffff81051ac6>] [<ffffffff81051ac6>] native_write_msr_safe+0x6/0x10
Sep 13 03:45:58 server1 kernel: [383595.308115] RSP: 0018:ffff8801bfc03e08 EFLAGSSep 13 13:16:53 server1 rsyslogd: [origin software="rsyslogd" swVersion="8.4.2" x-pid="581" x-info="http://www.rsyslog.com"] sta$
Sep 13 13:16:53 server1 systemd-modules-load[145]: Inserted module 'fuse'
Sep 13 13:16:53 server1 systemd-modules-load[145]: Inserted module 'loop'
Sep 13 13:16:53 server1 keyboard-setup[168]: Setting preliminary keymap...done.
Sep 13 13:16:53 server1 quotaon.sh[270]: Checking quotas...
Sep 13 13:16:53 server1 systemd[1]: quota.service: main process exited, code=exited, status=1/FAILURE
Sep 13 13:16:53 server1 systemd[1]: Failed to start Check And Enable File System Quotas.

Alles anzeigen

Bin für jede Hilfe dankbar!

[gelöschtes Individuum]

Guten Abend,

bitte fahren Sie die Maschine einmal vollständig herunter.

Im Anschluss bitte ich Sie, dass System über Ihr VCP neu zu booten und zu prüfen, ob der Server weiterhin so langsam reagiert.

Turbokeks

Hey,

ja hab das mal gemacht und schaut ganz gut aus. Wirklich komisch, dass das in Folge zwei Mal aufgetreten ist...

Danke!

vladat

Hallo,

auf einem vServer bei Netcup, den ich verwalte, kommen ebenfalls rcu_sched stall Fehler, das komplette Herunterfahren und Neustarten hat leider nichts gebracht.

Code

[65701.500030] INFO: rcu_sched detected stalls on CPUs/tasks:
[65701.500091] 	0: (3 GPs behind) idle=f6b/140000000000000/0 softirq=576191/576191 fqs=2101 
[65701.500094] 	(detected by 1, t=2102 jiffies, g=411375, c=411374, q=3)
[65701.500097] Task dump for CPU 0:
[65701.500102] crond           R  running task        0 10192   2327 0x00000008
[65701.500115]  000000000000270f 00007f18b4957b40 0000000000000230 00007f18b4957b98
[65701.500118]  00007f18b4957b98 00007f18b4957dd8 00007f18b3d924a0 0000000000000000
[65701.500121]  0000000001e2eb90 0000000000000231 000000000001cf71 0000000001e30e70
[65701.500124] Call Trace:

Code

[65876.126410] INFO: rcu_sched self-detected stall on CPU
[65876.126410] 	1: (1 ticks this GP) idle=e99/1/0 softirq=516890/516890 fqs=0 
[65876.126410] 	 (t=5076 jiffies g=411558 c=411557 q=87)
[65876.126410] rcu_sched kthread starved for 5076 jiffies! g411558 c411557 f0x0 s3 ->state=0x1
[65876.126410] Task dump for CPU 1:
[65876.126410] swapper/1       R  running task        0     0      1 0x00000008
[65876.126410]  ffffffff816b8d00 ffff8801bfd03dd0 ffffffff81068b54 0000000000000001
[65876.126410]  ffffffff816b8d00 ffff8801bfd03de8 ffffffff8106a9f2 0000000000000002
[65876.126410]  ffff8801bfd03e18 ffffffff81087f29 ffff8801bfd14300 ffffffff816b8d00
[65876.126410] Call Trace:
[65876.126410]  <IRQ>  [<ffffffff81068b54>] sched_show_task+0xa4/0x100
[65876.126410]  [<ffffffff8106a9f2>] dump_cpu_task+0x32/0x40
[65876.126410]  [<ffffffff81087f29>] rcu_dump_cpu_stacks+0x89/0xe0
[65876.126410]  [<ffffffff8108ae23>] rcu_check_callbacks+0x413/0x700
[65876.126410]  [<ffffffff8109340f>] ? update_wall_time+0x41f/0x690
[65876.126410]  [<ffffffff8108ce74>] update_process_times+0x34/0x60
[65876.126410]  [<ffffffff8109a311>] tick_sched_handle.isra.16+0x31/0x40
[65876.126410]  [<ffffffff8109a358>] tick_sched_timer+0x38/0x70
[65876.126410]  [<ffffffff8108d880>] __hrtimer_run_queues+0xc0/0x160
[65876.126410]  [<ffffffff8108db43>] hrtimer_interrupt+0xa3/0x190
[65876.126410]  [<ffffffff8102f5b3>] smp_trace_apic_timer_interrupt+0x63/0x90
[65876.126410]  [<ffffffff8102f5e9>] smp_apic_timer_interrupt+0x9/0x10
[65876.126410]  [<ffffffff814e8162>] apic_timer_interrupt+0x82/0x90
[65876.126410]  <EOI>  [<ffffffff81037ef6>] ? native_safe_halt+0x6/0x10
[65876.126410]  [<ffffffff8100dfb9>] default_idle+0x9/0x10
[65876.126410]  [<ffffffff8100e5ba>] arch_cpu_idle+0xa/0x10
[65876.126410]  [<ffffffff81078e15>] default_idle_call+0x25/0x30
[65876.126410]  [<ffffffff81078f8c>] cpu_startup_entry+0x16c/0x1e0
[65876.126410]  [<ffffffff8102df02>] start_secondary+0x132/0x140
[65926.880697] INFO: rcu_sched detected stalls on CPUs/tasks:
[65926.880897] 	1: (2 ticks this GP) idle=e99/1/0 softirq=516890/516891 fqs=0 
[65926.880899] 	(detected by 0, t=5077 jiffies, g=411558, c=411557, q=88)
[66808.890035] INFO: rcu_sched detected stalls on CPUs/tasks:
[66808.890035] 	(detected by 1, t=2587 jiffies, g=418317, c=418316, q=33)
[66808.890035] All QSes seen, last rcu_sched kthread activity 2586 (4301620771-4301618185), jiffies_till_next_fqs=1, root ->qsmask 0x0
[66808.890035] postgres        R  running task        0 10336   3367 0x00000008
[66808.890035]  ffffffff816b8d00 ffff8801bfd03e18 ffffffff81068b54 ffff8801bfd14300
[66808.890035]  ffffffff816b8d00 ffff8801bfd03e88 ffffffff8108b10b 0000000000000000
[66808.890035]  ffffffff810c2e27 ffff8801bfd03e50 0000000000000021 000000000006620d
[66808.890035] Call Trace:
[66808.890035]  <IRQ>  [<ffffffff81068b54>] sched_show_task+0xa4/0x100
[66808.890035]  [<ffffffff8108b10b>] rcu_check_callbacks+0x6fb/0x700
[66808.890035]  [<ffffffff810c2e27>] ? acct_account_cputime+0x17/0x20
[66808.890035]  [<ffffffff8108ce74>] update_process_times+0x34/0x60
[66808.890035]  [<ffffffff8109a311>] tick_sched_handle.isra.16+0x31/0x40
[66808.890035]  [<ffffffff8109a358>] tick_sched_timer+0x38/0x70
[66808.890035]  [<ffffffff8108d880>] __hrtimer_run_queues+0xc0/0x160
[66808.890035]  [<ffffffff8108db43>] hrtimer_interrupt+0xa3/0x190
[66808.890035]  [<ffffffff8102f5b3>] smp_trace_apic_timer_interrupt+0x63/0x90
[66808.890035]  [<ffffffff8102f5e9>] smp_apic_timer_interrupt+0x9/0x10
[66808.890035]  [<ffffffff814e8162>] apic_timer_interrupt+0x82/0x90
[66808.890035]  <EOI>  [<ffffffff8124fe17>] ? clear_page_c_e+0x7/0x10
[66808.890035]  [<ffffffff810db870>] ? get_page_from_freelist+0x560/0x9a0
[66808.890035]  [<ffffffff8106be87>] ? __enqueue_entity+0x67/0x70
[66808.890035]  [<ffffffff810dbeee>] __alloc_pages_nodemask+0x14e/0x8b0
[66808.890035]  [<ffffffff81257b05>] ? find_next_bit+0x15/0x20
[66808.890035]  [<ffffffff812448ec>] ? cpumask_any_but+0x2c/0x40
[66808.890035]  [<ffffffff81042548>] ? flush_tlb_page+0x38/0x80
[66808.890035]  [<ffffffff8107a6f1>] ? __raw_callee_save___pv_queued_spin_unlock+0x11/0x20
[66808.890035]  [<ffffffff810f59f7>] ? wp_page_copy.isra.70+0x2a7/0x3e0
[66808.890035]  [<ffffffff8110f139>] alloc_pages_vma+0xa9/0x190
[66808.890035]  [<ffffffff810d43e4>] ? filemap_map_pages+0x1f4/0x200
[66808.890035]  [<ffffffff810f9b7e>] handle_mm_fault+0xe3e/0x1070
[66808.890035]  [<ffffffff814e8ad8>] ? async_page_fault+0x28/0x30
[66808.890035]  [<ffffffff8103c1c7>] __do_page_fault+0x157/0x360
[66808.890035]  [<ffffffff8107a6f1>] ? __raw_callee_save___pv_queued_spin_unlock+0x11/0x20
[66808.890035]  [<ffffffff8103c422>] do_page_fault+0x22/0x30
[66808.890035]  [<ffffffff81037854>] do_async_page_fault+0x14/0x70
[66808.890035]  [<ffffffff814e8ad8>] async_page_fault+0x28/0x30
[66808.890035] rcu_sched kthread starved for 2586 jiffies! g418317 c418316 f0x2 s3 ->state=0x0
[66880.770006] INFO: rcu_sched detected stalls on CPUs/tasks:
[66880.770006] 	0: (0 ticks this GP) idle=903/140000000000001/0 softirq=585629/585629 fqs=0 
[66880.770006] 	(detected by 1, t=2487 jiffies, g=418338, c=418337, q=8)
[66880.770006] Task dump for CPU 0:
[66880.770006] postgres        R  running task        0 10338   3367 0x00000000
[66880.770006]  ffffffff810fddde 0000000000000006 0000000002142000 ffff8801b79aff58
[66880.770006]  ffff880037d8b480 ffff8801b7150000 ffff8801b79aff18 ffffffff8103c1c7
[66880.770006]  0000000000100073 0000000000000002 ffff880037d8b4e8 0000005500000000
[66880.770006] Call Trace:
[66880.770006]  [<ffffffff810fddde>] ? do_brk+0x1fe/0x2e0
[66880.770006]  [<ffffffff8103c1c7>] ? __do_page_fault+0x157/0x360
[66880.770006]  [<ffffffff8103c422>] ? do_page_fault+0x22/0x30
[66880.770006]  [<ffffffff81037854>] ? do_async_page_fault+0x14/0x70
[66880.770006]  [<ffffffff814e8ad8>] ? async_page_fault+0x28/0x30
[66880.770006] rcu_sched kthread starved for 2487 jiffies! g418338 c418337 f0x0 s3 ->state=0x1
[66900.063957] INFO: rcu_sched self-detected stall on CPU
[66900.063957] 	1: (1 GPs behind) idle=4bf/140000000000001/0 softirq=531333/531334 fqs=0 
[66900.063957] 	 (t=2998 jiffies g=418345 c=418344 q=85)
[66900.063957] rcu_sched kthread starved for 2998 jiffies! g418345 c418344 f0x0 s3 ->state=0x1
[66900.063957] Task dump for CPU 1:
[66900.063957] run-crons       R  running task        0 10341  10340 0x00000004
[66900.063957]  ffffffff816b8d00 ffff8801bfd03dd0 ffffffff81068b54 0000000000000001
[66900.063957]  ffffffff816b8d00 ffff8801bfd03de8 ffffffff8106a9f2 0000000000000002
[66900.063957]  ffff8801bfd03e18 ffffffff81087f29 ffff8801bfd14300 ffffffff816b8d00
[66900.063957] Call Trace:
[66900.063957]  <IRQ>  [<ffffffff81068b54>] sched_show_task+0xa4/0x100
[66900.063957]  [<ffffffff8106a9f2>] dump_cpu_task+0x32/0x40
[66900.063957]  [<ffffffff81087f29>] rcu_dump_cpu_stacks+0x89/0xe0
[66900.063957]  [<ffffffff8108ae23>] rcu_check_callbacks+0x413/0x700
[66900.063957]  [<ffffffff810c2e27>] ? acct_account_cputime+0x17/0x20
[66900.063957]  [<ffffffff8106b4e8>] ? account_system_time+0x78/0x110
[66900.063957]  [<ffffffff8108ce74>] update_process_times+0x34/0x60
[66900.063957]  [<ffffffff8109a311>] tick_sched_handle.isra.16+0x31/0x40
[66900.063957]  [<ffffffff8109a358>] tick_sched_timer+0x38/0x70
[66900.063957]  [<ffffffff8108d880>] __hrtimer_run_queues+0xc0/0x160
[66900.063957]  [<ffffffff8108db43>] hrtimer_interrupt+0xa3/0x190
[66900.063957]  [<ffffffff8102f5b3>] smp_trace_apic_timer_interrupt+0x63/0x90
[66900.063957]  [<ffffffff8102f5e9>] smp_apic_timer_interrupt+0x9/0x10
[66900.063957]  [<ffffffff814e8162>] apic_timer_interrupt+0x82/0x90
[66900.063957]  <EOI>  [<ffffffff8124fe4a>] ? copy_page+0xa/0x10
[66900.063957]  [<ffffffff810f582d>] ? wp_page_copy.isra.70+0xdd/0x3e0
[66900.063957]  [<ffffffff810f6e8d>] do_wp_page+0xbd/0x620
[66900.063957]  [<ffffffff8106be87>] ? __enqueue_entity+0x67/0x70
[66900.063957]  [<ffffffff810f994e>] handle_mm_fault+0xc0e/0x1070
[66900.063957]  [<ffffffff8103c1c7>] __do_page_fault+0x157/0x360
[66900.063957]  [<ffffffff8103c422>] do_page_fault+0x22/0x30
[66900.063957]  [<ffffffff81037854>] do_async_page_fault+0x14/0x70
[66900.063957]  [<ffffffff814e8ad8>] async_page_fault+0x28/0x30

Alles anzeigen