Very high disk write/fsync latency on RS 12000 G12, services stalling, storage optimization did not help

  • Hello,

    My RS 12000 G12 (Debian 13, kernel 6.12, ext4 on vda ~3 TB, only 20% used) has had persistent, very high disk latency for days, even under light load. My self-hosted Sentry (Postgres/ClickHouse/Kafka) keeps stalling and is currently down.

    iostat -x 5 from inside the VM (scheduler=none, steal ~0%, CPU mostly idle):
    - w_await: 150–580 ms (e.g. 152.9, 251.4, 254.7, 581.0, 329.4 ms)
    - f_await (flush/fsync): 27–266 ms
    - %util: 54–78% at only ~30–70 write IOPS (~1 MB/s)
    - iowait: 8–19%

    What I already tried:
    - Checked the guest side: no discard/atime issues, scheduler already "none", I/O pressure is not from CPU or RAM.
    - Ran storage optimization in the SCP [confirm result: finished / stuck]. Latency is unchanged and Sentry is still down.
    - Cleaned ~150 GB of unused Docker build cache and reduced write load.

    The server also restarted on 2026-10-23 ~06:34; the kernel then logged "EXT4-fs (vda4): orphan cleanup on readonly fs" before remounting r/w, so the previous session likely did not end cleanly.

    I found a similar thread (RS8000 G11, high iowait) that was resolved by moving the server to another node, so this looks like a host-side storage problem.

    Has anyone seen the same on RS G12 recently, and did a node migration fix it for you? I have also opened a support ticket requesting a migration to another host node. netcup staff, if you can have a look at the storage node I'm on, I'd really appreciate it.

    Thanks!