Ubuntu Version:
Ubuntu 20.04.4 LTS
Desktop Environment (if applicable):
N/A this is an Azure VM
Problem Description:
we have a buncu of VM’s in Azure that the ‘unattended upgrade’ has been faithfully installing the latest kernel and not rebooting. following a recent request from our cyber security team, we implemented reboots across all our vm’s which enabled the 5.15.0-1089-azure kernel.
now we have an issue with one of those VM’s (a production database server) where about 1 1/2 days after, it had a kernel panic and stopped responding. We used the “redeploy” option in the azure portal to restart the vm on different azure hardware. but then 2 days later it had another kernel panic, (this time though azure automatically restarted it.
Relevant System Information:
VM type: Standard E8s V3 (8 vcpu’s, 64 gb Mem)
2 disks, 1tb and 2tb premium ssd storage
accelerated networking
(no swap defined)
this is a postgresql VM running PG13 with 28gb shared buffer
Screenshots or Error Messages:
kernel dump following the 1st crash:
de-prusapge01-postgresql-vm-0 login: [ 21.299685] cloud-init[1669]: Cloud-init v. 23.1.2-0ubuntu0~20.04.2 running 'modules:config' at Sun, 15 Jun 2025 23:27:13 +0000. Up 21.13 seconds.
[ 57.818127] cloud-init[2378]: Cloud-init v. 23.1.2-0ubuntu0~20.04.2 running 'modules:final' at Sun, 15 Jun 2025 23:27:49 +0000. Up 57.62 seconds.
[ 57.829391] cloud-init[2378]: Cloud-init v. 23.1.2-0ubuntu0~20.04.2 finished at Sun, 15 Jun 2025 23:27:50 +0000. Datasource DataSourceAzure [seed=/var/lib/waagent]. Up 57.81 seconds
2025-06-15T23:42:04.809719Z INFO Daemon Agent WALinuxAgent-2.13.1.1 launched with command 'python3 -u bin/WALinuxAgent-2.13.1.1-py3.9.egg -run-exthandlers' is successfully running
[95113.554362] general protection fault, probably for non-canonical address 0x9bbabd39b592f90a: 0000 [#1] SMP PTI
[95113.564422] CPU: 0 PID: 1 Comm: systemd Tainted: G W 5.15.0-1089-azure #98~20.04.1-Ubuntu
[95113.574041] Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS Hyper-V UEFI Release v4.1 03/08/2024
[95113.587243] RIP: 0010:__kmalloc_track_caller+0xfa/0x4c0
[95113.591560] Code: 05 43 c4 ec 75 49 8b 50 08 49 83 78 10 00 4d 8b 38 0f 84 74 03 00 00 4d 85 ff 0f 84 6b 03 00 00 41 8b 46 28 49 8b 3e 4c 01 f8 <48> 8b 18 48 89 c1 49 33 9e b8 00 00 00 4c 89 f8 48 0f c9 48 31 cb
[95113.613864] RSP: 0018:ffffbd2500043b50 EFLAGS: 00010286
[95113.619365] RAX: 9bbabd39b592f90a RBX: ffff9dfc446dfae0 RCX: 00000000000081a4
[95113.625549] RDX: 00000000000c7a94 RSI: 0000000000000cc0 RDI: 0000000000036060
[95113.642005] RBP: ffffbd2500043ba8 R08: ffff9e0bdfc36060 R09: 0000000000000000
[95113.648003] R10: ffffbd2500043d1e R11: 000000000000000d R12: 0000000000000000
[95113.655368] R13: 0000000000000cc0 R14: ffff9dfc40042300 R15: 9bbabd39b592f902
[95113.661143] FS: 00007f8eec623980(0000) GS:ffff9e0bdfc00000(0000) knlGS:0000000000000000
[95113.668469] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[95113.674499] CR2: 000055a9dcf4b558 CR3: 0000000100134005 CR4: 00000000003706f0
[95113.681024] Call Trace:
[95113.683625] <TASK>
[95113.686502] ? show_regs.cold+0x1a/0x1f
[95113.689989] ? __die_body+0x20/0x70
[95113.692952] ? die_addr+0x3e/0x60
[95113.695779] ? exc_general_protection+0x1c2/0x3d0
[95113.701525] ? asm_exc_general_protection+0x27/0x30
[95113.707298] ? __kmalloc_track_caller+0xfa/0x4c0
[95113.711033] ? mntput_no_expire+0x4c/0x260
[95113.714416] ? kstrdup_const+0x28/0x31
[95113.718437] kstrdup+0x32/0x70
[95113.725691] kstrdup_const+0x28/0x30
[95113.730199] __kernfs_new_node+0x4c/0x1e0
[95113.734930] ? pcpu_block_update_hint_alloc+0xd5/0x300
[95113.742809] kernfs_new_node+0x56/0x80
[95113.746628] __kernfs_create_file+0x29/0xd0
[95113.751085] cgroup_addrm_files+0x155/0x370
[95113.757540] ? pcpu_alloc+0x3cb/0x710
[95113.760925] ? cgroup_propagate_control+0x15b/0x190
[95113.767207] ? group_init+0x1a9/0x1e0
[95113.771627] css_populate_dir+0x11e/0x150
[95113.776188] cgroup_mkdir+0xb5/0x190
[95113.779418] kernfs_iop_mkdir+0x5c/0x90
[95113.782821] vfs_mkdir+0x20a/0x2f0
[95113.785738] do_mkdirat+0xf9/0x120
[95113.790507] __x64_sys_mkdir+0x2d/0x40
[95113.794190] x64_sys_call+0x94/0x1fa0
[95113.797392] do_syscall_64+0x54/0xb0
[95113.800728] entry_SYSCALL_64_after_hwframe+0x6c/0xd6
[95113.806080] RIP: 0033:0x7f8eed4aae7b
[95113.809861] Code: ff ff c3 0f 1f 40 00 48 8b 05 11 e0 0d 00 64 c7 00 5f 00 00 00 b8 ff ff ff ff c3 0f 1f 40 00 f3 0f 1e fa b8 53 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d e5 df 0d 00 f7 d8 64 89 01 48
[95113.829746] RSP: 002b:00007ffd10dec198 EFLAGS: 00000206 ORIG_RAX: 0000000000000053
[95113.837689] RAX: ffffffffffffffda RBX: 000055a9dcdd0ca0 RCX: 00007f8eed4aae7b
[95113.847023] RDX: 00007ffd10dec020 RSI: 00000000000001ed RDI: 000055a9dcf5bc20
[95113.855398] RBP: 00007f8eed320d9b R08: 0000000000000001 R09: 0070756f7267632f
[95113.863151] R10: 0000000000000100 R11: 0000000000000206 R12: 0000000000000000
[95113.869977] R13: 000055a9dcdd0ca0 R14: 0000000000000000 R15: 0000000000000000
[95113.879615] </TASK>
[95113.882789] Modules linked in: iptable_filter mlx4_en mlx4_core xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c xt_owner iptable_security xt_tcpudp bpfilter nls_iso8859_1 dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua kvm_intel binfmt_misc kvm crct10dif_pclmul serio_raw joydev crc32_pclmul hid_generic hyperv_drm ghash_clmulni_intel drm_kms_helper sha256_ssse3 syscopyarea sha1_ssse3 sysfillrect sysimgblt aesni_intel crypto_simd fb_sys_fops cryptd cec hid_hyperv hv_netvsc rc_core hid hyperv_keyboard sch_fq_codel msr drm efi_pstore i2c_core ip_tables x_tables autofs4
[95113.942668] ---[ end trace 9e9a016fcd132e7d ]---
[95113.995839] RIP: 0010:__kmalloc_track_caller+0xfa/0x4c0
[95114.001409] Code: 05 43 c4 ec 75 49 8b 50 08 49 83 78 10 00 4d 8b 38 0f 84 74 03 00 00 4d 85 ff 0f 84 6b 03 00 00 41 8b 46 28 49 8b 3e 4c 01 f8 <48> 8b 18 48 89 c1 49 33 9e b8 00 00 00 4c 89 f8 48 0f c9 48 31 cb
[95114.026510] RSP: 0018:ffffbd2500043b50 EFLAGS: 00010286
[95114.033811] RAX: 9bbabd39b592f90a RBX: ffff9dfc446dfae0 RCX: 00000000000081a4
[95114.042999] RDX: 00000000000c7a94 RSI: 0000000000000cc0 RDI: 0000000000036060
[95114.051068] RBP: ffffbd2500043ba8 R08: ffff9e0bdfc36060 R09: 0000000000000000
[95114.057839] R10: ffffbd2500043d1e R11: 000000000000000d R12: 0000000000000000
[95114.068313] R13: 0000000000000cc0 R14: ffff9dfc40042300 R15: 9bbabd39b592f902
[95114.076809] FS: 00007f8eec623980(0000) GS:ffff9e0bdfc00000(0000) knlGS:0000000000000000
[95114.087256] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[95114.092836] CR2: 000055a9dcf4b558 CR3: 0000000100134005 CR4: 00000000003706f0
[95114.100949] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
Console log following the 2nd kernel panic:
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.322537] general protection fault, probably for non-canonical address 0x8b29c7b3c0cf3aa8: 0000 [#1] SMP PTI
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.332008] CPU: 1 PID: 645684 Comm: kworker/u16:3 Tainted: G W 5.15.0-1089-azure #98~20.04.1-Ubuntu
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.341696] Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS Hyper-V UEFI Release v4.1 03/08/2024
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.351257] Workqueue: mlx4_en mlx4_en_do_get_stats [mlx4_en]
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.358198] RIP: 0010:kmem_cache_alloc_trace+0xda/0x420
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.363029] Code: 14 b7 4c 4e 49 8b 50 08 49 83 78 10 00 4d 8b 30 0f 84 f9 02 00 00 4d 85 f6 0f 84 f0 02 00 00 41 8b 45 28 49 8b 7d 00 4c 01 f0 <48> 8b 18 48 89 c1 49 33 9d b8 00 00 00 4c 89 f0 48 0f c9 48 31 cb
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.382132] RSP: 0018:ffffa8d55217fc78 EFLAGS: 00010282
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.387766] RAX: 8b29c7b3c0cf3aa8 RBX: ffff9818912b0180 RCX: 0000000000000000
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.394545] RDX: 000000000019dd75 RSI: 0000000000000cc0 RDI: 0000000000036060
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.402791] RBP: ffffa8d55217fcb8 R08: ffff98281fc76060 R09: ffff981890a9e000
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.414549] R10: ffff9818fdc04000 R11: 0000000000000000 R12: 0000000000000000
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.426745] R13: ffff981880042300 R14: 8b29c7b3c0cf3aa0 R15: ffffffffc05caaf5
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.442001] FS: 0000000000000000(0000) GS:ffff98281fc40000(0000) knlGS:0000000000000000
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.457532] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.467153] CR2: 00007f7c28e5e940 CR3: 0000000132fdc004 CR4: 00000000003706e0
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.480832] Call Trace:
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.486876] <TASK>
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.491523] ? show_regs.cold+0x1a/0x1f
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.500088] ? __die_body+0x20/0x70
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.508684] ? die_addr+0x3e/0x60
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.513903] ? exc_general_protection+0x1c2/0x3d0
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.520870] ? asm_exc_general_protection+0x27/0x30
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.528848] ? mlx4_alloc_cmd_mailbox+0x25/0x80 [mlx4_core]
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.539193] ? kmem_cache_alloc_trace+0xda/0x420
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.547001] ? kmem_cache_alloc_trace+0x44/0x420
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.556483] mlx4_alloc_cmd_mailbox+0x25/0x80 [mlx4_core]
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.566577] mlx4_get_counter_stats+0x53/0x1b0 [mlx4_core]
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.576989] mlx4_en_DUMP_ETH_STATS+0x148/0xc80 [mlx4_en]
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.588129] mlx4_en_do_get_stats+0x105/0x4d0 [mlx4_en]
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.597667] ? raw_spin_rq_unlock+0x10/0x30
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.603443] ? finish_task_switch+0x8c/0x2f0
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.610175] process_one_work+0x225/0x3d0
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.614811] worker_thread+0x4d/0x3e0
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.622489] ? process_one_work+0x3d0/0x3d0
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.629170] kthread+0x12a/0x150
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.633180] ? set_kthread_struct+0x50/0x50
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.638375] ret_from_fork+0x22/0x30
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.642605] </TASK>
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.645148] Modules linked in: iptable_filter xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c xt_owner iptable_security xt_tcpudp bpfilter nls_iso8859_1 dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua mlx4_en binfmt_misc mlx4_core joydev serio_raw hid_generic kvm_intel kvm crct10dif_pclmul crc32_pclmul hyperv_drm drm_kms_helper ghash_clmulni_intel syscopyarea sysfillrect sha256_ssse3 sysimgblt sha1_ssse3 fb_sys_fops aesni_intel cec hv_netvsc hid_hyperv rc_core hid crypto_simd hyperv_keyboard cryptd sch_fq_codel msr drm i2c_core efi_pstore ip_tables x_tables autofs4
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.695873] ---[ end trace c04e56f86ccdd281 ]---
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.790412] RIP: 0010:kmem_cache_alloc_trace+0xda/0x420
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.795657] Code: 14 b7 4c 4e 49 8b 50 08 49 83 78 10 00 4d 8b 30 0f 84 f9 02 00 00 4d 85 f6 0f 84 f0 02 00 00 41 8b 45 28 49 8b 7d 00 4c 01 f0 <48> 8b 18 48 89 c1 49 33 9d b8 00 00 00 4c 89 f0 48 0f c9 48 31 cb
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.815879] RSP: 0018:ffffa8d55217fc78 EFLAGS: 00010282
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.822294] RAX: 8b29c7b3c0cf3aa8 RBX: ffff9818912b0180 RCX: 0000000000000000
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.830897] RDX: 000000000019dd75 RSI: 0000000000000cc0 RDI: 0000000000036060
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.838424] RBP: ffffa8d55217fcb8 R08: ffff98281fc76060 R09: ffff981890a9e000
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.846196] R10: ffff9818fdc04000 R11: 0000000000000000 R12: 0000000000000000
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.852890] R13: ffff981880042300 R14: 8b29c7b3c0cf3aa0 R15: ffffffffc05caaf5
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.860220] FS: 0000000000000000(0000) GS:ffff98281fc40000(0000) knlGS:0000000000000000
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.868995] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.875249] CR2: 00007f7c28e5e940 CR3: 0000000132fdc004 CR4: 00000000003706e0
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.884945] general protection fault, probably for non-canonical address 0x8b29c7b3c0cf3aa8: 0000 [#2] SMP PTI
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.896024] CPU: 1 PID: 607082 Comm: kworker/1:2 Tainted: G D W 5.15.0-1089-azure #98~20.04.1-Ubuntu
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.906958] Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS Hyper-V UEFI Release v4.1 03/08/2024
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.919407] Workqueue: events drm_fb_helper_damage_work [drm_kms_helper]
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.925955] RIP: 0010:__kmalloc+0xfc/0x4b0
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.929835] Code: e2 cd 4c 4e 49 8b 50 08 49 83 78 10 00 4d 8b 30 0f 84 67 03 00 00 4d 85 f6 0f 84 5e 03 00 00 41 8b 45 28 49 8b 7d 00 4c 01 f0 <48> 8b 18 48 89 c1 49 33 9d b8 00 00 00 4c 89 f0 48 0f c9 48 31 cb
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.947751] RSP: 0018:ffffa8d5404ebce8 EFLAGS: 00010282
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.954053] RAX: 8b29c7b3c0cf3aa8 RBX: 0000000000000dc0 RCX: 0000000000000000
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.961853] RDX: 000000000019dd75 RSI: 0000000000000dc0 RDI: 0000000000036060
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.968492] RBP: ffffa8d5404ebd28 R08: ffff98281fc76060 R09: ffff981884b5e680
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.976649] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000000
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.985064] R13: ffff981880042300 R14: 8b29c7b3c0cf3aa0 R15: ffffffffc049c0d5
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226876.992989] FS: 0000000000000000(0000) GS:ffff98281fc40000(0000) knlGS:0000000000000000
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.002844] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.008921] CR2: 00007f7c28e5e940 CR3: 000000010022c003 CR4: 00000000003706e0
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.015559] Call Trace:
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.018185] <TASK>
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.020564] ? show_regs.cold+0x1a/0x1f
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.025386] ? __die_body+0x20/0x70
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.029528] ? die_addr+0x3e/0x60
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.034030] ? exc_general_protection+0x1c2/0x3d0
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.039797] ? asm_exc_general_protection+0x27/0x30
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.045809] ? drm_atomic_helper_dirtyfb+0x95/0x280 [drm_kms_helper]
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.052092] ? __kmalloc+0xfc/0x4b0
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.056658] ? __kmalloc+0x66/0x4b0
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.061142] drm_atomic_helper_dirtyfb+0x95/0x280 [drm_kms_helper]
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.067363] drm_fb_helper_damage_work+0x1ac/0x340 [drm_kms_helper]
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.074292] ? save_fpregs_to_fpstate+0x3f/0xa0
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.078761] process_one_work+0x225/0x3d0
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.083763] worker_thread+0x4d/0x3e0
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.088342] ? process_one_work+0x3d0/0x3d0
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.096294] kthread+0x12a/0x150
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.100236] ? set_kthread_struct+0x50/0x50
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.104378] ret_from_fork+0x22/0x30
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.108497] </TASK>
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.111012] Modules linked in: iptable_filter xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c xt_owner iptable_security xt_tcpudp bpfilter nls_iso8859_1 dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua mlx4_en binfmt_misc mlx4_core joydev serio_raw hid_generic kvm_intel kvm crct10dif_pclmul crc32_pclmul hyperv_drm drm_kms_helper ghash_clmulni_intel syscopyarea sysfillrect sha256_ssse3 sysimgblt sha1_ssse3 fb_sys_fops aesni_intel cec hv_netvsc hid_hyperv rc_core hid crypto_simd hyperv_keyboard cryptd sch_fq_codel msr drm i2c_core efi_pstore ip_tables x_tables autofs4
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.164470] ---[ end trace c04e56f86ccdd282 ]---
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.251577] RIP: 0010:kmem_cache_alloc_trace+0xda/0x420
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.259156] Code: 14 b7 4c 4e 49 8b 50 08 49 83 78 10 00 4d 8b 30 0f 84 f9 02 00 00 4d 85 f6 0f 84 f0 02 00 00 41 8b 45 28 49 8b 7d 00 4c 01 f0 <48> 8b 18 48 89 c1 49 33 9d b8 00 00 00 4c 89 f0 48 0f c9 48 31 cb
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.282578] RSP: 0018:ffffa8d55217fc78 EFLAGS: 00010282
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.289442] RAX: 8b29c7b3c0cf3aa8 RBX: ffff9818912b0180 RCX: 0000000000000000
Jun 19 19:48:17 de-prusapge01-postgresql-vm-0 kernel: [226877.296720] RDX: 000000000019dd75 RSI: 0000000000000cc0 RDI: 0000000000036060
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226877.304161] RBP: ffffa8d55217fcb8 R08: ffff98281fc76060 R09: ffff981890a9e000
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226877.312593] R10: ffff9818fdc04000 R11: 0000000000000000 R12: 0000000000000000
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226877.319484] R13: ffff981880042300 R14: 8b29c7b3c0cf3aa0 R15: ffffffffc05caaf5
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226877.328689] FS: 0000000000000000(0000) GS:ffff98281fc40000(0000) knlGS:0000000000000000
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226877.335899] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226877.342692] CR2: 00007f7c28e5e940 CR3: 000000010022c003 CR4: 00000000003706e0
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226877.862618] general protection fault, probably for non-canonical address 0x8b29c7b3c0cf3aa8: 0000 [#3] SMP PTI
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226877.874550] CPU: 1 PID: 649483 Comm: postgres Tainted: G D W 5.15.0-1089-azure #98~20.04.1-Ubuntu
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226877.884869] Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS Hyper-V UEFI Release v4.1 03/08/2024
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226877.905443] RIP: 0010:kmem_cache_alloc_trace+0xda/0x420
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226877.917457] Code: 14 b7 4c 4e 49 8b 50 08 49 83 78 10 00 4d 8b 30 0f 84 f9 02 00 00 4d 85 f6 0f 84 f0 02 00 00 41 8b 45 28 49 8b 7d 00 4c 01 f0 <48> 8b 18 48 89 c1 49 33 9d b8 00 00 00 4c 89 f0 48 0f c9 48 31 cb
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226877.953032] RSP: 0018:ffffa8d54f47fa80 EFLAGS: 00010282
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226877.960155] RAX: 8b29c7b3c0cf3aa8 RBX: ffffa8d54f47fd90 RCX: 0000000000000000
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226877.969242] RDX: 000000000019dd75 RSI: 0000000000000a20 RDI: 0000000000036060
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226877.975924] RBP: ffffa8d54f47fac0 R08: ffff98281fc76060 R09: 0000000000000004
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226877.984546] R10: ffff981884c6e3c0 R11: 6d6f6f2f666c6573 R12: 0000000000000000
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226877.991638] R13: ffff981880042300 R14: 8b29c7b3c0cf3aa0 R15: ffffffffb1c34d44
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226877.999138] FS: 00007f66ac5f8ec0(0000) GS:ffff98281fc40000(0000) knlGS:0000000000000000
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.007259] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.012463] CR2: 00007f66af12d230 CR3: 00000001726b0006 CR4: 00000000003706e0
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.020594] Call Trace:
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.023978] <TASK>
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.026405] ? show_regs.cold+0x1a/0x1f
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.031152] ? __die_body+0x20/0x70
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.036091] ? die_addr+0x3e/0x60
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.040525] ? exc_general_protection+0x1c2/0x3d0
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.046863] ? asm_exc_general_protection+0x27/0x30
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.052850] ? proc_self_get_link+0xa4/0xc0
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.058663] ? kmem_cache_alloc_trace+0xda/0x420
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.064135] proc_self_get_link+0xa4/0xc0
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.068905] pick_link+0x27a/0x410
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.072561] ? proc_ns_get_link+0xe0/0xe0
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.076990] step_into+0x238/0x3b0
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.081415] walk_component+0x70/0x1c0
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.085058] link_path_walk.part.0+0x231/0x390
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.089889] ? path_init+0x2b2/0x390
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.094231] path_openat+0xd0/0xe00
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.097536] do_filp_open+0xb2/0x120
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.101864] ? audit_alloc_name+0x125/0x150
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.107223] do_sys_openat2+0x249/0x330
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.110952] do_sys_open+0x46/0x80
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.115464] __x64_sys_openat+0x20/0x30
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.120338] x64_sys_call+0x1a55/0x1fa0
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.125598] do_syscall_64+0x54/0xb0
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.129839] ? irqentry_exit_to_user_mode+0x10/0x30
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.134324] ? irqentry_exit+0x1d/0x30
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.137797] ? exc_page_fault+0x7d/0x160
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.142651] entry_SYSCALL_64_after_hwframe+0x6c/0xd6
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.147250] RIP: 0033:0x7f66afe7eabb
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.150656] Code: 25 00 00 41 00 3d 00 00 41 00 74 4b 64 8b 04 25 18 00 00 00 85 c0 75 67 44 89 e2 48 89 ee bf 9c ff ff ff b8 01 01 00 00 0f 05 <48> 3d 00 f0 ff ff 0f 87 91 00 00 00 48 8b 4c 24 28 64 48 33 0c 25
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.169529] RSP: 002b:00007ffce79a68e0 EFLAGS: 00000246 ORIG_RAX: 0000000000000101
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.178021] RAX: ffffffffffffffda RBX: 0000562cff0b2620 RCX: 00007f66afe7eabb
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.185565] RDX: 0000000000000001 RSI: 0000562d3e43b8a3 RDI: 00000000ffffff9c
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.192584] RBP: 0000562d3e43b8a3 R08: 0000000000000000 R09: 00007f66afe892d0
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.199199] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000001
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.205998] R13: 0000562cff0b2720 R14: 0000000000000006 R15: 0000562d3e472550
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.212190] </TASK>
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.214512] Modules linked in: iptable_filter xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c xt_owner iptable_security xt_tcpudp bpfilter nls_iso8859_1 dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua mlx4_en binfmt_misc mlx4_core joydev serio_raw hid_generic kvm_intel kvm crct10dif_pclmul crc32_pclmul hyperv_drm drm_kms_helper ghash_clmulni_intel syscopyarea sysfillrect sha256_ssse3 sysimgblt sha1_ssse3 fb_sys_fops aesni_intel cec hv_netvsc hid_hyperv rc_core hid crypto_simd hyperv_keyboard cryptd sch_fq_codel msr drm i2c_core efi_pstore ip_tables x_tables autofs4
Jun 19 19:48:18 de-prusapge01-postgresql-vm-0 kernel: [226878.265941] ---[ end trace c04e56f86ccdd283 ]---
Jun 19 19:48:19 de-prusapge01-postgresql-vm-0 kernel: [226878.333758] RIP: 0010:kmem_cache_alloc_trace+0xda/0x420
Jun 19 19:48:19 de-prusapge01-postgresql-vm-0 kernel: [226878.340101] Code: 14 b7 4c 4e 49 8b 50 08 49 83 78 10 00 4d 8b 30 0f 84 f9 02 00 00 4d 85 f6 0f 84 f0 02 00 00 41 8b 45 28 49 8b 7d 00 4c 01 f0 <48> 8b 18 48 89 c1 49 33 9d b8 00 00 00 4c 89 f0 48 0f c9 48 31 cb
Jun 19 19:48:19 de-prusapge01-postgresql-vm-0 kernel: [226878.360139] RSP: 0018:ffffa8d55217fc78 EFLAGS: 00010282
Jun 19 19:48:19 de-prusapge01-postgresql-vm-0 kernel: [226878.366467] RAX: 8b29c7b3c0cf3aa8 RBX: ffff9818912b0180 RCX: 0000000000000000
Jun 19 19:48:19 de-prusapge01-postgresql-vm-0 kernel: [226878.374813] RDX: 000000000019dd75 RSI: 0000000000000cc0 RDI: 0000000000036060
Jun 19 19:48:19 de-prusapge01-postgresql-vm-0 kernel: [226878.383320] RBP: ffffa8d55217fcb8 R08: ffff98281fc76060 R09: ffff981890a9e000
Jun 19 19:48:19 de-prusapge01-postgresql-vm-0 kernel: [226878.390234] R10: ffff9818fdc04000 R11: 0000000000000000 R12: 0000000000000000
Jun 19 19:48:19 de-prusapge01-postgresql-vm-0 kernel: [226878.397933] R13: ffff981880042300 R14: 8b29c7b3c0cf3aa0 R15: ffffffffc05caaf5
Jun 19 19:48:19 de-prusapge01-postgresql-vm-0 kernel: [226878.407102] FS: 00007f66ac5f8ec0(0000) GS:ffff98281fc40000(0000) knlGS:0000000000000000
Jun 19 19:48:19 de-prusapge01-postgresql-vm-0 kernel: [226878.417512] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 19 19:48:19 de-prusapge01-postgresql-vm-0 kernel: [226878.424361] CR2: 00007f66af12d230 CR3: 00000001726b0006 CR4: 00000000003706e0
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.441060] general protection fault, probably for non-canonical address 0x8b29c7b3c0cf3aa8: 0000 [#4] SMP PTI
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.462858] CPU: 1 PID: 649508 Comm: postgres Tainted: G D W 5.15.0-1089-azure #98~20.04.1-Ubuntu
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.473129] Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS Hyper-V UEFI Release v4.1 03/08/2024
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.483566] RIP: 0010:kmem_cache_alloc_trace+0xda/0x420
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.489664] Code: 14 b7 4c 4e 49 8b 50 08 49 83 78 10 00 4d 8b 30 0f 84 f9 02 00 00 4d 85 f6 0f 84 f0 02 00 00 41 8b 45 28 49 8b 7d 00 4c 01 f0 <48> 8b 18 48 89 c1 49 33 9d b8 00 00 00 4c 89 f0 48 0f c9 48 31 cb
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.508280] RSP: 0018:ffffa8d54f4c79f0 EFLAGS: 00010282
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.513654] RAX: 8b29c7b3c0cf3aa8 RBX: ffffa8d54f4c7d00 RCX: 0000000000000000
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.522426] RDX: 000000000019dd75 RSI: 0000000000000a20 RDI: 0000000000036060
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.533488] RBP: ffffa8d54f4c7a30 R08: ffff98281fc76060 R09: 0000000000000004
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.547340] R10: ffff981884c6e3c0 R11: 6d6f6f2f666c6573 R12: 0000000000000000
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.560694] R13: ffff981880042300 R14: 8b29c7b3c0cf3aa0 R15: ffffffffb1c34d44
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.575894] FS: 00007f66ac5f8ec0(0000) GS:ffff98281fc40000(0000) knlGS:0000000000000000
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.590080] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.602890] CR2: 00007f66af12d230 CR3: 0000000179120003 CR4: 00000000003706e0
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.611191] Call Trace:
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.615016] <TASK>
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.618766] ? show_regs.cold+0x1a/0x1f
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.622455] ? __die_body+0x20/0x70
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.626042] ? die_addr+0x3e/0x60
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.630346] ? exc_general_protection+0x1c2/0x3d0
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.635225] ? asm_exc_general_protection+0x27/0x30
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.640442] ? proc_self_get_link+0xa4/0xc0
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.644960] ? kmem_cache_alloc_trace+0xda/0x420
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.649697] proc_self_get_link+0xa4/0xc0
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.654842] pick_link+0x27a/0x410
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.659793] ? proc_ns_get_link+0xe0/0xe0
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.664271] step_into+0x238/0x3b0
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.668571] walk_component+0x70/0x1c0
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.672217] link_path_walk.part.0+0x231/0x390
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.679917] ? path_init+0x2b2/0x390
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.685493] path_openat+0xd0/0xe00
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.691248] ? page_counter_try_charge+0x34/0xc0
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.697246] do_filp_open+0xb2/0x120
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.705337] ? audit_alloc_name+0x125/0x150
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.712608] ? alloc_fd+0xf3/0x190
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.716709] do_sys_openat2+0x249/0x330
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.721369] do_sys_open+0x46/0x80
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.724831] __x64_sys_openat+0x20/0x30
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.728298] x64_sys_call+0x1a55/0x1fa0
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.733090] do_syscall_64+0x54/0xb0
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.736895] ? handle_mm_fault+0xcb/0x2a0
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.741346] ? exit_to_user_mode_prepare+0x54/0x270
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.747903] ? do_user_addr_fault+0x1d3/0x600
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.752565] ? irqentry_exit_to_user_mode+0x10/0x30
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.758201] ? irqentry_exit+0x1d/0x30
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.761434] ? exc_page_fault+0x7d/0x160
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.766548] entry_SYSCALL_64_after_hwframe+0x6c/0xd6
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.771145] RIP: 0033:0x7f66afe7eabb
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.775417] Code: 25 00 00 41 00 3d 00 00 41 00 74 4b 64 8b 04 25 18 00 00 00 85 c0 75 67 44 89 e2 48 89 ee bf 9c ff ff ff b8 01 01 00 00 0f 05 <48> 3d 00 f0 ff ff 0f 87 91 00 00 00 48 8b 4c 24 28 64 48 33 0c 25
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.793485] RSP: 002b:00007ffce79a68e0 EFLAGS: 00000246 ORIG_RAX: 0000000000000101
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.804658] RAX: ffffffffffffffda RBX: 0000562cff0b2620 RCX: 00007f66afe7eabb
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.812424] RDX: 0000000000000001 RSI: 0000562d3e43b8a3 RDI: 00000000ffffff9c
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.818958] RBP: 0000562d3e43b8a3 R08: 0000000000000000 R09: 00007f66afe892d0
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.825782] R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000001
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.832093] R13: 0000562cff0b2720 R14: 0000000000000006 R15: 0000562d3e464f30
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.839911] </TASK>
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.842203] Modules linked in: iptable_filter xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c xt_owner iptable_security xt_tcpudp bpfilter nls_iso8859_1 dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua mlx4_en binfmt_misc mlx4_core joydev serio_raw hid_generic kvm_intel kvm crct10dif_pclmul crc32_pclmul hyperv_drm drm_kms_helper ghash_clmulni_intel syscopyarea sysfillrect sha256_ssse3 sysimgblt sha1_ssse3 fb_sys_fops aesni_intel cec hv_netvsc hid_hyperv rc_core hid crypto_simd hyperv_keyboard cryptd sch_fq_codel msr drm i2c_core efi_pstore ip_tables x_tables autofs4
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.893204] ---[ end trace c04e56f86ccdd284 ]---
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.962747] RIP: 0010:kmem_cache_alloc_trace+0xda/0x420
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226883.969352] Code: 14 b7 4c 4e 49 8b 50 08 49 83 78 10 00 4d 8b 30 0f 84 f9 02 00 00 4d 85 f6 0f 84 f0 02 00 00 41 8b 45 28 49 8b 7d 00 4c 01 f0 <48> 8b 18 48 89 c1 49 33 9d b8 00 00 00 4c 89 f0 48 0f c9 48 31 cb
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226884.001279] RSP: 0018:ffffa8d55217fc78 EFLAGS: 00010282
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226884.008201] RAX: 8b29c7b3c0cf3aa8 RBX: ffff9818912b0180 RCX: 0000000000000000
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226884.017422] RDX: 000000000019dd75 RSI: 0000000000000cc0 RDI: 0000000000036060
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226884.028910] RBP: ffffa8d55217fcb8 R08: ffff98281fc76060 R09: ffff981890a9e000
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226884.038372] R10: ffff9818fdc04000 R11: 0000000000000000 R12: 0000000000000000
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226884.044732] R13: ffff981880042300 R14: 8b29c7b3c0cf3aa0 R15: ffffffffc05caaf5
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226884.053146] FS: 00007f66ac5f8ec0(0000) GS:ffff98281fc40000(0000) knlGS:0000000000000000
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226884.067512] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun 19 19:48:24 de-prusapge01-postgresql-vm-0 kernel: [226884.076146] CR2: 00007f66af12d230 CR3: 0000000179120003 CR4: 00000000003706e0
What I’ve Tried:
We have tried re-deploying the VM
We have added a parameter to the nessus-agent to limit the nessus agent to 1gb
Metrics don’t show any abnormal load on the VM at the times of the crashes
We raised a support call with Microsoft, but they brushed us off and said we need to talk with the Ubuntu Distribution Support team, hence asking in this forum.
Hoping someone is able to help us get to the bottom of why this VM has started crashing and how to resolve.
Regards,
John