Kernel crash dump

Documentation Server Guide
1

Note:
This documentation has moved to a new home! Please update your bookmarks to the new URL for the up-to-date version of this page.

2

Hello Powersj,
I followed the instructions on Ubuntu 20.04LTS running on a EFI only system with secure boot enabled and I can’t seem to active kdump. I get the following error:

 #kdump-config show
DUMP_MODE:        kdump
USE_KDUMP:        1
KDUMP_SYSCTL:     kernel.panic_on_oops=1
KDUMP_COREDIR:    /var/crash
crashkernel addr: 0x64000000
   /var/lib/kdump/vmlinuz: symbolic link to /boot/vmlinuz-5.4.0-66-lowlatency
kdump initrd:
   /var/lib/kdump/initrd.img: symbolic link to /var/lib/kdump/initrd.img-5.4.0-66-lowlatency
current state:    Not ready to kdump

kexec command:
  no kexec command recorded


 #kdump-config load
 * Creating symlink /var/lib/kdump/vmlinuz
 * Creating symlink /var/lib/kdump/initrd.img
kexec_file_load failed: Operation not permitted
 * failed to load kdump kernel

#cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-5.4.0-66-lowlatency root=/dev/mapper/vgdata-lvroot ro console=tty0 console=ttyS6,115200n8 rootdelay=5 net.ifnames=0 ipv6.disable=1 crashkernel=512M-:192M

Any suggestions?

3

Bonjour Powersj, je n’arrive plus Ă  demarer le linux ni le Windows. Quand j’allume l’ordinateur la console apparaĂźt et on me demande de tĂ©lĂ©charger d’abord le kernel. Cependant je ne sais pas comment procĂ©der Ă  partir de la console

4

Vu que les commande sudo ne sont pas trouvable

5

Hi haimiko, sorry you hear you had trouble with kdump and secure boot. Hopefully you’ve long since resolved the issue. If you did, and you have some copyedit suggestions to improve this document, please follow up with the suggested text. Unfortunately this Discourse document is not a great way to do troubleshooting of usage issues, so if you happen to still be looking for help I’d recommend other technical support channels.

1 Like
6

The kernel crash dump guide needs updates

Here is a link to each of the events that can cause a kernel disruption
https://wiki.ubuntu.com/Kernel/CrashdumpRecipe

echo 1 > /proc/sys/kernel/hung_task_panic # panic when hung task is detected
echo 1 > /proc/sys/kernel/panic_on_io_nmi # panic on NMIs from I/O
echo 1 > /proc/sys/kernel/panic_on_oops # panic on oops or kernel bug detection
echo 1 > /proc/sys/kernel/panic_on_unrecovered_nmi # panic on NMIs from memory or unknown
echo 1 > /proc/sys/kernel/softlockup_panic # panic when soft lockups are detected
echo 1 > /proc/sys/vm/panic_on_oom # panic when out-of-memory happens

Changes under the default prompts after installing linux-crashdump.
During the installation, you will be prompted with the following dialogs.
“you can select no” for kexec-tools handling reboots. In fact it recommended to select no as this is the current default and
this step is unrelated to kernel panics and kernel crash dump collection and it applies to all
reboots. 'Yesis not actually required; This does change reboot behavior for the system not to actually reboot in the traditional fashion (soft power cycle) which might have issues (eg, some devices might not correctly handle a software-only re-initialization, instead of an actual soft power cycle), soNois a valid and safe choice, whileYes` may be seen as a reboot time ‘optimization’ if the system/devices handle it well.

‘Yes’ should be selected here as well, to enable kdump-tools.
Remove the ‘as well’, per the above.

You can also edit /etc/default/kexec and set parameters directly:
this needs to be updated to say /etc/default/kdump-tools

If you enable kdump-tools after a reboot, you will only need to issue the kdump-config load command to activate the kdump mechanism.

I don’t believe this is a correct statement
“If you enable kdump-tools after a reboot with the crashkernel= parameter in place, you will only need
”, since without the memory reservation for the crash kernel to be kexec’ed on, there’s little that kdump-tools itself can do about it.

Begin: Saving vmcore from kernel crash 

This log line is out of date.

needs to say this

Started Kernel crash dump capture service

Troubleshooting -
If you are trying to crashdump on a cloud like AWS please add the following lines
to /etc/sysctl.conf and put at bottom of file.

kernel.sysctl=1
kernel.unknown_nmi_panic=1

7

yeah, i agree with @hypothetical-lemon this looks out of date a lot.

Is there server technical author to improve this?

8

Thanks for this heads up. I took a note internally to discuss with the team how to handle this topic.

9

Also, how to crash dump in the cloud like AWS is slightly different from doing it in say Azure.

10

Could you additionally clarify the statement:
Starting with 16.04, the kernel crash dump mechanism is enabled by default.

This (to me) implies that linux-crashdump is installed in 16.04 and beyond by default, but this isn’t the case on my 22.04 jammy stock image, nor is it the case on public cloud images (I’m not sure it is anywhere). Does this mean something else maybe?

11

Hi! As a possible point of improvement, I think it would be helpful to try to answer the question of how much disk space should be available on a system to house a full crash dump. Ostensibly that answer would be “as much RAM as is on the system”, but there’s also the defaults that are used for consideration (#MAKEDUMP_ARGS="-c -d 31" - compression and in-use-only pages).

It might also be useful to make mention of the ability to add NFS/FTP/SSH info into /etc/default/kdump-tools for saving dumps to remote locations!

12

Hi @foxmulder2004 - since this moved (as the header says) but your suggestion is good I converted it into an issue tracker entry at: