From Fedora Project Wiki

(There is no kdump.conf man page. Substitute the mkdumprd man page)
Line 19: Line 19:
#: <pre>yum install --enablerepo=fedora-debuginfo --enablerepo=updates-debuginfo kexec-tools crash kernel-debuginfo</pre>
#: <pre>yum install --enablerepo=fedora-debuginfo --enablerepo=updates-debuginfo kexec-tools crash kernel-debuginfo</pre>
#:
#:
# Next, edit /etc/grub.conf and add the "crashkernel=128M" command line option.  An example command line might look like this:
# Next, edit /boot/grub/grub.conf or /boot/grub2/grub.cfg and add the "crashkernel=128M" command line option.  An example command line might look like this (for grub2, "kernel" is replaced by "linux"):
#: <pre>kernel /vmlinuz-2.6.29.5-191.fc11.x86_64 ro root=/dev/VolGroup00/LogVol00 rhgb console=tty0 console=ttyS0,115200 crashkernel=128M</pre>
#: <pre>kernel /vmlinuz-3.1.4-1.fc16.x86_64 ro root=/dev/VolGroup00/LogVol00 rhgb LANG=en_US.UTF-8 crashkernel=128M</pre>
# Next, consider editing the kdump configuration file {{filename|/etc/kdump.conf}}.  When the kdump service starts it creates an initramfs for use with the crash kernel.  In the default configuration, kdump creates an initramfs that locates and mounts the root file system, pivots to it and runs {{command|/sbin/init}}, which also is the normal boot process.  While this is functional, it is somewhat limiting, in that it mandates the vmcore be saved to a local file system, and also implies the starting of all the other system services which can cause problems normally associated with operating in low memory environments (remember the system is acting here like you are running with only 128M of available RAM).  By using {{filename|/etc/kdump.conf}}, the kdump service will attempt to capture the vmcore file from the initramfs, which avoids starting all those unneeded services, and allows you access to other kdump features, such as saving the vmcore to a remote system via ssh, or nfs share, or a raw disk.  For additional information, consult the mkdumprd man page.
# Next, consider editing the kdump configuration file {{filename|/etc/kdump.conf}}.  This will allow you to write the dump over the network or to some other location on the local system, rather than to the default location of /var/crash.  For additional information, consult the mkdumprd man page and the comments in /etc/kdump.conf.
# Next, reboot your system
# Next, reboot your system
# Finally, active the kdump system service
# Finally, active the kdump system service
Line 31: Line 31:
# Above shown parameter reserves 128MB of physical memory. This reserved memory is used to preload and run the capture kernel.
# Above shown parameter reserves 128MB of physical memory. This reserved memory is used to preload and run the capture kernel.
# Init scripts take care of pre-loading the capture kernel at system boot time.
# Init scripts take care of pre-loading the capture kernel at system boot time.
# It is recommended to either set up a serial console or switch to run level 3 (init 3) for testing purposes. The reason being that kdump does not reset the console if you are in X or  framebuffer mode, and no message might be visible on console after system crash.
# It is recommended to either set up a serial console or switch to run level 3 (init 3) for testing purposes. The reason being that kdump does not reset the console if you are in X or  framebuffer mode, and no message might be visible on console after system crash. You may also see screen corruption in graphics mode during capture.
# Capturing a crash dump can take a long time, especially if the system has a lot of memory. Be patient. The system will reboot after the dump is captured.


=== Step 2: Capturing the Dump ===
=== Step 2: Capturing the Dump ===

Revision as of 13:11, 9 December 2011

Kernel and kdump

Kdump is a kernel crash dumping mechanism and is very reliable because the crash dump is captured from the context of a freshly booted kernel and not from the context of the crashed kernel. Kdump uses kexec to boot into a second kernel whenever system crashes. This second kernel, often called the crash kernel, boots with very little memory and captures the dump image.

The first kernel reserves a section of memory that the second kernel uses to boot. Kexec enables booting the capture kernel without going through the BIOS, so contents of the first kernel's memory are preserved, which is essentially the kernel crash dump.

How to Use Kdump

Step 1: Configuring Kdump

  1. First, install the kexec-tools, crash and kernel-debuginfo packages. Use following command line to install the packages.
    yum install --enablerepo=fedora-debuginfo --enablerepo=updates-debuginfo kexec-tools crash kernel-debuginfo
  2. Next, edit /boot/grub/grub.conf or /boot/grub2/grub.cfg and add the "crashkernel=128M" command line option. An example command line might look like this (for grub2, "kernel" is replaced by "linux"):
    kernel /vmlinuz-3.1.4-1.fc16.x86_64 ro root=/dev/VolGroup00/LogVol00 rhgb LANG=en_US.UTF-8 crashkernel=128M
  3. Next, consider editing the kdump configuration file /etc/kdump.conf. This will allow you to write the dump over the network or to some other location on the local system, rather than to the default location of /var/crash. For additional information, consult the mkdumprd man page and the comments in /etc/kdump.conf.
  4. Next, reboot your system
  5. Finally, active the kdump system service
    /sbin/chkconfig kdump on
    /sbin/service kdump start

Considerations:

  1. Above shown parameter reserves 128MB of physical memory. This reserved memory is used to preload and run the capture kernel.
  2. Init scripts take care of pre-loading the capture kernel at system boot time.
  3. It is recommended to either set up a serial console or switch to run level 3 (init 3) for testing purposes. The reason being that kdump does not reset the console if you are in X or framebuffer mode, and no message might be visible on console after system crash. You may also see screen corruption in graphics mode during capture.
  4. Capturing a crash dump can take a long time, especially if the system has a lot of memory. Be patient. The system will reboot after the dump is captured.

Step 2: Capturing the Dump

Normally kernel panic() will trigger booting into capture kernel but for testing purposes one can simulate the trigger in one of the following ways.

  1. Trigger through /proc interface
    echo c > /proc/sysrq-trigger
  2. Trigger by inserting a module which calls panic().

The system will boot into the capture kernel. A kernel dump will be automatically saved in /var/crash/<dumpdir> and the system will boot back into the regular kernel. The name of the dump directory will depend on date and time of crash. For example, /var/crash/2006-02-17-17:02/vmcore.

Step 3: Dump Analysis

Once the system has returned from recovering the crash, you may wish to analyse the kernel dump file using the crash tool.

  1. First, locate the recent vmcore dump file:
    find /var/crash -type f -mtime -1
  2. One you have located a vmcore dump file, call crash:
    crash /var/crash/2009-07-17-10\:36/vmcore /usr/lib/debug/lib/modules/`uname -r`/vmlinux
Note.png
Missing debuginfo?
Cannot find any files under /usr/lib/debug? Make sure you have the kernel-debuginfo package installed.

For more information on using the crash tool, see #More Documentation.

More Documentation