====== Debugging the Linux kernel with KGDB ======

===== Preparing a kernel =====
To prepare a kernel for testing, apply a KGDB patch to a linux kernel source. Then enable Remote (serial)
kernel debugging with GDB from kernel hacking, which will enable KGDB code in the kernel.This will enable
choice of a few more config options with change KGDB behavior. These are described below:

  * Thread analysis: With thread analysis enabled, gdb can talk to kgdb stub to list threads and to get stack trace for a thread. This option also enables some code which helps gdb get exact status of thread. Thread analysis adds some overhead to schedule and down functions. You can disable this option if you do not want to compromise on execution speed.
  * Console messages through GDB: With this option enabled, kgdb stub will route console messages through GDB. Console messages from the test machine will appear in a terminal on the development machine where gdb is running. Other consoles will not be affected by this option.After the kernel is configured, build it and and add it to Grub. KGDB stub requires following options in a kernel command line.
  * kgdbwait: This option causes KGDB to wait for a GDB connection during kernel bootup.

  kgdb8250=<port number>,<port speed>

Where port number can be 0 to 3 for ports ttyS0(COM1) to ttyS3(COM4) respectively.Supported port
speeds are 9600, 19200, 38400, 57600 and 115200.

An example of above procedure is shown below.  The steps are split between development machine (the machine from which you're debugging) and test machine (the machine running the kgdb patch kernel).

==== On development machine: ====

**1.** Download the 2.6.21.7 kernel from [[http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.21.7.tar.gz]], and extract it
  $ wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.21.7.tar.gz
  $ tar xzf linux-2.6.21.7.tar.gz
**2.** Download and apply the {{linux-doc:kgdb-2.6.21.tgz|kgdb patch}}. Unzip the kgdb patch and apply one by one as specified below.
  $ tar -zxf kgdb-2.6.21.tgz
  $ cd linux-2.6.21.7
  $ patch -p1 < ../kgdb-2.6.21/core-lite.patch
  $ patch -p1 < ../kgdb-2.6.21/core.patch
  $ patch -p1 < ../kgdb-2.6.21/i386-lite.patch
  $ patch -p1 < ../kgdb-2.6.21/i386.patch
  $ patch -p1 < ../kgdb-2.6.21/8250.patch
**3.** Configure the kernel
  $ make menuconfig
Configure drivers and other kernel options to enable kgdb, enable following config options (in this order).
   Kernel hacking ->
   [*] KGDB: kernel debugging with remote gdb ->
   [*]     KGDB: Console messages through gdb
   Method for KGDB communication (KGDB: On generic serial port (8250)) --->
          ( ) KGDB: Use only kernel modules for I/O
          (X) KGDB: On generic serial port (8250)
   --- KGDB: On generic serial port (8250)
   [*]     Simple selection of KGDB serial port
           (115200) Debug serial port baud rate
           (0) Serial port number for KGDB
**4.** Build the kernel
  $ make
**5.**Copy the kernel to target machine
  $ tar xzf ../linux-2.6.21.7-kgdb.tgz .
  $ scp ../linux-2.6.21.7-kgdb.tgz root@<test_machine>:/usr/src

==== On test machine: ====

**6.** Login as root, go to the ''/usr/src/linux-2.6.21.7'' directory then install the kernel
  $ make modules_install
**7.** Copy the files need to boot and make the initrd image
  $ cp arch/i386/boot/bzImage /boot/vmlinuz-2.6.21.7-kgdb
  $ cp System.map /boot/System.map-2.6.21.7-kgdb
  $ mkinitramfs -o /boot/initrd.img-2.6.21.7-kgdb 2.6.21.7
**8.** Add to grub entry:
  title           <OS NAME>, kernel 2.6.21.7 (KGDB)
  root            (hd0,1)
  kernel          /boot/vmlinuz-2.6.21.7-kgdb root=/dev/sda2 ro splash kgdbwait kgdb8250=1,115200
  initrd          /boot/initrd.img-2.6.21.7-kgdb
  quiet


===== Connecting to Debug Kernel  =====
 
Set the speed of the serial line on development machine to the value you have given to KGDB kernel on the test machine. Start GDB from the kernel source directory giving it the vmlinux file on command line as the object file. Run the KGDB kernel on the test machine and wait till it prints a message:
  Uncompressing Linux... Ok, booting the kernel.
Then connect to the target machine from GDB using target remote command. This command needs the serial line path to be specified as an argument. At this point GDB and the KGDB stub will be connected and GDB will have a control of the target kernel. You can now use GDB commands for inserting breakpoints, printing values of variables, breaking and continuing execution. An example of above procedure is given below:

==== On test machine ====

**1.** Select kgdb kernel from the boot loader prompt, the kernel will start and after doing some initializations, wait for gdb to connect. It will write following prompt on the console:
  Uncompressing Linux... Ok, booting the kernel.

==== On development machine ====

Here we separate the instructions in what to do if the test machine is a real machine, and what to do if is a VMWare virtual machine:

=== Real Machine ===

**2.** Set appropriate speed for serial line.
  $ stty ispeed 115200 ospeed 115200 < /dev/ttyS1
**3.** Start GDB in the same directory were the compiled kernel is locate. It will take some time because vmlinux contains a lot of debugging information. 
  $ gdb vmlinux
  GNU gdb 6.4
  Copyright 2003 Free Software Foundation, Inc.
  GDB is free software, covered by the GNU General Public License, and you are
  welcome to change it and/or distribute copies of it under certain conditions.
  Type "show copying" to see the conditions.
  There is absolutely no warranty for GDB. Type "show warranty" for details.
  This GDB was configured as "i686-pc-linux-gnu".
  (gdb)
**4.** Connect to the test machine using gdb command "target"
  (gdb) target remote /dev/ttyS0

=== VMWare Virtual Machine ===

**2.** Power down your virtual machine and click "//Edit Virtual Machine Settings//". Add a serial port, select "//Use a physical serial port on host//", and provide the pipe name "/tmp/com_1". Repeat step **1.** in the the test machine.
**3.** GDB can't connect directly to a unix-domain socket, so you'll need to use the socat ("socket cat") utility.  If the socket created by VMWare is /tmp/com_1, then invoke socat as 
  $ socat -d -d /tmp/com_1 PTY
Note: most likely it is not installed! please go ahead and install it using
  $ aptitude install socat
**4.** Find the line in the output identifying the PTY that was created (in this case is /dev/pts/2)
  2008/03/26 16:51:51 socat[13859] N successfully connected via 
  2008/03/26 16:51:51 socat[13859] N PTY is /dev/pts/2
  2008/03/26 16:51:51 socat[13859] N starting data transfer loop with FDs [3,3] and [4,4]
**5.** Start GDB in the same directory were the compiled kernel is locate. It will take some time because vmlinux contains a lot of debugging information. 
  $ gdb vmlinux
  GNU gdb 6.4
  Copyright 2003 Free Software Foundation, Inc.
  GDB is free software, covered by the GNU General Public License, and you are
  welcome to change it and/or distribute copies of it under certain conditions.
  Type "show copying" to see the conditions.
  There is absolutely no warranty for GDB. Type "show warranty" for details.
  This GDB was configured as "i686-pc-linux-gnu".
  (gdb)
**6.** Connect to the test machine using gdb command "target" with the port provided by socat
  (gdb) target remote /dev/pts/2

====== More Information ======
  * [[http://kgdb.linsyssoft.com/|KGDB Homepage]]
  * [[http://sources.redhat.com/gdb/current/onlinedocs/gdb_toc.html|Debugging with GDB]]
  * [[http://www.cs.dal.ca/studentservices/refcards/gdbref.pdf|GDB Quick Reference]]
  * [[http://oslab.info/index.php/Misc/KGDB|Using KGDB with VMWare]]