How to use gdb
==============
:toc:
Cyril Brulebois <kibi@debian.org>


Foreword
--------

One should note that X is responsible for VT switching, meaning
switching between an X session and console terminals. In other words,
`Ctrl+Alt+Fn` is handled by X. If X is stopped, for example because
it’s running under `gdb`, one can no longer switch to another
VT. That’s why we’re recommending using a second machine to debug
X. Nevertheless, here are some instructions to attempt debugging X
with a single machine.

One-machine approach
~~~~~~~~~~~~~~~~~~~~

This is a _post-mortem_ approach. The idea is to run X with the
`-core` option. Once it dies, a core file (`/etc/X11/core`) is
generated, and can be loaded from `gdb`.

Follow these steps:

 1. Getting a core file.
 2. Loading a core file.
 3. Displaying/saving a backtrace.

Two-machine approach
~~~~~~~~~~~~~~~~~~~~

You pay the “need a second machine” price, but that buys you
interactivity. Just log into the first machine from the second one,
using `ssh`.

Follow these steps:

 1. Attaching/Starting X from gdb.
 2. Displaying/saving a backtrace.

Needed packages
~~~~~~~~~~~~~~~

Obviously, `gdb` is needed; `xserver-xorg-core-dbg` contains the
debugging symbols for the server itself. Other needed packages can be
determined by looking at the backtrace. **FIXME: More info about
that.**


<<<
Actual gdb work
---------------

Getting a core file
~~~~~~~~~~~~~~~~~~~

 * Using `gdm3` 3.4.1 and above: uncomment `Enable = true` in the
   `[debug]` section of the `/etc/gdm3/daemon.conf` file.

 * Using an older `gdm3` package: The idea is to tweak the daemon’s
   `LocalXserverCommand` setting, adding the `-core` option. As of
   `gdm3 2.30`, the defaults can be found in
   `/usr/share/gdm/gdm.schemas`. Sample `/etc/gdm3/daemon.conf`
   excerpt:

----
[daemon]
LocalXserverCommand=/usr/bin/Xorg -br -verbose -audit 0 -novtswitch -core
----

 * Using `kdm`: One should look for the `ServerArgsLocal` variable in
   the `/etc/kde4/kdm/kdmrc` file, and add `-core` there. Example:

----
ServerArgsLocal=-br -nolisten tcp -core
----

 * Using `xdm`: It’s sufficient to add `-core` to the command
   configured through `/etc/X11/xdm/Xservers`, for example:

----
:0 local /usr/bin/X :0 vt7 -nolisten tcp -core
----

Loading a core file
~~~~~~~~~~~~~~~~~~~

That’s trivial, one just needs to pass both the core file and the path
to the binary:

----
# gdb -c /etc/X11/core /usr/bin/Xorg
----

Now `gdb` is ready to display backtraces.

Attaching X from gdb
~~~~~~~~~~~~~~~~~~~~

The way of starting X doesn’t really matter, as `gdb` makes it
possible to attach a running process. If there’s a single X instance
running, that will do the job:

----
# gdb attach $(pidof X)
[---GDB starts---]
(gdb) handle SIGPIPE nostop
(gdb) cont
----

If there are several instances, one can use `ps aux` to determine the
PID of the appropriate instance (2nd column → `$pid`), and then attach
it:

----
# gdb attach $pid
[---GDB starts---]
(gdb) handle SIGPIPE nostop
(gdb) cont
----

Starting X from gdb
~~~~~~~~~~~~~~~~~~~

In case X crashes at start-up, one can start X from `gdb` in the
following way. In this example, the only parameter is the display, but
more parameters can be added.

----
# gdb --args Xorg :0
[---GDB starts---]
(gdb) handle SIGPIPE nostop
(gdb) run
----

What is SIGPIPE?
~~~~~~~~~~~~~~~~

`SIGPIPE` is a signal that can reach the X server quite easily,
especially when performing a VT switch, or refreshing large parts of
the screen. That’s why we ask `gdb` not to stop when such a signal is
caught, thanks to the `handle SIGPIPE nostop` command.

How to display a backtrace?
~~~~~~~~~~~~~~~~~~~~~~~~~~~

Once X is crashed, for example because it received a `SIGSEGV`
(segmentation fault, usually because of a NULL pointer dereference),
or a `SIGBUS`, one gets back to the `gdb` prompt. One can then request
a backtrace (`bt`) or a full backtrace (`bt full`). The latter is what
developers are usually interested in, because variable values are also
available.

----
(gdb) bt
(gdb) bt full
----

How to save a backtrace?
~~~~~~~~~~~~~~~~~~~~~~~~

To save a recording of the gdb session to a file (`gdb.txt` by
default):

----
(gdb) set logging on
----

To save in a different file, use this instead:

----
(gdb) set logging file my-file.txt
(gdb) set logging on
----

Once logging is enabled, you can request a (full) backtrace using the
previous commands.