Silviocesare’s Weblog

gdb leaves file descriptors open in debugee

May 13, 2008 · 6 Comments

I have my emulator running reasonably successfully on upx now.  It’s actually an auto unpacker, and identifies when the program is unpacked by monitoring execution on previously written memory.  In the process of emulating file io I came across a particular bug in gdb.

The file descriptor returned from an open call inside the debuggee, was 6.  I was expecting 3.

stdin=0, stdout=1,stderr=2

gdb must be using file descriptors 3,4,5, and forgot to close them before calling execve.

I’m not sure what the descriptors are used for.  Anyone care to take a look?

In the best case scenario, this bug can be used for another test to see if a debugger is present, and in the worst case if these file descriptors were used for control, *gasp* control gdb?  Probably they arent used for anything important, but I havent looked any furthur..

Categories: C Bugs · Reverse Engineering

6 responses so far ↓

  • Thanasis K // May 18, 2008 at 12:23 am

    If this indeed is a bug, be a sport and submit a bug report :)

  • Lee Hinman // May 28, 2008 at 6:18 pm

    Can you detail how you reproduced or found this problem? I was curious so I tried with the following script:

    #!/usr/bin/env ruby
    puts “STDIN: #{STDIN.to_i}”
    puts “STDOUT: #{STDOUT.to_i}”
    puts “STDERR: #{STDERR.to_i}”
    sleep(20) # give time to attach with gdb
    i = 3
    while (i ruby fds.rb
    STDIN: 0
    STDOUT: 1
    STDERR: 2
    trying fd #3..
    fd #3 closed.
    trying fd #4..
    fd #4 closed.
    trying fd #5..
    fd #5 closed.
    trying fd #6..
    fd #6 closed.
    etc etc etc all the way to 20

    I’m very interested in how you found it and if it’s reproducable.
    (I tested on Ubuntu 8.04 and Mac OSX)

  • Lee Hinman // May 28, 2008 at 6:20 pm

    Hmm… last comment doesn’t like characters in the script, the script is here: http://pastebin.ca/1032349

  • silviocesare // May 29, 2008 at 5:10 am

    I’m not sure if your test will work, I would presume attaching to a process from gdb wont work, and the bug will only reveal itself when the target is run from within gdb (presumably its because the bug is some missing close’s before the execve). Also, running from a script will probably cause some problems too.

    This bug is in my testing of Fedora Linux 8, and also Debian etch.

    gdb version 6.6-45.fc8rh
    gdb version 6.4.90-debian

    $ cd /tmp
    $ cat test.c
    int
    main(int argc, char *argv[])
    {
    int ret;
    ret = write(atoi(argv[1]), “hi”, sizeof(”hi”));
    printf(”ret %i\n”, ret);
    }
    $ gcc test.c
    $ gdb a.out
    > run 4
    ret 3

    > break main
    > run 4
    ^Z
    $ ps a|grep gdb|grep -v grep
    28798 tty1 S+ 0:00 gdb a.out
    $ ls -la /proc/28799/fd
    total 0
    dr-x—— 2 silvio None 0 2008-05-29 14:48 .
    dr-xr-xr-x 6 silvio None 0 2008-05-29 14:47 ..
    lrwx—— 1 silvio None 64 2008-05-29 14:48 0 -> /dev/tty1
    lrwx—— 1 silvio None 64 2008-05-29 14:48 1 -> /dev/tty1
    lrwx—— 1 silvio None 64 2008-05-29 14:48 2 -> /dev/tty1
    lr-x—— 1 silvio None 64 2008-05-29 14:48 3 -> pipe:[192216]
    l-wx—— 1 silvio None 64 2008-05-29 14:48 4 -> pipe:[192216]
    lr-x—— 1 silvio None 64 2008-05-29 14:48 5 -> /tmp/a.out


    Silvio

  • Lee Hinman // May 29, 2008 at 3:51 pm

    Interesting, on OSX with gdb 6.3.50-20050815 file descriptors 3, 4 and 5 won’t open (no proc filesystem to check either). On Ubuntu with gdb 6.8-debian, file descriptor 4 opens with a return value of 3, but 3 and 5 return -1, even though 3 is shown in proc’s fd list:

    total 0
    lrwx—— 1 hinmanm hinmanm 64 2008-05-29 09:44 0 -> /dev/pts/2
    lrwx—— 1 hinmanm hinmanm 64 2008-05-29 09:44 1 -> /dev/pts/2
    lrwx—— 1 hinmanm hinmanm 64 2008-05-29 09:44 2 -> /dev/pts/2
    lr-x—— 1 hinmanm hinmanm 64 2008-05-29 09:44 3 -> pipe:[21982]
    l-wx—— 1 hinmanm hinmanm 64 2008-05-29 09:44 4 -> pipe:[21982]

    I suppose it is because fd 3 doesn’t have write permission. Still, I’m really interested in what you can do with these pipes.

  • Lee Hinman // May 29, 2008 at 5:32 pm

    Sorry to be littering your blog with comments, but I did find something similar to this when using gdbserver to debug remotely.

    Using the program from here, which shows all the information read or written on a fd for that particular program:
    http://www.dgp.toronto.edu/~ajr/209/a3/readall.c

    I did:
    gdbserver TCP4-LISTEN:5000 fdwatch
    (fdwatch is the compiled readall.c program)

    Then on the same machine:
    shell> gdb fdwatch
    GNU gdb 6.8-debian
    This GDB was configured as “i486-linux-gnu”…
    (gdb) target remote localhost:5000
    Remote debugging using localhost:5000
    [New Thread 21850]
    0xb7f79810 in ?? () from /lib/ld-linux.so.2
    (gdb) run
    The program being debugged has been started already.Start it from the beginning? (y or n) y
    Starting program: /home/hinmanm/fdwatch
    file descriptor 0 seems to be open
    file descriptor 1 seems to be open(I’ll assume it’s open for write, rather than read)
    file descriptor 2 seems to be open(I’ll assume it’s open for write, rather than read)
    file descriptor 3 seems to be open
    file descriptor 4 seems to be open
    file descriptor 5 seems to be open
    fd 5 says: ild-tree/i386-libc/csu/crti.S00/build/buildd/glibc-2.7/build-tree/glibc
    fd 5 says: -2.7/csu00GNU AS 2.18.00001\37777777600\377777776110000000200]0000000401\37777777655000000 000000/build/buildd/glibc-2.7/bfd 5 says: uild-tree/i386-libc/csu/crtn.S00/build/buildd/glibc-2.7/build-tree/glib
    … etc etc etc etc for quite a bit …

    Perhaps the other file descriptors are being used for remote debugging as well as fd 5?

    Suppose I could start lookging through the source :)

Leave a Comment