Debugging

In the past year I've been given a couple of very tough bugs to find and fix. It doesn't surprise me because while I'm really good at designing and building large systems, I'm also really good a finding and fixing the tough bugs. So I thought I would talk a little about debugging and debuggers.

There was a time, when I left college, and wouldn't choose a IDE for a project unless it had a really good debugger. This was even after four years of doing most of my software on systems that didn't have proper debuggers. This includes a bunch of pc titles using Turbo Pascal, Turbo C, Turbo Prolog and then all of my school assignments on Ultrix (the DEC version of SysV). No one ever told me about dbx on unix so I always used printf to find my bugs no matter what platform I was on. So it does seem strange that I required a good debugger once they started to become more readily available. The question is, did I use them? Well, yes and no.
They tended to work really well for simple bugs in logic, but for the tough problems I rarely found them useful and tended to keep using printf and logging. Especially for some of the more complicated servers I created. Debuggers and remote debugging just seemed too complex and wouldn't deal with threading properly to help me find the bugs I was trying to find. Add to that 100 instances of twelve different games running on the same JavaVM and logging was the only option.

I couldn't even imagine using a debugger to find a race condition in a threaded piece of software (most of the bugs I get lately). The AIX 64 bit device driver port that I fixed at the beginning of the year was crashing the system every time it would fail. I couldn't even get a debugger to latch onto the process before the crash.

So what is the trick? For me there is nothing special about the task of debugging something very complicated. I just start putting in printlns where I think the error is and seeing what the data looks like when things start to go haywire. On the device driver it was quite painful, but I found a way to println into shared memory that stayed active after reboot and I could then read the log finding the last few hundred messages. It was my first task on my new job and I spent three months pouring over logs. Family would ask when I got home. How was work? What do you say after you spent the entire day watching a machine reboot, pouring through logs and not making any progress except making the logs bigger for the next reboot? I teach classes in programming and I tend to say that programming is not difficult, it is just tedious.

As tedious as that was I knew that very tiny steps would pay off and they finally did. There were layered problems, but through the print statements I was able to see that two processes made device requests and only one got a response. Once you see that sort of thing in a log it's pretty easy to find the code that is not being properly locked and fix it.

On another problem I did use a debugger (both on windows and unix/dbx) when I just wanted to see a string variable (sql statement) as it was dynamically built. I guess I could have used printfs, but the debugger was readily available and easy enough to create a break point.
Learning Second Life script and Flash for a class I'm currently teaching, it seems the only debugger is printf (trace() in Flash and LLOwnerSay() in Second Life) which suits me just fine.
I guess the real answer is just to be flexible and use the right tool for the right job and when multiple tools will work just use your gut.

Do you use debuggers of printf?

All Things Software Development

I've decided to create a blog on some of the fun development stuff I'm always working on. This will be an on again off again blog as I go through cycles of development. Most of my fun code writing is for Gaming so this will probably have a strong gaming tilt. I've been working a little with Flash, Linden Scripting Language (lsl/Second Life), and Torque on the fun side. On the business side, it's all about Java, C and C++. Hopefully this is useful for more than just me. I'm planning on using this as a sort of notebook so I can come back here and find code samples and reminders when I'm working on projects that have become a little stale in my mind, which takes about 30 seconds.