Does Anyone Like GridBagLayout??

Does anyone like GridBagLayout? I know I've fought it at times and this animated blog entry is a hillarious parody. Although I don't think parody is the right word since it does seem to mimic reality.

Doom Slayer III

One of the student games in the contest I ran two night's ago did a two minute trailer for his game. It was actually really fun to watch and I thought I would link to it here since he just uploaded it. Most of the content from this video was in game capture. It is a real game that is fully playable, created by a sophmore at Chapman University in about 6-7 weeks. A huge piece of work for that short of time. He is going to be an amazing programmer.

Doom Slayer III
http://www.youtube.com/watch?v=PN5E3E_nIhA

I've already posted it, but the final results of the contest are here.
http://www.gamedev360.com/contest07/

Game Development Contest Results

Last night's contest for my class was amazing.

http://www.gamedev360.com/contest07

The RailGunner game is an amazing piece of work built in about six weeks. Jess and Noah really outdid themselves. Even the second (T-Orc) and third place (Doom Slayer III) games were very nicely done. Rail Gunner and T-Orc are fully playable. Doom Slayer III while not completely done had a ton of code to it. He even did a trailer that was hillarious and everyone enjoyed. I'm hoping he uploads the trailer for others to see.

All in all I think everyone enjoyed themselves and I think a number of the students will get jobs. There was enough time at the end for the industry folks to mingle with the students and I'm sure a number of them will get calls.

One of the things that became apparent and I had sort of forgotten about it was the fact that half of my class were seniors and half were freshman. I went really fast in the first few classes knowing the freshmen were out of their experience level and hoping they would drop. A number of them did, but a few of them stuck with it and while they didn't have winning games they did have some impressive pieces of work. Although I should mention that Jess (one of the winners) is only a sophmore. Brilliant.

One of the crowd favorites was a freshman game called Beer Sweeper. It was fun to watch as Luis kept raching around the virtual recreation of an actual dorm room (it was very close, I once lived in that same dorm) the keys started to change and it became very hard for him to control. It got some good laughs.

The fourth place game Ork Fight had a ton of AI code and Chris got the orcs to fight on teams with the other players. It was very fun to watch. He had some funny stories about problems while he was working on the AI. Some of the problems sounded almost human.

The fifth place game was a huge endeavor. Given a few more weeks I think Vinnie would have taken first place. He's going to be the Peter Jackson type some day. Double majoring in Film and Philosophy and a minor in computer science he made some major engine modifications and is well on his way to a significant creation.

All in all it was a great semester. I worked way too hard at it and didn't make much money, but I love doing it and I'm looking forward to at least taking a semester off. I'll decided later in the year if I'll do it again.

Tonight's Contest

Tonight is the night I'm hosting a bunch of local game industry folks to view the student projects and presentations. The best game get's a $100 gift card, but the students have done most of the work with the opportunity to get in front of a bunch of folks from industry. It's an experiment, but right now the wave looks a lot bigger than I ever expected.

This is how judging will be weighted. It doesn't have anything to do with their grade on the project, just who 'wins' best game.

Completeness: (25 %)
Replay/fun factor (20 %)
Uniqueness (15 %)
Story (10 %)
Graphics (10 %)
Simple graphics that are used in a unique way will also receive high scores here.
Audio: (10 %)
Student Choice: (10%) (max 90 points = 18 students x 5 points each)

I've tried to break completness out into the following sub items: Splash screen, main menu, Scoring, Game has a story, Game controls/play, multiple levels, game ends/menu cycle/quit, high score system.

I have this obscene spreadsheet created which will automatically calculate the winner based on judges input. Once I have the number (1-100) from the judges for each game and category I simply imput them and come up with a winner. I did this last year, but I put the numbers in as I went. I just hope it doesn't take forever to enter the data, but things have a way of working themselves out. I'll report again tomorrow with the winners.

Just as I finish writing this I may dump the spreadsheet and just ask for the judges first, second and third choices for the winner. Hmmm.

Game development class - huge finale

If you've been following my blog you know I've been teaching a game development class. The student's have been working on their own creations for almost half the semester and here is a (two week old) preview of the results.

http://www.gamedev360.com/contest07/preview.html

I have about 10 people from five of six of the biggest game companies in the area coming to judge the final presentations. I offered the student with the best game a $100 EB giftcard. Last year they were focused on the gift card, this year I'm sure they are focused on the job so putting them together with people who are looking for talent seems like a win win for everyone. All sides are excited.

I went into the lab on Saturday to help students with last minute problems and I have to say this is one of the best group of students I have ever had. The quality of the work is phenominal. The screenshots on the contest preview are almost three weeks old and the games since then are looking very very polished. A couple of them are going to be standouts.

There was the funniest quote the other day in one of the extra lab sessions. About halfway through the class I stopped assigning homework and allowing the student's to work on their own projects. One of the student's said to me, "Man, when you stopped assigning homework this class got a lot harder!". That has to be one of my all time favorite teaching quotes. The pressure of the final presentation in front of industry seems to have really worked.

I'll try and post results of the contest on Wednesday. I've been spending every minute of my free time on this contest. What am I going to do with all that extra free time?

Torque Development Process

I find that I like the workflow in developing games in Torque. As long as you are just using the base engine without any c++ changes to the engine itself it is as simple as keeping the engine exe file in the same directory tree as your script code and just running the engine for each test pass. The engine will automatically compile your scripts to bytecode if the source has changes since the last compile. Once you are ready to release you simply copy everything to a release directory and remove all the source.

I'm betting you could do 99% of your game this way unless you really chose to do something that was out of scope for the actual engine. In that case you would either need to choose a different engine, or make a lot of mods to the C++. In any case, I do like the workflow in developing for Torque. Even those mods are pretty simple.

It is a little strange since I remember at first thinking it was crazy that I had to re-run the engine after every change. It just seemed to go against the grain of what I'm used to, but now that I think back on it, I do tend to re-run everything no matter what platform I'm developing for. Maybe it was just that I wasn't used to the engine actually doing the compile and expected a traditional environment of having to compile before running? In any case, it is a good system.

Torque Introductory Tutorial

I missed class last week because I've been sick. I'm feeling better now and I've been working on this week's lecture. I found that when I got stuck on one of the examples I was building I immediately went back to the introductory tutorial that comes with the engine. It's file name is GettingStarted.pdf and it is in the examples directory. This really is a great place to start and I find myself returning to it for quick refreshers. I found one online here, but I don't think it is supposed to be there.

Anyway, if you are going to use Torque, keep this tutorial handy.

The student's marketing materials are due tonight. Just a simple screen shot and 1 page of html. Just enough for me to get something out to the judges so they can get a preview of the games they will be judging.

Apples To Apples - Fun Party Game

It's Thursday, time for some fun. When I worked at a game company we used to have Thursday game night every week where we would turn off the computers at 5 and head to the conference room for some old fashioned gaming. We were a small group (about 8) and had two exceptional game designers who also had large collections of board games. I'm going to take a quick diversion from game development each Thursday and talk about some of my favorites.

One of our favorites was a party game called Apples To Apples. It had just come out and is now available everywhere, but I'm sure not everyone has heard about it now? Very very simple to play and a lot of fun and laughes for a group of 4-10.

The basics, one person (let's say my friend Hank) plays a random green card face up. It says "annoying". Everyone else has to play a red card (face down) from their hand that they think Hank will choose for the word 'annoying'. You have a hand of 8 preprinted cards and the choice of card you play would probably be different depending on which person in the group is choosing. The cards in my hand might have things like. "Marilyn Monroe", "My Social Life", "Hockey", "Celine Dion", "Tornados", and "Strawberry Pie". After everyone has played a red card from their hand face down, Hank might end up with cards like: "Craying babies", "The NRA", "My Haircut", "A bar room brawl", and "Celine Dion". (you can see which card I chose. feel free to leave an angry comment. Even though I think the NRA is annoying that person doesn't know Hank is actually a gun lover). Hank turns them up one at a time and reads them out loud. After, he picks the one he think matches "annoying" the best. Now, if Hank thinks the way I think he should think, he'll choose "Celine Dion". If he does, I first give a Homer Simpson Whoooo Hooo, then I get to keep the green card. First one to some number of green cards (typically 4 or 5) wins. See, simple.

Many of the card combinations are hillarious and you'll have a great time with any group. A very fun game and did help build cohesion within our team.

Is Torque the best game engine for my project?

I was asked in an e-mail if Torque would be the best engine for a specific project. I wasn't given much information on the project, but this is my opinion on the matter.

For commercial games Unreal is the king and owns about 90% of the market. It is in the $100-200k range to license so it is out of reach for anyone that isn't very very serious. I'm not completely sure on the pricing because it's something that isn't advertised. They price it on a case by case basis and we tried to get a license for Chapman University and that was the range we were given.

The next most popular engine for commercial games is a tie between Quake and Half Life. Both have similar licensing expenses, but rumor has it that the Half Life engine is very hard to understand. I don't know first hand. Of the three I think Unreal is probably the easiest to learn having spent some time with it. Torque has a very strong following, but is mostly seen as a hobbyist engine. That doesn't make it bad by any means, but that is the view. It is well supported and while there is a steep learning curve, it isn't any steeper than the other engines out there. There are a number of commercial games available on it and the company just released a next gen version of the renderer so it is still growing and doing well.

The next thing to keep in mind is the game itself and if the engine supports the type of interface you are designing. Each engine has different specialities and one engine may be best for one design and wrong for another. Torque is best at FPS or third person multiplayer games. There is also a 2D version of the engine which I would also highly recommend.

Torque Game Engine (3D) does not have a good physics engine. I think the others are better, but there are a lot of threads out there on the Torque forums that talk about integrating other physics libraries. The 2D version has a great physics library.

Game Development or Game Playing

Not all game development is without it's fun side. One of the main reasons I'm into game development as a hobby is that I've loved to play games since I was a kid. It all started with that pong game my dad bought when I was about 9. The console wars have been going on ever since.

My current console is a ps2 which I have not played in a while. I was playing a lot of game boy until I gave it away to an orphanage in Africa when we were on a mission trip. I do miss it, but those kids really loved it a lot more that I ever would.

My current game is World Of Warcraft (Character Ironwood, Realm: Duskwood) One of the most addictive games I have played. Probably the most addictive for me since I got to 67% complete on GTA III, Vice City and San Andreas. Anyone who has played those knows 67% complete is too much fun time and not a lot of TV. I'm not sure where I stand in WoW, but I'm sure it's only in the 20% range at level 31. Lately I've slowed my progress and have been working on my Leather Making and Cooking skills. The game is very open ended and has a lot of ways to play. My only complaint is that I'm a bit bored with the same monsters over and over and over with a slightly different skin or model, but it's still a lot of fun.

With 8 million subscribers and still growing don't take my word for it, this is a fun game. Don't start if you don't have some time to waste. I started in January and three months later I'm finally starting to get a little bored with it. I'll probably drop my subscription in June if my usual game life cycle runs it's couse. A very fun game.

Torque - Torsion IDE or Eclipse

I've been trying the Torsion IDE for Torque Game Engine this week. Before I was using TextPad or CodeWright and just managing the projects by hand. It wasn't a big deal before, but I did have to find the error lines by hand.

The biggest difference is that the error output has a clickable interface and keeping the errors red makes it much easier to find the errors in the log. My only gripe so far is that the log window is not searchable which is annoying, but it has really sped up my workflow in so many other ways that I think I'll probably buy it when the 20 day trial is over.

I've seen some mention a that there is an eclipse plugin for Torque. I've been a long time user of NetBeans. I typically find an environment that I like and stick with it until it's either irrelevant or something very compelling gets me to switch. I'm doubting the Eclipse plugin is as closely tied to the Torque engine (it has a built-in debugger I have not yet tried) as the Torsion IDE and wonder if the error logs are hot linked to the source lines?

Has anyone tried the Torque plugin for Eclipse and have any insight? It might save me some time and frustration of learning yet another IDE although that will probably start some sort of flame war with people pitching one IDE against another. My view is that the tool that works for you/me is the best tool and it's just not worth arguing over.

Torque Script - datablocks - very confusing

It took me quite a while to really understand datablocks in Torque. All the books say exactly the same thing.

Of all the features in TorqueScript, Datablocks are probably the most confusing. To make things worse, they are central to the creation of most objects, which means you need to understand them relatively early.

I think all the books plagiarized this from the official documentation on datablocks.

Datablocks are just shared objects that contain static data that doesn't change from one object to the next. So the engine can send it once over the network and reference it by id from then on. I get this and I think every explanation made this perfectly clear. The problem is that all the callbacks from the engine into the script code comes through these datablocks. For instance:

datablock StaticShapeData(SomethingData)
    {
    // Basic Item properties
    shapeFile = "./mymodel.dts";
    };
function SomethingData::onCollision(%this, %obj, %col)
    {
    // your code here
    }

This really confused me and I was trying to create my own object types to no avail. The code above creates a type SomethingData which will show the 3d model contained in the file mymodel.dts. This is fine, but the call to onCollision is done on the datablock object (SomethingData) versus the object itself which in this case ends up being a StaticShape. Since the datablock is static and shared across all instances I couldn't figure out why or how you kept data in individual objects.

I finally had an epiphany when I noted that %obj is always passed into the callbacks and is the actual object instance. Individual changes and values can be made in this object. Seems simple enough now, but this killed some brain cells for me.

So I placed this comment in one of my pieces of sample code for class.

function SomethingData::onCollision(%this, %obj, %col)
{
  /*   * MANTRA - REPEAT OVER AND OVER UNTIL DATABLOCK UNDERSTANDING SETS IN
   * 1. %this is the datablock   * 2. %obj is the object instance
   *
   * of much less importance
   * 3. %col is the object colliding

Here is the lecture where I tried to explaing this. I think the notes are still a little jumpy, but the in person version seemed to clear things up for students. Your fuel economy may vary.

Torque Script - Behind The Scenes

Torque script is the scripting language behind all of the Torque Engines is a simple, but robust scripting language that is very easy to learn. All game engines include a scripting engine and Torque is no different. For non-game developers, the idea of a scripting language may seem like a negative, but there aren't any commercial games that don't use a scripting language.

Most non-game developers may at first think that this means the source to parts of your game are shipped with the game, but this is not the case. Torque script is compiled into a bytecode format and saved to a separate file (.dso extension). Everytime the game engine runs it does a date comparison between the source (.cs) and the compiled files. If the source is newer, it re-creates the bytecode file. If the source file does not exist, it simply uses the bytecode file. You simply ship your final version without the source files.

This is transparent and very fast. When you are developing games using this system you never see the compile taking place. One drawback of this system with Torque is that it isn't blatent when compile errors happen and you have to go back and look in the logs when you've made a bunch of changes. In most cases I've seen it will simply use the older .dso file and you'll be caught wondering why that echo/printf you just put in the code isn't appearing. As with most tools, once you get used to it, it is not really a problem. There is an editor called Torsion that will check/monitor your syntax, but I have not tried it. Everything I've read says it is highly recommended.

Torque Game Builder - TGB

Torque Game Builder (TGB) is another product I used in my game development class. It is a 2D version of the game engine that uses the same scripting language as the Torqe Game Engine (TGE) which is 3D.

Torque Game Builder was much more polished as a product, but it doesn't include source with the indie or edu license which is a disappointment, but not a good reason not to use it. This class was supposed to be a 3D class, but I did use the TGB for two weeks at the beginnning of class to give students an easier introduction to the scripting language. It was a lot easier to deal with the scripting language without the added third dimension. Once you add that third dimension everything gets more complicated.

Another big difference between the two is that TGB has a much better physics enginee. The physics engine in TGE is not as robust, but it does work for most FPS style games. The TGB physics engine has all the bells and whistles you would expect. As with TGE, TGB includes a multiplayer networking engine at it's core. Even single player games would have the networking code attached at the core. It's not a lot of over head, just the design for multiplayer at it's core. If you are going to develop a multiplayer game I do suggest buying the full license ($495) so you have the source and can run multiple clients on the same machine during development.

I think the TGB is very easy to learn and a great platform for building 2D games. For $100 there is no comparison.

Motivating students with a contest

A $100 EB gift card prize goes a long way in motivating a group of students. Sure it costs me a large percentage of the money I make teaching the class, but if I cared about the money I wouldn't teach the class in the first place. Adjunct teaching doesn't pay, at least not for me. I love to teach and the little I do make is just an added bonus.

I've done the contest once before and the work was amazing. This year I've upped the ante and invited a bunch of execs and hiring mangers from local game companies to be the final judges. This will be a great opportunity for the students to show their talent and for these game company folks to hire amazing young talent. It also motivates the students to spend extra time on their projects which is the only way to learn to program. All sides are primed and ready and it seems like the perfect matchup so far.

We still have 6 more weeks of development time left for the students, but after two weeks officially working on their final projects I am throughly impressed with the results. This is going to be much better than last year.

For those that are worried about it, the contest placing has no bearing on the grade for the students' final projects. I'll grade under the same rules as the contest, but that grade is separate from contest placing.

Broader individual knowledge helps the entire team

More thoughts on the 3d game development class I'm teaching.I'm about halfway through the semester and I just stopped giving 'busy work' assignments. Everyone is working on their own projects. The last busy work assignment I gave I introduced them to the installer. My choice of installer is NSIS, since it's free and something I have used on other commercial projects.

What does this have to do with a Game Development class? A ton. From the very first busy work assignment I forced my students to think in terms of a full cycle game. Everything they turned in had to have a splash page, a main menu and a way to cycle back to the main menu. The installer extends this concept of a full cycle game. Isn't this what they are going to deal with in the real world?

Most gaming books seem to only focus on the mechanics of the actual games, but to me that is only about 30% of the overall work that needs to get done. The real work is in the details and those details really matter. As an added bonus, once this class is over each of my students will have an full game with an installer they can give to a potential employer or put on a portfolio website. These students will have a distinct advantage over someone that just says they took a game development class.

To me this has a lot to do with getting a job, but it also has to do with the reality we all face. It is okay to specialize in one area of a project, but you still need to understand a little about the entire process. If everyone has broader knowledge the team will function more efficiently and increase the value of the products you create. Thus increasing the value of the individual to the team. I'm trying to train students I would want to have on my own team.

Blender. Free 3d modelling tool

As I mentioned in the previous post, I'm using Blender for modelling in my 3D game development class I'm teaching. I think Blender is great and the best value for the price (free!) of any modelling tool available. That said, it isn't for everyone. I did a full lecture on Blender and found that my love of Blender is related to my love of the text editor vi. In fact I used this opportunity to give the students a free introduction to vi. If you hate vi and think it is the worst editor ever, I guarantee you will not like Blender. If you like (or simply accept) vi then you will probably find that once you know about 20 key combos you'll be proficient with all things Blender.

Once I learned those key combinations I found that it is the fastest modelling tool I have ever used. I actually love the interface. I still find some process require a revisit of the documentation (especially for animations), but once I have them down it is fast.

The best way I found to learn Blender is with this set of Blender video tutorials.

The next best piece of documentation is this Blender Wiki book.The third place is the official Blender documentation.

If you have some extra time on a new modelling project give Blender a week and I think you will be hooked.

3d game development tools and Torque

The tools we chose for the game development class were driven mostly by the choice of books, but many of them are cheap/free and industrial strength as well and I would use most of them on my own projects even if I had some sort of budget. The engine we are using is Torque, but all these tools would be useful no matter which engine you use.

PaintShopPro is a great paint program. The first edition of the book used this tool, while the second edition uses Gimp. Since I already had a number of students in an earlier class that had used PSP I chose to stick with it. Also, it is my favorite paint program and in my opinion better than Photoshop for everything except CMYK. I've done a ton of image and print work over the years and PSP is an amazing value. Corel bought it from an independent developer a few years ago and I've been expecting a $400 price increase. I'm sure I'll grudgingly switch to Gimp if that happeens.

Milkshape is the tool used in the book and we made sure it was available for the students. It's not the best 3D tool by a long shot. It is missing a lot of features which also makes it a lot easier to learn. A double edged sword, but still a great value for $25.

For texture mapping you have to used a second tool called UVMapper. It's free and easy to use.

QuArk (Quake Army Knife) (free) is the tool we used to create interiors. Interriors are BSP trees that have a lot of properties that speed up rendering and collision detection. The cost is that the tools that do BSP are all very confusing and QuArk is no different. If you use QuArk for Torque, make sure you read the PDF tutorial that comes on the Game Programming All In One CDROM (it's not printed in the book any longer). It is a great tutorial that will take a lot of the guess work out of a confusing UI. Also, make sure you download QuArk from the Garage Games site and not the one main site as it's already configured for Torque. The Unreal editor is a BSP editor and it is just as confusing. Just because you pay more you are not going to find a good BSP editor.

Audacity (free) is not my first choice of audio tools, but my first choice (CoolEdit by Syntrillium) was purchased by Adobe and the price raised by $400. I'm not one to pay hefty fees for software and have found that I can live with Audacity. I'm going to be really mad if Corel does the same thing with PaintShopPro.

Blender (replacement for Milkshape) is my last tool of choice and it is a controversial one. You either love it or hate it. Blender was a commercial 3D editing tool that moved into the open source community driven world and is well updated and supported. It is absolutely free. That's not the controversial part. You either love it or you hate it as the UI takes a little getting used to, but once you have it down it is extremely fast. The commercial equivalent is 3D Studio Max and is used by 98% of the game studios. My only problem with it is the $3500 price tag. I'll do another blog entry just on Blender.

That's my list of (cheap/free) software to use in 3D game development.

Books to use with Torque

In choosing books for the game development class I am teaching we really only had three choices for the class. I don't belive I made the best choice with the limited time I had to choose and would choose differently now.These are the two books I would choose for the class if I had to choose again.

3D Game Programming All In One and Advanced 3D Game Programming All In One.

These two books compliment each other very well. The first book is more oriented towards a general overview and the tools to use and the second one has a much better focus on Torque Script and programming.

The other book that I would leave off the list is:The Game Programmers Guide To Torque

I thought this book would serve as a better reference, but it seems to be a wiki that was turned into a book and isn't detailed enough as a reference, or descriptive enough as a book. It jumps around too much and doesn't help give a clear understanding.

Why use Torque as a game engine?

As I said in my previous blog entry I've been teaching a game development class using the Torque Game Engine. There were a number of factors that lead to the choice of Torque.

The early meetings with the curriculum board lead to the adoption of Unreal as the engine of choice. As the fall arrived and I had been spending months working with Unreal, reality began to sink in. Since this was a programming class there would have to be more to it than just level editing. While Unreal script is programming I also wanted to be able to have the students understand the actual C code behind the engine. Without access to the engine source this would mostly be a level editing class. The Computer Science department wanted to make sure this was more of a programming class than a level editing class.

In general, there are always C code tweaks to an engine to make any game so anything short of source code would not be a realistic environment for the students. Workable, but not realistic.

When the department attempted to license the Unreal engine it was out of the budget. Rumor had it that it was in the 100s of thousands which was way outside the budget for an edu setting! Another factor was the majority of the official documentation for Unreal requires that same license. While there is a lot of independent documentation on Unreal and this may not be a problem it was a factor in our decision.

Torque had come up in earlier discussions and a review showed it was actually perfect for the educational setting. At around $100 a seat, it included the source. That decision was made in December and I've spent the past few months trying to stay ahead of the students while I learned the ins and outs of Torque. Stressfull, but I think the results I am seeing from the students are amazing.

Torque Game Engine

This semester I've been teaching a class on game development at Chapman University. The class website is http://www.gamedev360.com/. It is the biggest reason I have not blogged in a while. Secondary was my last post on procrastination which sort of helped me see blogging as a distraction. We are using the Torque Game Engine for the class and so far I am impressed. There are some things I really like about it and some things I don't and I'll come up with some posts along those lines in the future.

My previous gig was called GameWorld.com developing mobile games. During that time I was teaching some other classes (Assembly Language, Computer Architecture) and lobbied the head of the department to teach a game development class. After five years, that finally came through last year and I taught the first class. Last summer the department created a curriculum board of industry professionals to come up with a minor and certificate program. This semester is the first class as part of the minor. Very exciting.

The website for the current class is http://www.gamedev360.com/. Please feel free to comment or contact me about Torque. I'll be writing more about Torque and game development in the future.

Procrastination Study

I read this really interesting study on procrastination on Friday. It started by reading this article about a decade long study of procrastination.

http://biz.yahoo.com/ap/070111/procrastination_nation.html?.v=1T.

Then I went to the source and did a little more reading on the authors website.http://www.procrastinus.com/The best part was this page that talked about the various theories. It uses frames so the link looks wierd, but it is the same site.

http://webapps2.ucalgary.ca/~steel//Procrastinus/theories.php

It really changed how I worked on my weekend. There were a couple of tasks that I had been procrastinating on and this really convicted me. Now I have both tasks done and I feel much better. It seems the Delayed Expectancy Theory is the one that really made a difference to me. Those tasks were just drudgery with little reward and I just needed to get them done without putting other quick reward projects first. Now that they are done, the actual reward of relief is surprisingly high. With more analysis, I'm finding a lot of distractions fall under this quick reward category he talks about. For starters, I turned off the ringer on my incoming e-mail.

I hope you enjoy the reading. It really made a difference for me.

I'm Never Wrong...

I'm never wrong when I'm sleeping.

Is using a title about operating systems going to get anyone to look at this? If I'm going to talk about the future of operating systems there is one thing I've learned. Marketing comes first. And besides, I'm wrong the rest of the time so you may as well stop reading.

The last blog entry ended with a comment a few minutes after I wrote it that we need to look at operating sytems more as a video feed than a finite state machine (FSM). The root of the problem goes back to my hero Alan Turing. Did I know I was going to mention Turing when I started this? No way. I'm just typing and the last thing I would ever do is question his absolute brilliance, but I may be wrong in saying I would never question it since I'm not sleeping. On a side note, anyone read Cryptonomicron? Too long, but a fun book.

Finite State Machines work great to compute a result, but we no longer use computers to simply compute results. They run infinitely have become more of a linear medium than a finite one. The results being that our management tools were all designed to look at an operating system as snap shot of a single picture while in reality they are highly complex video cameras. Why are we still stuck using one dimensional tools on a linear two dimensional object?

The answer is that our tools were always limited by the resources we could bring to bear on the problem. That and the fact that the godfather of our industry (Turing) was only interested in breaking codes during WWII. He only cared about a single end result and once the ticker tape stopped, the tape WAS the result. Our resources were limited in that we/he always had tiny pieces of tape (memory and disk) to process and store everything (overwrite, overwrite, overwrite). We're stuck in a design from 1950 and our complacency is the root of the problem. So the big question I have is: What are you doing with that 500GB hard disk in your everyday computer? Unless you are a videographer, you probably wasted your money and have only used a few gigs.

I'm not saying we need to throw out the FSM at the root of our industry, but maybe it's time to rethink the one-dimensional tape. Did I lose you? I feel like I lost myself, but let me try and explain. I'm talking about a Temporal Finite State Machine. There seems to be an opportunity to build a new type of operating system that relies heavliy on readily available massive amounts of storage space to journal everything (copy that tape) and allow for much tighter contols on security and increased reliability with the ability to diagnose problems (non-linear operation) or changes that have occured in the past instead of only looking at the operating system as a single image of the here and now. Everyone is groaning right now with only thoughts on performance and I agree, performance would be an issue, but this is fantasy land and I'm just brainstorming and you are just along for the ride. I also work at FileNet and we have all kinds of crazy ideas for storing and organizing massive amounts of information so it's not a complete fantasy. I at least hope I stretched your brain a little. I'll write more on this.

Temporal Finite State Machine.... Hmmmm... This is going to kill some brain cells I can tell.
When am I ever going to get to the point and finally talk about why source code causes most of our problems? For now I think I'll just continue down this rabbit hole.

Computers are too complicated

I left my last blog post convinced that computer viruses are allowed by too much complexity, not the cause of some inept software developer. I love to complain about computers that don't work just as much as the next person, but I'd rather help make a change than just complain. This is half ramble and a creative endeavour. I'm just throwing this out here as it leaves my fingers. Brainstorming is never wrong, it's just a creative tool.

To repeat my earlier stance, computers (operating systems) have become too complex for the tools we use to monitor them. Would we run a nuclear power plant with gauges strewn all over the entire operation? No. We bring all the gauges and controls into a central control room. This is the problem with todays operating systems. We've strewn the gauges all over the place and made them next to impossible to read. I've been working with computers for a long time (I graduated computer science in 88) and I still can't tell you the health of a computer when I walk up to it. I'm not even sure I could tell you the health after a few hours of looking at it. There are tools out there that help with this (Registry Mechanic on Windows comes to mind), but even those don't give much information, they just fix and forget.

Where is the information that we need better access to to monitor the health of a computer? Three places. 1) Archival storage. 2) RAM 3) Process status.

What are the tools we use to monitor these today? 1) File managers (nothing much more complicated than ls). 2) vmstat is the only thing I can think of and that is a crude as it gets. 3) ps (Windows ctrl/alt/del, process tab) and that doesn't give any historical information about the history of the process.

What would it take in those three areas to find a virus just by looking at a proper monitoring tool? (This is sort of fun and I feel like I'm on to something as I start to visualize a new tool that monitors a running operating system). I worked at Sun in the early 90s and Rich Pettit's setool comes to mind, but even that was way too complicated, but it did bring together a lot of disparate data. That's the sort of thing I'm trying to visualize.

You look at a list of process that are currently running. You can see a graph of individual process cpu time and memory usage since it started. You click on the file that is running from that same view and see the change history and mechanism (human or computer) that changed the executable. Maybe some sort of finger printing to see who or what actually made the change and when (Journaling disk drives in VMS were really cool, but never became mainstream). You could see the processes interaction with the network over time and actually drill down into the network traffic to view the traffic as a video stream. You could click on the current memory and see a map of memory and drill down to see what's using what and how much of it. Maybe a sysadmin could see the actual contents of memory with tools to view different types of memory in different ways (this is a big stretch, but part of brainstorming). From the archival side of the house you could look at an executable on disk and see when and where it was run over the past days and what process was starting it and even drill down into those processes as well. Just a bunch of random ideas and I'm sure there are more where those came from.
Given a better view of the internals and history of an operating system, would a layman be able to detect a virus immediately? Given enough journaled information could you reverse the effects of a rogue process? Given a lot of centralized gauges, can an engineer decide when a nuclear power plant is going to melt down and what to do? I say yes to all of these, but I'm not asking normal people to be nuclear engineers. With the current state of the art even the engineers are blind when it comes to an operating system. Even a nuclear power plant's control rooms have alarms and I doubt most of them know what every gauge and switch does, they probably have a big manual and a big panic button. But I'm not talking about the dangers of a nuclear plant. I'm talking about an operating system and the problems we face today. Not enough information about it's current and historical state to be able to diagnose a problem or to know if a change was critical or malicious.

I never even got to the actual root cause. The real problem lies in how we program computers and the complexity of programming languages. Maybe I'll get to it in another entry.

Anti-virus software and software reliability

Last night I had to purchase and install Windows XP on a laptop that had Windows 2000 on it that had become corrupted with no media. I'm getting off on the cheap and yes I like to save money, even though I find it completely insane to buy a copy of xp only days before Vista ships, but that's me and that's a different story. After the install, I start the patch, reboot, patch, reboot, patch reboot, patch reboot cycle. I've only been through two so far, but it is SP2 out of the box so two is one too many for me.

Anyway, I keep getting the message pop up that says my machine may not be secure because there is no anti-virus software. I usually think nothing of it, but last night it sort of got to me. I've been paying $10 a year to Symantec forever because the software I use is insecure out of the box. Insecure out of the box!

I drove the car off the lot and I need to go to pep-boys to get an a key system for it.

I bought a brand new house, but I have to go to home depot to get door locks before my family moves in.

I bought an airline ticket, but I have to buy a seat belt in the terminal before getting on the plane.

I'm going on a cruise, but I have to bring my own life jacket.

I ate at Taco Bell, but I may get salmonella. Oh, I guess it is similar, in that one's a virus and one's a bacteria...

I know this isn't just a Windows problem. No OS is completely secure out of the box, they're all hard to use and highly insecure. It's a systemic problem with our industry. An interest in marketing features vs. marketing durability. If anyone finally figures out how to teach the public that features aren't as important as reliability then we'll finally be on the right track.
Maybe there is a lesson here from the car industry? Maybe the Japanese will come along and create Hondalinux or Toyatlux and finally show us that reliability and usability is 1000 times more important than a glossy paint job and slick sales rep.

There has to be a better way to program a computer than cryptic source code and that is another blog entry.