MPL Madness

August 31, 2009

The time has finally come. For years now I’ve known that of all my C++ shortcomings, the most obvious is my irrational fear of Template Metaprogramming. I’ve been telling myself that when the C++0x standard comes out, Concepts will change the face of metaprogramming, so I should just keep avoiding it until I can learn the new system. Well as Sutter reports, Concepts is out of the (newly renamed) C++1x Standard, and might not make it into the language for another few years. So it’s time for me to bite the bullet and learn something new.

With that in mind I’ve bought the big scary book, C++ Template Metaprogramming and am trying to read it. In the first weekend I managed to get through chapter 1 (mostly background) and struggle through chapter 2. Then I attempted the exercises in chapter 2 and almost gave myself a stroke. I got through half of them before moving on to chapter 3. A few pages into chapter 3 I was bleeding from the ears, so I backtracked and re-read chapter 2. Long story short, it’s gonna take me a lifetime to get through this book.

However I have managed to come up with an example where MAYBE I have put it to good (or at least interesting) use. I’ve been toying with the idea of re-writing the game engine on which Rift is built. So today I thought I’d tinker around with the math a bit, see if anything is interesting. One of the things I’m trying to do is create specific classes for specific purposes (I’m tired of losing track of whether or not my rotation vector is relative to the global coordinate space or the object’s). So I’ve been trying to come up with ways to create 1 template that can then be tricked into being different types for different purposes. So with that in mind, let’s take a look at the humble Vector.

For purposes of this discussion there are 2 kinds of Vectors, regular ones and normalized ones. Obviously a normalized one has a length that’s always 1. So the trick was to write a vector class that can be 2 types, one for regular vectors and one for normal vectors. Before templates I would just include a run time flag, m_IsNormal, and check it to see if I should normalize. Not a bad solution, but a bit prone to error and chews up runtime cycles (okay not a lot, but bear with me). So how do I create a vector that picks the correct type at compile time and normalizes accordingly? Here’s how I started:

template class Vector
{
public:
    Vector( float X, float Y, float Z )
      : m_X( X )
      , m_Y( Y )
      , m_Z( Z )
    {
    }
    void Normalize();
private:
    float m_X, m_Y, m_Z;
};

This works pretty well as a regular old Vector. But as you can see, it doesn’t normalize unless you specifically tell it to. So it fails as a Normalized Vector. I could call Normalize in the constructor, but of course then it would always normalize, which would make it fail as a plain vector. So I decided that I needed was a static helper class:

template struct Normalizer;

And of course, what good is a static helper class if it doesn’t perform an action? And not only that, but we have to be able to select that action at compile time. Thankfully this is all what I learned in chapter 2 of the book. You can select different actions by creating different specializations of the static helper class:

template
struct Normalizer
{
    static void Perform( Vector& V )
    {
        V.Normalize();
    }
};

template
struct Normalizer
{
    static void Perform( Vector& V )
    {    }
};

How’s that look? Now if you compile a Normalizer with true then it will call Normalize, and if you compile it with false, it won’t. Just what we need. How do we use it? I’m going to change the constructor a bit:

    Vector( float X, float Y, float Z )
      : m_X( X )
      , m_Y( Y )
      , m_Z( Z )
    {
        Normalizer::Perform( *this );
    }

Do you see the magic? When you set ShouldNormalize to true at compile time it will compile the version of Normalizer that calls Normalize(). If you set ShouldNormalize to false, it will do nothing. My hope is that since it’s a static function the optimizer will get rid of the function call and efficiency will be obtained.

But WAIT! There’s a problem. Now we have 2 vectors that are completely different types. Logically they’re both vectors, one’s just stored normalized, but to the compiler they’re completely different things. So the following reasonable code will fail:

typedef Vector PlainVector;
typedef Vector NormalizedVector;

PlainVector Plain( 10, 20, 10 );
NormalVector Normalized = Plain;

Damn! So close. So what do we do now? Well this is the point where I start to wonder if my solution is clever, or horrible. I hope at the very least it’s interesting. I figured what I needed was a copy constructor to convert between the two types. The clever part is that I only have to write one copy constructor to achieve this goal:

    Vector( const Vector& Other )
      : m_X( Other.m_X )
      , m_Y( Other.m_Y )
      , m_Z( Other.m_Z )
    {
        Normalizer::Perform( *this );
    }

Pretty sneaky huh? That will create a copy constructor for the normalized vector that accepts a plain vector, and vice versa. Pretty smooth…. only one problem, it won’t compile. Did you already figure out why? Feeling smart enough to guess? I’m gonna tell you in the next sentence to you better hurry up. It won’t compile because Vector
is not allowed to see the private members of Vector. Remember, they’re completely different objects. However the fix is pretty trivial. As my old professor used to say “only friends can see your private parts.” You just need to expose each type of vector to the other. You can use the same trick you used with the copy constructor:

    friend class Vector;

And that’s the end of that story. I now have one body of code that compiles cleanly into a normalizing and non-normalizing vector. This leaves one huge question in my mind, was this a good thing? If any of you have experience with this kind of thing I would LOVE to hear your feedback. Maybe when I get to the second half of chapter 3 I’ll learn that I just committed the classic rookie mistake. I’ll let you know.

posted by james wells at 4:49 pm

Video!

August 12, 2009

I’ll be damned, it worked! I managed to stick an AVI into the game and have it play on a texture. Turns out it wasn’t so tough. A raw AVI is a pretty straightforward file format, very well documented on MSDN. It’s basically just a bunch of BGR values with some location markers. Turning them into a handful of static textures is trivial. Then you just have to update the timer and pick the right frame.

I also created an animation in Blender. It’s pretty silly, some white rings and flashes and spinning bars. But when you throw it in the game it’s alright. Well, it’s not great, but at least there’s something to let you know it’s warping. It’s much better then the old system, which was just a hard cut to black.

Not sure what to add next. Back to play testing.

posted by james wells at 2:52 pm

Play Testing

August 10, 2009

The last couple of programming sessions have been sort of directionless for me. I wasn’t really sure what I wanted to change, so I opted for trying to make the game more “fun.” I know, I know, a crazy concept. With no idea how to do that I decided to just play a bit and see if there was anything that could be made a bit better. Well it didn’t take too long to start compiling a list of stuff. Right off the bat was the mouse pointer. Every other game with a first person view has a cool little crosshair type thing to chase around, whereas Rift has a mouse pointer. I also noticed that the steering was twitchy to the point of being stupid, which makes it really hard to dogfight. Also, why no shields? Seriously, how did I never get to that?

So these things and others have now been crossed off the list. It’s surprising how little things like this can go a long way. The next thing I decided to add was spawn points. You know how it works, you jump into an area and after a while some bad guys show up. Then a bit later some more show up. And so on until you either leave or die. Well I created a spawn point and immediately ran into a new and interesting problem. I found that when you had about 6 ships chasing me down my refresh rate would drag down to 1 fps.

This was cause for concern. Oddly enough I noticed that my Physics rate was holding steady at my pre-defined 40 frames per second. So how is it that Physics could remain unchanged, while refresh was dragged down? That’s when I started to remember something from those physics articles I posted a while back. The way the physics loop works is every time through the loop I accumulate time. When enough time has passed I chew up that time in physics. This is all well and good when the game is humming along in empty space. However there’s a real problem when things get busy. What happens when it starts taking longer then a second to calculate all the physics for the past second? The shit hits the fan and your frame rate goes down to 1 frame per second.

Just to pound the point home a bit, let’s say you decide to run 10 physics loops a second. At time 0 you start your game loop and your physics takes 0.18 seconds. Then you run your render and that takes 0.02 seconds. Now you start your game loop over and you have 0.2 seconds worth of physics to run. That’s 2 physics loops. So those two loops takes you 0.36 seconds and then you do your render for another 0.02 seconds. Now you start your third loop and you have 0.38 seconds of physics, or 3 full runs. That now takes you 0.54 seconds. You see where this is going. Every loop you get further behind. Eventually your game just becomes a huge physics number cruncher and stops responding to your input. That’s about what Rift did when 6 bad guys jumped into the spawn point.

Thankfully a long time ago I created a neat little utility program for Rift. The idea is that in debug mode you can break into Rift, flip a bit, and then the current state of the physics will dump to an xml file. Then you can open up that xml file in my test program and poke it with sticks to see how it responds. Well it turns out that with 500 lasers on the screen, 40 physics loops was taking around 3 seconds. Oops. What do you do? Well I started off by getting a nice little profiling tool called Very sleepy:
http://www.codersnotes.com/sleepy/
I fired that up and ran through my code, but it didn’t tell me anything I thought was useful. According to it, all of my time was spent doing things like dereferencing smart points and adding vectors. Surely that couldn’t be the problem? So I started adding counters and timers and what not, and looking at the code path. I found some troublesome square routes, and I copied my heavy STL lists into raw arrays. All of that helped a bit, but I was still taking over a second, which was still WAY too long. That’s when I started to wonder about the sheer quantity of calculations.

The thing about lasers in Rift is that they don’t actually collide. This is by design because lasers aren’t supposed to bounce. So why was my physics loop taking so long? I had 500 lasers that weren’t running any collision detections, should be cake. Well it looks like the problem is simply that when you have 500 objects in a container, and they all need to check against each other, and you need to do that 40 times a second, you end up doing over 5 million comparisons. Turns out this takes a bit of time, even in release mode. So yes, Very Sleeps was telling me the truth, all of my time was spent doing fast things. The problem was just that I did so many fast things.

The fix? I split up the physics a bit so that all of the lasers go in their own container that doesn’t bother to check against itself. Is there a moral, a lesson? I guess it’s that sometimes you have to abandon generality when things go wrong. I’m actually running into this a lot more as I try to work less on the engine and more on the game. Suddenly all my nice abstractions are falling down as I try to add specifics. Some of that is just because my engine has flaws, and some of that is just due to trying to make game specific things.

Next on my todo list is video. I want to make an animation for jumping to a new area. There are a number of articles out there about playing an avi on top of a texture. I just have to figure it all out. Looks interesting. Let’s see how far I get.

posted by james wells at 3:37 pm