I appologise in advance for a whole lot of boring detail you don't want to read, but I want to get off my chest.
I spent a good bit of time trying to get the animation system to deform meshes correctly, but I just couldn't quite get it to work and I couldn't figure out why. So I stopped working on it for a good long while. Now when I get into something, I really get into it - for a while. It may take months, but eventually I cool down and my enthusiasm dies. From the moment I came up with my current game dev plans I knew that once my obsession cooled I would stop working on the game, so I worked as hard as I could while the idea was still fresh, hoping that the progress I was making would sustain me. And it did, for a while. But eventually work, video games and Warhammer just pushed the project to the back of my mind.
So I've barely touched the code over a month. I wrote a bit a couple of weeks back, but then I undid all my changes and reverted to what was then month-old code. But it still gave me a better idea of how to solve the problem, and every once in a while I would spare a little thought to the issues I was having. And then today I finally sat down and in under two hours I re-wrote a good deal of the animation code and fixed all the problems. Under two hours. Days of coding in circles, six weeks of avoiding the problem, then two hours to fix it. In a way it's amazing when it happens, but on the other hand it makes you wonder what you were doing wrong the rest of the time.
Anyway, my mistake was that I was not applying bone length correctly, this wasn't a problem sometimes but it screwed up the inverse bind matrix, which made the problem hard to spot. I also removed the bone offset (this can be replaced by intermediate bones, it may be less efficient in some cases but typically the offset is not used so this is slightly more efficient for most rigs), but now that I've figured out the way the bone length is applied I beleive I can easily put the offset back in later if I decide I want it back.
So here's a couple of quick, and rather unspectacular, screenshots. Now I think I need to replace the linear interpolation of the matrices with a spherical linear interpolation, perhaps replace some of the 4x4 matrices with 3x3 for efficiency, then finish implementing the system in the actual animation pipeline. Then of course figure out how to get actual model bone and animation data out of blender and into my engine (either write custom exporters in Blender, which requires learning python, or writing an importer or converting from some other format - which I actually expect to be just as hard of not more so), which of course I can't do until I figure out animation and skinning in Blender... ugh, this is going to take forever, and those Warhammer armies aren't going to paint themselves either.