GSoC 2020

Refactoring the renderer interface and getting some work done on binary .X files

So last time we were at a point were the engine port works with OpenGL ES 2 shaders. One reason to implement this was actually to find out what a proper base interface for the rendering part would look like. This interface should abstract away the differences between the fixed and programmable pipeline APIs being used and also at least allow for a potential software renderer based on TinyGL.

The next step was thus to reorganize the current renderer code in a somewhat more proper way and also move some rendering API calls into more suited spots like the renderer classes for example. Also the old OpenGL matrix stack is not used anymore, since it was kind of unnecessary and there is a clear boundary now on which matrices are going to be transposed so that they can be passed to OpenGL (the boundary being the renderer interface actually).

One more thing that has happened is the implementation of a binary .X file lexer. Originally I though that this step would be straightforward, but actually the grammar for binary .X files is slightly different to save some space. Instead of writing a new parser I decided to hack the lexer in such a way (and also introduced some new functions for it) that the parser could stay essentially the same.

Here is a screenshot of “On the track of dinosaurs” demo, which has a binary. X file:

There are some oddities with this file, for example, it has less normal vectors than actual vertices. This might explain the dark spots because no or wrong normal vectors means messed up lighting. Also the animation data is contained in an additional file and support for loading of such a file is still missing.

GSoC 2020

Finishing basic shader rendering

Now we can render our WME games even with shaders. The most work was actually related to setting up the surrounding code and not so much the actual shaders (minus the one which is responsible for rendering .X models). At the moment, things are somewhat unorganized however and the current code should be refactored, interfaces improved and common code factored out. I would claim that at the moment one can see the fact that WME was designed for the old fixed function pipeline and that the addition of the shader renderer is not completely straight forward. In contrast changing from Direct3D to OpenGL was easier, the only real change that I made was using the OpenGL matrix stack, which now turned out to perhaps been the wrong direction. Well, it can be reimplemented (and is so at the moment), but the old WME code didn’t rely on it either, so I might change this back.

I hoped that lighting done by shaders would produce the same results as the original code but this is not the case.

Let me just assure you that Rune is not supposed to look like he is almost getting sick. Apart from that though, things seem to look just fine.

GSoC 2020

Starting rendering with shaders

Ok, what’s so special about the following picture:

Well, nothing, except that the scene was rendered with OpenGL shaders instead of the old fixed function pipeline. Which in this case (2d graphics) was not the biggest change, essentially only the 2d projection matrix has to be setup for the shader. 3d graphics will require more work, though. Also some refactoring will be necessary, some of which has already happened. For example, there is a 3d renderer interface class now with two implementations, which can be selected from the ResidualVM menu (a third one, based on TinyGL, is supposed to follow at some point in the future).

For some reason, when walking into the house and also when leaving it, during the animation the cursor becomes completely black, but only with the shader renderer. Now I am not too unhappy about this, at least it shows that the two renderers indeed operate differently.

Speaking of different behavior, the shader renderer might also be helpful when going back to lighting, since currently, the fixed function renderer does not give the same results as the original Wintermute Engine. If the shader renderer can reproduce WME lighting, than something would be wrong with the fixed function settings. If, on the other hand, shader and fixed function renderer produce the same results, something will be wrong with the rest of the code, possible normal vectors. If all three implementations are different, than it’s kitchen sink, though.

GSoC 2020

Rendering shadows

Another major step is done, rendering shadow volumes of 3d objects:

The shadow looks beautiful, I have to say, stencil shadows are great. Also Rune does not appear to be floating over the ground anymore in this scene.

WME has some more shadow techniques (looks like it got real time shadow mapping, if the graphics card supports it), but they don’t seem to be too relevant, so it might be better to postpone their porting until a later point.

One thing which is bugging me is the fact that lightning does still not give the same results everywhere compared to WME. But I am going to accept this for now. Now is the time to add some more missing stuff and cleanup the existing code.

GSoC 2020

Turning the lights on .. well, almost

So, saving and loading seems to be fully functional now, including compability with ScummVM save files. The last bug was caused by a mistake during code import from the original Wintermute Engine. I had similar bugs on multiple occasions, which is rather annoying, as these mistakes should be unnecessary, but so is life I guess. Next thing on the menu are lights and shadows, to make the visual output more enjoyable, for example:

The faces in particular look much more round and plastic than before. Now what I don’t like so much is that at this spot Rune should appear a little bit darker but I am not sure what is wrong here.

This is my new favourite:

No textures, but normal vectors are being rendered, just so that I could check if they are correct. Now in this picture they are actually not correct, which might be hard to see, but if you want to, you take a closer look at the chest for example. There a vertices with the same position but different normal vectors, which means that these vertices will receive light with different intensity. This will create more distinct edges which will be desireable in certain situations, but not here. The first picture by the way does not have this issue, hence the smooth skin.

The problem was created in the first place because I decided to recalculate the normals during animation updates (I am not 100% sure anymore if this is really needed, although it probably should be). But different vertices (even if they have the same position) mean different normals, as long as you don’t specify, which vertices should get the same normals. Since this information does not seem to be contained in the .X files, instead now I just update the normals directly via the bone transforms.

GSoC 2020

Saving the game and memory leaks

So the next thing I wanted to do was getting saving and loading to work. The Wintermute Engine has a, I would say, rather sophisticated save system. Basically it is possible to save an entire object graph into a file and then getting it back later, without the user code having to do anything about it apart from saving it’s own state (actually, the engine handles things in a way such that loading and saving can be handled by the same function for the object being saved/loaded in). Essentially this is achieved by registering newly instantiated objects (via new overload) and keeping them in a special container class.

Now there are some caveats here, for example when one is dealing with a subclass of a class with such a new overload, which actually is not supposed to be registered. In this case one has to disable the whole registering process via a flag, something I had to find out via debugging and comparing with the WME code.

Another problem is compability with ScummVM savegames. The 3d code adds new classing and member variables, which are going to end up in the save file. This would lead to inconsistencies betweem save files from ScummVM and ResidualVM. The proposed solution here is to only save/load the 3d classes and variables in case we are playing a 3d game, which should work but I have not tested this yet.

Also loading a save file in Alpha Polaris seems to introduce at least one bug, certain info texts disappear too quickly and thus are not readable. Also text input (which is required at certain times in the game) is not possible anymore.

One reason I started too work on save files was actually to be able to play the whole game since after a while the performance would drop so much that I had to stop the game. Turned out that the problem here were memory leaks. Fixing them made it possible to play the whole game in one go, without any saves (which basically are broken at the moment). This is very good news as it means there is no big stuff missing which would be required to finish the game. It also showed that the leaks were indeed fixed since memory usage became stable after a while (although 2GB still seems a little bit too much to me).

GSoC 2020

3D object selection

We are now able to select 3d objects by mouse clicks (2d object selection was already part of the ScummVM Wintermute port). Take a look at the following screenshot:

The cursor indicates that it is pointing onto our main protagonist.

Essentially what happens is that we have a bounding rectangle (constructed from a bounding box) for our model, which is added to a list. Every frame the list is traversed. If the mouse coordinates land in one of the rectangles, a more precise test is performed. In case of our model this means that we send a ray into the scene and check if it hits one of the triangles of the model. If so, the model becomes the active object, which means it can be selected by a mouse click.

GSoC 2020

Adding more 2d stuff and fixing 3d movement

We are now at a point where the 2d graphics side of things has reached a certain stableness. There are still parts of the original wme engine which are not implemented yet, but the main functionality is there, at least for Alpha Polaris (and the demos as well). Some screenshots:

This actually comes from an in-game video. Now the speech bubble in the lower left corner is not supposed to be there.
Rune’s not happy about being abruptly awakened. The debug messages in the upper left indicate that there is still enough work to do.
Looks like they found something. Unfortunately, shadows are still missing, so the scene doesn’t look quite right yet.
The laboratory. Rune looks almost floating, this will hopefully change once shadows are rendered.

What cannot be seen on the screenshots is movement of course. There were some more issues with it since the scripting code of Alpha Polaris is expecting a Direct3D coordinate system. So some conversions were needed. But now things seem to work as expected or at least nothing unexpected has happened. But then this is only the very beginning of the game.

One thing that worked straight out of the box was 2d object selection. On the other hand, 3d object selection together with attaching these objects to characters is still missing. Implementing this will be the next goal. After that, we will be able to have some fun with rifles and bears!

GSoC 2020

Getting coordinate systems right

So last Tuesday I thought that I had the .X file stuff basically finished, up to missing details at least. Unfortunately this was not correct. After getting everything to work in the Wintermute 3D demo, I decided to try out and see, what would happen in Alpha Polaris. Well, this was the first result:

The black something flying over the bed is supposed to be the main protagonist of the game. So this didn’t work out so well. Issues were negative texture coordinates together with clamping and a wrong mesh update function, which did not set all vertex position values to zero before calculating the new positions during an animation.

After fixing this, I realized, that my model was still using the Direct3D coordinate system, which caused it to appear mirrored. So this meant some more work figuring out how to correctly change the coordinate system to an OpenGL one and fixing some more bugs along the way. Lesson learned here: The coordinate system has definitively to be consistent.

Now things are looking much better. Pictures will follow over the weekend, there are still some other issues in Alpha Polaris, which have to be fixed.

GSoC 2020

Animating .X models

So, most of the necessary Wintermute 3D code is now imported. Also, the .X loader is almost finished, meaning in particular that the animation data is loaded in and applied. Here is the current state in a video (rightclick -> play should start the video):

So basic animations are essentially working (although there could be buggy edge case which so far did not appear). Also path-finding seems to work already. On the other hand there are still some glitches like the model being visible in the initial screen, the teapot disappearing when reentering the front part of the scene (although it will reappear again if you just go towards the back and then to the front again), letters two far away, etc. .. .