Introduction: Debugging OpenGL, what's the problem?

We can easily write complex graphics routines which involve many changes to OpenGL states, off-screen buffers and complex shaders.

One typical problem I see students encounter is that simple OpenGL programs are fairly straightforward to manage, but once they start building up their games or programs the graphics calls become more spread out. This means it can be much harder to see where the problems are and to work out what state the system is currently in, as you have to follow the flow of code between multiple functions and classes.

One of the biggest problems with programming OpenGL is that if we get something wrong we can get weird results or just a blank screen.

Why does this happen? OpenGL is a state machine. We put it into different states through our drawing calls and getting the correct answer (or picture) out depends on whether you got the states right. It's easy to lose track of what state OpenGL is in, or to just get it plain wrong due to lack of understanding. It also doesn't help that OpenGL's official debugging outputs are cryptic at best!

Nowadays we also have to use shaders as part of the regular graphics pipeline. These are small programs that run on the graphics card, so they operate in a totally different execution environment to your main program. This makes them really difficult to debug as you can't properly interrupt the program to step through the code.

For a beginner I know this can be overwhelming so in this article we're going to look at some tips to help you debug your OpenGL code and shaders.

 

Tip: Use OpenGL's Built-in Debugging Features

I've rarely found these helpful but for completeness they belong in this list. An internal OpenGL error will generally occur if you call a function to perform an operation when OpenGL isn't in the right state, or if you pass the wrong data or enums into a function. Internally, OpenGL will make a note of the error and essentially ignore the function you called. This means the cause of an error might not quite be where you think it is.

OpenGL keeps track of the errors in a queue. The OpenGL API then gives us two main mechanisms for interacting with this queue. The first is a set of functions for getting messages out of the queue. The second is a callback mechanism that is triggered at the point where the error should be added to the queue. This second method is far more useful as the first one can only work if you call it literally everywhere.

Getting Errors From The Queue

You need to call this function as often as you can:

GLenum glGetError()

This will return an enum that is supposed to describe the error that occurred, and remove it from the queue. If you want to check the whole queue you'll need a loop.

The main difficulty I find with this approach is that the enum is too cryptic to be of any use and the only way of figuring out which API call triggered it is to call glGetError literally after every OpenGL call. Once you know which function caused the error you can consult the documentation which should say under what situations it will be generated. You certainly don't want to be doing this in release code though as it gets quite inefficient.

Getting Errors As They Are Generated

The second mechanism is to write a simple callback function which OpenGL will trigger whenever an error situation is encountered. This is a more recent addition to the API and gives a bit more information about the nature of the error.

This is the format for the callback function:

void GLAPIENTRY
ErrorCallback( GLenum source,
                 GLenum type,
                 GLuint id,
                 GLenum severity,
                 GLsizei length,
                 const GLchar* message,
                 const void* userParam )
{

   /* handle error message */
}

Then during your initialisation you need to send the callback to OpenGL with:

glEnable( GL_DEBUG_OUTPUT );
glDebugMessageCallback( ErrorCallback, 0 );

You need to watch out about the volume of messages this generates though as by default it'll tell you about various internal graphics card points that are more for information purposes than actual errors.

Personally I'm still not convinced this is a great mechanism for debugging but it's better than nothing and perhaps it's a step in the right direction.

 

Tip: Use External Tools: especially NVIDIA Nsight

Link to NVIDIA's developer tools

The general lack of useful in-built debugging tools led to the development of external ones. There are several available and some work better than others. Some work by emulating the graphics pipeline in software, which means they're fully accessible but extremely slow. Others work by introducing a layer between your application and the graphics driver which is potentially more limited in features but they're usually faster.

However, the tool I really can't recommend enough is NVIDIA's Nsight. This is a tool that works directly with their drivers and graphics card to tell you what's really going on. It works with OpenGL but also DirectX and CUDA. It has extensive support for performance monitoring and will show you the current OpenGL state and resources.

There are now Nsight plugins available that integrate directly with Visual Studio or Eclipse, which give them a faster workflow and make them much easier for beginners. The ability to debug a single frame is especially useful.

I strongly suggest you go and check it out. Here's the link again.

 

Tip: Categorise the Problem

Ok, this tip is more of an intro into the next tips. In my experience OpenGL problems tend to fall into three general categories. There is some overlap between these and if you're really unlucky you might encounter all of them at once. These are ranked from easiest to fix to hardest to fix:

• Data-flow issues, or "where's my polygon?"
• State-based issues, or "why has this object got that other object's texture?"
• Shader-based issues, or "why does my lighting screw up when I move the camera?"

Let's look at each in turn.

 

Tip: Solving Data-Flow Issues

These are usually the easiest of problems to solve because a lot of it is outside of OpenGL and in your own code.

What I mean with this classification is that data is failing to make its way through the pipeline. In fact, it may be failing to get to the pipeline to begin with. The data that we work with is usually either an array of vertex attributes or a texture array. There are plenty of opportunities for either to go wrong.

Typical syptoms of this type of problem include:

• Completely missing objects
• Completely missing textures, including multi-textures
• Shading that doesn't change when lights or the camera moves (or model may be completely black, or whatever ambient colour you've set)

The best approach to this is to start with working through your code. First, make sure the data is flowing from the file to its storage in OpenGL. Then, when you make your drawing call, you need to check that all the data is actually going down the pipeline.

Here's a checklist to work through:

• Does the data file exist in the correct location?
• Is your parser working correctly? I.e. are your final arrays containing the data in the correct format?
• Are you correctly sending the data to OpenGL? Do you bind the relevant buffers and set up any associated state?
• Is the data correctly being received by the shader?

Solutions to the last two points will probably cross over into my two other categories of problem.

I know some of these are simple points, but believe me it's quite often the simple problems that are the hardest to solve! I've seen people check and double check their pipelines and shaders over and over again only to find a file was in the wrong directory and they hadn't put in any error checking code.

Similarly, I've quite often seen people trying to fix what they thought were broken shaders when it actually turned out the data wasn't arriving in the first place. As a first point of OpenGL debugging, I would always check all the expected data is arriving.

 

Tip: Solving State Issues

Unfortunately these can be quite tricky to solve.

So, what I mean with this class of problem is that the OpenGL 'machine' isn't set up right.

The pipeline is a bit of a black box and the way we work with OpenGL is that we activate things, change settings and deactivate things. But if we don't have the right things activated at the right time, we can easily lose track of what state the system is in, or it might ignore some of our changes.

The graphics pipeline is now quite complex, so it's easy to make mistakes and for things to go wrong. As a beginner you need to expect things to go wrong, it's an important part of the learning journey.

Typical syptoms of this type of problem include:

• Everything was working fine
• Then I added something simple
• Now it's all broken, or something completely unrelated is broken

As I mentioned in the introduction, this is especially common when you start working on your first larger OpenGL projects. State changes that were in one place in a simple demo can be spread about as you introduce classes to handle shaders, textures, materials, lights, cameras or whatever. As your use of these different classes changes, so too do the state changes that are being called.

Unfortunately, there's no easy answer to solve this. You need to put the work in to understand the underlying pipeline and how the API functions affect it. Make sure you understand how each feature works and what states you need to be in. The official documentation can really help here, though some can be a little cryptic at times.

Work through the code step-by-step, going into all the classes and functions to check where you're making OpenGL state changes. Do you end up in the correct state when you make the drawing call? Are you making the state changes in the correct order? Sometimes this is important and simply swapping around a couple of calls will fix a problem. Are you accidentally clearing or deactivating something? Again, easily done when moving stuff into separate classes.

A debugging tool can really help here. Nsight can give you details of the state OpenGL is in, which takes away a lot of the guessing. Each time you encounter a problem, treat it as a learning experience to discover more about the pipeline.

Unfortunately, even as you get more experienced you'll still find you fall prey to this type of problem. We tend to only remember things we use frequently (or I certainly do). Quite often you'll write code that wraps some feature and then bring it to your future projects. Then you forget how the internal workings (and actual OpenGL calls) worked, so you get confused if you have to do it again manually! I'd still have to look up things like texture loading, shader loading / linking and FBOs.

 

Tip: Solving GLSL Shader Issues

Shaders are an integral part of the modern OpenGL pipeline. We have two required shaders currently: the vertex and fragment shaders. Their main purposes, respectively, are to transform vertices and to calculate pixel colour. We can do some quite complex things in shaders nowadays, so there's plenty of scope for it to all go wrong.

Typical symptoms of shader issues are:

• Transforms - models not in the right place
• Shading - wrong colours, lighting going wrong etc

The first step I suggest is that you check your data flow. Is the right data getting to the shader? This is usually straightforward to check, as described previously, but it can be a state problem. Most transformation issues I've seen are in this category, possibly with uniforms not being sent properly or things like transposed matrices or incorrect inverse matrices. Double check the maths of what's being sent.

Once you're confident the data is getting to the shader, and if that didn't fix the problem, you need to look at the data processing. This is much harder and I would split it into two categories: conceptual and technical. A conceptual problem is when your theory is wrong. A technical problem is when the code is wrong. It's possible for your code to be perfectly correct, but you just got the theory wrong.

The conceptual problems are the hardest to solve because they require brainpower. You have to go back to the maths and work through all of the theory.

Here's a checklist:

• Mathematically, does what you're doing actually work?
• What space are your vectors supposed to be in?
• What space are they actually in?
• Are you doing operations between vectors in different spaces?
• Are the relevant vectors normalised?
• Are you getting values (especially inputs) within expected ranges?

As you can tell from the list, most of the problems I see are related to vectors not being in the correct spaces. This is mostly difficult from a conceptual perspective. Once you know what space they're in and what space they need to be in, it's just a matter of applying the relevant matrix.

I'll finish with the most important tip for shader debugging:

Tip: Using Colour To Get Output From Shaders

In regular C/C++ code, if we didn't have access to a debugger we could at least print something out to console or to file. We don't have access to things like that from within a shader, but what we can do is to output colour. Assuming you have an object actually showing, this is a great way to debug shaders.

The concept is quite simple, just output whatever vector is relevant as the colour output from the fragment shader. For example, to check if your surface normals are being sent through you could just set them as the fragment colour. If the object is black, or one colour, it probably means the normals aren't there.

You can also use this method to check if your values change when the camera moves. While it's unlikely that you'll be able to tell the actual values from the colours, it can give you an idea of which space they're in.

 

General Quick Tips:

Tip: Check GLSL Shader Compilation

When you compile your shaders you can retrieve error messages which can help identify compiler errors, including what line syntax errors are on. You really need to output these in some way while you're developing your program. Also, don't forget to check your shaders linked properly.

 

Tip: FBO Completeness

When they first came out, it felt like frame buffer objects (FBOs) had a whole list of restrictions and it was a real dark art trying to get them right. It seems some of these rules are relaxed now, but they're still awkward to set up. What you're aiming for is framebuffer 'completeness', which is OpenGL's way of saying you got it right.

My main tip here is actually not for helping you to achieve 'completeness' but that once you've got it, wrap all your FBO stuff up into a class and don't ever throw it away!

For debugging FBOs, a couple of things to watch out for: if your resolutions are different, make sure you're setting the viewport correctly; don't forget a depth attachment if you need the depth test!

 

Tip: Compare to known working code

I find this simple method especially useful for debugging state issues. Basically, you find a piece of code you know works - usually from something like a demo, and you compare the relevant OpenGL state changes line by line. Any differences could be causing the problem you encounter.

 

Tip: Deployment to a new platform

Once you have a working program and decide to test it on another platform, quite often you'll find it doesn't work as expected. This can be due to driver differences and hardware differences, including the vendor. I've found when porting from NVIDIA hardware to ATI/AMD hardware that the ATI/AMD drivers are a lot less forgiving. Technically, this may be because they more rigidly adhere to the OpenGL specification but it's something to watch out for.

You also need to be aware of what extensions or specific OpenGL version features you're using, which may not be present on different systems. If you're writing code that's destined for different platforms you must include a variety of checks during initialisation and fallback methods for older hardware.

All of this can be a pain to debug, especially when you may not have direct access to the machine. You'll need to make extensive use of log files and patiently work things through.

 

Conclusion

There are lots of opportunities for mistakes in graphics programming, but making mistakes is an important part of the learning journey. Often we learn more from making mistakes than we do from getting everything right. You should therefore aim to make mistakes and to develop a confident approach to problem solving that will allow you to handle anything you encounter.

Let's conclude with a general checklist:

• Is your data coming in correctly?
• Are your vectors in the right spaces?
• Are your vectors and colours within expected bounds? Do they need to be normalised?
• Do you know what state OpenGL is in?
• Do you know what state it's supposed to be in?

I hope this helps. Happy bug hunting ;)