Logging/Debugging of computer vision apps

Question

When I'm writing an computer-vision program (using OpenCV and Python) I need to print/show a lot of intermediate results in form of images (using cv2.imshow(..)) for debugging purposes to find out what is happening. After finishing the program I have no idea what to do with this "debugging code". Because this code is useless in production code, but it might be very useful in case of bug fixing in the future and it will make much easier for someone else to understand the logic of the code.

In case of plain text logging people usually use some logging library (in python it is logging library) that allows to print the redundant information only while debugging.

I did some research and I found nearly no discussion on that topic and in case of python just one unmaintained library that does this kind of logging so I am wondering whether my idea of logging of computer vision programs is wrong and I should rather think about dividing my program into processing part and visualization part?

Answer 1

If the debugging aspect of your program is important, include it as part of your program. Similar to Chrome, have the user press some sort of function key or key sequence to activate. Don't include this in your public documentation.

Another possibility is to add a log level to your program. Typical log levels are Verbose (Debug), Information, Error. In production the log level is set to Error, while locally developers can set it to Verbose to see everything (all your debug prints. Just set the level in your configuration file and it can be altered as needed.

Answer 2

I should rather think about dividing my program into processing part and visualization part?

Yes.

I don't have a good comprehensive answer (i.e. I don't think I have found a solution to the problem), but here is my approach.

I added "debug image listener interface" to the main algorithm classes. The interface contains a method that will "receive" some images which are generated during the normal execution of the main algorithm. The implementer of that listener class will decide what to do with that image, such as saving it.

The separation of "extension points" (the interface, which serves as callbacks or hooks) and the implementations (the debug image listener implementations, which are responsible for saving them to disk or analyzing them on-the-fly) is what allows separation of production code and development code.

This is acceptable in C++ because, in C++, OpenCV matrix handles (cv::Mat) behave similar to references to the underlying matrices. Passing a handle for read-only purpose does not require a copy of the underlying memory. This means handle-passing is memory-efficient.

For more complicated use cases, this is what I have tried:

• Pass a descriptive string along with each image. The string may describe the step which generates that intermediate image.
• Pass a string which can be used as a filename (i.e. characters from the descriptive string which are not valid as filename characters had been stripped out)
• Add a second method to the interface, which allows interrogating whether a particular intermediate image should be saved or not.
• Convert between multi-channel and high-bit-depth (integer-valued) images via Morton code, so that matrices that are not PNG-compatible can be encoded in PNG.

As you can see, logging of 2D images requires a lot of small tricks, and appears to be far from being a solved problem.

Finally, image algorithm development sometimes require a lot of R&D, which is typically done as a separate software project (or "software artifact" - tangible by-products from software development process). These artifacts are worth maintenance. Without these artifacts, fixing of algorithm-level defects or algorithm-level improvements, or even parameter tuning (as opposed to implementation or coding defects) can become impossible.