Night vision camera in Android using OpenCV

Introduction

Night vision in software is normally achieved either using histogram equalization or gamma correction, and these 2 methods are already supported in OpenCV. For any Android devices, to process the images captured from the device, we can use OpenCV4Android.

Background

The code is developed when I am developing myMobKit. You can find the source code here.

Using the code

When programming in Android, during the camera preview you will get a byte array which is a YUV420sp image.

Using OpenCV, convert the image into a grayscale image and use the OpenCV APIs to apply histogram equalization to the image.

@Override
public byte[] process(final byte[] source) {
    try {
        if (source == null) return null;
        if (sourceFrame == null) {
            if (!isColor) {
                this.sourceFrame = new Mat(height + (height / 2), width, CvType.CV_8UC1);
                this.processedFrame = new Mat(height + (height / 2), width, CvType.CV_8UC1);
            } else {
                this.sourceFrame = new Mat(height + (height / 2), width, CvType.CV_8UC1);
                this.processedFrame = new Mat(height + (height / 2), width, CvType.CV_8UC3);
                this.ycrcb = new Mat();
            }
        }
        sourceFrame.put(0, 0, source);

        // convert sourceFrame to gray scale
        if (!isColor) {
            Imgproc.cvtColor(sourceFrame, processedFrame, Imgproc.COLOR_YUV420p2GRAY);

            // Apply Histogram Equalization
            Imgproc.equalizeHist(processedFrame, processedFrame);

            // Convert to JPEG
            return CvUtils.grayToJpeg(processedFrame, imageQuality);
        } else {
            // Histogram equalization using YCrCb
            Imgproc.cvtColor(sourceFrame, ycrcb, Imgproc.COLOR_YUV2RGBA_NV21, 4);
            Imgproc.cvtColor(ycrcb, ycrcb, Imgproc.COLOR_RGB2YCrCb);
            Core.split(ycrcb, channels);
            Imgproc.equalizeHist(channels.get(0), channels.get(0));
            Core.merge(channels, processedFrame);
            Imgproc.cvtColor(processedFrame, processedFrame, Imgproc.COLOR_YCrCb2BGR);
            return CvUtils.toJpegByteArray(processedFrame, imageQuality);
        }
    } catch (Exception e) {
        LOGE(TAG, "[process] Unable to process byte[]", e);
        return null;
    }
}

The result

For Gamma Correction,

private void configureLut() {
    releaseMat(lutMat);
    lutMat = new Mat(1, 256, CvType.CV_8UC1);
    double invGamma = 1.0 / gamma;
    int size = (int) (lutMat.total() * lutMat.channels());
    byte[] temp = new byte[size];
    lutMat.get(0, 0, temp);
    for (int j = 0; j < 256; ++j) {
        temp[j] = (byte) (Math.pow((double) j / 255.0, invGamma) * 255.0);
    }
    lutMat.put(0, 0, temp);
}

@Override
public byte[] process(final byte[] source) {
    try {
        if (source == null) return null;
        if (sourceFrame == null) {
            sourceFrame = new Mat(height + (height / 2), width, CvType.CV_8UC1);
            processedFrame = new Mat(height + (height / 2), width, CvType.CV_8UC1);
        }
        sourceFrame.put(0, 0, source);

        // Gamma correction
        if (isColor) {
            Imgproc.cvtColor(sourceFrame, processedFrame, Imgproc.COLOR_YUV2BGRA_NV21);
            Core.LUT(processedFrame, lutMat, processedFrame);

            // Convert to JPEG
            return CvUtils.toJpegByteArray(processedFrame, imageQuality);
        } else {
            Imgproc.cvtColor(sourceFrame, processedFrame, Imgproc.COLOR_YUV420p2GRAY);
            Core.LUT(processedFrame, lutMat, processedFrame);

            // Convert to JPEG
            return CvUtils.grayToJpeg(processedFrame, imageQuality);
        }
    } catch (Exception e) {
        LOGE(TAG, "[process] Unable to process byte[]", e);
        return null;
    }
}

To see the full source code, check out the GitHub repository.

History

2016-10-01 - Initial release.