Android + OpenCV: Part 6— OpenCV Native Library

6 min readMay 4, 2020

In the OpenCV SDK folder, there is a native library. You can use the code directly from the Cpp. In this part, I will show you the demo. This time, I will try to use C++ to convert the Mat variable from color into grayscale instead of Java. Here is the menu:

1. Link to the Source
2. Create OpenCV C++ File
3. CMakeLists.txt: Link OpenCVLib + YourLib + app-glue + log-lib
4. Android.mk & Application.mk
5. OpenCV Native Call
6. Grayscale Conversion
7. Logcat
8. Modify MainActivity
9. Summary

😎1. Link to the Source

In your OpenCV SDK folder which unzipped from online, there is a native directory, “{cvsdk}/sdk/native”. You’d better put the CVSDK folder at the same drive to your Android project. I prefer to use them directly instead of copying them here and pasting them to your JNI folder. It’s a high risk to mess up the path. Assume they are in the same drive, you can link them by Gradle right away.

android {
    ...
    sourceSets {
        def opencvsdk = 'D://Android/OpenCV-Android-Lib/OpenCV-android-sdk-3410/sdk'
        main {
            jni {
                srcDirs 'src/main/jni', opencvsdk+'/native/jni'
            }
        }
    }
}

There it is. I am using the REAL path by a defined variable. Sync…

😄2. Create an OpenCV C++ Files

Let’s add a new C++ class, opencv_jni. The system will create two files: opencv_jni.cpp and opencv_jni.h. They will help us to call the OpenCV Cpp library.

☝️3. CMakeLists.txt: Link OpenCVLib + YourLib + app-glue + log-lib

Next, let’s link them to the project by CMakeLists.txt.

Version:

# The minimum version
cmake_minimum_required(VERSION 3.4.1)

CV include directory:

# Configure path to OpenCV include directories
include_directories( D://Android/OpenCV-Android-Lib/OpenCV-android-sdk-3410/sdk/native/jni/include )

CV Library: You have to use “${ANDROID_ABI}” for CPU auto-selection or you’ll get an error with Wrong File Format.

# Set up OpenCV shared .so library
# Import Library matched with your phone CPU
add_library(
        cv_core-lib
        SHARED
        IMPORTED )
set_target_properties(
        # Specifies the target library.
        cv_core-lib
        # Specifies the parameter you want to define.
        PROPERTIES IMPORTED_LOCATION
        # Provides the path to the library you want to import.
        D:/Android/OpenCV-Android-Lib/OpenCV-android-sdk-3410/sdk/native/libs/${ANDROID_ABI}/libopencv_java3.so )

My CV library:

# Your libraries
add_library(
        opencv_jni  # Library
        SHARED
        opencv_jni.cpp # Source file(s)
)

I also rename the one in build.gradle(Module: app).

android {
    ...
    defaultConfig {
        ...

        ndk{
            moduleName "opencv_jni"         //create so file
            //32bit and 64bit cpu
            abiFilters "arm64-v8a", "armeabi-v7a", "x86", "x86_64"
        }
        ...
    }
...
}

Let’s add some options for CMake to compile in defaultConfig{}, too.

externalNativeBuild {
    cmake {
        // Macro constants for the C compiler.
        cFlags "-D__STDC_FORMAT_MACROS"
        // Sets optional flags for the C++ compiler.
        cppFlags "-std=c++14 -frtti -fexceptions"
        // Passes optional arguments to CMake.
        arguments "-DANDROID_TOOLCHAIN=clang",
                "-DANDROID_STL=c++_shared",
                "-DANDROID_PLATFORM=android-21"
    }
}

Let’s continue with CMakeLists.txt.

Android Native Activity Manager:

# Android App Glue
add_library( app-glue
        STATIC
        ${ANDROID_NDK}/sources/android/native_app_glue/android_native_app_glue.c )

Log library:

# Logcat
find_library(
        log-lib  # Path
        log  # Locate the NDK library
)

Linkage:

target_link_libraries(
        opencv_jni  # Your library
        app-glue
        cv_core-lib
        ${log-lib}  # To the log library
)

👉🏻4. Android.mk & Application.mk

We need two files to tell Android about the configuration, called Android.mk & Application.mk. Right-click the “cpp” folder to add a file in the Android view.

Let’s put in the path of OpenCV, source files, and Cpp options. I need “jni” path because OpenCV.mk is staying over there.

LOCAL_PATH := $(call my-dir)

CVROOT := 'D://Android/OpenCV-Android-Lib/OpenCV-android-sdk-3410/sdk/native/jni'

include $(CLEAR_VARS)
OPENCV_INSTALL_MODULES:=on
OPENCV_LIB_TYPE:=STATIC
include $(CVROOT)/OpenCV.mk

LOCAL_MODULE += opencv_jni

LOCAL_C_INCLUDES += opencv_jni.h
LOCAL_SRC_FILES += opencv_jni.cpp

LOCAL_CFLAGS += -std=c++11 -frtti -fexceptions -fopenmp -w
LOCAL_LDLIBS += -llog 
LOCAL_LDFLAGS += -fopenmp

include $(BUILD_SHARED_LIBRARY)

Let’s add another file, Application.mk. It also needs to match your CPU at APP_ABI.

APP_STL := gnustl_static
APP_CPPFLAGS := -frtti -fexceptions
APP_ABI := all
APP_PLATFORM := android-21

😙 5. OpenCV Native Call

In NativeCall.kt,

external fun convertGray(matAddrRGBA:Long, matAddrGray:Long): Booleancompanion object {
        init {
            // Cpp library file
            System.loadLibrary("opencv_jni")
        }
}

This will create a function header in jnitest.h. This is how special about Cpp. It can separate code to the headers and functions. Now, I cut the line and paste into opencv_jni.h. And rename the class name to OpenCvNativeCall.

extern "C"
JNIEXPORT jboolean JNICALL
Java_com_homan_huang_opencvcamerademo_OpenCvNativeCall_convertGray(
        JNIEnv *env, 
        jobject thiz,
        jlong mat_addr_rgba,
        jlong mat_addr_gray);

Include,

#include <jni.h>
#include <string>
#include <stdio.h>
using namespace std;

I have tried to type “ std:: ” many times in the last part; so I use the namespace to relax my hands.

Next, I move this line from NativeCall.kt

// convert color to gray
external fun convertGray(matAddrRGBA:Long, matAddrGray:Long): Int

to OpenCvNativeCall.kt which is a new Kotlin class file. It shall be NO red at all.

class OpenCvNativeCall {
    // convert color to gray
    external fun convertGray(matAddrRGBA:Long, matAddrGray:Long): Boolean    companion object {
        init {
            // Cpp library file
            System.loadLibrary("opencv_jni")
        }
    }
}

In opencv_jni.cpp, I copy the header into this file and turn it into a function.

#include "opencv_jni.h"

extern "C"
JNIEXPORT jboolean JNICALL
Java_com_homan_huang_opencvcamerademo_OpenCvNativeCall_convertGray(
        JNIEnv *env,
        jobject thiz,
        jlong addr_rgba,
        jlong addr_gray) {
}

👦: Pretty neat, isn’t it?

✍️6. Grayscale Conversion

If you import the OpenCV library correctly, you can use them directly.

#include "opencv2/core.hpp"
#include "opencv2/core/mat.hpp"

#include "opencv_jni.h"


extern "C"
JNIEXPORT jboolean JNICALL
Java_com_homan_huang_opencvcamerademo_OpenCvNativeCall_convertGray(
        JNIEnv *env,
        jobject thiz,
        jlong mat_addr_rgba,
        jlong mat_addr_gray) {

    cv::Mat& mGr = *(cv::Mat*) mat_addr_gray;


}

Your code shall have no Unresolved Type Error. By the way, “cv::” can cause finger fatigue, too. Let’s add the namespace.

using namespace cv;

So I can remove the “cv::” in the code.

Mat& mGr = *(Mat*) mat_addr_gray;
Mat& mRgb = *(Mat*) mat_addr_rgba;

Color conversion:

bool converted = color2Gray(mRgb, mGr);

Right-click “corlor2Gray” to create the header and function.

bool color2Gray(Mat &img, Mat &gray) {
    cvtColor(img, gray, CV_RGBA2GRAY);

    return false;
}

Alt+Enter to import:

#include <opencv2/imgproc.hpp>

It needs to return the result instead of false.

// compare rows and cols
return img.rows == gray.rows && img.cols == gray.cols;

Next, let’s return to the result with false first.

extern "C"
JNIEXPORT jboolean JNICALL
Java_com_homan_huang_opencvcamerademo_OpenCvNativeCall_convertGray(
        ...) {
    ...
    bool converted = color2Gray(mRgb, mGr);

    if (converted) {
        string convResult = "Grayscale conversion: failed!!!";
        lge(convResult);
    }

    return (jboolean) false;
}

😺7. Logcat

Here is my Logcat code in Cpp.

// Logcat
#define TAG "MyLog/OpenCVJni"
void lgi(const string s);
// Android Logcat: debug
void lgd(const string s);
// Android Logcat: error
void lge(const string s);

There are no preset Logcat in the NDK. You have to add them manually. Or save them in your backup for further usage. You have to add Cpp function headers before the JNI headers in the header file; otherwise, your Cpp functions will be red. Cpp uses waterfall flowing to call. If you don’t place them in the front, they won’t be in memory. There’re a lot of functions and variables placed in front of the main() which is the executable in all Cpp code.

Here is the function in the Cpp file.

// Android Logcat: info
void lgi(const string s) {
    __android_log_print(ANDROID_LOG_INFO, TAG, "%s", s.c_str());
}
// Android Logcat: debug
void lgd(const string s) {
    __android_log_print(ANDROID_LOG_DEBUG, TAG, "%s", s.c_str());
}
// Android Logcat: error
void lge(const string s) {
    __android_log_print(ANDROID_LOG_ERROR, TAG, "%s", s.c_str());
}

😁8. Modify MainActivity

MainActivity.kt: Replace the line of “inputFrame!!.gray()”.

override fun onCameraFrame(inputFrame: CvCameraViewFrame?): Mat {
    imageMat = inputFrame!!.rgba()

    val javaEnabled = false;
    
    if (!javaEnabled) {
        // C++        
        if (!OpenCvNativeCall().convertGray(
                imageMat.nativeObjAddr,
                grayMat.nativeObjAddr
            )
        ) {
            throw CvException("Data is corrupted!")
        }
        finish()         
    } else {
        grayMat = get480Image(inputFrame!!.gray())
        ...
    }
    
    return imageMat
}

Now, let’s run. The app will be crashed and check the Logcat.

31404-31404 D/MYLOG MainActivity: OpenCV started...
31404-31404 I/MYLOG MainActivity: OpenCV Loaded Successfully!
31404-31404 D/MYLOG MainActivity: OpenCV started...
31404-31404 D/MYLOG MainActivity: CameraView turned ON...
31404-31404 D/MYLOG MainActivity:  System Ui Visibility Change
31404-31433 E/MyLog/OpenCVJni: Grayscale conversion: failed!!!
    
    --------- beginning of crash

The Logcat is working. Let’s fix the C code back to normal.

if (!converted) {
        string convResult = "Grayscale conversion: failed!!!";
        lge(convResult);
    }

    return (jboolean) converted;
}

Run again. No Logcat error has found.

😅9. Summary

There is an advantage of C code. It gives you a choice to not to copy a lot of data between functions like Java. You can pass the object address so the system can change right away. In the next part, I will continue to move the OpenCV functions from the MainActivity to the C code. So I can compare the performance between Java code and C code.

Before: Part 5 — JNI Implement Test: getMessage() and askQuestions()

Next: Part 7 — Face Detection in Native Library

Android + OpenCV: Part 6— OpenCV Native Library

1. Link to the Source
2. Create OpenCV C++ File
3. CMakeLists.txt: Link OpenCVLib + YourLib + app-glue + log-lib
4. Android.mk & Application.mk
5. OpenCV Native Call
6. Grayscale Conversion
7. Logcat
8. Modify MainActivity
9. Summary

😎1. Link to the Source

😄2. Create an OpenCV C++ Files

☝️3. CMakeLists.txt: Link OpenCVLib + YourLib + app-glue + log-lib

👉🏻4. Android.mk & Application.mk

😙 5. OpenCV Native Call

✍️6. Grayscale Conversion

😺7. Logcat

😁8. Modify MainActivity

😅9. Summary

Written by Homan Huang

No responses yet

Android + OpenCV: Part 6— OpenCV Native Library

1. Link to the Source2. Create OpenCV C++ File3. CMakeLists.txt: Link OpenCVLib + YourLib + app-glue + log-lib4. Android.mk & Application.mk5. OpenCV Native Call6. Grayscale Conversion7. Logcat 8. Modify MainActivity9. Summary

😎1. Link to the Source

😄2. Create an OpenCV C++ Files

☝️3. CMakeLists.txt: Link OpenCVLib + YourLib + app-glue + log-lib

👉🏻4. Android.mk & Application.mk

😙 5. OpenCV Native Call

✍️6. Grayscale Conversion

😺7. Logcat

😁8. Modify MainActivity

😅9. Summary

Written by Homan Huang

No responses yet

1. Link to the Source
2. Create OpenCV C++ File
3. CMakeLists.txt: Link OpenCVLib + YourLib + app-glue + log-lib
4. Android.mk & Application.mk
5. OpenCV Native Call
6. Grayscale Conversion
7. Logcat
8. Modify MainActivity
9. Summary