Let’s Have Fun: GStreamer+Android Tutorial #4 — Common Module to Share; JNI Calls in Hilt &. MVVM

Published in

The Startup

10 min readFeb 15, 2021

Good News😍: Tutorial 4 is a real player! It loads a 52s online video🎬, so you can test the player. The C code has no much increased than Tutorial 3. It’s simpler than the other tutorial. But I like you to challenge this tutorial to decouple the C code from the activity and redesign the pattern from MVC to MVVM. Don’t you like challenges?

🤔: Yeah, can I …
🦸‍♂️: Good! Let’s begin.

— === Menu === —

🏗️ 1. Insert a Common Module
🏳………🔌 Modify Gradle
🏳………🔗 Add Shared Functions
🏳………📦 Add GStreamer.java
♋️ 2. Translate Tutorial 4 to Kotlin
🏳………🍮 Java → Kotlin
🌁 3. Reconstruct MVC with Hilt + MVVM
🏳………🔌 Project: Build.gradle: Hilt plugin
🏳………🔌 Tutorial-4 Module: Build.gradle
🏳………📲 Add Application
🏳………➰ PlayerViewModel
🍗4. Prepare for a New GStreamer Player
🏳………👷🏻 NDKBuild — Android.mk
🏳………🗽 Theme Error
✂️5. Cut off Tutorial4 to Build NativePlayer
🏳………📢 Add public method in NativePlayer
🏳………✉️ setMessage()
🏳………➗ onGStreamerInitialized()
🏳………🆕 updateTimeWidget()
🏳………🔰 setCurrentPosition()
🏳………🔲 SurfaceView
🏳………🔳 onMediaSizeChanged()
🏳………➖ Seekbar

🏗️ 1. Insert a Common Module

< === Menu

What is the Common module? The GStreamer tutorials contain five projects which share the same GStreamer library. Anything that is common among them can be collected into the same module, so we can save a lot of steps and save code to resync the Gradle module. I call it a common module.

Let’s add a New Module

Choose Android Library => Module name: common

Let’s check settings.gradle:

include ':android-tutorial-1'
include ':android-tutorial-2'
include ':android-tutorial-3'
include ':android-tutorial-4'
include ':android-tutorial-5'
include ':common'

🔌Modify Gradle ( →Menu)

Now, you have a new common module. Let’s add some shared libraries:

dependencies {
    api "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"
    api "org.jetbrains.kotlin:kotlin-stdlib-jdk8:$kotlin_version"
    api 'androidx.core:core-ktx:1.3.2'
    api 'androidx.appcompat:appcompat:1.2.0'

    // test
    api 'junit:junit:4.13.1'
    api 'androidx.test.ext:junit:1.1.2'
    api 'androidx.test.espresso:espresso-core:3.3.0'
}

It’s easy to share by replacing all head commands with API. It’s ready and sync.

🔗 Add Shared Functions ( →Menu)

In the common module, you can add your own stuffs. I’d like to add my log and toast. Furthermore, the GStreamerSurfaceView is share among Tutorial 3 to Tutorial 5. Let’s move it into the common, too.

LogHelper.kt

const val TAG = "MTAG"
fun lgd(s:String) = Log.d(TAG, s)
fun lgi(s:String) = Log.i(TAG, s)
fun lge(s:String) = Log.e(TAG, s)
fun lgw(s:String) = Log.w(TAG, s)
fun lgv(s:String) = Log.v(TAG, s)

ToastHelper.kt

// Toast: len: 0-short, 1-long
fun msg(context: Context, s: String, len: Int) =
    if (len > 0) Toast.makeText(context, s, LENGTH_LONG).show()
    else Toast.makeText(context, s, LENGTH_SHORT).show()

📦 Add GStreamer.java

Create a package in common\java:

Choose main\java:

Insert: org.freedesktop.gstreamer

Copy Gstreamer.java into the package:

Add Common Module into Gradle Modules

Tutorial1 —

Gradle Module:

Run…Pass!

Tutorial2 —

Change Gradle Module like Tutorial1.

Run…Pass!

Tutorial3 —

Change Gradle Module like Tutorial1.
Remove common folder which holds duplicates
Remove GStreamerSurfaceView and modify the main.xml

<com.homan.huang.common.ui.GStreamerSurfaceView
    android:id="@+id/surface_video"
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:layout_gravity="center_vertical|center_horizontal" />

Run…Pass!

♋️ 2. Translate Tutorial 4 to Kotlin

< === Menu

Let’s add the common module into Tutorial4.

Change Gradle Module like Tutorial1.
Remove GStreamerSurfaceView and modify the main.xml

Run it. If you can watch the video, congratulations! Now, you’d better add VCS to backup💾 the code because we need to do a lot of changes to this tutorial.

To speed up the compilation, you can remark(Ctrl+/) all other project except Tutorial4 and common in settings.gradle:

//include ':android-tutorial-1'
//include ':android-tutorial-2'
//include ':android-tutorial-3'
include ':android-tutorial-4'
//include ':android-tutorial-5'
include ':common'

🍮 Java → Kotlin

Ctrl+Alt+Shift+k:

Date(pos) → Date(pos.toLong())
Date(duration） → Date(duration.toLong())
nativeClassInit() attached over with @JvmStatic

Run:

Nice, it’s success. Let’s backup💾.

🌁 3. Reconstruct MVC with Hilt + MVVM

< === Menu

🔌 Project: Build.gradle: Hilt plugin

buildscript {
    ext.kotlin_version = "1.4.30"
    repositories {
        google()
        jcenter()
        mavenCentral()
        maven { url "https://oss.jfrog.org/libs-snapshot" }
    }    ext.hilt_version = '2.29-alpha'
    dependencies {
        classpath 'com.android.tools.build:gradle:4.2.0-alpha15'
        classpath "org.jetbrains.kotlin:kotlin-gradle-plugin:$kotlin_version"        // Hilt
        classpath "com.google.dagger:hilt-android-gradle-plugin:$hilt_version"
    }
}

🔌 Tutorial-4 Module: Build.gradle

plugins {
    id 'com.android.application'
    id 'kotlin-android'
    id 'kotlin-kapt'
    id 'dagger.hilt.android.plugin'
}android {
    ...    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }
    kotlinOptions {
        jvmTarget = '1.8'
    }
}...dependencies {
    implementation fileTree(dir: 'libs', include: ['*.jar'])
    implementation project(path: ':common')    // Design
    implementation 'com.google.android.material:material:1.3.0'
    // Layout
    implementation 'androidx.constraintlayout:constraintlayout:2.0.4'
    implementation "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"
    implementation 'androidx.legacy:legacy-support-v4:1.0.0'    // Hilt
    implementation "com.google.dagger:hilt-android:$hilt_version"
    kapt "com.google.dagger:hilt-android-compiler:$hilt_version"
    // Hilt+Lifecycle
    def hilt_lifecycle_version = '1.0.0-alpha03'
    implementation "androidx.hilt:hilt-lifecycle-viewmodel:$hilt_lifecycle_version"
    kapt "androidx.hilt:hilt-compiler:$hilt_lifecycle_version"
    // Hilt Tests
    androidTestImplementation "com.google.dagger:hilt-android-testing:$hilt_version"
    kaptAndroidTest "com.google.dagger:hilt-android-compiler:$hilt_version"    // by viewModels() ext
    def activity_version = "1.2.0-rc01"
    def fragment_version = "1.3.0-rc02"
    implementation "androidx.activity:activity-ktx:$activity_version"
    implementation "androidx.fragment:fragment-ktx:$fragment_version"    // Lifecycle
    def lifecycle_ktx = '2.3.0'
    implementation "androidx.lifecycle:lifecycle-runtime-ktx:$lifecycle_ktx"
    implementation "androidx.lifecycle:lifecycle-livedata-ktx:$lifecycle_ktx"
    implementation "androidx.lifecycle:lifecycle-viewmodel-ktx:$lifecycle_ktx"
}

📲 Add Application

Add a new package: application
Add a new class: PlayerApp.kt

@HiltAndroidApp
class PlayerApp: Application()

Add android:name in AndroidManifest

<application
    android:label="@string/app_name"
    android:name="org.freedesktop.gstreamer.application.PlayerApp"
    android:icon="@drawable/gstreamer_logo_4"
    android:theme="@style/PlayerTheme.NoActionBar">

➰ PlayerViewModel

Add a new class: PlayerViewModel.kt

@HiltViewModel
class PlayerViewModel: ViewModel() {
}

Insert PlayerViewModel into Modify Tutorial4.kt

@AndroidEntryPoint
class Tutorial4 : AppCompatActivity(), SurfaceHolder.Callback, SeekBar.OnSeekBarChangeListener {
    // VM
    private val playerVM: PlayerViewModel by viewModels()

Add @AndroidEntryPoint
Replace Activity() to AppCompatActivity()

If you cannot import viewModels(), please check the steps above.

🔺 🏃Run, pass? Good, then back up💾.

🍗4. Prepare for a New GStreamer Player

< === Menu

👷🏻 NDKBuild — Android.mk

Let’s rename tutorial-4 to player at Android.mk:

LOCAL_MODULE    := player
LOCAL_SRC_FILES := player.c dummy.cpp

Save and Sync.

Rename tutorial-4.c to player.c.

Tutorial4.kt:

companion object {
    @JvmStatic
    private external fun nativeClassInit(): Boolean // Initialize native class: cache Method IDs for callbacks

    init {
        System.loadLibrary("gstreamer_android")
        System.loadLibrary("player")
        nativeClassInit()
    }
}

Run, an error shows up:

> Unexpected native build target tutorial-4. Valid values are: gstreamer_android, player...

Check Gradle:

externalNativeBuild {
    ndkBuild {
        def gstRoot

        if (project.hasProperty('gstAndroidRoot'))
            gstRoot = project.gstAndroidRoot
        else
            gstRoot = System.env.GSTREAMER_ROOT_ANDROID

        if (gstRoot == null)
            throw new Exception('GSTREAMER_ROOT_ANDROID must be set, or "gstAndroidRoot" must be defined in your gradle.properties in the top level directory of the unpacked universal GStreamer Android binaries')

        arguments "NDK_APPLICATION_MK=jni/Application.mk", "GSTREAMER_JAVA_SRC_DIR=src", "GSTREAMER_ROOT_ANDROID=$gstRoot", "GSTREAMER_ASSETS_DIR=src/assets"

        targets "tutorial-4"

        // All archs except MIPS and MIPS64 are supported
        abiFilters  'armeabi-v7a', 'arm64-v8a', 'x86', 'x86_64'
    }
}

Change targets to “player”. So NDKBuild must require same name of module among the files:

Android.mk — LOCAL_MODULE :=abc
Gradel —externalNativeBuild { ndkBuild { targets “abc” } }
Java class with NDK call — System.loadLibrary(“abc”)

🗽 Theme Error

🔺 🏃Run, error:

Caused by: java.lang.IllegalStateException: You need to use a Theme.AppCompat theme (or descendant) with this activity.

The AppCompatActivity is looking for a theme. Let’s add a full-screen version in res/values/style.xml:

<?xml version="1.0" encoding="utf-8"?>
<resources>
    <style name="PlayerTheme.NoActionBar" parent="Theme.MaterialComponents.DayNight.NoActionBar">
        <item name="windowNoTitle">true</item>
        <item name="windowActionBar">false</item>
        <item name="android:windowFullscreen">true</item>
        <item name="android:windowContentOverlay">@null</item>
    </style>
</resources>

AndroidManifest:

<application
    android:label="@string/app_name"
    android:name="org.freedesktop.gstreamer.application.PlayerApp"
    android:icon="@drawable/gstreamer_logo_4"
    android:theme="@style/PlayerTheme.NoActionBar">

🔺 🏃Run, pass? Good, then back up💾.

✂️5. Cut off Tutorial4 to Build NativePlayer

< === Menu

Add a new package called player
Add a new Kotlin file called NativePlayer.kt

Move the native data into NativePlayer from Tutorial4 activity

Move companion objects to NativePlayer from Tutorial4 activity

companion object {
    @JvmStatic
    private external fun nativeClassInit(): Boolean // Initialize native class: cache Method IDs for callbacks

    init {
        System.loadLibrary("gstreamer_android")
        System.loadLibrary("player")
        nativeClassInit()
    }
}

player.c: modify klass to NativePlayer at JNI_Onload()

jint
JNI_OnLoad (JavaVM * vm, void *reserved)
{
  JNIEnv *env = NULL;

  java_vm = vm;

  if ((*vm)->GetEnv (vm, (void **) &env, JNI_VERSION_1_4) != JNI_OK) {
    __android_log_print (ANDROID_LOG_ERROR, "tutorial-4",
        "Could not retrieve JNIEnv");
    return 0;
  }
  jclass klass = (*env)->FindClass (env,
      "org/freedesktop/gstreamer/tutorials/tutorial_4/player/NativePlayer");
  (*env)->RegisterNatives (env, klass, native_methods,
      G_N_ELEMENTS (native_methods));

  pthread_key_create (&current_jni_env, detach_current_thread);

  return JNI_VERSION_1_4;
}

Insert the NativePlayer back to the Tutorial4 activity

class Tutorial4 : AppCompatActivity(), SurfaceHolder.Callback, SeekBar.OnSeekBarChangeListener {
    // VM
    private val playerVM: PlayerViewModel by viewModels()
    
    // NativePlayer
    private val nplayer = NativePlayer()

📢 Add public method in NativePlayer

To solve the redline in Tutorial4, you need to add public access method for those private functions.

// native fun
// Initialize native code, build pipeline, etc
private external fun nativeInit() 
fun initJni() { nativeInit() }
// Destroy pipeline and shutdown native code
private external fun nativeFinalize() 
fun finalize() { nativeFinalize() }
// Set the URI of the media to play
private external fun nativeSetUri(uri: String?) 
fun setUri(uri: String?) { nativeSetUri(uri) }
// Set pipeline to PLAYING
private external fun nativePlay() 
fun play() { nativePlay() }
// Seek to the indicated position, in milliseconds
private external fun nativeSetPosition(milliseconds: Int) 
fun setPos(ms: Int) { nativeSetPosition(ms) }
// Set pipeline to PAUSED
private external fun nativePause() 
fun pause() { nativePause() }
// A new surface is available
private external fun nativeSurfaceInit(surface: Any) 
fun initSurface(surface: Any) { nativeSurfaceInit(surface) }
// Destroy surface
private external fun nativeSurfaceFinalize() 
fun surfaceFinalize() { nativeSurfaceFinalize() }

Now, we can fix some spots in the activity:

onCreate()

play.setOnClickListener {
    nplayer.is_playing_desired = true
    nplayer.play()
}
pause.setOnClickListener {
    nplayer.is_playing_desired = false
    nplayer.pause()
}

Remark the savedInstanceState because we’re going to use LiveData.

lgi("GStreamer -- "+
        "\nplaying:$nplayer.is_playing_desired"+
        "\nposition:$nplayer.position"+
        "\nduration: $nplayer.duration"+
        "\nuri: $nplayer.mediaUri")

// Start with disabled buttons, until native code is initialized
play.isEnabled = false
pause.isEnabled = false
nplayer.initJni()

Destroy()

override fun onDestroy() {
    nplayer.finalize()
    super.onDestroy()
}

✉️ setMessage(): Move it in NativePlayer; set LiveData to update the message.

The NativePlayer needs PlayerViewModel to transfer the data back to the activity, so we need to inject the VM into the player.

// inject vm
lateinit var vm: PlayerViewModel
fun setVM(vm: PlayerViewModel) { this.vm = vm }

onCreate() of Tutorial4:

// layout
setContentView(R.layout.main)

// vm => player
nplayer.setVM(playerVM)

Now, we can use PlayerViewModel to handle the message:

// player data
val message = MutableLiveData<String>()

init {
    message.value = ""
}

NativePlayer:

fun setMessage(inMessage: String) {
    vm.message.postValue(inMessage)
}

onCreate() of Tutorial4: add an observer

// vm => player
nplayer.setVM(playerVM)

playerVM.message.observe(this, {
    msgTV.text = it
})

♌️ setMediaUri(): Move it to NativePlayer

fun setMediaUri() {
    if (mediaUri == null || mediaUri!!.isEmpty())
        mediaUri = defaultMediaUri
    nativeSetUri(mediaUri)
    is_local_media = mediaUri!!.startsWith("file://")
}

Add defaultMediaUri.

➗ onGStreamerInitialized()

This one is special, you can keep the UI part in the activity.

NativePlayer: Move in the upper part of onGStreamerInitialized()

private fun onGStreamerInitialized() {
    lgi("GStreamer -- GStreamer initialized:")
    lgi("GStreamer --\nplaying:$is_playing_desired\nposition:$position\nuri: $mediaUri")

    // Restore previous playing state
    setMediaUri()
    nativeSetPosition(position)
    if (is_playing_desired) {
        nativePlay()
    } else {
        nativePause()
    }

    vm.gstInitialized.postValue(true)
}

PlayerViewModel:

// player data
val message = MutableLiveData<String>()
val gstInitialized = MutableLiveData<Boolean>()

init {
    message.value = ""
    gstInitialized.value = false
}

Tutorial4: onCreate()

// observers
playerVM.message.observe(this, {...})
// initialize GStreamer
playerVM.gstInitialized.observe(this, {
    runOnUiThread {
        play.isEnabled = true
        pause.isEnabled = true
    }
})

🆕 updateTimeWidget()

Update Tutorial4:

@SuppressLint("SimpleDateFormat")
fun updateTimeWidget() {
    val pos = sb.progress
    val df = SimpleDateFormat("HH:mm:ss")
    df.timeZone = TimeZone.getTimeZone("UTC")
    val message = df.format( Date(pos.toLong()) ) +
            " / " + df.format(Date(nplayer.duration.toLong()))
    timeTV.text = message
}

🔰 setCurrentPosition()

It’s the seekbar related data. Let’s remove setCurrentPosition() at the activity.

NativePlayer：

var seekbarPressed = false
fun setCurrentPosition(position:Int, duration:Int) {
    if (seekbarPressed) return

    // update seekbar
    vm.seekb.postValue(SeekData(position, duration))
    this.position = position
    this.duration = duration
}

SeekData.kt: Add a new class in player

data class SeekData(
    val position:Int,
    val duration:Int)

PlayerViewModel:

// player data
val message = MutableLiveData<String>()
val gstInitialized = MutableLiveData<Boolean>()
val seekb = MutableLiveData<SeekData>()

Tutorial4: onCreate()

// initialize GStreamer
playerVM.gstInitialized.observe(this, {...})
// update seekbar
playerVM.seekb.observe(this, {
    sb.max = it.duration
    sb.progress = it.position
    updateTimeWidget()
})

🔲 SurfaceView

override fun surfaceChanged(
    holder: SurfaceHolder, format: Int, width: Int,
    height: Int
) {
    lgd("GStreamer -- Surface changed to format " + format + " width "
                + width + " height " + height
    )
    nplayer.initSurface(holder.surface)
}

override fun surfaceCreated(holder: SurfaceHolder) {
    lgd("GStreamer -- Surface created: " + holder.surface)
}

override fun surfaceDestroyed(holder: SurfaceHolder) {
    lgd("GStreamer -- Surface destroyed")
    nplayer.surfaceFinalize()
}

🔳 onMediaSizeChanged()

Remove it from the activity.
Create a class in player to hold the screen data, MediaSize.kt:

data class MediaSize(
    val width:Int,
    val height:Int)

NativePlayer:

private fun onMediaSizeChanged(width: Int, height: Int) {
    lgi("GStreamer--Media size changed to " + width + "x" + height)
    val mediaSize = MediaSize(width, height)
    vm.mediaSize.postValue(mediaSize)
}

PlayerViewModel:

val seekb = MutableLiveData<SeekData>()
val mediaSize = MutableLiveData<MediaSize>()

Tutorial4:

// update seekbar
playerVM.seekb.observe(this, {...})
// get media size to surfaceview
playerVM.mediaSize.observe(this, {
    gsv.media_width = it.width
    gsv.media_height = it.height
    runOnUiThread { gsv.requestLayout() }
})

➖ Seekbar

Update Tutorial4:

// The Seek Bar thumb has moved, either because the user dragged it or we have called setProgress()
override fun onProgressChanged(sb: SeekBar, progress: Int, fromUser: Boolean) {
    if (!fromUser) return
    nplayer.desired_position = progress
    // If this is a local file, allow scrub seeking, this is, seek as soon as the slider is moved.
    if (nplayer.is_local_media)
        nplayer.setPos(nplayer.desired_position)
    nplayer.seekbarPressed = sb.isPressed
    updateTimeWidget()
}// The user started dragging the Seek Bar thumb
override fun onStartTrackingTouch(sb: SeekBar) {
    nplayer.pause()
    nplayer.seekbarPressed = sb.isPressed
}// The user released the Seek Bar thumb
override fun onStopTrackingTouch(sb: SeekBar) {
    // If this is a remote file, scrub seeking is probably not going to work smoothly enough.
    // Therefore, perform only the seek when the slider is released.
    nplayer.seekbarPressed = sb.isPressed
    if (!nplayer.is_local_media)
        nplayer.setPos(nplayer.desired_position)
    if (nplayer.is_playing_desired)
        nplayer.play()
}

…

🔺 🏃Run… Pass? Good then back up 💾.

😤: So far, I have decoupled the GStreamer code from the activity and couple them with the ViewModel. I hope you have the same result. Now, we can use ViewModel for different activities and fragments on Android.

Let’s Have Fun: GStreamer+Android Tutorial #4 — Common Module to Share; JNI Calls in Hilt &. MVVM

🏗️ 1. Insert a Common Module

♋️ 2. Translate Tutorial 4 to Kotlin

🌁 3. Reconstruct MVC with Hilt + MVVM

🍗4. Prepare for a New GStreamer Player

✂️5. Cut off Tutorial4 to Build NativePlayer

Written by Homan Huang