Anbox Streaming SDK

The Streaming Stack comes with

  • a Javascript SDK designed to help you get started with the development of a web based client. It handles all aspects of streaming, from the WebRTC protocol to handling controls, gamepads, speakers and screen resolutions.
  • a Native SDK offers a C API to provide full-featured video streaming that Javascript SDK provides but aims for a low latency to your application based on Anbox Cloud. The Native SDK is intended for C and C++ based application and currently supports Android and Linux.

You can find the SDK in the download page. The SDK comes bundled with examples to help you get started. They are located in the examples directory.

NOTE: the Native SDK is currently in beta and APIs under the beta version in Native SDK are subject to change. Use of these APIs in production applications is not supported.

Requirements

This guide assumes you have deployed the Anbox Streaming Stack.

Javascript SDK

The following HTML document should give you the minimum to get started.
Detailed explanations can be found below.

<!DOCTYPE html>
<html>
<head>
    <meta content="text/html;charset=utf-8" http-equiv="Content-Type">
    <meta content="utf-8" http-equiv="encoding">
    <title>Anbox Streaming SDK Example</title>
</head>
<body>
    <script type="module">
    import {AnboxStream, AnboxStreamGatewayConnector} from './anbox-stream-sdk.js';

    const connector = new AnboxStreamGatewayConnector({
        url: 'https://gateway.url.net',
        authToken: 'YOUR_AUTH_TOKEN',
        session: {
            app: "com.foo.bar",
        },
    });

    let stream = new AnboxStream({
        connector: connector,
        targetElement: "anbox-stream",
        screen: {
            width: 1280,
            height: 720,
            fps: 25,
        },
        controls: {
            keyboard: true
        },
        callbacks: {
            error: error => {
                console.log("AnboxStream failed: ", error);
            }
        }
    });
    stream.connect();
    </script>

    <div id="anbox-stream"></div>
</body>

Let’s go through each step here:

const connector = new AnboxStreamGatewayConnector({
    url: 'https://gateway.url.net',
    authToken: 'YOUR_AUTH_TOKEN',
    session: {
        app: "com.foo.bar",
    },
});

Behind the scenes, the SDK is actually comprised of two parts. The connector takes care of talking to the stream backend (in this case the Stream Gateway, but this could be replaced with your own middleware) and initiating the WebRTC setup.
The Anbox Stream takes care of displaying the video and audio feed and handle controls, lifecycle events, and more.

The distinction between the two is made to make it easier to plug your own software in the SDK rather than having to re-write everything again.

In this case, the connector is made to talk directly to the Stream Gateway and thus needs its location and an access token. It also needs to know which Android application to start.

gateway.url.net should be replaced with the Stream Gateway IP/Domain name. You can get this information by running juju status.
YOUR_AUTH_TOKEN is a token [generated with the Stream Gateway](manage-gateway-access.md.
com.foo.bar is the name of an application added via ams. You can list all applications by running amc list.

let stream = new AnboxStream({
    connector: connector,
    targetElement: "anbox-stream",
    screen: {
        width: 1280,
        height: 720,
        fps: 25,
    },
    controls: {
        keyboard: true
    },
    callbacks: {
        error: error => {
            console.log("AnboxStream failed: ", error);
        }
    }
});

This is the main class. It takes the previously created connector and prepares the browser to handle the stream properly.

targetElement: "anbox-stream",

The SDK needs an HTML element where it can attach the video, targetElement is the ID of that element.
In this case you’d need to add the following to your HTML body:

<div id="anbox-stream"></div>

Note: Make sure events can reach this element, otherwise controls will not work.

The default behavior of the video is to fill the maximum space given by this element while keeping aspect ratio intact.

screen: {
    width: 1280,
    height: 720,
    fps: 25,
},
controls: {
    keyboard: true
},
callbacks: {
    error: error => {
        console.log("AnboxStream failed: ", error);
    }
}

The rest is mostly optional and is made to customize the stream. You can find a complete inline documentation in the SDK.
Note that you can register callbacks to be notified at specific points in the stream lifecycle.

stream.connect();

Once everything is ready, you can start the connection and start the stream.

Native SDK

This native SDK contains a header file to include in your C or C++ based program and the shared library to link against.

To integrate the native SDK to your program. You need to initialize AnboxStreamConfig object first, and set the concrete config items via the object.

    std::unique_ptr<AnboxStreamConfig, void(*)(AnboxStreamConfig*)> cfg(
      anbox_stream_config_new(),
      [](AnboxStreamConfig* cfg) { anbox_stream_config_release(cfg); });

    // The SDK does handle the signaling process for us. All we have to do is
    // to tell it the URL of the signaling endpoint
    EXIT_ON_FAILURE(anbox_stream_config_set_signaling_url(cfg.get(), session_url));
    EXIT_ON_FAILURE(anbox_stream_config_set_use_insecure_tls(cfg.get(), use_insecure_tls));
    EXIT_ON_FAILURE(anbox_stream_config_add_stun_server(cfg.get(), stun_server_urls, size_of_urls, username, password));

After all configurations are set you can create the Anbox Stream object. You probably need to hold the stream by a custom context object as the stream is required for a few other functions to be invoked later on.

    Context ctx;
    ctx.stream = anbox_stream_new(cfg.get());

Then in order to monitor the WebRTC peer connection state, disconnection signal, or error occurring during streaming. The following callback functions are expected to be registered. Those callback functions accept a valid anbox stream object that is created above.

    EXIT_ON_FAILURE(anbox_stream_set_connected_callback(ctx.stream, [](void* user_data) {
        auto ctx = reinterpret_cast<Context*>(user_data);
        # Start rendering thread after streaming gets connected.
    }, &ctx));


    EXIT_ON_FAILURE(anbox_stream_set_disconnected_callback(ctx.stream, [](void* user_data) {
       auto ctx = reinterpret_cast<Context*>(user_data);
       # Release the resource when streaming get disconnected.
    }, &ctx));


    EXIT_ON_FAILURE(anbox_stream_set_error_callback(ctx.stream, [](AnboxStatus status, void* user_data) {
       # Respect the status and run error handling routine.
    }, nullptr));

To capture the audio output data and playout on the client side. You need to register the following function and process the received PCM audio data via platform specific API for playback

    EXIT_ON_FAILURE(anbox_stream_set_audio_data_ready_callback(ctx.stream, [](
        const uint8_t* audio_data, size_t len, void *user_data){
      auto ctx = reinterpret_cast<Context*>(user_data);
      # Process PCM audio data for playback
    }, &ctx));

After all set, you can call the following function to connect the stream

    EXIT_ON_FAILURE(anbox_stream_connect(ctx.stream));

To forward input events occurring on the client to the Android container, you need to construct the AnboxStreamControlMessage based on actual input event via platform specific API.

      AnboxStreamControlMessage msg;
      memset(&msg, 0, sizeof(msg));

      ```
         # Construct control message according to different platform API
      ```
      anbox_stream_send_message(ctx.stream, &msg);

In the user defined render thread, it’s expected to set proper viewport size and render each frame immediately once a frame is available.


void run_render_thread(Context* ctx) {
  ...
  ...
  eglMakeCurrent(display, surface, surface, context);

  while (ctx->running) {
    anbox_stream_set_viewport_size(ctx->stream.get(), width, height);
    anbox_stream_render_frame(ctx->stream.get(), 100);
    eglSwapBuffers(display, surface);
  }
}

And when streaming is done, the client is in charge of releasing the Anbox stream object.

    EXIT_ON_FAILURE(anbox_stream_release(ctx.stream));