Multiscreen (App-to-app) Communication

From SAM
Jump to: navigation, search

This page describes solutions for Multiscreen App-to-app Communication which are out there in the field. The most important technologies for it is NSD and DIAL, so these get most of the attention. The DIAL and NSD protocols and its components are briefly explained by using a few sequence diagrams.

Introduction

For Multiscreen App-to-app Communication, there are 5 important technologies available as per today: Samsung's Multiscreen SDK[1], Qualcomm's Alljoyn[2], Network Service Discovery (NSD) [3], DIAL [4] and Google Cast[5]. As the first two are proprietary in usage and thus out-of-scope for this project, this document will only focus on the freely available NSD, DIAL and Google Cast technology. This page describes what they are, how they work and show some explanatory sequence diagrams. Google Cast is used as a basis for the Chromecast dongle[6], developed by Google.

Relevance to SAM

In the SAM project there are 2 devices in play: there is a 1st Screen device, which mainly shows the video and some metadata and there is a 2nd Screen device, e.g. a tablet that has widgets and feeds with extra information for the video that is being watched on the 1st Screen device. Because these 2 devices are not independent of each other, there is a natural need for communication between the 2. On both devices, there will be some kind of an app, that will display metadata information, so these 2 apps can make use of of app-to-app communication. To make the SAM platform user friendly, both devices should discover their counterpart app and launch it. That is where Multiscreen app-to-app communication comes into the picture. This section can also be used in WP3: Architecture, Specification and Integration as well as WP7: Multi-Device Media Representation and Interaction.

State of the Art Analysis

NSD

What is NSD?

  • NSD[3], short for Network Service discovery, is a simple protocol which on adding to the app allows the users of app to discover other devices on the local network that support the services the app requests. This is useful for a variety of peer-to-peer applications such as communication between 2nd screen application with 1st screen for exchange of information. Android's NSD APIs simplify the effort required to implement such features.

Service Registration on the Network

NSD[3] allows the app to advertise the app's services over the local network. To register and advertise the service on the local network, an NsdServiceInfo object needs to be created. This object is used to feed the information that other devices on the network use when they're deciding whether to connect to the registered service. While registering a service name is given. The name is visible to any device on the network that is using NSD[3] to look for local services. The name must be unique for any service on the network and Android automatically handles conflict resolution.

Note: The International Assigned Numbers Authority (IANA) manages a centralized, authoritative list of service types used by service discovery protocols such as NSD[3] and Bonjour.

Discover Services on the Network

For the client and server app to communicate with each other or before they start to exchange information with each other, the client app needs to discover the server app over the network. This functionality is known as service discovery. This mechanisms allows a NSD client to discover NSD servers as long as both are connected to the same local network.The client application needs to listen to service broadcasts on the network to see what services are available, and filter out anything the application can't work with.

Service discovery, like service registration, has two steps: setting up a discovery listener with the relevant callbacks, and making a single asynchronous API call to discoverServices().

The NSD[3] API uses the methods in the interface to inform the application when discovery is started, when it fails, and when services are found and lost (lost means "is no longer available"). It does several checks when a service is found.

  • The service name of the found service is compared to the service name of the local service to determine if the device just picked up its own broadcast (which is valid).
  • The service type is checked, to verify it's a type of service your application can connect to.
  • The service name is checked to verify connection to the correct application.

Connect to Services on the Network

When the application finds a service on the network to connect to, it must first determine the connection information for that service.

Unregister the Service on Application Close

It's important to enable and disable NSD[3] functionality as appropriate during the application's lifecycle. Unregistering your application when it closes down helps prevent other applications from thinking it's still active and attempting to connect to it. Also, service discovery is an expensive operation, and should be stopped when the parent Activity is paused, and re-enabled when the Activity is resumed.

DIAL

What is DIAL?

  • DIAL[4], short for DIscovery And Launch, is a simple protocol which can be used by a 2nd Screen device for discovering and launching apps on a 1st Screen device, such as televisions, set-top boxes, HDMI-dongles (like Chromecast) or even Blu-ray players.
  • Used by NetFlix and YouTube
  • DIAL Protocol consists of two components:
    • DIAL Service Discovery
    • DIAL REST Service
  • The DIAL specification[7] is freely available on their website.

DIAL Service Discovery

This mechanisms allows a DIAL client to discover DIAL servers as long as both are connected to the same local network. The DIAL servers are hosted on 1st Screen devices. Once such a server is discovered, it can get access to the DIAL REST Service that is available on those 1st Screen devices. The DIAL Service Disovery relies on SSDP (Simple Service Discovery Protocol) version 1.1. This protocol is a part of the the UPnP protocol [8]. The advantage of using SSDP is that there is no pairing necessary between the devices, although communication for client to server is possible.

Below the mechanism of discovery is depicted. Essentially, the DIAL client sends an M-SEARCH request (including a Search Target header) over UDP to the server, which then responds with an M-SEARCH response, including a LOCATION header. The client then issues a HTTP GET request to the URL that it received in the LOCATION header. Finally, when the server will respond with a HTTP response which contains the UPnP device description and an additional header Application-URL containing the absolute URL of the DIAL REST Service.

Sequence diagram that explains the DIAL Service Discovery

DIAL REST Service

  • Contains ways to
    • Launch applications
    • Pass on parameters for launch
    • Provide new arguments to a running application
    • Stopping of an application
    • Querying for application information (e.g. application is running, installable, etc.)

The DIAL REST Service represents applications (like YouTube, Netflix, etc.) by URL's. The list of applications that are available for launch is defined by the DIAL Server. If a DIAL Client wants to perform a certain operation in an application, then it issues a HTTP request against the corresponding URL. Launching apps & passing parameters or new arguments to these apps is done by using HTTP POST requests. Stopping applications can be done by using HTTP DELETE request and HTTP GET requests can be used for querying application information.

Extensions to DIAL

  • Add a DIAL Communication Service
  • The DIAL Communication Service URL is communicated via the application information
  • DIAL REST Communication Service provides
    • HTTP GET interface to retrieve data from the running application
    • HTTP POST interface to the sent data to the running application

Example of DIAL REST Communication Service

Sequence diagram that shows an example which uses DIAL REST Communication

Google Cast

What is Google Cast?

Similar to DIAL, Google Cast is a multi-screen technology that allows users to control their 1st Screen device with a 2nd Screen device (phone, tablet or even a PC). The software on the 2nd Screen device is developed in the appropriate Google Cast SDK (Android, iOS or Chrome) and is able to discovery the 1st Screen device. The 2nd Screen will launch a receiver application on the 1st Screen device is able to send content to large screen through the sender API's.

Features of Google Cast

Chromecast dongle
  • Start Web App + cast content
    • Push the URL of Cloud content to Chromecast dongle and play it on the TV which uses a downloaded, lightweight web app to stream the content, from the Cloud and render the UI.
    • Mobile device can control the content that is being streamed (play, stop, volume, seek).
    • Users can add video’s to the current Playlist (e.g. YouTube usecase)
    • Users can take over an active Google Cast session
  • Mirroring
    • Mirror content of Chrome browser tab to the TV
    • Mirroring entire screen (experimental, platform dependent)
  • Supports HDMI-CEC
    • For automatically waking up the TV upon a Google Cast request
    • For automatically switching to the HDMI source the dongle is connected to

Interface description

Google has great documentation on how to write both sender and receiver apps. In the context of SAM, the documentation for developing a receiver application would be necessary for the 1st Screen device, while sender application documentation is necessary for the development of the 2nd Screen device application.

Ecosystem

Currently, the Google Cast feature works on the popular Chromecast dongle, manufactured by Google, which is being sold through Google Play Store or through various resellers, such as Amazon.

The Google Cast-enabled apps quickly rose to more than 400 apps across platforms. An overview of the most relevant apps can be found here.

References

  1. Samsung's Multiscreen SDK website
  2. AllJoyn website
  3. 3.0 3.1 3.2 3.3 3.4 3.5 3.6 NSD website
  4. 4.0 4.1 DIAL website
  5. Google CAST website
  6. Chromecast website
  7. DIAL specification (pdf)
  8. UPnP website

Categories