2nd Screen is a term that refers to an additional electronic device or devices that allow the user to retrieve extra data about the content they are watching on the so-called “first screen", usually the TV which may be a SmartTV or linked to a connected Blu-ray player, set-top-box or similar device.
- 1 Introduction
- 2 State of the Art Analysis
- 3 SAM Approach
- 4 References
2nd Screen can be called a “companion device” and so whilst watching or engaging with content on the “first screen”, additional contextual information and content can be displayed on a companion device such as a tablet, smartphone or computer screen but for a true 2nd Screen experience there are three basic conditions that must be met. The first is connectivity, the second is typically app functionality and the third, obviously is, it must have a screen. Therefore laptops, handheld gaming units, tablets, and smartphones are all able to be used as a second screen, however the latter two are currently the most popular options.
Relevance to SAM
SAM will develop an advanced asset delivery platform based on “2nd Screen and Content Syndication within a Social Media context”. The SAM 2nd screen ecosystem will provide open and standardised ways of characterising, discovering and syndicating media assets interactively. These digital assets will then be deployed to the different 2nd screens the user can be using (computers, tablets, smartphones, smart TVs, etc.) to create richer consumer experiences.
A number of tasks within the SAM work packages will deliver key elements which will potentially impact on the development of the 2nd Screen ecosystem including:
- WP2: 2nd Screen, Social Media Vision and Challenges: Tasks within WP2 will describe the current state of the 2nd Screen target market sector and its ‘players’ and will identify the opportunities and challenges and be used to guide the RTD to develop the prototypes and also aid the evaluation in WP8 and the exploitation in WP9. In addition information core requirements for the 2nd Screen ecosystem will be gathered from the expertise, experience and industry knowledge of the partners.
- WP5: Content Syndication and Delivery: SAM will work to define an open asset format and Open Specification API so that it could be shared with other initiatives dealing with 2nd Screen technologies, entertainment data and social media.
- WP6: Context Analysis & Dynamic creation of social communities: The social components that will compose the SAM core social tools, will allow sharing user-generated content like comments and ratings both on SAM's and in different social networks and these components will be rendered at the 2nd Screen SAM Dashboard
- WP7: Multidevice media representation and interaction: Tasks within WP7 will allow synchronisation of consumer information and context, based on the user interactions and data provided such as the different assets presented at the different 2nd screens the user can be using (computers, tablets, smartphones, smart TVs, etc.). In addition a SAM 1st screen media interaction component will apply an interaction and presentation layer for viewing media content and related media assets. It will implement and support the communication tasks with 2nd screen applications, like synchronization and screen swapping (Moving a media asset with a gesture from 2nd to 1st screen) and will render the optimum view of the displayed media content and its assets. Configuration of the different devices and contents that a user is using at a specific time will be managed by a 2nd screen focused on REST and HTML5 technologies and generic widget viewers apps will also be considered.
- WP8: Use Cases: SAM in Action: The validation of primary principles and functionalities of SAM will be carried out in use cases which will aim to implement many parts of the content creation, sharing and social consumption through the 2nd Screen usage as applicable to its case. The scenario for testing the advanced social capabilities of SAM is based on the consumption of consuming common media resources (e.g. DW Euromaxx programme) simultaneously from different locations and the interaction of the users on the dynamic social communities created. DW and BDS will prepare and provide complex assets, including services, and editorially curated information for consumption via the 2nd Screen devices.
Within the 2nd Screen ecosystem a number of opportunities have been identified as SAM will enable:
- Creation of dynamic context driven communities
- Social exchange of information via various social media types
- Creation of personalised user experiences based on dynamically changing context, comments and ratings
- Provision of an abundance of rich curated content
- Metadata providers to enhance rich content through linked ratings, reviews and recommendations
- Multi-device data representation and interaction based on syndicated assets within a social/televisual context.
- Capture and analysis of user activity within a ‘protected’ environment
- Advanced business intelligence techniques to detect trends, tendencies, comments, reviews etc. allowing for fact-based, business decision making
- Targeted advertising, interactive ads and the ability to disseminate through social media.
- Marketeers to providers to create cost-efficient media-rich campaigns to drive business
- Allow broadcasters to develop and amend programmes and plots based on social media responses
State of the Art Analysis
A 2nd Screen application must have some kind of ‘discovery and launch’ system to allow the applications to synchronise and this is usually based on Automatic Content Recognition (ACR). There are a number of technologies which have developed to enable ‘trigger actions’ and these include:
- Audio ACR via audio fingerprints and watermarks captured and recognised by microphone on the 2nd screen device.
- Operating system-level (OS-level) ACR via a ‘timecode aware’ device which identifies the timecode and triggers an event in the 2nd Screen application.
- Video stream capture (NOC) ACR whereby video stream recognition techniques are usually found in the broadcaster Network Operations Centre and can remotely trigger events in the relevant 2nd Screen app. This technology is being successfully used for synchronisation of advertisements and in live events such reality shows, award ceremonies and live sporting events such as the Olympics.
- Video ACR is an emerging technology based on video fingerprinting and watermarking. This is especially being seen in connected SmartTVs or set-top-boxes (STB) where the SmartTV/STB synchronises information via a data interface with the 2nd Screen.
- Close Caption ACR is a technology gaining interest, as it allows an app to recognise and action events through a set of triggers in the closed caption text. Furthermore 20 February 2014 the US Federal Communication Commission set new rules for TV closed captioning linked to news, entertainment and information for individuals who are deaf or hard-of-hearing which may well stimulate greater use for 2nd Screen.
2nd Screen Opportunities
The first mainstream second screen sync app around a major television show was Grey's Anatomy Sync for iPad which was released in Feb. 2011 and which started a concept now extends to dozens of shows on prime time and cable across the major networks and continents. Since then an ecosystem has developed which is transforming the experience of hundreds and millions of households across the U.S. and Europe by offering greater viewer engagement across a few primary activity sets related to future 2nd Screen revenue streams:
- Discovering content to watch within the ecosystem, through guide, ‘search and discovery’, reviews, ratings and recommendations.
- Enhancing the viewing experience through searching for or receiving additional (perhaps synchronised) related information to the programme along with 2nd Screen-based commerce and interacting with programmes. Delivering relevant information to the consumer when it is happening e.g. identifying the background music, links to actor filmographies, biographies and photos with director’s notes, etc. all of which creates a more compelling and engaging experience.
- Sharing views and opinions through contextual and spoiler-proof social feeds and through comments and voting, influencing decisions around content. This has already been trialled by the broadcaster CBS in January 2014 in an episode of 'Hawaii Five-0' when, after all of the suspects' motives are revealed, viewers went to CBS.com or Twitter to select one of three options: #TheBoss, #TheTA or #TheStudent. Votes were tallied immediately during the episode, and the most popular ending became part of the broadcast during both the East and West broadcasts. Based on the results different endings were shown on East and West Coasts, with all three endings were available on CBS.com following the broadcast.
- Contextual advertising is seen as the Holy Grail for interactive advertising, allowing for positioning of suitable brand messaging not only based on demographics of the viewer, but also based on what is happening on the first screen and previous consumer actions, so reinforcing brands in a dynamic contextual setting.
- Contextual commerce will provide the prospect to finally deliver ‘opportunities to buy’ by offering buy (or want) clicks to the consumer based on analysis of viewing, previous interest and activities.
- Controlling the SmartTV or STB, whilst perhaps the hardest to value in monetary terms for 2nd Screen, this is an important feature for device makers and those hoping to win control of the digital video ecosystem. Whilst gesture control is now available through Kinect or Leap devices, voice and speech recognition offers the opportunity for a better experience for the consumer.
2nd Screen Key Trends
At the beginning of 2103, the 2nd Screen Society predicted it thought would be key trends with the following results:
- There was a convergence of viewing and companion experiences as consumers choose increased use of mobile devices leading to greater potential growth for inline video advertising.
- DIAL becomes a serious protocol control (Discover And launch) http://www.dial-multiscreen.org
- An ecosystem developed by some players whereby any app using DIAL protocol can use available technologies for 2nd Screen synchronisation as exemplified by and Microsoft’s Smartglass where one app delivered a companion experience for video, music and games.
- Monetisation was bigger than forecast for many companies as exemplified by Viggle who reported revenue for the latest quarter of $4.6 million, a growth of 289% of the previous 12 months. Projected revenue for 2014 is more than $20 million.
- Consolidation continues and discovery takes off as demonstrated by acquisitions such as Dijits’ of Miso, i.TV’s of GetGlue and Tivo’s purchase of Digitalsmiths
- Device makers embraced and committed to 2nd Screen e.g. Samsung, Google Chromecast.
- There was an explosion of ad supported video due to the targetable nature of 2nd Screen and the developing easy interactive capabilities.
- There was a rise of responsive HTML 5 apps as consumers chose engagement with entertainment content through mobile browser.
- Social feeds became much more than a feature as shown by the success of Twitter which has made ‘social’ something much larger, as revenues increased.
- Gamification continues to grow and Viggle is leading the way making TV more engaging by rewarding the viewer who ‘checks in’ whilst watching, playing trivia games and interacting via Twitter or Facebook.
- eCommerce once thought to be ripe for exploitation via Ebay and Amazonhas not grown as anticipated as current second-screen development practice makes it uneconomical as apps are developed as part of individual commissions making the costs of integration with ecommerce systems uneconomical.
- The rise of companies developing features to allow consumers to block displays and video advertising (ad blockers), which was once seen as the best method of monetisation.
However as well as opportunities there are the key challenges (in addition to ad blockers) as along with the potential of 2nd Screen comes the requirement for meaningful, high-quality content creating the demand for resources to generate it. Many broadcasters assert that currently there is not enough time, talent or funding to give 2nd Screen the attention it and the audience deserve, and they struggle with using technology to create an immersive engaging experience which will show a real return on investment. Many in the TV industry have concerns that real-time viewing on both screens will pull attention from the primary content, leaving viewers with a disjointed experience, which could hurt the brand. As broadcasters seek ways to monetise the Second Screen experience, many feel they cannot risk neglect first screen material to create 2nd Screen content that is truly complementary and not just promotional.
Companies involved in ACR and ‘discovery and launch’ include
- Mufin - http://www.mufin.com/
- Civolution - http://www.civolution.com/
- Audible Magic - http://www.audiblemagic.com/ (http://developer.audiblemagic.com)
- Vobile - http://www.vobileinc.com/
- Samba TV - http://www.samba.tv/
- Netflix (DIAL) - http://www.dial-multiscreen.org/
- Intrasonics - http://www.intrasonics.com/
- Zample - http://zample.com/
- Gracenote - https://developer.gracenote.com/
- Advanced Television Systems Committee (ATSC) http://www.atsc.org/cms/standards/ATSC-3-Watermark_final-CFP.pdf
- SeeSpace InAiR – http://www.seespace.co/
Whichever ACR system is used, while watching a TV show or movie, the app knows to display certain information at specific times in the broadcast and because the content is able to be pre-programmed, users can access the content even if they don't see the show live. This is especially significant for the ‘catch up’ viewing experience.
The 2nd Screen component enables the prosumption of content corresponding to the current video which the End User is viewing.
Architecture and Dependencies
The architecture of the 2nd Screen component is presented in the following image. The component provides the Generic Dashboard component to the End User and. More specifically these features are or will be provided:
- The 2nd Screen component acts as a container for the Generic Dashboard component.
- It will also contain the Inter-Device-Communication subcomponent, which will enable the communication between the 2nd Screen and 1st Screen component on a local network.
- Additionally the 2nd Screen component will provide the detection of the 1st Screen component based on a DIAL-like approach and enable Android notifications.
Implementation and Technologies
After extended analysis and comparison Android as the most appropriate technology has been selected. More specifically, this example implementation of an 2nd Screen app is based on Android 5.0.
The 2nd Screen component is connected to two different components:
- the Identify and Security Services (ISS) component for login purposes and
- the Generic Dashboard component for additional content display.
The ISS component provides user information, which are needed to retrieve additional information through the Generic Dashboard component. The GD component is integrated using an Android WebView component and provides the End User additional information.
A summary of the tasks carried out for each subcomponent of the first version of the prototype is shown in the following table.:
|Voice Dialogue||Basic integration into the Android application and setup of the TDM back end|
|Device Controller||Provision of device specifications and GPS data to the Generic Dashboard component using the Dashboard Viewer subcomponent|
Functionality and UI Elements
The Login section enables the End User to login into the SAM 2nd Screen app.
The Content section will be shown if the End User has successfully logged in. At the top of the screen it provides the user with an so-called action bar, which contains remote control features, the Voice Dialogue button and a settings menu. The rest of the screen is covered by the Generic Dashboard component, which provides additional information depending on the video consumed on the 1st Screen component.
The following features have been added to the 2nd Screen application:
- The 2nd Screen application is using Network Service Discovery protocol (NSD) to discover local 1st Screen devices and connect to them.
- Voice Dialogue supports many different ways for user interaction.
- Support for social interactions (Facebook, Twitter, SAM communities)
- Support for controlling video playback
- Support for PTI (Push-To-Initiate) where whole interactions can be accomplished without the user pressing any button
- Latitude, The Future of Storytelling
- 2nd Screen Society - Quarterly Second Screen Market Trend Update http://digitalvideospace.blogspot.co.uk/2014/02/quarterly-market-trend-update-on-second.html