Asset Aggregation

From SAM
Jump to: navigation, search

Introduction

The era of Web 2.0 brought an exponential increase of the data creation and distribution through the Internet. This effect makes the search, distribution and representation of information through internet more complex and difficult to accomplish correctly. The difficulty and complexity is created because simple search results are not enough. More specifically, users need a cluster of information (video, photos, articles etc.) presented to them in order to have an overall view and be well informed regarding their topics of interest. Data aggregation is the technique for accomplishing this presentation of data. Search engines or recommendation systems use data aggregation techniques in order to provide a strong context aware environment to the users.

Relevance to SAM

Data Aggregation is an important feature of SAM Platform, as SAM connects several different types of sources that provide their data and digital assets. More specifically, SAM connects social media, television, e-commerce and other sources. So, in order to present them correctly, a mechanism to aggregate all these digital assets is needed. An aggregator will manage all the assets and create a cluster of related digital assets. The asset aggregation in the context of SAM is described in the T5.3 Asset Aggregation and Composition in WP5 Content Syndication and Delivery.

State of the Art Analysis

Aggregated search

Aggregation in essence relies on the relation between digital assets. Therefore aggregation techniques found fertile ground to grow and be established in the most common and used system. This system is search engines, which not only return results for specific queries, but create a cluster of information and digital assets to present a more advance and useful search results. These results contain multiple digital assets (images, video, news, etc.) linked to the search queries. In addition the search results are not only related to the search query but in the relation the query has with different digital assets.[1]

Cross-vertical Aggregated Search (cvAS)

This framework is used widely in today's Search engines. The major attribute that has is including a variety of content such as (images, video, news, etc.) within the basic search results by using vertical search engines. Vertical search engines specialize work well with this kind of content (maps, images, video, news, etc.). Taking under consideration the Content-digital asset SAM produces this kind of search engines will be very suitable to be used with-in SAM platform.

Relational Aggregated Search (RAS)

Relational Search is a search framework that produces relational results on search queries. More specifically, the results returned by the systems are not only related to the search query itself but also to the relations this query has with different digital assets. Relational creation is a key ingredients in aggregating data, because digital assets should have a reason to be aggregated and serve a purpose. The relations and correlations between them is the most major reason to aggregate them.

Aggregation and recommendation

Data aggregation is used in a variety of systems, one of them is recommender systems. In order for these systems to create group recommendation they must establish a data aggregation mechanism. This necessity is created from evaluating typical recommended systems that use user group aggregation or user group preferences aggregation in order to create group recommendations. The evaluations shows that embedding data aggregation technics and strategies improve significantly the performance of the recommendation systems.[2]

Aggregation techniques

Single Stream Aggregation

Single stream aggregation takes multiple sources and combines them into a single stream of information.

Aggregator

Aggregator is a client software or a web application which aggregates syndicated content such as news, headlines, blogs podcasts and vlogs in a single location unifying the content.

Edited Aggregation

Edited aggregation combines content from trusted sources and is monitored and controlled by human editors.

Data Aggregation Tools

Sqoop

Sqoop is a product created by Apache, it is designed to move mass data between Hadoop and structured data stores effectively. The process is mainly automatic and uses MapReduce to import and export data, providing also fault tolerance and supports parallel operations.

Flume

Flume is also created by Apache and is an efficient and distributed system for collecting, aggregating and moving huge amounts of log data. The Flumes aims to create a framework for moving data between applications and Hadoop.

Chuhwa

Chuhwa is built on top of MapReduce framework as well as Hadoop Distributed file system. This attribute makes it robust and scalable. It also has a flexible toolkit for monitoring, analyzing ad displaying results[3].

Popcorn.js

Popcorn.js is an HTML5 media framework written in JavaScript for filmmakers, web developers, and anyone who wants to create time-based interactive media on the web[4]

Aggregation techniques

Single Stream Aggregation

Single stream aggregation takes multiple sources and combines them into a single stream of information.

Aggregator

Aggregator is a client software or a web application which aggregates syndicated content such as news, headlines, blogs podcasts and vlogs in a single location unifying the content.

Edited Aggregation

Edited aggregation combines content from trusted sources and is monitored and controlled by human editors[5].

Asset Aggregation and Asset description

A very important element for asset aggregation creation is the schemas and vocabularies used by the aggregation mechanisms (aggregators). In order to accomplish simple or dynamic aggregation is important to use the appropriate schemas. EDM is a European initiative for open data linking. It uses the RDF schema to describe the data provided. More information and Schemas can be found in the Asset Description article.

SAM Approach

The Linker component is responsible for the Asset aggregation and composition for the SAM Platform. With this tool, the Content Providers will be able to enrich the Assets with additional information coming from other Assets, web or social network sources. The Linker allows the Content Providers to define easily but also effectively all aspects of the new, composite Asset covering among others the aspects of asset time-line, audience types, delivery channels etc. This component produces new Assets, annotated and enriched with information from various sources so as to create a rich 2nd Screen experience for the End Users. In the next paragraph an overview of the Linker Component architectural diagram is provided in order to understand the component linking and interaction.

Architecture and Dependencies

The Linker component, consisting both from GUI elements and services, as presented in the following image, provides several key features to the content providers for creating complex assets. More specifically:

  • The main feature of this component is the creation and editing of asset compositions so as to define what additional content will be syndicated, when and to whom. As part of this process content providers are able to search for related assets through specific keywords and criteria.
  • The GUI of the Linker also supports “on-the-fly” preview of the asset composition which is under production and finally, the Linker produces the description document for the asset that is produced following the asset description language defined in SAM.
  • The Linker allows content providers to load and configure modules which enable the manipulation of the respective content types as part of their asset linking project. To this direction the Linker component is not limited to support specific content types, but it can be extended to additional ones through these modules or the creation of new ones.

ArchitectureLinker.jpg

Implementation and Technologies

After Extended Analysis and comparison the most appropriate technologies for the frontend and the backend have been selected.

Frontend Technologies (User Interface)

The user interfaces Asset Editor, Asset Preview and Asset Module Configurator will be implemented using the technology AngularJS. Also because the aforementioned user interfaces will be embedded inside the marketplace it is reasonable to choose this technology to provide a common look and feel [6].

Backend Technologies (Web Services)

The Linker prototype will use the JAX-RS technologies and more specifically the spring.io framework in order to implement the RESTful Web Services for Linker. Even though Jersey is more suitable for a simple RESTful implementation, because it is not overloaded with features and dependencies, spring.io is powerful, robust, complete in terms of features and well-maintained. This should considerably ease the implementation, deployment and testing of the Linker-backend environment[7].

Subcomponents

A summary of the tasks carried out for each subcomponent of the first version of the prototype is shown in the following table.

Subcomponent Task
Linker Multiscreen Timeline editing Create a module in the UI to support the timeline editing of the Assets
Linker Asset Composition The basic module for Asset Linking, can load connect

and create Assets

Linker Project Manager The project manager is a UI module for accessing,

saving and creating new Linking Projects

Linker UI Include SAM,look and feel.

Integrate the,interface in the Marketplace

Linker Services Define services for accessing, storing, deleting and

creating new Assets.

Functionality and UI Elements

This section describes the procedure to create a linked Asset, manipulate an existing one or previewing linked Assets. The two basic procedures are the Asset Composition and the Asset Timeline Linking. The first one allows the user to create linkage between Assets and the second one takes the linking a step further by allowing the linking of Assets for a specific time frame during a film or other video type Assets. The purpose of the preview is to help the users to check the correct alignment of the connected assets in order to avoid errors such as overlapping or wrong placement of Assets.

SAM Marketplace and Linker

The Linker is embedded inside the SAM Marketplace, so users need to be registered in the Marketplace to access the Linker. In the Marketplace interface users can access the Linker from the left side vertical menu.

Ui.jpg

Linking

The first step towards creating a Linking Project and consequently Link Assets is defining or accessing a Linking Project. In order to access, an existing Linking Project users should select from the dropdown list in the top menu the Load Project field.

Loadasset.jpg

Asset Composition

Asset Composition is the core component of the UI for linking Assets. The first linking a user can create through this UI is a generic one. The generic linking is the association of the Root Asset with specific keywords and also giving a style for presenting these keywords (i.e. Default Theme). Going further, users can search for available Assets or create new ones in order to enrich the linking that is created.By clicking on the “plus” icon in the actions field, users can link the selected asset with the Root Asset. In the pop up form that is shown users can define the specific way the Asset will be linked.

Composition.jpg

Multi-Screen Timeline

The Multi-screen Timeline editor enriches the functionality of the Asset Composition by adding some additional features for the Linking. More specifically the users can Link Assets for specific periods of time (Figure 51), given that the Root Asset is a movie or an audio file. As it is shown in the image below, the connected Assets are displayed in an array and users can change the starting and the finishing time of each Asset by adjusting the slider.

Timeline.jpg

Latest Developments

The asset Aggregation component which is essentially the linker of Assets has reached its final state. Linker now contains advanced modules for Asset creation, editing and linking. This component also contain a recommendation mechanism to help users identify related Assets for linking. The connection of external entities such as Wikipedia articles and social media is an important part of media linking and it’s established in this component. One of the major developments of the Linker is the preview functionality that allows the user to preview the experience created in the Linker. The image below depicts the latest UI of the Linker component.

LinkerUI.PNG

References

  1. Kopliku, Arlind, Karen Pinel-Sauvagnat, and Mohand Boughanem. "Aggregated search: A new information retrieval paradigm." ACM Computing Surveys (CSUR) 46.3 (2014): 41.
  2. De Pessemier, Toon, Simon Dooms, and Luc Martens. "An Improved Data Aggregation Strategy for Group Recommendations." Human Decision Making in Recommender Systems (Decisions@ RecSys’ 13) (2013): 36.
  3. http://www.bigdata-startups.com/open-source-tools/data-aggregation/
  4. http://popcornjs.org/
  5. http://itlaw.wikia.com/wiki/
  6. https://angularjs.org/
  7. https://jax-rs-spec.java.net/