Context Awareness

From SAM
Jump to: navigation, search

Context Awareness is an area of Computer Science which deals with the adaptation of computing systems to the user’s current context. The most recognised definition for the term “context” was from Dey[1]: “Context is any information that can be used to characterise the situation of an entity. An entity is a person, place, or object that is considered relevant to the interaction between a user and an application, including the user and applications themselves.” By being “aware” of the user’s context, a computing system can make deductions about the current situation of the user, from various hardware sensors such as GPS receiver (i.e. for positioning), surrounding Bluetooth equipment (and thus, surrounding people), and even software sources like one’s agenda and list of currently opened documents. Context awareness will be a keystone in user-centred social networks in the next years.

Introduction

Example context factors that may be useful in SAM
The computing systems that provide context-aware services have to perform a variety of distinct functions such as collection of raw data about the users and their environment, applying different reasoning techniques on such data to infer higher-level context information, storage of the context information in a retrievable and indexed format to make available when required, management and coordination of context and related information among different components of the systems, and to provide an environment for building, hosting or enabling context-aware applications and services.[2]

Relevance to SAM

Context-Aware Dynamic Communities

SAM will incorporate a system module that will analyse user data and context data and will create, manage and analyse dynamic social communities. The module will interface with the relevant external data sources and the applicable data consumers within the SAM Platform in order to make the relevant functionalities available to the respective target user groups (end users and commercial users of the platform). Interfacing will include access to all relevant data available through SAM while respecting any privacy and security restrictions that apply, communication with external social network providers through SAM, provision of data to SAM components for social network analysis (again respecting privacy and security restrictions that apply to this particular activity) and the interactive provisioning of community content to end users.

Context data will be used as an important data source for dynamic community creation. By means of its context-centric social communities, SAM will foster in the research and development of different and innovative aspects related to context. These aspects will include geo-localisation, 2nd Screen device information, characterised data and NLP techniques will compose the context awareness mechanism of SAM. Context awareness will be used in order to be able to "know" what is the user talking about (based on "understanding" its comments), extract from there its immediate interests, and find related information or dynamically create a contextual social community that can fit better with it.

Context-Aware Selection of Content

Context information can be meaningfully employed in search or information filtering applications that have access to relevant contextual information. In SAM, identified context parameters can be used to filter available content with respect to its utility given the context of a user at a given point in time.

State of the Art Analysis

To the best of the author's knowledge, no previous work on context awareness for social communities around media consumption has been published at the time of writing. The analysis below briefly reflects on the state of the art for topics related to context awareness that involve media and social communities and/or networks.

Context Modelling

The context knowledge needs to be modelled so as to be machine interpretable and processable as well as exchangeable using well-defined interfaces. Context modelling techniques and requirements have been surveyed and a number of different approaches for representing contextual information and semantic knowledge can be found in literature. Strang and Linnhoff-Popien [3] identify high requirements applied on any context modelling approach: • Be able to cope with high dynamics and distributed processing and composition • Be able to partially validate contextual knowledge independently of complex interrelationships • Support quality and richness indication • Deal with incompleteness and ambiguity of contextual information • Have a shared understanding with each affiliate component • Be applicable to existing environments Context models can be classified into six different model categories with respect to the scheme of data structures, which are used to exchange contextual information, namely key-value models, markup scheme models, graphical models, object oriented models, logic based models and ontology based models.

Key-value models

The model of key-value pairs is the simplest data structure for modelling context. An environmental variable acts as a key and the value of this key holds the actual contextual information. In this way, easy lookup is enabled by applying a matching algorithm. Pair recursion is also supported. The key-value pairs are easy to manage and powerful enough to allow pattern-matching queries, but they do not allow for efficient context retrieval as they lack structuring.

Markup scheme models

Markup scheme models include a hierarchical data structure of markup tags, attributes and content. The content of the markup tags is usually recursively defined by other markup tags. The typical representatives of this approach, namely profiles, are based on XML (Extendible Markup Language). Some of them are defined as an extension to the Composite Capabilities/Preferences Profile (CC/PP) and User Agent Profile (UAProf) standards, which use RDF (Resource description Framework) and a serialisation of XML. With the introduction of new attribute vocabularies, these approaches extend and complete the basic CC/PP so as to cover the higher dynamics and complexity of contextual information compared to static profiles. Such an approach is the Comprehensive Structured Context Profiles (CSCP) by Held et al [4], which overcomes the shortcomings of the CC/PP regarding structuring, as it supports the full flexibility of RDF to express natural structures of profile information. Furthermore, it extends the mechanisms to express user preferences. A similar approach to CSCP is the CC/PP Content Extension by Indulska et al [IRR+03], which includes additional vocabulary for user context information (user profile describing, user applications, user devices, user current location), device capabilities (device profile which includes hardware and software capabilities, device location, networks to which the device is and can be attached, potential and current network QoS on the device’s network interfaces) as well as application requirements and their current status (application profile). There are also several other context modeling approaches in the group of markup scheme models, often either proprietary or limited to a small set of contextual assets or both such as Centaurus Capability Markup Language (CCML), which is built on top of XML and used within a framework for intelligent services in a mobile environment and Context Meta Language (ContextML), which represents not only the context information but also the context metadata.

Graphical Models

Graphical models take advantage of the general structure of the Unified Modelling Language, which allows for a graphical description of a context model and for deriving an Entity Relationship model as required in rational databases. In [5], UML has been extended in order to model contextual information. Another graphical model is the one introduced by Henricksen et al [6] which is based on Object-Role Model (ORM) and designed to overcome the fact that several context models are implicit and informal and largely lack in the expressive power required to accommodate features such as varying information types, quality and temporal characteristics.

Object Oriented Models

Object oriented models offer powerful capabilities of inheritance, reusability and encapsulation while the access to contextual information is provided by well-defined interfaces. In this category and within the ESPRIT Project TEA “Technology for Enabling Awareness [7], the cues were developed providing an abstraction from physical and logical sensors. A cue is regarded as a function taking the values of a single sensor up to a certain time as input and providing a symbolic or sub-symbolic output. A set (finite or infinite) of possible values for each cue is defined. Each cue is dependent on a single sensor but different cues may be based on the same sensors. The context is modeled as an abstraction derived from the available cues. Thus, the cues are objects providing contextual information through their interfaces, hiding the details of determining the output values. In the same category, the object model, developed within the GUIDE project [8], supports the provision of dynamic and personal and environmental contextual information by adopting an approach relatively straightforward that does not require the rewriting of large sections of application code while it maintains scalability. Last, a similar approach is the one of Bouzy and Cazenave [9] who used Object Oriented Paradigm (OOP) mechanisms to model temporal, goal, spatial and global contextual information in computer Go game. Their choice was driven by the need of reutilisation and inheritance capabilities, resulting thus, in organising the knowledge and simplifying the rules.

Logic Based Models

When a high degree of formalism is required, the logic based models are the most suitable as they typically comprise facts, expressions and rules. A logic defines the conditions on which a concluding expression or fact may be derived (a process known as reasoning or inferencing) from a set of other expressions or facts. Usually, contextual information is added to, updated in and deleted from a logic based system in terms of facts or inferred from the rules in the system respectively. One of the first logic based approaches was developed by McCarthy, formalising context as first class objects. More specifically, context is modelled as abstract mathematician entities with properties useful in artificial intelligence. In a similar perspective, Giunchiglia’s [10] approach focuses more on context reasoning, taking a context to be the set of facts used locally to prove a given goal plus the inference routines used to reason about them. Another approach in the same category is the Extended Situation Theory by Akman and Surav [11]. As the name implies, it extends the Situation Theory proposed by Barwise and Perry. Barwise and Perry tried to cover model-theoretic semantics of natural language in a formal logic system. The former ones used and extended this system to model the context with situation types which are ordinary situations and thus they are treated as first-class objects in situation theory. The variety of different contexts is addressed in form of rules and presuppositions related to a particular point of view. They represent the facts related to a particular context with parameter-free expressions supported by the situation type which corresponds to the context. Last, Gray and Salber [12] propose an approach, where first-order predicate logic is used as a formal representation of contextual propositions and relations.

Ontology based models

Ontologies are a promising tool for context-aware applications as they are suitable at describing concepts and interrelations while in parallel allow the determination of service interoperability on the context level. Their well-defined declarative semantics provide a means to project parts of the contextual information onto a data structure processable by computers. An ontology based approach [13] is the Context Ontology Language (CoOL), which is a collection of several fragments of languages describing ontologies, grouped into two subsets, rather than a single, monolithic language. The first subset includes the model of the language, namely the ASC model, named after its core concepts, which are aspect, scale and context information. Each aspect aggregates one or more scales, and each scale aggregates one or more context information. The second subset is a collection of schema and protocol extensions as well as common subconcepts of the model, enabling the use of CoOL in several service frameworks, particularly Web Services.

The CONON context modelling approach by Wang et al [14] is based on the same idea of the CoOl approach, namely to develop a context model based on ontologies because of its knowledge sharing, logic inferencing and knowledge reuse capabilities. CONON provides an upper context ontology and a specific ontology. The upper ontology captures general concepts about basic context, and also provides extensibility for adding domain-specific ontology in a hierarchical manner. The specific ontology is a collection of ontology sets which define the details of general concepts and their features in each sub-domain. Another context modeling approach based on ontologies is the COBRA ontology, used in the CoBrA (Context Broker Architecture) system [15]. This system provides a set of OWL ontologies developed for modelling physical locations, devices, temporal concepts, privacy requirements and other kinds of objects in intelligent spaces. CoBrA employs reasoning for detecting inconsistent context information, evaluating privacy policies and inferring additional context information based on properties such as temporal and spatial relations. In this category, FOAF (Friend of a Friend) is a popular ontology for the representation of personal profile information and social relationships among groups of peers, as it contains terms for describing personal information, membership in groups, and social connections [16]. Built on the Resource Description Framework (RDF2) Semantic Web language, FOAF gives a representation of a social relationship network as a graph structure, where people connected to each other by a given relationship are represented as graph nodes connected by edges. Personal profile information is also represented by RDF subgraphs connected to people nodes. FOAF has become a widely accepted standard vocabulary for representing social networks, and many large social networking websites use it to produce Semantic Web profiles for their users. The main strengths of the FOAF ontology, stemming from its RDF/Semantic Web nature, are simplicity and extensibility. In fact, starting with the FOAF core vocabulary, virtually any RDF vocabulary can be embedded in FOAF, extending its profile representation capabilities, while the social network representation capabilities can be extended with increasingly expressive social relationship sub-properties [17].

Context Awareness and Television

Research and development efforts relating to context awareness and media consumption without focusing on social communities around media, however, have been undertaken, both related to content offered to viewers[18][19][20] and related to context-aware devices such as television sets that switch off screens when no persons are detected within viewing distance of the screen. While such services are of some interest with respect to the goals for using context in SAM, their contributions cannot readily be transferred to the domain of interest for SAM.

Context Awareness and Social Networks

Some research has been undertaken on topics concerning context awareness and social networks, in particular regarding mobile users and mobile usage of social networks[21][22]. While some of the referenced works provide interesting approaches towards integrating contextual data and also integrate time- and location-sensitivity in their approaches, no predominant state of the art can be identified from the scientific work that has been published.

Commercially available systems for context awareness and social networks tend to focus on single specific contextual aspects such as physical proximity of persons that are within a social network of the user of interest.

Related Projects

  • CATVS Context-Aware TV Systems[23]

SAM Approach

The Context Control component is responsible for storing and managing context information used for the effective syndication of assets as well as for the creation of dynamic communities. The main idea behind this component is that it collects contextual information from Social Media, including SAM dynamic communities, the Syndicator and the Dashboard, stores it according to an extended OpenSocial [24] context model and makes it available when another SAM component requests it.

Architecture and Dependencies

In this subsection, the functionalities of the component associated with context management and analysis are identified with respect to the identified goals and user requirements. The top component is the Context Control, as defined in SAM deliverable D3.2.1 “Global Architecture Definition”. The subcomponents that address the desired features and functionalities are the Context Manager and the Data Listener Controller along with the Social Data, Content Data and User Data Listeners and the Data Analyser.

ContextControlArchitecture.png

The key functionalities needed in order to perform context management and analysis are summarised below:

  • Subscribe and listen to available incoming data:
    • Receive messages sent and received together with social network interactions made by the user from the Social Components
    • Receive metadata about content sent to the user via the Dashboard
    • Receive data entered by the user and data on user interaction behaviour from the Dashboard
  • Convert the incoming data into a suitable format for further processing as well as aggregate and buffer it if needed
  • Carry out analysis of the data received via data listeners; for instance statistical analyses of the frequency of use of components and widgets
  • Create, maintain, and update a representation of context parameters of interest for each active end user
  • Use rich context metadata in order to keep the contextual information accurate and up to date
  • Store and retrieve context models from the Cloud Storage Component whenever necessary; provide an API to give external services access to context data
  • Allow other components to selectively retrieve specific parts of one or more users’ context models
  • Communicate with the Semantic Services component in order to carry out sentiment analysis for selected Social Media content
  • Apply context reasoning in order to derive new conclusions based on existing context and an available knowledge base

Implementation and Technologies

Following the goal, requirement, and functionality identification regarding the context modelling and provisioning; the solution approach that will be used in SAM is the OpenSocial specification. This section provides an overview of the specification along with a description of the related technical infrastructure to be used with OpenSocial. Furthermore, a short description of how both the context model and the related software address the desired goals and requirements are provided.

SAM Context Model

After a thorough survey of the available context models and context provisioning frameworks, the OpenSocial specification transpired to be the most suitable approach, serving as a context model. As mentioned before, OpenSocial is user-centric, but it does not take into account the user’s context. However, it provides a large variety of data objects that allow the realisation of the SAM Context Control Component functionalities.

Model Overview

The OpenSocial API is divided into three main areas: People and Relationships, Activities and Persistence. People and Relationships provide functionality to access user information and the relationships that users have with each other. The Activities functionality allows the developer to access the users’ social activities. The persistence functionality enables the storage and the retrieval of application specific information. Furthermore, the API views users from three different perspectives: viewer, owner and friends. The viewer is the person in whose browser the application is being displayed while the owner is the person who owns the profile or information being displayed. Friends refer to the social connections of either the viewer or the owner. OpenSocial provides developers with several approaches for developing apps depending on their project requirements. Most social application designs have the same structure: application data, social data and a template are combined to provide a rendered view to the end user. Social applications normally use one of three types of development patterns:

  • Social Mashup [SMT10] is a lightweight OpenSocial application which runs inside a social network. Because such applications do not rely on a server, they typically scale extremely well but may be limited in terms of data storage and/or processing. A social mashup is typically created using HTML, JavaScript, CSS, OpenSocial templates, and/or Flash.
  • Social Application [SAT10] run inside of a social network but rely on an external server for processing and rendering data. These applications can provide advanced functionality but may run into scaling problems when they become very popular. Social applications can be created using a variety of technologies, including HTML, JavaScript, CSS, OpenSocial Templates, Flash, PHP, Python, Java, Perl, .NET, or Ruby.
  • Social Website / Mobile application [SWT10] approaches run outside of social networks but consume social data through REST or RPC APIs. Users of these apps can grant access to their data using 3-legged OAuth without needing to add an application on a social network. This design pattern grants the most flexibility - almost all languages and platforms can take advantage of data using this approach.

The easiest and most common way to utilise OpenSocial is by creating “gadgets” - small applications that utilise social data from the underlying social network containers they run on. Gadgets are written using a mix of simple client-side technologies including HTML, JavaScript and XML. Due to the requirements of SAM and the need to extract data from social networks rather than create applications that run within social networks, the Social Website pattern is most applicable. Thus SAM looks to utilise data-centric REST and RPC APIs, and client libraries that may help simplify the development in general and across multiple social network containers.

Model Extensions

OpenSocial provides an extension model that has the potential for the specification to mature by adding depth and richness to a set of existing features while at the same time fostering organic growth to enable new capability. In this way, it is possible to enhance existing features as well as add new ones. A robust extension model enables the OpenSocial community to manage the overall complexity of the specification in a clearly documented and consistent manner. Indeed, SAM requirements for the effective content syndication and dynamic community creation cannot be met without extending the OpenSocial model. Below, the initial approach towards the required extensions is described.

  • Media Asset-Related Information

The first extension includes the handling of on-demand videos (SAM assets) such that when a user watches an on-demand video, contextually relevant and personalised content should accompany it.

  • Community-Related Information

The third extension is related to the dynamic community creation. More particularly, a user should receive recommendations to join specific communities related to their context and the assets that they consume. Furthermore, when a user joins a community, they should also receive default recommendations for related or similar communities.

SAM Context Management

As far as the context management and provisioning are concerned, Apache Shindig [25] provides an OpenSocial reference implementation that has been considered as a candidate for use in SAM. However, although Apache Shindig is the reference implementation of OpenSocial and is fully compliant with the core OpenSocial specification, its adoption in SAM would lead to technical problems in particular concerning the implementation of extensions to the core OpenSocial specification. After careful analysis of the Apache Shindig design and implementation as well as extensive experimentation and testing, it has been concluded that the Apache Shindig components and services cannot be effectively customised for SAM and most importantly that the necessary extensions to OpenSocial cannot be implemented effectively. Hence, a custom context control framework for SAM will be implemented which on the one hand will follow the architectural design proposed in D3.3.1 Technical Specification and on the other hand will be built upon the solid foundations of the well-established OpenSocial data model. If a social network wants to host OpenSocial applications, then it must implement the OpenSocial specification. The SAM Context Control component will use the Spring Framework [26] for implementing the business logic and the service interfaces for the context management operations and will be supported by a graph database for the serialisation of the data and their effective querying. A popular and feature-rich graph database is Neo4j [27] . The combination of Spring and Neo4j offers great advantages to our work since they are fully compliant and incorporate very effective features. It is expected that with this approach the context management will be able to provide the required performance levels, enabling simultaneously simplified access to services and data through Cypher, the powerful query language of Neo4j.

SAM Context Analysis

Following the study regarding user behaviour analysis approaches; the user modelling-based solution seems the most suitable for SAM as it is already defined that user profile models will be used prior to the context analysis. Furthermore, such an approach will benefit from the Semantic Services that enrich the content, in the sense that the same component can be extended in order to perform semantic processing on user-generated content (posts, tweets, etc.). Furthermore, it is an approach aiming at achieving better recommendations, which is similar to the main goals of SAM Dynamic Social Communities. Within the user modelling approach, the user profile is a set of weighted concepts. These concepts include entities, topics and hashtags and thus there are entity-based, topic-based and hashtag-based profiles. In order to build the aforementioned profiles, the messages a user posts to the social network services are enriched via semantic processing in order to detect and identify entities such as persons, events, products or music groups etc. The next step is to enrich these messages with external Web resources with linking strategies that detect whether and, if so, to which news article a post refers – even for posts that do not explicitly provide a hyperlink to the corresponding news article. In the context of SAM, the messages can be enriched with tags associated with dynamic communities or SAM assets. Thus consideringrecommendation, the candidate items will be ranked according to a specific algorithm so that the user receives the most prominent results.

Subcomponents

A summary of the tasks carried out for each subcomponent of the first version of the prototype is shown in the following table.

Subcomponent Task
Context Manager
  • Extend the models so that all the requirements of the cooperative components are met:
    • Support all the Asset types of SAM
    • Store Community related information
    • Active Assets
  • Integration with the Cloud Storage
Social Data Listener Automated ingestion of contextual data from the listeners
Content Data Listener
Data Listener
Data Listener Controller

Letest Developments

In the latest version of Context controller component, services for delivering personalized recommendations are implemented. The collected data from the data listeners implemented in the previews versions are now used as input to recommendations algorithms to produce recommendation for users. There are two types of recommendations, the first one recommend movie Assets for users to consume and the second one is used to deliver a more personalized 2nd screen experience by orchestrating the way widgets and generally second screen content is delivered to the user.

RecommendM.PNGRecommendW.PNG

References

  1. A. K. Dey, “Understanding and Using Context,” in Personal and Ubiquitous Computing Journal, vol. 5, no. 1, 2001, pp. 4-7.
  2. Knappmeyer, Michael, et al. "Survey of context provisioning middleware." IEEE Communication Surveys and Tutorials 15.3 (2013): 1492-1519.
  3. T. Strang and C. Linnhoff-Popien, "A Context Modeling Survey," in In Workshop on Advanced Context Modelling, Reasoning and Management, UbiComp 2004 - The Sixth International Conference on Ubiquitous Computing (September 2004) Key: citeulike:249347, 2004
  4. A. Held, S. Buchholz and A. Schill, "Modeling of context information for pervasive computing applications," in Proceeding of the World Multiconference on Systemics, Cybernetics and Informatics, Florida, 2002
  5. J. Bauer, R. Kutsche and R. Ehrmanntraut, "Identification and modeling of contexts for different information scenarios in air traffic," Technische Universität Berlin, Berlin, 2003.
  6. K. Henricksen, J. Indulska and A. Rakotonirainy, "Generating Context Management Infrastructure from High-Level Context Models," in 4th International Conference on Mobile Data Management (MDM) - Industrial Track, 2003
  7. A. Schmidt, M. Beigl and H.-W. Gellersen, "There is more to context than location," in Computers & Graphics , 1999, pp. 893-901
  8. K. Cheverst, K. Mitchell and N. Davies, "Design of an object model for a context sensitive tourist GUIDE," in Computers & Graphics, 1999, p. 883–891
  9. B. Bouzy and T. Cazenave, "Using the object oriented paradigm to model context in computer go," 1997, Proceedings of the First International and Interdisciplinary Conference on Modeling and Using Context
  10. F. Giunchiglia, "Contextual Reasoning," EPISTEMOLOGIA, SPECIAL ISSUE ON I LINGUAGGI E LE MACCHINE, vol. 16, pp. 345-364, 1992
  11. V. Akman and M. Surav, "The use of situation theory in context modeling," in Computational intelligence, Blackwell Publishers Inc, 1997, pp. 427-438
  12. P. Gray and D. Salber, "Modelling and Using Sensed Context Information in the Design of Interactive Applications," in Engineering for Human-Computer Interaction, Toronto, 2001
  13. T. Strang, C. Linnhoff-Popien and K. Frank, "CoOL: A context ontology language to enable contextual interoperability," in Distributed applications and interoperable systems, Springer Berlin Heidelberg, 2003, pp. 236-247
  14. X. H. Wang, D. Q. Zhang, T. Gu and H. K. Pung, "Ontology based context modeling and reasoning using OWL," in Second IEEE Annual Conference on Pervasive Computing and Communications Workshops, 2004
  15. H. Chen, T. Finin and A. Joshi, "Using OWL in a Pervasive Computing Broker," MARYLAND UNIV BALTIMORE DEPT OF COMPUTER SCIENCE AND ELECTRICAL ENGINEERING, 2005
  16. J. Golbeck and M. Rothstein, "Linking Social Networks on the Web with FOAF: A Semantic Web Case Study," AAAI, vol. 8, pp. 1138-1143, 2008
  17. L. Burano, "Exploiting social context information in context-aware mobile tourism guides," Mobile Guide, 2006
  18. G. Adomavicius and A. Tuzhilin, "Context-Aware Recommender Systems", in F. Ricci, L. Rokach, B. Shapira and P. Kantor (eds.), "Recommender Systems Handbook", 2011, pp. 217-253.
  19. W. Lee and C. KaoLi, "A Context-Aware Smart TV System with Body-Gesture Control and Personalized Recommendation", in D. Tjondronegoro (ed.), "Tools for Mobile Multimedia Programming and Development", 2013, pp. 115-130.
  20. T. De Pessemier, T. Deryckere and L. Martens, "Context aware recommendations for user-generated content on a social network site", in Proceedings of the EuroITV '09 Seventh European Conference on European Interactive Television, 2009, pp. 133-136.
  21. M. Raento, A. Oulasvirta, R. Petit and H. Toivonen, "ContextPhone: a prototyping platform for context-aware mobile applications", in IEEE Pervasive Computing, vol. 4, no. 2, 2005, pp. 51-59.
  22. A. Beach, M. Gartrell, X. Xing, R. Han, Q. Lv, S. Mishra, K. Seada, "Fusing mobile, sensor, and social data to fully enable context-aware computing", in Proceedings of the HotMobile '10 Eleventh Workshop on Mobile Computing Systems & Applications, 2010, pp. 60-65.
  23. CATVS. Context-Aware TV Systems. http://www.ideal-ist.eu/ps-it-81735
  24. OpenSocial http://www.w3.org/blog/2014/12/opensocial-foundation-moves-standards-work-to-w3c-social-web-activity/
  25. Apache Shindig https://shindig.apache.org/
  26. Spring http://spring.io/
  27. Neo4j, the World's Leading Graph Database http://neo4j.com/