• If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • Buried in cloud files? We can help with Spring cleaning!

    Whether you use Dropbox, Drive, G-Suite, OneDrive, Gmail, Slack, Notion, or all of the above, Dokkio will organize your files for you. Try Dokkio (from the makers of PBworks) for free today.

  • Dokkio (from the makers of PBworks) was #2 on Product Hunt! Check out what people are saying by clicking here.


Additional Readings Summaries

This version was saved 10 years, 7 months ago View current version     Page history
Saved by Chris Bayruns
on December 4, 2011 at 2:53:12 pm

Type your reading summaries here in the following format:


Title: A brief history of wearable computing

Name: Edison Thomaz

Summary: Your summary...

Additional Readings Summaries


Title: Accurate Activity Recognition in a Home Setting

Name: Abhishek Shroff



The researchers aimed to create a probabilistic model for activity recognition based on input from various sensors placed in a household. In doing so, they needed to use a sensor network setup that can be easily installed in houses and an inexpensive and accurate method for annotation. Since these were not used in a lot of existing experiments, they decided to create their own using the RFM DM 1810 wireless networking kit, and the Jabra BT250v bluetooth headset. The RFM DM 1810 has the capability to be attached to arbitrary sensors, both digital and analog.


Using this sensor network, the researchers collected a series of binary signals from each sensor when they were triggered, and simultaneously had the subject annotate the activity via the bluetooth headset. For the purposes of this study, the researchers mainly focused on detecting 8 specifically chosen Activities of Daily Life, or ADLs, that included:

Idle, Leaving, Toileting, Showering, Sleeping, Breakfast, Dinner, and Drink.


As mentioned before, the classification model was probabilistic. The researchers tried two different approaches to classification: Hidden Markov Models(HMM), and linear-chain Conditional Random Fields(CRFs). For creating the models, they broke down the training period into a granularity of 60s, and were not concerned with the timing of sensor data within that granularity. At each timestep t, the current activity yt was generated by the activity at t-1, and produced the sensor output xt. The researchers trained their probabilistic models on this data. For test data, they took one full day of sample data omitted it from their training data, and used that day's data to test. The researchers were able to achieve upto 80% classification accuracy with HMMs, and 70% accuracy with CRFs.


Additionally, with both these models, the researchers had the option to run the model algorithms online or offline. In offline mode, the algorithm has access to all data, whereas in online mode, the algorithm only uses data from events that have happened on or before time t while computing step t. They found around a 6 - 9% decrease in accuracy when training online.




Title: Living in a Glass House: A Survey of Private Moments in the Home

Name: E.J. Layne



The focus of the researchers was to identify the activities that the participants do not want to be recorded in the home. The justification for the research is the development and acceptance of ubiquituous systems and sensors in the home, such as the Microsoft Kinect and Samsung Televisions with sensing technology. To determine the activities that users would not want recorded, the researchers conducted an anonymous survey asking what the activity was, and where in the house it was performed. They collected 489 surveys, 475 of which they considered valid. 


The collected taboo activities were categorized into 19 high level categories such as "Self-Appearance", "Intimacy", Media Use, etc, and then further collated into 75 sub-categories. The categories of Self-APpearance, Intimacy, and Cooking & Eating had the highest percentages of participants listing them, with 22.5%, 18.3%, and 9.3% respectively. The percentage results were also correlated to the other collected demographics, such as gender, and location in the home where the given activity would be taboo to record. The bedroom was the most recorded location, with 33.7%.


Overall, I felt that this paper took a step in the right direction, but it also could have collected more data if it had collected some of its own sensor data, and further correlated its results. Furthermore, the conclusions that the paper came to were somewhat obvious, and so it should have been apparent to the researchers that they needed to take their experiment a step further to get past the obvious. An example of sensor data that they could have collected without breaching the participants privacy would be to take 100 recordings of themselves or some friends in different areas of the house, and survey the reactions of the participants watching the videos, based on how they felt about someone being recorded in that part of the house. 



Title: Accurate Activity Recognition in a Home Setting, Karsteren et al

Name: Hafez Rouzati

Interpretation of sensor network data is a complex task that requires both accurate sensing infrastructure as well as well-formed activity models. Kasteren et al’s approach involves baselining sensor data with user-provided input in the form of audio annotations. Using a small number of sensors (14) and development of probabilistic Markov and Conditional Random Field models, the researchers compare the effectiveness of each model, the effect of training data on classification accuracy, and the effectiveness of offline vs online inference methods. Their results illustrate that with just 12 days of training data, high-accuracy results can be produced and how different modeling strategies are biased with respect to their particular parameterization.


Title: Mining Models of Human Activities from the Web

Name: Hafez Rouzati

Contextual inference is often proposed for use in ubiquitous computing applications. The premise of such reasoning is that by providing models of events, states, and entities, and associated sensing infrastructure to detect when events occurs, then systems can reason about the current context that the user / system is in. The system can then present information or help the user with tasks since it is now able to anticipate the users needs based on the current context. Pekowitz et al attempt to address a major problem with contextual reasoning - in order to reason about context, the system must have detailed models that parameterize context and building these type of model for human “activity” algorithmically can be extremely challenging, often resulting in tedious description of activity models by hand, potentially resulting in an impoverished level of detail or a lower number of models. The researchers propose using NLP techniques to mine “how-to” websites such as ehow.com to build up a large number of models and by using RFID to sense which objects the user is interacting with, to infer the user’s activity based on comparison to the web-generated models. The primary contribution of using a data-mining approach to constructing model is simply the number of models that can be generated. By combining NLP with relevance metrics in the form of Google searches, they provide a potentially interesting methodology for automated activity model generation.


Title: On the Limitations of Query Obfuscation Techniques for Location Privacy

Name: Abhishek Shroff




Privay is an important concern among users of Location Based Services. One of the techniques to try to ensure privacy is to provide false data along with real data in

order to obfuscate the user's actual location. The researchers used a tool called SybilQuery that, given the real start and end points, and a number k, generates k-1

similar pairs of start and end points along with roughts, for some notion of similarity described in the paper in detail. This paper talks about adverserial Location Based

Services trying to identify the user's real location from the false location queries, as a means to an unknown end.


This paper studies users' location as trips, where trips are defined as temporally close queries. Queries more than 10 minutes apart are considered to be separate trips.

According to the authors, studying trips gives more information than individual queries, and is generally easier to deal with.


The paper considers two types of adverserial LBSs:

1. An LBS that has access to trip data from before when the user started using the obfuscator, known as the strong adversary

2. An LBS that does not have access to this, i.e. the user started using this technique before using this service, known as the weak adversary.


The researchers used Weka, an open source machine learning tool on the data set for the strong adversary in order to correlate old trips with new trips.

They achieved close to a 94% success rate with only 2% false positives when using restricted classification on end points, i.e. not taking waypoints into account.


For the weak adversary, the researchers tried to correlate the destinations of trip i-1 in order to determine which one of trip i was valid, because the source of trip i

would presumably be closer to the destination of trip i-1. For a k value of 5, i.e. 5 total trips including the real one, they improved the classification rate from 20% as

achieved by random guessing, to 40%.





Title: Living in a Glass House: A Survey of Private Moments in the Home

Name: Megha Sandesh


This paper, which was carried out by a group of researchers from University of Washington, Microsoft Research and Intel Labs explores the reactions of people when their activities were recorded using sensors. While the project involved the participants taking a survey (either offline or online using Mechanical Turk), it did not actually involve planting sensors in their homes.


After an initial screening done on Mechanical Turk, a total of 475 out of 489 surveys were aggregated from Mechanical Turk, Craiglist and local US postcards. Other responses were left out as the participants were outside the US, under 18 years of age or were casual survey takers. Around of the respondents reported that they did not want to be recorded while in their bedroom or bathroom, especially during intimate and private moments. The other notable result was that people had an aversion toward being recorded in general. They did not want to be caught up in situations which could demean their social standing within their peer group, family etc. They also wanted to preserve their privacy during worship and almost any form of activity in general. This shows that people have an uneasy feeling about prying eyes.


Comments: I thought that this paper did not go beyond finding out the obvious. The researchers reported that people did not want to be seen or recorded by others during private or intimate moments. This was to be expected as almost no one wants to be caught by others in awkward moments. After all, these sort of incidents carry an amount of social stigma with them. This paper servers the purpose of finding out the distribution of activities and the demographics associated with it.   Also, the lack of the actual use of sensors has prevented the researchers from deriving meaningful real world results which could have backed up the numbers.




Title: Route Classification Using Cellular Handoff Patterns

Name: Megha Sandesh


This paper explores the possibility of predicting routes by tracking the handoff patterns between cellphone towers. The paper is based on the following basic premises/assumptions:

  • People usually carry cellphones with them wherever they go.
  • The location of the cellphone is a good indicator of the location of the user.


The authors use  nearest neighbour algorithm to predict routes.

The researchers have explored 3 different ways of predicting routes:

1. Handoff between common antennas

2. Handoff between common sectors

3. Handoff between common towers


Using a training set and GPS data they improved the efficiency of each method. It was found that each method was successful in predicting routes, with the common antennas method being most accurate. The authors thus prove that cellular handoff patterns do have a strong correlation to route travelled. I think that this experiment can be better validated if conducted on a larger scale, like an entire city instead of just the one 3 mile stretch used in the original experiment. Overall, I think this paper demonstrates that mobile apps which help in commuting can be designed with the basic premise that people will be near to their phones.


Title: Software Engineering Issues for Ubiquitous Computing

Name: Oriol Collell Martin

Summary: Ubiquitous computing has been emerging as a new research field during the last years. Because of their nature ubicomp applications evolve organically, that means that often the best research approach to ubicomp is based on rapid prototyping. This presents a challenge to Software Engineering which lacks appropiate toolkits to rapidly build and test ubciomp application prototypes. The problems of the toolkits, more specifically, are that they do not effectively support the implementation of the three common features of every ubicomp application which are the following: transparent interfaces, context-awareness and automated capture.

The first one refers to the fact that interfaces and interaction methods available in ubicomp applications should be as "transparent" as it is the use as everyday objects. For example, the best way to interact with an "electronic whiteboard" would be to just grap an "electronic chalk" an start writing, just as it would be with a regular whiteboard.

The second one claims that the behaviour of an application should vary depending on the context, that is, the current location of the user, the time, etc. For example, an application may indicate you the closest bars depending on your location.

The third one states that the everyday experiences should be recorded for later recall. For example, lectures you take could be recorded so that you can later review them.

Toolkits, then, should address this issues which by the time of writing the paper were not well addressed. Toolkits should let us treat other forms of input rather than standard keybord or mouse as easy as we treat those. There should also be a standard way to access context information and a way of building context-aware applications just as we do with GUI (with predefined interactors and building blocks).

Another important issue is in the way we structure software. Designers tend to tightly couple different interacting parts which later difficults the modification of that software. Succeding in the achievement this separtion is crucial given the rapid prototyping research model proposed by the author.

The author ends by pointing out that the hope is to overcome all these issues by the building of software upon third-party components, that is, different components addressing different issues, which at that present time was available but in a non-universal way (that is, it was only available for a little set of devices an operating systems).

Comment: I have found the title of the article a little misleading. It says "Software Engineering issues...", but, actually, it focuses only one Software Engineering stage, the implementation. It leaves some sentences or some insights about the other SE stages (Specification, Design, Testing, ...), but I expected to find more about these other stages.




Title: DigitalDesk

Name: Chih-Pin Hsiao



We interact with the documents in two separated worlds, the digital one and the physical one because we can take advantages from both of the worlds. However, each world has its own constraints. Working on one side means we need to abandon the advantages providing from the other. Due to the fact that people seem not to give up the properties provided from physical papers, the author tried to bring the computer power to our physical desktops instead of making the GUIs more like a physical desk. Thus, the author claims that the properties from digital and physical worlds can be integrated. DigitalDesk uses two overhead cameras pointing down to the desk with one projector for projecting interacting contents. One of the cameras is used to see the users’ hands on the desks. The other one is for scanning the contents in the physical documents into digital formats. This prototype employs computer vision to detect the user’s fingertip or pen as the pointing device instead of mice.


DigitalDesk Calculator is one of the applications that addresses the issue of transcribing the numbers on physical papers into computational device. Users can just copy and paste the numbers on physical documents into the virtual calculator to get the desired calculations. PaperPaint helps users to construct a mixed paper and electronic paintings. That is, the system can operate the captured sketch patterns or objects in a computational ways with their free-hands, such as copy, move, and rotate. Since the digitized information can be transferred to a remote location, the author also provides a prototype that helps people to engage in the same task and document in the separated locations.

The author also addresses some implementation issues and the strategies to resolve the issues. How to recognize the finger tip for pointing on the screen is the first issue. DigitalDesk uses the strategy of detecting moving pixels against the static objects on the desk. Detecting finger taps is another issue when the system like digital task uses an overhead camera. In the implementation, DigitalDesk facilitate the sound when knocking on the desk for indicating the tap. The shadows from the projector are other issues. However, after the user testing, the author found that participants seem not to be bothered by the shadows.


Overall, the author claims that the work is the one step toward to the integration of physical paper documents with the digital capabilities.


Title: One.world: Experiences with a Pervasive Computing Architecture

Name: Shaun Wallace




One of the attractive properties of pervasive computing is this idea of being able to access information at any time and almost anywhere. A major challenge in accomplishing this goal includes being able to develop applications that continually change based on the environment they are in as people move through the physical world. one.world project aims provides developers with the tools they need to build and deploy these types of applications. The architecture provides services such as discovery which helps developers locate and connect service and migration, that helps applications follow users as they move around an environment.


To overcome the limitations of distributed systems, there are three requirements for system support of pervasive computing:


  1. As people move through the physical world, an application’s location and execution context constantly changes. So the system support must embrace contextual change, not hid it from applications.
  2. System support must encourage ad hoc composition and not assume a static computing environment with just  a few interactions.
  3. As users collaborate, they must share information, so the system must facilitate sharing between applications and between devices.


One.world’s system employs the foundation and system services on the kernel and the libraries, system utilities, and applications run in the user space.

All code in one.world runs in a virtual machine because developers can not predict what devices the code will actually need to run on. All data is presented as tuples, which define a common data model which in-turn simplifies sharing. All communication is expressed, whether local or remote, through asynchronous events. Lastly, environments are used as the main structuring mechanism. They host running applications and isolate them from one another. They also serve as containers for persistent data, thus making it possible to group running applications with their persistent data.


One.world was evaluated based on the following criteria:


completeness - could useful applications be built using one.world’s primitives,

complexity - how hard is it to write code in one.world?,

performance - does the architecture perform well enough to support actual application workloads?.

utility - can others build real pervasive application atop one.world?


Services and applications atop one.world:


  1. replication service - service providing  access to data items even if several people share the same data and access it from different, possibly disconnected networks.
  2. user and application manager - includes support for setting up users, running applications for a user, and check-pointing all of the user’s applications, lets users move or copy applications and data through a simple drag-and-drop interface.
  3. text-and-audio-messaging system - chat is based on a simple model uder which users send text and audio messages to a channel and subscribe to that channel to see and hear messages sent to it.
  4. Labscape - experimental data follows researchers as they move throughout the laboratory. Additionally, there is no need to move the entire application, only a small component to capture and display experimental data.


One.world’s study showed that nesting of environments shows that this paradigm is powerful for controlling and composing applications.


The event-based programming model enables a more familiar synchronous programming style and reduces the complexity of building event-based applications.


Data-centric data models provide better interoperability than programmatic data models.


Defining data-models and communication protocols to better integrate pervasive systems with one another and with Internet services remains a challenge.


Title: Automatic Partitioning for Prototyping Ubiquitous Computing Applications


Name: Caleb Southern
The authors discuss a major software engineering challenge for ubiquitous computing, distributed systems. Ubiquitous computing, by its very nature, involves a distributed architecture. The authors assert that most of the problems in this area arise during the R&D and prototyping stages of the development of ubiquitous applications. They then describe a solution to this problem, along with their implementation and evaluation of this solution. 
The general approach they propose is called automatic partitioning, which "is the process of adding distribution capabilities to an existing centralized application without needing to rewrite the application’s source code or needing to modify existing runtime systems." In other words, the programmer can write the software as if it were a centralized application designed for a single location (which is much simpler), and then apply automatic partitioning to implement it as a ubiquitous, distributed application. Such a tool would allow programmers to develop at a higher level, where they only need to specify the location of the distributed resources. The tool would take care of the underlying communication protocols and the details of the distributed implementation. The authors compared automatic partitioning to traditional distributed programming toolkits, which employ techniques such as remote procedure calls. In these other approaches, the toolkit only automates the communication layer; the programmer still has to think about the distributed architecture.
The authors built and evaluated an automatic partitioning toolkit called J-Orchestra. This tool offers programmers a GUI which allows them to allocate groups of Java classes to specific locations. With J-Orchestra, programmers can take a centralized, non-distributed Java application and automatically distribute across locations, transforming it into a ubiquitous application. They implemented a test case with an application called Kimura. First they rewrote Kimura as a centralized Java application, removing the distributed toolkit it was built with. Then they took this modified code and partitioned it with J-Orchestra, calling the result Kimura2. 
Their evaluation of Kimura2 showed benefits from using J-Orchestra. "The main benefit is in the new software architecture’s simplicity, which resulted in more understandable and maintainable code without sacrificing any of the original functionality. . . . [T]he developers can focus on the desired functionality without worrying about the distribution specifics." However, there are some software engineering aspects for which J-Orchestra is less suitable, and the traditional distributed systems toolkits should still be used. "[W]e believe that automatic partitioning’s benefits for ubicomp are most pronounced during prototyping." In terms of commercial or mission critical applications, "if handling tough distribution issues—such as asynchrony, fault tolerance, or load balancing—is essential for an application, the best option continues to be the use of a flexible middleware technology and a program design that exerts full control over the application structure."

Title: Activity Sensing in the Wild
Name: Jeremy Duvall
This paper discusses a field study where 12 participants used a system of devices designed to motivate those with sedentary lifestyles to be more fit. The system was composed of a cell phone application, a 'glanceable' display, and a small device worn on-body. The device was trained using machine learning techniques to infer a users physical activity and track it automatically. The glanceable display was a dynamic image that provided the user with a fast way to track his or her activity and even compare those activities versus goals that were set. As the on-body device collected more information, the garden in the display appeared more vibrant. Specific activities were correlated with specific components of the image--a pink flower represented a 10 minute cardio activity, for example. If the user completed a weekly goal, a large yellow butterfly would appear in the garden. Overall, the field study was quite robust, and produced some valuable insights into how users interact with an on-body ubiquitous computing system. For example, the ability to manipulate data inferred by the device was discovered to be very valuable to users.

Title: Sensors and Surveys

Name: Patrick Mize



Field studies are extremely important to ubiquitous computing research. These studies allow researchers to collect data about how people use and what they think about specific technologies. MyExperience is a tool to help researchers in the data collection process. It is an open source tool for mobile phones that can collect qualitative and quantitative data on the behaivor, attitudes, and technology usage of humans. This tool allows the researcher to collect data easily and remotely in the field.

MyExperience has been used in many studies, mostly health related, but with access to over 140 sensor events and a plug-in architecture for adding other sensors, the possibilities are many. MyExperience is designed to run on the subject's personal phone to prevent carrying an extra monitoring device for the study. Not only can sensor data be captured but it can also use time based or sensor-triggered self-report surveys to obtain information that a sensor cannot (e.g. beliefs, feelings, or attitudes). These surveys can be dynamically changed if enough data has been collected in a certain area. And because the software is running on the subject's phone, real time updates can be sent appropriately according (e.g., text messaging, WiFi, or GPRS) to their data plan and hardware.


Title: Farther Than You May Think

Name: Spencer Border



With the fast-paced growth of the mobile computing technology, it is assumed that these devices can be suitable for determining the proximity of the owner’s location. Consequently, the ubiquitous computing world has made the mobile phone the choice for location-aware computing.  In this paper, the authors present a study reflecting this assumption. The study included 16 different participants over a three week period in which the researchers measured the proximity of the participant and their mobile device.


This study was done using a mixed approach with both automatic logging done on the mobile device and personal interviews with the participants.  The researchers conducted their experiment using several technologies. First, they had all participants wear plastic beacons as lanyards nearly all the time in order to measure the phones’ distance from the beacon. This beacon uses the Bluetooth technology and transmits a Bluetooth signal that is then detected by a custom-built application on the users’ mobile device. This was done about every minute and was used as a distance approximation between the user and mobile device based off of the signal strength.  Using signal strength as a distance approximation allowed for three different proximity levels that included:

1. Within arm’s reach (1-2 meters of the beacon)

2.Within the same room (5-6 meters of the beacon)

3.Unavailable (beyond 6 meters of the beacon)

The interview was thus used to allow the users to self-report reasons why they either did or did not have their mobile phones on them indicated by data collected during the week.


The researchers found that participants varied in their proximity levels with all but two users keeping their phones on them more than 85% of the time.  On average participants had their cell phones within arm’s reach 58% of the time and within the same room 78% of the time. Participants had their phone within arm’s reach more when they were either away from home, during weekday’s,  or while they were awake. One interesting finding was that the number of minutes used per month on the phone had no correlation to the proximity relationship.  An open question that the study was trying to address was whether the phone itself could be used as a proximity indicator without the Bluetooth tag. From the experiment, it was shown that some features do have predictive power. The most important features found for proximity indication includes cell tower ID, hour of the day, or day of the week. These findings help to reveal some interesting ideas that mobile phone application designers can consider in their development.


Title: How to Nudge In Situ: Designing Lambent Devices to Deliver Information Salience in Supermarkets

Name: Angela Mingione



Most shoppers make decisions by ignoring most information and rapidly processing the information they find most useful or prominent. For example, they’re likely to select an item based on price, brand, or packaging. The researchers wanted to see if they could provide a tool to help shoppers easily determine whether food had traveled far and if it is organic. To change a user’s behavior, their tool needed to be simple, display only limited information, and be difficult to ignore.


They created a snap-on handle, the lambent trolley handle, for shopping carts that had several lights, an emoticon, and a cancel button. The lights lit up based on the distance the food had traveled. If it was local, only a few lights lit up. If it came from outside of Europe, most of the lights lit up. The color of the lights determined whether the food was organic or not, with green meaning organic and orange for non-organic foods. The emoticon changed based on the average distance traveled of all the food in the cart compared to a social norm. If the user’s cart contained mostly local foods, the emoticon would be a smiley face. If it fell about average, the face was neutral and if it was above the average then it was a sad face. The cancel button allowed users to remove an item from the cart.


They tested their handle with several shoppers by giving them scenarios that asked them to select particular items for a friend or relative that wanted organic/locally grown food. Some users received carts with the lambent handle and some received typical grocery carts. They found that in most cases, the handle was successful in nudging people to buy food that had not traveled far. Users also reported that it took longer to shop with the handle and that the emoticon made them feel bad in cases when they had no other options. For example, all chillies in the store were from Africa but this displayed a far travel distance. However, users seemed pleased when their overall average for the travel distance of their food was lower than the average.



TITLE: How to Nudge In Situ: Designing Lambent Devices to Deliver Information Salience in Supermarkets

MY NAME: Ed Anderson


AUTHORS: Vaiva Kalnikaite, et al at PIL Open University, Microsoft Research UK, University Bath, Indiana University, and the Innovative Retail Lab, Saarbrucken

Presented at Ubicomp 2011




What is the subject: Informed decision making in supermarkets.

Why is this important: Specifically in this case, environmental impact concerns -- food miles.  More generally, delivering relevant information to the shopper as they select products could lead to better choices.

What is the problem: Communicating an increasing amount of product information to shoppers without overwhelming or burdening them.

What is the solution: A "Lambent Shopping Handle display" that visualizes food miles and indicates organic items as they are placed in the cart.  The display is an example of "calm technology" delivering the information in a way that requires minimal attention.




The use of the Lambent Shopping Handle as an "in situ" (during shopping product selection) aid would cause a "nudge effect" that would motivate shoppers to select products with lower food miles and more organic ingredients.




In a one time experiment, a group of 18 shoppers were tasked with purchasing a shopping list of items from a normal UK grocery store.  They completed this task both with and without the use of the Lambent Shopping Handle.  As they selected items that fit the criteria on the shopping list, they could scan them and the handle would momentarily light up, indicating the amount of food miles associated with that product and whether or not it was organic.  More lights mean more miles, yellow lights for conventional and green for organic.  As the shopping cart filled up, the handle would remain lit in proportion to the total food miles for all items in the cart.  Additionally, there is a separate light with a small icon of a face indicating the current total food miles in the cart relative to the social norm.  If the total miles are under the norm, the face has a happy expression, as the norm is reached, the face is neutral, and once the norm is exceeded, the face is sad.




The Lambent Shopping Handle did have some significant nudge effect for products that met certain criteria: 


1) very notable difference in food miles between two choices for one product

2) shopper had no strong preference for a particular brand, size, price, or origin


The indication of organic ingredients was not effective, as most organic products were already clearly labeled.  All shoppers completed the experiment with a total of food miles below the social norm, and the effect of the display was ambiguous.



Title: Getting Closer: An Empirical Investigation of the Proximity of User to Their Smart Phones

Name: Isaac Kulka



Following up on the 2006 study, “Farther Than You May Think” that focuses on participants’ daily proximity to their cellphones, this 2011 study explores a similar theme, investigating what effect the rising popularity of smartphones has had on how close owners keep their devices.


In “Farther than you may think,” the authors discovered that participants kept their cell phones within arm’s reach about 58% of the time, and within the same room 78% of the time.  In the current 2011 study, the authors were surprised to discover that, of the 28 subjects who participated in their study, smartphones were kept within arm’s reach only 53% of the time, a 5% decrease since 2006.  The researchers did discover, however, that current participant’s keep their smartphones within the same room as themselves 88% of the time, an increase of 10%.


The study was conducted with 28 participants over a period of four weeks.  Each participant used their own Android smartphone, on which they installed the Android AWARE data collection framework, an application designed to monitor basic statics about the phone such as time of day, location, battery usage and battery temperature.  Each participant also wore a bluetooth device around their neck used to approximate their distance from the phone using Received Signal Strength Indication (RSSI).  It was these measurements that were used to calculate the percentage from each day where the phone was within arm’s reach, in the same room, or further away.


The study concludes that, while it was surprising to see the “within arm’s reach” percentage fall between 2006 and 2011, it was notable that the overall “within room” percentage did increase from 78% to 88%, indicating that smartphone owners are keeping closer to their devices.   Interestingly, at the beginning of the study, participants had assumed they were keeping their smartphones within arm's reach an average of 22 hours per day, a gross overestimation.  The authors did cite their sample size of 28 participants as being too small from which to make general predictions about smartphone use.  However, based on the measurements taken by the Android AWARE data collection framework, the authors were able to conclude that time-of-day was the number one predictive factor in determining whether a given participant was likely to have their smartphone nearby or not.  Concerning future work, the authors suggest that being able to more accurately predict (with up to 90% accuracy, they claim) when a user will be within arm's reach of or within the same room as their smartphones ought to aid mobile application developers in making decisions such as when the most opportune time is to deliver notifications to their users and by what modality (vibration, alarm, status bar icon).


Title: From the War Room to the Living Room: Decision Support for Home-based Therapy Teams

Name: Sunil Garg



This paper describes a system called Abaris, designed to support therapists who collaborate in autism interventions. Because numerous therapists may interact one-on-one with a single patient, it becomes important for them to reliably collect data about their interactions, so that it can be properly compiled and analyzed to support informed decisions about the patient's treatment. While existing CSCW systems have addressed collaboration both for remote and co-located team meetings, there has not been a focus on collaborative technologies for the home.


The Abaris system is designed around a technique called Discrete Trial Training, in which "a therapist instructs the child in a variety of skills in a highly structured, repetitive manner, helping the child master correct behavior through errorless teaching and positive reinforcement". Throughout each session, the therapist records grades and notes about the patient's performance, which are then used by the next therapist. Abaris augments this existing data-collection practice through the use of video recording and digital pen-based input. The technology is able to automatically create visualizations of the data and annotate the video stream.


To evaluate the impact of the system, the authors conducted a long-term study. Interestingly, they themselves became trained as therapists in order to be able to perform therapy and participate in team meetings as participant-observers. Abaris proved to serve as a valuable tool by helping the team use more reliable data-driven methods and artifacts in their decision-making process.


Title: Getting Closer: An Empirical Investigation of the Proximity of User to Their Smart Phones

Name: Dan Huang



Mobile computing has been growing and becoming even more pervasive everyday. There is an assumption that a phone is always with its owner, and can be used as a medium to collect data from the user, as well as relay information to users at anytime. From a previous study five years ago, participants had their phone on them 81% of the time and kept their phone within arm’s reach 58% of that time. This new study employed a series of surveys and interviews, as well as employing an Android application for 28 participants over a span of 4 weeks to observe their proximity to these devices.


The authors of the previous experimental setup were contacted in order to replicate the proximity study from Farther Than You May Think: An Empirical Investigation of the Proximity of Users to Their Mobile Phones. The AWARE Android framework was developed to help gather information about the owner’s proximity to his or her phone. The framework gathered information regarding phone activity, battery, Bluetooth, phone, location, network, screen, sensor, messages, weather, wi-fi, etc. Data on the proximity of the users were collected via Bluetooth gps devices worn around the participant’s neck. Using RSSI measurements, few minutes of data were collected within arm’s reach (1-2 meters), within the same room (5-6 meters), and unavailable (6+ meters).


The results show that there was no an increase in amount of time the phone was within arm’s reach, but a substantial increase in amount of time the phone was in the same room. Otherwise, the proximity results are very similar to the previous study.


Title: Sensors and Surveys: Collecting Qualitative and Quantitative Data on

Human Attitudes, Behaviors, and Activities via Mobile Phones

Name: Harrison Jones

Summary: This article was a fairly short one. The authors start off defining how important context-aware mobile computing is. They give examples of research being conducted to detect users falling or a user's fitness.  They then go into their platform "MyExperience." MyExperience is a tool they've developed for mobile phones which senses the world around it, takes user input, and does calculation to collect data about human behavior and technology usage. They then give examples of successful usage of MyExperience in studies: It has been used to promote physical activity, monitor stress, and study obesity. One of the big benefits of MyExperience, they continue, is that it is capable of pulling data from a wide array of different sensors and that adding new sensors to the network is trivial with their plug-in architecture. Self-reports, such as "how are you feeling," can be triggered based on time and/or gps location.


Title: Route Classification Using Cellular Handoff Patterns

Name: Joseph Lin

Summary: This article seeks to understand the utilization patterns of the roads within a city through cell phone data and correlate them with actual utilization statistics recorded through conventional means. They approach this by utilizing Call Detail Records (CDRs) that are collected whenever a call on a cellphone is made. These records contain information about which antennas were used by the phone. When multiple antennas are used, then it is likely that the caller was moving along a particular route. Their approach to this problem was to first establish that along a particular route, the same set of antennas are consistently used during a call.


Then they devise an algorithm to the antennas used during a call to a particular route. The authors solve this particular algorithm in two methods. One involves driving along each route and collecting training and test data. The test data is matched with the training data using the EMD (Earth Mover's Distance) algorithm to identify which route the test data came from. This is shown to have remarkable accuracy when enough antennas are used along the route.


They propose an alternate method which does not require the use of test drives. They do so by looking at scanner data along the roads which record the signal strength (SNR) of each antenna within range. The handoff patterns are then matched with the largest sum of SNR values and that tells you which route the handoff patterns belonged to.  This strategy has the advantage of not requiring test drives along each route as the scanner data is already readily available.


The authors go into some further work discussing how their data was privatized before showing a map of road utilization on a particular day. They compare this traffic data with existing data collected from vehicle counts and show remarkable correlation. The strengths of this study was being able to leverage existing infrastructure and build a system to gather traffic data that did not rely on expensive computation or capital. Some applications of this study are mapping trajectories as well as providing real time travel time estimation.


Title: When Recommendation Meets Mobile: Contextual and Personalized Recommendation On The Go

Name: Ravi Karkar


The paper discusses the exploratory analysis done by the team in the field of mobile search queries. The team, a collaboration of Nanyang Technological University and Microsoft Research Asia conducted a 6 month study to collect the search log of a mobile search engine (nearly 400k queries per day) to conduct their research of probabilistic entity recommendation approach in search.


The previous work in the field is:

-          Query Suggestion: which reduces user effort by auto completing the query

-          Object level Vertical/Local Search: uses predefined recommendations for the user to reduce user input by limiting the searching range

-          Recommendation system: provides a ranked list to the user based on the query

Compared to the previous work, easylit(the application created for the survey) does not require query suggestion since it is intent based search, i.e. user is returned a list of sorted entities without any input. Also, the object level search is implicitly vertical, providing user listing of nearby entities only. The problem with recommendation system is that it uses the user’s history along with similar users’ history to provide recommendations which is resource heavy. In easylit they choose to adopt a probabilistic approach for modeling the conditional probability of generating entities for the user.


The team conducted a 6 month long analysis of the mobile search query logs, and the following characteristics:

-          Mobile search query is short: average query contains 2.52 words or 18.76 letters

-          Mobile search is location and context sensitive: depending on the location of the user, the search results may vary for the same query.


Approach adopted for the intent based search consists of 2 components:

-          Entity extraction

-          Entity ranking


The Entity extraction process involves the creation of a db of the extracted entities from various sources. The entity ranker estimates the conditional probabilities of entities and entity types for a given condition.


The probabilistic entity ranking algorithm computes the final list of entities to be given as output to the user by calculating the query against the entity type, context (location and time), user similarity, popularity of the entity and other parameters. User similarity is divided into: entity based, entity type based and entity attribute based similarity. There is also the notion of session. Session is the time interval between two consecutive queries. Note: entity attributes are often ignored when calculating the similarity, since even if the 2 entities are in the same group the attributes can lead to some totally different end result eg: entity – fastfood burgers, however depending on user’s taste, higher preference can be given to either McDonalds or KFC.


The team conducted 3 experiments after performing the analysis of the query logs and developing the easylit application. For the experimentation, the day was divided into 7 time intervals. And 3 measures: entity accuracy, intent accuracy and summation of top level intent accuracy, were used as recommendation factors. Also, for measuring the accuracy 8 parameters were used which consisted of location, popularity, time, etc.

-          Measure the recommendation accuracy

  • It was found that popularity is more important than distance and time also acts as a context for the recommendation
  • Personalized Context-Aware entity Ranking algorithm scored 10% higher than the traditional approaches

-          Measure the sensitivity to context

  • Performance of entity ranker relies on the quality of the queries

-          Conduct user study

  • The application is able to recommend the entity and entity types
  • 12 users were asked to estimate the profile of the user based on the search history provided to them. Each subject rated the list of entities provided on a scale of 1-5.
  • It was found that the entity model was the most successful with a score of 4.31, followed by user model at 4. Interesting finding was that user rating is worse than recommendation by distance.


In conclusion, the team did a detailed analysis of the search logs and developed an intent based search model which proved to be more efficient.


Title: Bridging the Gap Between Physical Location and Online Social Networks

Name: Alex Samarchi



This paper examines the location traces of users to infer relationships between user mobility patterns and structural properties of their underlying social network. They will evaluate these properties based on two tasks: predicting whether two co-located users are friends on Facebook and predicting the number of friends a user has in the social network.


They designed Locaccino, an experimental framework capable of observing both offline and online user behavior. Locaccino works as a location sharing application based on Facebook’s social network. It allows user to share their location with their Facebook friends subject to robust privacy preferences. Their results are based on analysis of the location trails of 489 participants who were tracked using GPS and WiFi positioning technologies installed on their mobile phones and laptops. Locaccino is a web/mobile application developed at Carnegie Mellon.


Of the 489 participants some used the system over a period of 7 days to several months. These participants were recruited through various media such as fliers, electronic message boards, and built invite mechanisms. Throughout the study over 3 million location observations were collected, with nearly 2 million of these falling in the Pittsburgh region. At the end of the study they had accumulated over 20 years of cumulative human observational data.


Locaccino triangulated a user’s position into 0:0002 X 0:0002 latitude/longitude grids (30X 30 meter grid). Co-location of two users is defined as an observation of the users within the same location grid within the same discrete 10 minute interval. Based on this data 3 networks were defined: The Social Network, The Co-Location Network, and The Co-Located Friends Network. The social network was the network of the users friends on Facebook, the co-location network was the network of users who were co-located at one point in time during the study, and the co-located friends network was the network of users where were friends on Facebook and who were co-located at one point in time.


Study results showed that the Co-Location network had 3 times the number of edges as the social network but the social network was better connected. It was also determined that the entropy of a location is a valuable tool for analyzing social mobility data. Entropy is defined as both the number of users observed at the location as well as the relative proportions of their observations. Locations of high entropy are precisely the places where chance encounters are most probably, thus Co-Locations at high entropy locations are thus much more likely to be random occurrences than Co-Locations at low entropy locations. Thus if two users are only observed together at a location of high entropy such as a shopping mall or university center, they are less likely to actually have a tie in the online social network than a place with low entropy.


Data also showed instances where users are not friends in the online social network, yet exhibit very convincing co-location patterns for friendship, and similarly there were numerous instances of friendships in the online social network where little to no evidence for friendship in the co-location data

The authors believe that their system/data could be useful to help redefine the friend recommendation systems currently employed by social networks. They also believe their work could be used to segregate and categorize online connections into groups by building privacy rules and organizing the social graph.

One issue that should be explored further was the homogeneity of their study participants, who were mostly students.



Title: The Domestic Panoptican: Location Tracking in Families

Name: Spencer Border

Summary: Location-Based Services (LBS) for tracking used to be just an idea and prototypical design. Since the advance in this technology, LBS are now being used in the accepting society. This paper presents a qualitative study that examines the use of LBS usage by families and how well it has integrated into their everyday lives. The study consisted of four households that were already participating with LBS applications. The study consisted of a number of interviews, diary studies, and observations for a two-week period. The study consisted of four different phases. In the first phase of the study, participants were interviewed and asked a number of questions pertaining to their LBS use. These questions included: why LBS was adopted in their household, what application was being used, who was tracking who, and how individuals felt about tracking or being tracked. The second phase of the study involved the adult users keeping a diary describing each instance that they had tracked someone. In particular, the authors wanted to know whether the person being tracked was where they were supposed to be and if not, how the user felt and what actions they would take. Phase three was a debriefing session in which the participants were asked to comment on their diary entries. Phase four was intended to be used as an observation time of location tracking in daily use in order to gain an understanding of LBS in an unfamiliar, crowded, and potentially dangerous environment.


There were some key findings that the authors found during their study and evaluation. They were focused on exploring the ways in which location-based systems are changing domestic relationships. In particular, the authors focused on how LBS is being used to apply different forms of digital nurturing. They found that a lack of trust was the initial motivator for participants with children to use LBS (their children were not where they said they were).  After their evaluation, the authors found that using LBS had resulted in a change in their children’s behavior and found that the parents became less dependent on the service.  An instance of this is that one of the participants reported that her children were now more honest with her due to the use of LBS. Limitations in the technology were exploited by the children that allowed them to be viewed in one location when they were elsewhere. Although some of the children took steps to avoid being tracked, children in the households appeared to like being tracked for their safety.


LBS also created a form of trust work (trust as something that is “done”, enacted and maintained). The authors argue that LBS limits the opportunities to maintain and display trust, where everyday trust activities are replaced by a technology medium. It seemed that a LBS technology fosters distrust in the use of adult tracking of one another. This technology provides power to individuals and can provide evidence of an individual’s whereabouts.  In one instance, a participant admitted to tracking her partner for promiscuous behavior. It was noted that the one being tracked felt a lack of trust from the one doing the tracking. This suggests that appropriate use of this technology is being negotiated within households and there seems to be a considerable debate by individuals about its use. This in turn shows that LBS is not socially acceptable yet.


This study has shown that LBS changes how trust work is viewed and how LBS is being incorporated in the household. This domestic panopticon is still only a possibility until its full technology is socially acceptable allowing for mechanisms to protect and maintain trust within the household.



Title: Exploring End User Preferences for Location Obfuscation, Location-Based Services, and the Value of Location

Name: Hans Reichenbach

Summary: This paper explored how users would react to sharing long-term location based data publicly or with specified groups when using various methods of obscuring the identifiability of the data. The researchers gave 32 individuals (12 households) GPS receivers to keep with them at all times (they could leave them or turn them off if they wanted but were asked to keep them on as much as possible) for a 2 month period. Afterwards they were interviewed and given a questionnaire asking about how comfortable they would be with sharing the data based on 5 obfuscation types and 4 levels per type. The types of obfuscation were Deleting (removing data near sensitive locations such as a person's home), Randomizing (adding noise to the location plots), Discretizing (use larger grids instead of distance points when marking the datapoints), Subsampling (increasing the time between location samples), and Mixing (mixing user data with other users' data in the same general area). The majority of participants chose Mixing as their preferred method, with Deleting and Randomizing being solid seconds. Neither Discretizing and Subsampling were favored methods with only 2 people choosing Discretizing and nobody choosing Subsampling as a preferred method. Users largely seemed to understand how the different types worked and typically chose methods consistent with their largest areas of concern relating to sharing data. They did tend to underestimate the ability of people to infer data over a long time period with some methods of obfuscation, though, as well as seeming to think that the higher levels within an obfuscation type implied higher levels of privacy (which is not always true). The main 4 areas of concern seemed to be keeping home location private, obscuring identity, obscuring location, and keeping the data useful.


Participants were also asked how comfortable they would be with various levels of sharing their long-term location data. Overall around 50% of the people in the study said they would be ok with their location data being available online (with most of them preferring it to be anonymous), 15% said they would be ok with corporations or academia having access to their data, and 23% not wanting to share their data at all. There was very little differences between obfuscation types, the participants were largely consistent with their sharing level regardless of what method was employed to safeguard their data. At the end of the study the participants were asked if they would consent to anonymized data collected during the study to be posted online where the public can view the data. 21 of the 32 participants agreed, with most of them being male and renters.


The researchers also asked the participants to try and put a value to their location data by asking how much they would want to be paid to allow them to continue the study for a longer period of time. The participants had a very tough time with this part, with the values they asked for varying greatly. They didn't seem to be able to come up with a good point of reference to base it off of, with most complaining about not knowing what the competitive price was. They consistently gave discounts for longer period observations, though, when compared with shorter term studies. The prices for a one month extension were roughly in the $100 range, while prices for a full year were around $500 on average.



Title: Bridging the Gap Between Physical Location and Online Social Networks

Name: Jon Pelc

Summary:  The authors of this paper aimed to examine how data received from a location sharing social network could give insight into their social network.  Specifically, if by using the ubiquity of mobile phones, they examined the data using a series of statistical formulas to examine features of the data that contextualize the locations users visited and analyzing the predictive power of those features.


The authors used a location sharing application called Locaccino based on Facebook’s social network.  It involved client software which ran in the background of their laptops or mobile phones which transmitted their position using GPS, WiFI and IP geolocation.  There were 489 participants the majority of which were recruited for three separate studies from Carnegie Mellon’s campus.  The rest were either invited by this majority or found their way through other means such as academic papers and online press.  They used the system anywhere from seven days to several months.  In order to control the study the authors ignored any location data from outside the Pittsburgh metropolitan region.  This reduced the recorded location observations from 3 million to 2 million.


To determine co-location, latitude and longitude space was split into grids consisting of .0002 x .0002 degree (30 meter x 30 meter) cells based on a balance of the accuracy of the location technology and the noise caused by larger windows.  If two users were in the same cell they were considered to be co-located.  The co-location data formed a network, referred to as the Co-location Network, where two users would be connected if they were co-located.


In addition to the co-location network the authors used two others.  The social network based on the users’ Facebook network.  Two users are connected if they are friends on Facebook.  The final network is called the Co-located Friends Network and is an intersection of the Social and Co-location Networks. 


The authors then proceeded to model the co-location of two users and individual user mobility of the Co-location network.  To model the co-location the authors extracted a number of features for each edge in the network which were placed into one of four categories: intensity and duration (how long and how much user used the system), location diversity (discussed below), specificity (how specific is a location to a pair of users), and structural properties (strength of the structure relationship between the two connected users).


They measured the diversity of a location based on three factors: frequency (how many location observations), user count (how many unique users have been there), and entropy.  Entry works as such: if many users were observed at a location with equal proportion the entropy will be high.  It will be small if it is heavily concentrated on a few users.


Features of user mobility were also abstracted for each vertex of the co-location network.  These included intensity and duration, location diversity, and mobility regularity.  For individual user mobility they first determined the regularity of a user’s routine by restricting their observed locations by location, day of the week, and hour.  By restricting on a particular attribute they could see how a user’s schedule varied based on that attribute.  If it was concentrated on a few spots their schedule was regular based on that value.

The results showed that when using classifiers to predict Facebook friendships based off co-location, co-location by itself is not a strong predictor of online friendship but can be improved by using additional contextual social properties of the locations the users visit.  They also found that an examination of the context of the locations a user visits and the user’s routine can provide knowledge into the social behaviors of the user.


The authors conclude by asserting that properties of the locations a user visits will provide a valuable context to the user observations.  Entropy in particular.   They also believe that entropy of the locations of a user’s visits can provide insight into the number of ties the individual has in the social network and the strength of their ties but mention more study is required.  Potential uses for their work include enhancing friend recommendation systems by including offline world behavior, aiding systems to segregate and categorize connections, and acting as window into the relationship between online and offline social behavior.


Title: The Domestic Panopticon: Location Tracking in Families

Name: Lauren Schmidt

Summary: Coming soon

Title: Route Classification Using Cellular Handoff Patterns

Name: Josh Fierstein


This paper takes a look at creative ways to analyze call detail records (or CDRs) to extract useful information regarding the route the phone traveled based on the call handoffs from cell tower to cell tower.  A variety of algorithms are assessed including variations of Common Subset Differences, Earth Mover’s Distance algorithm, and the Nearest Neighbors algorithm. Using common subsets, (i.e. looking at similar patterns of commonality between sets of data from Antennae) results in a loss of some crucial information, so a variation of the Earth Mover’s Distance (EMD) was posed to account for the sequential nature and the location information of the handoffs.


Researchers took a sample of CDR data for 60 days in a modeled environment. The best results came from a combination of the EMD algorithm and NN (nearest neighbors) however other aspects such as signal strength where assessed and factored into the route classification algorithms. They also discussed how the algorithms can be used in practice to predict traffic volume along the proposed routes.




Title: The Domestic Panopticon

Name: Dylan Demyanek

Summary: With Location-Based technologies becoming more widely-used, it is important to understand how such technologies impact the privacy, trust-relationships and safety of users. The authors of the paper conducted a two week long study involving four households in London that regularly use Location Based Services (LBS) to track one or more family members. In some instances the parents used LBS to track their children. However, other situations also occurred; spouses tracking one another, children tracking each other and even children tracking their parents. The focus of the study was the relational and psychological impact of the technology on the families. The researchers found that the families' motivation for using the technology ranged from remote parenting to curiosity to vouyerism. Interestingly, they concluded that it mattered little what the motivation was, as just using the technology at all was seen as a breech of trust. The study found that the use of LBS has great potential for undermining trust-based relationships. Also, using LBS allows for less opportunities for building trust in general.


I found this article interesting, given the recent release of iOS5 and the new "Find Friends" feature that allows a person to track the location of certain contacts. The feature requires users to "release" their location to others who wish to follow them. In addition, there is a settings option that allows users to temporarily hide their movements and location information from others. In light of the paper's conclusion, these features seem like a welcome change from older, less flexible tracking applications. The primary issues that relate to this study are privacy and trust, both of which are important to everyone. LBS invades on both of these issues, and consequently remains outside of mainstream acceptance. However, in the future I believe we will see more socially acceptable options and more widespread adoption.



Title: The Domestic Panopticon: Location Tracking in Families

Name: Cheng-Pin Lee

Summary: This paper dealt with the issue of location tracking within families. They studied four households in the London area, in which LBS (Location based-services) were used to track the participants' children, spouse, cousin, or uncle. In each case, the participants had their reasons for using the technology. The most prevalent reason was to ensure the safety of the person or persons they were tracking. For example, two mothers used it to track their children (and in some cases, their spouses), and one participant used LBS to check up on his uncle. Another reason for using the technology was just to try the technology, as seen with the household that used the technology to track their cousins. In all cases but one (the case where the nephew was tracking his elderly uncle), there were "trust work." Trust work is defined in the paper as basically complications having to do with trust. The paper talks about the fine line and "boundary" between the "well-intended self-assurance of safety to voyeuristic behavior," and can be seen when the parents (more specifically the mother in household 1 & 2) continually check up on their children and finding that they are not where they are supposed to be, or when one spouse checks the location of the other. The children sometimes feel that they are in the metaphoric panopticon in that they are "prisoners." The spouses sometimes feel the same way and feel that this act displays distrust more than any other sentiment. In all cases, the paper notes that the dynamic of the family has changed due to the technology. It is concluded that (within this study) LBS has a profound effect on the workings of the family, and the technology should focus on the issue of "trust work" before it becomes even more prevalent in consumer lives.


I found that the paper noted some good arguments about the use of LBS technologies, in particular the use of the technology to check up and ensure the safety of family members. I agree that the trust issue and privacy issue is definitely there but I found it interesting that they did not comment on how household 3 (the nephew and uncle pair) thought of the trust and privacy issue. The paper notes how participants began to find ways of tricking the system and faking information, and the paper also notes that this causes people to weigh the benefits of having their privacy invaded vs. all other benefits of having their phone with them. I think this phenomenon would have happened with anything in which parents monitoring their kids or spouses monitoring spouses is involved. Furthermore, I do agree that in order for this technology to be more prevalent, researchers have to focus on the trust and privacy issue.



Title: When Recommendation Meets Mobile: Contextual and Personalized Recommendation On The Go

Name: Oriol Collell Martin

Summary: This paper proposes a method jointly with an example application to recommend entities (such as a Sturbucks) and entity types (such as coffee bar) to mobile phone users based on the context. The paper starts by describing an empirical study they conducted to examine how mobile phone users search for entities in query based systems, based on a large-scale query log data set that they collect. With the study, they conclude that searching on the mobile phone is popular, that people do not like to write long queries to search for what they want and that queries are context-sensitive with respect to location and time. The latter means, for instance, that more queries searching for shops or restaurants will be made on commerical areas and during the afternoon.

The study motivates the authors to create a method to search for entities that is both context-aware and that does not require any input from the user. The method contains two major parts:

  • Entity Extraction, which consists on gathering entity data from the Internet using a rule-based approach.
  • Probabilistic Entity Ranking, which relies on a user model built from previously issued queries by him, and can get a list of best matching entities and entity types according to the current context and that query base. The basic idea of the algorithm is to find the queries issued on the actual context (location and time), and then weigth these queries according to the user similarity. The algorithm calculates the probability for each entity according to the user and then ranks each entity according to that probability.

The authors also perform a set of 3 experiments to evaluate the proposed approach and to evaluate the user satisfaction when using an example moible application that they build.

In the first experiment they conclude that the proposed method delivers better results that other more naive methods. They also conclude that personalization exists in this type of searches, that popularity is more important than geometric distance and that time is an effective context for recommendation.

In the second experiment they first test the accuary of the approach, concluding that the accuary is not very high (around 40%), but that it still provides valuable suggestions about the nearby entities. They also test the recommendation method against 4 invented user profiles and observe that the recommendations vary depending on the user profile and the current context, which verifies the personalization target of the approach.

In the last experiment, they conduct a user study with 12 people. They provide each subject with an invented profile and two different contexts (two times and two lactions). Subjects, then, are presented search results according to the different contexts and according to different methods, and they are asked to give the satisfaction score for each method. They conclude that methods that take into account the user profile are more user-friendly since they are given greater satisfaction scores.


Comment: The paper proposes an innovative method to recommend entities to users which seems to be appropiate given the current needs of users. I  find that this desire of completely removing the need for explicit user input moves in a very good direction and is a good effort to reach the goals of Ubicomp and the visions of Weiser. However, the method builds in several assumption. I find two of their assumptions problematic. First, they build upon the assumption that the method will have available a user model which is the result of a large query based of previously perfromed queries by the user, but it is not clear that this query base will be always available. Second they obsever the fact that textual queries performed by users are most of the time very short, and claim that this means that people do not like to write long queries; however, this could also mean that people have enough with short queries to find what they want.


Title: When Recommendation Meets Mobile: Contextual and Personalized Recommendation On The Go

Name: Tarun Chakravorty

Summary: This paper discusses an entity recommendation system and demonstrates this system through a WP7 app called easylife. The system ranks both entities (like ‘i like pizza’) and entity types (‘restaurant’).  The system is both context aware (geolocation and tme service), and personalized (history of user queries). The system is then put through performance and UX tests. The system is similar to the Local Scout feature on WP7 phones.

The authors research finds that a users intention of using a mobile device is largely local (context-aware). So they focus on the context signals available on the phone to tailor the entities to the users interests. They use 3 components on the cloud:
1. Entity crawler - collects entities from the web
2. Entity extractor - recognizes entities from queries
3. Entity ranker - ranks based on context

The paper then proceeds to explain the analysis of the mobile click through data which was taken from a commercial search engine. Some results:
1. Mobile search is more popular in Big cities.
2. Mobile search is short - around 2.5 words / 17-18 letters per search
3. the search is very sensitive to location and context aware (time of day when searching)

The paper then explains the recommendation approach (which includes the entity extraction system and the ranking system). More focus is kept on the ranking system. The ranking system takes a probabilistic approach. Some real world example of what this is:

“For example, considering two users, one of them is interested in McDonalds, the other always  refers KFC.When they come to a district where only BurgerKing is available, they may both like it since they both need fast food.”

The above example can be categorized as User similarity. The other such category is random walk. An example for this is:

“Considering the meeting of some friends, they probably query the restaurants before dinner.  fter having dinner, they may be interested in the bars nearby for night life.”

The mobile application employing this system had entities from a dataset of 700 million local businessesand 30 entity types in the US. The results of the user study are explained after the recommendation system.  12 random users were selected and were asked to estimate the profile of a user, whose search history was provided. The they were asked to select two time slots and 2 locations.
Based on this the subjects were provided with 4 recommendation results:
1. Entity types ranked by the built user model.
2. Entities ranked by user model belonging to  top ranked entity type.
3. Ranked by distance to current location.
4. Ranked by user rating

The satisfaction score of the entity type recommendation was found to be the highest. User model achieved a better performance as compared to recommendations by distance and ratings, which shows that this model is more user friendly.


This project takes on the idea that “users don’t know what they are looking for”. And I think it does a good job of solving the problem. Apps similar to this project are already on the market and I find myself using such features a lot. It fits well with the goals of ubicomp.

The fact that the system looks for what other people in your present location and time searched for and uses results from that to provide you with relevant results is a very strong idea. I wonder how it would work for a tourist visiting a new city for example. This can actually help tourists get to what you can say is the “life” of a city or neighbourhood.




Title: The domestic panopticon: location tracking in families

Name: Suchit Dubey


Technology today has enabled location-based services like Google Latitude, Foursquare, Find my friends iPhone app etc to track people for both social and personal reasons. The authors in this paper examined protracted usage of location-based services by families and how it is integrated in their lives. They conducted an ethnographic study with four households and tried to ask questions like what motivated people to use LBS, what kind of LBS did they used, when and how did they used LBS and what effect did they it have on their family privacy and relationship. They examined how technology is adopted in the domestic work of parenting, elder care and romantic and friendship-based relationship(called digital nurturing). Finally they argued that LBS by allowing surveillance shortchanges the trust building activity(called trust work) in a family, thus creating a tension between digital nurturing and trust work.


The study consisted of four households(14 individuals)in London who were already using LBS applications and they participated in interviews, diary study and observation for two-week period. Two households consisted on mothers tracking their children, one consisted of a nephew who was tracking his older uncle and one household in which two cousins were tracking each other. The authors found that their study exposed the tensions found in most households and that the technology had further exacerbated them.


Key findings:


Motivations for using LBS services

Mothers took the responsibility to monitor their children. They were concerned that children might stray into areas which are unsafe. One participant use LBS to take care of elderly because of their disability. With the use of LBS the participant didn't have to call him or be physically present to monitor him. People also reassured themselves that the family members were safe by monitoring their family member's location. People were also curious to know whether their partners were actually in the place they said they were going to. This may be due to general curiosity and not due to suspicion.


Observed behavior

LBS provided concrete information to mothers about their children which they used to discipline them. 

They were less dependent on LBS overtime. Mothers also felt like spies using the technology with their own family and overtime, instead of confronting them, they would strategically engage their children. Parents also felt that real motivation of being concerned about their child's safety might go unnoticed

and they might look like showing a lack of trust.


They also realized how teens change their plans and their hangout locations. Children's behavior improved because of the fear of getting caught. However, children were displeased to learn about the technology. Some tried to exploit the weaknesses of the technology by leaving the phone at home or friend's place. They also argued that the service malfunctioned and sometimes turned off the phone. Children chose between having a phone and their parents not being able to track them.


The authors concluded that the children behavior demonstrated a need for privacy beyond that LBS currently allows. Since trustworthy actions occur under surveillance, good action cannot be attributed to good behavior to fear. This paradox might in turn develop distrust between parent and child. Thus, LBS might encourage distrust. It is not socially accepted yet, and its role is being still negotiated in home.

Route Classification Using Cellular Handoff Patterns

Name: Angela Mingione


In this paper, researchers wanted to find a way to track traffic patterns. Knowing traffic patterns is useful for urban planning, but since current methods are expensive, the tracking is rarely done or is done with small populations.


The researchers used Call Detail Records (CDRs) to track users on a call while traveling. Testing was done over 8 months using a variety of phones. The researchers used all the major roads into the city for testing and conducted tests in different weather and traffic conditions to see how stable the patterns were. During the drive, they also tracked the actual route using an iPhone app they had built and then used two algorithms to look at the data - modified Earth Mover's Distance and an algorithm based on signal strength.

Title: Bridging the Gap Between Physical Location and Online Social Networks

Name: Travis Wooten


Summary: Researchers explored the connection between offline, real-world interactions and online social network interactions. Studying 489 users, they compared the users' location information with that of their online social network friend information. They used Loaccino, a web-application that allows users to share their location through Facebook, to track the locations of users. Location was determined done using GPS data, Wi-fi location techniques, and IP geolocation. Location was gathered from laptops and mobile smartphones alike, with 93.7% of the data coming from laptop computers.


Researchers were trying to ascertain whether or not co-location, defined roughly as being in the same place at the same time, corresponded to being actually socially connected, i.e. Facebook friends. Researchers discovered that, in fact, co-location and online social connections are connected. If two people were recorded to be co-located at a location marked as one with high entropy, or a record of very diverse visitors, they were less likely to be connected through an online social network. However, those same people were also found to have more total friends on the social network. Conversely, if two people were co-located at a location defined as having low entropy, they were much more likely to be connected through an online social network. Furthermore, the users who visited low entropy locations were found to have fewer total friends on the social network.


Route Classification Using Cellular Handoff Patterns

Name: Sean Swezey


Millions of people commute everyday to and from work. In the U.S., the most common way people way get to and from work is by car. This paper looks at ways to track current road (and train) route usage to and from a city to better determine how the routes are being used and where to spend the money for them. Currently, the DOTs use cameras and sensors along with sometimes user surveys to figure out who is going where, but they are not very good at determining routes and user surveys are not so easy or cheap to do.


In this paper, they want to match cell phone records to routes that people travel. They are mainly focusing on freeways and train routes which people use to commute to and from work, and not the side roads and access roads to them. They implement this using the cell phone records from a cell phone company for phone calls which take place along the route and two separate algorithms to match the records to the routes. The first algorithm they use is a derivative of the Earth Mover's Distance, in which they use the location of the antennas and common subsets between antennas to determine the route from the cell phone record. The second uses the signal strength from an antenna to the cell phone during the course of the call to determine the route. The EMD method requires test drives to give accurate information, whereas for the signal strength algorithm, it can use information which the cell phone companies already collect, preventing the need for test drives. They both end up performing well, and are able to correct determine the route from the cell phone log.


The major limitation with their study is for the algorithms to work and to have the information needed, the person has to be on an active call for the duration of the drive. And according to the paper the DOT only estimates 9% of people driving to be on an active call (non-handsfree). Also, they studied their findings in a city with a hub and spoke architecture with the city being the final destination, or acting under that assumption. With a different highway layout, they would have to perhaps work on how they gather their information, and to insure that they could differentiate unique routes through the city. Also, they only worked with one cell phone company, and thus only data from customers from that company, and would need further companies' participation in order to more accurately determine information.


The study was successful in what they set out to achieve. There is potential to use the findings further on in tracking car usage on roads, but it is also is something which is not readily available to the people who need it, the Department of Transportation. Only time will tell what comes of the findings of this study.


Title: Bridging Gap Between Physical Location and Online Social Networks

Name: Bilal Anwer


This work looks at the real world location traces of 489 users (based on an average of 74 days usage) to correlate them with their online social networks. The work infers properties of social behavior of users from their location trails. This paper uses data from already developed facebook based location sharing application called "Locaccino". It uses GPS,IP Geo-Location and WiFi positioning technologies and sends them after every 10-minues to Locaccino server. There are 3 main contributions including, Model of friendship in online social networks based on user's location information, establishing relationship between mobility patterns and number of users and affect of entropy on social interactions.


This work divides the location into grids of 30x30 meters and then captures user coordinates after every 10 minutes. It then considers three networks, Social,Co-location and Co-location Friends networks.The model developed in this work uses diversity measures, co-location features and user mobility features.

 To measure the diversity of a locations three measures are used: frequency(raw count of user observations at a location), user count(total unique users),entropy(number of users at the location and their relative proportions of observations). For the col-location features it considers different independent variables that measure, Intensity and Duration of the system, Location diversity, Specificity,Structural Properties etc. Similarly for user mobility features: Intensity and Duration,Location diversity and Mobility Regularity are used.


For evaluation of the developed model and the validity of the features extracted they use different machine learning techniques like AdaBoost,SVMs etc. with different parameters. The performance of each classifier was measured to see if the users are friends on facebook using 50-fold corss validations. Among different algorithms AdaBoost performed the best in inferring social network ties from the co-location features mentioned previously. Similarly in inferring the number of friends from user mobility data, users with highly diverse locations in their location traces end to have more social network ties. Similarly, users with irregular schedules tend to have more ties in online social networks.


Regarding locations of high entropy, if two users are only observed at locations of high entropy, mall,university center, they are less likely to have online social network than if they are seen at a location of low entropy. Users who visit location of higher entropy tend to have more ties. Although this work is promising, but the authors point out location traces have many privacy concerns and their data-set was homogeneous as it contained mostly students.



Title: Route Classification Using Cellular Handoff Patterns 

Name: Caleb Southern



The authors attempt to use location data from cell phones in order to determine the routes that drivers choose on road networks. This is based on the assumption that users have their phones with them and the phones are a proxy for their locations, which may be more valid when driving than during other activities. One challenge is that cellular antennas often cover over one square mile, and thus provide only course location data. However, the authors hypothesize, and later confirm, that the sequence of antennas can be used to determine which route a driver has taken.
The first test the authors performed was the stability of handoffs between cell towers. They traveled several routes, with two phones calling each other, and examined the CDRs (call detail records) from the telecom network. They determined that the handoffs were sufficiently stable to use as a data source for their position algorithms. There was a lag between the same trip going in opposite directions.
Next, they tested two algorithms for determining the route traveled: (1) Nearest Neighbor using Earth Movers Distance (EMD); and (2) Signal Strength. The Nearest Neighbor algorithm proved largely successful in identifying routes traveled, with some problems distinguishing between routes close to each other. This algorithm required training data for each route. The Signal Strength algorithm relied on data from the telecom company that included GPS-stamped signal-to-noise measurements of cell towers taken at one-second intervals along various routes; thus, this algorithm did not need additional training data.The Signal Strength algorithm was better for shorter trips (with less than 9 handoffs), but was otherwise comparable to the Nearest Neighbor algorithm.
Finally, the authors tested their algorithms against the ground truth of DOT traffic analyses, and found a correlation of .77. One disadvantage of their method is that the traveler must be engaged in a phone call, so they had to estimate total traffic based on the percentage of drivers assumed to be using cell phones. Advantages of their system over existing methods is that it can be frequently updated, is cheaper than traffic sensors and driver surveys, and is potentially more accurate than surveys.

Title: Bridging the Gap Between Physical Location and Social Networks
Name: Kevin Kraemer

This paper involved analyzing the relationship between peoples' physical locations and their social networks. The researchers tracked the physical location of 489 subjects over the course of a few months. They then compared the location data to each subject's network of friends on Facebook to see if they could predict whether two subjects are Facebook friends based on their location data.

The location data was gathered by using a webapp called Locaccino that lets users share their location through Facebook. Locaccino uses GPS, WiFI, and IP geolocation to calculate the user's location, and it runs on mobile devices and laptops. 94% of the observations were collected from the laptop software, which could lead to accuracy problems since laptops can often be on for prolonged periods of time while away from the user. The majority of the subjects were Carnegie Mellon students. The researchers limited the location data they would collect to the Pittsburgh metropolitan region.

To determine if subjects were co-located (in the same location at the same time), the space was divided into a grid of 30m by 30m squares. If two people were in the same square within a 10 minute interval, they were marked as co-located.The researchers considered three networks: the social network of Facebook friends, the Co-located Network of subjects who were co-located, and the Co-located Friends Network that was an intersection of the first 2 networks.

For each location they measured the frequency of user visits, the number of unique visitors, and the entropy. The entropy of a location aggregated the number of user visits with the relative proportion of visits. A location with high entropy had a large amount of users with equal proportion, while low entropy locations had a few users with high proportion.

Ultimately, the researchers found that they could not accurately predict whether two users were Facebook friends just by using the co-location data, but the prediction was significantly improved by taking into account additional contextual properties of the locations. In relation to location entropy, they found that people who were co-located in high entropy areas were much less likely to be Facebook friends than those who were co-located in low entropy areas. Similarly, people who frequented high entropy locations tended to have more Facebook friends than those who mainly went to low entropy locations.


Title: From Awareness to Connectedness: The Design and Deployment of Presence Displays

Name: Isaac Kulka


In this paper, Anind K. Dey and Edward S. De Guzman describe a 2006 study conducted using a pair of peripheral presence displays designed to promote feelings of awareness and connectedness toward remote loved ones.  The study was conducted over multiple phases including an initial contextual inquiry, a three week cultural probe, a 10 day focus group, and a five week field study.  Each phase in the study relied on the participation of groups of young college students (primarily women) who lived in shared apartments or houses, who were not members of the same department as the authors, and who used instant messaging (IM) clients regularly.


Contextual Inquiry & Cultural Probe

The five participants in the contextual inquiry were recruited via paper fliers and online bulletin boards.  The purpose of the inquiry was to understand how participants maintained awareness of and connectedness to friends and family (loved ones).  The cultural probe was conducted over a three week period, where the seven participants were paid $30 each to describe how they keep in contact with loved ones, and hence maintained feelings of awareness and connectedness.  The contextual inquiry and the culture probe produced similar findings.  Participants reported they rely on peripheral artifacts such as pictures, small toys and stuffed animals to remind them of distant loved ones.  These artifacts were usually present in the participant’s bedrooms, which is where they spent the majority of their at-home time.


Presence Display Designs & Focus Group

The authors produced designs for a total of 10 presence displays.  The displays were based on modified designs for common household objects ranging from stools and stuffed animals, to picture frames and photographs, to wall-mounted mirrors and information appliances such as clocks and thermometers.  The 10 design ideas were shared with an online focus group of eight participants.  The focus group was conducted asynchronously over a period of 10 days, where the participants were paid $30 to share their thoughts about the designs with the moderators and with one another.  Participants selected the augmented mirror and picture frame as the two designs they believed were most likely to promote awareness of and connectedness to distant loved ones.  The designs were selected due to their unobtrusive, wall-mounted form factor, non-distractive peripheral nature, and a perceived connection between the participant and others.  The augmented mirror was instrumented with colored LEDs and a USB connection to the participant’s PC.  Open source software running on the PC was alerted when a specific loved one’s IM status changed or when an IM was sent to the participants.  The picture frame had a rotating icon that displayed the loved one’s current IM status.


Field Study & Conclusions

A field study of nine participants was held over a period of five weeks.  Participants were paid $100 each to participate.  Three participants had augmented mirrors mounted in their bedrooms, three had picture frames installed, and three had no display installed in order to server as a baseline group.  The field study was divided into three phases: one week without presence displays installed, three weeks with one installed, and then one more week without.  Participants were randomly polled during the five week period to measure their awareness of and feelings of connectedness to their chosen remote loved one.  Awareness was objectively measured by how well users could recall their loved one’s current IM status (available, busy, unavailable).  Connectedness was measured subjectively by asking participant’s how close they currently felt to their loved one at the time of the poll.  Results from the five weeks of polling concluded that the six participants with displays did increase their awareness of and connectedness to  remote loved ones.  The three week period where displays were installed in the participant’s bedrooms had a much higher success rate of recalling the loved one’s current IM status and higher feelings of closeness than during the before and after weeks.  The three control group participant’s had much lower success rates of recalling IM status and lower feelings of connectedness during the five weeks, but the small sample size of the control group was statistically insignificant.  The authors concluded that presence displays were effective for promoting awareness and connectedness.  The authors also suggest that future studies be carried out to explore the possible connection between awareness and connectedness, rather than treating them as separate concepts.


Title: From Awareness to Connectedness: The Design and Deployment of Presence Displays

Name: Kartik Agarwal


The paper outlines the development of peripheral physical Presence Displays of online presence of close family and friends and aims to improve a user's sense of connectedness to loved ones. The process follows a user centered design process in which the end users are involved from conception to completion to validation of the two hypothesis that 1.presence displays lead to better awareness(defined as knowing about one's environment and surroundings and the activities of others) and 2.presence displays lead to better connectedness(defined as a positive emotional bond with a loved one). The prototypes visually represent a loved ones Instant Messaging(IM) status(available, offline, busy) as an indicator of their presence and are incorporated into daily objects like a mirror and a photo frame.


The authors work with college students as subjects as they were most likely to use IM, were early adopters of new technology and had recently moved away from family and friends to attend college and expressed a desire to stay in touch. The design process begins with a contextual inquiry of 5 students and the purpose was to understand their personal spaces and objects with which they had a personal connection and why, and whom did the objects reminded them of. The finding was that the subjects spent most of their time in their bedrooms and many of their personal effects were located there and hence it would be the place where awareness about a loved one would be most useful.


The next phase was the cultural probe exercise in which 7 subjects were given 17 tasks to complete in 3 weeks, which included describing a loved one as a personal object, how they stayed in touch, how they were aware and connected, why they maintain this connection and what personal objects remind them of these loved ones. The goal was to infer the form, function and location of the presence displays from the responses. The findings reinforced the belief that displays should be physical, should present information peripherally, be small and existing and should provide an obvious connection between the user and the loved one.


The results from the contextual inquiry and the cultural probe were used to create 10 Presence Display concepts which included a stool that would change color to indicate presence, a picture frame that would represent presence as an icon on the frame, a photo wall grid of loved ones, a stuffed toy that would change color/temperature or vibrate to indicate presence, a toy that would release scents when a person's status changes, a mirror with colored LEDs embedded into the frame which would change colors on status changes. The decision to use change of color was to make the displays unobtrusive so that they do not demand immediate attention.


A focus group of 8 subjects was run online for 10 days to collect feedback about the concepts and to gain a consensus about the ones which can be implemented. The online focus group allowed participants to anonymously respond to questions posed and to see and comment on the responses of others. The subjects were also asked to think about how each design might affect their sense of awareness and connectedness to a loved one. The top 2 concepts that emerged were the picture frame and the augmented mirror, which were then implemented.


Finally, quantitative and qualitative results from subjective and objective measures from a 5 week long field study of 9 subjects was presented. The field study was divided between two groups, one used the presence displays and one did not and represented the baseline. The group that used the presence display went through three phases, in phase 1 the awareness and connectedness data was collected without any presence display for 1 week, in phase 2 the presence displays were kept at the subjects' bedroom for 3 weeks and in phase 3, the displays were again removed. The findings validated the authors' hypothesis that presence displays significantly increase a user's sense of awareness and connectedness towards a loved one than a traditional GUI and lead to better understanding of a loved one's daily routine such that the subjects were able to more accurately guess a loved one's online status in phase 3 after using the presence displays for 3 weeks in phase 2.


Title: Bridging the Gap Between Physical Location and Online Social Networks

Name: Harish Kothandaraman



The paper talks about how physical location and online social networking relationships and interactions can be used to predict the nature of their relationships, besides analysing the social context of a geographical location and its influence on social networks online. The premise of the paper is to track the location of the users through their smart phones and in certain cases, foot-scale devices.Using Locaccino's location sharing ability, tracking of Facebook users was done and friends who were colocated and other similar cases were taken as a positive entropy and a model of friendship was built. The paper goes on to talk about how the diversity of the location with the increased uniqueness of visitors there may be used to analyze the social interactions at that location. They justify their method of using Wi Fi as tracking medium instead of Bluetooth handshake. Using all the data, they created a graph of social network based on a grid like geographical map. They produce networks called Social Network (FB network), Co-location Network and Co-located Friends network (Colocated FB friends. The measure diversity in three ways - frequeny, user count and entropy. Frequency is the raw count of occurences of a user at a location, user count is the number of users there and entropy is a measure of proportion of the users at the location at a time.Logically, a location with a high entropy such as a crowded mall is where chance occurences are highly likely to occur.


Using the structural nature of the graph and the measured diversity, the regularity of a user's routine could be studied.The method they use to track a user's routine is that they restrict their observations to just the location and day of the week and observe how the pattern changes over the week. The observed joint probability distribution seemed to help track the regularitiy of routine of users.


Using these methods and highly trained AdaBoost classifiers they were able to infer the social networking ties of a user based on co-location and the number of friends using mobility data.For the latter, they tracked the mobility patterns of the users and found how interacted with the rest.


They claim that the results are enough evidence that analysis of the context of a user visit to a location and that of regularity of his routine is a good tool to analyze the social behavior of the user.




Title: PreHeat: Controlling Home Heating Using Occupancy Prediction

Name: Caleb Southern



[coming soon]




Title: PreHeat: Controlling Home Heating Using Occupancy Prediction

Name: Donovan Hatch


Summary: This article detailed the efforts of the preHeat project worked on by Microsoft Research Divisions and Mike Hazas from Lancaster University which aimed to make heating homes more efficient through occupancy prediction.  The experiment consisted of 5 different homes with 3 being in the U.S. and 2 in the U.K and lasted for 61 days.  The houses in the U.S. used hot water space heating while the houses in the U.K. used radiators and pipes in the floor to heat the rooms.


The PreHeat project uses Occupancy prediction to calculate the probability that there will be somebody in the house at any given time based on historic data. When a user walked into the house they left their keys at a specified place that could be detected by the server for the house so that it would know you were home.  Users were required to take their keys with them with they left so that the server would know when they were not there.  In the project they found that the best number of days to look back in time when calculating probability is 5 days.


They were specifically looking at two factors in determining how efficient their system was: savings in gas and the MissTime, the time that the temperature was more than 1 degree C than the designated time for a user.  The results were that overall it improved MissTime in all houses by substantial amounts. When looking at Gas savings, the results were a little different.  When comparing their results to a Scheduled Thermostat mode, U.S.  houses used more gas while U.K. houses saved a lot of gas.  When comparing PreHeat to an Always on Model, there were gas savings across the board in every house.



Title: Toolkit to Support Intelligibility in Context Aware Application

Name: Ramik Sadana

The paper discusses the construction of a toolkit that aims to standardize the process of providing explanations about the existing state of a context aware application to the user. Understanding the state of a machine/system is highly desirable, both in terms of understanding what is going right and what is not, as well as to perceive the privacy and security issues of what the system is actually doing. Since the realm of context aware applications is so large, it is a difficult task to standardize to a process of communicating since it takes a large amount of time for the developer to generate and maintain consistency for such a task.


The authors, Brian Lim and Anind Dey, build on the existing context-aware platform they have created in previous research and increase the scope to afford two-way communication. By hard-coding rules and mechanisms for the machine -> man communication, they take away the responsibility of detailing each process from the developer and make it a system task.


Decision models used in the context aware computing

Reviewing past literature in the field, the authors first decide upon 4 models which are used at any given stage by a context aware system for decision making. (This is important because this is one of the main things the user is interested to know)


  • Rules: If I receive a signal from X, I shall do Y. e.g. RFID, where a particular object is pre-identified.
  • Decision-tree: If this happens, followed by this, and finally this, I shall do this. 
  • Naive Bayes: Uses probability. If x, y, and z have happened, it is quite likely that m is the reason.
  • Hidden markov models: If x happened, followed by y, and finally z, 0.6, 0.2 and 0.2 are the probabilities of the a, b and c being the reason.


Based on the above decision systems, the authors produce a list of 8 explanation types to provide to the users

  • Inputs: What input sensors and information sources is the application using. e.g. GPS, Accelerometer, etc.
  • Outputs: What output options the application can produce. e.g the app can tell if you are standing or sitting or walking
  • What: What is the current state of the application. What was the previous state of the application. e.g. logging or not, sending data or not.
  • What If: This explanation allows the users to speculate the application would do given a set of user-set input values.
  • Why: Why did the application take a particular action.
  • Why-not: Why did the application not take a particular action
  • How to: How did the application reach the current state. How could an application reach a particular state.
  • Certainty: How sure is the application of the output values produced.


The aim of the toolkit, built on complex calculations and models, is to facilitate in answering these questions about an application. The way it is done is the following.\

  • Architecture: Modify the structure of each object storing any information to have an 'output' parameter which can be accessed at any point in time
  • Explainer: Hard code all possible set of rules, decision trees, etc. and have the explainer be able to just look at the outputs and generate explanations
  • Reducer: Take the output of the explainer and 'clean' it up. This could mean reducing multiple explanations for single state to one, or modifying results on the basis of the user is calling for.
  • Presenter: This takes care of elegantly presenting the explanations generated above so that they can be intuitive for the user.


Using this structure, the authors presented a toolkit which could be used to provide explanations, thus supporting intelligibility in Context Aware Apps.



TITLE: Toolkit to Support Intelligibility in Context-Aware Applications

NAME: Ed Anderson


AUTHORS: Brian Lim and Anind Dey at CMU HCII

Presented at UbiComp 2010




What is the subject: Effective feedback about automated decisions from context aware applications.

Why is it important: Increasingly, applications are more aware of context, and use it as a proxy for user intent to make decisions automatically on behalf of the user, lowering their cognitive load.

What is the problem: Users lose trust in applications that make context based decisions that are not intelligible (transparent).

What is the solution: A toolkit to automate the generation of explanations of decision model choices in context aware software.


NOTE: Rather than propose a hypothesis and then run an experiment to prove or disprove it, this paper is a survey of decision models and explanation types, followed by a description of the toolkit created to automatically generate explanations for those decisions.




The researchers identified four major decision models used in a selection of 114 context-aware applications gathered from submissions to the CHI, Ubicomp, and Pervasive conferences.


Rules: This is the most common type (more than the other three combined), with decisions based on simple logical rulesets, often in an "if <condition> then <action>", or a "this ID maps to this entity" format.


Decision Tree: The tree model considers one input factor at a time, branching through a series of decisions until a result is reached.  The tree is created by feeding the model a dataset and training it at each choice.


Naive Bayes: This is a probability based model, where all the input factors are treated as independent, and a decision is generated based on the simultaneous evaluation of all input factors.  Training is accomplished by adjusting the probabilities until satisfactory performance is achieved.


Hidden Markov Model: This extends the Naive Bayes model to a multi-step evaluation, where the result of one step is fed into the next step for several iterations.


Explanations fall into two categories, those which are independent of the type of decision model being used, and those that are dependent.


Independent: Input -- Possible inputs, Output -- possible outputs, What -- current system output, What if -- a possible system output given a set of inputs.


Dependent: Why -- why the current output was selected based on the inputs, Why Not -- why an alternative output was not selected based on the inputs, How to -- how can a certain output value be selected, Certainty -- how certain the selection of the output is




The goal of the toolkit is to make it easier for application developers to incorporate appropriate decision explanations.  To that end, the toolkit is flexible with regard to differing explanation types, combining explanations, generalizing across explanations and even being extended to support new types of explanations.


The toolkit consists of four functional units...


Explainer: generates the basic explanation based on the current model type.

Querier: gathers user input and system state history for explanations that require this data in addition to current inputs and output.

Reducer: reduces the detail or quantity of factors listed in cases where the basic explanation is too long or complex.

Presenter: chooses a mode of presentation for the data contained in the explanation, either as text, a visualization, or an interactive graphic.




IM Autostatus - Rules/Decision tree

Uses a combination of rules and a decision tree to automatically reply to IMs with an estimate of how long it will be before the recipient actually replies, based on their recent activity and other factors.  After the initial auto-generated reply, the sender can query the auto-responder about the various inputs and how they were processed to generate the time-to-reply estimate.


Mobile and Physical Activity Recognizer - Naive Bayes

Uses the accelerometer in an Android phone to generate a bar graph showing the probability that the user is sitting, standing or walking.


Home Activity Recognizer - Hidden Markov Model

Uses binary data from 14 sensors (on doors between rooms and various kitchen and bathroom surfaces/controls) to render a map with probability that the subject is engaging in a particular activity, e.g. sleeping, making breakfast.




The primary advantage of using the intelligibility toolkit is that it allows context aware application developers to more easily implement different types of explanations and see which are the most effective.  Further research should be conducted by building a variety of applications and incorporating various explanation methods, and studying the impact on intelligibility and resulting user trust.


Title: PreHeat: Controlling Home Heating

Using Occupancy Prediction

Name: Harrison Jones

Summary: This paper details research done in the field of intelligent learning thermostats. The paper centers on the "PreHeat" system. The PreHeat system is an intelligent learning heating system that uses sensors to determine if a room or house is occupied and then, together with historical data, determines if the house needs to be heated. If the house is occupied it sets the temperature to a pre-set value. If the house is not occupied but has a high probability of being occupied soon the system heats the home. The system and the predictive learning algorithm are the key elements of the research. The paper opens with a description of the field of intelligent/programmable thermostats. It explains that while programmable thermostats are effective in reducing energy consumption they are often not used effectively. The paper then turns to research done in the field by other researchers. It concludes that section saying that while that research was able to improve upon programmable thermostat they were not as effective as PreHeat. The paper then goes into the hardware setup of the experiment. Sensors and actuators were placed in several US and UK homes. In the US homes the system was capable of controlling the heating system and sensing occupancy via RFID. In the UK homes the system was capable of controlling the heating system and sensing occupancy via motion sensors. The system collected and stored historical occupancy data and then in real time used a predictive algorithm to determine if heating should start or end. The system was capable of keeping the time the house was at temp when occupied maximized (They attempted to minimize what they called "MissTime") and accurately predict when the house was going to be occupied. The system resulted in minimized "MissTime" and greater fuel savings.


Title: PreHeat: Controlling Home Heating Using Occupancy Prediction

Name: Yoo Mi Pyo


This paper discusses about a field study done in five homes, three in the U.S. and two in the U.K., using occupancy sensing and occupancy prediction. The study took place over a period of an average of 61 days per house to automatically control the home heating system more efficiently while removing the need for the users to program the thermostat schedules. In the three U.S. homes, each house was controlled by forced air heating systems and participants used RFID for occupancy sensing. Each tag sent its identity to the receiver every 5s when it was in range. In the homes in U.K., each house was controlled by radiators and an underfloor heating system per room; motion sensors that could detect motion by any occupant were added in each room.The occupancy sensors collected sensor data and ran the heating algorithm. When a space was occupied or the system failed to contact the server, it used the Occupied setpoint. Otherwise, it predicted when the space would be next occupied by matching the occupancy data from the current day against historical occupancy data. During Phase 0 (> 7 days), the system was run using the Scheduled algorithm where participants could adjust their setpoint temperatures or scheduled times if desired. During Phase 1(14 days), they ran a 14 day phase without Prediction so that they could collect some occupancy history data to work with the Prediction Algorithm. During Phase 2(61 days average), they used PreHeat's prediction algorithm and the Scheduled algorithm alternately each day. Overall, the houses in U.K. had little MissTime, but the PreHeat succeeded in decreasing this whereas the houses in U.S. only improved the MissTime by a large factor. The PreHeat system is able to dynamically heat day-by-day to more closely match the occupancy and provide better energy use and MissTime than a thermostat program.


Title: ThinSight: Integrated Optical Multi-touch Sensing through Thin Form-factor displays
Name: Spencer Border
ThinSight is an optical multi-touch sensing technology that is capable of being embedded behind a regular LCD. display. It is made up of an array of infrared emitters and detectors. These are encapsulated in a retro-reflective optosensor. These infrared emitters and detectors allow for the recognition of fingertips, hands, and the outlines of objects. The retro-reflective optosensor contains two components: a light emitter and an isolated light detector. The detection happens when a reflective object is placed in front of the display and some of the emitted light is reflected back. This reflected light is then detected. By placing retro-reflective optosensors in a uniform grid behind the display, it is possible to detect any number of objects on the display surface. The main advantages of ThinSight over other camera and projector based optical systems is that ThinSight is compact, has a low profile making interaction more practical, and the ability to be deployed in real-worl d settings.
IR light is a suitable choice for this technology for the following reasons:
- Though the IR light is attenuated by the LCD layers, portions of the signal are still detectable through the display.
- Though not limited to fingertips, a human fingertip reflects around 20% of incident IR light.
- IR light is not visible to the user and thus does not cause a distraction.
Due to the attenuation of the IR light when passed through the LCD display, a fingertip has to be at most around 10mm from the display to be detected. To date only touch/non-touch differentiation has been implemented but further research can prove that ThinSight can differentiate between touch and hover. This is due to the increasing intensity of the image blobs as they get closer to the screen. Another step in future research that will increase the capabilities of this system is to identify objects by placement of visual codes on the bottom of their surface.

Title: ThinSight: Integrated Optical Multi-touch Sensing Through Thin Form-factor Displays

Name: James O'Neill


This paper outlines multitouch hardware which uses retro-reflective infrared sensors to detect the presence of reflective objects on a display surface. It was written by researchers at Microsoft Research Cambridge in 2007 (which, for reference, was the year the first iPhone was released). The researchers seek to create multi-touch hardware which overcomes the disadvantages of existing technologies such as those based on FTIR and capacitive electrodes. They use a 2D matrix of Avago HSDL-9100 retro-reflective infrared sensors mounted behind an LCD. These are sensors that are typically used for sensing proximity of an object up to 100mm away, but because the LCD attenuates both the emitted and reflected infrared light, an obstruction must be approximately 10mm from the LCD screen to be detected. The detected signal is a low resolution monochrome “image” which can then be processed as multi-touch input via computer vision techniques. In their implementation there are three PCBs, each of which contains a 5x7 array of the IR sensors. They mount the three PCBs behind the LCD in a Dell laptop, taking measures to reduce IR interference. The PCBs cover only a subsection in the center of the LCD. They smooth the resulting signal in order for it to be displayed on the LCD as a 150x70 greyscale image in realtime. One of the key advantages to this type to multi-touch sensing is its thin form-factor, which incurs an aesthetic and portability advantage over camera-based technologies such as those that rely on FTIR as a sensing mechanism. In particular, these retro-reflective sensors do not require the use of a distant camera to capture reflections, and do not require a layer of material on top of an LCD screen, which can distort the image of an LCD display. Since the sensing of an obstruction is not dependent on its conductivity (as is the case with capacitive arrays), there is an opportunity to implement a tangible interface by sensing particular objects of any sort. The authors propose future work which involves computer vision techniques to identify touches or tangible input, active electronic identification via embedded IR emitters in particular objects, achieving improved resolution of sensing, framerate, and power consumption, and extending the hardware to cover an entire LCD.

Title: ThinSight: Integrated Optical Multi-touch Sensing through Thin Form-factor Displays

Name: Connor Wakamo


Researchers at Microsoft Research Cambridge developed a prototype hardware system for sensing multi-touch input on an LCD display.  Rather than apply a film or some other layer to the top of the LCD, these researchers instead put an array of IR emitters and detectors behind the LCD.  Even though the LCD panel is on top of the sensor array, enough IR signal is detectable through the panel to enable this to work.  With the sensor array behind the LCD, they were able to detect a fingertip at a distance of about 10mm from the LCD.  Upon gathering the data, the prototype interpolates an image of what it is detecting, and from there it uses computer vision image processing to determine what it sees.  The ThinSight prototype was able to detect not only fingertips but any IR-reflective object, and the researchers described the possibility of a future study to investigate identification of other objects using visual codes.  The researchers also believe that the ThinSight prototype is sensitive enough to detect the difference between hovering and touching the display, although they note that they did not attempt to implement this functionality.  The researchers conclude by stating that they believe technology like this will be integrated directly into displays, so that sensing pixels will be placed alongside RGB pixels on the panel.



Title: ThinSight: Integrated Optical Multi-touch Sensing through Thin Form-factor Displays

Name: Shubhshankar Murugaiah Sankachelvi


The paper discusses an optical sensing system called ThinSight which can be embedded behind a regular LCD screen. Traditional touch displays based on resistive and capacitive overlays don’t scale to multi touch efficiently. The basic component of ThinSight is a sensing element called retro reflective optosensor.It is made of a light emitter and a light detector and hence can both emit light and detect intensity of incident light. A 2D grid of these optosensors is constructed and placed behind a LCD screen. When a finger is placed in front of the screen, a fraction of the light emitted by the optosensors is reflected back and its intensity detected by the detector. The sensors operate in the infra-red spectrum since IR light would not interfere with the image displayed on the screen and is sufficiently reflected by a fingertip. Using bi-cubic interpolation the raw sensor data is scaled by a factor of 10 and smoothened by a Gaussian filter. The prototype developed was able to detect hands and fingers successfully. It can also detect any other object that reflects IR light. Further an object with an IR transmitter can be developed to transmit a code which can be detected by ThinSight.



Title:  ThinSight: Integrated Optical Multi-touch Sensing through Thin Form-factor Displays

Name: Kartik Agarwal


The paper presents the prototype for ThinSight,an optical sensing system integrated onto a regular LCD and is capable of detecting multi-touch by fingers and can detect physical objects and their shapes. The sensory data obtained is processed using computer vision techniques and a response to the interaction is generated. The advantage of ThinSight over conventional camera and projector systems is in its compactness and deployability in real world settings.


A comparison with single touch displays is drawn and their disadvantage is that they cannot scale robustly to support multi-touch and has led to purpose built sensing electronics like arrangement of capacitive electrodes and using camera to capture or process images of hands or objects on the display surface. Although these approaches provide rich data and are flexible in processing arbitrary objects, they limit portability and can be deployed to limited systems which is addressed by ThinSight, because of its form-factor can be extended to existing systems like laptops without much modification.


ThinSight consists of a 2D grid of retro-reflective optosensors placed behind a conventional LCD panel, which contents two elements: a light emitter and a light detector, when a reflective object is placed in front of it, the emitted light is reflected back and detected. The grid of sensors allows detection of any number of fingertips on the screen and the raw data is a low resolution monochromatic image of objects/fingers on the screen. Computer vision techniques are then applied to generate information about number and position of multiple touch points. Use of IR light is significant as (1)IR light is detectable through the display, unaffected by the image on the screen, (2)human fingertips are passable as reflective object because they reflect about 20% of incident IR light and (3)IR light is invisible and doesn't interfere with image on the screen. ThinSight can also detect any reflective object and can generate a silhouette to determine the object's location, orientation and shape. The object may include a barcode or a small infrared transmitter to increase the probability of detection.


The construction of the prototype is built using three identical custom made 70X50mm 4 layer PCBs each of which has a 7X5 grid of HSDL-9100 devices on a 10mm pitch. Thus a total of 105 devices cover 150X70mm area in the center of the LCD screen. The pitch ensures that at least one sensor would detect a fingertip even if the fingertip was placed in between four adjacent sensors. The prototype was attached to the back of a Dell laptop and the the hinges were reversed to have a tablet style construction.


In operation, fingertips appear as small blobs in the image, which increases in intensity as they get closer to the screen, thus it is possible to detect both touch and hover but the implementation currently focuses on establishing  a touch/no touch differentiation. A user may be able to apply zero force to touch and interact with the display. The pressure can be calculated by the increase in area and intensity of the fingertip blob upon touch detection. Ideas for further improvements include improving the resolution of sensing, framerate and power consumption and to expand the sensing area to cover the entire screen, which would be possible given the scalable nature of the hardware. Applications can be in the home to support media management and gaming using multi-touch interaction and physical objects.



Title: Low-Cost Multi-Touch Sensing through Frustrated Total Internal Reflection

Name: Ravi Karkar

Summary: The paper was written by Jefferson Han of Media Research Laboratory of NYU in 2005, before the (multitouch) touchscreen displays bombarded the market. The paper talks of a novel, low-cost method for multi touch sensing by using the phenomenon of frustrated total internal reflection. Now, the phenomenon itself is nothing new and infact has been around since 1960s. When light encounters an interface to a medium with a lower index of refraction (e.g. glass to air), the light becomes refracted to an extent which depends on its angle of incidence, and beyond a certain critical angle, it undergoes what is known as total internal reflection (TIR). However, another material at the interface can frustrate this total internal reflection, causing light to escape the waveguide there instead, resulting in the phenomenon of Frustrated total internal reflection (FTIR). The author used a single layer of acrylic sheet (16"x12"x0.25") and used an array of LEDs on the edge of the sheet to result in the TIR of the light of the LEDs. Whenever someone touches the surface, their finger causes the light to scatter out of the glass at the point of contact. A camera captures the image of the sheet and using machine vision software interprets the different touches and gestures.


Some features of the FTIR is that, it can distinguish hover from actual touch. It is highly accurate and inexpensive to construct. The main feature which sets it apart from traditional touch screens using capacitive and resistive displays is, it is easyily scalable to large displays. The drawbacks of this technology are it does not sense pressure and requires a significant amount of space behind the surface for the camera to pick up the entire image. In a setup using rear projection display a diffuser is introduced between the surface and the projector. There is a small gap in between the surface and diffuser to avoid frustrated interference. The downside with this setup is that it loses some of the scalability factor.


Finally, the author found a common vinyl material which can eliminate the use of a separate screen for display, thus fusing the display and the interactive surface. Some of the generic problems in such multiscreen displays are, a bit difficulty when the user has dry fingers (needs more pressure which leads to faster fatigue), oily finger prints on the screen over a long duration can cause interference which the vision softwares’ recognition, leading to difficulty in differentiating between live fingers and latent residues.



Title: System Guidelines for Co-located, Collaborative Work on a Tabletop Display 

Name: Chih-Pin Hsiao


The form factors of Table Top devices lead to many uncovered research questions regarding to collaborative interactions. Due to the tabletop technology was at the time to be mature when the authors wrote the paper, the question of what is the most appropriate configuration of tabletop system was becoming popular. The author investigated the existing tabletop systems and literatures. They categorize the tabletop system into four different classes: digital desks, workbenches, drafting tables, and collaboration tables. This paper focuses on the last one – collaboration tables. Then, through analyzing these literatures and existing tabletops, this paper presents eight design guidelines for supporting the activities around the tabletops:

(1) Natural interpersonal interaction: Interfering with the interactions can cause the breakdowns in collaboration. The authors also point out the bulky components under the table, such as cameras and projectors will force the users sit or stand awkwardly and impact the comfort level. 

(2) Transitions between activities,

(3) Transitions between personal and group work,

(4) Transitions between tabletop collaboration and external work,

(5) The use of physical objects,

(6) Accessing shared physical and digital objects,

(7) Flexible user arrangements

(8) Simultaneous user interactions

The results also indicate the future researches in the Table Top field. 





Title: Exploring End User Preferences for Location Obfuscation,Location-Based Services, and the Value of Location

Name: Madhura Bhave

Summary: The paper was written by A.J. Bernheim Brush, John Krumm and James Scott at Microsoft Research and presented at Ubicomp 2010.

There are a large number of location based services available today, such as providing navigational assistance or letting people share their location. These services use location data which may either be just the user’s current location or long term location data. While using location based services that use the long term location data, such as GPS logs, it is important to consider the privacy risks involved.


To understand the end users concerns about collection and sharing of their location data, the authors conducted a study involving 32 participants from 12 households. They collected GPS logs of the participants for 2 months. They were then shown visualizations of their own data. Hardly any participants were surprised as the data shown to them was what they expected to see. Those who were surprised could easily recall the relevant events. The authors demonstrated the effects of the different obfuscation techniques on the data and then interviewed the participants about which obfuscation technique they preferred most.  Mixing obfuscation technique was preferred by most participants followed by deleting and randomizing.


Of the 32 participants, 21 signed a consent form at the end of the interview to share their anonymized data collected during the study on a public website with a non-regular polygon removed around their house. When deciding with whom to share with, many participants always shared with the same recipient (e.g. public anonymous or academic/corporate) if they shared at all. However, participants did not realize the privacy interrelationships and differences were observed within a household as to whether to share and at what level.



TITLE: Exploring End User Preferences for Location Obfuscation, Location-Based Services, and the Value of Location 

NAME: Ed Anderson


AUTHORS: A.J. Bernheim Brush, John Krumm and James Scott at Microsoft Research

Presented at UbiComp 2010




What is the subject: Recording and publishing continuous user location data.

Why is this important: Location data can be used to provide valuable services.

What is the problem: Privacy issues make users resistant to allowing access to location data.

What is the solution: A survey of available techniques and if/how they are accepted suggests strategies that may be adopted to encourage users to share location data.




If subjects' continuous location data is collected over a period of time and then visualized for them, with the addition of several different methods of obfuscation, the subjects will be able to comprehend the privacy issues surrounding the publishing of their data, choose a preferred obfuscation method, decide whether or not they feel comfortable sharing the processed data, and put a monetary value on that data.




The locations of 32 participants from 12 households were recorded for two months using GPS trackers that they carried.  Participants could opt out at any point by turning the tracker off.  In exchange, they each received $600 of Microsoft software and $0.50 per day that they reported data.  After two months, the participants were shown a visualization of their own location data mapped around the two locations where they spent the most time, and were given a semi-structured interview and survey about their reaction to the data that had been collected, and the various factors around sharing that data.  This included showing the subjects illustrations of the various obfuscation methods that could be used to hide personally relevant aspects of their data, such as the exact location of their home.  After being shown their data and the obfuscation alternatives, the subjects were asked which obfuscation method they would prefer, if they would consent to have their own data published, and potentially what price they would put on their own data, depending on who would be paying to use it and what the potential use would be.




Most subjects found that the collected location data matched their own sense of their movement patterns.  The majority (21 out of 32) consented to share an anonymized version of their GPS data.  Preferences for the various obfuscation methods showed that subjects did at least partially understand the methods, but their choices were varied and tied to individual concerns.  Translating those individual concerns for privacy into intelligible settings on a UI was seen as a major issue.  Within households, there was limited understanding of how one user's choice to share at a certain level would affect the privacy of other housemates -- some members would choose to share more detailed information without realizing that this could compromise the privacy of others.  Although removing data points from the immediate area around a sensitive site such as the home was seen as the primary tool for safeguarding such a location, there are significant issues with this tactic because the location can still be inferred from the remaining data by analyzing movement patterns.  Creating better obfuscation methods to defeat this sort of analysis should be a topic of further research.




Title: Living in a Glass House:  A survey of private moments in the home

Name: Tarun Chakravorty


This paper discusses the privacy concerns that can arise due to advances in technologies such as sensors and recording devices which are integrated into homes. The authors conduct an anonymous survey which focuses on activities and habits that people do at home that they would not want to be recorded.

The paper begins by talking about how sensors are becoming pervasive within home environments, and the existing literature in this area. They say that although the benefit of having sensors in a home is clear, this is still a private and intimate environment with multiple people who may have differing views on whats socially acceptable and useful.
While talking about the existing literature around privacy concerns over emerging technologies, the authors point out that much work has been done towards guidelines or theoretical tools to build privacy sensitive Ubi comp systems and while the authors work is similar, they focus more on the privacy concerns due to the technologies in the home. Their focus is to understand activities that people do not want recorded.

Details on the survey:

The survey used online and offline recruiting methods. The questions were iterated using Amazon’s Mechanical Turk , which is an online crowd sourcing system. 7 iterations with 114 respondents were conducted before coming up with the final questions and scenarios.
The online survey was distributed through MTurk, the researchers personal networks and craiglist. The offline survey was distributed via postcards to places such as coffee shops and medical center waiting rooms. Anonymity was preserved in all cases.

A scenario was given in the survey which was intentionally less emotionally loaded and still generally understandable to layman people. People were asked to describe at least 3 regular habits that they would not want to be recorded.

A total of 489 people responded out of which 475 were considered valid. 468 provided their gender - 71.6% were female and 28.4% were male. Respondents were from a wide range of professions with 20.9% being students.

A total of 1433 activity descriptions were collected which were then analyzed into 19 high level and 75 sub categories. These were also analyzed based on the locations in the home. The activity type consisted of categories such as self appearance, cooking and eating, intimacy etc. A beakdown of all the activities is given in the paper and example excerpts from real users are also given describing the activities. It was observed that male respondents were more likely to report intimacy and media use while females reported self appearance and oral expressions.
For the locations, bedroom was most frequently mentioned and that there was usually a path or sequence involved such as bedroom to bathroom, bathroom to laundry, etc

The authors also discuss the limitations in that its not their intention to define privacy by generic categories. They rather show that relatively safe activities such as cooking can suddenly become sensitive activities by subtle change in context. They say that the next step is to explore how particular sensing modalities may affect people’s privacy concerns. They say that in conclusion, this work provides a better understanding of the types of private moments that occur int he home, which can help designers and developers be more mindful of the types of activities that need to remain private.


COMMENTS: Although the results of this survey seem pretty obvious, I think it is useful to know that there is actually proof of these "obvious"things. An interesting fact was that this survey left out people aged below 18 and non-US residents, and that they did not give any reason for it. I think while looking at the home, considering that children are an integral part of a home, they should have been included (albeit given less importance perhaps?). Also in the context of UbiComp I think its important to look at non-US families as well.





When Recommendation Meets Mobile: Contextual and Personalized Recommendation On The Go

Name: Saie Deshpande


This paper describes an effort to develop a context-aware and personalized system that does not require the user to type in any query. Based on a study conducted by the authors, they find that mobile searches involve queries that are short, local and personalized. This led them to propose an application that guesses the user’s intent and provides relevant results. It was deployed on Windows Phone 7 devices.

Conditional probability was used to guess what the user might be looking for. Based on the history of queries that a particular user tends to look up, they try to find entities belonging to a certain category of entities. The history of queries of other users is also taken into consideration. The suggested entity types and entities are relevant to user context i.e. his history of queries. It is also relevant to the current time and location of the user. They try to find out which categories the user looks up more often and which entities from that category. They do this for other n users and try to find out the value of similarity between the users. Higher the similarity, higher is the chance that this user will look for the same entities. A three-level similarity function is used:

  •          Entity-based similarity,
  •          Entity-type-based similarity and
  •          Entity-attribute based similarity

Based on the user’s intent, the appropriate entities are guessed, searched and filtered before they are presented to the user. These are then ranked depending on their relevance. The approach used includes 3 steps which are as follows:

  •          Entity crawling- It collects entities with attributes from the Web
  •          Entity extraction- It detects and recognizes entities from a query
  •          Entity ranking- It ranks the entities in a context- sensitive and personalized manner.

The evaluations of the application indicated that it achieved a high level of user satisfaction and the best user experience.                                                                                                                                                                                                                                                                                                                                                                             


Living in a Glass House: 

A Survey of Private Moments in the Home 


Name: Mebaa Kidane


This paper details the need for and the distribution and results of a survey by researchers that aimed to provide insight into the privacy concerns of typical Americans as they relate to the home and personal privacy. The article briefly discusses the increasing trend for everyday devices to have sensing capabilities, and it aims to provide a better understanding of the general attitude towards this trend and the privacy lines that people believe should not be crossed.


As stated before, a survey was used to gather the information. The final questions for the survey were derived after 7 iterations of 114 pilot respondents. The participants were compensated 10 cents per survey. The final survey was distributed via MTurk (Amazon's crowdsourcing system), personal networks, Craigslist, and postcards. The researchers received 475 valid survey responses. Over 70% of the respondents were female and they had varying occupations. As the survey as respondents to identify personal activities that would be considered too personal, embarrassing, etc. to be recorded, the researchers gained valuable information about where the line between security/convenience and privacy issues should be drawn.


The responses that including activity descriptions were organized various "high-level" (19 of them) categories and sub-categories (75 of them). In total, there were 1533 coded activities, which were also categorized based on the location within the home that the activities occur. As far as common activities are concerned, the most common deal with self-appearance, intimacy, cooking & eating, media use, and oral expressions. The most private location within the home according to the respondents is the bedroom.


Interestingly, the researchers note that the demographics and household structure of the respondents obviously affect the reported activity types. Though this is true, they do not offer any insight into the reasoning for surveying the people that were surveyed. Knowing the reported activities are affected so heavily on the demographics, why was there no explanation as to why certain demographics were left out of the survey (children, for example)?




Living in a Glass House: 

A Survey of Private Moments in the Home 

Name: Joseph Lin



This paper conducted an anonymous survey of 475 participants focusing on any activities or habits that they do around the house that they would not like to be recorded. The purpose of this survey was to bring to light potential issues regarding privacy as the prevalence of sensors and recording devices increases. The results were collected and aggregated into several major categories.


Data was collected using both online and offline methods. Amazon's Mechanical Turk, Craigslist and physical mail were used. In the surveys, the participants were asked to imagine a scenario where sensors and recording devices were placed within their home for security reasons. The goal of this particular statement was to provide a neutral emotional reaction to the idea of recording devices. 


The survey collected 1443 activity descriptions that users would not want recorded. The main category of activities that participants do not want recorded were related to self-appearance. This consisted of situations where the participant did not look at their best such as being partially dressed. The other major category was that of intimacy. However, there were many situations discovered through the survey where participants took a scenario where recording was typically permissible but then gave a specific context of a situation where it wasn't. An example was binge eating or picking food dropped on the counter. The article also brings to light potential scenarios where recording naturally happens, but users may bring to light specific situations where it is not appropriate to be recorded. An example would be shirtless players in front of the Kinect. The authors feel this paper brings to light some situations where recording devices may not be appropriate for recording within the home. Their next step would be to explore how various data sensing modalities such as how the data is processed or their retention time influences the user's privacy concerns.


From Spaces to Places

Emerging Contexts in Mobile Privacy

Name: Chris Bayruns


The concept of what construes mobile privacy can be a somewhat subjective and difficult set of perspectives to define. The authors give by an example of the problem asking self proclaimed healthy eaters the question if they prefer to snack on a piece of fruit or on a candy bar. In actuality the responders say they prefer the fruit , but actually go for the candy bar. So the question posed by Mancini et al is to answer what construes privacy in mobile devices, but more important to this paper is how they go about surveying users of mobile and communication devices. They chose to investigate privacy in mobile devices in a "qualitative exploratory" study.
The methods they undertake in the study are: (Old methods) Experience sampling, Experience sampling to memory triggering, Deferred contextual interviews, and then a qualitative study on the practices of mobile Facebook practices. In their method of the latter they advertized the study through the use of mailing lists, word of mouth and volunteers to document the practices of Facebook users. Personal interviews with two Facebook user episodes were described. The authors reviewed all of the results of the studies on their Facebook research and identified five types of personal boundaries that exist in Facebook land: 1. Personal Policy Boundaries, 2.Inside Knowledge Boundaries, 3. Etiquette Boundaries, 4. Proxemic Boundaries, and 5.Aggregation Boundaries.
The resulting conclusions from the studies presented by the authors is that there exists a socio-cultural notion of "place" as defined by emerging socio-cultural knowledge, functions,relations and rules. The authors also made clear the distinctions between "space" and "place". In the end of the paper they discuss future research needed to help develop designs for mobile Ubicomp technology.


Title: Living in a Glass House: A Survey of Private Moments in the Home

Name: Dan Huang



The home is often considered a private and intimate place, with different people who may have competing priorities and tolerances for what is acceptable or useful. This article discusses what activities and habits people wouldn't recorded by the advances in sensor technology, and provides suggestions for the design of UbiComp systems in the sensing space.


Privacy concerns over emerging technologies isn't new. Back in 1890 when photographs began emerging in newspapers, the private life of an individual or entity could easily become public knowledge. The situation is even more convoluted in today's technology environment. The research team from the University of Washington, Intel Labs Seattle, Microsoft Research, and Lab126 set out to investigate privacy concerns around sensing and recording technologies in the home.


An anonymous survey of 475 people revealed activities and habits people do at home that they would not want to be recorded. The analysis focused on the activity/task and the location in the home. 489 surveys were received, which 475 were valid (71.6% female, 28.4% male). There were 1433 activities that respondents reported doing in their homes that they would not want recorded. The activities were broken down into 19 high-level types and 75 sub-categories. The most frequently reported activity that respondents do in their home that would not want to be recorded fell into the categories of appearance, intimacy, cooking, eating, media use, and oral expression. Further investigation of the respondent's demographic information using a chi-square test of independence revealed that the relationship between gender and activity type was significant. On the subject of location, the bedroom was the most frequently mentioned place (33.7%).


While the results seem obvious, the researchers believe that privacy concerns from these sensing technologies have not been previously studied in detail. The point the researchers are trying to make is that activities normally viewed as "safe" could suddenly become a sensitive activity by subtle changes in context. The next step is to explore how sensing methods affect people's concerns about privacy.


Title: Who's your best friend? Target privacy attacks in location-sharing social networks.

Name: Carter Templeton:



This paper centers around an experiment designed to determine which user(s) in a social network are more likely to reveal information on their location, and if it is possible to determine which of a user's friends knows the most about that user's whereabouts. The experimenters did this by creating a small social network and identifying connections to the target user that have a low level of privacy, i.e. where a large amount of personal data is shared.


In an attack against a targeted user, the attacker would identify a "weak link" in the target's list of friends - theoretically any friend who the target shares any location information with, but ideally one who the target shares as much information as possible with. Weak likes are identified based on how many friends the weak link has in the network, or how many friends in common the weak link has with the target. The attacker would then befriend the weak link, thus becoming a friend of a friend of the target on the network, and (likely, but not necessarily) having greater access to the target's information.


The study found that users more "central" in the network (that is, with a greater amount of connections) are more likely to share their location with others, making them ideal targets for an attacker. The more information the weak link shares with the target, the more the target trusts the weak link and shares their info in return. As a result, the attacker can gain much information on the target by seeing what their "friend's friend" has shared with their new "friend".


The implications of this can be unsettling for privacy advocates. Something you share with your friends may be available for viewing by people who not only are not your friend, but people who are looking to exploit your information.

Title: Living in a Glass House: A Survey of Private Moments in the Home

Name: Connor Wakamo



Researchers from the University of Washington and Intel Labs Seattle (two had since left Intel for Microsoft Research and Lab126) conducted an anonymous survey to determine what in-home activities people do not want recorded.  They received 475 responses: some were received from Amazon's Mechanical Turk, while others were received using offline recruiting methods such as leaving blank surveys and envelopes in public places like coffee shops. The vast majority of responses (405 of 475) came from Mechanical Turk.  71.6% of respondents were female and 28.4% were male.


For the purposes of the survey, the researchers had respondents imagine a scenario in which their home had security cameras and microphones in every room except the bathrooms.  (They chose this scenario because it was more easily understandable to laypeople.)  They asked respondents to describe at least three regular activities they would not want recorded and where in the home they did these activities.


The two largest categories of activities were self-appearance and intimacy, with 22.5% and 18.3% of activities coming from these categories respectively. The only other categories to score 5.0% or above were, in descending order, cooking & eating (9.3%), media use (8.3%), oral expressions (8.0%), socially awkward acts (5.9%), and personal hygiene (5.0%).


Of these categories, men were more likely than women to report activities in intimacy or media use.  Women on the other hand were more likely to report activities in self-appearance and oral expressions.  Of locations where activities occur, the bedroom was the most frequently mentioned location.  Other common locations were anywhere in the home and the living room.


Understanding How Visual Representations of Location Feeds Affect End-User Privacy Concerns

Name: Joseph Lin



This paper aims to understand how visual representations of historical location information influence a user's perception of privacy. The three visualizations that were used in this study were map based, time based and text based.


The key features of each visualization are described here. In a text based representation, each row shows when the user arrived at a location, where they were and how long did they stay. The time moves forward down the list. The spatial information about each location is not represented visually and each row visually looks no different than the next. In a map based visualization,  halos are used to represent physical locations on a map and clicking on a halo reveals information about when a user arrived and their duration there. Finally, the third visualization studied is the time-based visualization. This is a colored-block timeline showing when a user arrived and left a place. 


The experiment consisted of giving participants mobile phones to carry around as their primary phones for a two week study. These phones logged location data using GPS or Wi-Fi positioning over this period. The raw location data was then converted to both specific geographic labels (the nearest address) as well as semantic labels (i.e Starbucks). This data was also supplemented with user supplied labels. Furthermore, the user participated in validating those labels. 


At the end of the study, the users were asked to look at the three visualizations their historical location data and decide which of the them they were most comfortable sharing for each relationship group. The relationship groups were family, close friends, acquaintances and supervisors. Based on the feedback, participants mainly felt the time-based visualizations were too privacy-invasive. This was because they felt the location durations would lead to incorrect inferences about what was happening at a particular location. Users also preferred to share lower granularity labels when there are power dynamics involved, but more descriptive labels when sharing with close friends. Another interesting result is that their choice of markers to use in a map-based visualization were not consistent with their location labels. They wanted to combine general semantic labels with pushpin markers to give the appearance of visually moving from one location to the next. The halo markers would barely wobble after movement as they encompassed a wide area.


In summary,  all the visualizations were isomorphic. The same data could be gathered from any visualization. However, the time based visualization seemed to suggest or incorrectly suggest what activities a user was doing a a place and most participants did not like that. The greatest discomfort was the suggestion that these apps have of what a participant was doing a a particular location. The work in this paper suggests that participants should possibly have the power in a location sharing app to tag their presence at location with a particular activity.


Title: Living in a Glass House: A Survey of Private Moments in the Home

Name: Ravi Karkar

Summary: omw



Title: Living in a Glass House: A Survey of Private Moments in the Home

Published in Ubicomp ’11

Name: Shaun Wallace




As more and more sensing technologies are introduced into the market, and more specifically those that are designed for in-home monitoring of some kind, privacy becomes a major concern. This article was written to explain research that was done to find what people would not want recorded within their homes. The specific study focussed primarily on privacy concerns around sensing and recording technologies in the home and therefore differed from other similar studies in that prior research was more concerned with proposing guidelines and principles to build privacy-sensitive Ubicomp systems. This particular study investigated the types of activities that occur in the home that could cause concern if they were recorded.


To gather data from individuals, the researchers surveyed users with the aid of online and offline recruiting methods. Of the surveys released they gathered information from 475 participants and gathered 1433 activity descriptions that respondents reported. Those activities were analyzed and a category scheme was developed which placed activities within 19 high-level types and 75 sub-category types.


The research showed that the most frequent activities that respondents did in their homes that they would not want recorded fell into the categories of self-appearance, intimacy, cooking & eating, media use, and oral expressions.


The outcome of the study as stated by the researchers is as follows: “the work has provided a better understanding of the types of private moments that occur in the home, which can help developers and designers of in-home sensing systems be more mindful of the types of activities that need to remain private”.


Title :Living in a Glass House: A Survey of Private Moments in the Home

Name: Shubhshankar M.S 


The paper tries to explore how the use of sensing mechanisms affect the privacy of an individual, an understanding of which would allow development of better Ubicomp systems.

A survey was carried out in which the participants were given a scenario of how sensors may be embedded in their homes and were asked to describe at least three regular habits they wouldn’t want to be recorded. A total of 475 valid surveys were received among which the majority (405) were received from MTurk, a crowd sourcing system. The rest came from physical postcards, craigslist and personal networks.


A total of 1433 activity descriptions were obtained which are discussed with regard to the type of the activity and the location.

Activity Types :- As expected Self-appearance(22.5%) and intimacy(18.3%) ranked highest in the list of activities followed by cooking, media use and oral expressions. It also shows significant correlation between gender and activity type. Males were likely to report activities in Intimacy and Media use while females reported activities in Self appearance and Oral Expressions.Bedroom was the most frequently mentioned location followed by living room which corresponded to activities such as media use.


Though most of the responses are obvious in nature, the paper carries out a comprehensive survey about the privacy concerns in homes with ubiquitous systems. 



Title: Privacy Attacks in Location-sharing Social Networks

Name: Patrick Mize


This paper was written by three individuals from the University of Madeira, another from Carnegie Mellon, one from Coventry University and a last from the University of Bath. It was published in UbiComp 2011. The paper was written to find a general strategy for finding the best “target” in an online social network to get his or her location information. It was assumed that being friends with the target was “being too close”; therefore, the writers also sought a friend of the target to befriend to get the target’s location information. Therefore, two main questions were: given a group of users and their social graph, is it feasible to determine which is likely to reveal the most about their location and who among them knows the most about his or her location.

The hypotheses in the study were: individuals more central in their social graph are likely to reveal the most about their location, the individual’s friend who has the most friends himself has higher probability of knowing more about the individual, and the individual’s friend who has the most friends in common knows most about the target. The study used Locaccino, a real-time location sharing application on Facebook, and collected data on more than 300 users. The results they found confirmed their hypotheses, although the writers acknowledge that there may be other factors that can affect location sharing that the study did not take in to account. While most of the paper is written as a way to acquire location from an individual in a nefarious way, the purpose was to find a way of protecting users against such attacks. The protection that they listed included more stringent sharing settings (i.e. only to immediate friends), limiting users on how often their update location information, and informing the individual if anyone is requesting location information too often through a pull-base location-sharing model.


Title: Living in a Glass House

Name: Yoo Mi Pyo


Summary: This paper shows the results of an anonymous survey with 475 out of 489 participants who are over 18 years old, focusing on activities and habits that people do at home that they would not want to be recorded. Privacy concerns around sensing and recording technologies in the home was focused on this survey since there's a recent work by Srinivasan et al has shown that private activities in the home can be inferred by eavesdropping on the wireless transmissions of data.

The survey was conducted an anonymous survey using online and offline, distributed the final version of the survey online through MTurk. The survey was available to U.S. residents, and posted the link to nine U.S. cities on Craigslist.org and sent emails to like forward to others who were not students or engaged in technology-related jobs. "at least three regular habits" that participants do at home that they would not want to have recorded, and where in the home they do the activities. As a result, people tend to mind the types of activities like self-appearance, cooking & eating, intimacy, media use, and oral expressions in the bedroom(79.7%), anywhere in the home(16.2%), or living room(13.9%), etc. This study may look obvious, however, this can help designers and developers to better understanding of the types of activities householders would like to  keep private.



Title: Living in a Glass House

Name: Donovan Hatch



This paper was written by Microsoft Research, Intel Labs Seattle, University of Washington, and Lab126.  They wanted to research how people felt about having everything in the house recorded by microphone and video camera except for in the bathrooms. To do this they created an anonymous poll and issued it via Amazon’s MTURK service, post card, and email. They received a total of 485 responses and used 475 that they considered to be real responses of valid candidates. 

                The overall response to the survey was that they did not want to be recorded inside of their house.  The top categories picked for where they did not want to be recorded were bedroom (33.7%), anywhere in the house (16.2%), and the living room (13.9%).  Of the different areas listing where they would not like to be recorded and why they wouldn’t want to be recorded there, there were usually activities associated in that location that also took place inside of the bedroom.  The top activities that they did not want being recorded included activities involving self-appearance (22.5%), intimacy (18.3%), cooking and eating (9.3%), and media use (8.3%).  




Title: Living in a Glass House: A Survey of Private Moments in the Home

Name: Jon Pelc

Summary: The authors of this paper sought to gain some more insight into what privacy people expect and anticipate in the home environment.  This is highly relevant as technology allows for monitoring and recording data is becoming more prevalent in living environments.  As we have seen, previous work has discussed privacy concerns with ubiquitous computing systems the authors focused exclusively on privacy in the home.


They did this by conducting an anonymous survey recruiting participants through Amazon’s Mechanical Turk, physical postcards in public environments, and craigslist.org, and the researcher’s personal networks.  Anonymity was important to ensure that participants would be willing to share accurate data.  The survey questions consisted of scenarios of how sensors and recording devices could be embedded in their home and asked the participant to respond.  In addition the survey asked them simple questions such as to name three regular habits that occur in the home they would not want to be recorded.  There were a total of 489 responses of which 475 were used in the final analysis.


The results were broken down as by types of activities and location of activities.  The major types of activities (with an example) respondents didn’t want recorded were self-appearance (walking around partially dressed), intimacy (sexual activities), cooking & eating (sneaking junk food), media use (watching bad tv), and oral expressions (expressing frustration about a family member or friend).   Others included hygiene activities (showering), socially awkward acts (picking your nose), and physical activity (yoga).  They found a significant relation between gender and type.  Males were more likely to report intimacy and media use activities while females were more likely to express self-appearance and oral expression activities.


The overwhelming majority of participants that mentioned a location in their activities mentioned the bedroom as a place where they would not like to be recorded.  The second most common response was “anywhere in the home” involving nose picking, arguments, private conversations etc.  Others involved a specific path such as from the bathroom to the bedroom after showering.


The authors hoped that these results would better serve to become more aware of the activities their users consider private because they may not be aware.  Their next step will be to explore how different sensing modalities may affect people’s privacy concerns.


Title: Understanding How Visual Representations of Location Feeds Affect End-User Privacy Concerns
Name: Kevin Kraemer

This paper involves tracking users' historical locations, visualizing the data in three different ways, and examining perceptions of privacy issues between the different visualization methods. The three visualization techniques used were text-based, map-based, and time-based. The text-based view simply showed a list of locations visited with the arrival time and length of stay for each. The map-based view showed a map with visited locations represented by halos with the user's true location being somewhere within the ring's boundaries. Clicking on a halo would pop-up the arrival time and duration. The halos also showed different properties such as frequency of visit and time since visit using other visual properties like transparency. The time-based view shows a timeline of color-coded blocks with each block representing a different location and the length of the block showing the duration. Each of the three visualization types contained the same data. By varying the visualization type, labeling type (geographic vs. semantic), and labeling specificity (general vs. specific), 18 different visualizations were created.

For the study, twelve people were recruited and given phones to use over a two-week period. These phones collected GPS and Wi-Fi positioning data, and the data were visualized using the three methods described above. One problem found when gathering the data was inaccuracies in labeling locations. Geographical labels could be extrapolated reliably, but semantic labels were less accurate. This problem was solved by having the subjects validate and correct the labels halfway through and at the end of the observation period.

After the end of the observation period, each subject was shown his or her location data in each of the 18 visualization types and asked which view they would be most comfortable sharing for four different groups of people: family, close friends, acquaintances, and supervisors. The subjects unanimously disliked the time-based visualizations, feeling that they were too privacy-intensive. In general, subjects were less comfortable with sharing the temporal attributes of the data (arrival times and duration). A majority of the subjects also preferred visualizations with inconsistent label-and-marker combinations; for example, they would like specific location labels with more general map markers, and vice versa. With regards to sharing the data with different groups, subjects consistently chose more general labels for close friends and supervisors.

Ultimately, the authors conclude that location-sharing applications need to be flexible and allow users to control who can see their location data and how that data is visualized, because different visualizations can change how the data is perceived by others.


Title: On the Limitations of Query Obfuscation Techniques for Location Privacy
Name: Ramik Sadana

The aim of the paper is to measure the effectiveness of obfuscation techniques for location privacy. Obfuscation techniques are methods used for anonymizing the user data, so that no real information about the person the data is taken from is available. This is important because all the Location based services that we use, like maps, deal finder, etc. send the GPS coordinates to the server which then parses it for useful information and sends back a response. In this process, the server can store the location information, and over a period have enough information about our eating, shopping and spending habits that the data gains a lot of commercial value. More so, since the data is very specific about the person, it is private information and can be used in much more negative, targeted ways.


To counter the possibility of the service knowing the exact location, the data is run through a obfuscation method which cleans it in any particular way. This could be introducing random noise which taken the precision off the coordinates. Another similar method takes a coordinate and hard removes the precision of the floating values, so that the coordinates now refer to a broader region of a square mile instead of a couple of yards. The problem with these methods is that since the original data is changed, if the information required depends highly on the precision of the value, e.g. in case of maps, then the techniques fail to help. 


To counter these issues, a technique called k-anonymity is used and this is the technique discussed in the paper. In this technique, if the user needs information on the basis on location x, the system would generate k different queries of which k - 1 would be queries using locations similar to x. This way, the LBS receives multiple query requests from the same system, and each of the k queries could be the real location of the user. Since there is no way to effectively tell (discussed later), this adds anonymity. In case of driving direction request, the user needs to put in the start and end location of the trip. The system then calculates k-1 other trips between k-1 different source and destination points which mimic the main trip. 


The authors look at one such system, called SybilQuery, which generates the other k-1 queries. The aim of the research is to understand whether it is possible for the server to predict the right location of the user from the set of k locations. The authors look at two separate cases and present results.


1. Assuming the server has knowledge of user's previous trips, using machine learning on those trips, the server can identify the real trip 94% of times, with only 2% of false positives.

2. If the server does not have previous trip information, by accumulating a set of 5 trips, the authors use a technique which increases trip identification probability from 20% to 40%.



Title: Whoʼs Your Best Friend? Targeted Privacy Attacks In Location-sharing Social Networks

Name: Cheng-Pin Lee



This paper is a location privacy paper written for the 2011 Ubicomp Conference in Beijing, China. It was written by three members of the 

University of Madeira, two members from Carnegie Mellon, and two British members from Conventry University and University of Bath.


The authors were researching how best to go about a privacy attack using Locaccino and Facebook check-ins. They had a sample group of 340 people and created statistics such as, but not limited to, how many friends a person has (degree centrality) in a social graph of all their friends, a person's willingness to share his/her location, and a person's number of people in which other friends share their location with him or her. The statistics were gathered using a person's friends list and the privacy settings the users set. Using these statistics, they ran statistical analysis to come up with correlations in order to determine a worthy "target" of a privacy attack and a "weak link" in which to obtain information about the "target" from. 


The authors hypothesized that people who had lots of friends and trusted many friends were subject to sharing his or her location information with more people. This was proven correct and helped determine who would make a good "target" for a privacy attack. The next two hypotheses dealt with how to go about attacking the "target." They hypothesized that the targetʼs friend with the highest degree has higher probability of knowing more about the target. This was also proven correct by studying the relationship of the number of friends a person has and the trust rank of a person on a "target's" list. A trust rank is a list sorted according to how much a subject trusts another subject. And the final hypothesis stated that the targetʼs friend with most common ties with the target knows most about the target. This was also proven correct through analysis of a subject's trust rank and mutual rank, or the statistic of how many friends a person has in common with the "target." 


With the proven hypotheses and the statistics the authors gathered, the authors could create a hypothetical scenario for a privacy attack. They found that it was possible to "track" a target indirectly through a mutual friend of a "target" that they deemed a "weak link" and also one that scored high based on the analysis from the hypotheses. The authors proved that one could be "targeted" without the "target's" awareness. The thing to take away from the study is to use the information to secure one's privacy better as well as make suggestions for automatic settings that could facilitate one's use of this technology and information.


Title: At the Flick of a Switch: Detecting and Classifying 

Unique Electrical Events on the Residential Power Line

Name: Joseph Lin

Summary: In this paper, folks from GVU at Georgia Tech look into activity sensing using the power line infrastructure. A small device is connected to a computer via USB and this device is connected to a power outlet. The device is some kind of oscilloscope that analyzes data coming from the power line and feeds it back to the computer. The goal here is to sense changes in noise along the power lines to detect when a particular device is switched on or off as well as sense the ambient noise that a device generates when it is turned on.


In this project, machine learning was used to train the device learning the various signals generated within the house. The report states that some devices are more detectable than others depending on the amount of power they consumed. In general, by training the devices on more samples, the accuracy improved up to about 80-90% accurate.


The strengths of this approach are being able to detect when certain household activities are occurring. The data can be studied for applications in household monitoring, power conservation, entertainment and more. The device does not require additional infrastructure and is cheap to deploy. In addition, only one sensor is required, however, this can potentially be a vulnerability as a single point of failure. The device does require machine learning training data so that can also be a disadvantage for configurations that are constantly changing from day to day or if the testers are not able to access all the electrical equipment in the area.


Future applications of this technology may be implemented in commercial or large office buildings. This may complicate or make it more difficult to accomplish as now there are more events happening and more noisy systems such as a central HVAC that interferes with the readings. 



Title: At the Flick of a Switch: Detecting and Classifying Unique Electrical Events on the Residential Power Line

Name: Oriol Collell Martin

Summary: This paper describes an activity recognition sensing system that uses the power line to detect activity in a home. They claim that their approach has two main benefits over existing approaches:
   1. They use an already existing infrastructre (the power line) to perform the sensing, which is chepar than building a whole new infrastructure.
   2. They only require a device to be plugged into the power line. In contrast, other approaches that also use an existing infrastructre (such as water pipes), require a more complex deployment.

Activity sensing is a popular area in Ubicomp which aims at detecting where activity is being held at a home. The purposes are diverse, they go from adjusting the HVAC system of a home by maintaining the appropite level of comfort only on the rooms that are occupied to recommending ways of saving energy.

The goal of the system is to infer the activity that is being held in a home by sensing changes on electrical noise in the power line over the time. For doing so they, first, profile each device and switch in isolation by turning the rest of devices off and learning the noise that it produces. Then, with a machine learning approach, they are able to predict what devices/switches are on by sensing the electrical noise. Knowing the devices that are on and off and how that changes over time, they can infer where people is in the home and what
they are doing.

For they approach to work, however, each device needs to have its own electical signture, so that it can be differentiated from others. To see that they profile different devices and compre them to see if they are different enough. Specifically they capture the electrical noise corresponding to three events: turning on a device, turning off a device and maintaining a device in on state. It results that different type of devices produce noise on a different subset of events, moreover, each device produces different noise in the same events.

Their system consist simply on a device plugged to an electrical outlet and to a computer via USB. The device consists some hardware components, such as an oscillator, an is able to both profile the different devices and then apply a machine learning algorithm to predict the on and off devices. They profiling is done via a user interface where the house owner can survey the different devices on the home.

The authors performed a user study in which they installed the system in 6 homes. In the first home they conducted a 6-week study while in the others the study lasted 1 week. The idea was to use the short studies to confirm the results obtained in the long study.
During the study, they installed the data collector device in each home and collected and labelled data several times per week. The data collection consisted on profiling each of a set of devices by labeling the electrical events produced. With that, they measured the accuary of the classification algorithm by counting the number of events that were properly detected.
The results of the study in the different houses show the classification algorithm has an average accuarcy of 85% to 90%. The also note two important observations: In the case of the long study, by using only samples collected during the first week, the accuarcy decreases very slowly during the time (reaching 79% in the 6th week); and as more samples are available the accuarcy increaes, however, having around 2 samples per device is enough to deliver good accuarcy, this means that the training process time can be fairly short.

Finally they comment on some limitations of their approach and how it could be improved. They comment, for instance, that the system should be able to detect and adapt to random noise events or that the system would not properly detect an event where two devices/switches are turned on simultaneously. They also envision the possibility of deploying the system in an office environment, but their big space and full of people nature offers difficulties for the approach.

Title: At the Flick of a Switch: Detecting and Classifying Unique Electrical Events on the Residential Power Line 

Name: Akshay Goil 



This paper presents an approach that uses a single plug-in sensor to detect a variety of electrical events throughout a home as a solution to support activity detection and context-aware applications within the home. Since the proposed solution takes advantage of the residential power line and only requires the installation of a single, plug-in module that connects to an embedded or personal computer, it proves to be an inexpensive and easy to deploy solution that is truly ubiquitous in nature.


The computer records and analyzes electrical noise on the power line caused by the switching of significant electrical loads. Machine learning techniques applied to these patterns identify when unique events occur. Examples include human-initiated events, such as turning on or off a specific light switch or plugging in a CD player, as well as automatic events, such as a compressor or fan of an HVAC system turning on or off under the control of a thermostat. By observing actuation of certain electrical devices, the location and activity of people in the space can be inferred and used for applications that rely on this contextual information. For example, detecting that a light switch was turned on can be an indication that someone entered a room, and thus an application could adjust the thermostat to make that room more comfortable. The system also additionally provides a low-cost solution for monitoring energy usage within the household.


This system is built using 3 primary components: a custom powerline interface which can connect to any electrical outlet in the home, an oscilloscope which connects to the powerline interface, and a pc which connects to the oscilloscope via USB. The system aims to detect three types of electrical noise in the powerlines: the noise associated with switching a device on, the noise associated with switching a device off, and the noise associated with continuous use of the device. For each electrical device of interest, the authors singled out the device and within this controlled environment, visually observed and collected noise signatures for turning the device on, turning it off, and its stable on state. For many of the devices tested, the authors found that the actions showed not only strong, but also consistently reproducible signatures. These signatures formed the raw data for the machine learning techniques implemented.


The 10 bits of resolution provided by the custom data collection device limited the range of detection. Most loads drawing less than 0.25 amps were practically undetectable, whereas loads above that amount produced very prominent electrical noise (transient and/or continuous). Additionally, the sampling and processing latency of the device affected the required delay between switching a given device from one state to another, requiring a 500 ms. period between consecutive actions.


To evaluate the feasibility and performance of the system, the system was put through a six-week test in a real-world home environment where the classification accuracy of various electrical events was determined. The system was then deployed in five other homes for a one week period to reproduce the results from the first home. Results indicated that the authors’ presented approach could learn and classify various electrical events with accuracies ranging from 85-90%.

Title: Detecting Human Movement by Differential Air Pressure Sensing in HVAC System Ductwork: An Exploration in Infrastructure Mediated Sensing 

Name: Akshay Goil 



Before I begin, it is worth noting that the authors of this paper also contributed to the previous paper on infrastructure mediated sensing, ‘At the Flick of a Switch: Detecting and Classifying Unique Electrical Events on the Residential Power Line’. This paper presents an approach for whole-house gross movement and room transition detection through sensing at only one point in the home. Unlike the previous solution which took advantage of existing residential power lines, this approach leverages the existing ductwork infrastructure of central heating, ventilation, and air conditioning (HVAC) systems found in many homes.


Disruptions in airflow, caused by human inter-room movement, result in static pressure changes in the HVAC air handler unit. This is particularly apparent for room-to-room transitions and the opening and closing of doors involving full or partial blockage of doorways and thresholds. The system detects and records this pressure variation from sensors mounted on the air filter and classifies where movements are occurring within the house. Similar to the previously discussed approach, this method requires the installation of a single sensing unit connected to an embedded or personal computer that performs the classification function. As such, this method also proves to be an inexpensive and easy to deploy solution that is truly ubiquitous in nature.


The HVAC’s air filter is instrumented with five bi-directional pressure sensor units to allow for easy installation in standard HVAC units. These sensors do not interfere with the air filter or HVAC and capture the pressure differential across the filter in the air handler chamber. The magnitude of the pressure change across all the sensors is used to identify unique disruptions in airflow in the physical space. Machine learning techniques then classify these disruption signatures allowing them to be re-detected and classified later.


When a door is closed, there is first an initial abrupt change in static pressure followed by persistent change until the door is reopened. After opening the door, the static pressure gradually drops to its previous state. Unlike the door events, the change in pressure when a person passes through a doorway is short-lived. There is a slight change in the static pressure and then the pressure settles back to its original state. This effect is dependent on the location of the supply and return vents relative to the doorway and the ratio of the size of the person to the size of the doorway. The sensitivity of the pressure sensor units make it possible to detect airflow reaching the sensors for door transition events even when the HVAC is switched off. Under these circumstances, there is no static pressure build-up in the air handler. Instead, the pressure is equal to the atmospheric pressure of approximately 1 bar. Any significant airflow generated in the conditioned space is guided through either the supply or return ducts and eventually reaches the sensor units on the filters. The pressure values from both sides of the filter are then used to help determine where the airflow originated from.


The authors conducted feasibility experiments to determine if and how often they could detect transition movements (adults walking through doorways and the opening and closing of doors) and how accurately they could classify unique transition events. A total of six different HVAC units and spaces were evaluated for a period ranging from three to four weeks. Using this method of infrastructure mediated sensing, it was determined that one can classify unique transition events with up to 75-80% accuracy. Although this is a relatively high accuracy level, when compared to the previously discussed method which used electrical events on the residential power line, this method which leverages the existing ductwork infrastructure of central heating, ventilation, and air conditioning (HVAC) systems found in many homes is approximately 10% less accurate at detecting human presence. Nevertheless, the methods of detection make the two approaches completely independent of one another. The previously discussed approach required a conscious physical action such as switching on a light bulb to detect the location or presence of an individual within a given area. On the other hand, this approach does not require a conscious physical action of that sort and can detect human movement as the individual simply walks through a doorway.

Title: A Logitudinal Study of Pressure Sensing to Infer Real-World Water Usage Events in the Home

Name: Jeremy Duvall


This study is from the authors of the previous work: HydroSense, which is a pressure water pressure sensing system designed to detect and infer water usage. Pressure waves propogate throughout a homes plumbing system when a specific fixture is in use, and HydroSense (and it's improved predecessor described in this paper) leverages that fact. This paper is a feasibility study designed to determine if pressure based sensing works in practice.


To answer this question, an improved system was installed in three homes and two apartments with data collected over a 5 week period. So called "ground truth sensors" were installed on all fixtures in the house and were designed to track hot and cold water usage. These ground truth sensors were used to determine if the pressure sensing framework could be used to infer hot/cold water usage along with water usage patterns.


The original template matching algorithm used with Hydrosense was not enough to adequately extract useful data from a noisy system. The system was extended to leverage probabilistic algorithms--which performed significantly well over templating.


Typically, flow trace analysis is the de-facto standard on water sensing, and it categorizes about 83% of isolated water usage events. However, the performance drops to 24% when compound events of two fixtures occur, and 0% when three or more were used. The technique described by the authors gets much better, and is able to classify between compound events.


Each site has two pressure sensors along with all the ground truth sensors. Three types of ground truth handle sensors were installed on the various fixtures: Reed switches, accelerometers and hall effect sensors. Additionally, three types of sensors were installed for washing machines and dishwashers: power usage sensors, push buttons, and thermistors. Power consumption patterns were used to reconstruct when those applicances used water. Additional interfaces to 3/8" access points were built to allow the system to be deployed to apartments in addition to the standard 1/4" access points. It took two people two full work days to install everything at each deploymet site. During the trial, data was collected, and the pressure stream was annotated manually or automatically where possible. After the experiment labels were reviewed, and the total time to analyze the data was 8-12 hours per week of data collected.


In addition to all the hardware, a software architecture was deployed consisting of a custom data logger that uploaded information to a web server at 30 minute intervals--along with an email mechanism to inform the researchers of failures. The researchers also built annotation software to help instrument the pressure stream.


As it turns out, Compound/collision events are hard to classify using pressure-based approches. However, baysian filters and statistics magic help define pressure transient sequences (collisions/compounds). Once the source and filter are separated, the event can be translated into a bigram and classified using traditional speech recognition algorithms.


The language used defines a Bigram as an open->open event, and there are transition probablities for every valve pair, and Katz smoothing is used to assign non-zero probability to every sequence. A set of grammar rules is applied across the language. These rules consist of three values: 1. an opening of a valve must be followed by a closing of that same valve, 2. a valve's closure must be preceded by its opening, 3. the temperature state of a valve must be consistent. The grammar is soft, in that, it doesn't eliminate impossible sequences but instead probabilistically penalizes them. The gist is to idenitfy the closest match of valve tuples given a real event--hence all the fuzziness and probabilities.


By increasing the number of pressure sensors installed in the sensing network to two and leveraging a sophisticated statistical model based on speech recognition algorithms, the researches netted anywhere from a 2% to 14% increase in the accuracy of events. Unfortunately this is only the upper bound, and there's nothing guaranteeing such high performance. Plus they cheat by using the ground truth data to segment transient wave forms. As a result they are able to analyze the discriminability and consistency of pressure transients, but this doesn't really do much to validate the feasibility of using such a system with a high degree of success in real-world applications where transient events are common (a large building for example.)



At the Flick of a Switch: Detecting and Classifying

Unique Electrical Events on the Residential Power Line


Name: Mebaa Kidane


This paper was written by researchers from Georgia Tech's College of Computing and GVU Center. Essentially, this paper discusses a way to sense the activity within a home using the noises sensed from electrical devices connected to a power line. The sensing device connects to a computer via USB and is plugged into an electrical outlet. The signals are "heard" from the devices and are used to distinguish which devices in the building (which are similarly plugged into an electrical socket) are being used.


Obviously, for this to work, every device connected to the power supply needs to be sensed individually to determine its electrical signal. With this information, each device can be uniquely identified. The device is coupled with a machine learning algorithm to identify devices based on their signals.


A study was conducted by this group of researchers. This study tested the system in various homes (one for several weeks and five for one week). Over the course of the study, data was collected and labeled many times. The accuracy of their identification system was then tested and the number of correct identifications was recorded. The results of this study indicate that sensing of devices via this device is very promising and can be extremely accurate (85 to 90 percent according to the trials). Though this is true, it does have its limits. For example, it is important (for the sake of accuracy) to have many samples. As samples increase, so too does the learning algorithm's accuracy. In addition, each device must have a uniquely identifiable signal (or set of signals when being turned on/off and when idle). 


This research has very far-reaching goals. Using the information from the device allows for the identification of a resident's activities via a simple, non-intrusive manner. This information proves useful in many fields, including, among others, healthcare and security.




At the Flick of a Switch: Detecting and Classifying Unique Electrical Events on the Residential Power Line

Name: Bilal Anwer


Human activity sensing is of interest due to its application in many ares of life; health, security, power consumption etc. "At the Flick of a Switch" paper talks about using the variations in power signals and resulting noise to detect on and off events inside the home. This paper proposes an approach that uses infrastructure mediate sensing instead of lying down a whole new infrastructure for sensing. Key observation for this work was that there is a certain noise signals generated when a switch is flicked and the noise signature is different for on/off switch events. This noise can be of transient or continuous nature and can be used to detect devices inside home.

Paper presents a simplified model of electrical noise source and classifies them in three kinds of sources. 1- Resistive loads is produced by stove, coil heaters,lamps because of thermal noise or Johnson noise.  2- Inductive Loads such motors,fans generate noise by continuous breaking and connecting of motor brushes. 3- Solid State Switching devices like computers, microwave ovens etc. produce synchronous noise that is different between the devices.

  For the basic exploration of detection of noise in power lines the system was built to produce three outputs. 1- Standard 60 Hz power signal. 2- bandpass filtered power line output in range of 100Hz to 100KHz. 3- Another bandpass filtered power line output between 50Khz and 100Khz. Then they visually detected noise for different devices to confirm electrical noise signature produced by different devices. Many of the devices produced more detectable continuous noise at higher frequencies and at lower frequencies they produced low amplitude continuous noise but a higher transient noise.

They used Support Vector Machine to train the system to learn the transient noises and detect them. For system evaluation, first they evaluated the system for detection of transients. They conducted five tests to detect transients noises and found 90% plus accuracy in 4/5 tests. For classification of transient events they did a study for 6 weeks in one home and for 6 weeks in one home and in almost all weeks(weeks 1-6) SVM accuracy was >80%.

The system has its own limitations for example in case of compound events system can be confused. Similarly in case of commercial settings the noise generated by heavy duty equipment can cause the system to not function properly, one solution can be to increase the spectrum range.
Similarly, power strip noise is different from the wall socket noise. But overall system provides a great solution to detect electrical events inside home using single system placed in normal wall socket, without requiring changing existing infrastructure.




At the Flick of a Switch: Detecting and Classifying Unique Electrical Events on the Residential Power Line

Name: E.J. Layne



There were two aims of this research paper: the first was to measure the electrical noise on the power line based upon the switching on and off of eletrical loads from appliances and switches around the house. The second was to accurately correlate this data with human activity and interaction with those eletrical appliances around the house, and to use machine learning to predict what a user just interacted with based on the electrical data. In their approach, they did not want to install an intrusive, complex system into the user's house that might interfere either physically or visually with the user's lifestyle; instead, they piggybacked off of the existing powerline infrastructure in the house to record their measurements.


During their machine learning process, they focused on classifying events so that the system would be able to recognize them later when the system was deployed in other houses. They removed large unpredictable draws from the powerlines such as an HVAC or a refrigerator, and then tested each event individually. They trained the system in one house, and through 5 tests, they found that they system's accuracy ranged from 88% to 98% accuracy. Then, over the next 6 weeks, they focused on retraining their machine-learnt system in other houses to see if the system would still accurately recognize events on the powerlines. In those houses, the accuracy ranged from 84% to 92%. Finally, they tested the first house again with the retrained system to see if the preliminary results still held.


This study was very well done, as the researchers realized that there could be many unpredictable variables in their study, so they did they best to isolate and simplify those variables while still collecting valuable information. They appear to address my biggest concerns in their limitations section: in a large house, there could be events happening simultaneously, so in addition to recording events individually, they should also test events happening together, and see if there are predictable patterns. For example, if a HVAC begins to blow cold air at the same time as a light switch is flipped on, is there a predictable signature?




Detecting Human Movement by Differential Air Pressure Sensing in HVAC System Ductwork: An Exploration in Infrastructure Mediated Sensing

Name: Dylan Demyanek



The aim of the study on which this paper was based was to develop a non-invasive approach to whole-house movement sensing. Particularly, the goal was to monitor the movement of individuals between spaces in a household as well as record door opening and closing events. The key to this study, and what distinguishes it from many similar studies is the fact that it only requires the installment of a single sensing unit, opposed to several units. The approach is part of a new group of sensing technologies that use infrastructure mediated sensing. That is, the sensors leverage existing infrastructures within the home to record data and make inferences based on that data. In this instance, the researches from Georgia Tech utilized the heating, cooling and ventilation (HVAC) systems in homes to monitor movements between rooms. The study required placing a single group of sensors on an air filter and capturing data from the system. When the air-flow of a room is disturbed by a door opening or closing or a person walking through a passage, there is a change in static pressure that the sensors are able to detect. Using this style of sensing via air flow could be used in conjunction with electrical sensors and water-flow sensors to greatly improve energy efficiency in households and larger buildings as well. The researchers found that they were able to accurately record inter-room movements with a 75-80% accuracy rate, which proves that this method of sensing is viable for possible future use. 


Title: At the Flick of a Switch: Detecting and Classifying Unique Electrical Events on the Residential Power Line

Name: Saie Deshpande


This paper focuses on detecting all the electrical activities taking place inside a house. They have used a single plug-in switch to detect the electrical noise caused by turning a switch ON and OFF and based on the noise certain devices make while in operation. The authors tested the system in one home for six weeks and then tested five houses, for one week each.  They found that the accuracies of electrical activities range from 85-90%


This paper uses abruptly switched electrical supplies for the study which generate broadband electrical noise, either transient or continuous. For the testing, they used a few devices of interest and collected their noise signatures while turning the device ON and OFF and while it was running. They observed that the devices that drew less than 0.25 AMP were not detected while the ones that drew over 0.25 AMP generated a significant amount of electrical noise. They recorded and observed about 100 events in one home for six weeks and collected about 3000 samples. They measured the accuracy of classification of all the electrical devices.


I feel that their approach was very interesting. It uses a single switch for detecting the electrical activity in an entire house which is impressive. They tested their module in six homes only but, used six different types of homes.  Also, large amounts of result data were collected that helped them draw a conclusion. Large data helped them generate more accurate classification results. However, this approach works only for stationary devices. Since the use of mobile devices nowadays is are inevitable it is essential to extend this approach to mobile and portable devices too.  

Title: Living in a Glass House: A Survey of Private Moments in the Home 

Name: Madhura Bhave


Due to the advances in sensing technologies, it is very common to have sensors and recording devices integrated into homes. There are a number of benefits that these technologies provide but they come with a price, mainly privacy. In this paper, the authors have discussed activities which people prefer not to be recorded. They conducted an anonymous survey involving around 475 people.  They used offline and online recruiting methods such as Amazon’s Mturk, the researchers’ personal networks and Craiglist. In addition 150 physical postcards were distributed along with another 150 prepaid postcards at local places such as coffee shops, medical waiting rooms etc.


Pilot testing was conducted before deciding the survey content with 114 respondents. To help respondents imagine how sensors might be embedded in their homes they found that the best way was to make the scenario neither too vague nor too specific.  Thus they provided the following scenario:

Imagine a future where you live in a smart home in which security cameras and microphones are used to protect you, the other members of your household, and your household itself. These devices would be integrated into every room in your home except for the bathroom(s).


Of the 475 respondents, a majority of them were female. Respondents included students, homemakers, unemployed, having IT related jobs and people from a number of other occupations such as attorney, writer, artist, cashier, teacher, marketer, statistician, professor, consultant etc.

A total of 1433 different activities were reported which people would not want to be recorded. They were divided into 19 high-level and 75 sub-categories. They analyzed the types of activities and also where in the home these activities tend to be performed. The most common activities recorded were those of intimacy, cooking and eating, oral expressions, self-appearance and media use. Also, some locations in the home were considered to be more private than others.


Although the results of the study seem very obvious, the authors feel that enough attention hasn’t been paid towards privacy concerns around recording and sensing technologies. Using these results, the authors want to help designers and developers become better aware of the types of activities that their potential users consider private.


Title: Accurate Activity Recognition in a Home Setting

Name: Alex Samarchi

Summary: The authors of this paper are investigating the potential benefits of a sensor system capable of automatically recognizing activities because this could allow for ubiquitous applications. As a result they designed an easy to install sensor network with an inexpensive annotation method and recorded a data-set consisting of 28 days. They published this data-set, along with its annotations, on their website for the community. Through a number of experiments they showed how the hidden Markov model and conditional random fields perform in recognizing activities.


This paper had 4 major contributions:

  • a sensor network setup that can easily be installed and used in different houses
  • an inexpensive and accurate method for annotation along with the software for doing so 
  • the sensor data nad its annotation online so that it can be shared by the research community
  • a number of experiments on our sensor data showing how to effectively use a probablistic model for recognizing activites


Their system was composed of sensors and annotations. The sensors were wireless network nodes to which simple off the shelf sensors were attached. They chose the RFMDM 1810, because it had a rich well documented API and the ease of adding new sensors by way of a simple pairing procedure. The annotations were created using a bluetooth headset combined with speech recognitcion software. The starting and end point of an activity were annotated out of predefined set

of commands. They utilized a Jabra BT250v bluetooth headset, with a custom annotation software built by their research team. The software was written in C and combines elements of bluetooth API and Microsoft Speech API. 


For this software they created their own speech grammar which contains possible combinations of commands the engie could expected. Commands such as "begin use toilet" and "begin take shower" allowed for almost flawless recognition results during annotation. The recognized sentence is outputted using the text-to-speech engine, with a "correct last" command in case of error.


They evaluated the data set with two probabilistic models: Hidden Markov Model and Conditional Random Fields. They ran three experiments using theses models: Model Comparison, Minimum amount of training data, and offline vs online interference. The First experiment yielded better CRF timeslice accuracy than the HMM. But the HMM received the overall highest class accuracy. The second experiment concluded a minimum of 12 days was need for accurate parameter estimation. The third experiment concluded that HMMs still give good results when using the online interface.




Title: Fine-Grained Activity Recognition by Aggregating Abstract Object Usage

Name: Travis Wooten



Researchers from the University of Washington were exploring the advantages and challenges of using an RFID-glove to determine what activities a person might be performing. The researchers applied models to the data collected to determine how effective one can differentiate between activities being performed.


To test their models, the researchers had a subject wear gloves outfitted with RFID antennae so they could track objects, with RFID transmitters attached, that were being used in activities done during a morning routine. RFID was chosen because it does not require line of sight and can tell the difference between different instances of a class of object. About 60 objects were tracked during 11 different activities.


The problems the researchers ran into were fairly expected. Some objects were used in multiple different activities. In addition, several activities were not done in completion all at once. Most were done simultaneously while doing other activities.


Concluding, the researchers discovered that as their models became more and more complex, they were able to more accurately determine which activity a subject was performing.




Title: Mining Models of Human Activities from the Web

Name: James O'Neill



Information regarding the current day-to-day activity a person is involved in is a an area of interest in application programming. Activities (i.e. "cooking" or "sleeping") can be characterized by object terms that describe the surrounding environment (i.e. "pot" or "bed"). Object terms can be generated from RFID tags placed on household items, and statistical relationships between a set of object terms and activities can be derived by web mining. These relationships are "activity models," which are probabilistic translations between activity names and the objects involved in the activity. 


The paper is divided into three tasks:


1. Theoretically address the problem of creating a mapping between natural language object terms and activities (e.g. an activity model).

2. Build the activity model automatically via web crawling techniques.

3. Validate the results.  


This paper is written within the larger context of the Proactive Activity Toolkit (PROACT), developed at Intel Research (http://luci.ics.uci.edu/websiteContent/weAreLuci/biographies/faculty/djp3/LocalCopy/irs_tr_03_013.pdf). 


The authors build a model extractor from definitions of activities written in natural language, such as recipes, device use manuals, training manuals, and experiment protocols (all of which they generically call "directions"). In order to build a formal model, they make assumptions about the syntactic structure of the directions mined from the internet. They assume that each direction has:


1. A title.

2. A list of steps, each containing a natural-language paragraph that details instructions. Each instruction step probably mentions a subset of the objects involved in the task and may contain a keyword delimiting duration.


The authors use natural language parsing techniques to extract the activity label and related objects. First, object extraction is performed to identify words that could correspond to objects. Then, noun phrase extraction more accurately whittles down the extracted object words to those that are nouns. To compute the conditional probability of an object being associated with an activity, they compute the "Google Conditional Probability," which is GoogleCount("l" + o)/GoogleCount(l) (where GoogleCount in the number of Google results, 'l' is the activity label, 'o' is the object label, and '+' is the string concatenation operation). The GCP gives reasonable likelihoods of objects being involved in particular activities. For instance, the GCP of "'making tea' + cup" is 0.30, whereas the GCP of "'making tea' + keyboard" is 0.03, so, given that the current activity is "making tea," it is more likely that a cup is involved than a keyboard. Finally, "tagged object filtering" reduces the set of possible activities to those which have a surrounding real-world object in their model. A Sequential Monte Carlo approximation is then used the solve for the most likely activity at a given point in time.


The authors mine approximately 21,300 activity models from ehow.com (for domestic task directions) and epicurious.com (for recipes). They then evaluate the system in the context of PROACT, mentioned earlier. 



Title: Transferring Knowledge of Activity Recognition across Sensor Networks

Name: Sunil Garg



In order for activity recognition to be performed, a labelled data set needs to be available because most models for activity recognition require such data to learn their parameters. This causes problems at scale, because creation of such data sets is expensive and the data set for one house cannot be used for another house because of differences in a homes' layouts and behavior of their occupants. The authors propose to use "transfer learning" to use existing data sets from various homes to support classification in a different target home.


The existing sensor data is used to create a hidden Markov model, and then a prior distribution is created based on the same existing datasets, with which the authors algorithmically learn the target model parameters.


To test this method, the authors recorded datasets in three houses with varying floor plans. Occupants of the houses were asked to annotate their behavior in a standard format, for a duration ranging from 13 to 25 days depending on the house. They conducted three experiments with this data, finding that the transfer learning method works well, especially when no labelled data is available for a target house. It can outperform previous methods in which only a single model was used for all houses. However, in some cases, this method can result in lower performance than other approaches that do not utilize transfer learning.



Title: A Long-Term Evaluation of Sensing Modalities for Activity Recognition

Name: Pradyut Pokuri



The authors of this paper evaluated various sensors and their ability to recognize human activities in a home setting.  There are a number of ways that such computer systems can assist in our daily lives.  The authors specifically point on health care as an example application.  Systems that monitor health and provide automated care reduce health care costs and improves quality of life.


There were a few other experiments related to this paper but none were as thorough and none simulated real-world conditions as these authors did.  Most related work involved sensing of one or two activities.  The authors of this paper sensed all movements of the subjects and attempted to extrapolate what activity was being undertaken.  Furthermore, other experiments were usually held in a lab setting unlike this one which involved a married couple continuously being sensed and recorded for 10 weeks.


A variety of sensors including reed switch, infra-red, RFID, water-flow etc were used. RFID tags were placed on a number of devices and new ones were installed over the course of the experiment.  The subjects were advised to maintain a normal routine and at the end both subjects felt they were able to do so.


The activities were categorized and statistics such as cumulative time, mean time, number of occurrences were recorded.  However due to technological limitations and complex human behavior, there were a some irregularities.  For example, the subject wrote the RFID bracelet on his right hand, if he grabbed something with his left hand, the RFID on the item would not fire and so would not have been recorded.


The data was analyzed using decision tree classifiers, as they had superior performance. The authors also cross-validated by conducting "leave one day out experiments" for each activity. The authors noted that motion-based sensors stood out as the best sensor category while RFID was the worst.


Eating was the most difficult activity to detect.  Most people do not sit at the same location every time when eating.  Also humans do not necessarily eat 3 meals, but rather eat a number of times throughout out the day. Overall it was noted that even with the large dataset studied, it was not sufficient to designed an automated recognition system.



Title: Mining Models of Human Activities from the Web

Name: Pradyut Pokuri



The paper points out the importance of activity recognition systems.  The authors point out that modeling activities does not scale well as low-level details but be manually specified.  They present a new approach that scales to a much larger class of activities using RFID tags. 


The two main problems with designing a recognizing system are presented.  A system should not be incapable of recognizing new activities. Second, they must be able to extract models without a considerable human burden.  


They use components of a large system called PROACT.  Input is composed of two things.  When an activity is initiated, the objects touched by a subject in this activity are recorded using RFID tags. These tags are placed on various objects around the house.  The system records the time and sequence followed of the various tags.


The system then crawls the web to find documents which contain the same objects and approximately the same time and sequence.  The authors explain the mathematical model used to create a quantitative score to infer the activity.


For example, the user touches a teacup, followed by a water faucet and then the stove.  He grabs a teabag and then the milk and sugar.  The system mines the web for documents containing these objects.  It uses natural language processing to convert text directions to activity models.  Finally the system can then infer that the user was making tea.


The authors used the techniques to create more than 20,000 models for activities mined from ehow.com (domestic tasks) and epicurious.com (for recipes). This novel technique is implemented for the first time by these authors.  There analysis shows that the technique is fairly accurate and further research will incorporate more sensors as well as other websites.




Title: Mining Models of Human Activities from the Web

Name: Sean Swezey



A system for determining what a person is doing and providing information in relation to what that person is doing has previously been difficult to implement and also difficult to reuse between any two systems. This paper proposes a new method using information garnered from the internet along with RFID tags to allow for a more flexible system in determining what a user is currently doing.


The way they have their system PROACT designed is a model is written for an activity with keywords in it to help define the activity. These keywords correspond with objects which would be used during the activity and that are tagged with an RFID tag. They then translate this human description of an activity into a model which they can they lookup from common instruction websites such as eHow. The system collects the information from the sensors and the models and then does a probabilistic approach to determining which activity a user is doing based on the objects they are using, the order they are using them, and in some cases the duration of using each step.


The RFID system works by tagging as many objects as possible and making an item database with the corresponding noun for each tag. The tags are then read with a mobile robot and the objects in-context are determined using the user's location or using a mobile reader worn on the hand. This allows the PROACT system to determine which objects are within reach of the user - which typically represents which objects a user is currently using.


Each activity is modeled using a sequence of probabilistic item interactions and then using a database of these, when the sensor data is retrieved, it is compared against these models to determine a probability for a set of activities, and the highest one being, presumably, the correct activity.


To check their models, they compared activities mined from eHow, ffts.com and epicurious.com. For ffts.com and epicurious.com they compared the recipes for both to check to see if an activity was correctly mapped to both, and that the recipes mapped to each other. However, with recipes, some steps were optional, or out of order, which confused the model. Also, they tested activities of daily living (ADL) by having participants over a 3 months perform a subset of 55 chosen activities and then checked the accuracy of the results from these tests. 


This paper was presented as a first step in ongoing work to use the web to help determine activities from sensors. They plan to further refine their model, and to add more activities and tags to allow for more activities to be modeled. Also, they plan to use sensors other than RFID tags, with more of 'cues' to help determine an activity as well as just objects to determine which activity is being performed. 




Title: ThinSight: Integrated Optical Multi-touch Sensing  through Thin Form-factor Displays 

Name: Harish Kothandaraman


Thinsight is a new technology that makes use of IR optical sensors to create a multitouch display. It is not huge like the other multi-touch LCD displays because it does not have a camera-projector included in the form factor.


The prototype uses InfraRed emitters and detectors embedded in the LCD display to take care of the multi-touch interaction. The sensing is of a very low resolution in the current prototype but it is effective enough to detect fingertips and hands. The recognition is done using simple image processing techniques on the data.


ThinSight has a compact, low profile form factor. It has retro-reflective optosensors in the form of IR emitters and detector mounted on the LCD panel. Light is reflected by any object that touches the display. This reflected light is detected by the detectors, producing the data required for touch recognition.


The output of the initial data collection is a monochrome image. This monochrome image is proccessed using computer vision techniques to get the data in a form that can be used to identity touch locations etc.


The LCD display's multiple layers do not attenuate the IR greatly, as we think they might. Fingers are clearly identifiable despite the attentuation. Plus IR is invisible, so displayed content is not adulterated.


Using ThinSight, any object can be detected including its shape and orientation. Barcode markings may also used on touch devices such as styluses. Also, IR emitters may be attached to such devices making room for more possibilities.


In the current prototype, fingers have to be at least 10 mm withtin the screen to be detected. A grid of 105 devices covering the area of the LCD panel collects data in to a PIC microcontroller to construct a "frame" of data which can be sent to a PC.


Behind the LCD backlight od a 17" dell laptop, holes were made in the white reflector opposite each emitter. Accurate alignment was made. The arrangement was made to resemble a tablet pc.




Sensor data is processed and filtered to generate an image to detect objects. TOuch/Non touch differentiation has been done using thresholding.Disambiguation hover is possible too. The prototype cannot, however, do sophisticated image recognition.



Interaction is done by moving fingers to the display. Pressure can also be detected.


Future Work:

Future work may be aimed at refining the detection and recognition. It also promises potential in the multi-touch multi-user interaction,gaming and media management.


Title: Accurate Activity Recognition in a Home Setting

Name: Chris Bayruns



The details of typical human activities can be a very useful tool for designing and planning ubiquitous

mobile computing applications. Tim van Kasteren et al implement an RF sensor system consisting of 14 state change sensors in a 26year old's apartment to record and analyze his activities. The locations of the sensors include doors, cup-boards, refrigerator and a toilet flush sensor. In order to make sense of the activities the subject used a bluetooth headset to voice annotate the activies during the 28days of collecting data in his apartment. This resulted in 2120 sensor events and 245 activity instances. Of note the highest number of instances recorded was toilet flushing but the largest percentage of time was leaving, followed by sleeping.


As stated by the authors, the task of recognizing and infering activities from sensor readings can be a formidable task. They statistically model the activites using temporal probabilistic models, being the Hidden Markov Model and Conditional Random Fields (CRF) model.The Hidden Markov Model (HMM) directly applies since it looks for a hidden variable and an observable variable, in their case being the activity performed and the sensor reading. The two models are compared using a Confusion Matrix. They found that the CRF achieves a higher timescale accuracy while the HMM achieves the overall highest class accuracy using a changepoint+last representation.


In the end I believe the authors work introduces a useful sensor and annotation system for measuring and analyzing activity recognition in a living setting.


Comments (0)

You don't have permission to comment on this page.