Our Three Good Things application (3GT) is based on a positive psychology exercise that encourages people to record three good things that happen to them, as well as the reasons why they happened. By focusing on the positive, rather than dwelling on the negative, it is believed that people can train themselves to be happier.

Example 3GT tag clouds

Limited success for Word Clouds

When we interviewed 3GT users, we expected to find that the participants believed the word clouds helped them notice and reinforce trends in their good things. Results here were mixed. Only one participant we interviewed described how the combination of listing reasons and seeing them summarized in the word clouds had helped her own reflection:

“You’ve got tags that show up, like tag clouds on the side, and it kind of pulls out the themes… as I was putting the reasoning behind why certain [good] things would happen, I started to see another aspect of a particular individual in my life. And so I found it very fascinating that I had pulled out that information… it’s made me more receptive to that person, and to that relationship.”

A second participant liked the word cloud but was not completely convinced of its utility:

I like having the word cloud. I noticed that the biggest thing in my reason words is “cat”. (Laughs). And the top good words isn’t quite as helpful, because I’ve written a lot of things like ‘great’ and ‘enjoying’ – evidently I’ve written these things a lot of times. So it’s not quite as helpful. But it’s got ‘cat’ pretty good there, and ‘morning’, and I’m not sure if that’s because I’ve had a lot of good mornings, or I tend to write about things in the morning.

Another participant who had examined the word cloud noticed that “people” was the largest tag in his good things cloud and “liked that… [his] happiness comes from interaction with people,” but that he did not think that this realization had any influence over his behavior outside of the application.

One participant reported looking at the word clouds shortly after beginning to post. The words selected did not feel representative of the good things or reasons he had posted, and feeling that they were “useless,” he stopped looking at them. He did say that he could imagine it “maybe” being useful as the words evolved over time, and later in the interview revisited one of the items in the word cloud: “you know the fact that it says ‘I’m’ as the biggest word is probably good – it shows that I’m giving myself some credit for these good things happening, and that’s good,” but this level of reflection was prompted by the interview, not day-to-day use of 3GT.

Another participant did not understand that word size in the word cloud was determined by frequency of usage and was even more negative:

It was like you had taken random words that I’ve typed, and some of them have gotten bigger. But I couldn’t see any reason why some of them would be bigger than the other ones. I couldn’t see a pattern to it. It was sort of weird… Some of the words are odd words… And then under the Reason words, it’s like they’ve put together some random words that make no sense.

Word clouds did sometimes help in ways that we had not anticipated. Though participants did not find that they helped them identify trends that would influence future decisions, looking at the word cloud from her good things helped at least one participant’s mood.

I remember ‘dissertation’ was a big thing, because for a while I was really gunning on my dissertation, and it was going so well, the proposal was going well with a first draft and everything. So that was really cool, to be able to document that and see… I can see how that would be really useful for when I get into a funk about not being able to be as productive as I was during that time… I like the ‘good’ words. They make me feel, I feel very good about them.

More work?

The importance of supporting reflection has been discussed in the original work on Three Good Things, as well as in other work that has shown how systems that support effective self-reflection can improve users’ ability to adopt positive behaviors as well as increase their feelings of self-efficacy. While some users found benefit in word clouds to assist reflection, a larger portion did not notice them or found them unhelpful. More explanation should be provided about how word clouds are generated to avoid confusion. They should also perhaps not be shown until a participant has entered a sufficient amount of data. To help participants better notice trends, improved stop-wording might be used, as well as detecting n-grams (e.g. “didn’t smoke” versus “smoke”) and grouping of similar terms (e.g., combining “bread” and “pork” into “food”). Alternatively, a different kind of reflection exercise might be more effective, one where participants are asked to review their three good things posts and write a longer summary of the trends they have noticed.

Navigation to Apps
Formerly, the application dock made it easy to access an application from anywhere in Facebook. One click to get to a bookmarked application; two clicks (without waiting for page loads) to get to other applications. In the new homepage, this has been removed.

Not so with the new design. If an application doesn’t have a tab on your profile page, the only way to access it is from the home page. From my profile or someone else’s profile, this means: click to the Facebook home page, wait for the page to load, click the app icon (or, if the application is not one of your top three book marks, click more and then click the app icon). Yes, this is only one more click, but it requires waiting for an entire page load, and it’s worse for non-bookmarked apps: one click to the home page, wait for it to load, one click to the application dashboard, wait for it to load, one click for “see all of your applications,” wait for that to load, and finally click on the application.

One possible remedy might be to add an “applications” drop-down next to the new notifications, requests, and messages icons.

Notifications
At the end of the month, Facebook will turn off the ability of applications to send notifications. This is a method I’ve been using to send reminders in Three Good Things, for both automatically generated reminders and user-to-user reminders.

3GT Notifications. Left: system generated reminder. Right: user-to-user reminder.

I understand that notifications have gotten a lot more intrusive with the addition of push notifications to the iPhone app, and that some app developers have used them more than some Facebook users would prefer. Facebook has also added additional integration points. On the balance, though, I think notifications are going to be an unfortunate integration point to lose.

Application Privacy
Along with some others building health and wellness applications for the Facebook platform, I’ve felt fairly strongly that Facebook needs to give users and developers enhanced privacy controls for applications. At a minimum, this should include the ability to hide one’s use of an application from friends (i.e., not appearing under “friends using this application” in the application’s profile page).

With the recent updates Facebook’s designers and developers appear to have recognized some of these concerns. Application developers are able to set an application as “private,” causing one’s use to not appear in the new application dashboard. This is a good start, but it feels incomplete for a number of reasons. First, users, not developers, should have control of privacy. What’s to stop a developer from later reverting to a more public setting, instantly and completely changing what user activity is revealed? Currently, users do not have any way to remove this information once it appears, either.

Second, this level of privacy does not extend far enough. Friends who use private applications still appear on the application’s profile page under “friends using this application.” Furthermore, the model of application use and content being either private or public is insufficient. In health and wellness applications, for example, participants may benefit from sharing and interacting with other participants in the intervention as well as their friends or family members on Facebook, while also wanting to keep their activity private from coworkers.

This is something that Facebook has already discovered and addressed with the newsfeed (now stream) content, but application content does not enjoy the same privacy controls. To share with only a subset of one’s friends within an application, however, application developers must implement their own social graph features and users must built a second network within the application. Enabling privacy controls similar to those for the newsfeed for application content and use could help people to feel more comfortable using health wellness applications on Facebook while creating more possibilities for designers. A user could only allow an application to be aware of relationships in one or more friend lists or networks, or to select some to exclude from the application while allowing the other connections to remain visible. Assuming a user had created the necessary friend lists or that their privacy preferences mapped to their networks, this would allow someone to filter out their coworkers or to allow only close friends to see their participation.

Update: As of 17 February, Facebook has added a privacy argument to calls for publishing methods (e.g. Stream.publish that gives applications much more control over shared content.

In this study, Paul Resnick and I explored individuals’ preferences for diversity of political opinion in online news aggregators and evaluated whether some very basic presentation techniques might affect satisfaction with the range of opinions represented in a collection of articles.

To address these questions, we needed subjects with known political preferences, from the United States, and with at least some very basic political knowledge, and we wanted to collect some demographic information about each subject. Each approved subject was then assigned to either a manipulation check group or to the experimental group. Subjects in the manipulation check group viewed individual articles and indicated their agreement or disagreement with each; subjects in the experimental group viewed entire collections and answered questions about the collection. The subjects in the experimental group were also assigned to a particular treatment (how the list would appear to them). Once approved, subjects could view a list up to once per day.

Screening. To screen subjects, we used a Qualification test in AMT. When unqualified subjects viewed at task (HIT – human intelligence task, in mTurk parlance), they were informed that they needed to complete a qualification. The qualification test asked subjects two questions about their political preferences, three multiple choice questions about US politics, and a number of demographic questions. Responses were automatically downloaded and evaluated to complete screening and assignment.

To limit our subjects to US residents, we also used the automatic locale qualification.

Assignment. We handled subject assignment in two ways. To distinguish between the treatment group and the manipulation check group, we created to additional qualifications that were automatically assigned; an approved subject would be granted only one of these qualifications, and could thus could only complete the associated task type.

Tasks (HITs). The task implementation was straightforward. We hosted tasks on our own server using the external question HIT type. When a subject loaded a task, AMT passed us the subject’s worker ID. We verified that the subject was qualified for the task and loaded the appropriate presentation for that subject. Each day, we uploaded one task of each type, with many assignments; assignments are the number of turkers that can complete each task.

Because we needed real-time access to the manipulation check data, the responses to this task were stored in our own database after a subject submitted the form; the subject could then return to AMT. This was not necessary for the experimental data, and so the responses were sent directly to AMT for later retrieval.

Quality control. Careless clicking or hurrying through the task is a potential problem on mTurk. Using multiple raters does not work when asking subjects about their opinions. Kittur, Chi, and Suh recommend asking Turkers verifiable questions as a way to deal with the problem¹. We did not, however, ask verifiable questions about any of the articles or the list, because that might have changed how turkers read the list and responded to our other questions. Instead, we randomly repeated a demographic question from the qualification test. 5 subjects changed their answer substantially (e.g. aging more than one year or in reverse or shifting on either of the political spectrum questions by 2 points or more). Though there are many possible explanations for these shifts – such as shared accounts within a household, careless clicking, easily shifting political opinions, deliberate deception, or lack of effort – all of these explanations are not desirable for study subjects, and so they were excluded. We also examined how long each subject took to complete tasks (looking for implausibly fast responses); this did not lead to the exclusion of any additional subjects or responses.

Some reflection. We had to pay turkers a bit more than we expected (~$12/hr) and we recruited fewer subjects than we anticipated. The unpaid qualification task may be a bit of a barrier, especially because potential subjects could only complete one of our paid tasks per day (and only one was listed at a time). Instead, we might have implemented the qualification as a paid task, but that might result in paying for subjects who would never return to complete an actual task.

Further resources

• Guide to experiments on Amazon’s Mechanical Turk: Winter Mason has put together a great getting started guide, including an overview of the AMT command line tools. Also see his paper from HComp 2009.
• Experimental Turk: Blog about using Mechanical Turk for social science experiments, including the reproduction of several classic experiments on mTurk.
• Mechanical Turk @ A Computer Scientist in a Business School: Panos Ipeirotis has written several blog posts about AMT and turkers, including turker motivations and demographics and using AMT in other contexts (particularly labeling data).
• Experimenting on Mechanical Turk: 5 How Tos: Another set of tips on Mechanical Turk experiments, from Markus Jakobsson (PARC).
• Dolores Labs blog: Lots of great crowdsourcing examples and tips.

1. Kittur, A., Chi, E. H., and Suh, B. (2009). “Crowdsourcing User Studies With Mechanical Turk,” Proc. CHI 2009: 453-456. (ACM | PARC)
2. Mason, W. and Watts, D. J. (2009). “Financial incentives and the ‘performance of crowds,’” SIGKDD Workshop on Human Computation: 77-85. (ACM | Yahoo)

This study is part of the BALANCE project and was funded by NSF award #IIS-0916099.

I’ve been meaning to post a simple update to my previous visualization of political blogs’ link similarities. In the previous post, I used GEM for layout, which was not, in hindsight, the best choice.

In the visualization in this post, the edges between blogs (the nodes, colorized as liberal, independent, and conservative) are weighted as the Jaccard similarity between any two blogs. The visualization is then laid out in GUESS using multidimensional scaling (MDS) based on the Jaccard similarities.

Three Good Things supports a positive psychology exercise in which participants record three good things, and why these things happened. When completed daily – even on the bad days – over time, participants report increased happiness and decreased symptoms of depression. The good things don’t have to be major events – a good meal, a phone call with a friend or a family member, or a relaxing walk are all good examples.

I’m interested in identifying best practices for deploying these interventions on new or existing social websites, where adding social features may make the intervention more or less effective for participants, or may just make some participants more likely to complete the exercise on a regular basis. Anyway, feel free to give the app a try – you’ll be helping my research and you may end up a bit happier.

My guess is that if Twitter does take off among teens and Dylan’s friends feel pressured to let peers and parents and everyone else follow them, the same problem will arise and Twitter will become public in the same sense as Facebook. This of course raises a critical question: will teens continue to be passionate about systems that become “public” (to all that matter) simply because there’s social pressure to connect to “everyone”?

I believe that Twitter may actually be much more resistant to both this pressure and subsequent switch to less “public” platforms than Facebook for two reasons: account norms and Twitter clients.

Account Norms, Privacy, and Collapsed Contexts
On Facebook, everyone pretty much gets one account.¹ This leaves me with a choice of collapsed contexts (same profile for everyone) or only friending people from a particular context or set of context. There are many fine-grained privacy controls, but this all adds up to a more-is-less experience, at least for me. There are enough many controls that I don’t particularly remember what I’ve set to be visible to whom. When I comment on something in friend’s profile (or am tagged in one of their photos), I don’t know who can see that.

With Twitter, people can have multiple accounts, and for private accounts, they know exactly who can see their posts: only people who I give permission. This is not to say Twitter is not without some privacy pitfalls – e.g. plenty of private tweets get retweeted or replies on others’ public accounts – but I have a much clearer idea of who can see a status update or reply on Twitter than I do of who can see similar content on Facebook at the time of posting. I suspect that many users of private Twitter accounts do so just to avoid the “what if so-and-so sees this?” question. So it seems reasonable that people could have different accounts for their work, family, friends, etc personas, though there’s a point at which it probably would be too many.

Twitter Clients
Having multiple accounts wouldn’t work well without an appropriate interface, and here Twitter benefits hugely from its API and the many, many Twitter clients available. Using more than one Facebook account, especially simultaneously, is an ordeal – multiple web browsers, no aggregation. With the right client, reading from and posting to multiple Twitter accounts is a breeze.

So while there may eventually be an exit from a more public Twitter, I think there is more room to move within the same service, diversifying accounts, than there might be on Facebook. This will only, work, though if people are willing to set boundaries and accept boundaries – and probably not if mom and dad insist on following the Twitter account their kids use to communicate with friends from school, or if colleagues regularly feel insulted when a coworker-acquaintance declines their request to follow an account they use to communicate with close friends.

¹I believe this used to be part of the terms of service, but I don’t see it anymore and can’t be sure that it was ever there.

Opinion and topic diversity in the output sets can provide individual and societal benefits. If news aggregators relying on votes and links to select and subsets of the large quantity of news and opinion items generated each day simply select the most popular items may not yield as much diversity as is present in the overall pool of votes and links.

To help measure how well any given approach does at achieving these goals, we developed three diversity metrics that address different dimensions of diversity: inclusion/exclusion, nonalienation, and proportional representation (based on KL divergence).

To increase diversity in result sets chosen based on user votes (or things like votes), we developed the sidelines algorithm. This algorithm temporarily suppresses a voter’s preferences after a preferred item has been selected. In comparison to collections of the most popular items, from user votes on Digg.com and links from a panel of political blogs, the Sidelines algorithm increased inclusion while decreasing alienation. For the blog links, a set with known political preferences, we also found that Sidelines improved proportional representation.

Our approach differs and is complementary to work that selects for diversity or identifies bias based on classifying content (e.g. Park et al, NewsCube; ) or by classifying referring blogs or voters (e.g. Gamon et al, BLEWS). While Sidelines requires votes (or something like votes), it doesn’t require any information about content, voters, or long term voting histories. This is particularly useful for emerging topics and opinion groups, as well as for non-textual items.

Here’s a sampling of the final projects:

• Space News Aggregator, Ben Schoenfeldt
• Financial Buy/Sell Recommender, Jennifer Wolf
• Boo-Moo: Books and Movies Recommendation, Azalea Ayuningtyas
• Aggregator, Jia Jin Kee
• SouthQuad menu to SMS (presentation), HeeJung Byun
• Your Perfect Style, Erica Willar
• Hang Ninja (or on Facebook), Matt Hornback & Clint Sweet
• Flickr Weather, Emily Rosengren
• GoodDayGenerator, Andrew Olmsted
• Intelligent Design (Color Palette picker), Allie Ghaman
• Male Tennis Statistics, Danielle Lamy
• Weather Generator, Khizar Jahangir
• The Big Ten Tracker, Jeff Green
• Twitter:Rise&Fall in stock market, Somin Yoo
• Online PGN Interpreter, Andrew Konishi
• visualSearch, Lee Overbeck
• Weather using Geolocation, Mitchell Overbeck
• What’s new A2? (Ann Arbor review highlights), Evan Schuetz

Also, a huge thanks to Chuck Severance, who got this course started and gave us early chapters of his book Using Google App Engine, which gave us the confidence to use App Engine in the course and which we were able to rely on for class readings.

So, after growing impatient with existing tools, put together a map of Ann Arbor Craigslist housing listings (sorry, I don’t include homes for sale. It’s not the right time to buy, anyway). You can:

• simultaneously display each of the listing types,
• Filter by price, date posted, and number of bedrooms (all as sliders with min and max), and
• adjust the price filters to work on price per bedroom.

This was pretty fast to throw together thanks to BeautifulSoup, the Google Maps API, and YUI, but I’m sure that there are some rough edges that will need to get worked out.

Also, I don’t know or care whether it works in IE.

Update, 19 February: More locations now available.