Step Tracker Goal-setting Study

Overview

Context

Step trackers keep track of daily step numbers for people. They also include features that are designed to encourage users to be more active. One example is goal-setting, where a user can set a step number that he/she wants to achieve everyday. If successful, the user will be granted some ‘honor’ by the step tracker.

However, most often, the goal is arbitrarily set by the user (or worse, the step tracker itself). Studies have proved that inappropriate goals can not be effective in encouraging the behaviors as intended. We believe that the goals should be set around the user’s actual active level (i.e. average step number).

Goal

To investigate the effectiveness of different levels of ‘difficulty’ in goals in encouraging users’ walking activities.

Questions

  1. Does goal setting improve people’s physical activity level?

  2. How does difficulty of goals affect people’s physical activity level?

Approach

Field experiment with manipulations in goal difficulty among groups; interview of individual users; comparative statistical analysis and qualitative content analysis

Outcome

Too difficult goals will harm people’s activity level, while easier goals are more effective in encouraging activities.

Study method

Study setting

  • Participant N = 64

  • Factorial design: 8 groups, 4 goal level × 2 activeness level

  • Measure: steps each participant walks every day for 2 weeks

  • Procedure: as shown in the graph below

Timeline.png

  • We asked the users to choose from the two most popular step-tracking apps and use it throughout the study:

Tools.png

Data analysis

  • Independent two-sample T test

  • Two-way ANOVA

 

Findings

No significant difference in step number before and after goal-setting

Active level does affect the step difference in the two weeks

Thus, we believe it is the effect of the study itself - the users already active are more likely to be encouraged by the study and willing to put in more steps

 

Post-study interview

Some interesting excerpts from our user interview:

“I did try hard at first but then the goal was harder than that so I just kind of gave up”
“This was interesting to me because I’ve always felt that I’m lazy, but you’re telling me that I’m in the ‘active’ half, and this actually motivated me a little bit more than the goal itself. :D”
“I always forget about having the goal... until the end of the day when I report my daily steps to you...”
 

Lessons learned

Details on study design, e.g. time of reporting

Value of user input at the end

An interesting incident in our study:

We ended up having to restart the study after collecting all the data after the first week, because we found that the data of the WeChat users are very different from the data of the iOS Health users. The variance of the WeChat users’ data was so big that we did not feel comfortable with using it. So we tried to figure out why.

So we asked those users to provide step tracking data through that week from other sources - and it turned out to be that the WeChat data were not accurate. However, this only happened to those who do not live in the same time zone as us. Then we realized that the WeChat app did consider time zone issue when displaying the data - which screwed up our study (but luckily we realized it!).

After realizing this, we asked those who use WeChat to send their daily step at the end of their day, instead of us reading their daily step numbers from our friend list at the end of our day, which solved the issue.

Feedback

Please feel free to let me know your thoughts about this study and I would be more than happy to chat with you :D