Thermostat Launch

Role

User Researcher

Activities

Field Trial Recruitment & Planning, Surveys, Interviews

The Challenge

In the summer of 2018, I partnered with the Product Manager to identify ways to measure customer satisfaction while the product was in field trials. With this goal in mind, we developed four main questions that informed the months-long research plan :

1. Quality Assurance

   • Is the product functioning reliably?

2. User Experience

   • Are users satisfied with the overall product experience?

3. Customer Satisfaction

   • Are we delivering on our brand promise?

4. Product Usage

   • In what ways is this product being used? 

Approach

PLANNING

To avoid knowledge gaps and unassigned responsibilities, a multi disciplinary team consisting of the Product Manager, UX Designer, Agile Coach and Customer Support Lead were brought in to outline the scope and responsibilities for each element of recruitment.

This allowed for clear, up-front communication of expectations to participants who were being enrolled into the months-long feedback program.

RECRUITMENT

I screened and subsequently recruited about 150 participants across various demographic segments to capture diverse sentiments and ensure quality assurance across different types of heating and cooling systems.

METHODS

The feedback process was a mix of bi-weekly surveys, interviews and home visits and self initiated feedback from participants through emails, customer support and a portal the team had set up for the purpose of the beta program.

The survey were staggered based on the participant’s phase in the beta process and targeted all elements of the thermostat features using metrics like the NPS (Net Promoter Score), CSAT (Customer Satisfaction Score) and an internally developed tool called the FUSS (Feature Usage Satisfaction Score) which correlates feature value and satisfaction.

SYNTHESIS

The surveys were tagged and synthesized bi-weekly with an emphasis on rapid feedback to allow the team to make changes and prioritize.

At the end of the Field Trials, I conducted interviews with 10 participants to get deeper insights into their experience with the product. Thereafter, I facilitated a group synthesis session with representatives including Product Managers and UX across the Thermostat Team.

TOOLS/FRAMEWORKS: Jobs-To-Be-Done, Excel, Mural

Insights/Impact

The insights from the research:

• Informed the decision on whether or not to include print manuals and subsequently what information was most valuable to include

• Aided the prioritization of the software features necessary for a successful launch

• Gave the team confidence to proceed with launch

• Have been used inform to the roadmap for future features and products