Data Driven Product Design Enabled By Application Usage Tracking

A couple of years back I read a book about how buildings are reshaped by their occupants after the architects who built them are long gone. It is called How Building Learn. What happens after they are built.   It got me thinking about what happens to software products after the designers who build them move on to other assignments. I wrote my thoughts down in a post titled Landlords Vs Building Architects to document my thought on how product managers of cloud products needs to think about product design.

Six months back I joined Castlight Health to build a product called Castlight Action. Action is a tool that helps benefits leaders get the most out of their benefits strategy by improving benefits utilization, reducing costs, and improving benefits satisfaction.

Right from day one I requested my team to think like a land lord rather than think just like an architect. I asked them to think not only about building a great product, but also learn how it is used after it is built and use that learning to improve the product in the long run. I told them that we are a cloud product and we have the responsibility to not only build the product but also operate it flawlessly, at a reasonable cost and improve it continuously based on what we learn.

David Tischler under the guidance of Alka Tandon, took responsibility for putting the underpinnings in the product that help us track usage, conduct experiments and, based on our findings, prioritize our roadmap to reshape the product. We are using some cutting edge tools, chosen by Anumeha Goel Dhanrajani from our strategic analytics team, to track not only pages but also every action and every user flow in the application. Rather than make tracking and internal reporting an after thought, we are putting the necessary instrumentation in place behind every event to do the following.

1. Help our product managers with real time information about how how their user flows are performing.
2. Help our user experience designers with real time information about how their experience design is performing.
3. Help our content strategy managers and clinicians with real time information about how each and every piece of content is performing.

Since every one of them will have this information, we are laying the foundation for a better chance of success from day one.

Predictive Insights, Personalized Recommendations and Evaluation of Engagement

I have not written a post in many months. That is because for the past six months I have been spending all my time building a product called Castlight Action. At Castlight Health I have the privilege and honor of leading a team of world class experts in the fields of data science, clinical research, product design, product management and behavior change. I decided to write this post to explain what Action does and shine light on some of the many talented people building the Action product.

Castlight Action's goal is to make American workers healthier by connecting them with better care in a timely manner, at a lower cost. This is a worthwhile cause because American employers spend over $ 620 Billion on healthcare out of which 30 percent is wasted. Despite having the most expensive health care system, the United States ranks last overall among 11 industrialized countries on measures of health system quality, efficiency, access to care, equity, and healthy lives, according to a Commonwealth Fund report.

Castlight Action helps American workers manage their health better at a lower cost to them and their employers. To accomplish this Alka Tandon  and the Action team worked with many large employers to design a unique product that is very much like three products in one package.

First, it is a identification and prediction product. Our data scientists, led by Arjun Kulothungan, and epidemiologist Dr. Tara Yun Zhang, have built models that identify employees who need care and predict their behavior based on a variety of data.

Second, it is a personalized recommendations product. Our clinicians, led by Dr. Zach Landman, and our content and behavior change experts,  led by Christa Dahlstrom, design recommendations that are delivered via email and our web application, called Castlight Essentials. Our personalization experts led by Vicki Mar have designed a personalization engine to deliver the right recommendation at the right time via the right channel to users.

Third, it is a benefits engagement evaluation product. Our Analytics experts led by Sunil Ravichandran have designed data visualizations that track the utilization and inform benefits managers about the efficacy of their benefits design.

The identification and prediction module

Sample recommendation

Evaluation of benefits utilization and engagement

I have always enjoyed building things. I consider Castlight Action among my best and most meaningful work. We hope to reach and improve the health of millions of American workers, increasing their happiness and productivity and reducing the cost of healthcare for hundreds of employers, making them more competitive.

There are few things that I am particularly proud of.

1. We followed the principle that the best interface is no interface. So we have designed the product in such a way that we don't need the benefits manager to do anything. In fact they can get most of the value associated with the product even if they never login to the product. Since, from my SAP days, I understand how busy they are, I was particularly keen on not burdening them with one more tool to administer.  This unique design was done by a team of designers led by Phuong Tran.

2. Unlike traditional analytics products that analyze a snapshot of data, we run our predictive models every day so that timely and relevant recommendations can be made to employees.

3. Rather than wait for the end of the quarter or year to provide benefits leaders with feedback about how their benefits design is doing, Action provides feedback on employee engagement with benefits every day to benefits leaders.

4. Our plan is also to provide insights as a service to all health care partners who serve our customers. This insights as a service design is led by Alvin Cadman.

5. While our predictive models have performed well in initial tests, we believe that we will learn more in the next few months and fine tune even more. Same is the case with our recommendations. We are building a completely new infrastructure to track usage and engagement on a very granular level and make that information available to all product managers so that they can improve the product every month. We are investing in some cutting edge technology for this. This effort is led by David Tischler. I might be able to share more about tracking and continuous improvement in about 6 months.

You can read more about Action in the Castlight Health web site. Please note that all images are from the public website of my employer Castlight Health.