Human Expertise Vs Data

In the movie  Moneyball, there is a scene where Billy Beane the general manager of the Oakland Athletics baseball team discusses the problem the team is facing with the scouts of the baseball team. It is a fantastic scene that depicts the problem faced by human experts in todays data driven world.



Another interesting scene is the one where the an economist from Yale, who works as a data scientist for the Oakland Athletics explains the medieval approach taken by most baseball scouts. He also talks about how to look at the problem differently, from a data point of view.



This different points of view creates friction. If you are developing data driven products for any industry, you will face the problem of friction between subject matter experts and data scientists. Unfortunately for human experts, almost all technologies eventually becomes faster, better, cheaper and win in most cases.

Human experts do play an important role in the process of build data driven products and artificial intelligence products. They train the machine to interpret data and take right decisions. Experts play the important role of teaching machines, like a parent teaches a child. No machine in the world, not even IBM Watson, has inherent knowledge. A human has to train it.

Product managers who understand this and product teams that appreciate this reality and work together will build the most successful data products of the future.

What is IBM Watson and how can product managers use it?

IBM Watson is IBM's artificial intelligence technology that can be taught to answers questions posed in simple English. IBM Watson learns from documents, the same way a human being learns by reading. The difference is Watson can learn from millions of documents in a short period of time and keep up with the growing body of written knowledge on a topic.

IBM Watson learns a topic very quickly.
Experts I spoke with said that when provided with access to adequate written material, IBM Watson can answers questions as accurately as an expert in the field 60% of the time. When an expert coaches IBM Watson for 8 weeks on a topic, IBM Watson can answers questions as accurately as an expert in the field 90% of the time.

You don't program Watson. You teach it
There is no programming language for IBM Watson. Product managers and domain experts can simple teach Watson by providing written material and training it.

I think there are interesting possibilities with Watson in the areas of education and healthcare.

Prototypes For Data Driven Products and A new Role Called Data Product Managers

For many years I prototyped my products very early in the design process and used it a tool for conversation and design participation. I have even tried it for home improvement projects. Recently I started building predictive analytics products that help enterprises identify a segment of people, take actions to influence the behavior of those people, track the engagement of such interventions and measure the eventual impact of such behavior change.

For such products I learned, by trial and error, that we will have to build multiple prototypes. These fall into two main categories; user experience prototypes and data prototypes.

1. User Experience Prototypes
You may have to build a user experience prototype for every persona that encounters the product. In our case, there is a professional user who looks at segments of people and takes action and an end user who receives such interventions.

We built one prototype to show the experience of the professional user and another prototype to show the experience of the end user.

There are good tools to build such prototypes. My team uses Axure RP. It is the best in the business. Product managers can develop skills to use this tool. User experience designers and visual designers also developed skills to build these prototypes within days.

Prototype could also be a sales demo tool
Our visual designer built a version of the prototype with the final visual design screens. We plan to use this prototype for early sales conversations with customers and prospects.

2. Data Prototypes
Data prototypes may fall into multiple categories. The first prototype might be to visualize insight from existing data. The second prototypes might be to visualize insights from engagement data. The third prototype might be to visualize the impact the action taken has had.

The purpose of building a data prototype is the following.

a. Ensure that the data you want to visualize exists.
b. Verify the quality of the data you want to derive insight from.
c. Check to ensure that the data is ready to tell the story you want the data to tell.

A team of experts built the data prototype and determined if the insight we want to visualize were meaningful, useful and reliable. I anticipate that this data prototype significantly improved the quality of our design, improved the communication of the design to our engineers and significantly reduced possible errors in the product. I called the experts who built the data prototype and arrived at the final design, data product managers.

Data Product Managers
Building data products might require skills that traditional software product managers may not have. So I created a role called data product managers who have technical skills such as querying a data base, manipulating it and visualizing the data in a spreadsheet.

If you would like to learn more, drop me a note and I will be glad to share what I can.