Product Managers Could Explain Data Science Results In Plain English

One of the key responsibilities of a product manager working with a data science team could be to articulate the results of a data science project in plain English to other team members and stakeholders. I take the following approach to do this.

First, I request the data science team to aim for a small success within three months of work. In collaboration with the data scientists and a subject matter expert, I create a concept story (1) that outlines the specific results we aim to achieve. We aim for modest results in a short period rather than aim for very ambitious results in a year (4).

Second, I sit down and have a conversation with one of the data scientists (2) to understand the results of the data science project. I do this during the research phase of the project as soon as the team reaches the projects desired research goals (3). The data scientist will usually share a data file with the results of the data science project. I normally request the data scientist to point out the top three highlights of the research. We then verbalize the results and convert the results into a plain english sentence in a short work session. For example, in a data science project to match data sets, the plain English sentence might say "We were able to improve the match rate between dataset A and dataset B from about 2000 to about 10,000." I then build on the sentence by stating what it means for an end user. For example, the plain English statement might be "When a person looks at a doctor, she is five times more likely to see a hospital affiliation compared to before."

Third, I provide a screen shot of the application area where the data manifests itself to make the data easier to understand for all team members and stakeholders.

A product manager who takes on these responsibilities in the data product team can play a meaningful role (4). It is also a good way to gain credibility not only with the data scientists but also with stakeholders who may not always have a data science background. It might take 6 months and a couple of successful releases for the data scientists and stakeholders to appreciate the role of a product manager. Don't let that stop you. Keep at it and you will succeed.

_______________________________________________________

1. I might share a sample concept story in a future post, if possible.
2. Experienced data scientists are good at articulating the results achieved.
3. Data science research outcome is later turned into scalable code by a data engineering team.
4. Overstating the scope and impact of a data science project is a common mistake.

Keep Application Teams Posted About Data Science Projects

In some cases the results of a data science project might lead to the creation of vast quantities of useful data. If the resulting data is used by applications, it is necessary to keep application engineering teams informed in advance so that they can be prepared for the increase in available data. They may have to invest in improving their infrastructure and performance to accommodate the new data.

Think of data as water and applications as the hydro electric dam that uses the water to generate electricity. A sudden unexpected deluge of water might overwhelm the turbines. So keep those responsible informed about the possible deluge.

This could be a key responsibility of a product manager working with a data science team.

Data Science Teams Are Twice As Big & Work Twice As Long As App Teams

At Castlight Health, I have worked with  two data science teams that developed multiple data products. Based on my experience, I noticed that in a data product development team, the data science and engineering team is usually twice as big as the application development team. In other words, if you are developing a data product, your invest twice as much in the data science and engineering team as you would in an application engineering team in any given period.

Another important fact is that the data science and engineering team needs to work about twice as long as the application engineering team. Think about it this way. If you are developing a Maps product, the maps application engineering team might be about 4 engineers who work for an year to build the product. However the maps data teams will be about 8 people and will work for two years to create and operationalize the first version of the product.

If you are a product manager involved in planning a data product, this is a good insight to share with your stakeholders and investment decision makers. This of course is a rough idea based on data products in the healthcare industry.

R Code for Reading an XML File And Extracting Data From It

Let us say you use the services of a cloud provider who gives you data in XML format via a web site and you want to periodically look at that data and extract some information to make your decisions. The file is big. So loading it in Excel and manipulating the data is cumbersome and error prone. You can spend a lot of money to build a special software for it or you can write some simple R code to extract the data yourself. This is how you can do it.

Let us assume that the file I am working with is the master data file of 100,000 employees. At any given point of time I want to find out how many employees live in a certain zip code.

Step 1 is to load the web address of the XML file in a vector.
fileURL < - "http://www.website.com/filename.xml"

Step 2 is to load all the content of the XML file in another vector.
documentcontent <- xmlTreeParse(fileURL, userInternal=TRUE)

Step 3 is to parse the root node of the XML content and store it in another vector.
rootNode <- xmlRoot(documentcontent)

Step 4 is to extract all zip codes into a vector.
allzipcodes <- xpathSApply(rootNode, "//zipcode", xmlValue)

Step 5 is to count the number of people who have the zip code "90210".
sum(allzipcodes == "90210")

In 5 simple steps you have performed meaninful data extraction from XML data, which normally requires very sophisticated and costly tools.

To perform  data extraction like this, you will need some basic understanding of XML and some logical thinking. If you are a cloud professional services or an SAP ERP HCM functional consultant, I believe you can perform basic data extraction like the one I described below using R, with a little bit of effort .

R Code for Reading a Specific Portion of an Excel File

Most data sharing in organizations is done using Microsoft Excel. So the code to import Excel data into R for manipulation is a good thing to know. The import function is powerful enough to let you read a specific section of an Excel file and load it into a dataframe..

Here is the sample code.

mydata <- read.xlsx("file.xlsx",sheetIndex=1,colIndex=7:15, rowIndex=18:23)

The sample code reads an Excel file and imports data from columns 7 to 15 and rows 18 to 23 into a dataframe. An R dataframe is a table of data.

Basic R Code For Getting And Cleaning a File

The most basic steps in getting and cleaning data are like this. I am using a data file that has US housing data. I want to analyze that data the same way a web site such as Zillow might analyze that data.

1. First, you have to fetch the data into R. The code for that might look like this. Here you are reading a csv file from your working directory and loading it into a dataframe called housing.

housing <- read.csv("us-housing-data.csv", stringsAsFactors = FALSE)

The sample code above shows how to fetch data from a CSV file in a local directory. Similarly there are functions to fetch data from an XML file, an EXCEL file, a JSON file, and an HTML web page. Once you understand the fundamentals of fetching data, it is only a matter of knowing the function.

2. Second, you may want to remove some rows where data is missing for a particular column. So you create another dataframe which only has the rows where column 37 has some valid data.

cleanhousingdata  <-  housing[complete.cases(housing[,37]),]

3. In the third step you may want to filter that column for a certain condition. In this case, I am looking for homes that are valued at more than 1 million USD. VAL is the name of the column.

costlyhouses <- subset(cleanhousingdata, VAL >1000000)

Once you do these basic steps, you can start looking for answers to you questions in the data. Coming up with questions is another interesting area.