Infusing Systematic Innovation Into Product Design and Development

When I build new products from concept to launch, I ask my teams to adopt the design thinking process. For example, when I built a product called Castlight Action, the first several weeks of the project was devoted to listening to benefits leaders from companies such as Walmart, and Mondelez to understand their pain points around health benefits administration and communication. The next several weeks were devoted to coming up with a hypothesis, concept story creation, rapid prototyping, concept validation with targeted users followed by detailed definition, design and development. The product was a commercial success when we launched. Hundreds of customers adopted it and it made millions in revenue every year for Castlight Health. In this project, there was a dedicated team of 70 people with a very clear goal of creating a commercially successful product. 

However, when there is no major program with an ambitious goal, applying the design thinking process became sporadic and inconsistent. Thanks to the efforts of Vince Loomba, head of product operations, Castlight Health had a very robust quarterly planning process for development projects. However, systematic innovation that is sustainable, became particularly hard. Innovation did happen during quarterly execution. But it happened in a hidden factory, led by individual initiative, where progress was hard to track and success hard to measure. Product managers were more focused on getting executives and engineering to invest in their ideas rather than identifying products and features that bring value for customers and become a commercial success. When specific functionality was approved for investment in a quarter, designers and user researchers were given limited time and shorter notices to test the concept and do detailed design. Ruta Raju, Castlight’s head of design at that time pointed out to me that innovation and design projects should be treated as separate projects with specific deliverables rather than hurried predecessors of development projects. I agreed with his observation and promised to work on it. 

I also noticed that in some companies I knew, the executive team and product managers decided to build and release major functionality to customers with little or no customer co-innovation session or concept testing with targeted real users. Such products were not used by customers and months of investment was wasted.

When I had an opportunity to rethink the product management process at Jemstep, I requested my colleague Jason Kim, Director of product operations to separate innovation assignments from development assignments. I suggested that, apart from the regular development assignments for a quarter, we have a set of  innovation assignments with specific deliverables that will become candidates for development assignments in the subsequent quarter. I also requested product managers and designers to plan their time commitments for such assignments every quarter. Jason then created detailed and separate project plans for development assignments and innovation assignments. I identified the skills needed and the tools we need to invest in to support this process. Product managers were trained on specific skills and supported when necessary.

A. Quarterly Innovation Assignments

For innovation assignments, product managers and designers worked together to do the following.

1. Create concept stories with low fidelity mockups.

2. Conduct customer co-innovation sessions to validate hypothesis.

3. Create Medium fidelity prototypes of proposed functionality

4. Conduct concept testing with real users 

5. Calculate the  return on investment for large innovation objectives.

6. Review the solution with internal teams including executive sponsors

For example, when we chose to invest in Individual Retirement Accounts for our robo advice product at Jemstep, Laura Lewison, the lead product manager responsible, did a return on investment analysis for the functionality and had the numbers ready for executive review of quarterly investment. She had co-innovation sessions with our customers to validate the need. She also created medium fidelity prototypes that simulated the functionality completed and tested the concept with real users.

Such innovation projects were given a full calendar quarter. They were treated like any other quarterly objective. The key difference was that the deliverables were user-validated concepts with fully developed medium fidelity prototypes. These deliverables and insights were then used to determine if and when we should invest in developing such functionality.

Every functionality we developed was put through a concept validation test with carefully chosen end users. For example, we could choose users in their forties, living in Texas, who have an individual retirement account, and a bank account to validate a concept. All this can be done in a matter of minutes and insights can be derived in a matter of hours. We used UserTesting.com to recruit users and conduct such validation. When possible, product managers met customers and validated the concepts as well.

B. Quarterly Development Assignments.

For Development projects product managers created product definition documents, epics, user stories, acceptance criteria, and test cases. Designers created high fidelity mockups. Engineers programmed the functionality and demoed their progress every week.

Skills required for such an approach

Because every product and capability was required to go through this framework, product managers and designers will have to learn specific skills. The skills required to perform these tasks are customer co-innovation skills, ability to create a concept story with low fidelity mockups, ability to create a medium fidelity prototype that explains the user journey, ability to conduct moderated and unmoderated concept validations using a service such as UserTesting.com, and the ability to write a product definition document.

Challenges a product leader will face

A product leader taking this approach will face the following challenges. 

First, you may not get executive support for such investment. To get such support, convey that such an approach leads to the development of high quality products in a sustainable manner, while significantly increasing the chances of success. Unfortunately, not every senior executive understands the importance of design thinking and innovation. There may be good business reasons for that belief. My recommendation is to adapt the process to suit the company or find another company to work for. In today's hyper competitive world, companies that take this approach will most likely beat companies that do not take this approach.

Second, a product leader may face resistance from product managers who may not see some of these tasks as their responsibilities. Product designers may see this as product managers encroaching upon their area of expertise. These are genuine concerns and challenges that will have to be addressed. In my experience, particularly at Jemstep, my colleagues found a way to make this happen. There may be exceptions, which the product leader will have to manage.

Why was this not done before?

The above framework sounds logical and simple enough. So, why is this not implemented more broadly in more companies? I believe that design thinking, concept story creation and user validation functions were considered very specialized and expensive activities. In SAP for example such research was conducted in separate buildings where users were recruited and paid to participate in person. Testing one feature used to cost tens of thousands of dollars. That has changed significantly with the advent of online services such as UserTesting.com. Prototyping tools have evolved significantly in the past few years. Today, it is possible for a skilled product manager and designer to simulate a final product pretty quickly. Product managers and the design team at Jemstep led by David Ma built medium fidelity prototypes for every feature and validated most features with real users before the feature was presented as candidates for development. I know from experience that this is possible and doable.

Taking this approach requires hard work, skill and commitment. However, this is a foolproof way to build commercially successful products that customers value.  If there is enough interest, I may expand on this topic a bit more in the future.

Designing the machine that makes the machine

In the past decade, I was hired as a product leader to execute multiple assignments in the human capital management, health benefits and financial services industries. Over the years I learned that when an organization hired me as a product leader they not only hired me to design the machine (the product) but also to design the machine that makes the machine. 

Elon Musk famously pointed out that “The machine that makes the machine is vastly harder than the machine itself”. He added "The extreme difficulty of scaling production of a new technology is not well understood. It is 1000% to 10,000% harder than making a few prototypes."

A product leader is hired to design the machine that makes the machine

Based on my conversations with a few CEOs and business leaders who hired me or wanted to hire me, I determined that the most important value a product leader is expected to bring to the table is the ability to define what the machine that makes the machine is, build it and run it. Very few job descriptions for a product leader (Director, Vice President or Chief Product Officer) call this out clearly, let alone define what that is. Even the hiring managers, who were CEOs of small companies or business leaders in a big company, were not able to articulate this very clearly to me. Instead they said one of the following things. These are real scenarios and quotes from CEOs of companies that hired me or wanted to hire me.

1. Scenario 1: We believe we have a good idea to solve a customer need and have smart engineers who have some solutions. We need a leader to architect the product to ensure that it brings value for customers, validate it with customers and then rapidly scale it for hundreds of customers.

2. Scenario 2: We have identified a major opportunity in our industry. We have smart people who have extensive experience in the industry. We are working with two customers to provide solutions. They are excited about what they see. But every project requires our engineering team and client services team to be involved every day. Our CEO, Client Services and Sales can’t agree on what is priority. We think a product leader can step in and provide a roadmap and deliver high quality products reliably.

3. Scenario 3: We have a product that a few companies bought. We poured smart people and money into it. However we are struggling with delivering the product because there are competing priorities for our customers and we have a hard time delivering anything.
   
   
   Scenarios A, B and C are companies usually at the Series A funding stage if they are private. They may be small divisions within a larger public company. Such divisions resemble a Series A company in terms of purpose, autonomy, responsibility and investment.

4. Scenario 4: We have delivered a very successful product. Millions of users are using it. They love it. We grew rapidly and hired many smart individual contributors who are great at defining and delivering features. However, we need product leaders who make the team work together well to serve customers at a scale we are not used to. The processes, skills, tools, artifacts and frameworks that worked for 1 Million users may not work for the number of users we have now. We need a product leader who can bring experience, hire new talent, mentor them, help them develop professionally, develop new leaders, bring wisdom, and execute with discipline in a regulated industry.  

Let’s now look at how a machine could be designed to meet these needs.

Ray Dalio, the leader of the most successful hedge fund in the world said that the role of a leader is to design a machine that can produce outcomes that meet an organization’s goal. He defined the machine as people and culture. I extended that framework a few years back to include what a product leader’s machine for delivering outcomes for a product company might look like. This is based on my experience executing about 4 assignments in the past decade. It has evolved over time.

Figure: The Product Management Machine

This is how a product leader can go about using this framework when starting a new assignment within the company or in a new company. This could be the first 90 day plan for a product leader.

Before accepting the assignment.

1. Determine the maturity level of product management in the organization. Are you the first product leader? Is product management at an experimental stage? Is it valued and respected? The importance of product management for a company may change over time.

2. Understand the mission of the company. For example, the mission could be to ‘democratize financial advice’, or ‘simplify navigating the healthcare system’. Ensure that you believe in that mission and its impact on society.

3. If you are not familiar with the industry or subdomain, learn and get certified in the specific subdomain. For example, a year before I joined Castlight Health, I got certified in data science and machine learning from Johns Hopkins. When I joined the financial services industry, I called a few of my expert friends and learned the domain through a few work sessions. Recently I added some certifications on Financial Technology from Wharton University. Certifications and learning by conversations are effective ways to understand the domain. Now-a-days, you can learn almost any topic from top universities in a matter of weeks. One of the main skills a product leader needs to have is to know how to learn ten times faster than normal. I highly recommend this course that teaches you how to learn.

After accepting the assignment but before starting it.

1. Listen  to engineering leaders to build credibility. Meet them for coffee or over the phone to understand their point of view and challenges. Convey that you are there to serve them well and you value engineering.

2. Listen to client services leaders or user growth leaders and the support team leader to understand their challenges. Since they are responsible for retaining customers, they can give you raw insights into what is happening on the ground.

Month ONE

1. Understand the top three priorities for the year from the CEO or the business leader responsible. For example, when I joined Jemstep, my manager and Jemstep CEO, Simon Roy made it clear to me that delivering the product to the first major bank customer was the top priority. At Castlight Health, my manager and current CEO Maeve O’Meara made it clear to me that delivering the predictive analytics and programmatic marketing product to Walmart and Mondelez was the top priority for the first year.
   
   

2. People: It is important to understand people, their motivations and their character as quickly as you can.  Once you get the right people in the bus, the bus can be taken to the right destination even if it went in the wrong direction for a few days or weeks. But the wrong people in the bus will ensure that you fail. 
   
   

1. Focus on people. Understand their current assignments and individual feature roadmaps. Recognize overlaps on assignments or features without a clear custodian.

2. Understand their aspirations and mental models.

3. Learn about their prior experience and place them in the pragmatic product management framework. For example, some may be closer to customers and not very close to engineering. Some may have deep experience in analytical thinking but not much experience in day to day execution. Some may be good at design thinking.

4. Look at the artifacts they produce.

5. Assess their mastery of the craft of product management.

6. Design a learning map for everyone based on skill levels and gaps.

7. Do not make any changes to people or organizations at this time.

Unfortunately there is no college degree for product management. It is likely that you will have a team or hire a team of people with varied backgrounds. All product managers irrespective of their responsibilities need to have a basic set of skills. For example, they needed to know how to write a user story. Verify the artifacts of even the most experienced product managers. You would be surprised by the lack of current skills. I normally put everyone through Pragmatic Product Management and key product managers through IDEO’s Design Thinking training programs.

3. Process: The purpose here is to understand if there is a commonly understood way of doing things to collaborate with customers or users, define features, design them, define them for engineering and deliver them.
   
   

1. Understand if there is a process for getting things done.

2. See actual artifacts delivered by the product and design teams. A person’s writing pretty accurately reflects their thinking, knowledge, communication skills and attention to detail. Written communication is critical in today’s world of distributed teams.

3. Understand how the product team collaborates with engineering. Impress upon them that they need to partner with engineering and earn their trust to be successful. 

4. Understand the sharing of responsibilities between product managers and designers. 

5. Make  minor process changes if needed. But do not change the process dramatically in the first month.

MONTH  TWO

4. Platform: The product management team and the design team need effective tools to do their work. These tools do not have to be expensive. Many of these tools can be simple document creation tools or spreadsheet based tools. Some will be off the shelf tools. Ensure that everyone uses them and is skilled at using them.

1. Understand tools used for product definition and design

2. Understand the tools used to deploy or implement the product.

3. Understand the underlying technology platform from the engineering leaders.

5. Product Design: You have to be familiar with the product in month 1. However, in month 2 you need to develop a deeper understanding of every persona, every user Journey, and every possible alternative flow. You may have to work with product managers and designers to accomplish this.

1. Review every persona and the job they want the product to do.

2. Review every user journey in the product. Ask the team for the user flow diagrams. Ask them to create it if needed.

3. Review an entity relationships diagram if available with engineering. If not ask senior product managers to create an entities and logical relationships diagram. Train them to create one if required.

4. Visit three top customers or prospects and conduct co-innovation workshops . If possible, spend a day at your customer support team’s offices and listen to 50 calls. I have not done this in my previous jobs. Instead I listened to recordings of calls that my colleagues made at Castlight. However, colleagues who did this in-person told me that it is very valuable. I plan to do this for future assignments.

MONTH THREE

6. Product Strategy: By month three it will be clear if the team is executing towards the company’s goals. However month 3 is the time to take a deep look at data to verify if the current and planned roadmaps bring value to customers. It is also time to verify if the company is getting back some of the value or will get back some of the value in the future.

1. This is the time to ask senior product managers to perform ROI analysis on major functionality if they have not already done so. 

2. This is also a good time to have a workshop with the senior product managers to review if the features under development align with the goals of the company. Shreyas Doshi of Stripe has done a very good job of defining good and bad product strategy. It is worth a read to understand what needs to be done.

Communicating your work to the organization.

It is important to communicate what you are doing as a product leader to those who do the work following your directions and senior leaders who expect you to produce the desired outcomes. I normally send a weekly work plan email to critical leaders and colleagues outlining the main priorities of the week and what I plan to do and who I plan to work with. It serves many purposes. It helps the product leader earn credibility with product, engineering and key business leaders. It keeps the CEO and senior leaders informed about the areas of focus for the week. More importantly it also points out what the product leader is not planning to do. It gives others an opportunity to point out important tasks if they don’t see it. This is hard to do. But very valuable.

Applying this framework

This framework is based on my experience. It may not apply to every scenario. I am certain it won’t apply to any scenario and organization without some modification. I also want to point out that this is a machine that requires continuous monitoring and nurturing. It is not a self-driving car. It needs a driver, the product leader.

Product leaders could use this framework as a starting point when they start an assignment or a new job. Product managers who aspire to become product leaders could use this framework to understand how their product leaders might be approaching their work. CEOs and business leaders considering hiring product leaders (Directors, Vice Presidents, Chief Product Officers) can use this framework to understand what a product leader can and should bring to their organization. If you use it, let me know. I look forward to learning from all of you.

Ideation and Innovation In The Product Design and Development Life Cycle

For the the past 8 years or so, I have been designing, developing and delivering cloud based enterprise software in a quarterly cadence for use by employees and customers of organizations. This model is normally referred to as business to business to consumer (B2B2C) model. I have done this in the human capital management, health benefits and recently in the financial technology industries. In all these organizations, I followed a quarterly development cycle where senior executives and product leaders set the business goals for a year and product managers defined features that need to be developed to meet those goals with quarterly planning and delivery cycles. 

When developing functionality rapidly quarter after quarter, I noticed that sometimes product teams become delivery oriented and lose their ability to innovate. They stop meeting users and customers in a planned and systematic way. Market research and innovations becomes the responsibility of a few individuals. No matter how smart those individuals are, they eventually run out of energy, ideas, and lose touch with the capabilities of the organization. When this happens, product managers who deliver a product, start looking up to senior executives to point them to the next big thing. If they do not hear a clear direction from senior executives, they become demotivated and may leave the organization or become less engaged with the with the mission of the company. Slowly but steadily, talented people leave and the organization fails.

There is another extreme scenario that happens in some startup organizations where there is no successful product yet. In such organizations, every work session and every meeting becomes a new product brain storming session with no frameworks or business processes to listen to the voice of the market, identify the pain of a customer, test the feasibility of a solution, confirm a hypothesis, clarify the details of a solution, pilot it for a customer and build a product in a phased manner. In this scenario, the organization makes poor investment decisions, product decisions and commercialization decisions. The organization either fails or takes a very long time to become successful.

These are not  good results for any organization.

I believe it is necessary for the sustainability of an enterprise software business to incorporate the process of ideation and innovation in the product design and development process. It is also important to tap the creativity and talent of many people in the organization rather than rely on the vision of one or two people in the company. I developed the following framework a few years back to do this. It has worked well for me and has helped me created products that bring multiple millions in annual recurring revenue. Please note that this framework has been tested only in business to business to consumer software models. This is designed to work in an existing medium sized software organization with 100 to 500 employees. Smaller organizations may take a different more nimbler approach. 

1. Proof of concept : This is the stage where executives have a hypothesis about a market need and a product manager or engineer has an idea to build a solution. At this stage, one senior product manager working with a senior engineering architect builds a proof of concept to test the feasibility of the solution. They may show the solution to end consumers and get their inputs at this stage. No formal customer co-innovation or user testing is done. At the end of this session they produce a functioning proof of concept that proves that their idea works. I normally prefer to make this a project to be completed within a quarter. This proof of concept is formally tracked as a quarterly objective. It should take about 25 percent of an experienced and motivated product manager's time and about 25 percent of an engineering architect's time.. That translates to about 200 hours of work in a quarter.

2. Co-Innovation: Once a solution is proven to be feasible with the capabilities available in the organization, it is time to go meet 2-3 customers for 2 hours work sessions. Product managers will have to create detailed concept stories with low fidelity mock-ups to tell the story to customers. usually 2-3 concept stories explaining 2-3 user cases will work. Once they complete the co-innovation sessions with 2-3 customers or prospects, they should be able to define a possible solution and prototype it using medium fidelity mocks. It is important that they prototype almost 90 percent of the functionality to ensure that prototype conveys the user experience and value created for a customer. Co-Innovation sessions require senior product leaders who understand the design thinking process, a user experience designer and a sales executive. The sales executive's role is to guide the team to work with appropriate prospects and customers who are most likely to co-innovate and most likely to buy the proposed solution. The co-innovation session should be led by the product team and not the sales team. If done well, this should lead to a paid pilot project with a customer.

3. Pilot: This is the stage where one customers is willing to pay to test the solution. At this time, it is important to commit to the delivery of the solution as long as the customer signs a contract or statement of work. The project will be considered a pilot only if there is a paying customer. If no customer is willing to pay, then the co-innovation process continues until the product meet the needs of a customer. Even when one customer is using the solution, it is still not appropriate to call it a product, until the solution has been implemented for at least two customers.

4. Minimum Viable Product: I consider any functionality or solution to be a product, even a minimum viable one, only when it meets the needs of at least two paying customers. Once the solution has been implemented for two customers, the sales team can start selling the product to several customers. This stage will require multiple senior product managers  and multiple engineering teams working full time to build features that meet the needs of the market and strategic customers. It is important to note that a product considered minimum viable for one customer may not be so for another customer. this will be the case until the product reaches about 10 customers.

5. Growth: When a product goes beyond two customers, it is time to think about scaling the product for dozens of customers. This is when a platform product manager, usually a senior leader in the team, and a senior engineering architect join the team to build foundations such as configuration tools, reports, scalable infrastructure and data models. This usually takes about on 1-2 product managers working with multiple engineering teams for 2-3 quarters. The skills required to do this are very different from the skills required to build a minimum viable product.

The key take aways:

• It is possible to incorporate an innovation framework into the quarterly product planning and development process to tap into the talent and creativity of many people in the organization in a systematic and sustainable manner.
• Product organizations that recognize the different stages of product design and development and apply different execution frameworks to each type of work will outperform those who don't.

Executive teams that recognize this and invest in such frameworks and people will outperform those who do not do so.

The Anatomy of an Effective Product Definition Document

Imagine a tool that not only organizes your team's thoughts but also becomes the cornerstone of collaboration and debate in product development. That's the power of a well-crafted product definition document. As a Vice President of Product Management, I've seen firsthand how this crucial artifact serves dual purposes. It's not just about gathering ideas; it's a platform for engaging discussions with customer success managers, professional services teams, and the product team itself.

Empowering Product Managers with a Structured Approach

During each quarterly planning stage, I encourage every product manager, especially the newcomers, to craft a product definition document. For those unfamiliar with this process, I provide a structured framework. This framework is designed to:

1. Develop empathy for the user.
2. Garner inputs from customer success and professional services.
3. Foster collaboration with technical architects during the definition phase.
4. Shift the mindset from 'inventor' to 'custodian' of the solution.
5. Encourage the development and defense of a unique point of view.
6. Build credibility in proposing solutions.

Distinguishing the Product Definition Document

It's crucial to understand that a product definition document is distinct from a vision document. Its primary function is to outline functionalities achievable within a planning period, typically a quarter or 3 months.

Key Characteristics of an Effective Product Definition Document

1. Meaningful Title: The title should be clear and understandable across the organization. A well-chosen title, like "Contact Center Agent Experience for Web Chat - Phase 1," indicates thoughtful consideration of the problem's nature and scope.
2. Date: Including the creation and last updated dates provides valuable context, especially when the document is revisited months later
3. Authors and Contributors: This section should reflect a diverse range of inputs, highlighting the collaborative nature of the project.
4. Document Purpose: Clearly articulate whether the document is for collaboration, development readiness, or requires legal or clinical review. This reflects the product manager's awareness of the document's audience and its multi-stage development process.
5. Key Word Explanation: Define key terms in simple language to avoid confusion and facilitate clear, concise discussions. For example, explaining terms like "Sweepstakes" or "Call Disposition" ensures everyone is on the same page.
6. Executive Summary: This is not just for executives; it's a concise overview for all readers, outlining the user persona, their challenges, and the proposed solutions. A well-written summary is often a hallmark of an experienced product manager. The executive summary should explain the JOB the customer is trying to get done. The PAIN they will experience, if they do not have a solution and the GAIN. The gain is the solution and its value proposition for the user. For example, a tenant in an apartment wants to get a broken faucet fixed quickly by explaining the problem to the customer service person of the property manager. In this case the JOB is to resolve the problem fast. The PAIN is wasted time and a flooded apartment. The GAIN: Attaching a picture of the leaking faucet to her text message to the customer service team will explain the problem better and will lead to faster resolution.
7. User Interaction Diagram: Include a persona description, a simple, abstract diagram showing how the user will interact with the product or feature. This visual element can be more effective than lengthy descriptions. Figure: Sometimes all it takes is a hand drawn diagram.
8. Feature Details: Use sentences and diagrams to describe the feature. This section evolves with each work session, often containing links to detailed use cases, mocks or prototypes.
9. Risks, Assumptions, and Limitations: A transparent discussion of these aspects shows a realistic understanding of the project scope.
10. Acceptance Criteria: Define what criteria will be used to verify the completion of work.

Conclusion: The Art of Conciseness and Clarity

The best product definition documents are concise, typically starting as a 2-page document and potentially extending up to 5 pages. Anything more comprehensive is better suited for a prototype or can be broken down into multiple phases. As a product manager, your goal is to convey your vision clearly and succinctly, setting the stage for successful development and collaboration.