We have Mr. PO with us today. He has over nearly 20 years of tech experience and has recently taken over as the Product Owner for one of the biggest Enterprise products.. Today we have a great discussion on how Product Ownership works in his company and he shares a wealth of his ideas. Previously, he worked as Senior Manager, Product Development for a different product in the same company. He prefers to stay anonymous.
What do you consider the role of a Product Manager to be?
The Product Manager role primarily requires interacting with customers. We are always in touch with customers. We need to understand how usable the product is, we talk directly with the customers about the features, you have to have a mechanism to collect data on how customers are using the product and see if there is anything you can simplify for them so that the workflow is as simple as possible. You basically own the product which means that you need to make sure that you work with product marketing to get inputs on their marketing needs and ensure the message presented to the outside world is inline with your product, work with sales to understand what kind of environments customers have and their needs, to make sure those needs are satisfied.
So what are the tools you using on your job(Ex: communication tools, planning tools, design etc)?
We are currently using Slack, Outlook in Office 365, Jira, and Zendesk.
How do you currently conduct a user research?
Currently we send surveys to the customer. We are lucky to have customers who are invested in our product. We are also trying to see if we can collect data (if customers agree) from our tools directly about the product usage. Our challenge is that our product is on-premise software. So the constraints are very different from that of SaaS.
Currently, how do you decide on what bugs and features to address in each release?
Bugs are chosen based on criticality. If the customer is not able to really move ahead with whatever they are doing, that’s very critical. So we kind of addressed them immediately and provided a hotfix asap. In cases where the bug is not really blocking them, we will identify what kind of priority it is and decide when to really address it. If it is a corner case and there are workarounds, we will try to provide that.
Otherwise, we will give the customer a timeline on when this is going to get addressed. If it is a feature then it depends on whether it is in line with our roadmap. If it aligns to our roadmap then we can give them some time line on when they can get the feature. If it is not in line with our roadmap, then we tell the customer this is not something we can do and will recommend some other source/alternative to achieve their goals.
So when you give them a time frame for when you can expect a feature. How do you communicate? Do you keep them apprised of the timelines?
Yeah, if they are priority customers, they have the highest level of support. In such cases, we actually have regular calls with them including members of support and customer success teams. In those cases, we will be able to communicate the cadence for the requested features during calls. If we are not on regular calls it is usually updated through Zendesk. The customer comes back with further questions and we respond through Zendesk on the expected timeline.
So you have regular calls with high priority customers. Do you have the history of all the communication that has happened with a particular customer when you go on call? Like what are their past conversations with your company, their requests,and their status?
Every customer will have an associated customer success manager. The CSM would know what the top priorities are and maintains that record of what is really needed for that customer. It’s all usually recorded in zendesk or jira.
The CSM may be dedicated for one customer or a group of customers. So her role is to really keep a tab of what is going on with respect to her customers. She’s kind of the customer representative within our organization. So she makes sure the customer really gets whatever is needed.
When you decide on the features and bugs to work on or what direction to take with your product: Do you use any particular tools and models to make the decision?
We use ranking in Jira to prioritize the tasks but the ranking itself is actually based on the judgment from our team. Some of our considerations when we come up with a ranking are: how critical is this task for our customers, how complex is the feature, is it requested by other customers, Is it inline with our roadmap, etc.
So do you keep track of the Slippages in your bugs and features? If so, how do you keep track of that?
We have features we have committed to the customer for that particular quarter. With that, there will be a planning meeting for each Sprint. So if something slips in one Sprint, then we will know what to expect in the next and our position with respect to our timeline commitments.
At the end of every Sprint there is a review/grooming meeting for the next sprint. We pick up tasks that have slipped the previous sprint, review them and add to the current sprint after we determine the reason for the slippage. Coming to think about it, if there is some feature which has not been picked up in the sprint because of other high priority items and there is a slippage, we do not have a system for detecting that on the engineering side.
We review the committed tasks and sync up with customer success team once in two weeks. We also keep the customer success team updated on what we do on the engineering side.
So how is each bug or feature mapped to a customer who raised it?
We have plugins in jira that will help associate the support ticket of a customer with a task in Jira. If there are multiple customers facing the same problem, we have the ability to associate multiple support tickets with a task in Jira. The company name of each customer will also be associated.
When you commit about making features to the customer is the timeframe recorded somewhere?
It is recorded in zendesk currently. We are looking for ways to make it available to the Engineering team too.
Do you track your customers’ usage patterns? And how does it affect your decision-making?
We are currently not tracking the userflow and other patterns. Our case is a bit different as our products are run on-premise. This is definitely on our roadmap though as it will help us understand the features the users prefer and what has fallen into desire.
How do you schedule a release?
We try to communicate the deadlines as much as possible. If a customer is waiting on a critical feature, We communicate with the customers in case of slippages.
Our teams use burndown charts to track their sprints. With the knowledge of past sprints, they assign points for each story that they have to address. A story assigned 2 points would be doable in 2 hours. Whereas, a 5 point story would indicate a couple of days of work. A 10 point story would need an entire week to finish. Since this is a relative scoring based on the difficulty level of different tasks and the post performance of the team, the values lose relevance outside the setting of the current team.
We do our own estimates called ‘Sizing’ at the epic level. Again, we use the progress achieved in past epics to inform us and help us estimate upcoming ones.
So, how do determine how many features would go into a release?
Based on the velocity of each and every team within a product. One of the perennial problems of software development is time and effort estimation as each problem is unique. For previous releases, we know how much of the items of similar size they were able to really pick up. We use that as a benchmark with the current release to come up with a realistic estimate.
Who participates and collects data during customer discovery? And how is it logged? Is it documented assiduously?
We have an established set of customers already. As we have already discussed, we have regular meetings with our customers. During the meetings, we interview the customer on how they use it and what could make it better. The customer might come up with features specific to their usecases. Before we decide to implement that idea, we go talk with more customers to understand if this is a real need and how it can be implemented for everyone rather than that one customer.
How do customer requests like features/bugs flow into your system?
There are many touch points between each customer and our company. Support and customer success add any actionables that they find to Zendesk. Sales engineers add either to Zendesk or Jira. One of the recent issues we had was Sales engineers had some of their requests as emails and we missed them. If a request is added in Zendesk, a corresponding ticket is added to Jira with a specific type set. The Jira ticket is masked open and added to the release bucket.
If the release of the feature is planned within a year, it is put in the quarterly bucket, else it is marked for review in the future. everything else is marked ‘ Won’t Fix’ and those will not be reviewed in the future.
If a request originated from a customer, the ticket will have the customer information associated with it. The info will come from Zendesk.
How many incoming tickets do you see typically in a week?
5 to 10 support tickets come in each week to Zendesk
2-5 issues a week percolate to Jira. 0-5 are chosen to work on.
How do you handle the tickets?
ERT(Emergency Response Team) immediately responds to burning requests without affecting the sprint. They follow a kanban approach, with a list of tickets ordered by priority. They start churning the tickets from the top. This keeps the Sprint from getting interrupted.
The Dev team follows a Scrum methodology. They pick up bugs and feature requests on biweekly sprints and see them to completion.
ERT is best suited for teams that are good at self-directing.
How many issues are open at any point in time?
Bugs/Field incidents are addressed immediately. They are considered high priority and any week max 2 or 3 issues will be filed.
When a new release comes out, it is tested in the local environment. Once satisfactory, it is deployed to a few initial customers, with not too much of a complex setup. It is ideal to release to one customer at a time. We have a team on hand to address any burning issues immediately. Inevitably, quite a few bugs will be filed. These bugs are squashed first and then it is ensured that the initial customers are settled before it is deployed to a wider customer base.
When a new release happens, we can expect 10-20 issues a day. Both the ERT and Eng team will be deployed to handle this on top priority.
On the other hand, the number open feature requests will vary from anywhere between 200 to 1000 open requests for our products. Any working large product would easily have 100-400 open feature requests.
How many features do you cover in a release?
It depends on the size of the product and the teams. One of our bigger products has 8 teams. The product is in the growth phase. The customers are also bigger. We typically address 10-15 big-ticket features in a release. We will also add around 50 low hanging tasks to be completed in a release.
On the other hand, another product of ours has been quite stable for sometime. So we need only a single team to handle this. We address 3-4 big-ticket features and around 20 low hanging issues in a release.
What is your burning problem?
Time, or the lack of it. I have to work with the marketing team on the message. I also have to work with the sales people educating them on the capabilities of the product with relation to the external ecosystem so they can sell effectively. I have also been involved in many of the engineering meetings to gain a deeper understanding of the product. All of these are causing a strain on my time. Now that I have a good understanding of how the engineering process and the product, I can take a step back and focus more on product, marketing and sales.
Are there any other issues that you handle day to day?
We get to meet a lot of customers and we have a steady stream of great ideas coming in. But once in a while Customers come up with really great ideas but they are out of scope of the current roadmap. It is really difficult to go back to them to tell them it cannot be done now. A counterpart of this is when customers come up with ideas related specifically to their environment. Saying yes would lead to lots of feature creep.
You have been a product owner for 4 months now? What has been one of the most rewarding moments for you?
It is always a gratifying experience to deliver a solution and see it is working, helping a lot of customers out there. The cherry on top is the feedback from the customer saying that the product really is fitting well in his environment and solves their problems.
I’m currently working on a new sister product. The customers have seen the demo of the beta release. They are happy about how it is developing. They want to try it now though we are still in testing. We want to make sure the SLAs are met before giving it to them. Actually. They are really looking forward to using it.
But currently, my worry is whether we will be able to satisfy all their expectations.
And that’s a good problem to have.