Tweet about this on TwitterShare on FacebookShare on LinkedInShare on Google+

In Part 2 of our chat with Tomasz Tunguz, Tom gives his take on prioritizing as a product manager. In a nutshell, PMs should think of making product decisions as if they are running a business, complete with financial modeling.

Watch the interview to learn:

  • Why being sensitive to certain metrics (e.g. churn rate) can help a company optimize its business, as told via an example with NetSuite
  • The importance of identifying which variable(s) influence your business most, and heavily weigh those numbers in decision making
  • How Tom learned from a personal “disaster story” and applied the resulting lessons to future successes

Watch Part 3 to find out how Tom determines impact of a given product. Check out Part 1 if you missed it the first time!

Create Your Free Wizeline Account

Video transcript:

Matt Pasienski: I found personally that's that an extraordinarily useful task because it does bring up those things that you weren't thinking about, and it puts into perspective what is more relatively important. How do you recommend product managers take an approach where they can use that type of comprehensive or even financial planning to get to an understanding of what the world can be like in the future?

Tomasz Tunguz: I think really great product managers think of themselves not as project managers but as general managers. They pretend that they actually are operating a business. One of the things that we never did at Google that I wish we would have done more of is this financial modeling. I learned all of that here at Red Point. What a financial model does, like you said, is it basically forces you to distill how you think about the business and how it works in a very simple way, so that everybody else can understand. The other thing it forces you to do is to write down your assumptions. This has to be true, and this has to be true, and this has to be true, and then there's a result. Then you can go into a meeting, and you can say, "Hey, guys, conversion rate, how much is that going to move, quotas, how much is that going to move? What do we think we can do on pricing? What are the implications there?" People are going to have different points of view, and you can play with those figures. All of a sudden you can see the business is actually much more sensitive to this particular metric.

There's this great interview of the CFO of NetSuite, and he talks about ... It's a publicly traded SaaS company. I can't remember the market cap, but it's $5 or $6 billion the last time I looked. They were really trying to grow their business starting in 2008. They went through this period where the CFO who's really smart ran this sensitivity analysis, and he figured out that churn was the thing that was going to move the growth rate of the business the most for the next few years, so they focused on churn rate. They identified all the different leading indicators of churn, and they basically were able to optimize the churn. Then at some point a few years later, they realized that they couldn't do any more on churn. It was going to be average contract size that was really going to increase the revenue of the business. They needed to move upmarket, which had ramifications for sales, and product and marketing. Then they went, and they focused on that.

Matt Pasienski: I think if you haven't done it before, and you haven't really gone through and modeled out your business, 18 months or 2 years and looked at how all these things interplay, it sometimes doesn't seem like it would be that useful, but the second you do it it immediately jumps at you. You spend 10 or 15 hours playing with the model, and then it just ...

Tomasz Tunguz: You can start out with a super basic model, and then you're going to realize I'm going to make this assumption. I'm going to fill in a number here, but that number actually had 4 or 5 different inputs that I could then model independently. Then you keep doing that, and you keep doing that. As businesses grow, that's really what you want to do because then you really understand which of the variables have actually moved my business.

Matt Pasienski: Once you've done that, you have a great framework for making decisions. You have a great way to come up with the key decisions that need to be made. What are we going to do about churn? How are we going to increase contract size? Now you've got something really concrete to talk about and to plan against, but then it also allows you to be data-driven. It is that ontology for saying this is the way we're going to take all these crazy, different numbers and integrate them into something that allows us to make decisions.

Tomasz Tunguz: When I started at Google, there was a guy named Scott who became my manager. Scott was a PhD from Princeton, and he was at McKinsey for a while. Then he came to Google. He spent the first month creating the equation that governed the business. He had never been in the advertising world before, and he didn't really understand.

Matt Pasienski: What did he have his PhD in, by the way?

Tomasz Tunguz: I think it was aerospace engineering, super sharp guy. He said, "I'm going to take this apart. There's one top-level equation that governs the business, and there are all these different pieces that feed into it. Let me understand what all those different pieces are, but let me also understand the history about why we've made certain decisions about different facets of the product." Starting with that equation to decompose he really understood both the history of the business and the rationale of decisions that we made, but also how the business worked. The most important thing for me and what he really enforced in us was is the project you're proposing going to meaningfully impact revenue because you can see right away whether or not it will. That prioritization and that focus really helped.

Matt Pasienski: Even though it's a PhD in aerospace engineering, they're not that complicated.

Tomasz Tunguz: No, it's a bunch of multiplication.

Matt Pasienski: We're going to have this many users. They're each going to pay this amount, and after a month we'll have this many more because of growth and this many less. It's not hard stuff, but it's incredible how much information you get from going through and simply modeling based on those types of things.

Tomasz Tunguz: The simplification has a lot of power.

Matt Pasienski: I think with SaaS there's a lot of talk about the funnel, and we generally mean by the funnel you're going to have this many leads coming in out of your marketing channels. You're going to qualify them. You're going to have a certain number that convert, and then you obviously get a certain contract value. It's a customer acquisition and growth funnel, but there are a lot of different funnels. Again, this is a way of simplifying your business and understanding the impact of different things. What are the different funnels that people need to be paying attention to across their business in terms of going from something that's unknown or not necessarily set in stone to becoming something that is concrete and beneficial for revenue or whatever?

Tomasz Tunguz: I think every product is a series of funnels. There's the one that we talk about a lot, which is how do you convert a lead to a paying customer? Then there's also smaller funnels. There's a user onboarding funnel. How do I maximize the number of people who sign up to the number of people who ultimately use the product and are daily active users and monthly active users? Then we're launching a feature there's a feature funnel. What fraction of total daily active users ultimately end up using my feature?

Matt Pasienski: Could you maybe give an anecdote of how something like that can actually impact major decisions, if you think about people using your features as a funnel?

Tomasz Tunguz: I'll give you a disaster story. I remember I just transferred from [AdSense 00:06:22] Operations, and my first project I was taking over this project called demographic targeting at Google. This was a product initiative that had been started maybe 6 months before I started, and it was scheduled to launch probably 3 or 4 months after the transition in product management had happened, and I was really naïve. I handled the situation so poorly. It's [hard 00:06:49] for me to say. I can remember walking in, understanding the product a little bit, what they were doing. Basically, what was happening is publishers were telling us this visitor is male and in this age range. The idea was that AdWords advertisers could then say on the Google content network, "I really only want to advertise to only 35-year-old females or something like that. After doing a little bit of analysis I suspected that the fraction of AdWords users who would ultimately end up using this product was very small on a relative basis, single digit percentage.

I walked into the meeting, and I said, "We are canceling this project." I didn't share any of the results. I didn't get any buy-in. I didn't talk to my manager. I just said, "We shouldn't do this," and it was a complete disaster. People were up in arms. The engineers didn't know what to do. Should they continue working, should they not? We ultimately took a step back, and reanalyzed and tried to figure out how do we actually increase the adjustability. Clearly, I handled everything poorly, but I think it did raise the question of hey, we've got this feature. We're going to take 4 engineers. We're going to allocate several man years this product, and there's going to be a bunch of marketing initiatives. There's going to be a bunch of effort.

Matt Pasienski: How do you recommend project managers or portfolio companies go about that process of we're going to ... You talk about being data-driven, having this master equation. You need to have a way of saying, "Is this product going to impact, is it going to put 1% of users?" Is it going to be 100% of users? Is it going to actually go out and triple our total [impact 00:08:36]? These are questions that can be answered. What is your normal method of suggesting to get that?

Posted by on Wednesday, April 20, 2016.


Leave a Reply