Nabeel Hyatt, Founder and CEO of Conduit Labs has an interesting post that examines how to predict the success of a Facebook app. Essentially, he looks at the ratio of two metrics made available by Facebook, Daily Active Users/Monthly Active Users (DAU/MAU).

I love it when people dig into the data to find answers - even when they are slightly off like this case! Nabeel's point is that a high DAU/MAU ratio predicts strong DAU growth (DAU counts is how Nabeel quantifies app success) as indicates by the high R-squared value.

Of course this is a bit of a self-fulfilling prophecy as DAU is used to compute the DAU/MAU ratio (the term of this bias I believe is called autocorrelation). Not to say this is a useless exercise. In fact, every entrepreneur should engage in a form of this exercise. The better you understand the key variables that affect your success, the easier it will be for you to devise strategies that directly impact your sucess.

In this case, I would like to see DAU or DAU/MAU regressed against app revenues. I think a person may want to incorporate a lag in those variables). The best thing you could do it collect a wealth of different metrics and use statistics to find the most important ones. I did something similar when I worked at THQ. I built a dataset that helped me identify the key factors that influence a video games revenue and unit sales success for the Kids fighter game genre. The process still can be applied to any situation, but I think it could be very valuable for web-based games where data is very abundant.

I think this would be a really interesting exercise. If anyone has a revenue-generating Facebook app out there and is willing to collect a dataset for me, I would be willing to try to identify the key determining metrics for you. No charge - I just need to know you are willing to put in the time to collect the data. Let me know in the comments.

Henry Blodget's post on display advertising brought me back to a post I made over a year ago. You need to go down to the 6th from last paragraph to hear my thoughts, but essentially I was criticizing the recent movement saying that Facebook display advertising was a failure because of low click-thru rates. I urged people to consider the "awareness" benefits of such advertising. Henry's post highlights some research that appears to support my assertion. Feel guilty for tooting my own horn, but it is nice to see some research examining this important area.

While the value that the Annual Revenue Growth (ARG) from New Products returns is a good indicator about the success of an innovation program, in conjunction with the other data derived in its calculation it can become a strategic tool. Here is a suggest best practice on how to use this metric.

1. Evaluate the Health of the Core - By deriving Annual Revenue Growth and  Annual Revenue Growth from New Products, you can calculate Annual Revenue Growth from Core Products (I = G - H from last post's example). This number will give you an indication of the health of the business's core product line (often the products that brought you to the dance). There are 3 categories for Core Product health:
   • Growing (>20%) - The core is still in growth mode. Like most growing things, it needs support so focus efforts on enhancing the core products.
   • Hitting Steady State (0%-20%) - Core growth is maturing and heading to a steady state level (often between 0%-5%). Now is time to begin thinking about moving into product or market adjacencies to fuel new growth.
   • Declining (<0%) - Core has begun to shrink usually as the result of some exogenous shock. If not already underway, New Product development into product or market adjacenies should be a priority. If the shock is expected to be permanent other strategies may need to be considered such as harvest/exit strategy or refocusing the core.
2. Determine Your New Product Growth Needs - Once you understand and predict how your core will grow next year, you can set a New Product Innovation Strategy. If your core is:
   • Growing - You probably don't need much revenue growth from New Products. So focus on building out your core products, but you should be slowly doing research on potential adjacent markets to enter when growth slows.
   • Steady State - Two good options exist: 1) segment your customers; and 2) move into adjacent markets or products. Segmenting your customers allows you to identify which segments are growing and find new needs and uses for your product that you haven't considered. If your core is well rounded, another strategy is to identify new adjacent markets to offer your existing products or services; or identify new products/technologies to offer your existing customers (customer  segmenting helps identify this).
   • Decline - Not only does your New Product Growth need to be strong, it needs to offset the declines from your core. Note as mentioned above, if this is viewed to be a permanent decline you will need to determine a strategic response such as harvest/exit strategy or refocusing the core.

What Should Your Target ARG from New Product Be? This will really depend on what stage your company is in. Obviously a more mature company would expect more modest growth rates compared to a new start up. So assuming we're talking about a mature company, the first question is what is the overall ARG that the company expects to get in a sustainable steady state? Chris Zook in his great book, Beyond the Core, references a US study in the 1990's that found that public companies that grew revenue by <5% saw a return to shareholders of 4.1% per year; and companies that grew between 5-10% per year, saw a return to shareholders an average of 12.1%! So let's say our target ARG is 8%. Next we ask, how do we expect our core to grow in the future. Maybe a healthy company sees a 4% Core ARG. That leaves us with 4% ARG from New Products (note there could be other sources such as acquisitions).  Now we have very visible metric targets for growth from our core and new products. If a company sees that its new product growth is falling to 2%, then it immediately knows that it has an issue with its new product development and should be doing a more thorough analysis. The ARG from New Products is a great metric for innovation because it leaves no doubt to its interpretation.

In my last post, I talked about the common innovation metric Percentage of Revenue from New Products Released in the Past 5 Years and discussed it was a poor metric because it was not actionable or had a common interpretation. I would like to propose a modified metric that will eliminate these problems. To recap the Juicy Analytics blog pointed out that the four dimensions of a good metric are 1) Actionable; 2) Common Interpretation; 3) Accessible, Credible Data; and 4) Transparent, Simple Calculation. By taking the data from the percentage of revenue from new products, one can construct a new metric that is Actionable and has a Common Interpretation: Annual Revenue Growth from New Products. What this metric attempts to do is to separate the annual revenue growth that came from new products (released in the past 5 years) and the revenue growth that came from the core (products release more than 5 years ago). The formula relationships is:

% of Annual Revenue Growth (ARG) = %ARG from New Products + %ARG from Core

Let's review the four dimensions and see how this metric will stack up:

1.  Actionable: While you will need to do some digging to better understand the source, the results are very consistent - if the %ARG from New Products increases that is good and if it decrease that is bad. A definite improvement from percentage of revenue from new products.
2. Common Interpretation: Once people understand the metrics construction, it can be painfully clear what the metric is telling you.
3. Credible, Actionable Data: Data is sourced from your financial system, so I hope that it is good data.
4. Transparent, Simple Calculation: As I show below, the metric is slightly more difficult to construct, but once you understand the concept it is simple to construct.

To construct the metric: To demonstrate the metric consider this example. A company has the following financial results in 2007 and 2008

		2007	2008
A.	Annual Revenue	2000	3000
B.	Revenue from New Products Released in the Previous 5 Years	500	1200
C.	Revenue from Core Products (C=A-B)	1500	1800
D.	Change in Annual Revenue (D=2008A - 2007A)		1000
E.	Change in New Product Revenue (E=2008B - 2007B)		700
F.	Change in Core Product Revenue (F=2008C - 2007C)		300
G.	Annual Revenue Growth (G=D/2007A)		50%
H.	Annual Revenue Growth from New Products (H=E/2007A)		35%
I.	Annual Revenue Growth from Core Products (I=F/2007A)		15%
J.	Percentage of Annual Revenue from Products Released in the Previous 5 Years (J=B/A)	25%	40%

Using the ARG from New Products metric it becomes clear where a companies revenue growth is coming from. An added bonus is that you also compute ARG from Core which in the next post on using these metrics can be quite insightful.

A common metric associated with measuring innovation is the % of revenue from products released in the past five years. The theory behind the metric is that a company with a strong percentage of revenue coming from new products is innovative. I believe this metric is very dangerous for a company to rely on. The Juice Analytics blog notes that the four key dimensions of a good metric are 1) Actionable; 2) Common Interpretation; 3) Accessible, Credible Data; and 4) Transparent, Simple Calculation. *Source: Juice Analytics Blog The problem with the percentage of revenue from new products is that it is not actionable and doesn't have a common interpretation. Let's say a business that moves from value of 10% to 20% over the course of a year, most people would say that this company improved it's innovative capacity over the past year. This is not necessarily true. What if, over that year, the companies revenues dropped by 20% and revenues from the new products dropped by 5%. In this case the Percentage of Revenue from New Products would increase, but I doubt we would say the company is improving innovation. Therefore, if the metric goes up or down, it is not clear if this is good or bad. That is why this is a dangerous metric to use to evaluate innovation efforts. In our next post, I will propose a slight variant on this metric that will eliminate this common interpretation weakness.

Measuring innovation presents a conflict of interest. If you build a measurable process around innovation are you not destroying the openness needed to fuel innovation? Maybe so, but when trying to instill a culture of innovation in a large, successful corporation it will be hard to change anything that isn't measured. So a CEO has two options: 1) Let the final performance of a business unit be the ultimate judge of successful innovation or 2) Find some metrics that will provide indications of how well a business unit is executing in all phases of innovation.

The first option demands that business units meet a threshold level of Revenue and profitability growth (GE sets it at 8% per year) . The problem is that a CEO will never know there is a problem until they miss their numbers. After heads roll, getting the business unit back on track will be a 2 - 4 year process. I know of few CEOs who could survive such a setback. The second option tries to find a delicate balance of measurement without dictating process. Measuring innovation provides a second challenge that most innovation will take multiple years to come to fruition (often with losses in the front end). As Robert Louis Stevenson said, "Don't Judge each day by the harvest you reap, but by the seeds you sow."Any interpretation of innovation metrics must keep this statement as the overriding context. A fair innovation metric exercise should try to incorporate a balanced scorecard approach - there is no dominant innovation metric. Try to look at innovation in multiple phases. I propose Inputs, Process, Execution, and Value Creation.

• Inputs - Are you putting sufficient resources and generating enough ideas to fuel your target growth (see my post on innovation math)? Track the number of ideas you generated, how much resources you had dedicated to innovation (not evolutionary product developement), and what types of opportunities do you have in your pipe (is your pipeline evaluating enough ideas of a sufficient market opportunity).
• Processes - How quickly are you processing ideas through each stage of your pipeline from early evaluation through to launch?
• Execution - How many launches have you had this year? How many launches attack large market opportunities? How accurate have your forecast efforts been?
• Value Creation - What is the contribution margin of your recent years' launches? What percent of revenue growth is contributed by innovation projects?

You can find the best measures for the four headings based on your company environment. The goal of these metrics is not to judge the success or failure of a business unit. It needs to be a collaborative effort to help business units identify any red flags in their innovation processes so that they can course-correct before it becomes a big problem.

Josh Bernoff of Forrestor's Research posted a blog that social applications will thrive in a recession. I have no problem with his conclusions, but he makes a controversial statement that awareness advertising is ineffective in a recession and that marketers should consider advertising that targets the consideration phase with which I take issue. I will argue that awareness advertising is precisely the type of advertising a marketer should pursue in a recession. Let's start with Josh's statement about awareness advertising:

Advertising (or as we often call it, "shouting") is mostly about generating awareness and reinforcing brands. In a recession, ordinary consumers like you and me aren't as willing to spend. Sure, we'll be aware of the product, but that doesn't make so much difference when you're worried about your future.

and let's see his point about the consideration phase:

You may resist advertising if your finances are tight, but if your bud tells you that new movie is really worth seeing or that the Gap has the cutest new tops, that's more persuasive than advertising. Basically, in a recession, the consideration phase is more important than awareness

Did you see the problem here? In the awareness comment he notes that in a recession that consumers are not willing to spend, then he promotes consideration phase advertising to encourage purchase. The problem is that the pot these ads are competing for are much smaller than usual and definitely much smaller that it will be when the economy picks up. Now, I am not saying consideration advertising is not important - you still need to sell and friend recommendations are important - but, many consumer decisions in a recession are made based on price and value. In a thriving economy, if we get 3 friend recommendations for restaurants, we may actually go to all three in a month. In a recession, we may not even go to one. My key point here is that in a recession immediate conversion should not be your primary goal; Awareness should be your primary goal.

In a recession, it is less about what consumers buy today and more what will they buy when they start spending again. Research shows that in markets with a great deal of choice where the product is not a commodity consumers tend to buy recognizable brands. What better time to build brand than in a recession. As Josh observes, "In the 2001 recession, US advertising dropped 9% and Internet advertising plummeted 27%". A drop in ad spending means a drop in ad prices as less companies are bidding up the prices. This makes awareness ads relatively cheaper in a recession than in a boom cycle. Therefore, awareness advertising in a recession has two compelling benefits: 1) It is relatively inexpensive to build brand recognition; and 2) it will improve your brand's positioning for when the economy recovers.

If we agree that awareness advertising is important in a recession (and I hope I persuaded you), then I want to comment on challenges with awareness advertising on the internet. The internet has allowed granular level tracking that traditional advertising never could. This has made advertisers and analysts in this new age focused on click-thru rates or some other type of conversion. I remember hearing Jason Calacanis in an episode of Steve Gilmor's The Gang state social networking ads are no good because they don't convert! It is a fascinating comment that is shared by many people, but I would like to examine the underlying assumptions of the comment. The comment assumes conversion is the only successful result for advertisements. Search got people focused on this, but I argue that it is incorrect. Why does Pepsi spend so much money on TV ads? Why do people buy space on bulletin boards in Times Square? Why does Starbucks invest so much in ensuring every Starbuck's location has the same look and feel? These do not directly cause conversions. What they do is build brand that raises the chances their good will be selected over others when people make purchases.

This article is not meant to discuss the power of brand, but I ask that you take it as true. Now, I ask if building brand is so powerful, why do ads on the edges of Facebook must have click-thru's in order to be considered successful. Often (brand) awareness advertising just wants to get impressions. Expose consumers to enough impressions and sales should increase. They do it all the time for TV ads. Proctor & Gamble would calculate the frequency of an ads appearance multiplied by the % of the product's target audience that would be exposed to the ad to get Gross Rating Points. They would use these GRPs to predict future sales. I believe that CTM ads on sites like Facebook need to be considered in the same manner. Accept that they are not made to convert, but to build brand awareness (a potentially more powerful effect).

The great news is that the cost of these ads are incredibly low, thus providing a great source of value to a smart marketer. Last point is to remember that the style of awareness advertising should be different from the style of conversion advertising. You are just trying to reinforce brand image - it is a subconscious effect. Conversion's demand engagement from a consumer. Different styles for different goals - be aware. I appologize for the length and scope of this article, but I strongly feel it is an important discussion to have. I encourage other marketers to share their thoughts.

UPDATE: February 22, 2008 - I just saw a Feb 14th post by Aaron Wall who also advocates that companies need different strategies when advertising on social media sites. He suggests that companies find sites that relate to their brand and co-brand with them. I agree this is another strategy that companies can look at. At the heart of it, it is trying to build brand awareness. Great post.