R&D performance measurement and benchmarking

Rajesh Uppal December 16, 2022 Industry, Management Comments Off on R&D performance measurement and benchmarking 394 Views

There are many deﬁnitions of technological innovation: “technological innovation is a process which includes the technical, design, manufacturing, management and commercial activities involved in the marketing of a new (or improved) product or the ﬁrst use of a new (or improved) manufacturing process or equipment”

A national system of innovation, “is that set of distinct institutions which jointly and individually contribute to the development and diffusion of new technologies and which provides the framework within which government forms and implements policies to influence the innovation process” (Freeman, 1995).

The backbone of an innovation system is the business sector, which “is the central axis of the process of transformation and innovation”, while a supporting structure of organizations such as universities, laboratories, and policy-related institutions contributes to the ongoing transformation (Teubal, 2002). RTO are organizations in the supporting structure.

The core mission of Research and Technology Organizations is to harness science and technology in the service of innovation, to improve quality of life and build economic competitiveness” (EARTO, 2015).

The main focus of RTOs in three terms:
1. Tackle the needs of industry for knowledge-related services.
2. Focus on user or problem-oriented research for the benefit of society.
3. Assume some of the risks of industrial innovation, helping companies go beyond what they would be able to do alone.

RTOs support the innovation process by engaging in activities. Below are examples of the activities that RTOs engage in.

Ideation. Ideation, technology foresight.
Research: Basic research, applied research, contract research.
Development, demonstration and production: product development, prototyping, proof of concept, compliance, technical support, pilot projects and plants, production process development.
IP Protection and Commercialization: patenting, IP portfolio management, licensing, commercialization.
Business and market development: entrepreneurship and startup support, market identification, business model innovation.
Collaboration and networking: convening interest groups, networking sessions, university collaboration, faculty exchange, researcher exchange.
Funding: grant funding for projects, seed funding, subsidized services.

A traditional classi- ﬁcation of R&D activities distinguishes between (Chiesa, 2001):

(i) basic research, which is “an activity aimed to generate knowledge related to the working principles of natural and social science without direct relation to industrial applications (products, services, production processes)”;

(ii) applied research, which is “aimed to the production of knowledge required to deﬁne the means to fulﬁl a speciﬁc and explicit need”;

(iii) development, which consists of “the systematic use of knowledge oriented to the development of materials, methods, tools and systems”

Research and development (R&D) was once considered to be a unique, creative and unstructured process that was difﬁcult, if not impossible, to manage and control. Today, considerations about R&D have changed, although it is generally recognised that it is challenging to establish accountability for many R&D activities. The evaluation of R&D is thought to be a complex task mainly because effort levels are difﬁcult to observe, success is highly uncertain, typically inﬂuenced by unmanageable factors and it can be assessed only after long delays.

Metrics that measure the effectiveness of ongoing R&D have been difficult to obtain. Four types of R&D metrics have traditionally been used:

R&D Investments / Expenses
- Total R&D Headcount
- Total R&D Expense
- R&D Expense as % of Revenue
- R&D Expense increase/decrease from prior year
- R&D Expense compared with peers/industry average
- …
Project execution status
- Performance relative to plans (costs and schedule)
- Concept to Market Time
- Number of Projects in the Pipeline
- …
Historic results-based Metrics
- Fraction of Revenues from New Products
- Number of Patents generated
- Number of Papers Published
- Customer satisfaction with new products
ROI-based metrics
- Return on Innovation Investment
- Target NPV for each new product

The top five metrics fall into two categories, “input” and “output.” The first two metrics are input metrics, “R&D spending” and “R&D headcount.” Each year, management determines how much to reinvest into R&D. R&D spending is typically measured as a percentage of sales. Some industries reinvest 1 to 2%, others 20 to 25%. The right amount depends on one’s industry, the company strategy, and what the competition is spending.

Headcount is a key metric for R&D. In operations, the unit of production capacity is generally “a piece of equipment assisted by people.” In R&D, however, the unit of production capacity is generally “people assisted by a piece of equipment.” An R&D organization’s capacity is best measured by its people. Appropriately, these are the top two metrics.

Product releases are big events in every company and nearly all employees have responsibilities associated with new product releases. Product releases start the meter for counting the revenues associated with new products, in support of the third ranked metric.

The problem with most of these metrics is that they are not actionable. As we discussed earlier, R&D effectiveness needs to drive real business results. It is hard for managers to take concrete actions using these metrics to improve R&D effectiveness.

Engineering utilization, productivity and throughput are among the most important metrics for measuring R&D performance. If you want to improve your R&D capability, focus on these three metrics.

Productivity and utilization directly determine throughput, and throughput is the most important of all R&D performance metrics. It measures the rate at which an R&D team develops production-ready products. The higher the productivity and utilization, the higher the R&D throughput. Higher throughput means the R&D team churns out more products in a given period of time. That usually translates to revenue and profits— assuming the rest of the enterprise pulls its weight. Big assumption, right?

Every R&D organization should constantly be boosting throughput, and the quickest way is to increase utilization. Utilization measures the percentage of the engineering workforce’s time spent on revenue-generating activities. It’s the percentage of time it spends on developing products – those intended to generate revenue. Here’s the key point though—only tasks directly contributing to a specific product’s development qualify as revenue-generating. All others are non-revenue-generating, and time spent on them reduces R&D utilization. Although there are “gray” areas, these activities typically include attending trade shows and conferences, pre-and post-sales support, corporate improvement initiatives, paid time off, etc.

In a nutshell, anything that diverts resources from revenue-generating activities reduces utilization. According to PRTM, a top management consulting firm, the utilization level of best-in-class semiconductor companies is around 73 percent. In contrast, many organizations are in the 40 to 50 percent range.

Boosting utilization is a two part process, the first of which determines the percentage of time the R&D organization truly spends on revenue-generating activities. Management consultants accomplish this fairly quickly via interviews and surveying the R&D organization. After tallying the numbers, “resource leakage” immediately reveals itself, and this enables the executive team to take appropriate action.

Annual planning. Understand how well your R&D function is performing in its current state and prioritize opportunities for improvement.
Functional transformation. Identify next steps for critical areas and plan a long-term path to your desired maturity level.
Strategy execution. Ensure alignment of strategy and resources with market opportunities and business needs. Save time acting on your plans with recommended resources and guidance from a service partner.

OPEXEngine launches breakthrough technology R&D benchmarking solution

OPEXEngine, the world’s leading SaaS benchmarking platform, today launches its new R&D benchmarks solution. The benchmarks help technology executives invest in R&D with more confidence by comparing their own costs and performance with similar organizations.

New R&D benchmarking solution at a glance:

Benchmarks for 65+ R&D metrics
Comparisons of direct hire vs outsourcing models
KPIs for resource allocation and engineering execution (including speed, quality, and efficiency)
Available as a standalone solution, or as part of OPEXEngine’s BenchmarkEngine™platform
The new benchmarks have been developed in conjunction with executives at leading technology organizations and build on OPEXEngine’s best-in-class analytics, which are used by CFOs, investors, and leadership teams at many of the world’s most important SaaS companies.

With development tools and engineering practices being continuously improved, these new R&D benchmarks will inform organization investments in R&D, where there are often challenging trade-offs between costs, speed, and efficiency. The solution helps technology organizations make well-informed planning and strategy decisions, give themselves a common discussion framework to rally internal teams, and find opportunities to drive operational excellence.

“An efficient and effective R&D process is increasingly the critical factor to determining the long-term success of technology organizations,” said Lauren Kelley, Founder and CEO. “Our new solution extends the insights we already offer SaaS businesses, and means that every technology organization’s leadership team can benchmark themselves to their peers and to market leaders. With highly relevant and recent data, they can inform every strategy and investment decision with the best possible context.”