Project management tracking through performance metrics to measure team productivity, quality and customer satisfaction

The complexity of managing projects within the Triple constraints has been increasing day by day. Various factors contributing include shortening product development cycles, changing customer expectations, exponentially increasing usage of the internet as well as more millennials in the project teams.

 

Measurement is key to successful project management. As the old adage says, “You can’t manage what you don’t measure.” Collecting and measuring data is at the heart of any worthwhile endeavor. For project management success, you can use project management metrics and Key Performance Indicators (KPIs) to help you strategically meet your business goals.

 

Relevant project management metrics will enable us to improve our understanding by removing uncertainty so that we can make well informed decisions. If a department does not produce or contribute to the measurable objectives of a company, a smart company would dissolve the department and move resources to another area that produces results.

 

Data, metrics, and analytics

Data is information. It’s the numbers and feedback available to you about different aspects of your project. Metrics are how you measure your data. They define the important or specific information (data) you need to know about your project, such as productivity, quality, or engagement.

 

Once you determine your project’s metrics, you analyze the data according to those metrics to find patterns and answer questions about your project. This process is called analytics: using data to answer questions, discover relationships, and predict unknown outcomes. When analyzing data, ask: What do the metrics mean to you? How do you want to use the metrics you’ve chosen? Can you find patterns to make predictions about your project? Can you find ways to improve—or optimize—certain aspects of your project? What lessons can you draw from your project’s data?

 

As a project manager, you can use data daily to make better decisions, solve problems, improve performance and processes, and understand your users.  There are many types of project data you can use to determine your team’s progress and efficiency, evaluate the success of your project, and inform project decisions.

 

For example, if you have data on customer buying patterns, you can identify your best-selling products, and you’ll be able to make smarter decisions when placing new product orders with your suppliers. This data will also help you better understand your users and their preferences so you can improve your product offerings and performance.

 

Project management metrics are an effective way to evaluate the progress of a project. Measuring your project progress against specific factors helps clarify the management process. By following and analyzing the right indicators, you’ll gain key insights into your team’s performance — from both a high-level and individual perspective.

 

You’ll be able to pinpoint bottlenecks in workflows and other inefficiencies that you can then correct, adding to your project assumptions list and improving future project performance. You can also use project team data to help you refine your processes. For example, if your team is experiencing an issue, analyzing data from your project tracker about the number of tasks completed, escalations, or internal process problems can help you find the source. This will allow you to make an informed decision about where to focus your efforts to improve  processes. Through critical analysis, application, and execution, data becomes a powerful tool to guide any project in the right direction.

 

How do you choose metrics?

Each project has different requirements and needs unique project performance metrics to align with its goals. First and foremost, determine your goals and understand the purpose of your project. It is essential to pick the project metrics that are appropriate for your project’s needs and success factors. Based on your project goals, decide on the metrics that help you identify and execute your objectives.

 

Choosing Right metrics

Each business or project requires unique metrics that align with its purpose or goal. There are three steps to choosing metrics:

1. Understand the purpose or goal of the project or work

2. Determine what critical success factors need to be fulfilled in order for you to succeed and achieve the goal

3. Take each critical success factor for the project or program and identify how you will measure its fulfillment

In addition all the project management metrics you use should be:

• Easy to measure
• Easy to understand, and
• Pertinent to your organization and its current goals

Any metric you measure should be clearly and accurately measurable. There are two aspects to accurate measurement:

• The metric you’re trying to measure must be consistent and predictable
• The tools you’re using to measure the metric must have a low error rate

 

Productivity

Productivity shows the relationship between inputs and outputs. How much are you getting out after all that you put into a project? The ideal productivity outcome is creating more for less.

Productivity = Units of Input/Units of Output

 

Productivity metrics

Productivity metrics typically measure progress and output over time. They allow you to track—or predict—the effectiveness and efficiency of your project team. To track your team’s productivity over time, analyze the number of tasks or milestones completed in a given time frame. Ask questions like, what percentage of tasks are completed on time, and how long do they usually take? Or, if tasks were not completed on time, how much longer than anticipated did it take to complete all the tasks?

 

Return on Investment (ROI)

Return on investment specifically looks at the dollar amount earned for the amount invested in a project. Like gross margin, this is a financial equation. Instead of looking at overall profit, it looks at the specific benefit from the project divided by the costs.

To use this metric, a dollar amount needs to be assigned to each unit of data to determine the net benefits—benefits may include contribution to profit, cost savings, increased output, and improvements. Costs may include resources, labor, training, and overhead.

ROI = (Net Benefits/Costs) x 100

 

Relevant metrics in project management can demonstrate the productivity of a team. For instance, the on-time delivery (OTD) rate or service level agreement (SLA) rate can measure the ROI of the project. ROI looks specifically at the value of a project’s outcomes vs. the dollar amount spent to complete the project.

 

On-time completion rates can help illustrate to clients and stakeholders how the project is progressing and when they can expect certain deliverables to be ready. If your project’s completion rates are high, it means you’re doing a good job of meeting your completion goals. If the rates are low, it means you’re missing deadlines.

 

Analyzing data can help you make decisions about things like improving or implementing new processes, or re-evaluating how you estimate project scope, complexity, and timeline. Calculating duration (how long something takes) can be useful for setting and evaluating tasks and milestones and determining if you’ll meet project deadlines. Tracking task duration can improve the accuracy of estimating a project’s timeline. This data is broken down into hours, days, weeks, months, and sometimes years.

 

Planned Value

The PMBOK prioritizes Planned Value (PV), Earned Value (EV), and Actual Cost (AC) as three crucial metrics for measuring project performance.

Planned Value, which is sometimes also called Budgeted Cost of Work Scheduled (BCWS), is a measure of the estimated cost of planned activities at any given time. It’s important to note that this metric focuses on the scheduled value, not the actual value. That is, it tells you the ideal value you should have realized if the project was going 100% on track.

Earned Value

Earned value provides strategic guidance by showing how much value you have earned from the money spent to date on a project. It compares the value of the work completed by a specific date in relation to the approved budget for the project.

Earned value is also called Budgeted Cost of Work Performed (BCWP). This metric provides a reality check during the process of a project.

Earned Value (EV) = % of Completed Work / Budget at Completion (BAC)

 

Comparing Earned Value with Planned Value gives you a quick estimate of whether you’re behind or ahead of schedule.

More importantly, Earned Value is used to calculate a host of useful metrics such as Schedule Variance, Cost Performance Index, etc. Like Planned Value, think of it as a baseline metric you must calculate for all projects.

 

Schedule variance: This metric considers the budgeted cost of work scheduled and work completed. It tells if the project is under budget or exceeding its budget. Earned value is calculated by subtracting the budgeted cost of work completed from the budgeted cost of work scheduled. Negative schedule variance means that the project is running late.

For example, a project with a budget of $100,000 has a duration of 12 months. After 6 months, you realize that only 40% of the project is completed vs the estimated 50%.

Your Earned Value, therefore, would be $40,000 (40% * $100,000).

After 6 months, 50% of the project’s time has elapsed. Thus, the Planned Value of the project at 6 months would be $50,000 (50% * $100,000).

you’re $10,000 behind schedule.

 

Cost or budget variance is the difference between the actual amount of money spent on a project and the amount that was budgeted for the project. This shows the discrepancy between the predicted budget and the actual costs within a specified period. A negative cost variance shows the project runs over budget, whereas if it’s positive, it means the project is under budget.

 

Over time, this data can help you understand how well you’re estimating budgets for your projects. A low variance means you’ve estimated your project budget accurately. A high variance means you should reevaluate your estimation process. You could be under- or over-estimating costs for your budget, or you may not be tracking expenses effectively.

 

You can also analyze current information to predict future outcomes and make projections (or forecasts) about productivity trends, project durations, costs, performance or quality. This kind of data empowers you to proactively manage your project and its resources and measure the accuracy of your projections over time. For example, analyzing your team’s overall performance or velocity can answer questions such as, is the team completing its tasks and milestones? What percentage of tasks is the team finishing on time? Predicting the future may be impossible, but building a better understanding of it and refining your method for making projections is achievable and valuable.

 

Agile metrics

Team Velocity

Team velocity is an Agile KPI used to measure the amount of work that gets done during a sprint. It is typically measured in terms of hours or story points. Project managers use team velocity to calculate how fast development teams can work through a backlog of tasks. This system is particularly useful for forecasting, planning future sprints, and identifying whether the team is dealing with any roadblocks.

 

Burndown

Burndown represents how fast Agile teams are “burning” through their work. Sprint burndowns measure how much work was completed during a sprint. This metric is useful for keeping teams aware of any potential blockers they might face during the development process and allows managers to track team progress toward reaching the sprint goal. Release burndown provides an overview of the release progress. It’s similar to sprint burndown but tracks progress on the project level and allows teams to check whether they’re on track to release the project by a specific date.

 

Commit-to-Deploy Time (CDT)

This metric measures the time it takes to take the code from the commit stage to the point of deployment—testing, QA, and staging, depending on your organization. CDT tells managers how long it takes to move code through the pipeline and helps ID roadblocks if there are any.

Throughput

Similar to velocity, but more granular, throughput measures features, as well as bugs, tasks, and other activities performed during a given time period. The benefit here is that measuring throughput allows managers to get a better sense of the team’s workload and how they’re spending their time.

 

Communication Index

Communication index measures the communication capabilities between an outsourced team and the internal teams impacted by the project. You can start measuring developer communications by performing a subjective evaluation of communication quality across the following areas:

• Social skills
• English proficiency
• Cultural differences

But don’t stop there—make sure to capture feedback from multiple stakeholders—each of whom should perform an independent assessment before discussing their scores with the team.

 

Quality metrics

Quality metrics relate to achieving acceptable outcomes and can include metrics such as a number of changes, issues, and cost variance, all of which affect quality.

 

Changes refer to differences in any aspect of the project from what was originally planned or required. Issues are problems that may affect task completion—and often result in a change. Track the number of changes and issues to identify patterns, refine processes, and share information about the project with stakeholders.

 

Cost of Quality: Cost of quality is one of the most important, yet often overlooked, metrics to monitor. The true cost of quality includes both the cost of poor quality and investments in good quality.  ASQ, or the American Society of Quality, developed the following formula for

Cost of Quality:  Cost of poor quality (COPQ) + Cost of good Quality (COGQ)

COPQ includes internal and external failures, such as:

• Internal COPQ such as scrap, rework and re-inspection
• External COPQ when defects reach the customer, including adverse event reporting, warranty, corrections and removals, product liability and loss of brand reputation

COGQ is comprised of what you spend to create conforming products, including:

• Appraisal costs such as inspection and testing, quality audits and calibration
• Prevention costs such as statistical process control (SPC), quality planning and training

 

DPPM = Defective Parts per Million; A measure of quality performance. One DPPM means one (defect or event) in a million or 1/1,000,000. To calculate, for example, let’s say you had 25 pieces defective in a shipment of 1,000 pieces. 25/1000= .025 or 2.5% defective. 0.025 X 1,000,000 = 25,000 PPM.

 

DPMO = Defects per Million Opportunities; A measure of quality performance calculated as:  DPMO= 1,000,000 x number of defects/number of units x number of opportunities per unit. This metric is more useful when looking at defects in subassemblies, which may have multiple opportunities for failure.

 

Sigma(σ) Level = A statistical term used to measure how much a process varies from perfection and the symbol for Standard Deviation, calculated from the squares of the deviations of measured samples from the mean value. In other terms, Six Sigma would mean the process maintains six standard deviations from the mean value of the process to the nearest tolerance limit.

 

Maintenance Metrics

Maintenance metrics are important leading indicators of quality, providing early warning of when you’re headed for quality issues. Leading metrics to monitor here include:

• On-time completion of scheduled maintenance
• Ratio of planned maintenance activities completed to unplanned emergency maintenance
• Downtime as a percentage of total operating time

 

Uptime

If you have a product that’s currently online and live, how often is that product unavailable to customers? Sudden increases in downtime can point toward issues like aging existing infrastructure or an unreliable cloud provider that must be addressed ASAP as they can introduce security risks, friction, and delays.

 

Customer Satisfaction

Tracking customer satisfaction over time is a clear indicator of whether the team is delivering value back to the end-consumer, and by extension, the bottom line. For mobile app development, you might keep a close eye on user reviews in the App Store and Play Store. Ideally, the average user rating should either increase over time or remain steady. Otherwise, it means users are unhappy with the app experience.

 

Each company can develop a score unique to its business by weighing each variable based on its importance. Variables may include customer survey results, revenue generated from clients, repeat or lost clients, and complaints.

 

The Customer Satisfaction Index (CSI) is the most widely used system for measuring customer satisfaction. The Net Promoter Score (NPS) is another method to capture customer satisfaction. NPS reveals customer loyalty by probing the likelihood of a customer recommending the product or service.

Customer Satisfaction Score = (Total Survey Point Score / Total Questions) x 100

 

Project managers at Google use a sub-set of metrics called happiness metrics that also relate to quality. These are metrics that relate to different aspects of the user’s overall satisfaction with a product or service, like visual appeal, how likely they are to recommend, and ease of use. Happiness metrics can generally be captured with a well-designed survey or by tracking revenue generated, customer retention, or product returns.

 

Customer satisfaction scores reflect user attitudes, satisfaction, or perceived ease of use. These scores measure how well the project delivered what it set out to do and how well it satisfies customer and stakeholder needs. Customer satisfaction scores generally represent a combined metric—the sum of several different happiness metrics. For example, on a satisfaction survey, a customer might separately rate a product’s appearance as 6/10, ease of use as 7/10, and likeness to recommend or use again as 8/10. The overall customer satisfaction score would then be 7/10. You will need to determine what scores are acceptable for your project by discussing with stakeholders what the most important aspects of the project are.

 

Adoption and engagement

Another set of metrics related to quality are adoption and engagement. Adoption refers to whether or not a product, service or process is accepted and used. Engagement refers to the degree to which it is used—the frequency of use, amount of time spent using it, and the range of use. It might help to think of these in terms of throwing a party: your adoption metrics would reveal to you whether or not people accepted the invitation and showed up. The engagement metrics would tell you how active they were at the party—whether they participated in activities or interacted with other attendees, if they invited their friends to come with them, and how long they stayed.

 

Adoption metrics for a product or service release, like an app, software program, delivery service, or gym membership, would be similar to the party example. However, they can be a bit more complex if you need to track metrics for more than one thing, like whether users make additional purchases or sign up for premium features.

 

Each project will need to define its own set of successful adoption metrics, such as: Conversion rates, Time to value (TTV), Onboarding completion rates, Frequency of purchases, Providing feedback (rating the product or service), Completing a profile. Engagement metrics tell you to what degree a product, service, or process is being used. They reveal the frequency and type of customer interaction and participation over time. Engagement metrics might include the daily usage rate of a design feature or tracking orders and customer interactions.

 

As a project manager, you’re not only concerned with the end user’s level of engagement. It’s just as important to monitor stakeholder and team member engagement as well. Measuring stakeholder participation by tracking the frequency of communication, responses to emails or updates, attendance at meetings, or level of input can give you a sense of whether or not stakeholders are finding value in the project. A lack of meaningful engagement could put your project at risk. Stakeholders may not be aware of changes or the overall progress of the project, and therefore the final outcome of the project may not meet their expectations. Measuring team member engagement is vital to the success of your project because the more engaged they are, the more productive they are, and the more likely they are to produce high-quality results.

 

Ideally, you want your adoption and engagement metrics to increase or to at least meet the goal metrics that were established with stakeholders earlier in the project. If there is no increase, or the metrics drop, then your rates are low and therefore not as successful. Check out the resources below for a more in-depth understanding of how and why to measure adoption and engagement.

 

Engineering metrics

1. Lead Time

Lead time refers to the time needed from feature description to feature implementation in the production environment. It’s a great metric to evaluate the smoothness of your engineering processes. For instance, processes that cause a lot of friction, lack ownership, or don’t have a clear description will negatively affect the lead time. The level of automation is another crucial element that determines your engineering team’s lead time. For that reason, it’s a must-have metric to track your overall performance.

On top of that, your lead time can help your product team plan new features and their capacity. It helps them to make realistic feature roadmaps and allows for clear communication to your clients. Also, a realistic roadmap puts less pressure on your developers, which is good for the overall mood and retention. High developer stress is one of the primary reasons why developers decide to look for an alternative job.

 

2. Number of Pull Requests (PRs) vs. Story Points

Some teams prefer to count the number of PRs developers complete per sprint. However, it’s not a great metric to measure because it doesn’t represent your team’s actual velocity. On top of that, it’s not a fair metric as some developers prefer to create smaller PRs, and senior developers will often complete more PRs. The same is valid for counting the number of commits per sprint. It’s not a metric that you can easily compare week by week.

Therefore, it’s better to assign story points to each issue so you can measure the total number of story points you complete per sprint. You can then map this number week by week to get a graph.

Story points provide an effective way to detect technical debt. In other words, measuring sprint velocity is a great strategy to make technical debt more visible. It’s important to quickly detect the on-ramp of technical debt as it has a significant impact on your team’s performance in the long run.

To make technical debt visible, add bookmarks and TODOs to your code, and collaborate with your team on maintenance work, try out Stepsize VS Code and JetBrains extensions.

 

3. Time to Complete a Code Review

You may have many automated processes to deploy your code to a test environment and run tests against it. However, code reviews can still slow you down. Therefore, you should measure the time needed to complete a code review.

Often, pull requests lack clear ownership, or the process lacks clear acceptance criteria. In these situations, teams don’t know when a pull request is ready to be accepted or not. This uncertainty will slow down the review process.

Also, many teams have more experienced developers who take care of most code reviews. They can easily get overwhelmed by all the pull requests they have to review besides their work. It’s a major blocker for development teams who put too much stress on their senior developers. For that reason, it’s important to diversify the knowledge across multiple developers so not only your senior developers are responsible for accepting pull requests.

In short, track who’s doing code reviews and how long it takes to accept pull requests into your codebase. It will help you to understand choking points that slow down your team’s code review completion time.

 

Clear Ownership

When doing a code review, it’s essential to tag the right people to review your code. As a best practice, you want to tag at least one person that has a good understanding of the part of the codebase you’re working on. Moreover, you should tag at least one other developer who knows less about this specific code so they can review if the code meets the style guide and acceptance criteria.

 

 

References and Resources also include:

https://www.stepsize.com/blog/3-most-important-metrics-for-engineering-team-performance

https://www.workamajig.com/blog/project-management-metrics

 

Cite This Article