7 Key Performance Indicators When Testing PLM Solutions

Evaluating and testing PLM solutions require a clear understanding of business objectives, processes, levels of automation of enterprise platforms and interfaces. Key performance indicators (KPIs) are used to measure and track success. Success is defined in context of given critical success factors (CSFs) which describe business expectations and high-level requirements in tangible terms.

KPIs are used in context of business benefit realization; they are also used in context of enterprise solution implementations, tracking delivery progress and quality. Business value and delivery quality are two different things, both important, interconnected, but not to be confused. Like with any balanced scorecard, KPIs combine elements of strategy, execution, support and other functions. When it comes to testing PLM solutions, there are KPIs to consider when tracking progress toward verification and validation of enterprise solutions, ahead of their rollout to the business.

In this post, I elaborate on 7 KPIs to consider when testing PLM solutions (non-exhaustive list).


In project management terms, “testing” is a subset of quality control (PMBOK, PRINCE2 and other practices will support this principle). Verification and validation refer to different levels of testing, checking, double-checking, confirming, etc.

Adding a level of nuance, IEEE-STD-610 defines “verification” and “validation” as follows:

  • Verification: “a test of a system to prove that it meets all its specified requirements at a particular stage of its development.”
  • Validation: “an activity that ensures that an end product stakeholder’s true needs [a.k.a. requirements] and expectations are met.”

In software development terms, verification refers to "building the product right" (i.e., ensuring the solution is built as designed), while validation refers to "building the right product" (i.e., ensuring it is meeting expectations and delivering value).

There are different types of testing activities: unit testing, functional testing, non-functional testing (performance, usability, reliability, fail-over, etc.), branch testing, integration testing, end-to-end testing, stress and load testing, boundary testing, branch testing, smoke testing, bug testing, regression testing, etc. (non-exhaustive list).

Testing PLM solutions refers to more than just “software testing”. PLM platform, apps and other tools are COTS solutions, hence prebuilt. It is important to check that they are “fit-for-purpose” and configured / customized / integrated per a given scope and as per agreed business storyboards (tailoring to a given organizational context without compromising future upgrades or changes). Hence, testing PLM solutions involves elements of verification, combined with throughout validation; it is not about testing each and every out-of-the-box unit testing, which is done upfront by vendors.

Typical KPIs for testing PLM solutions include:

  1. Defined “minimum viable product” related use cases with traceability to associated met / not met requirements: it is not a matter of testing each and every out-of-the-box capabilities, but to focus on step-by-step capabilities and processes which will be formally applied / used so satisfy a given set of business requirements.
  2. Tangible (and sufficient) business benefits, with associated rationale: there should be no doubt about the value that the solution is to deliver; focusing on verifying and validating the PLM solution.
  3. Quality of business storyboards, strategic alignment with the PLM platform “good” practices; deviations must be accordingly justified and approved by knowledgeable solution architects and business analysists.
  4. Decision tracking metrics, with risk and impact assessment against the relevant elements and implications (quality, timing and cost).
  5. Number and type of open issues, incidents, defects, bugs, etc. based on severity and importance (impact in the wider enterprise vs relative business context, including reducing possible “emotional” implications).
  6. Triage and response time in addressing issues and implementing corrective actions and workarounds.
  7. Severity and impact from potential showstoppers and blocking issues, remediation and associated risk mitigation plans.

What are your thoughts?


This is an original publication by:

Momentum PLM and written by Lionel Grealou


Share this:

A profile picture for Lionel Grealou

Lionel Grealou

31st March

Insight

Latest Jobs

Architect, Windchill PLM

West Chester, PA, USA Competitive Permanent

Posted on behalf of

Johnson & Johnson

About the job Job DescriptionJohnson & Johnson Technology (JJT), the Enterprise Technology group supporting all business functions under Johnson & Johnson is currently recruiting for an Architect,...

PLM Analyst

Manhattan Beach, CA, USA Competitive Permanent

Posted on behalf of

Fisker

About the job Description: Company Overview Fisker Inc. is a design led pioneering mobility technology company researching, developing and producing next generation electrically power...

Application Consultant - Data Management Solutions

Quakertown, PA, USA Competitive Permanent

Posted on behalf of

Synergis Engineering Design Solutions

About the job Synergis Technologies, the region’s leader in data management, engineering, architectural, civil and manufacturing design solutions is searching for an engineering professional who is...

Looking for people to join your team?

Advertise your role with Momentum PLM. Get Started 

Looking to become a contributor?

Submit an article to Momentum PLM.  Get Involved 

Our Contributors

See the minds behind Momentum PLM.  Learn More 

Meet the Vendors

Read up on the latest vendor updates  Learn More 

Get the latest PLM updates in your inbox

Subscribe to our newsletter

Email Address

*

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

By submitting your email address and any other personal information on the website, you consent to it being collected, held, used and disclosed in accordance with our Privacy Policy.

*