Release of ISPE Quality Metrics Wave 2 Report Enables a Continued Data-Driven Dialogue with the FDA

06 June, 2016
Regulatory Matters

(Bethesda, Maryland, USA, 6 June 2016) – The International Society for Pharmaceutical Engineering (ISPE) will release the results of its Quality Metrics Pilot Program, Wave 2, at the ISPE/FDA/PQRI Quality Manufacturing Conference, 6–8 June 2016 in North Bethesda, Maryland, USA. Insights from this key initiative will be presented by the ISPE Quality Metrics team members on June 8, during a half-day session devoted to the topic.

Mairead Goetz, Head of Compliance, Group Compliance & Audit, Group Quality Assurance, Novartis Pharmaceuticals Corporation, and Chair of the ISPE Quality Metrics Project Team, reported that Wave 2 enhances understanding of relationships revealed in Wave 1, which was published in June 2015.

John Bournas, ISPE CEO and President said, “We are very excited that Wave 2 of our Pilot Program built and expanded on the successful findings from Wave 1. These Pilots help to further inform the industry on the selection of small metrics for the USFDA Quality Metrics initiative by continuing the dialogue between ISPE and the FDA.”

Wave 1 results and preliminary findings from Wave 2 were used to develop ISPE’s response to the FDA’s “Request for Quality Metrics” Draft Guidance and Federal Register Notice (FRN).

Full reports on Wave 1 and Wave 2 are available through ISPE.

Pilot Program Wave 2

ISPE initiated Wave 2 of the Quality Metrics Pilot Program in July 2015. The initial goals were to:

  • Expand the data set across segments, geographies, and time, in order to expand the knowledge gained from Wave 1 and evaluate trends;
  • Continue to develop measures, tools, and dialogue related to quality culture, and process capability to facilitate industry self-development and assessment;
  • Enable continued objective and data-driven dialogue with the FDA and other health authorities.

During Wave 2 planning and setup, the FDA issued its “Request for Quality Metrics” FRN and Draft Guidance. As a result, the Wave 2 design was adjusted to include the following additional objectives:

  • Test the proposed FDA metrics
    • Help develop appropriate definitions       
    • Understand data-collection challenges
  • Evaluate the logistics and effort of gathering data at a product-application level

Wave 2 also evaluated three of the four metrics proposed by the FDA in its Draft Guidance:

  • Lot Acceptance Rate
  • Product Quality Complaint Rate
  • Invalidated Out-of-Specification (OOS) Rate

Like Wave 1, Wave 2 was conducted in partnership with McKinsey & Company, who performed confidential data collection and analysis. The total number of sites contributing to the project increased from 44 sites from 18 companies in Wave 1 to 83 sites from 28 companies in Wave 1 and 2 combined.

Participating companies reported that they derived great value from the metric data they received, and from contributing to the confidential benchmarking exercise. John Bournas said “ISPE extends its sincere gratitude to the participating companies and their staffs for the excellent input, support, and enthusiasm they provided throughout the Wave 2 Pilot. We appreciate the confidence that these companies have bestowed on this data-driven project.”

Findings

ISPE Pilot Program Wave 2 met its objectives and confirmed findings from Wave 1. The findings from this study are fully explored in the Wave 2 report, but include:

  • The effort to collect FDA Draft Guidance metrics was approximately three times that given in the FRN. This is probably an underestimate, especially for over-the-counter companies and companies with complex supply chains.
  • Wave 2 confirmed the importance of quality culture, with some further relationships identified.
  • Process capability/performance measures are extensively used by companies to help control processes and identify continual improvement opportunities. Use of these indices varies between companies.

Background

Wave 1 of ISPE’s Quality Metrics Pilot Program was initiated in 2014 with results reported in June 2015. Wave 1 of the Pilot met all of its objectives, and also provided some key insights in relation to the prevailing quality culture within an organization that merit further exploration.

Following presentation of the Wave 1 Pilot results at the ISPE Quality Metrics Summit in Baltimore on 21–22 April 2015, it was broadly agreed that there was a continuing appetite within industry for additional learning about using metrics to measure and benchmark quality performance. ISPE therefore initiated a Wave 2 of the Pilot, which commenced in July 2015.

What Are Quality Metrics?

Following the passage of the Food and Drug Administration Safety and Innovation Act of 2012 (FDASIA), the FDA was authorized to collect manufacturing quality data and other metrics from pharmaceutical companies and obtain certain records from a drug manufacturer in lieu of, or in advance of, an inspection. FDA also was directed to replace the previous two-year drug inspection frequency requirement with a risk-based inspection approach.

These metrics include number of lot release tests, out-of-specification results, and lots attempted, rejected, reworked, and reprocessed. Historically, however, each company has collected data in its own way, using a range of definitions and sampling plans. The FDA has recommended that these metrics be standardized to ensure that all companies define and measure the metrics they report to the agency in the same way.

The FDA is considering the utilization of quality metrics as an input to its inspection models as well as to predict possible drug shortages, determine inspection schedules for a manufacturer, assess post-market change reporting, and re-structure the format of inspection. To that end, the FDA has invited input from industry on which metrics manufacturers use and find effective to assess processes and their risks.
About ISPE

The International Society for Pharmaceutical Engineering, is the world’s largest not-for-profit association serving its members through leading scientific, technical, and regulatory advancement across the entire pharmaceutical lifecycle. The 20,000 members of ISPE are building solutions in the development and manufacture of safe, effective pharmaceutical and biologic medicines and medical delivery devices in more than 90 countries around the world. Founded in 1980, ISPE has its worldwide headquarters in Bethesda, Maryland, USA; and an operations center in Tampa, Florida, USA. Visit www.ISPE.org for more information.