An Unprecedented View of Attrition in a Corporate Portfolio – AstraZeneca
When you work inside a drug company, especially as a portfolio data manager, you understand the intricacies and organic nature of a real R&D pipeline. You also understand that the consortia, that collect pipeline data from a subset of companies in the industry and average out the data, are providing a hybrid view of the R&D pipeline. In any consortium you see how the data from your company compares to the range of data from consortium members without knowing whose data is whose. Whose data it is could be useful but anonymity is understandable. Even then you can only surmise the other company’s R&D practices and culture.
So it is really unusual for the inner workings of a company’s R&D practices and culture to be revealed to the public, especially since one can only guess as to what trade secrets may be revealed let alone inadvertantly revealing intellectual property. Mergers reveal portfolios to the merging companies and occasionally to the public. 1 In the absence of such a transformation, AstraZeneca took the unusual step of publishing a paper on the inner workings of its development organization. 2 In particular they presented a discussion on attrition in their pipeline, the causes and what they intend to do about it.
As John La Mattina noted “the paper is rich in detail, remarkable in its candor, and pretty damning of scientific management and oversight.” 3 He notes that when Martin Mackay arrived at AstraZeneca, he commissioned a study to determine the reasons why compounds had such a high rate of attrition in progressing from the laboratory through clinical trials. The authors goal was to “determine the most critical factors that were likely to improve the health of the R&D organization, and increase the probability of successful transitions to Phase III trials and, ultimately, the launch of new medicines.”
A comprehensive review was undertaken of 142 drug development projects that had been active between 2005 and 2010 at the company. The authors hoped to understand “the major reasons for project closure and to identify the features of projects that correlated with successful outcomes.”
Project Success Rates. We were particularly delighted that the authors not only showed AZ success rates but also quoted industry metrics [the KMR group managed Pharmaceutical Benchmarking Forum (PBF)]. It’s always nice to have an update on such an important metric. Intriguingly they separate Phase II into IIa and IIb. Most often the two are combined. If we average the two medians we get a Phase II average success rate of 38% for AZ and 48% for PBF. The PBF number is comparable to 44% determined earlier by Brown and Superti-Furga (see the discussion in the section Attrition).
#click on figure to enlarge
Figure 1, Comparing Median Success Rates for Preclinical Evaluation, Phase I, Phase IIa, and Phase IIb between AstraZeneca and the PBF from 2005-2010.
Causes of Project Failures. The causes of project failure are displayed in Figure 2. They can be grouped into four categories.
#click on figure to enlarge
Figure 2, Causes for Project Failure in AstraZeneca Development Portfolio 2005-2010 (Ref 1)
Safety. Understandably 82% of project closures in in Preclinical Evaluation were due to Safety. That is where one would expect Safety issues to arise and lead to project closure if necessary. But as seen in the figure, projects continued to close due to Safey in Phase 1, IIa and IIb, albeit by a decreasing percentage. The authors of the study hold management responsible for failing to make the hard decision to terminate a project on Safety.
One wonders why Safety hadn’t been routed out at Preclinical Evaluation, which is the main reason for project failure in that phase. There are at least three reasons.
a) Animal Testing in Preclinical Evaluation. Safety issues are evaluated in species other than human in Preclinical Evaluation. It is only in the clinical phases that safety issues with humans are revealed. That is why Project Failure due to Safety can still be high in Phase 1 (62%) and why healthy young male volunteers tend to be the first subjects to receive a new drug.
b) Animal Safety Studies in a Clinical Stage May Kill a Project in that Stage. Safety studies specific to the stage are also run prior to the trials in that stage. Thus it is conceivable that the safety issues that came up in the clinical stages were found in animals in that stage and not in humans. We don’t know if the authors made that distinction.
c) The Acceptability of the Principle of the Risk-Benefit Ratio. The industry has accepted that the observed safety issues may be manageable if the benefit of the therapy is sufficiently high. Hence there is the temptation to continue, to see if the benefit is sufficiently high. To the project team the compound represents a breakthrough in disease treatment of a disease. To the team and managers who champion the drug there is the hope or even the belief that the risk to benefit ratio will be acceptable. It is often the case that safety issues uncovered in Preclinical Evaluation go on to close the project after Phase I or Phase II trials.
d) First In – First Out. It would be rare if only one clinical candidate came into development. The project team would have brought up a set of agents into the late stages of Lead Optimization, where preliminary Safety studies occur. The team tends to champion the first agent to meet all the criteria for candidate selection and usually management accepts it as the first into Development. The challenge with some safety issues is whether to drop the compound and bring along a suitable alternative candidate that is free of the safety issues or to continue. But AstraZeneca found that the back-up/follow-up rarely displaces the Lead agent, which is not unusual. They also found that back-ups/follow-ups did not have a significantly different success rate than the lead agents.
But the authors are holding management to the task, they take the position that these closures should have occurred earlier.
Efficacy. Lack of efficacy of the experimental medicine in clinical trials was the second leading cause of failure – no big surprise there. Even though it is the role of the project team to build a case for efficacy with animal and in vitro data, 40% of the projects that got into the clinic and closed, had a lack of data that demonstrated a clear linkage of the target to the disease, or lacked validated animal disease models. The authors found that the teams were often overly optimistic in the potential of their project. They found that roughly 45% of projects that failed due to lack of clinical efficacy, had initiated Phase II trials on the basis of weak clinical evidence in Phase 1 (“low confidence in target”).
Milestone Management. With Phase II, expense begins to ramp up considerably. The authors felt that management was insufficiently rigorous about preventing spending on poor bets at this milestone.
Back-up Candidate Differentiation. Quite often the first candidate to reach clinical trials is just that – the first and not necessarily the best. As the project team vets its set of potential clinical candidates excitement builds within the company around the possibility that success is just around the corner. So the company is quite ready to take into the clinic the first or “lead” compound that has a complete dossier. There may be better compounds coming up but their dossiers are not yet complete. So the company goes with what it has. After all, the patent clock is ticking and there may be competitors to think about.
When the dossier has been completed on one or more of these other candidates, they are accepted into clinical trials as back-up candidates ready to take over should the first candidate fail. Some companies add a second category of follow-up or follow-on to agents that they will take into the clinic along with the first or lead agent because the follow-up has attributes that are different or superior to the lead. (There is the portfolio management issue of lead/follow-up position change – the project data system needs to be able to note such events).
The AstraZeneca analysis showed that many of the failed compounds were back-up/follow-up candidates. Of course these candidates are often advanced without knowing the fatal flaw of the lead candidate. In the absence of such information the back-up may possess the same flaw. And so when the lead fails the back-ups/follow-ups often fail for the same reason. Back-ups and follow-ups unwittingly become a major contributor to project failure. In fact, the authors noted that “approximately 50% of the AstraZeneca portfolio was composed of back-up molecules.”
Some companies take pains to ensure that a follow-up is significantly different from the lead agent. It appears in the case of AstraZeneca, it didn’t matter. As mentioned earlier, AstraZeneca found that the back-up/follow-up rarely displaces the Lead agent. They also found that back-ups/follow-ups did not have a significantly different success rate than the lead agents. So if the back-ups aren’t really ensuring success for the new type of drug, why bother? Limiting the number of back-ups in the clinical portfolio would significantly impact the spend in Development – half of the portfolio contained back-ups/follow-ups.
Management Failure. LaMatinna attributes these categories of failure to a “breakdown in scientific management and leadership”. Despite the natural exuberance about projects and the conviction of major breakthrough discovery it falls to management to challenge project teams and make tough decisions. It is a management decision to allow compounds to advance into clinical trials with safety issues or insufficient data to justify advancement. Of course, Mackay came to AstraZeneca from a company that likely was no better in such decision making. It’s human nature to put the coin in the slot machine. Management wants and needs the success just as much as the project team. As LaMattina notes “one would hope that most organizations have recognized these issues and have corrected them. If that’s not the case, high project failure will continue throughout the industry.”
Indeed AstraZeneca implemented a “5R” decision making framework. The 5Rs stand for the following
- Right Target – ensuring a strong link to the disease along with usable and predictive biomarkers
- Right Tissue – ensuring adequate bioavailability and tissue exposure and again, usable and predictive biomarkers
- Right Safety – ensuring differentiated and clear safety margins along with a clear understanding of the risks
- Right Patients – ensuring the drug goes to the most responsive patient population with a clear understanding of the risk-benefit in that population
- Right Commercial Potential – ensuring a clear value proposition in the face of the future standard of care (it might be different by the time the drug comes on the market) for the payer and provider.
They surveyed project teams for their confidence in various decisions made by the project team, with some interesting findings.
- Confidence in Phase IIb patient selection tended to be low for closed projects.
- Confidence in patient selection decreased through the clinical phases, sadly lowest in Phase IIb.
- Confidence in perceived commercial value decreased through the clinical phases, sadly lowest in Phase IIb.
The authors concluded that the 5R decision making framework resulted in “a substantial reduction in the size and shape of the research and early development portfolio”. This is not a bad thing as long as the remaining projects have a higher probability of success. The portfolio can be quite small if the projects left in the portfolio are all sure winners.
- “The genetics of a pharma merger” P. Sanseau, M. Chabot-Fletcher, M. J Browne, Drug Discov. Today, 2009 14, 334-6 ↩
- D. Cook, D. Brown, R. Alexander, R. March, P. Morgan, G. Satterthwaite, M. Pangalos, “Lessons learned from the fate of AstraZeneca’s drug pipeline: a five-dimensional framework” Nat. Rev. Drug Disc. 2014, 13, pp. 419-429. ↩
- “AstraZeneca Provides An Inside Look Into R&D Practices” J. LaMattina, Forbes, May 2014, http://www.forbes.com/sites/johnlamattina/2014/05/19/astrazeneca-provides-an-inside-look-into-rd-practices/ ↩