Another Unprecedented View of Attrition in a Corporate Portfolio – Chorus

It is unusual for a company to divulge the inner workings of its R&D practices and culture. We were surprised by the publication from a group in AstraZeneca who revealed the causes of attrition in the AZ portfolio from 2005-2010 [1]. And now there is the more recent publication of the Chorus portfolio in January 2015 [2].  Chorus is an independent division of Eli Lilly established to cost-effectively advance candidate molecules from discovery through clinical Proof of Concept (PoC). After 10 years of operation they have published a health check on their corporate strategy.  The Chorus paper  describes in detail how the portfolio has changed through the period 2005-2010.

With both publications the companies felt they had unearthed lessons that they wanted to share. It’s great to have two peeks behind the curtains in the space of six months.

What’s even more fun about the Chorus publication is that the authors present in detail all projects that have passed through the pipeline – 46 projects in all, showing the therapy areas, the origins (Lilly or “external”) and the fates of all projects. They present the portfolio in the form of a Stock & Flow diagram which we have recreated in Figure 1. A Stock & Flow diagram can only be constructed when one has this type of information.

Stock & Flow Diagram of Chorus Portfolio 16Jan15b#click to enlarge the image

Figure 1, A Stock & Flow Diagram of 5 years of the Chorus Portfolio (2005-2010) [2]. The boxes represent the development stages employed by Chorus. Projects that were still in a stage at the time of the analysis are indicated as (WIP) or “work in progress” within the box. The arrows leading into the boxes show the numbers of projects feeding each stage. The arrows between each stage show the number of projects transitioning to the next stage above the arrow, along with the success rates for those stages shown below the arrow. The arrows coming in from the top display the number of projects stocking the portfolio – 34 at Preclinical Evaluation and 11 at Phase I. *={Footnote 1} The arrows leading out the bottoms of each stage show the number of projects that were terminated in each stage. Chorus has two “exits” that differ from the traditional pharmaceutical company. The first one is at the end of Phase I they call “Proof of Mechanism” (POM) which is a point where Lilly may pull the asset into its main R&D unit. The second one is “Proof of Concept” (POC) in Phase II. It is presumed that Lilly carried the POM assets forward into Phase II and the POC assets into Phase III but of course there is no mention of their fate. {Footnote 2}

The publication of this information allows us to comment on several aspects of the Chorus portfolio in particular, and a project portfolio in general. {Footnote 3}

Comparing Chorus Therapy Areas, Success Rates, Cycle Times and Capitalized Costs with a “Traditional” Pharmaceutical Company.

Therapy Areas. The Chorus portfolio has been heavily invested in Neuroscience assets (41.3%), rather unusual for a drug company. Neuroscience drugs have the longest development times and the lowest approval success rates [3].

Therapy Area Balance in the Chorus Portfolio#click to enlarge the image

Figure 2, Therapy Area Balance in the Chorus Portfolio. One asset in the paper (No. 33) appears to have changed therapy areas from Neuroscience to MSK/Inflammation. It is counted as Neuroscience here.

But it would make sense for Chorus to take over a disproportionate share of Neuroscience projects on behalf of its parent Eli Lilly, to spend as little as possible to see if there are any winners. To date, Lilly has taken back two Neuroscience assets with positive outcomes at Phase I (either they were most concerned about the safety of these targets or it made more sense for Lilly to invest in the expensive Phase II and III trials). All of the other Neuroscience targets were terminated – 84% failures! The other therapy area with a high failure rate was MSK/Inflammation with 71% failures (5 out of 7), Figure 3.

Percent of Terminated Assets in the Chorus Portfolio#click to enlarge the image

Figure 3, Percent of Terminated Assets in the Chorus Portfolio.

Chorus Success Rates. The percent of successful projects is often called the Success Rate or the Probability of Success (POS) as discussed in this section.  Table 1 compares the Success Rates determined by Chorus with two other groups, the Protein Benchmarking Forum (PBF) and AstraZeneca in the earlier reference [1]. It should be noted that, in Table 1, the Chorus data represent the mean while the AZ and PBF data represent the median. Thus we can only say that Chorus success rates are within the ballpark of industry data, and perhaps the Phase II POS is better than AstraZeneca.

Chorus Development Stage Success Rates POS 16Jan15#click to enlarge the image

Table 1, Comparing Chorus Development Stage Success Rates (POS) with the Pharmaceutical Benchmarking Forum and AstraZeneca [2]. (a) – data from [1].

Chorus Cycle Times. Table 2 compares Chorus cycle times with what the authors call a “traditional” pharmaceutical company [4]. {Footnote 4} While projects at Chorus tend to get through Preclinical and Phase 1 more quickly than the “traditional company”, Phase II actually takes longer in the Chorus model. This is because their goal is to get to POC (the end of Phase IIa) quickly. So at Chorus certain types of work are deferred to Phase IIb. Types of deferred work that could be deferred include defining the best formulation, extended toxicology studies, manufacturing process optimization and pilot studies to identify other indications.  The traditional company gets through Phase II in about 2.5 years, while it takes on average 3.2 years by the Chorus model. Chorus gets to Phase II from Candidate Selection in 2.1 years compared to 3 years for the traditional company but that advantage is lost in Phase IIb.

Chorus Development Stage Cycle Times 16Jan15#click to enlarge the image

Table 2, Comparing Development Stage Cycle Times between a “Traditional” Company and Chorus [2]. *Proof-of-concept is the end of Phase IIa. Not all companies make this distinction so the full cycle time for Phase II is quoted for the Traditional company.

Chorus Capitalized Development Costs. Where the difference between Chorus and the traditional company comes is in capitalized cost, Table 3. Because Chorus gets to Phase II quickly the cost for Preclinical and Phase I is considerably lower than for the Traditional company: $3.3 million (Chorus), versus $17.9 million (Traditional). Chorus is able to trim costs a bit more in subsequent phases as well, so the overall capitalized cost advantage in early development is maintained throughout the development cycle. So it was a good bet for Lilly to run the bunch of Neuroscience and MSK/Inflammation assets through Chorus, since most of them turned out to be failures!

Chorus Capitalized Costs 16Jan15#click to enlarge the image

Table 3, Comparing Capitalized Costs between a “Traditional” Company and Chorus [2]. *Proof-of-concept is the end of Phase IIa. Not all companies make this distinction so the full cycle time for Phase II is quoted for the Traditional company.

Ten Years is a Short Amount of Time. In its decade of existence Chorus has worked on 45 assets, and successfully passed 8 of them back to Lilly for further development, a 17.8% success rate – not bad. As seen in Table I, their cumulative Preclinical to End of Phase II POS is 8.1 % right in there with the Pharmaceutical Benchmarking Forum and better than AstraZeneca. That said, only three of the assets exited at POC and only one of the 34 assets that came in at Preclinical Evaluation and went all the way to a successful exit at POC.

A more rigorous analysis of the portfolio would need to await another ten years for the passage of a greater number of assets through the Chorus pipeline. By then Chorus should have even more Preclinical to Exit successes. Were Chorus an independent start-up, at this point they would likely be hoping for an IPO or take-over in the event that any of their POCs turn to blockbuster product launches. But being a branch of a larger drug company that appears to be satisfied with the results, Chorus may look forward to another decade of work. One wonders whether we can look forward to a subsequent disclosure ten years hence. In other words, this may be as good as it gets in terms of public disclosure of a pharmaceutical project portfolio.

Small versus Large Company Behaviors. The analysis provided by Chorus was intended to promote the virtue of their strategy of “pulling risk and attrition forward”. They note that the Chorus model may only work for larger pharma, since “pulling risk and attrition forward only makes sense when there is an abundance of actionable innovation at hand: that is, when the savings reaped from an early termination can be redirected to a more promising asset.”

But there is also another important point here: The Chorus model may not be that dissimilar from a typical small company strategy. Small companies and their financial backers typically lack the resources to “do it all”. Small companies will try to do only as much as they can to reach a positive POC, a very important investment inflexion point. After POC they hope to lure a partner into carrying the asset forward into late stage development. Inevitably the large pharma partner will come to realize what work needs to be completed prior to Phase III and do so in their own Phase IIb (which is what Lilly has done with Chorus assets brought back in). So let us take the Chorus Phase IIb and Phase III cost estimates as representative of what a large pharmaceutical company typically needs to spend to bring the in-licensed asset into late-stage development readiness – $21.4 million dollars in capitalized costs, Table 3. An added bonus to Lilly is that a Chorus asset brought back in occurs without the attendant mark-up that an external small company would put on the price tag for the partnership which would be considerably more than $21.4 million dollars.

Challenges to Portfolio Metrics.

The Chorus portfolio also presents three issues that come up when one determines portfolio metrics.

Work in Progress. Note that Chorus had six assets in Phase II that were WIP at the time of the analysis. Should they all fail the POC exit success would still be 3 of 14 or 21.4%. If they all succeed however, the POC exit success would still be 9 of 14 or 64.3%! It is often the case that some of the WIP assets are tantalizingly close to successful completion but haven’t crossed the finish line of the final corporate approval. The portfolio manager should resist the temptation. If necessary add a footnote to the analysis about such WIP.

Assets Brought into the Pipeline after the First Stage. The 11 assets brought into the Chorus pipeline at Phase I are similar to in-licensings or assets brought in through a partnership. How should they be treated in portfolio metrics? Since those external assets could been treated differently in preceding stages than they would in-house, such assets are often left out of the pipeline metrics. Preclinical Evaluation could be different at Lilly than at Chorus. One of those Phase I entries was an external asset. Chorus, in fact, did exclude the external assets in some of their analyses in the paper, but not the Phase I entries from Lilly. We left all such assets in our analysis. The reader may wish to compare the numbers.

Out-licensed Assets. The POM exits to Lilly could be considered out-licensed assets. In determining portfolio metrics care should be taken to treat out-licensed assets as some form of success and not as marks against later stage metrics. For example, the Phase II success rate should be 3 out of 14 assets (excluding the 5 POM exits) and not 3 out of 19 assets. Similarly the POM exits and POC exits should not be lumped together because the 5 POM exits may not be successful in Phase II, so POM and POC exits are not the same.

Therapy Area Changes. As noted in Figure 2, one of the  assets in the paper (No. 33) appears to have changed therapy areas from Neuroscience to MSK/Inflammation. Without further information we counted it as Neuroscience here (the change could have been a typographical error). But it raises a real issue – it is feasible that the range of potential indications for a drug candidate could straddle other therapy areas. It would actually be in Phase IIa where such issues get sorted. So the therapy area change to asset 33 could be real. In which case one should count it with the MSK/Inflammation assets and not Neuroscience.

More about Stock & Flow Diagrams.

The reader may be unfamiliar with the Stock & Flow diagram and may want to learn more about them. There are a number of ways to display movement within a project portfolio. We encourage the reader to consider our white paper “Examples of Portfolio Views in Biopharmaceutical Discovery” which discusses Stock & Flow diagrams and other ways to view the project portfolio, available in the Downloads section of

Again, Chorus had their own reasons for presenting this data. And we are grateful to have the chance to peek behind the curtains. There is much to be learned from such revelations. We hope the trends continues. We agree with the authors that Chorus seems to be working and constitutes a good strategy for managing risky assets. Its continued survival will depend on whether its parent company Eli Lilly agrees.

{Footnote 1} Chorus operates a non-development Clinical Candidate stage that feeds into Preclinical Evaluation. In that stage Lilly staff evaluates the robustness of the asset prior to accepting it into the Development portfolio. The authors indicate that this evaluation is what would normally happen at the end of Lead Optimization in a traditional company. The paper shows 35 projects had passed into the Clinical Candidate Stage with one WIP, and 34 projects moving through other stages in the pipeline.

{Footnote 2} While most assets exit at the end of Phase IIa, the traditional end of that phase, it appears that some assets may go further and exit at the end of Phase IIb, as evidenced by one WIP in Phase IIb (Phases IIa and IIb are combined in Figure 1).

{Footnote 3} Some of the analyses in the paper omit the external assets (e.g. Figure 2 of the paper), perhaps because these assets belong to another company. For the sake of continuity we present all assets as the Chorus portfolio assuming no bias in resource applied to Lilly or External assets.

{Footnote 4} The authors base all of their assumptions about a “traditional” pharmaceutical company on the work of Paul et al. [4] and the data presented therein.


[1] D. Cook, D. Brown, R. Alexander, R. March, P. Morgan, G. Satterthwaite, M. Pangalos, “Lessons learned from the fate of AstraZeneca’s drug pipeline: a five-dimensional framework” Nat. Rev. Drug Disc. 2014, 13, pp. 419-429.

[2] P. K. Owens, E. Raddad, J. W. Miller, J. R. Stille, K.G. Olovich, N.V. Smith, R.S. Jones & J.C. Scherer “A decade of innovation in pharmaceutical R&D: the Chorus model” Nat. Rev. Drug Disc. 14, 17–28 (2015)

[3]. J. A. DiMasi “Metrics on Technical Risks, Clinical Development Times, and Approval Times for Cancer Drugs” ASCO/IOM Workshop Washington, DC February 2013. Available at Files/Disease/NCPF/2013-FEB-11/DiMasi.pdf

[4] S. M. Paul et al. “How to improve R&D productivity: the pharmaceutical industry’s grand challenge” Nature Rev. Drug Disc. 9, 203–214 (2010).