Suppose we are planning a drug development program testing the superiority of an experimental treatment over a control treatment. Our drug development program consists of an exploratory phase II trial which is, in case of promising results, followed by a confirmatory phase III trial.

The drugdevelopR package enables us to optimally plan such programs using a utility-maximizing approach. To get a brief introduction, we presented a very basic example on how the package works in Introduction to planning phase II and phase III trials with drugdevelopR. Contrary to the basic setting, where only one phase III trial was planned, we now want to show how to plan several phase III trials using the package.

Suppose we are developing a new tumor treatment, *exper*. The
patient variable that we want to investigate is how long the patient
survives without further progression of the disease (progression-free
survival). This is a time-to-event outcome variable. Therefore, we will
use the function `optimal_multitrial`

, which calculates
optimal sample sizes and threshold decisions values for time-to-event
outcomes when several phase III trials are performed.

Within our drug development program, we will compare our experimental
treatment *exper* to the control treatment *contro*. The
treatment effect measure is given by \(\theta
= −\log(HR)\), where the hazard ratio \(HR\) is the ratio of the hazard rates
between the two groups. If we assume that the hazard of experiencing a
progression is reduced by 75% in the treatment group compared to the
control group, we have a hazard ratio of 0.75.

After installing the package according to the installation instructions, we can load it using the following code:

We insert the same input values as in the example for time-to-event endpoints. As in the basic setting, the treatment effect may be fixed or follow a prior distribution. Furthermore, some options to adapt the program to your specific needs are also available in this setting, however skipping phase II, setting predefined benefit categories and choosing different treatment effects in phase II and III are not possible.

The package implements a framework developed for phase II/III drug
development programs where several phase III trials are performed. This
is of particular relevance as regulatory agencies often require
statistical significance in two or more phase III trials. Different
*cases*, defined by the number of significant trials needed for
approval, are possible. For each case, different *strategies* are
possible. They are defined by the number of phase III trials to be
conducted in order to reach the goal of the case. For the success of the
drug development program, it is necessary that the treatment effects of
all phase III trials point in the same direction. For example, if we
select case 3 and strategy 4, we require four phase III trials, where
three need to be significant at level \(\alpha\) and the treatment effect of the
fourth must point in the same direction.

The following cases and possible strategies are implemented in the package.

Case | Possible strategies for this case |
---|---|

1 | 1, 2 |

2 | 1 (with significance level of \(\alpha^2\))* , 2, 3, 23* |

3 | 1 (with significance level of \(\alpha^3\))*, 3, 4 |

Note that setting `case = 1`

and `strategy = 1`

would lead to the same results as in the basic setting. There are two
special strategies which we marked with a * in the table:

- When using strategy 23 (the “two+one” strategy) in case 2, two phase III trials are conducted. If only one shows a significant positive treatment effect (and the other trial’s treatment effect points at least in the same direction), a third trial is conducted.
- Setting
`strategy = 1`

in cases 2 and 3 leads to a situation where one large phase III trial is conducted with adjusted significance level, i.e. \(\alpha_{new} = \alpha^2\) in case 2 and \(\alpha_{new} = \alpha^3\) in case 3. This feature is implemented as there is discussion that a larger trial with adjusted significance level may also may serve evidence on efficacy (see Koch (2005)).

So, in addition to the parameters from the basic setting, we provide the following parameters:

- The parameter
`case`

allows us to chose the number of trials in phase III that have to show a significant positive treatment effect for the whole program to be deemed successful. In the following example we set the parameter`case = 3`

, indicating that at least two trials in phase III have to show a significant treatment effect in phase III. - The parameter
`strategy`

defines the number of trials that are conducted in phase III. The strategies 1, 2, 3, and 4 (and 23) are implemented. Setting`strategy = TRUE`

returns the results of the optimization for all implemented strategies for a specific case. Note that not all strategies are implemented for each case (see table above). For the following example we set`strategy = TRUE`

.

Note that for the following example, some input parameters were changed compared to the basic setting in order to reduce computation time.

```
res <- optimal_multitrial(w = 0.3, # define parameters for prior
hr1 = 0.7, hr2 = 0.8, id1 = 210, id2 = 420, # (https://web.imbi.uni-heidelberg.de/prior/)
d2min = 20, d2max = 200, stepd2 = 5, # define optimization set for d2
hrgomin = 0.7, hrgomax = 0.9, stephrgo = 0.02, # define optimization set for HRgo
alpha = 0.025, beta = 0.1, xi2 = 0.7, xi3 = 0.7, # drug development planning parameters
c2 = 0.75, c3 = 1, c02 = 100, c03 = 150, # define fixed and variable costs
K = Inf, N = Inf, S = -Inf, # set constraints
b1 = 1000, b2 = 2000, b3 = 3000, # define expected benefits
fixed = TRUE, # choose if effects are fixed or random
case = 3, strategy = TRUE # choose case and strategy
num_cl = 3)
```

After setting all these input parameters and running the function, let’s take a look at the output of the program.

```
res
#> Optimization result with 3 significant trial(s) needed, strategy 1:
#> Utility: -231.07
#> Sample size:
#> phase II: 30, phase III: 368, total: 398
#> Expected number of events:
#> phase II: 20, phase III: 258, total: 278
#> Assumed event rate:
#> phase II: 0.7, phase III: 0.7
#> Probability to go to phase III: 0.57
#> Total cost:
#> phase II: 122, phase III: 454, cost constraint: Inf
#> Fixed cost:
#> phase II: 100, phase III: 150
#> Variable cost per patient:
#> phase II: 0.75, phase III: 1
#> Effect size categories (expected gains):
#> small: 1 (1000), medium: 0.95 (2000), large: 0.85 (3000)
#> Success probability: 0.19
#> Success probability by effect size:
#> small: 0.06, medium: 0.11, large: 0.02
#> Significance level: 1.5625e-05
#> Targeted power: 0.9
#> Decision rule threshold: 0.76 [HRgo]
#> Assumed true effect: 0.7 [hr]
#> Treatment effect offset between phase II and III: 0 [gamma]
#>
#> Optimization result with 3 significant trial(s) needed, strategy 3:
#> Utility: -411.1
#> Sample size:
#> phase II: 30, phase III: 258, total: 288
#> Expected number of events:
#> phase II: 20, phase III: 180, total: 200
#> Assumed event rate:
#> phase II: 0.7, phase III: 0.7
#> Probability to go to phase III: 0.5
#> Total cost:
#> phase II: 122, phase III: 483, cost constraint: Inf
#> Fixed cost:
#> phase II: 100, phase III: 150
#> Variable cost per patient:
#> phase II: 0.75, phase III: 1
#> Effect size categories (expected gains):
#> small: 1 (1000), medium: 0.95 (2000), large: 0.85 (3000)
#> Success probability: 0.08
#> Success probability by effect size:
#> small: 0, medium: 0.05, large: 0.03
#> Significance level: 0.025
#> Targeted power: 0.9
#> Decision rule threshold: 0.7 [HRgo]
#> Assumed true effect: 0.7 [hr]
#> Treatment effect offset between phase II and III: 0 [gamma]
#>
#> Optimization result with 3 significant trial(s) needed, strategy 4:
#> Utility: -436.79
#> Sample size:
#> phase II: 58, phase III: 416, total: 474
#> Expected number of events:
#> phase II: 40, phase III: 292, total: 332
#> Assumed event rate:
#> phase II: 0.7, phase III: 0.7
#> Probability to go to phase III: 0.5
#> Total cost:
#> phase II: 144, phase III: 716, cost constraint: Inf
#> Fixed cost:
#> phase II: 100, phase III: 150
#> Variable cost per patient:
#> phase II: 0.75, phase III: 1
#> Effect size categories (expected gains):
#> small: 1 (1000), medium: 0.95 (2000), large: 0.85 (3000)
#> Success probability: 0.2
#> Success probability by effect size:
#> small: 0, medium: 0.17, large: 0.02
#> Significance level: 0.025
#> Targeted power: 0.9
#> Decision rule threshold: 0.7 [HRgo]
#> Assumed true effect: 0.7 [hr]
#> Treatment effect offset between phase II and III: 0 [gamma]
```

The program returns a data frame where the output for all implemented strategies is listed.

For strategy 1 we get:

`res[1,]$d2`

is the optimal number of events for phase II and`res[1,]$d3`

the resulting number of events for phase III. We see that the optimal scenario requires 20 events in phase II and 258 events in phase III, which correspond to 30 participants in phase II and 368 in phase III.`res[1,]$HRgo`

is the optimal threshold value for the go/no-go decision rule. We see that we need a hazard ratio of less than 0.76 in phase II in order to proceed to phase III.`res[1,]$u`

is the expected utility of the program for the optimal sample size and threshold value. In our case it amounts to -231.07, i.e. we have an expected utility of -23 107 000$.- Note that the input parameter
`alpha`

is returned having a value of 0.000016, which corresponds to the adjusted significance level \(\alpha^3\).

The results for strategy 3 are:

`res[2,]$d2`

is the optimal number of events for phase II and`res[2,]$d3`

the resulting number of events for phase III. We see that the optimal scenario requires 20 events in phase II and 180 events in phase III, which leads to 30 participants in phase II and 258 in phase III. This corresponds to three trials in phase III with each trial having 86 participants.`res[2,]$HRgo`

is the optimal threshold value for the go/no-go decision rule. We see that we need a hazard ratio of less than 0.7 in phase II in order to proceed to phase III.`res[2,]$u`

is the expected utility of the program for the optimal sample size and threshold value. In our case it amounts to -411.1, i.e. we have an expected utility of -41 110 000$.

Strategy 4 yields the following results:

`res[3,]$d2`

is the optimal number of events for phase II and`res[3,]$d3`

the resulting number of events for phase III. We see that the optimal scenario requires 40 events in phase II and 292 events in phase III, which leads to 58 participants in phase II and 416 in phase III. This corresponds to four trials in phase III with each trial having 104 participants.`res[3,]$HRgo`

is the optimal threshold value for the go/no-go decision rule. We see that we need a hazard ratio of less than 0.70 in phase II in order to proceed to phase III.`res[3,]$u`

is the expected utility of the program for the optimal sample size and threshold value. In our case it amounts to -436.79, i.e. we have an expected utility of -43 679 000$.

In this article we presented an example where several phase III
trials are conducted. Note that this example is not restricted to
time-to-event endpoints but can also be applied to binary and normally
distributed endpoints by using the functions
`optimal_multitrial_binary`

and
`optimal_multitrial_normal`

. For more information on how to
use the package, see:

*Introduction to drugdevelopR:*See how the package works in a basic example.*Different outcomes:*Apply it to binary endpoints and time-to-event endpoints.*Interpreting the rest of the output:*Obtain further details on your drug development program.*Fixed or prior:*Model the assumed treatment effect on a prior distribution.*More parameters:*Define custom effect size categories. Put constraints on the optimization by defining maximum costs, the total expected sample size of the program or the minimum expected probability of a successful program. Define an expected difference in treatment effect between phase II and III. Skip phase II.*Complex drug development programs:*Adapt to situations with biased effect estimators, multi-arm trials, or multiple endpoints.*Parallel computing:*Be faster at calculating the optimum by using parallel computing.

Koch, G. G. (2005). Statistical consideration of the strategy for demonstrating clinical evidence of effectiveness one larger vs two smaller pivotal studies by z. shun, e. chi, s. durrleman and l. fisher, statistics in medicine 2005; 24: 1619–1637. Statistics in Medicine, 24(11):1639–1646.