Title: | Graphical Multiple Comparison Procedures |
---|---|
Description: | Multiple comparison procedures (MCPs) control the familywise error rate in clinical trials. Graphical MCPs include many commonly used procedures as special cases; see Bretz et al. (2011) <doi:10.1002/bimj.201000239>, Lu (2016) <doi:10.1002/sim.6985>, and Xi et al. (2017) <doi:10.1002/bimj.201600233>. This package is a low-dependency implementation of graphical MCPs which allow mixed types of tests. It also includes power simulations and visualization of graphical MCPs. |
Authors: | Dong Xi [aut, cre], Ethan Brockmann [aut], Gilead Sciences, Inc. [cph, fnd] |
Maintainer: | Dong Xi <[email protected]> |
License: | Apache License (>= 2) |
Version: | 0.2.6 |
Built: | 2024-11-08 21:22:22 UTC |
Source: | https://github.com/openpharma/graphicalmcp |
For an intersection hypothesis, an adjusted p-value is the smallest
significance level at which the intersection hypothesis can be rejected.
The intersection hypothesis can be rejected if its adjusted p-value is less
than or equal to . Currently, there are three test types
supported:
Bonferroni tests for adjust_p_bonferroni()
,
Parametric tests for adjust_p_parametric()
,
Note that one-sided tests are required for parametric tests.
Simes tests for adjust_p_simes()
.
adjust_p_bonferroni(p, hypotheses) adjust_p_parametric( p, hypotheses, test_corr = NULL, maxpts = 25000, abseps = 1e-06, releps = 0 ) adjust_p_simes(p, hypotheses)
adjust_p_bonferroni(p, hypotheses) adjust_p_parametric( p, hypotheses, test_corr = NULL, maxpts = 25000, abseps = 1e-06, releps = 0 ) adjust_p_simes(p, hypotheses)
p |
A numeric vector of p-values (unadjusted, raw), whose values should
be between 0 & 1. The length should match the length of |
hypotheses |
A numeric vector of hypothesis weights. Must be a vector of
values between 0 & 1 (inclusive). The length should match the length of
|
test_corr |
(Optional) A numeric matrix of correlations between test
statistics, which is needed to perform parametric tests using
|
maxpts |
(Optional) An integer scalar for the maximum number of function
values, which is needed to perform parametric tests using the
|
abseps |
(Optional) A numeric scalar for the absolute error tolerance,
which is needed to perform parametric tests using the |
releps |
(Optional) A numeric scalar for the relative error tolerance
as double, which is needed to perform parametric tests using the
|
A single adjusted p-value for the intersection hypothesis.
Bretz, F., Maurer, W., Brannath, W., and Posch, M. (2009). A graphical approach to sequentially rejective multiple test procedures. Statistics in Medicine, 28(4), 586-604.
Lu, K. (2016). Graphical approaches using a Bonferroni mixture of weighted Simes tests. Statistics in Medicine, 35(22), 4041-4055.
Xi, D., Glimm, E., Maurer, W., and Bretz, F. (2017). A unified framework for weighted parametric multiple test procedures. Biometrical Journal, 59(5), 918-931.
adjust_weights_parametric()
for adjusted hypothesis weights using
parametric tests, adjust_weights_simes()
for adjusted hypothesis weights
using Simes tests.
hypotheses <- c(H1 = 0.5, H2 = 0.25, H3 = 0.25) p <- c(0.019, 0.025, 0.05) adjust_p_bonferroni(p, hypotheses) set.seed(1234) hypotheses <- c(H1 = 0.5, H2 = 0.25, H3 = 0.25) p <- c(0.019, 0.025, 0.05) # Using the `mvtnorm::GenzBretz` algorithm corr <- matrix(0.5, nrow = 3, ncol = 3) diag(corr) <- 1 adjust_p_parametric(p, hypotheses, corr) hypotheses <- c(H1 = 0.5, H2 = 0.25, H3 = 0.25) p <- c(0.019, 0.025, 0.05) adjust_p_simes(p, hypotheses)
hypotheses <- c(H1 = 0.5, H2 = 0.25, H3 = 0.25) p <- c(0.019, 0.025, 0.05) adjust_p_bonferroni(p, hypotheses) set.seed(1234) hypotheses <- c(H1 = 0.5, H2 = 0.25, H3 = 0.25) p <- c(0.019, 0.025, 0.05) # Using the `mvtnorm::GenzBretz` algorithm corr <- matrix(0.5, nrow = 3, ncol = 3) diag(corr) <- 1 adjust_p_parametric(p, hypotheses, corr) hypotheses <- c(H1 = 0.5, H2 = 0.25, H3 = 0.25) p <- c(0.019, 0.025, 0.05) adjust_p_simes(p, hypotheses)
An intersection hypothesis can be rejected if its p-values are less than or
equal to their adjusted significance levels, which are their adjusted
hypothesis weights times . For Bonferroni tests, their adjusted
hypothesis weights are their hypothesis weights of the intersection
hypothesis. Additional adjustment is needed for parametric and Simes tests:
Parametric tests for adjust_weights_parametric()
,
Note that one-sided tests are required for parametric tests.
Simes tests for adjust_weights_simes()
.
adjust_weights_parametric( matrix_weights, matrix_intersections, test_corr, alpha, test_groups, maxpts = 25000, abseps = 1e-06, releps = 0 ) adjust_weights_simes(matrix_weights, p, test_groups)
adjust_weights_parametric( matrix_weights, matrix_intersections, test_corr, alpha, test_groups, maxpts = 25000, abseps = 1e-06, releps = 0 ) adjust_weights_simes(matrix_weights, p, test_groups)
matrix_weights |
(Optional) A matrix of hypothesis weights of all
intersection hypotheses. This can be obtained as the second half of columns
from the output of |
matrix_intersections |
(Optional) A matrix of hypothesis indicators of
all intersection hypotheses. This can be obtained as the first half of
columns from the output of |
test_corr |
(Optional) A numeric matrix of correlations between test
statistics, which is needed to perform parametric tests using
|
alpha |
(Optional) A numeric value of the overall significance level, which should be between 0 & 1. The default is 0.025 for one-sided hypothesis testing problems; another common choice is 0.05 for two-sided hypothesis testing problems. Note when parametric tests are used, only one-sided tests are supported. |
test_groups |
(Optional) A list of numeric vectors specifying hypotheses to test together. Grouping is needed to correctly perform Simes and parametric tests. |
maxpts |
(Optional) An integer scalar for the maximum number of function
values, which is needed to perform parametric tests using the
|
abseps |
(Optional) A numeric scalar for the absolute error tolerance,
which is needed to perform parametric tests using the |
releps |
(Optional) A numeric scalar for the relative error tolerance
as double, which is needed to perform parametric tests using the
|
p |
(Optional) A numeric vector of p-values (unadjusted, raw), whose
values should be between 0 & 1. The length should match the number of
columns of |
adjust_weights_parametric()
returns a matrix with the same
dimensions as matrix_weights
, whose hypothesis weights have been
adjusted according to parametric tests.
adjust_weights_simes()
returns a matrix with the same
dimensions as matrix_weights
, whose hypothesis weights have been
adjusted according to Simes tests.
Lu, K. (2016). Graphical approaches using a Bonferroni mixture of weighted Simes tests. Statistics in Medicine, 35(22), 4041-4055.
Xi, D., Glimm, E., Maurer, W., and Bretz, F. (2017). A unified framework for weighted parametric multiple test procedures. Biometrical Journal, 59(5), 918-931.
adjust_p_parametric()
for adjusted p-values using parametric tests,
adjust_p_simes()
for adjusted p-values using Simes tests.
alpha <- 0.025 p <- c(0.018, 0.01, 0.105, 0.006) num_hyps <- length(p) g <- bonferroni_holm(rep(1 / 4, 4)) weighting_strategy <- graph_generate_weights(g) matrix_intersections <- weighting_strategy[, seq_len(num_hyps)] matrix_weights <- weighting_strategy[, -seq_len(num_hyps)] set.seed(1234) adjust_weights_parametric( matrix_weights = matrix_weights, matrix_intersections = matrix_intersections, test_corr = diag(4), alpha = alpha, test_groups = list(1:4) ) alpha <- 0.025 p <- c(0.018, 0.01, 0.105, 0.006) num_hyps <- length(p) g <- bonferroni_holm(rep(1 / 4, 4)) weighting_strategy <- graph_generate_weights(g) matrix_intersections <- weighting_strategy[, seq_len(num_hyps)] matrix_weights <- weighting_strategy[, -seq_len(num_hyps)] adjust_weights_simes( matrix_weights = matrix_weights, p = p, test_groups = list(1:4) )
alpha <- 0.025 p <- c(0.018, 0.01, 0.105, 0.006) num_hyps <- length(p) g <- bonferroni_holm(rep(1 / 4, 4)) weighting_strategy <- graph_generate_weights(g) matrix_intersections <- weighting_strategy[, seq_len(num_hyps)] matrix_weights <- weighting_strategy[, -seq_len(num_hyps)] set.seed(1234) adjust_weights_parametric( matrix_weights = matrix_weights, matrix_intersections = matrix_intersections, test_corr = diag(4), alpha = alpha, test_groups = list(1:4) ) alpha <- 0.025 p <- c(0.018, 0.01, 0.105, 0.006) num_hyps <- length(p) g <- bonferroni_holm(rep(1 / 4, 4)) weighting_strategy <- graph_generate_weights(g) matrix_intersections <- weighting_strategy[, seq_len(num_hyps)] matrix_weights <- weighting_strategy[, -seq_len(num_hyps)] adjust_weights_simes( matrix_weights = matrix_weights, p = p, test_groups = list(1:4) )
Graph objects have different structures and attributes in graphicalMCP
,
gMCP
, and igraph
R packages. These functions convert between different
classes to increase compatibility.
Note that igraph
and gMCP
have additional attributes for vertices, edges,
or a graph itself. These conversion functions only handle attributes related
to hypothesis names, hypothesis weights and transition weights. Other
attributes will be dropped when converting.
as_initial_graph(graph) ## S3 method for class 'graphMCP' as_initial_graph(graph) ## S3 method for class 'igraph' as_initial_graph(graph) as_graphMCP(graph) ## S3 method for class 'initial_graph' as_graphMCP(graph) as_igraph(graph) ## S3 method for class 'initial_graph' as_igraph(graph)
as_initial_graph(graph) ## S3 method for class 'graphMCP' as_initial_graph(graph) ## S3 method for class 'igraph' as_initial_graph(graph) as_graphMCP(graph) ## S3 method for class 'initial_graph' as_graphMCP(graph) as_igraph(graph) ## S3 method for class 'initial_graph' as_igraph(graph)
graph |
An |
as_graphMCP()
returns a graphMCP
object for the gMCP
package.
as_igraph()
returns an igraph
object for the igraph
package.
as_initial_graph()
returns an initial_graph
object for the
graphicalMCP
package.
Csardi, G., Nepusz, T., Traag, V., Horvat, S., Zanini, F., Noom, D., and Mueller, K. (2024). igraph: Network analysis and visualization in R. R package version 2.0.3. https://CRAN.R-project.org/package=igraph.
Rohmeyer, K., and Klinglmueller, K. (2024). gMCP: Graph based multiple test procedures. R package version 0.8-17. https://cran.r-project.org/package=gMCP.
graph_create()
for the initial graph used in the graphicalMCP
package.
g_graphicalMCP <- random_graph(5) if (requireNamespace("gMCP", quietly = TRUE)) { g_gMCP <- as_graphMCP(g_graphicalMCP) all.equal(g_graphicalMCP, as_initial_graph(g_gMCP)) } if (requireNamespace("igraph", quietly = TRUE)) { g_igraph <- as_igraph(g_graphicalMCP) all.equal(g_graphicalMCP, as_initial_graph(g_igraph)) }
g_graphicalMCP <- random_graph(5) if (requireNamespace("gMCP", quietly = TRUE)) { g_gMCP <- as_graphMCP(g_graphicalMCP) all.equal(g_graphicalMCP, as_initial_graph(g_gMCP)) } if (requireNamespace("igraph", quietly = TRUE)) { g_igraph <- as_igraph(g_graphicalMCP) all.equal(g_graphicalMCP, as_initial_graph(g_igraph)) }
Built-in functions to quickly generate select graphical multiple comparison procedures.
bonferroni(hypotheses, hyp_names = NULL) bonferroni_holm(hypotheses, hyp_names = NULL) huque_etal(hyp_names = NULL) fallback(hypotheses, hyp_names = NULL) fallback_improved_1(hypotheses, hyp_names = NULL) fallback_improved_2(hypotheses, epsilon = 1e-04, hyp_names = NULL) fixed_sequence(num_hyps, hyp_names = NULL) simple_successive_1(hyp_names = NULL) simple_successive_2(hyp_names = NULL) random_graph(num_hyps, hyp_names = NULL) two_doses_two_primary_two_secondary(hyp_names = NULL) three_doses_two_primary_two_secondary(hyp_names = NULL)
bonferroni(hypotheses, hyp_names = NULL) bonferroni_holm(hypotheses, hyp_names = NULL) huque_etal(hyp_names = NULL) fallback(hypotheses, hyp_names = NULL) fallback_improved_1(hypotheses, hyp_names = NULL) fallback_improved_2(hypotheses, epsilon = 1e-04, hyp_names = NULL) fixed_sequence(num_hyps, hyp_names = NULL) simple_successive_1(hyp_names = NULL) simple_successive_2(hyp_names = NULL) random_graph(num_hyps, hyp_names = NULL) two_doses_two_primary_two_secondary(hyp_names = NULL) three_doses_two_primary_two_secondary(hyp_names = NULL)
hypotheses |
(Optional) A numeric vector of hypothesis weights in a
graphical multiple comparison procedure. Must be a vector of values
between 0 & 1 (inclusive). The length should match |
hyp_names |
(Optional) A character vector of hypothesis names. The
length should match |
epsilon |
(Optional) A numeric scalar indicating the value of the
|
num_hyps |
(Optional) Number of hypotheses in a graphical multiple comparison procedure. |
An S3 object as returned by graph_create()
.
Bretz, F., Maurer, W., Brannath, W., and Posch, M. (2009). A graphical approach to sequentially rejective multiple test procedures. Statistics in Medicine, 28(4), 586-604.
Bretz, F., Posch, M., Glimm, E., Klinglmueller, F., Maurer, W., and Rohmeyer, K. (2011). Graphical approaches for multiple comparison procedures using weighted Bonferroni, Simes, or parametric tests. Biometrical Journal, 53(6), 894-913.
Huque, M. F., Alosh, M., and Bhore, R. (2011). Addressing multiplicity issues of a composite endpoint and its components in clinical trials. Journal of Biopharmaceutical Statistics, 21(4), 610-634.
Maurer, W., Hothorn, L., and Lehmacher, W. (1995). Multiple comparisons in drug clinical trials and preclinical assays: a-priori ordered hypotheses. Biometrie in der chemisch-pharmazeutischen Industrie, 6, 3-18.
Westfall, P. H., and Krishen, A. (2001). Optimally weighted, fixed sequence and gatekeeper multiple testing procedures. Journal of Statistical Planning and Inference, 99(1), 25-40.
Wiens, B. L. (2003). A fixed sequence Bonferroni procedure for testing multiple endpoints. Pharmaceutical Statistics, 2(3), 211-215.
Wiens, B. L., and Dmitrienko, A. (2005). The fallback procedure for evaluating a single family of hypotheses. Journal of Biopharmaceutical Statistics, 15(6), 929-942.
Xi, D., and Bretz, F. (2019). Symmetric graphs for equally weighted tests, with application to the Hochberg procedure. Statistics in Medicine, 38(27), 5268-5282.
graph_create()
for a general way to create the initial graph.
# Bretz et al. (2009) bonferroni(hypotheses = rep(1 / 3, 3)) # Bretz et al. (2009) bonferroni_holm(hypotheses = rep(1 / 3, 3)) # Huque et al. (2011) huque_etal() # Wiens (2003) fallback(hypotheses = rep(1 / 3, 3)) # Wiens and Dmitrienko (2005) fallback_improved_1(hypotheses = rep(1 / 3, 3)) # Bretz et al. (2009) fallback_improved_2(hypotheses = rep(1 / 3, 3)) # Maurer et al. (1995); Westfall and Krishen (2001) fixed_sequence(num_hyps = 3) # Figure 1 in Bretz et al. (2011) simple_successive_1() # Figure 4 in Bretz et al. (2011) simple_successive_2() # Create a random graph with three hypotheses random_graph(num_hyps = 3) # Figure 6 in Xi and Bretz et al. (2019) two_doses_two_primary_two_secondary() # Add another dose to Figure 6 in Xi and Bretz et al. (2019) three_doses_two_primary_two_secondary()
# Bretz et al. (2009) bonferroni(hypotheses = rep(1 / 3, 3)) # Bretz et al. (2009) bonferroni_holm(hypotheses = rep(1 / 3, 3)) # Huque et al. (2011) huque_etal() # Wiens (2003) fallback(hypotheses = rep(1 / 3, 3)) # Wiens and Dmitrienko (2005) fallback_improved_1(hypotheses = rep(1 / 3, 3)) # Bretz et al. (2009) fallback_improved_2(hypotheses = rep(1 / 3, 3)) # Maurer et al. (1995); Westfall and Krishen (2001) fixed_sequence(num_hyps = 3) # Figure 1 in Bretz et al. (2011) simple_successive_1() # Figure 4 in Bretz et al. (2011) simple_successive_2() # Create a random graph with three hypotheses random_graph(num_hyps = 3) # Figure 6 in Xi and Bretz et al. (2019) two_doses_two_primary_two_secondary() # Add another dose to Figure 6 in Xi and Bretz et al. (2019) three_doses_two_primary_two_secondary()
Under the alternative hypotheses, the distribution of test statistics is
assumed to be a multivariate normal distribution. Given this distribution,
this function calculates power values for a graphical multiple comparison
procedure. By default, it calculate the local power, which is the probability
to reject an individual hypothesis, the probability to reject at least one
hypothesis, the probability to reject all hypotheses, the expected number of
rejections, and the probability of user-defined success criteria. See
vignette("shortcut-testing")
and vignette("closed-testing")
for more
illustration of power calculation.
graph_calculate_power( graph, alpha = 0.025, power_marginal = rep(alpha, length(graph$hypotheses)), test_groups = list(seq_along(graph$hypotheses)), test_types = c("bonferroni"), test_corr = rep(list(NA), length(test_types)), sim_n = 1e+05, sim_corr = diag(length(graph$hypotheses)), sim_success = NULL, verbose = FALSE )
graph_calculate_power( graph, alpha = 0.025, power_marginal = rep(alpha, length(graph$hypotheses)), test_groups = list(seq_along(graph$hypotheses)), test_types = c("bonferroni"), test_corr = rep(list(NA), length(test_types)), sim_n = 1e+05, sim_corr = diag(length(graph$hypotheses)), sim_success = NULL, verbose = FALSE )
graph |
An initial graph as returned by |
alpha |
A numeric value of the one-sided overall significance level, which should be between 0 & 1. The default is 0.025 for one-sided hypothesis testing. Note that only one-sided tests are supported. |
power_marginal |
A numeric vector of marginal power values to use when simulating p-values. See Details for more on the simulation process. |
test_groups |
A list of numeric vectors specifying hypotheses to test together. Grouping is needed to correctly perform Simes and parametric tests. |
test_types |
A character vector of test types to apply to each test
group. This is needed to correctly perform Simes and parametric
tests. The length should match the number of elements in |
test_corr |
(Optional) A list of numeric correlation matrices. Each
entry in the list should correspond to each test group. For a test group
using Bonferroni or Simes tests, its corresponding entry in |
sim_n |
An integer scalar specifying the number of simulations. The default is 1e5. |
sim_corr |
A numeric matrix of correlations between test statistics for
all hypotheses. The dimensions should match the number of hypotheses in
|
sim_success |
A list of user-defined functions to specify the success
criteria. Functions must take one simulation's logical vector of results as
an input, and return a length-one logical vector. For instance, if
"success" means rejecting hypotheses 1 and 2, use |
verbose |
A logical scalar specifying whether the details of power
simulations should be included in results. The default is |
A power_report
object with a list of 3 elements:
inputs
- Input parameters, which is a list of:
graph
- Initial graph,
alpha
- Overall significance level,
test_groups
- Groups of hypotheses for different types of tests,
test_types
- Different types of tests,
test_corr
- Correlation matrices for parametric tests,
sim_n
- Number of simulations,
power_marginal
- Marginal power of all hypotheses
sim_corr
- Correlation matrices for simulations,
sim_success
- User-defined success criteria.
power
- A list of power values
power_local
- Local power of all hypotheses, which is the proportion
of simulations in which each hypothesis is rejected,
rejection_expected
- Expected (average) number of rejected hypotheses,
power_at_least_1
- Power to reject at least one hypothesis,
power_all
- Power to reject all hypotheses,
power_success
- Power of user-defined success, which is the
proportion of simulations in which the user-defined success criterion
sim_success
is met.
details
- An optional list of datasets showing simulated p-values and
results for each simulation.
The power calculation is based on simulations. The distribution to simulate
from is determined as a multivariate normal distribution by power_marginal
and sim_corr
. In particular, power_marginal
is a vector of marginal
power values for all hypotheses. The marginal power is the power to reject
the null hypothesis at the significance level alpha
without multiplicity adjustment. This value could be readily available from
standard software and other R packages. Then we can determine the mean of the
multivariate normal distribution as
,
which is often called the non-centrality parameter or the drift parameter.
Here is the marginal power
power_marginal
of hypothesis .
Given the correlation matrix
sim_corr
, we can simulate from this
multivariate normal distribution using the mvtnorm
R package (Genz and
Bretz, 2009).
Each set simulated values can be used to calculate the corresponding
one-sided p-values. Then this set of p-values are plugged into the graphical
multiple comparison procedure to determine which hypotheses are rejected.
This process is repeated n_sim
times to produce the power values as the
proportion of simulations in which a particular success criterion is met.
Bretz, F., Posch, M., Glimm, E., Klinglmueller, F., Maurer, W., and Rohmeyer, K. (2011a). Graphical approaches for multiple comparison procedures using weighted Bonferroni, Simes, or parametric tests. Biometrical Journal, 53(6), 894-913.
Bretz, F., Maurer, W., and Hommel, G. (2011b). Test and power considerations for multiple endpoint analyses using sequentially rejective graphical procedures. Statistics in Medicine, 30(13), 1489-1501.
Genz, A., and Bretz, F. (2009). Computation of Multivariate Normal and t Probabilities, series Lecture Notes in Statistics. Springer-Verlag, Heidelberg.
Lu, K. (2016). Graphical approaches using a Bonferroni mixture of weighted Simes tests. Statistics in Medicine, 35(22), 4041-4055.
Xi, D., Glimm, E., Maurer, W., and Bretz, F. (2017). A unified framework for weighted parametric multiple test procedures. Biometrical Journal, 59(5), 918-931.
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 4 in Bretz et al. (2011a). alpha <- 0.025 hypotheses <- c(0.5, 0.5, 0, 0) delta <- 0.5 transitions <- rbind( c(0, delta, 1 - delta, 0), c(delta, 0, 0, 1 - delta), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) marginal_power <- c(0.8, 0.8, 0.7, 0.9) corr1 <- matrix(0.5, nrow = 2, ncol = 2) diag(corr1) <- 1 corr <- rbind( cbind(corr1, 0.5 * corr1), cbind(0.5 * corr1, corr1) ) success_fns <- list( # Probability to reject both H1 and H2 `H1andH2` = function(x) x[1] & x[2], # Probability to reject both (H1 and H3) or (H2 and H4) `(H1andH3)or(H2andH4)` = function(x) (x[1] & x[3]) | (x[2] & x[4]) ) set.seed(1234) # Bonferroni tests # Reduce the number of simulations to save time for package compilation power_output <- graph_calculate_power( g, alpha, sim_corr = corr, sim_n = 1e2, power_marginal = marginal_power, sim_success = success_fns ) # Parametric tests for H1 and H2; Simes tests for H3 and H4 # User-defined success: to reject H1 or H2; to reject H1 and H2 # Reduce the number of simulations to save time for package compilation graph_calculate_power( g, alpha, test_groups = list(1:2, 3:4), test_types = c("parametric", "simes"), test_corr = list(corr1, NA), sim_n = 1e2, sim_success = list( function(.) .[1] || .[2], function(.) .[1] && .[2] ) )
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 4 in Bretz et al. (2011a). alpha <- 0.025 hypotheses <- c(0.5, 0.5, 0, 0) delta <- 0.5 transitions <- rbind( c(0, delta, 1 - delta, 0), c(delta, 0, 0, 1 - delta), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) marginal_power <- c(0.8, 0.8, 0.7, 0.9) corr1 <- matrix(0.5, nrow = 2, ncol = 2) diag(corr1) <- 1 corr <- rbind( cbind(corr1, 0.5 * corr1), cbind(0.5 * corr1, corr1) ) success_fns <- list( # Probability to reject both H1 and H2 `H1andH2` = function(x) x[1] & x[2], # Probability to reject both (H1 and H3) or (H2 and H4) `(H1andH3)or(H2andH4)` = function(x) (x[1] & x[3]) | (x[2] & x[4]) ) set.seed(1234) # Bonferroni tests # Reduce the number of simulations to save time for package compilation power_output <- graph_calculate_power( g, alpha, sim_corr = corr, sim_n = 1e2, power_marginal = marginal_power, sim_success = success_fns ) # Parametric tests for H1 and H2; Simes tests for H3 and H4 # User-defined success: to reject H1 or H2; to reject H1 and H2 # Reduce the number of simulations to save time for package compilation graph_calculate_power( g, alpha, test_groups = list(1:2, 3:4), test_types = c("parametric", "simes"), test_corr = list(corr1, NA), sim_n = 1e2, sim_success = list( function(.) .[1] || .[2], function(.) .[1] && .[2] ) )
A graphical multiple comparison procedure is represented by 1) a vector of
initial hypothesis weights hypotheses
, and 2) a matrix of initial
transition weights transitions
. This function creates the initial graph
object using hypothesis weights and transition weights.
graph_create(hypotheses, transitions, hyp_names = NULL)
graph_create(hypotheses, transitions, hyp_names = NULL)
hypotheses |
A numeric vector of hypothesis weights in a graphical
multiple comparison procedure. Must be a vector of values between 0 & 1
(inclusive). The length should match the row and column lengths of
|
transitions |
A numeric matrix of transition weights between hypotheses
in a graphical multiple comparison procedure. Must be a square matrix of
values between 0 & 1 (inclusive). The row and column lengths should match
the length of |
hyp_names |
(Optional) A character vector of hypothesis names. If not
provided, names from |
An S3 object of class initial_graph
with a list of 2 elements:
Hypothesis weights hypotheses
.
Transition weights transitions
.
Inputs are also validated to make sure of the validity of the graph:
Hypothesis weights hypotheses
are numeric.
Transition weights transitions
are numeric.
Length of hypotheses
and dimensions of transitions
match.
Hypothesis weights hypotheses
must be non-negative and sum to no more
than 1.
Transition weights transitions
:
Values must be non-negative.
Rows must sum to no more than 1.
Diagonal entries must be all 0.
Hypothesis names hyp_names
override names in hypotheses
or
transitions
.
Bretz, F., Maurer, W., Brannath, W., and Posch, M. (2009). A graphical approach to sequentially rejective multiple test procedures. Statistics in Medicine, 28(4), 586-604.
Bretz, F., Posch, M., Glimm, E., Klinglmueller, F., Maurer, W., and Rohmeyer, K. (2011). Graphical approaches for multiple comparison procedures using weighted Bonferroni, Simes, or parametric tests. Biometrical Journal, 53(6), 894-913.
graph_update()
for the updated graph after hypotheses being deleted
from the initial graph.
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 1 in Bretz et al. (2011). hypotheses <- c(0.5, 0.5, 0, 0) transitions <- rbind( c(0, 0, 1, 0), c(0, 0, 0, 1), c(0, 1, 0, 0), c(1, 0, 0, 0) ) hyp_names <- c("H11", "H12", "H21", "H22") g <- graph_create(hypotheses, transitions, hyp_names) g # Explicit names override names in `hypotheses` (with a warning) hypotheses <- c(h1 = 0.5, h2 = 0.5, h3 = 0, h4 = 0) transitions <- rbind( c(0, 0, 1, 0), c(0, 0, 0, 1), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions, hyp_names) g # Use names in `transitions` hypotheses <- c(0.5, 0.5, 0, 0) transitions <- rbind( H1 = c(0, 0, 1, 0), H2 = c(0, 0, 0, 1), H3 = c(0, 1, 0, 0), H4 = c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) g # Unmatched names in `hypotheses` and `transitions` (with an error) hypotheses <- c(h1 = 0.5, h2 = 0.5, h3 = 0, h4 = 0) transitions <- rbind( H1 = c(0, 0, 1, 0), H2 = c(0, 0, 0, 1), H3 = c(0, 1, 0, 0), H4 = c(1, 0, 0, 0) ) try( g <- graph_create(hypotheses, transitions) ) # When names are not specified, hypotheses are numbered sequentially as # H1, H2, ... hypotheses <- c(0.5, 0.5, 0, 0) transitions <- rbind( c(0, 0, 1, 0), c(0, 0, 0, 1), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) g
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 1 in Bretz et al. (2011). hypotheses <- c(0.5, 0.5, 0, 0) transitions <- rbind( c(0, 0, 1, 0), c(0, 0, 0, 1), c(0, 1, 0, 0), c(1, 0, 0, 0) ) hyp_names <- c("H11", "H12", "H21", "H22") g <- graph_create(hypotheses, transitions, hyp_names) g # Explicit names override names in `hypotheses` (with a warning) hypotheses <- c(h1 = 0.5, h2 = 0.5, h3 = 0, h4 = 0) transitions <- rbind( c(0, 0, 1, 0), c(0, 0, 0, 1), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions, hyp_names) g # Use names in `transitions` hypotheses <- c(0.5, 0.5, 0, 0) transitions <- rbind( H1 = c(0, 0, 1, 0), H2 = c(0, 0, 0, 1), H3 = c(0, 1, 0, 0), H4 = c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) g # Unmatched names in `hypotheses` and `transitions` (with an error) hypotheses <- c(h1 = 0.5, h2 = 0.5, h3 = 0, h4 = 0) transitions <- rbind( H1 = c(0, 0, 1, 0), H2 = c(0, 0, 0, 1), H3 = c(0, 1, 0, 0), H4 = c(1, 0, 0, 0) ) try( g <- graph_create(hypotheses, transitions) ) # When names are not specified, hypotheses are numbered sequentially as # H1, H2, ... hypotheses <- c(0.5, 0.5, 0, 0) transitions <- rbind( c(0, 0, 1, 0), c(0, 0, 0, 1), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) g
A graphical multiple comparison procedure defines a closed test procedure, which tests each intersection hypothesis and reject an individual hypothesis if all intersection hypotheses involving it have been rejected. An intersection hypothesis represents the parameter space where individual null hypotheses involved are true simultaneously.
The closure based on a graph consists of all updated graphs (corresponding
to intersection hypotheses) after all combinations of hypotheses are deleted.
For a graphical multiple comparison procedure with hypotheses, there
are
updated graphs (intersection hypotheses), including the
initial graph (the overall intersection hypothesis). The weighting strategy
of this graph consists of hypothesis weights from all
updated
graphs (intersection hypotheses). The algorithm to derive the weighting
strategy is based on Algorithm 1 in Bretz et al. (2011).
graph_generate_weights(graph)
graph_generate_weights(graph)
graph |
An initial graph as returned by |
A numeric matrix of all intersection hypotheses and their hypothesis
weights. For a graphical multiple comparison procedure with hypotheses,
the number of rows is
, each of which corresponds to an intersection
hypothesis. The number of columns is
. The first
columns
indicate which individual hypotheses are included in a given intersection
hypothesis and the second half of columns provide hypothesis weights for each
individual hypothesis for a given intersection hypothesis.
Generation of intersection hypotheses is closely related to the power set
of a given set of indices. As the number of hypotheses increases, the memory
and time usage can grow quickly (e.g., at a rate of ). There are also
multiple ways to implement Algorithm 1 in Bretz et al. (2011). See
vignette("generate-closure")
for more information about generating
intersection hypotheses and comparisons of different approaches to calculate
weighting strategies.
Bretz, F., Posch, M., Glimm, E., Klinglmueller, F., Maurer, W., and Rohmeyer, K. (2011). Graphical approaches for multiple comparison procedures using weighted Bonferroni, Simes, or parametric tests. Biometrical Journal, 53(6), 894-913.
graph_test_closure()
for graphical multiple comparison procedures using
the closed test.
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 1 in Bretz et al. (2011). hypotheses <- c(0.5, 0.5, 0, 0) transitions <- rbind( c(0, 0, 1, 0), c(0, 0, 0, 1), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) graph_generate_weights(g)
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 1 in Bretz et al. (2011). hypotheses <- c(0.5, 0.5, 0, 0) transitions <- rbind( c(0, 0, 1, 0), c(0, 0, 0, 1), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) graph_generate_weights(g)
When multiple hypotheses are rejected by using graph_test_shortcut()
,
there may be multiple orderings or sequences in which hypotheses are rejected
one by one. The default order in graph_test_shortcut()
is based on the
adjusted p-values, from the smallest to the largest. This function
graph_rejection_orderings()
provides all possible and valid orders
(or sequences) of rejections. Although the order of rejection does not affect
the final rejection decisions Bretz et al. (2009), different sequences could
offer different ways to explain the step-by-step process of shortcut
graphical multiple comparison procedures.
graph_rejection_orderings(shortcut_test_result)
graph_rejection_orderings(shortcut_test_result)
shortcut_test_result |
A |
A modified graph_report
object containing all valid orderings of
rejections of hypotheses
Bretz, F., Maurer, W., Brannath, W., and Posch, M. (2009). A graphical approach to sequentially rejective multiple test procedures. Statistics in Medicine, 28(4), 586-604.
Bretz, F., Posch, M., Glimm, E., Klinglmueller, F., Maurer, W., and Rohmeyer, K. (2011). Graphical approaches for multiple comparison procedures using weighted Bonferroni, Simes, or parametric tests. Biometrical Journal, 53(6), 894-913.
graph_test_shortcut()
for shortcut graphical multiple comparison
procedures.
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 4 in Bretz et al. (2011). hypotheses <- c(0.5, 0.5, 0, 0) delta <- 0.5 transitions <- rbind( c(0, delta, 1 - delta, 0), c(delta, 0, 0, 1 - delta), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) p <- c(0.018, 0.01, 0.105, 0.006) alpha <- 0.025 shortcut_testing <- graph_test_shortcut(g, p, alpha, verbose = TRUE) # Reject H1, H2, and H4 shortcut_testing$outputs$rejected # Default order of rejections: H2, H1, H4 shortcut_testing$details$del_seq # There is another valid sequence of rejection: H2, H4, H1 graph_rejection_orderings(shortcut_testing)$valid_orderings # Finally, intermediate updated graphs can be obtained by providing the order # of rejections into `[graph_update()]` graph_update(g, delete = c(2, 4, 1))
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 4 in Bretz et al. (2011). hypotheses <- c(0.5, 0.5, 0, 0) delta <- 0.5 transitions <- rbind( c(0, delta, 1 - delta, 0), c(delta, 0, 0, 1 - delta), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) p <- c(0.018, 0.01, 0.105, 0.006) alpha <- 0.025 shortcut_testing <- graph_test_shortcut(g, p, alpha, verbose = TRUE) # Reject H1, H2, and H4 shortcut_testing$outputs$rejected # Default order of rejections: H2, H1, H4 shortcut_testing$details$del_seq # There is another valid sequence of rejection: H2, H4, H1 graph_rejection_orderings(shortcut_testing)$valid_orderings # Finally, intermediate updated graphs can be obtained by providing the order # of rejections into `[graph_update()]` graph_update(g, delete = c(2, 4, 1))
Closed graphical multiple comparison procedures, or graphical multiple comparison procedures based on the closure, generate the closure based on a graph consisting of all intersection hypotheses. It tests each intersection hypothesis and rejects an individual hypothesis if all intersection hypotheses involving it have been rejected. An intersection hypothesis represents the parameter space where individual null hypotheses involved are true simultaneously.
For a graphical multiple comparison procedure with $m$ hypotheses, there are
intersection hypotheses. For each intersection hypothesis, a test
type could be chosen to determine how to reject the intersection hypothesis.
Current choices of test types include Bonferroni, Simes and parametric. This
implementation offers a more general framework covering Bretz et al. (2011),
Lu (2016), and Xi et al. (2017). See
vignette("closed-testing")
for more
illustration of closed test procedures and interpretation of their outputs.
graph_test_closure( graph, p, alpha = 0.025, test_groups = list(seq_along(graph$hypotheses)), test_types = c("bonferroni"), test_corr = rep(list(NA), length(test_types)), verbose = FALSE, test_values = FALSE )
graph_test_closure( graph, p, alpha = 0.025, test_groups = list(seq_along(graph$hypotheses)), test_types = c("bonferroni"), test_corr = rep(list(NA), length(test_types)), verbose = FALSE, test_values = FALSE )
graph |
An initial graph as returned by |
p |
A numeric vector of p-values (unadjusted, raw), whose values should
be between 0 & 1. The length should match the number of hypotheses in
|
alpha |
A numeric value of the overall significance level, which should be between 0 & 1. The default is 0.025 for one-sided hypothesis testing problems; another common choice is 0.05 for two-sided hypothesis testing problems. Note when parametric tests are used, only one-sided tests are supported. |
test_groups |
A list of numeric vectors specifying hypotheses to test together. Grouping is needed to correctly perform Simes and parametric tests. |
test_types |
A character vector of test types to apply to each test
group. This is needed to correctly perform Simes and parametric
tests. The length should match the number of elements in |
test_corr |
(Optional) A list of numeric correlation matrices. Each
entry in the list should correspond to each test group. For a test group
using Bonferroni or Simes tests, its corresponding entry in |
verbose |
A logical scalar specifying whether the details of the
adjusted p-value calculations should be included in results. When
|
test_values |
A logical scalar specifying whether adjusted significance
levels should be provided for each hypothesis. When |
A graph_report
object with a list of 4 elements:
inputs
- Input parameters, which is a list of:
graph
- Initial graph,
p
- (Unadjusted or raw) p-values,
alpha
- Overall significance level,
test_groups
- Groups of hypotheses for different types of tests,
test_types
- Different types of tests,
test_corr
- Correlation matrices for parametric tests.
outputs
- Output parameters, which is a list of:
adjusted_p
- Adjusted p-values,
rejected
- Rejected hypotheses,
graph
- Updated graph after deleting all rejected hypotheses.
details
- Verbose outputs with adjusted p-values for intersection
hypotheses, if verbose = TRUE
.
test_values
- Adjusted significance levels, if test_values = TRUE
.
Test specification includes three components: test_groups
, test_types
,
and test_corr
. Alignment among entries in these components is important
for correct implementation. There are two ways to provide test specification.
The first approach is the "unnamed" approach, which assumes that all 3
components are ordered the same way, i.e., the $n$-th element of test_types
and test_corr
should apply to the $n$-th group in test_groups
. The
second "named" approach uses the name of each element of each component to
connect the element of test_types
and test_corr
with the correct element
of test_groups
. Consistency should be ensured for correct implementation.
Bretz, F., Posch, M., Glimm, E., Klinglmueller, F., Maurer, W., and Rohmeyer, K. (2011). Graphical approaches for multiple comparison procedures using weighted Bonferroni, Simes, or parametric tests. Biometrical Journal, 53(6), 894-913.
Lu, K. (2016). Graphical approaches using a Bonferroni mixture of weighted Simes tests. Statistics in Medicine, 35(22), 4041-4055.
Xi, D., Glimm, E., Maurer, W., and Bretz, F. (2017). A unified framework for weighted parametric multiple test procedures. Biometrical Journal, 59(5), 918-931.
graph_test_shortcut()
for shortcut graphical multiple comparison
procedures.
# A graphical multiple comparison procedure with two primary hypotheses # (H1 and H2) and two secondary hypotheses (H3 and H4) # See Figure 4 in Bretz et al. (2011). hypotheses <- c(0.5, 0.5, 0, 0) delta <- 0.5 transitions <- rbind( c(0, delta, 1 - delta, 0), c(delta, 0, 0, 1 - delta), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) p <- c(0.018, 0.01, 0.105, 0.006) alpha <- 0.025 # Closed graphical multiple comparison procedure using Bonferroni tests # Same results as `graph_test_shortcut(g, p, alpha)` graph_test_closure(g, p, alpha) # Closed graphical multiple comparison procedure using parametric tests for # H1 and H2, and Bonferroni tests for H3 and H4 set.seed(1234) corr_list <- list(matrix(c(1, 0.5, 0.5, 1), nrow = 2), NA) graph_test_closure( graph = g, p = p, alpha = alpha, test_groups = list(1:2, 3:4), test_types = c("parametric", "bonferroni"), test_corr = corr_list ) # The "named" approach to obtain the same results # Note that "group2" appears before "group1" in `test_groups` set.seed(1234) corr_list <- list(group1 = matrix(c(1, 0.5, 0.5, 1), nrow = 2), group2 = NA) graph_test_closure( graph = g, p = p, alpha = alpha, test_groups = list(group1 = 1:2, group2 = 3:4), test_types = c(group2 = "bonferroni", group1 = "parametric"), test_corr = corr_list ) # Closed graphical multiple comparison procedure using parametric tests for # H1 and H2, and Simes tests for H3 and H4 set.seed(1234) graph_test_closure( graph = g, p = p, alpha = alpha, test_groups = list(group1 = 1:2, group2 = 3:4), test_types = c(group1 = "parametric", group2 = "simes"), test_corr = corr_list )
# A graphical multiple comparison procedure with two primary hypotheses # (H1 and H2) and two secondary hypotheses (H3 and H4) # See Figure 4 in Bretz et al. (2011). hypotheses <- c(0.5, 0.5, 0, 0) delta <- 0.5 transitions <- rbind( c(0, delta, 1 - delta, 0), c(delta, 0, 0, 1 - delta), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) p <- c(0.018, 0.01, 0.105, 0.006) alpha <- 0.025 # Closed graphical multiple comparison procedure using Bonferroni tests # Same results as `graph_test_shortcut(g, p, alpha)` graph_test_closure(g, p, alpha) # Closed graphical multiple comparison procedure using parametric tests for # H1 and H2, and Bonferroni tests for H3 and H4 set.seed(1234) corr_list <- list(matrix(c(1, 0.5, 0.5, 1), nrow = 2), NA) graph_test_closure( graph = g, p = p, alpha = alpha, test_groups = list(1:2, 3:4), test_types = c("parametric", "bonferroni"), test_corr = corr_list ) # The "named" approach to obtain the same results # Note that "group2" appears before "group1" in `test_groups` set.seed(1234) corr_list <- list(group1 = matrix(c(1, 0.5, 0.5, 1), nrow = 2), group2 = NA) graph_test_closure( graph = g, p = p, alpha = alpha, test_groups = list(group1 = 1:2, group2 = 3:4), test_types = c(group2 = "bonferroni", group1 = "parametric"), test_corr = corr_list ) # Closed graphical multiple comparison procedure using parametric tests for # H1 and H2, and Simes tests for H3 and H4 set.seed(1234) graph_test_closure( graph = g, p = p, alpha = alpha, test_groups = list(group1 = 1:2, group2 = 3:4), test_types = c(group1 = "parametric", group2 = "simes"), test_corr = corr_list )
Shortcut graphical multiple comparison procedures are sequentially rejective
procedure based on Bretz et al. (2009). With $m$ hypotheses, there are at
most $m$ steps to obtain all rejection decisions. These procedure are
equivalent to closed graphical multiple comparison procedures using
Bonferroni tests for intersection hypotheses, but shortcut procedures are
faster to perform. See vignette("shortcut-testing")
for more illustration
of shortcut procedures and interpretation of their outputs.
graph_test_shortcut( graph, p, alpha = 0.025, verbose = FALSE, test_values = FALSE )
graph_test_shortcut( graph, p, alpha = 0.025, verbose = FALSE, test_values = FALSE )
graph |
An initial graph as returned by |
p |
A numeric vector of p-values (unadjusted, raw), whose values should
be between 0 & 1. The length should match the number of hypotheses in
|
alpha |
A numeric scalar of the overall significance level, which should be between 0 & 1. The default is 0.025 for one-sided hypothesis testing problems; another common choice is 0.05 for two-sided hypothesis testing problems. |
verbose |
A logical scalar specifying whether the details of
intermediate update graphs should be included in results. When
|
test_values |
A logical scalar specifying whether adjusted significance
levels should be provided for each hypothesis. When |
An S3 object of class graph_report
with a list of 4 elements:
inputs
- Input parameters, which is a list of:
graph
- Initial graph,
*p
- (Unadjusted or raw) p-values,
alpha
- Overall significance level,
test_groups
- Groups of hypotheses for different types of tests,
which are the list of all hypotheses for graph_test_shortcut()
,
test_types
- Different types of tests, which are "bonferroni" for
graph_test_shortcut()
.
Output parameters outputs
, which is a list of:
adjusted_p
- Adjusted p-values,
rejected
- Rejected hypotheses,
graph
- Updated graph after deleting all rejected hypotheses.
details
- Verbose outputs with intermediate updated graphs, if
verbose = TRUE
.
test_values
- Adjusted significance levels, if test_values = TRUE
.
Bretz, F., Maurer, W., Brannath, W., and Posch, M. (2009). A graphical approach to sequentially rejective multiple test procedures. Statistics in Medicine, 28(4), 586-604.
Bretz, F., Posch, M., Glimm, E., Klinglmueller, F., Maurer, W., and Rohmeyer, K. (2011). Graphical approaches for multiple comparison procedures using weighted Bonferroni, Simes, or parametric tests. Biometrical Journal, 53(6), 894-913.
graph_test_closure()
for graphical multiple comparison procedures using
the closed test,
graph_rejection_orderings()
for all possible rejection orderings.
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 1 in Bretz et al. (2011). hypotheses <- c(0.5, 0.5, 0, 0) transitions <- rbind( c(0, 0, 1, 0), c(0, 0, 0, 1), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) p <- c(0.018, 0.01, 0.105, 0.006) alpha <- 0.025 graph_test_shortcut(g, p, alpha)
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 1 in Bretz et al. (2011). hypotheses <- c(0.5, 0.5, 0, 0) transitions <- rbind( c(0, 0, 1, 0), c(0, 0, 0, 1), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) p <- c(0.018, 0.01, 0.105, 0.006) alpha <- 0.025 graph_test_shortcut(g, p, alpha)
After a hypothesis is deleted, an initial graph will be updated. The deleted hypothesis will have the hypothesis weight of 0 and the transition weight of 0. Remaining hypotheses will have updated hypothesis weights and transition weights according to Algorithm 1 of Bretz et al. (2009).
graph_update(graph, delete)
graph_update(graph, delete)
graph |
An initial graph as returned by |
delete |
A logical or integer vector, denoting which hypotheses to
delete. A logical vector results in the "unordered mode", which means that
hypotheses corresponding to |
An S3 object of class updated_graph
with a list of 4 elements:
initial_graph
: The initial graph object.
updated_graph
: The updated graph object with specified hypotheses
deleted.
deleted
: A numeric vector indicating which hypotheses were deleted.
intermediate_graphs
: When using the ordered mode, a list of
intermediate updated graphs after each hypothesis is deleted according
to the sequence specified by delete
.
When there are multiple hypotheses to be deleted from a graph, there are many
sequences of deletion in which an initial graph is updated to an updated
graph. If the interest is in the updated graph after all hypotheses specified
by delete
are deleted, this updated graph is the same no matter which
sequence of deletion is used. This property has been proved by Bretz et al.
(2009). If the interest is in the intermediate updated graph after each
hypothesis is deleted according to the sequence specified by delete
, an
integer vector of delete
should be specified and these detailed outputs
will be provided.
Bretz, F., Maurer, W., Brannath, W., and Posch, M. (2009). A graphical approach to sequentially rejective multiple test procedures. Statistics in Medicine, 28(4), 586-604.
Bretz, F., Posch, M., Glimm, E., Klinglmueller, F., Maurer, W., and Rohmeyer, K. (2011). Graphical approaches for multiple comparison procedures using weighted Bonferroni, Simes, or parametric tests. Biometrical Journal, 53(6), 894-913.
graph_create()
for the initial graph.
graph_rejection_orderings()
for possible sequences of rejections for a
graphical multiple comparison procedure using shortcut testing.
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 1 in Bretz et al. (2011). hypotheses <- c(0.5, 0.5, 0, 0) transitions <- rbind( c(0, 0, 1, 0), c(0, 0, 0, 1), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) # Delete the second and third hypotheses in the "unordered mode" graph_update(g, delete = c(FALSE, TRUE, TRUE, FALSE)) # Equivalent way in the "ordered mode" to obtain the updated graph after # deleting the second and third hypotheses # Additional intermediate updated graphs are also provided graph_update(g, delete = 2:3)
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 1 in Bretz et al. (2011). hypotheses <- c(0.5, 0.5, 0, 0) transitions <- rbind( c(0, 0, 1, 0), c(0, 0, 0, 1), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) # Delete the second and third hypotheses in the "unordered mode" graph_update(g, delete = c(FALSE, TRUE, TRUE, FALSE)) # Equivalent way in the "ordered mode" to obtain the updated graph after # deleting the second and third hypotheses # Additional intermediate updated graphs are also provided graph_update(g, delete = 2:3)
initial_graph
The plot of an initial_graph
translates the hypotheses
into vertices and
transitions
into edges to create a network plot. Vertices are labeled with
hypothesis names and hypothesis weights, and edges are labeled with
transition weights. See vignette("graph-examples")
for more illustration
of commonly used multiple comparison procedure using graphs.
## S3 method for class 'initial_graph' plot( x, ..., v_palette = c("#6baed6", "#cccccc"), layout = "grid", nrow = NULL, ncol = NULL, edge_curves = NULL, precision = 4, eps = NULL, background_color = "white", margins = c(0, 0, 0, 0) )
## S3 method for class 'initial_graph' plot( x, ..., v_palette = c("#6baed6", "#cccccc"), layout = "grid", nrow = NULL, ncol = NULL, edge_curves = NULL, precision = 4, eps = NULL, background_color = "white", margins = c(0, 0, 0, 0) )
x |
An object of class |
... |
Other arguments passed on to |
v_palette |
A character vector of length two specifying the colors for
retained and deleted hypotheses. More extensive color customization must be
done with |
layout |
An igraph layout specification (See |
nrow |
An integer scalar specifying the number of rows in the vertex grid. If row and column counts are not specified, vertices will be laid out as close to a square as possible. |
ncol |
An integer scalar specifying the number of columns in the vertex grid. If row and column counts are not specified, vertices will be laid out as close to a square as possible. |
edge_curves |
A named numeric vector specifying the curvature of
specific edges. Edge pairs (Where two vertices share an edge in each
possible direction) are detected automatically and get 0.25 curvature.
Adjust edges by adding an entry with name |
precision |
An integer scalar indicating the number of decimal places to to display. |
eps |
A numeric scalar. The transition weight of |
background_color |
A character scalar specifying a background color for
the whole plotting area. Passed directly to |
margins |
A length 4 numeric vector specifying the margins for the plot.
Defaults to all 0, since igraph plots tend to have large margins. It is
passed directly to |
An object x of class initial_graph
, after plotting the initial
graph.
There are a few values for igraph::plot.igraph()
that get their defaults
changed for graphicalMCP. These values can still be changed by passing them
as arguments to plot.initial_graph()
. Here are the new defaults:
vertex.color = "#6baed6"
,
vertex.label.color = "black"
,
vertex.size = 20
,
edge.arrow.size = 1
,
edge.arrow.width = 1
,
edge.label.color = "black"
asp = 0
.
Neither graphicalMCP
nor igraph
does anything about overlapping edge
labels. If you run into this problem, and vertices can't practically be
moved enough to avoid collisions of edge labels, using edge curves can help.
igraph
puts edge labels closer to the tail of an edge when an edge is
straight, and closer to the head of an edge when it's curved. By setting an
edge's curve to some very small value, an effectively straight edge can be
shifted to a new position.
Bretz, F., Posch, M., Glimm, E., Klinglmueller, F., Maurer, W., and Rohmeyer, K. (2011). Graphical approaches for multiple comparison procedures using weighted Bonferroni, Simes, or parametric tests. Biometrical Journal, 53(6), 894-913.
Xi, D., and Bretz, F. (2019). Symmetric graphs for equally weighted tests, with application to the Hochberg procedure. Statistics in Medicine, 38(27), 5268-5282.
plot.updated_graph()
for the plot method for the updated graph after
hypotheses being deleted from the initial graph.
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 4 in Bretz et al. (2011). hypotheses <- c(0.5, 0.5, 0, 0) delta <- 0.5 transitions <- rbind( c(0, delta, 1 - delta, 0), c(delta, 0, 0, 1 - delta), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) plot(g) # A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and four secondary hypotheses (H31, H32, H41, and H42) # See Figure 6 in Xi and Bretz (2019). hypotheses <- c(0.5, 0.5, 0, 0, 0, 0) epsilon <- 1e-5 transitions <- rbind( c(0, 0.5, 0.25, 0, 0.25, 0), c(0.5, 0, 0, 0.25, 0, 0.25), c(0, 0, 0, 0, 1, 0), c(epsilon, 0, 0, 0, 0, 1 - epsilon), c(0, epsilon, 1 - epsilon, 0, 0, 0), c(0, 0, 0, 1, 0, 0) ) hyp_names <- c("H1", "H2", "H31", "H32", "H41", "H42") g <- graph_create(hypotheses, transitions, hyp_names) plot_layout <- rbind( c(0.15, 0.5), c(0.65, 0.5), c(0, 0), c(0.5, 0), c(0.3, 0), c(0.8, 0) ) plot(g, layout = plot_layout, eps = epsilon, edge_curves = c(pairs = .5))
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 4 in Bretz et al. (2011). hypotheses <- c(0.5, 0.5, 0, 0) delta <- 0.5 transitions <- rbind( c(0, delta, 1 - delta, 0), c(delta, 0, 0, 1 - delta), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) plot(g) # A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and four secondary hypotheses (H31, H32, H41, and H42) # See Figure 6 in Xi and Bretz (2019). hypotheses <- c(0.5, 0.5, 0, 0, 0, 0) epsilon <- 1e-5 transitions <- rbind( c(0, 0.5, 0.25, 0, 0.25, 0), c(0.5, 0, 0, 0.25, 0, 0.25), c(0, 0, 0, 0, 1, 0), c(epsilon, 0, 0, 0, 0, 1 - epsilon), c(0, epsilon, 1 - epsilon, 0, 0, 0), c(0, 0, 0, 1, 0, 0) ) hyp_names <- c("H1", "H2", "H31", "H32", "H41", "H42") g <- graph_create(hypotheses, transitions, hyp_names) plot_layout <- rbind( c(0.15, 0.5), c(0.65, 0.5), c(0, 0), c(0.5, 0), c(0.3, 0), c(0.8, 0) ) plot(g, layout = plot_layout, eps = epsilon, edge_curves = c(pairs = .5))
updated_graph
Plotting an updated graph is a very light wrapper around
plot.initial_graph()
, only changing the default vertex color to use gray
for deleted hypotheses.
## S3 method for class 'updated_graph' plot(x, ...)
## S3 method for class 'updated_graph' plot(x, ...)
x |
An object of class |
... |
Arguments passed on to
|
An object x of class updated_graph
, after plotting the updated
graph.
Bretz, F., Posch, M., Glimm, E., Klinglmueller, F., Maurer, W., and Rohmeyer, K. (2011). Graphical approaches for multiple comparison procedures using weighted Bonferroni, Simes, or parametric tests. Biometrical Journal, 53(6), 894-913.
plot.initial_graph()
for the plot method for the initial graph.
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 1 in Bretz et al. (2011). hypotheses <- c(0.5, 0.5, 0, 0) transitions <- rbind( c(0, 0, 1, 0), c(0, 0, 0, 1), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) # Delete the second and third hypotheses in the "unordered mode" plot( graph_update( g, c(FALSE, TRUE, TRUE, FALSE) ), layout = "grid" )
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 1 in Bretz et al. (2011). hypotheses <- c(0.5, 0.5, 0, 0) transitions <- rbind( c(0, 0, 1, 0), c(0, 0, 0, 1), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) # Delete the second and third hypotheses in the "unordered mode" plot( graph_update( g, c(FALSE, TRUE, TRUE, FALSE) ), layout = "grid" )
graph_report
A printed graph_report
displays the initial graph, p-values and
significance levels, rejection decisions, and optional detailed test results.
## S3 method for class 'graph_report' print(x, ..., precision = 4, indent = 2, rows = 10)
## S3 method for class 'graph_report' print(x, ..., precision = 4, indent = 2, rows = 10)
x |
An object of class |
... |
Other values passed on to other methods (currently unused) |
precision |
An integer scalar indicating the number of decimal places to to display. |
indent |
An integer scalar indicating how many spaces to indent results. |
rows |
An integer scalar indicating how many rows of detailed test results to print. |
An object x of class graph_report
, after printing the report of
conducting a graphical multiple comparison procedure.
Bretz, F., Posch, M., Glimm, E., Klinglmueller, F., Maurer, W., and Rohmeyer, K. (2011). Graphical approaches for multiple comparison procedures using weighted Bonferroni, Simes, or parametric tests. Biometrical Journal, 53(6), 894-913.
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 1 in Bretz et al. (2011). hypotheses <- c(0.5, 0.5, 0, 0) transitions <- rbind( c(0, 0, 1, 0), c(0, 0, 0, 1), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) p <- c(0.018, 0.01, 0.105, 0.006) alpha <- 0.025 graph_test_shortcut(g, p, alpha)
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 1 in Bretz et al. (2011). hypotheses <- c(0.5, 0.5, 0, 0) transitions <- rbind( c(0, 0, 1, 0), c(0, 0, 0, 1), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) p <- c(0.018, 0.01, 0.105, 0.006) alpha <- 0.025 graph_test_shortcut(g, p, alpha)
initial_graph
A printed initial_graph
displays a header stating "Initial graph",
hypothesis weights, and transition weights.
## S3 method for class 'initial_graph' print(x, ..., precision = 4, indent = 0)
## S3 method for class 'initial_graph' print(x, ..., precision = 4, indent = 0)
x |
An object of class |
... |
Other values passed on to other methods (currently unused). |
precision |
An integer scalar indicating the number of decimal places to to display. |
indent |
An integer scalar indicating how many spaces to indent results. |
An object x of class initial_graph
, after printing the initial
graph.
Bretz, F., Posch, M., Glimm, E., Klinglmueller, F., Maurer, W., and Rohmeyer, K. (2011). Graphical approaches for multiple comparison procedures using weighted Bonferroni, Simes, or parametric tests. Biometrical Journal, 53(6), 894-913.
print.updated_graph()
for the print method for the updated graph after
hypotheses being deleted from the initial graph.
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 1 in Bretz et al. (2011). hypotheses <- c(0.5, 0.5, 0, 0) transitions <- rbind( c(0, 0, 1, 0), c(0, 0, 0, 1), c(0, 1, 0, 0), c(1, 0, 0, 0) ) hyp_names <- c("H11", "H12", "H21", "H22") g <- graph_create(hypotheses, transitions, hyp_names) g
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 1 in Bretz et al. (2011). hypotheses <- c(0.5, 0.5, 0, 0) transitions <- rbind( c(0, 0, 1, 0), c(0, 0, 0, 1), c(0, 1, 0, 0), c(1, 0, 0, 0) ) hyp_names <- c("H11", "H12", "H21", "H22") g <- graph_create(hypotheses, transitions, hyp_names) g
power_report
A printed power_report
displays the initial graph, testing and simulation
options, power outputs, and optional detailed simulations and test results.
## S3 method for class 'power_report' print(x, ..., precision = 4, indent = 2, rows = 10)
## S3 method for class 'power_report' print(x, ..., precision = 4, indent = 2, rows = 10)
x |
An object of the class |
... |
Other values passed on to other methods (currently unused) |
precision |
An integer scalar indicating the number of decimal places to to display. |
indent |
An integer scalar indicating how many spaces to indent results. |
rows |
An integer scalar indicating how many rows of detailed test results to print. |
An object x of the class power_report
, after printing the report of
conducting power simulations based on a graphical multiple comparison
procedure.
Bretz, F., Posch, M., Glimm, E., Klinglmueller, F., Maurer, W., and Rohmeyer, K. (2011a). Graphical approaches for multiple comparison procedures using weighted Bonferroni, Simes, or parametric tests. Biometrical Journal, 53(6), 894-913.
Bretz, F., Maurer, W., and Hommel, G. (2011b). Test and power considerations for multiple endpoint analyses using sequentially rejective graphical procedures. Statistics in Medicine, 30(13), 1489-1501.
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 4 in Bretz et al. (2011). alpha <- 0.025 hypotheses <- c(0.5, 0.5, 0, 0) delta <- 0.5 transitions <- rbind( c(0, delta, 1 - delta, 0), c(delta, 0, 0, 1 - delta), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) marginal_power <- c(0.8, 0.8, 0.7, 0.9) corr1 <- matrix(0.5, nrow = 2, ncol = 2) diag(corr1) <- 1 corr <- rbind( cbind(corr1, 0.5 * corr1), cbind(0.5 * corr1, corr1) ) success_fns <- list( # Probability to reject both H1 and H2 `H1andH2` = function(x) x[1] & x[2], # Probability to reject both (H1 and H3) or (H2 and H4) `(H1andH3)or(H2andH4)` = function(x) (x[1] & x[3]) | (x[2] & x[4]) ) set.seed(1234) # Bonferroni tests power_output <- graph_calculate_power( g, alpha, sim_corr = corr, sim_n = 1e5, power_marginal = marginal_power, sim_success = success_fns )
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 4 in Bretz et al. (2011). alpha <- 0.025 hypotheses <- c(0.5, 0.5, 0, 0) delta <- 0.5 transitions <- rbind( c(0, delta, 1 - delta, 0), c(delta, 0, 0, 1 - delta), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) marginal_power <- c(0.8, 0.8, 0.7, 0.9) corr1 <- matrix(0.5, nrow = 2, ncol = 2) diag(corr1) <- 1 corr <- rbind( cbind(corr1, 0.5 * corr1), cbind(0.5 * corr1, corr1) ) success_fns <- list( # Probability to reject both H1 and H2 `H1andH2` = function(x) x[1] & x[2], # Probability to reject both (H1 and H3) or (H2 and H4) `(H1andH3)or(H2andH4)` = function(x) (x[1] & x[3]) | (x[2] & x[4]) ) set.seed(1234) # Bonferroni tests power_output <- graph_calculate_power( g, alpha, sim_corr = corr, sim_n = 1e5, power_marginal = marginal_power, sim_success = success_fns )
updated_graph
A printed updated_graph
displays the initial graph, the (final) updated
graph, and the sequence of intermediate updated graphs after hypotheses are
deleted (if available).
## S3 method for class 'updated_graph' print(x, ..., precision = 6, indent = 2)
## S3 method for class 'updated_graph' print(x, ..., precision = 6, indent = 2)
x |
An object of the class |
... |
Other values passed on to other methods (currently unused). |
precision |
An integer scalar indicating the number of decimal places to to display. |
indent |
An integer scalar indicating how many spaces to indent results. |
An object x of the class updated_graph
, after printing the updated
graph.
Bretz, F., Posch, M., Glimm, E., Klinglmueller, F., Maurer, W., and Rohmeyer, K. (2011a). Graphical approaches for multiple comparison procedures using weighted Bonferroni, Simes, or parametric tests. Biometrical Journal, 53(6), 894-913.
print.initial_graph()
for the print method for the initial graph.
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 1 in Bretz et al. (2011). hypotheses <- c(0.5, 0.5, 0, 0) transitions <- rbind( c(0, 0, 1, 0), c(0, 0, 0, 1), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) # Delete the second and third hypotheses in the "unordered mode" graph_update(g, delete = c(FALSE, TRUE, TRUE, FALSE)) # Equivalent way in the "ordered mode" to obtain the updated graph after # deleting the second and third hypotheses # Additional intermediate updated graphs are also provided graph_update(g, delete = 2:3)
# A graphical multiple comparison procedure with two primary hypotheses (H1 # and H2) and two secondary hypotheses (H3 and H4) # See Figure 1 in Bretz et al. (2011). hypotheses <- c(0.5, 0.5, 0, 0) transitions <- rbind( c(0, 0, 1, 0), c(0, 0, 0, 1), c(0, 1, 0, 0), c(1, 0, 0, 0) ) g <- graph_create(hypotheses, transitions) # Delete the second and third hypotheses in the "unordered mode" graph_update(g, delete = c(FALSE, TRUE, TRUE, FALSE)) # Equivalent way in the "ordered mode" to obtain the updated graph after # deleting the second and third hypotheses # Additional intermediate updated graphs are also provided graph_update(g, delete = 2:3)