Compare NPAG or IT2B runs

Compare NPAG or IT2B runs. his function is superseded by PM_compare.

Usage

PMcompare(x, y, ..., icen = "median", outeq = 1, plot = F)

Arguments

x

The run number of the first object you wish to compare. This should be a folder in your working directory. To avoid confusion, this function does not use objects already loaded with PMload. This will serve as the reference output for P-value testing (see details).

y

The run number of the second object to compare.

...

Additional run numbers to compare. See details. Also, parameters to be passed to plot.PMop if plot is true as well as to mtsknn.eq. Order does not matter.

icen

Can be either "median" for the predictions based on medians of pred.type parameter value distributions, or "mean". Default is "median".

outeq

Number of the output equation to compare; default is 1

plot

Boolean operator selecting whether to generate observed vs. predicted plots for each data object as in plot.PMop

Value

A data frame with the following objects for each model to analyze:

run

The run number of the data

type

NPAG or IT2B data

nsub

Number of subjects in the model

nvar

Number of random parameters in the model

par

Names of random parameters

cycles

Number of cycles run

converge

Boolean value if convergence occurred.

ll

Final cycle -2*Log-likelihood

aic

Final cycle Akaike Information Criterion

bic

Final cycle Bayesian (Schwartz) Information Criterion

popBias

Bias, or mean weighted prediction error of predictions based on population parameters minus observations

popImp

Imprecision, or bias-adjusted mean weighted squared error of predictions based on population parameters minus observations

popPerRMSE

Percent root mean squared error of predictions based on population parameters minus observations

postBias

Bias, or mean weighted prediction error of predictions - observations based on posterior parameters

postImp

Imprecision, or bias-adjusted mean weighted squared error of predictions - observations based on posterior parameters

postPerRMSE

Percent root mean squared error of predictions based on posterior parameters minus observations

pval

P-value for each model compared to the first. See details.

Details

For backwards compatibility, objects can be specified separated by commas, e.g. PMcompare(1,2,3) followed by any arguments you wish to plot.PMop, mtsknn.eq. P-values are based on comparison using the nearest neighbors approach if all models are non-parametrics. Models may only be compared on parameters that are included in the first model. The P-value is the comparison between each model and the first model in the list. Missing P-values are when a model has no parameter names in common with the first model, and for the first model compared to itself, or when models from IT2B runs are included. Significant P-values indicate that the null hypothesis should be rejected, i.e. the joint distributions between the two compared models are significantly different.

See also

PMload, plot.PMop, mtsknn.eq

Author

Michael Neely