Skip to contents

r lifecycle::badge("stable")

Compare parameters, convergence, -2*log likelihood, AIC and bias and imprecision of population and posterior predictions.

Usage

PM_compare(x, y, ..., icen = "median", outeq = 1, plot = F)

Arguments

x

The first PM_result object you wish to compare. Unlike the legacy PMcompare this function only uses objects already loaded with PM_load. This will serve as the reference output for P-value testing (see details).

y

The second PM_result object to compare.

...

Additional PM_result objects to compare. See details. Also, parameters to be passed to plot.PM_op if plot is true as well as to mtsknn.eq. Order does not matter.

icen

Can be either "median" for the predictions based on medians of pred.type parameter value distributions, or "mean". Default is "median".

outeq

Number of the output equation to compare; default is 1

plot

Boolean operator selecting whether to generate observed vs. predicted plots for each data object as in plot.PM_op.

Value

A data frame with the following objects for each model to analyze:

run

The run number of the data

type

NPAG or IT2B data

nsub

Number of subjects in the model

nvar

Number of random parameters in the model

par

Names of random parameters

cycles

Number of cycles run

converge

Boolean value if convergence occurred.

ll

Final cycle -2*Log-likelihood

aic

Final cycle Akaike Information Criterion

bic

Final cycle Bayesian (Schwartz) Information Criterion

popBias

Bias, or mean weighted prediction error of predictions based on population parameters minus observations

popImp

Imprecision, or bias-adjusted mean weighted squared error of predictions based on population parameters minus observations

popPerRMSE

Percent root mean squared error of predictions based on population parameters minus observations

postBias

Bias, or mean weighted prediction error of predictions - observations based on posterior parameters

postImp

Imprecision, or bias-adjusted mean weighted squared error of predictions - observations based on posterior parameters

postPerRMSE

Percent root mean squared error of predictions based on posterior parameters minus observations

pval

P-value for each model compared to the first. See details.

Details

Objects can be specified separated by commas, e.g. PM_compare(run1, run2, run3) followed by any arguments you wish to plot.PMop, mtsknn.eq. P-values are based on comparison using the nearest neighbors approach if all models are non-parametrics. Models may only be compared on parameters that are included in the first model. The P-value is the comparison between each model and the first model in the list. Missing P-values are when a model has no parameter names in common with the first model, and for the first model compared to itself, or when models from IT2B runs are included. Significant P-values indicate that the null hypothesis should be rejected, i.e. the joint distributions between the two compared models for that parameter are significantly different.

Author

Michael Neely