Modeling attribute
relationships in diagnostic
models

With the R package measr

W. Jake Thompson, Ph.D.

Data: ECPE

• 28 items, measuring 3 attributes
  • Lexical, cohesive, and morphosyntactic language skills
• Used by Templin & Hoffman (2013) to demonstrate the LCDM
  • Templin & Bradshaw (2014) proposed a hierarchy among the attributes
  • Lexical -> Cohesive -> Morphosyntactic
• Available in the dcmdata package

# remotes::install_github("r-dcm/dcmdata")
library(dcmdata)

?ecpe

Specify a DCM with measr

• Unconstrained model with all possible profiles

# remotes::install_github("wjakethompson/measr@refactor")
library(measr)

lcdm_spec <- dcm_specify(
  ecpe_qmatrix, identifier = "item_id", measurement_model = lcdm(),
  structural_model = unconstrained()
)

• Constrained model that enforces the proposed hierarchy

hdcm_spec <- dcm_specify(
  ecpe_qmatrix, identifier = "item_id", measurement_model = lcdm(),
  structural_model = hdcm("lexical -> cohesive -> morphosyntactic")
)

Estimate the competing models

• The measr package wraps Stan for model estimation

mod_lcdm <- dcm_estimate(
  lcdm_spec,
  data = ecpe_data, identifier = "resp_id",
  iter = 1500, warmup = 1000, cores = 4
)

mod_hdcm <- dcm_estimate(
  hdcm_spec,
  data = ecpe_data, identifier = "resp_id",
  iter = 1500, warmup = 1000, cores = 4
)

1

Supply your model specification to dcm_estimate().

2

Simply swap the model specifications to fit a competing model.

Compare profile base rates

• Only profiles that conform to hierarchy are included in the HDCM

full_join(
  rename(measr_extract(mod_lcdm, "strc_param"), LCDM = estimate),
  rename(measr_extract(mod_hdcm, "strc_param"), HDCM = estimate),
  join_by(class)
)

#> # A tibble: 8 × 3
#>   class               LCDM          HDCM
#>   <chr>         <rvar[1d]>    <rvar[1d]>
#> 1 [0,0,0]  0.2976 ± 0.0165  0.32 ± 0.014
#> 2 [1,0,0]  0.0119 ± 0.0062    NA ± NA   
#> 3 [0,1,0]  0.0166 ± 0.0109    NA ± NA   
#> 4 [0,0,1]  0.1281 ± 0.0199  0.14 ± 0.019
#> 5 [1,1,0]  0.0093 ± 0.0057    NA ± NA   
#> 6 [1,0,1]  0.0184 ± 0.0099    NA ± NA   
#> 7 [0,1,1]  0.1731 ± 0.0198  0.18 ± 0.019
#> 8 [1,1,1]  0.3451 ± 0.0169  0.35 ± 0.016

Relative fit comparisons

• Model comparisons with relative fit indices such as leave-one-out cross validation (Vehtari et al., (2017))

loo_compare(mod_lcdm, mod_hdcm, criterion = "loo",
            model_names = c("LCDM", "HDCM"))

#>      elpd_diff se_diff
#> LCDM  0.0       0.0   
#> HDCM -3.7       4.7

• Because the difference in the elpd_diff is small (i.e., < se_diff), we would prefer the HDCM
  • Many fewer parameters, more parsimonious

https://measr.info

 wjakethompson.com
 wjakethompson@ku.edu
 in/wjakethompson
 @wjakethompson
 @wjakethompson.com
 @wjakethompson@fosstodon.org
 @wjakethompson
 @wjakethompson

Acknowledgements

The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grants R305D210045 and R305D240032 to the University of Kansas Center for Research, Inc., ATLAS. The opinions expressed are those of the authors and do not represent the views of the Institute or the U.S. Department of Education.