Intraclass correlation coefficient (ICC) of BERT models.

Interrater agreement of log probabilities (treated as "ratings"/rows) among BERT language models (treated as "raters"/columns), with both row and column as ("two-way") random effects.

Usage

ICC_models(data, type = "agreement", unit = "average")

Arguments

data

Raw data returned from FMAT_run().

type

Interrater "agreement" (default) or "consistency".

unit

Reliability of "average" scores (default) or "single" scores.

Value

A data.table of ICC.