ConQuest: preparation of data, work with the program, the interpretation

of output data

Galina Larina

28-31 of March, 2012University of Ostrava

About this program

Advantages Disadvantages• Works with a big

number of models, including Many-Facet and Multidimensional ones (Rasch)

• Reports a confidence interval for fit statistics

• Good item analysis• Creates a variable map• Many outputs

• Requires making of control file

• Requires special knowledge on interpretation outputs in complex analysis

• Doesn’t work with 1PL and 2PL models and their polytomous extensions

• Generalized Item Response Modeling Software• ConQuest developed by Australian Council for Educational Research (ACER) and University of California, Berkeley

https://shop.acer.edu.au/acer-shop/group/CON2

Application

• Performing item analysis• Exploring rater effects• Examining DIF• Estimating latent correlation and testing

dimensionality• Fitting a wide variety of item response models:– Rasch’s Model– Rating Scale Model– Partial Credit Model– Multifaceted Models– Multidimensional Item Response Models– etc.

Performing item analysis (Rasch analysis) .shw

These tables are for the term item (dichotomous items) and term item*step (polytomous items), as the first and second terms in the model statement

Performing traditional item analysis .itn

This table shows summary results

These outputs conclude tables showing classical difficulty, discrimination and point-biserial statistics for each items (dichotomous and polytomous)

Performing item analysisMap of latent distributions .shw

There are two maps in ouputs: - response model parameter estimates - generalized-items tresholds

This histogram illustrates the distribution of student’s achievement. In this example each ‘X’ means 9.7 cases.

Items are plotted to indicate their difficulty level

These are Thurstonian thresholdes for each of the items. The notation x.y is used to indicate the y-th thresholds of the x-th item.

ConQuest PlotsDichotomous item

ConQuest PlotsPolytomous item

ConQuestExaminees

ID number Raw score that Maximum Student’s latent Standard error student attained possible score ability

Steps of workData• No missings, recode ones– Only numerical or letter symbols in matrix data

• Individual file with matrix data– Without unique ID

– Or with unique ID (in columns 1 through 9)

• Save your data in Notepad and name it like ex1data.dat

Steps of workVariable labels• Individual file with variable labels looks like

• First line of the file is required===> item

• Amount of spaces doesn’t matter• In this example the label for item 1 is BSMMA01, the

label for item 2 is BSMMA02, and so on. • Save your data in Notepad and name it like

ex1names.dat

Steps of workCommand File• Example

• Save your command file in notepad and name it like ex1run.dat

ConQuest Commands• Datafile indicates the name and location of

the data file• Format statement describes the layout of

the data in the file ex1data.dat. In this example id 1-9 means unique id is located in columns 1 through 9. And responses 10-26 means that the responses to the items are in columns 10 through 26

• Labels indicates the name and location of the file with variable labels

• Export logfile indicates the name and location of the logfile

• Codes identifies all valid codes in data file

ConQuest Commands• Key statement identifies the correct response for each

of multiple-choice item. – Dichotomous test:

Key 14323487 ! 1;– Non-dichotomous test:

Key 4111111411231411 ! 1;Key xxxx22xxxxxx2xx2 ! 2;

• Model specifies the item response model that is to be used in the estimation.– model item in case of simple logistic model. We are

dealing with single-faceted dichotomous data– model item + item*step in case of PCM. We are

dealing with polytomous items or a mixture of dichotomous and polytomous data

– model item + step in case of RSM. We are dealing with polytomous items, where the step parameters are the same for all items

– And so on…

ConQuest Commands• Estimate statement initiates the estimation of the item

response model. You can select some special options for your analysis:– type of method– maximum number if iterations– etc.

• Show statement produces a sequence of tables that summarizes the result of fitting the item response model. The result are redirected to a file ex1.shw in this example.

• Show cases statement produces a display of the results of a examinee analysis. The result are redirected to a file ex1_stud.shw in this example. You can select the type of estimate - it can be eap, latent, mle or wle.

• Itanal statement produces a display of the results of a traditional item analysis. The result are redirected to a file ex1.ita in this example.

ConQuest Run the program

2. Run – Run all

1. File – Open – Find your

command file

ConQuest ManualManual consist of four sections:– Introduction provides a brief survey of the

models that ConQuest can fit– Tutorial contains nine samples of ConQuest

analysis and describes how to use the program to address particular problems without any underlying methodology

– Technical Matters provides underlying in ConQuest methodology

– Command Reference contains general information about the syntax of ConQuest statements

Exploring rater effectsRaters .shw

Fit statistics for the raters. These ones lap over it’s confident interval.

Exploring rater effectsCriteria .shw

Fit statistics for the criteria. These ones lap over it’s confident interval.

Exploring rater effectsMaps of the parameter estimates.shw

Examinee Rater.Criteria.Step

Examinee Rater Criteria

Exploring rater effectsPlots

Testing dimensionality

Multidimensional model Control file

Multidimensional model .shw Correlations/covariance between dimensions

COVARIANCE/CORRELATION MATRIX

Dimension ------------------Dimension 1 2

Dim 1 0.553 Dim 2 0.928 -------------------------------------------Variance 0.624 0.570 -------------------------------------------

Covariance coefficients

Correlations coefficients

Multidimensional model .shw Reliability coefficients

RELIABILITY COEFFICIENTS------------------------ Dimension: (Dim 1) ----------------------- MLE Person separation RELIABILITY: Unavailable WLE Person separation RELIABILITY: Unavailable EAP/PV RELIABILITY: 0.871 ------------------------Dimension: (Dim 2) ----------------------- MLE Person separation RELIABILITY: Unavailable WLE Person separation RELIABILITY: Unavailable EAP/PV RELIABILITY: 0.849

Between-Item

лалала

Multidimensional model .shw Examinees

Dimension 1

Dimension 2