Data Upload
- Each row should represent the responses of a single student.
- Each column should contain responses to a single item.
- The first row must contain item names as column headings.
- Use 0 for incorrect answers and 1 for correct answers.
The dataset should contain only the responses to the items. Remove other variables, if any.
Data Preview
Frequency Distribution of Total Scores
Descriptive Statistics for the Items and the Total Score
Parameters
When the purpose is to select a limited group of examinees, such as only the top 10%, the widely used Classical Test Theory (CTT) and Item Response Theory (IRT) item discrimination indices may be biased because they estimate how well the entire score range is discriminated.
In this case, a special case of Brennan’s index is proposed by Arikan & Aybek (2022) to identify items for selecting a limited number of high-achieving examinees. Inspired by Brennan’s formula, this index calculates the difference between the performance of the upper 10% and lower 90% of the group on the item. It's important to note that the proportion defining the upper group is adjustable based on user requirements and is not fixed at 10%.
The formula is expressed as:
B10−90 = pU10 − pL90
where pU10
represents the proportion of students in the upper 10 percentile who answered the item correctly and pL90
denotes the proportion of students in the lower 90 percentile who answered the item correctly.
Results
A Special Case of Brennan’s Index Plot
Parameters
This index is designed for criterion-referenced tests or absolute assessments where a specific cut-off score is used to make decisions.
The index is calculated using the following formula:
BI = p(A) - p(B)
where p(A)
is the proportion of examinees above the cut-off score who answered the item correctly, and p(B)
is the proportion of examinees below the cut-off score who answered the item correctly (Brennan, 1972).
Results
Brennan's Index Plot
Discrimination Indexes
The discrimination index is a measure of how well a question discriminates between high and low scorers. It's calculated in the following steps:
- Rank all students based on their total test score.
- Identify the upper 27% and the lower 27% of students.
- For each item, calculate the proportion of students in the upper group who answered correctly (U) and the proportion in the lower group who answered correctly (L).
- The discrimination index is D = U - L.
The item-total correlation is the correlation between a scored item and the total test score. Higher positive values indicate that the item is more consistent with the overall test.
Item-Rest Correlation (Rir):The item-rest correlation is the correlation between a specific item and the total score of all other items in the scale, excluding the specific item. It helps to determine how well an item fits with the other items in the test.
Results
Discrimination Indices Plot
Parameters
Item Response Curves
Dogac, A., Aybek, E. C., Arikan, S., & Coskun, S. (2024). DiscrimHub [R Shiny application]. https://shiny.eptlab.com/discrimhub/
Developers:
Aybüke Doğaç, https://www.linkedin.com/in/aybukedogac/
Eren Can Aybek,https://www.linkedin.com/in/ecaybek/
Serkan Arıkan, https://www.linkedin.com/in/serkan-arikan-a62a38a2/
Sinem Coşkun, https://www.linkedin.com/in/sinem-co%C5%9Fkun-7632b5214/
Please refer to the following publications for further information:
Arikan, S., & Aybek, E. C. (2022). A special case of Brennan's index for tests that aim to select a limited number of students: A Monte Carlo simulation study. Educational Measurement: Issues and Practice, 41(4), 35-49.https://doi.org/10.1111/emip.12528
Brennan, R. L. (1972). A generalized upper-lower item discrimination index. Educational and Psychological Measurement, 32(2), 289-303.
If you have any questions, concerns, or feedback about the Shiny app, please do not hesitate to get in touch:
dogacaybuke@gmail.com