IRW

The Item Response Warehouse (IRW) is an open-source, large-scale repository designed to advance psychometric research by standardizing and aggregating a large volume of item response datasets.

IRW Menu

• IRW Website: has a high-level overview of the datasets.
• IRW Paper: describes IRW in detail, including data format, inclusion criteria, and our vision in the future
• IRW Data Dictionary: maintains a record of the descriptions, origins, and licenses of the processed datasets.
• IRW Datasets: stores all processed datasets on Redivis.
• IRW Code: a list of code used to standardize datasets into IRW format.
• Contact Us: for any question, please contact us at itemresponsewarehouse@stanford.edu.

Installation & Getting Started

The easiest way to get access to IRW data is via the Redivis API

# 1. Make sure devtools is intalled from CRAN 
# install.packages("devtools")

# 2. The Redivis R package redivis-r needs to be installed
devtools::install_github("redivis/redivis-r", ref="main")

# 3. Generate a long-term Redivis API Token to avoid human intervention.
# Instructions at https://apidocs.redivis.com/client-libraries/redivis-r/getting-started

# 4. Install the IRW package to access essential APIs
devtools::install_github("ben-domingue/irw/irw_pkg")

For guidance to use IRW for data analysis in Python or R, please refer to the IRW website for comprehensive explanations and examples.

IRW Commandments

Below are critical instructions for formatting data for the IRW. More information about the IRW data standard is available in the preprint and by contacting the IRW maintainers.

1. Numeric values of a response should be meaningful.
2. If data come from an RCT, have a treatment column that is 1 if response comes from a treated respondent and 0 otherwise.
3. Response time should be in seconds.
4. Longitudinally collected responses should be in Unix time (seconds since Jan 1 1970 UTC).
5. If there are multiple scales available, split them into mutiple files.

Adding to the IRW

• To add data from the queue to the IRW repository there are three todo items:

  1. Create a Github issue for this repository that describes any decisions you had to make and also includes a file with | in IRW format. Note that this may not be appropriate if the data is not publicly sharable. Contact us at the below email if that is the case.
  2. Once we have confirmed that the data is appropriate, submit a pull request so that the code used to format the data gets added here. The pull request should go to the original repository (not your forked version of it) and to the main branch (unless you have a need to create a new branch).
  3. Finally, ensure the 'data index' page here gets updated with the relevant information.

• We have a queue of data that we aim to add to the IRW available here. We have typically tried to do some initial checks to ensure that these data are appropriate, but further investigation often suggests otherwise. Please feel free to each out to us at itemresponsewarehouse@stanford.edu.