Skip to content

Agenda

The times provided are approximated for the maximum amount of time spent on each section. The sections may be shorter depending on the session.

  1. Introduction to Open Science Practices (Lecture) - 1 hour
    1. Why should we do Open Science?
    2. A Brief Disclaimer
    3. Preregistration
      1. Side Note: Registered Reports
    4. Open Data
    5. Open Materials
    6. Publishing Papers
      1. Open Access
      2. Preprints
    7. Break and Q&A
  2. Working with the Open Science Framework (Interactive) - 30 minutes
    1. OSF Project
    2. OSF Preregistration
    3. Break and Q&A
  3. Debugging and Fixing Reproducibility (Interactive) - 50 minutes
    1. What software do you use?
    2. Reproducibility as relevant to the participants
    3. Break and Q&A
  4. Fixing Reproducibility within Others Work (Interactive) - 40 minutes
    1. Gurung, A., Botelho, A. F., Heffernan, N. T. (2021). Examining Student Effort on Help through Response Time Decomposition. In LAK21: 11th International Learning Analytics and Knowledge Conference (LAK21). Association for Computing Machinery, New York, NY, USA, 292–301. https://doi.org/10.1145/3448139.3448167
    2. Shaw, S. T., Luna, M. L., Rodriguez, B., Yeh, J., Villalta, N., Ramirez, G. (2022). Mathematical Creativity in Elementary School Children: General Patterns and Effects of an Incubation Break. Front. Educ. 7:835911. https://doi.org/10.3389/feduc.2022.835911

Tutorial Structure

The tutorial will occur over three hours and focuses on introducing some common open science principles and their usage within education technology, providing an example on using the Open Science Framework to create a project, post content, and preregister studies, and using previous papers to apply the learned principles and any additional reproduction mitigations.

  1. We will provide a presentation on an overview of a few problems when conducting research. Using this as a baseline, we will introduce open science and its principles and practices and how they can be used to nullify some of these issues and mitigate others. In addition, we will attempt to dispel some of the misconceptions of these principles and practices.
  2. We will provide a live example of using the Open Science Framework website to make an account, create a project, add contributors, add content and licensing, and publicize the project for all to see. Afterwards, we will provide a guide to creating a preregistration, explaining best practices, and identifying how to create an embargo. Additional features and concerns, such as anonymizing projects for review and steps required to properly do so, will be shown.
  3. We will discuss reproducibility metrics within work when providing datasets and materials. This will review commonly used software and languages (e.g. Python, RStudio, etc.) and how, without any steps taken, most work tends to be extremely tedious to reproduce or are not reproducible in general. Afterwards, we will provide some mitigation strategies needed to remove these concerns.
  4. We will take our own research from prior conferences that do not meet some open science principles or cannot easily be reproduced and apply what has been learned across the entire tutorial. We will use a few papers, each containing different issues, and apply the necessary steps needed to reproduce the results within the paper.