Transparent Practices for Quantitative Empirical Research

A CHI 2023 course

Hybrid

Instructors: Chat Wacharamanotham, Fumeng Yang, Xiaoying Pu, and Abhraneel Sarma

General Information

Transparent research practices enable the research design, materials, analytic methods, and data to be thoroughly evaluated and potentially reproduced. This course presents current best practices and tools for increasing research transparency.

After the course, we expect the participants to be able to get started on improving their own research practices, assessing research transparency in articles they read and review. Toward these goals, we design this course with the following specific learning outcomes:

The course participants…

  • will be aware of decisions made in the course of quantitative research and the importance of making them transparent,
  • will know a range of practices, methods, and tools to improve research transparency,
  • can apply principles to evaluate and compare the transparency of statistical and visualization techniques, and
  • can preregister and deposit their research artifacts in FAIR repositories.

Online sessions dates: Wednesday, April 12 Friday, April 14 Monday, April 17

Online sessions time: 9:00–10:15 (Pacific) = 11:00–12:15 (Central) = 12:00–13:15 (Eastern) = 18:00–19:15 (Europe)

On-site/hybrid sessions dates and time: Monday, April 24 late-morning session (exact time and location will be announced in the CHI program)

Schedule

April 12

18:00 – 19:15 (Germany) Planning, analyzing, and sharing quantitative research (online)

April 14

18:00 – 19:15 (Germany) Introduction to Transparent Bayesian Data Analysis (online)

April 17

18:00 – 19:15 (Germany) Introduction to Transparency-oriented Visualizations (online)

April 24

11:10 – 12:35 (Germany) Hands-on with transparency practices (on-site/hybrid)
End (TBA) Post-course survey

Registration

Course fee: $100 + CHI registration fee

  1. Go to CHI 2023 registration page
  2. Scroll to the bottom of the page and select the Member or Non-Member button to begin your registration.
  3. Fill in all of the required information needed for registration to attend CHI 2023.
  4. On the Courses and Workshops page – select the course CO8 from the list and click next at the bottom of the page.
  5. Continue through registration to complete by making a payment and clicking “Finish”. A confirmation email is automatically generated.

If you are adding a course to your existing registration:

  1. Go to CHI 2023 registration page
  2. Scroll towards the bottom and click on the “Already Registered” option. Do not begin a new registration if you are already registered for CHI 2023.
  3. Enter your email address and confirmation code – found in your confirmation email.
  4. Click “Login” to proceed, then select “Modify Registration” towards the bottom on the next page.
  5. Proceed through your registration (“Next” button until the Optional Items page) to add the desired course.
  6. Select the course CO8 from the list and click “Next” at the bottom.
  7. Follow through to complete the addition by making a payment and clicking “Finish”.

Support for the registration fee

  • Gary Marsden Travel Awards (Deadline February 9th, 11:59pm AoE). This fund prioritizes first-time attendees and presenters. You do not need to have a paper accepted to CHI to apply for this fund.

About instructors

Chat Wacharamanotham is a lecturer at Swansea University and a mandated instructor at the University of Zurich. The focus of his work is on understanding and developing tools for planning, reporting, reading, and sharing quantitative research. He is also a co-organizer of the Transparent Statistics in HumanComputer Interaction group and the Dagstuhl Seminar on Transparent Quantitative Research as a User Interface Problem. He has seven years of experience teaching a research method course for graduate students.

Fumeng Yang is a postdoctoral fellow at Northwestern University. Her recent research focuses on uncertainty visualizations for the general public and user modeling through statistical and machine learning models. She served as a Student Volunteers chair for IEEE VIS and co-instructed the precedent series of the proposed course.

Xiaoying Pu takes a human-centered approach to help data workers communicate uncertainty and data analyses. She has organized a CHI 2021 SIG on visualization grammars and studied the open science practice of preregistration.

Abhraneel Sarma is a Ph.D. candidate at Northwestern University. His research interests include studying how people make decisions using visualizations, and how visualizations can be used for improving statistical analysis or reporting statistical results. In addition, he has studied how users implement certain aspects of Bayesian models and has developed tools for conducting multiverse analysis which is an approach for more transparent statistical research.

Frequently Asked Questions

Is there any scholarships?

Gary Marsden Travel Awards (Deadline February 9th, 11:59pm AoE). This fund prioritizes first-time attendees and presenters. You do not need to have a paper accepted to CHI to apply for this fund.

Can I participate this course if I will only attend CHI online?

Yes. The first three sessions are online, and the last sessions is hybrid. See the information on the page of the last session above for the hybrid arrangement.

Do I need to know any programming language?

Some experience in R will be helpful but not strictly required. We will use R only for examples and will provide code for you to comment/uncomment to explore several scenarios.

Contact: chat@acm.org