AutoFrontCCS
The AutoFrontCCS project is part of the second funding phase of the DFG-funded priority program “META-REP: A meta-scientific research program to analyse and optimise replicability in the behavioral, social, and cognitive sciences.” The project develops tools and methods that support researchers in documenting studies to increase reproducibility and replicability early on in the publication process.
Project Objectives
The aim of the AutoFrontCCS project is to develop, test, and evaluate solutions that improve the replicability and reproducibility of research in the young but rapidly growing field of computational communication science (CCS). The project builds on the results of the predecessor project “RepCCS: What Defines and Affects Replicability in Computational Communication Science?”
The previous project showed that communication science in general, and CCS in particular, face several challenges that can hinder the replicability of research. Two issues have become especially evident. First, the distinction between reproduction and replication is less clearly defined in CCS than in many other research fields. Second, there is a substantial gap between the materials researchers typically provide and the information actually required for replication studies. Both issues are closely related to the topics, methods, and data commonly used in CCS.
To address these challenges, the AutoFrontCCS project develops tools to promote what is referred to as “proactive replicability.” These tools are intended to be used already during the submission and review of research papers in journals and conferences. In addition to established measures such as checklists, innovative approaches based on large language models are also employed. The development of these tools is carried out in close collaboration with the research community, and their effectiveness is systematically evaluated within the project.
“We develop and test solutions to improve reproducibility in the rapidly growing field of computational communication science. This includes both established tools such as checklists and innovative LLM-based approaches.“
Prof. Johannes Breuer
AutoFrontCCS - Fostering Proactive Replicability in Computational Communication Science via Frontloading Effort and Automating Protocols
Project Duration: 03/2025 to 03/2028
Funded by the German Research Foundation
Contact Person
Prof. Johannes Breuer
johannes.breuer@cais-research.de
Directly to …
Research Focus at CAIS
The researchers involved at CAIS are primarily responsible for the project’s second work package. This work package is titled “Evaluation and Community” and comprises three central areas with distinct tasks.
1. Assessing Needs and Engaging the Community
First, the perspectives and needs of researchers in the field of computational communication science (CCS) are systematically collected and incorporated into the project:
- Community Survey: An online survey gathers insights from CCS researchers about their experiences with replicability and reproducibility. Additionally, their assessments of the solutions planned in the project are collected.
- Qualitative Interviews: Complementary structured interviews are conducted with journal editors and division chairs of key CCS conferences. The goal is to obtain additional feedback and practical insights from the publishing and reviewing process.
2. Systematic Evaluation of the Tools Developed in the Project
Another focus is the systematic evaluation of the tools developed in the project:
- Experimental Study: Authors of studies already accepted for publication by a journal or conference are randomly assigned to one of three groups. The first group uses a checklist, the second group uses a tool based on large language models, and the third group works without any additional tool and serves as a control group.
- Subjective Evaluation: Users then assess their experiences with the tools and their usefulness in the workflow.
- Objective Evaluation: In addition, the replicability of the studies from the experiment is checked in so-called Replication Games, where participants attempt to reproduce each study based on the materials provided.
3. Dissemination and Training
Finally, materials are developed to support the use of the tools created in the project:
-
These include training materials (documentation and tutorials) for researchers, journal editors, and conference program chairs to facilitate and promote the use of the developed tools.
Xinyue Zhao & Prof. Johannes Breuer
Project Partners
The project is carried out in collaboration with Prof. Dr. Mario Haim from Ludwig-Maximilians-Universität München and is part of the DFG-funded priority program “SPP 2317: META-REP – A meta-scientific program to analyse and optimise replicability in the behavioral, social, and cognitive sciences.”
Funded by:
Team AutoFrontCCS
