Scientific Large Scale Infrastructure for Computing/Communication Experimental Studies

ACM SIGCOMM 2025 Non-Paper Session

Research Infrastructures and Tools for Reproducible Data Communication Research

Date, Time and Room

Organizers

Motivation and Goals

Experimental research on networked systems requires suitable research infrastructures (RIs). For many years, the dominating approach of research groups was to create a specific experimental setup tailored to the needs of a specific set of experiments, e.g., in the context of a PhD thesis. The community is well aware of the obvious shortcomings of this approach:

The need to address such shortcomings has been identified for a long time, e.g., in the 2003 workshop of ACM SIGCOMM “Models, Methods and Tools for Reproducible Network Research” [1]. To overcome such challenges, the networked systems community built testbed RIs, such as GENI [2], FABRIC [3], Chameleon [4], CloudLab [5], and Fed4Fire [6].

While these initiatives demonstrated significant progress, many areas for improvement remain. Artifact evaluation committees need to spend significant effort to evaluate papers with artifacts (software and data with metadata) as experimental artifacts and the involved tools are highly customized and heterogeneous. At the same time, experimenters have to invest significant effort to prepare their artifacts for evaluation and reusability. This approach clearly does not scale, in particular in the context of data-driven science powered by AI/ML. An area for improvement is experiment control. So far, scientists who use a large testbed for specific experiments need to solve how to orchestrate their experiments, how to collect, process, and store the data produced by the experiment, and how to add metadata to support other scientists. Consequently, there is a high heterogeneity concerning the artifacts of specific experiments. The SLICES Research Infrastructure [7] introduced several concepts for the evolution of networking testbeds into scientific instruments, with comprehensive support for experiment control and data management.

The recent Dagstuhl seminar 24462 [8] entitled “Research Infrastructures and Tools for Collaborative Networked Systems Research” focused on bridging the gap between the services provided by large-scale testbed infrastructures and the needs of researchers conducting cutting-edge experiments. In this seminar, the networking community, the users of RIs, and the testbed community, the developers and providers of RIs and data management solutions, exchanged their viewpoints. The seminar concluded with a set of recommendations. A key goal of the non-paper session is to share and discuss these conclusions and recommendations with the SIGCOMM community. Conclusions to be discussed address strategic investment and community engagement, open access and data sharing, amplifying impact and network effects. Recommendations to be discussed include scientific objectives, common abstractions, reproducibility and repeatability, FAIR principles (Findability, Accessibility, Interoperability, and Reusability), flexibility and adaptability interoperability and openness, user experience, international collaboration and support of sustainable development goals (SDGs).

Planned agenda

References