A Tool for Assessing the Quality of Reports of Agent-based Modeling (Adapted Version of the ODD Protocol)

Relevance. An agent-based approach to modeling is widely in demand across a wide range of scientific fields, including research into patterns of the spread of infectious diseases. Objective. To develop a Russian-language adapted version of the ODD protocol to make roprting of results from agent-based modeling more comprihensive and reliable in epidemiological studies. Materials and Methods. We used the authors' original methodological guideline in English, «Overview, Design Concept, Details», or ODD. Our adapted version of the ODD protocol was prepared with the identification of critical user points. The consistency of expert assessments was achieved by calculating the kappa coefficient according to the Fleiss method. All statistical analyses wereperformed in R 4.3.2 (RStudio). Results and Discussion. To the best of our kmowledge, we have first proposed a Russian-language version of ODD. This guidance has been designed to fully describe the process of model development and substantiate the results for use in biomedical research. The article primarily focuses on the "Design Concepts" section. The recommendations outlined therein help justify the construction of a design model and make its presentation less arbitrary and more modern. The Russian version of ODD presents all 11 elements of study design, with a brief description of their purpose and a checklist of topics to consider. The design elements include teh foloowing: basic principles, emergence of new results, adaptation, objectives, learning, prediction, sensing, interaction of agents, process stochasticity, and observation. We provide a user with the description of both a full and a summarized versions of a report based on the ODD recommendations. As for the summarized version, a reader will be provided with a short text template for manuscript formatting. Conclusion. Using the ODD protocol, researchers will be able to describe the agent-based model in detail as a whole, as well as reflect its essential, most characteristic and most sophisticated characteristics, which are not always easily represented by mathematical equations and flowcharts.

1. Akimkin V.G., Mikheeva I.V. Concept of digital transformation of the epidemiological surveillance system. Epidemiology and infectious diseases. Current issues. 2025;2:16–20. (In Russ). doi: 10.18565/epidem.2025.15.2.16–20

2. Pushkareva T.P. Mathematical modeling as a necessary component of mathematical training. Modern problems of science and education. 2014;5:796–798. (In Russ).

3. Hunter E, Namee BM, Kelleher J. An open-data-driven agent-based model to simulate infectious disease outbreaks. PLOS One. 2018;13(12):e0208775. doi:10.1371/journal.pone.0208775.

4. Gerasimov AN. Models and statistical analysis in the epidemiology of infectious diseases. TMZh. 2019;3:80–83. (In Russ).

5. Denisov NS, Kamenskikh EM. Fedorov OS. Trends in population studies: molecular and digital epidemiology (review). STM. 2022;14(4):60.

6. Kondratiev MA. Methods of forecasting and models of disease spread. Computer research and modeling. 2013;5(5):863–882.

7. Kosova AA, Chalapa VI, Kovtun OP. Methods of modeling and forecasting the dynamics of the epidemic process of infectious diseases. Ural Medical Journal. 2023;22(4):102–112.

8. Brauer F. Mathematical epidemiology: Past, present, and future. Infect Dis Model. 2017;2(2):113–127. doi: 10.1016/j.idm.2017.02.001.

9. Hackl J, Dubernet T. Epidemic spreading in urban areas using agent-based transportation models. Future Internet. 2019;11(4):92. doi: 10.3390/fi11040092.

10. Gryaznov SA, Kuznetsov MI. Factors influencing the reproducibility of scientific research. International Journal of Humanities and Natural Sciences. 2024;8-1(95);69–71.

11. Baker M. 1,500 scientists lift the lid on reproducibility. Nature. 2016;533:452–454. doi: 10.1038/533452a

12. Richter S. Systematic heterogenization for better reproducibility in animal experimentation. Lab Anim. 2017;46:343–349. doi:10.1038/laban.1330

13. Grant R, Rubin M, Abbas M, et al. Expanding the use of mathematical modeling in healthcare epidemiology and infection prevention and control. Infection Control & Hospital Epidemiology. 2024;45(8):930–935. doi:10.1017/ice.2024.97

14. Railsback SF, Grimm V. Agent-based and individual-based modeling: a practical introduction. Princeton (NJ): Princeton University Press; 2019.

15. Bennett C, Manuel DG. Reporting guidelines for modelling studies. BMC Med Res Methodol. 2012;12:168. doi: 10.1186/1471-2288-12-168.

16. Chaturvedi MB, Denkinger A, Klett-Tammen C, et al. Guidelines on reporting and assessing mathematical models for infectious disease dynamics: A scoping review. medRxiv. 2024. doi: 10.1101/2024.11.27.24318060.

17. Pollett S, Johansson MA, Reich NG, et al. Recommended reporting items for epidemic forecasting and prediction research: The EPIFORGE 2020 guidelines. PLoS Med. 2021;18(10):e1003793. doi: 10.1371/journal.pmed.1003793

18. Achter S, Borit M, Chattoe-Brown E, Siebers P-O. RAT-RS: a reporting standard for improving the documentation of data use in agent-based modelling. International Journal of Social Research Methodology. 2022. doi: 10.1080/13645579.2022.2049511

19. Vincenot C.E. How new concepts become universal scientific approaches: insights from citation network analysis of agent-based complex systems science. Proceedings of the Royal Society B: Biological Sciences. 2018;285(1874):20172360.

20. Grimm V, Berger U, Vincenot CE, et al. The ODD protocol for describing agent-based and other simulation models: a second update to improve clarity, replication, and structural realism. Journal of Artificial Societies and Social Simulation. 2020;23(2):7. https://www.jasss.org/23/2/7.html

21. Grimm V, Berger U, Bastiansen F, et al. A standard protocol for describing individual-based and agent-based models. Ecological Modelling. 2006; 198:115–126. doi:10.1016/j.ecolmodel.2006.04.023

22. Li M, Gao Q, Yu T. Kappa statistic considerations in evaluating inter-rater reliability between two raters: which, when and context matters. BMC Cancer. 2023;23:799. doi: 10.1186/s12885-023-11325-z

23. Wilkinson MD, Dumontier M, Aalbersberg IJ, et al. The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data. 2016;3:160018. doi:10.1038/sdata.2016.18

24. da Costa GG, Neves K, Amaral OB. Estimating the replicability of highly cited clinical research (2004-2018). PLoS One. 2024;19(8):e0307145. doi: 10.1371/journal.pone.0307145.

25. Fatima G, Alhmadi HB, Mahdi AA, et al. Monkeypox outbreaks: a comprehensive review of epidemiology, clinical management, and public health measures. Infection and immunity. 2025;15(2):227–234. (In Russ)

26. Ali M, El Guma F, Qazza A, et al. Stochastic modeling of influenza transmission: Insights into disease dynamics and epidemic management. Partial Differential Equations in Applied Mathematics. 2024;11(100886): doi: 10.1016/j.padiff.2024.100886.