From 21481bdc6ce9343929423af3939e1c892ac25948 Mon Sep 17 00:00:00 2001 From: "Maryann E. Martone" Date: Tue, 6 May 2025 15:00:39 -0700 Subject: [PATCH 1/5] Create longform.md Longer explanations of each of the principles --- content/about/principles/longform.md | 53 ++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) create mode 100644 content/about/principles/longform.md diff --git a/content/about/principles/longform.md b/content/about/principles/longform.md new file mode 100644 index 0000000..3b96f9e --- /dev/null +++ b/content/about/principles/longform.md @@ -0,0 +1,53 @@ +# ReproNIM Principle 1: Study planning + +Study planning represents a foundational step in ensuring reproducible neuroimaging research. Many problems that lead to reproducibility issues can be avoided by proper planning + +1. **Implement good science basics, e.g, power analysis, statistical consults** + If your study is underpowered or your study design is flawed, you will have trouble publishing your work and, more to the point, your work will likely not be reproducible ([Scuzs and Ioannidis, 2020](https://www.sciencedirect.com/science/article/pii/S1053811920306509#bib24)). + * ***ReproNim resources*****:** + * [Repronim training module in statistics for neuroimaging](https://www.repronim.org/module-stats/): covers all aspects of statistics for neuroimaging, including sampling, power analysis and more + + * ***Other resources:*** + * NINDS-funded [Community for Rigor (C4R)](https://c4r.io/): tools and resources for improving rigor and reproducibility, from basics to advanced + + * Statistical consult: Most universities have statisticians available for consultation. The most important time to consult with a statistician is at the conceptualization phase, ***not*** after the data are collected. + +2. **Use pre-existing data for planning and/or analysis** + Leveraging pre-existing data, whether your own or data available from public repositories, can help validate methodological approaches before collecting new data and potentially increase your sample size + + * ***ReproNim resources*****:** + ReproNim has tools and approaches to make it easier to find and reuse your own data or publicly available data + * [ReproLake](https://repronim.netlify.app/resources/tools/reprolake/): a searchable data store for public neuroimaging data, built on the NIDM and BIDS standards + * RepoPond: A local searchable metadata store for your neuroimaging data, built with the same standards as the ReproLake. + * [Neurobage](https://repronim.netlify.app/resources/tools/neurobagel/)l: A system for distributed data sharing and discovery built on common annotations across the workflow + * Tutorial: [Sharing and Searching Metadata Using a ReproPond and the ReproLake](https://repronim.netlify.app/resources/tutorials/pond-lake/) + + * ***Other resources:*** + * OpenNeuro (and other repositories) + +3. **Create a data management and sharing plan (DMSP)** + A key component of study planning is carefully considering data management needs from the start. By anticipating the volume and types of data that will be generated, researchers can allocate appropriate resources for storage, processing, and sharing. The NIH's requirement for a Data Management and Sharing Plan (DMSP) in all proposals helps formalize this planning process. The DMSP requires researchers to specify crucial details like which data standards will be implemented and where data will ultimately be shared. By addressing these requirements during the planning phase, researchers can integrate data sharing preparations into their workflow from the beginning, rather than treating it as an burdensome afterthought at the study's conclusion. + + * ***ReproNim resources:*** + * [How ReproNim can help you create an NIH-compliant DMSP](https://repronim.netlify.app/resources/tutorials/data-management-and-sharing/) + * [Planning ahead for multi-site data collection using ReproSchema](https://repronim.netlify.app/resources/tutorials/reproschema/) + + * ***Other resources:*** + * [NIH DMSP website](https://sharing.nih.gov/data-management-and-sharing-policy/planning-and-budgeting-for-data-management-and-sharing/writing-a-data-management-and-sharing-plan#after) + +4. **Adopt open consent to allow broad sharing of data** + To ensure that data can be broadly shared, it is important that research participants provide consent for broad public sharing of the data, rather than for narrow purposes. + + * ***Resources*** + * The [Open Brain Consent](https://open-brain-consent.readthedocs.io/en/stable/) project provides guidance, tools and templates for incorporating open consent into neuroimaging studies while still providing protections for human subjects through de-identification of data. + +5. **Pre-register your study to increase transparency and build trust** + Pre-registering a study involves documenting and publishing the research plan, including details about study hypotheses, design and analysis, before the study begins. Preregistration makes research more transparent and reproducible, helps reduce duplication of research, can improve study design and separates hypothesis-generating (exploratory) from hypothesis-testing (confirmatory) research. Some journals support Registered Reports, a peer reviewed study pre-registration which guarantees the publication of the subsequent results if the study adheres to the protocol, regardless of whether or not the hypothesis was supported. +* ***Resources*** + * [Center for Open Science pre-registration site](https://www.cos.io/initiatives/prereg) + + * Lakens, D., Mesquida, C., Rasti, S., & Ditroilo, M. (2024). The benefits of preregistration and Registered Reports. *Evidence-Based Toxicology*, *2*(1). [https://doi.org/10.1080/2833373X.2024.2376046](https://doi.org/10.1080/2833373X.2024.2376046) + + * Example of publication of a null result after pre-registration: + * Kliemann D, Galdi P, Van De Water AL, Egger B, Jarecka D, Adolphs R, Ghosh SS. Resting-State Functional Connectivity of the Amygdala in Autism: A Preregistered Large-Scale Study. Am J Psychiatry. 2024 Dec 1;181(12):1076-1085. [doi: 10.1176/appi.ajp.20230249](https://psychiatryonline.org/doi/10.1176/appi.ajp.20230249). + From e903d0737aeffcae094106aa5f2b3fb4055c3201 Mon Sep 17 00:00:00 2001 From: "Maryann E. Martone" Date: Tue, 6 May 2025 15:06:36 -0700 Subject: [PATCH 2/5] Create _index.md For the principle long form pages --- content/about/principles/_index.md | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 content/about/principles/_index.md diff --git a/content/about/principles/_index.md b/content/about/principles/_index.md new file mode 100644 index 0000000..e4af612 --- /dev/null +++ b/content/about/principles/_index.md @@ -0,0 +1,7 @@ +--- +title: "Principles" +description: "Additional materials for each ReproNim principle" +weight: 1 +--- + +Here we provide more information about each principle and resources that can help put the principle into practice From e3972153d013c2a41d308a950d082d1eeca96d58 Mon Sep 17 00:00:00 2001 From: "Maryann E. Martone" Date: Tue, 3 Jun 2025 12:09:30 -0700 Subject: [PATCH 3/5] Update longform.md Added header --- content/about/principles/longform.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/content/about/principles/longform.md b/content/about/principles/longform.md index 3b96f9e..b1a0ef7 100644 --- a/content/about/principles/longform.md +++ b/content/about/principles/longform.md @@ -1,3 +1,10 @@ +--- +title: "Planning" +type: docs +weight: 80 +--- + + # ReproNIM Principle 1: Study planning Study planning represents a foundational step in ensuring reproducible neuroimaging research. Many problems that lead to reproducibility issues can be avoided by proper planning From 459056ce31756aef0f4f077eb779e44bcb2e1028 Mon Sep 17 00:00:00 2001 From: Austin Macdonald Date: Tue, 3 Jun 2025 15:35:37 -0500 Subject: [PATCH 4/5] Reorganize based on discussion with memartone --- content/about/in-practice.md | 97 ------------------- content/about/principles/_index.md | 96 +++++++++++++++++- .../principles/{longform.md => planning.md} | 6 +- 3 files changed, 95 insertions(+), 104 deletions(-) delete mode 100644 content/about/in-practice.md rename content/about/principles/{longform.md => planning.md} (98%) diff --git a/content/about/in-practice.md b/content/about/in-practice.md deleted file mode 100644 index 736b08f..0000000 --- a/content/about/in-practice.md +++ /dev/null @@ -1,97 +0,0 @@ ---- -title: "Reproducible Neuroimaging: Principles and Actions" -type: docs -weight: 80 ---- - -## ReproNim's principles of reproducible neuroimaging - - - - - - - - -1. **Study planning** - - 1. Implement good science basics ([power analysis, statistical consults, etc.](https://www.repronim.org/module-stats/)). - - 2. Use pre-existing data for planning and/or analysis. - - 3. Create an [NIH-compliant data management and sharing plan](https://sharing.nih.gov/data-management-and-sharing-policy/planning-and-budgeting-for-data-management-and-sharing/writing-a-data-management-and-sharing-plan#after). - - 4. Adopt [open consent](https://open-brain-consent.readthedocs.io/en/stable/) to allow broad sharing of data. - - 5. [Pre-register](https://www.cos.io/initiatives/prereg) your study. - - -2. **Data and metadata management** - - 1. Use standard data formats and extend them to meet your needs. - - 2. Use version control from start to finish. - - 3. Annotate data using standard, reproducible procedures. - - -3. **Software management** - - 1. Use released versions of open source software. - - 2. Use version control from start to finish. - - 3. Automate the installation of your code and its dependencies. - - 4. Automate the execution of your data analysis. - - 5. Annotate your code and workflows using standard, reproducible procedures. - - 6. Use containers where reasonable. - - -4. **Publishing everything** (publishing re-executable publications) - - 1. Share plans (pre-registration). - - 2. Share software. - - 3. Share data. - - 4. Make all research objects FAIR. - -## ReproNim's four core actions - -As indicated by the blue highlights in the figure below, four core actions are key to implementing the above principles. - -1. **Use of standards** - - Using standard data formats and extending them to meet specific research needs is important for data and metadata management (Principle 2) in reproducible neuroimaging. - -2. **Annotation and provenance** - - Annotating data using standard, reproducible procedures ensures clarity and transparency in data management (Principle 2). _Provenance_ refers to the origin and history of data and processes, enabling researchers to track how data was generated, modified, and analyzed (Principles 2, 3, and 4). This is essential for understanding the context of data and ensuring reproducibility. - -3. **Implementation of version control** - - Version control is crucial for both data and software management. It allows researchers to track changes over time, revert to previous versions if necessary, and collaborate effectively. - - For data, version control helps manage different versions of datasets and track modifications made during processing and analysis (Principle 2). - - For software, version control helps track code changes, manage different versions of analysis scripts, and ensure that the correct version of the code is used for each analysis (Principle 3). - - And even publications can be versioned (Principle 4). - -4. **Use of containers** - - Containers provide a portable and self-contained environment for running software, ensuring that the analysis can be executed consistently across different computing environments (Principle 3). Containers encapsulate all of the software dependencies needed to run an analysis, making it easier to share software (Principle 4) and reproduce results. - -![A diagram showing how ReproNim's principles and core actions work together to reproducible neuroimaging.](/images/principles-of-neuroimaging.jpg) diff --git a/content/about/principles/_index.md b/content/about/principles/_index.md index e4af612..2657817 100644 --- a/content/about/principles/_index.md +++ b/content/about/principles/_index.md @@ -1,7 +1,97 @@ --- -title: "Principles" -description: "Additional materials for each ReproNim principle" +title: "Reproducible Neuroimaging: Principles and Actions" weight: 1 +type: docs --- -Here we provide more information about each principle and resources that can help put the principle into practice + + +## ReproNim's four core principles + + + + + + +1. **[Study planning](/about/principles/planning/)** + + 1. Implement good science basics ([power analysis, statistical consults, etc.](https://www.repronim.org/module-stats/)). + + 2. Use pre-existing data for planning and/or analysis. + + 3. Create an [NIH-compliant data management and sharing plan](https://sharing.nih.gov/data-management-and-sharing-policy/planning-and-budgeting-for-data-management-and-sharing/writing-a-data-management-and-sharing-plan#after). + + 4. Adopt [open consent](https://open-brain-consent.readthedocs.io/en/stable/) to allow broad sharing of data. + + 5. [Pre-register](https://www.cos.io/initiatives/prereg) your study. + + +2. **Data and metadata management** + + 1. Use standard data formats and extend them to meet your needs. + + 2. Use version control from start to finish. + + 3. Annotate data using standard, reproducible procedures. + + +3. **Software management** + + 1. Use released versions of open source software. + + 2. Use version control from start to finish. + + 3. Automate the installation of your code and its dependencies. + + 4. Automate the execution of your data analysis. + + 5. Annotate your code and workflows using standard, reproducible procedures. + + 6. Use containers where reasonable. + + +4. **Publishing everything** (publishing re-executable publications) + + 1. Share plans (pre-registration). + + 2. Share software. + + 3. Share data. + + 4. Make all research objects FAIR. + +## ReproNim's four core actions + +As indicated by the blue highlights in the figure below, four core actions are key to implementing the above principles. + +1. **Use of standards** + + Using standard data formats and extending them to meet specific research needs is important for data and metadata management (Principle 2) in reproducible neuroimaging. + +2. **Annotation and provenance** + + Annotating data using standard, reproducible procedures ensures clarity and transparency in data management (Principle 2). _Provenance_ refers to the origin and history of data and processes, enabling researchers to track how data was generated, modified, and analyzed (Principles 2, 3, and 4). This is essential for understanding the context of data and ensuring reproducibility. + +3. **Implementation of version control** + + Version control is crucial for both data and software management. It allows researchers to track changes over time, revert to previous versions if necessary, and collaborate effectively. + + For data, version control helps manage different versions of datasets and track modifications made during processing and analysis (Principle 2). + + For software, version control helps track code changes, manage different versions of analysis scripts, and ensure that the correct version of the code is used for each analysis (Principle 3). + + And even publications can be versioned (Principle 4). + +4. **Use of containers** + + Containers provide a portable and self-contained environment for running software, ensuring that the analysis can be executed consistently across different computing environments (Principle 3). Containers encapsulate all of the software dependencies needed to run an analysis, making it easier to share software (Principle 4) and reproduce results. + +![A diagram showing how ReproNim's principles and core actions work together to reproducible neuroimaging.](/images/principles-of-neuroimaging.jpg) diff --git a/content/about/principles/longform.md b/content/about/principles/planning.md similarity index 98% rename from content/about/principles/longform.md rename to content/about/principles/planning.md index b1a0ef7..68089ee 100644 --- a/content/about/principles/longform.md +++ b/content/about/principles/planning.md @@ -1,12 +1,10 @@ --- -title: "Planning" +linkTitle: "Planning" +title: "ReproNim Principle 1: Study planning" type: docs weight: 80 --- - -# ReproNIM Principle 1: Study planning - Study planning represents a foundational step in ensuring reproducible neuroimaging research. Many problems that lead to reproducibility issues can be avoided by proper planning 1. **Implement good science basics, e.g, power analysis, statistical consults** From d061517a04f4fab03537aee8f7deba76ec6db87d Mon Sep 17 00:00:00 2001 From: Austin Macdonald Date: Tue, 3 Jun 2025 15:44:52 -0500 Subject: [PATCH 5/5] fixup formatting and relative links --- content/about/principles/planning.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/content/about/principles/planning.md b/content/about/principles/planning.md index 68089ee..a35d2a0 100644 --- a/content/about/principles/planning.md +++ b/content/about/principles/planning.md @@ -5,11 +5,11 @@ type: docs weight: 80 --- -Study planning represents a foundational step in ensuring reproducible neuroimaging research. Many problems that lead to reproducibility issues can be avoided by proper planning +Study planning represents a foundational step in ensuring reproducible neuroimaging research. Many problems that lead to reproducibility issues can be avoided by proper planning. 1. **Implement good science basics, e.g, power analysis, statistical consults** If your study is underpowered or your study design is flawed, you will have trouble publishing your work and, more to the point, your work will likely not be reproducible ([Scuzs and Ioannidis, 2020](https://www.sciencedirect.com/science/article/pii/S1053811920306509#bib24)). - * ***ReproNim resources*****:** + * ***ReproNim resources***: * [Repronim training module in statistics for neuroimaging](https://www.repronim.org/module-stats/): covers all aspects of statistics for neuroimaging, including sampling, power analysis and more * ***Other resources:*** @@ -22,10 +22,10 @@ Study planning represents a foundational step in ensuring reproducible neuroimag * ***ReproNim resources*****:** ReproNim has tools and approaches to make it easier to find and reuse your own data or publicly available data - * [ReproLake](https://repronim.netlify.app/resources/tools/reprolake/): a searchable data store for public neuroimaging data, built on the NIDM and BIDS standards + * [ReproLake](/resources/tools/reprolake/): a searchable data store for public neuroimaging data, built on the NIDM and BIDS standards * RepoPond: A local searchable metadata store for your neuroimaging data, built with the same standards as the ReproLake. - * [Neurobage](https://repronim.netlify.app/resources/tools/neurobagel/)l: A system for distributed data sharing and discovery built on common annotations across the workflow - * Tutorial: [Sharing and Searching Metadata Using a ReproPond and the ReproLake](https://repronim.netlify.app/resources/tutorials/pond-lake/) + * [Neurobagel](/resources/tools/neurobagel/): A system for distributed data sharing and discovery built on common annotations across the workflow + * Tutorial: [Sharing and Searching Metadata Using a ReproPond and the ReproLake](/resources/tutorials/pond-lake/) * ***Other resources:*** * OpenNeuro (and other repositories) @@ -34,8 +34,8 @@ Study planning represents a foundational step in ensuring reproducible neuroimag A key component of study planning is carefully considering data management needs from the start. By anticipating the volume and types of data that will be generated, researchers can allocate appropriate resources for storage, processing, and sharing. The NIH's requirement for a Data Management and Sharing Plan (DMSP) in all proposals helps formalize this planning process. The DMSP requires researchers to specify crucial details like which data standards will be implemented and where data will ultimately be shared. By addressing these requirements during the planning phase, researchers can integrate data sharing preparations into their workflow from the beginning, rather than treating it as an burdensome afterthought at the study's conclusion. * ***ReproNim resources:*** - * [How ReproNim can help you create an NIH-compliant DMSP](https://repronim.netlify.app/resources/tutorials/data-management-and-sharing/) - * [Planning ahead for multi-site data collection using ReproSchema](https://repronim.netlify.app/resources/tutorials/reproschema/) + * [How ReproNim can help you create an NIH-compliant DMSP](/resources/tutorials/data-management-and-sharing/) + * [Planning ahead for multi-site data collection using ReproSchema](/resources/tutorials/reproschema/) * ***Other resources:*** * [NIH DMSP website](https://sharing.nih.gov/data-management-and-sharing-policy/planning-and-budgeting-for-data-management-and-sharing/writing-a-data-management-and-sharing-plan#after)