fedRBE

HowTo Guide Documentation GitHub FeatureCloud App

Reproduce the fedRBE Preprint

This guide explains how to reproduce the analyses from the fedRBE preprint, including centralized limma correction, federated fedRBE correction, and downstream evaluation.

Table of Contents

Prerequisites and setup

  1. Docker: Installation Instructions
  2. Git LFS (required for large files used in this workflow)
  3. Either Conda or Mamba for environment.yml, or separate Python 3.8+ and R 4.0+ installations.
  4. System resources:
    • ≥ 16 GB RAM (The script uses very close to 16 GB, so be careful with other programs running!)
    • ≥ 20 GB free disk space

Setup steps

  1. Set up Git LFS locally:

    Install git lfs following the git lfs documentation.

    Initialize Git LFS:

    git lfs install

  2. Get the repository and large files:

    For a fresh clone:

    git clone https://github.com/Freddsle/fedRBE.git
cd fedRBE
git lfs pull

    For an existing clone:

    cd fedRBE
git pull
git lfs pull

  3. Set up Python and R dependencies:

    Choose one of the following setup paths.

    Option A: Conda or Mamba environment

    The Conda environment installs Python 3.11, R, the CRAN and Bioconductor packages used by the notebooks, and the FeatureCloud Python package:

    conda env create -f environment.yml
conda activate fedRBE

    Mamba can use the same environment.yml. The file pins conda-forge::openblas=0.3.3 to avoid pthread_create() failures in affy::rma() on conda environments.

    If the older OpenBLAS build cannot be resolved for your platform or channel setup, use the preprocessCore source-install command from Option B after activating this conda environment.

    Install the GitHub-only R dependency:

    Rscript -e 'remotes::install_github("mwgrassgreen/RobNorm")'

    Option B: Separate Python and R installations

    Set up Python dependencies:

    python3 -m venv fedrbe_env
source fedrbe_env/bin/activate  # on Windows: fedrbe_env\Scripts\activate
pip install -r requirements.txt

    requirements_r.txt lists CRAN, Bioconductor, and GitHub dependencies in separate sections. Install them with:

    Rscript -e 'install.packages(c("data.table","ggpubr","gridExtra","ggsci","ggtext","glue","IRkernel","invgamma","jsonlite","knitr","nipals","patchwork","pheatmap","remotes","reshape2","scales","tidyverse","umap","viridis"), repos="https://cloud.r-project.org")'
Rscript -e 'if (!requireNamespace("BiocManager", quietly=TRUE)) install.packages("BiocManager", repos="https://cloud.r-project.org"); BiocManager::install(c("AnnotationDbi","affy","GEOquery","GO.db","hgu133acdf","hgu133plus2cdf","impute","limma","variancePartition"), ask=FALSE, update=FALSE)'
Rscript -e 'BiocManager::install("preprocessCore", configure.args=c(preprocessCore="--disable-threading"), type="source", force=TRUE, ask=FALSE, update=FALSE)'
Rscript -e 'install.packages("WGCNA", repos="https://cloud.r-project.org")'
Rscript -e 'if (!requireNamespace("remotes", quietly=TRUE)) install.packages("remotes", repos="https://cloud.r-project.org"); remotes::install_github("mwgrassgreen/RobNorm")'

Running the analysis

This section guides you through running both federated and centralized batch effect corrections and comparing their results. Any step can be skipped because the corresponding outputs are already committed in the repository.

Recommended order:

  1. Set up dependencies and fetch LFS data, see Prerequisites and setup.
  2. Prepare or reuse the per-dataset before/ inputs and central limma outputs, see 1. Preparing inputs and obtaining centrally corrected data.
  3. Run the federated correction script from the prepared before/ inputs, or use the fedRBE outputs already committed in evaluation_data/*/after/, see 2. Obtaining federated corrected data.
  4. Compare federated vs central corrections, then run the figure/table, classification, and clustering analyses.

1. Preparing inputs and obtaining centrally corrected data

Prepare the per-dataset before/ inputs and perform centralized batch effect correction using limma’s removeBatchEffect for comparison.

Run the dataset preparation and central-correction notebooks from their own directories, or set the notebook working directory to the listed folder so relative paths resolve correctly.

Dataset Run order
Simulated evaluation_data/simulated/01_data_prep_and_central_RBE.ipynb
E. coli evaluation_data/ecoli/01_data_prep_and_central_RBE.ipynb
Ovarian cancer evaluation_data/ovarian_cancer/00_harmonize_meta_load_data.ipynb (run time ~30 minutes), then evaluation_data/ovarian_cancer/02_central_RBE.ipynb
ccRCC proteomics python evaluation_data/ccRCC_studies/prepare_ccRCC_data.py, then evaluation_data/ccRCC_studies/01_central_RBE.ipynb
Quartet multiomics evaluation_data/quartet_multiomics/01_preprocess_eda.ipynb, evaluation_data/quartet_multiomics/02_prepare_RBE_inputs.ipynb, then evaluation_data/quartet_multiomics/03_central_RBE.ipynb

For simulated datasets, if all 30 runs were not re-generated, the evaluation_data/simulated/01_data_prep_and_central_RBE.ipynb notebook will only run the single-run central correction for the committed inputs. If you want to run the full 30-run evaluation, first generate the before/intermediate/ and after/runs/ files via evaluation_data/simulated/00_data_simulation.ipynb, which will take 1-2 hours, then re-run the notebook.

Output:

Note: The preprocessing and centralized correction notebooks have already been run for the committed outputs. You can skip this step when using the provided inputs and corrected data.

For simulated data, committed run-1 inputs support a quick check. The full 30-run evaluation requires generated before/intermediate/ and after/runs/ files via evaluation_data/simulated/00_data_simulation.ipynb.

2. Obtaining federated corrected data

Use the provided utility script to perform the configured federated batch effect correction experiments from the prepared evaluation_data/[dataset]/before/ inputs. Edit the experiment list in the script to enable or disable any of the datasets; currently all datasets are enabled by default.

python3 ./generate_fedrbe_corrected_datasets.py

Steps Performed by the Script:

  1. Sets up multiple clients: Simulates clients based on the datasets in evaluation_data/[dataset]/before/.
  2. Runs fedRBE on each client: Applies federated batch effect correction using FeatureCloud testing environment.
  3. Aggregates results: Combines corrected data and writes outputs to evaluation_data/[dataset]/after/ for each dataset.

Output:

Notes:

Customization:

For simulated data, the script checks that before/lab*/ inputs still match run 1 before writing the single-run FedApp_corrected_data*.tsv files used by analyse_fedvscentral.py.

3. Comparing federated and central corrections

Use the provided script to analyze and compare the results of federated and centralized batch effect corrections.

python3 ./analyse_fedvscentral.py

What this does:

Output:

4. Produce tables and figures

To reproduce the tables and figures from the preprint, run the provided Jupyter notebooks in the evaluation/ directory:

  1. evaluation/evaluation_simulated.ipynb
  2. evaluation/evaluation_simulated_30runs.ipynb (may take several hours to run if all 30 runs are generated)
  3. evaluation/evaluation_ecoli.ipynb
  4. evaluation/evaluation_ovarian_cancer.ipynb
  5. evaluation/evaluation_ccRCC.ipynb
  6. evaluation/evaluation_quartet_multiomics.ipynb

These notebooks expect the corrected data from Steps 1 and 2. evaluation_simulated_30runs.ipynb can use the committed precomputed 30-run LMPV summary cache for the final variance-partitioning plots; full recalculation still requires the generated simulated before/intermediate/ and after/runs/ files. Figures are written under evaluation/plots/ and related evaluation output folders.

5. Reproduce the classification analysis comparing fedRBE-corrected to uncorrected data

First run the classification experiments. This can take several hours; the repository already contains the generated results.

The classification experiments are split into the two different experiment types:

  1. train_test_split: Each client reserves 20% of the data as test data, trains on the other 80%, and reports test-set metrics. This takes up to an hour. 
    python3 evaluation_classification_after_correction/run_classification_train_test_split.py
  2. leave_one_cohort_out: The model is trained on all clients except one, then evaluated on the held-out client. This trains n_clients models and can take several hours. 
    python3 evaluation_classification_after_correction/run_classification_leave_one_cohort_out.py

The experiment results are saved in evaluation_classification_after_correction/results.

To visualize the experiments with plots, run the corresponding analysis script:

python3 evaluation_classification_after_correction/analyse_classification_metric_report.py

The resulting plots can be found in evaluation_classification_after_correction/plots. You can also use the helper shell scripts:

Note: If you want to inspect the RandomForest logic, this is located in evaluation_classification_after_correction/fc_fed_forest_simple_app.

6. Reproduce the clustering analysis comparing fedRBE-corrected to uncorrected data

To reproduce the clustering results from the preprint, run the real-dataset clustering notebooks under evaluation_clusterization_after_correction/.

Prerequisites

Reproducing the clustering analysis

Run the notebooks in evaluation_clusterization_after_correction/real_datasets/:

  1. 00_build_kmeans_matrices.ipynb — required only for Quartet multiomics.
  2. 01_data_preparation.ipynb
  3. 02_central_kmeans.ipynb
  4. 03_federated_runs.ipynb — optional; required only when regenerating federated k-means outputs.
  5. 04_analysis_metrics_plots.ipynb
  6. 05_multiple_runs.ipynb — optional repeated seeded federated runs.

See evaluation_clusterization_after_correction/real_datasets/README.md for detailed options and output paths.

Note: If you want to inspect the FedKMeans logic, this is located in evaluation_clusterization_after_correction/federated_kmeans_upd.

Repository structure

Understanding the repository layout helps in navigating the files and scripts.

fedRBE/
├── README.md                                   # General repository overview
├── batchcorrection/                            # fedRBE FeatureCloud app
├── evaluation_data/                            # Data used for evaluation
│   ├── ccRCC_studies/                          # ccRCC proteomics datasets
│   ├── ecoli/                                  # E. coli dataset
│   ├── ovarian_cancer/                         # Ovarian cancer datasets
│   │   ├── before/                             # Uncorrected data with structure needed to run the app
│   │   ├── after/                              # Corrected data
│   │   ├── 00_harmonize_meta_load_data.ipynb   # Data harmonization notebook
│   │   ├── 01_check_datasets_intersection.ipynb
│   │   └── 02_central_RBE.ipynb                # Centralized removeBatchEffect run
│   ├── quartet_multiomics/                     # Quartet multiomics datasets
│   └── simulated/                              # Simulated datasets
├── analyse_fedvscentral.py                     # Compares federated and centralized batch effect corrections.
├── generate_fedrbe_corrected_datasets.py       # Runs fedRBE on all configured datasets and saves results
├── run_sample_experiment.py                    # Runs fedRBE on the sample dataset
├── evaluation_utils/                           # Utility scripts for evaluations
│       ├── evaluation_funcs.R
│       ├── featurecloud_api_extension.py
│       ├── fedRBE_simulation_scrip_simdata.py
│       ├── filtering.R
│       ├── plots_eda.R
│       ├── simulation_func.R
│       ├── upset_plot.py
│       └── utils_analyse.py
├── evaluation/                                 # Main evaluation scripts to produce results and figures
│   ├── eval_simulation/                        # Evaluations on simulated data
│   ├── evaluation_ccRCC.ipynb
│   ├── evaluation_ecoli.ipynb
│   ├── evaluation_ovarian_cancer.ipynb         # Evaluation of ovarian_cancer datasets
│   ├── evaluation_quartet_multiomics.ipynb
│   ├── evaluation_simulated.ipynb
│   ├── evaluation_simulated_30runs.ipynb
├── evaluation_classification_after_correction/ # Classification comparison scripts and outputs
├── evaluation_clusterization_after_correction/ # K-means comparison notebooks and FeatureCloud app
└── [other directories/files]

Utility scripts overview

Main scripts live at the repository root; shared helpers live in evaluation_utils/.

File Purpose
generate_fedrbe_corrected_datasets.py Runs configured fedRBE experiments, assigns client inputs, executes the app, and stores corrected outputs.
analyse_fedvscentral.py Loads federated and central corrected datasets, checks shape/index consistency, and writes mean/max difference metrics.
evaluation_utils/featurecloud_api_extension.py Wraps the FeatureCloud testbed with automatic restart and result-extraction helpers used by the fedRBE correction script.
evaluation_utils/filtering.R Provides filtering and preprocessing functions used before centralized limma correction.
evaluation_utils/plots_eda.R Provides boxplot, PCA, UMAP, heatmap, and correction-diagnostic plotting helpers.
evaluation_utils/upset_plot.py Creates UpSet plots for feature-overlap comparisons.

Troubleshooting

Common checks:

For unresolved issues, open a GitHub issue.

Additional resources

Contact information

For questions, issues, or support, open a GitHub issue.

This site is open source. Improve this page.