Running the experiment

Run command

From the repository root:


python example/IBD/ibd_franzosa_multimodal.py

Configuration assembly and execution settings

The full ExperimentConfiguration, including the nested HyperoptConfig and ExperimentExecutionConfig, is assembled inside run_experiment():


def run_experiment():
    config = mll.ExperimentConfiguration(
        microbiome_file=MICROBIOME_FILE_PATH,
        metadata_file=METADATA_FILE_PATH,
        experiment_dir=EXPERIMENT_DIR,
        sample_id_column=SAMPLE_ID_COLUMN_NAME,
        taxonomic_configs=TAXONOMIC_RESOLUTIONS_CONFIGS,
        transform_configs=TRANSFORMS_CONFIGS,
        target_configs=[
            mll.TargetConfig(
                column=TARGET_COLUMN_NAME,
                task_type=TASK_TYPE,
            )
        ],
        primary_modality_models=MODEL_CONFIGS,
        multimodal_config=MULTIMODAL_CONFIG,
        nested_cv_config=NESTED_CV_CONFIG,
        hyperopt_config=mll.HyperoptConfig(enabled=False, ...),
        evaluation_thresholds=EVALUATION_THRESHOLDS,
        execution_config=mll.ExperimentExecutionConfig(
            n_jobs=1,
            use_threading=False,
            enable_early_termination=True,
            progress_backend="sqlite",
            progress_batch_size=10,
            consolidate_predictions=True,
            model_compression_level=3,
            consolidate_hyperparameters=True,
            deduplicate_nested_cv=False,
            incremental_results=True,
            stream_predictions=True,
            save_model_weights=True,
            save_predictions=True,
            save_processed_data=False,
            store_data_in_results=False,
            gc_between_configs=True,
            keep_original_data=False,
        ),
    )
    return run_evaluation(config)

For a parameter-by-parameter breakdown, see Experiment configuration.

Running the evaluation


def run_evaluation(config):
    evaluator = mll.Evaluator(config)
    evaluator.run_systematic_evaluation()
    summary = evaluator.save_results()
    return evaluator, summary

run_systematic_evaluation() runs every combination of (taxonomy, transform, model, fusion strategy). For each fusion strategy, the evaluator:

Early / CONCAT: builds a combined feature matrix from both modalities, then trains and evaluates each model on that matrix.
Late / MEAN: trains each model separately on the microbiome and metabolomics features, then averages the predicted probabilities.

Because run_siso=False, only the fused (multimodal) configurations are evaluated. Setting run_siso=True would additionally produce single-modality baselines for comparison.

Output structure


results/ibd_multimodal/
├── miso/                          # multimodal fused results
│   ├── early_concat/
│   │   └── target-Study.Group/
│   │       └── ...
│   └── late_mean/
│       └── target-Study.Group/
│           └── ...
└── evaluation_results.csv         # combined summary across all runs

evaluation_results.csv contains one row per evaluated configuration, making it straightforward to compare performance across fusion strategies.