Instructions to use InstaDeepAI/MOJO with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use InstaDeepAI/MOJO with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="InstaDeepAI/MOJO", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("InstaDeepAI/MOJO", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
MOJO
MOJO (MultiOmics JOint representation learning) is a model that learns joint representations of bulk RNA-seq and DNA methylation through bimodal masked language modeling and is tailored for cancer-type classification and survival analysis on the TCGA dataset.
Developed by: InstaDeep
Model Sources
- Repository
- Paper: Bimodal masked language modeling for bulk RNA-seq and DNA methylation representation learning (ICML 2026)
How to use
Until its next release, the transformers library needs to be installed from source with the following command in order to use the models. PyTorch should also be installed.
pip install --upgrade git+https://github.com/huggingface/transformers.git
pip install torch
Other notes
We also provide the params for the MOJO jax model in jax_params.
A small snippet of code is provided below to run inference with the model using bulk RNA-seq and DNA methylation samples from the TCGA dataset.
import numpy as np
import pandas as pd
from transformers import AutoModel, AutoTokenizer
from huggingface_hub import hf_hub_download
tokenizer = AutoTokenizer.from_pretrained("InstaDeepAI/MOJO", trust_remote_code=True)
model = AutoModel.from_pretrained(
"InstaDeepAI/MOJO",
trust_remote_code=True,
)
n_examples = 4
omic_dict = {}
for omic in ["rnaseq", "methylation"]:
csv_path = hf_hub_download(
repo_id="InstaDeepAI/MOJO",
filename=f"data/tcga_{omic}_sample.csv",
repo_type="model",
)
omic_array = pd.read_csv(csv_path).drop(["identifier", "cohort"], axis=1).to_numpy()[:n_examples, :]
if omic == "rnaseq":
omic_array = np.log10(1 + omic_array)
assert omic_array.shape[1] == model.config.sequence_length
omic_dict[omic] = omic_array
omic_ids = {
omic: tokens["input_ids"]
for omic, tokens in tokenizer.batch_encode_plus(omic_dict, pad_to_fixed_length=True, return_tensors="pt").items()
}
omic_mean_embeddings = model(omic_ids)["after_transformer_embedding"].mean(axis=1) # embeddings can be used for downstream tasks.
License
MOJO is released by InstaDeep Ltd and CentraleSupélec under the
Apache License 2.0. The sample data in data/ are from TCGA (GDC) and remain
under their source's terms.
Citing our work
@InProceedings{pmlr-v306-gelard26a,
title = {Bimodal masked language modeling for bulk {RNA}-seq and {DNA} methylation representation learning},
author = {G\'{e}lard, Maxence and Benkirane, Hakim and Pierrot, Thomas and Richard, Guillaume and Courn\`{e}de, Paul-Henry},
booktitle = {Proceedings of the 43rd International Conference on Machine Learning},
pages = {34463--34495},
year = {2026},
editor = {Zhang, Tong and Dudik, Miroslav and Jaggi, Martin and Agarwal, Alekh and Li, Sharon and Schuurmans, Dale and Zhu, Jerry and Berkenkamp, Felix and Dong, Hanze and Bietti, Alberto},
volume = {306},
series = {Proceedings of Machine Learning Research},
month = {06--11 Jul},
publisher = {PMLR},
pdf = {https://raw.githubusercontent.com/mlresearch/v306/main/assets/gelard26a/gelard26a.pdf},
url = {https://proceedings.mlr.press/v306/gelard26a.html},
}
- Downloads last month
- 62