Open weights · Hugging Face

Models.

Technical documentation for biomedical world models and modality-specific encoders. Extract patient embeddings, fuse modalities, and build clinical prediction models for oncology and healthcare — in minutes.

Install the Standard Model End-to-end example

Browse on Hugging Face

Available models

Two model collections
available open source.

Family World Model

smb-series

World-model encoders trained on longitudinal oncology data. Extract patient embeddings that are predictive across nearly every clinical task — readmission, staging, response, survival.

5 Variants Backbone · Paradigm

JEPA-SFT or curriculum training on longitudinal EHR from Memorial Sloan Kettering — 20K+ patients, 300K+ patient-years, 7 cancer types.

Feature extraction JEPA

Browse world-model variants

Family Modality Encoders

smb-encoders

Modality-specific encoders for 3D CT, somatic genomics, and proteomics. Drop in alongside smb-v1 for multimodal fusion or use standalone.

Variants

smb-vision-v1 0.6B params · CT
GenVarFormer 993K params · WGS
KRONOS 7B params · Proteomics

CT Volumes Genomics Proteomics

Browse encoder variants

How to use

Three ways to deploy Standard Model encoders — from a linear probe to multimodal alignment with an LLM backbone.

01

Linear probe · Frozen embeddings

Predict any clinical outcome from a frozen encoder

Freeze smb-v1, extract patient embeddings via last-token pooling, and train a lightweight head — logistic regression, Ridge, or CoxPH — to predict outcomes like readmission, staging, treatment response, or survival.

Run the demo
02

Multimodal fusion · Projection layer

Fuse modalities into one patient representation

Combine longitudinal EHR embeddings from smb-v1 with modality-specific encoders for CT, somatic genomics, and proteomics. A learned projection layer maps each modality into a shared latent space — producing one patient representation that captures clinical, molecular, and radiological signal together.

Architecture details
03

Multimodal alignment · LLM backbone

Align embeddings with an LLM for VQA and report generation

Align patient embeddings from smb-v1 and smb-encoders with external encoders or an LLM backbone for VQA, report generation, and other generative clinical tasks.

Learn more