In clinical research, data is almost always the bottleneck. The real data exists, but it is locked behind IRB approvals, HIPAA compliance requirements, data use agreements, and legal review that can take months. Meanwhile, the ML team needs data to develop and validate models.
At Tecla, working on a clinical research project for Incyte, we addressed this by generating high-fidelity synthetic tabular data using diffusion models. This post describes the approach, the constraints we worked within, and how we evaluated whether the synthetic data was actually useful.
The naive framing is: "we need synthetic data because we can't share real data." But the constraint is subtler. Synthetic data that is too faithful to the real data can still leak patient information through membership inference attacks or attribute inference. A synthetic dataset where every row is a slightly perturbed real patient is not private.
This means the generation method must balance two competing objectives:
These objectives are in tension. The right tradeoff depends on the downstream use case and the sensitivity of the data.
The standard approach for synthetic tabular data is a GAN-based model like CTGAN or TVAE. These work well for simple datasets but struggle with clinical data, which typically has:
Diffusion models handle these properties better in practice. The iterative denoising process learns the full joint distribution rather than a min-max game between generator and discriminator, which makes training more stable and the learned distribution more faithful to complex conditional structure.
We used a score-based diffusion model adapted for mixed-type tabular data, with separate noise schedules for continuous and categorical columns.
One of the most important lessons was to treat missingness as a feature, not a problem to impute away before training. In clinical data, whether a lab test was ordered carries information about the patient's condition. A model trained on imputed data loses this signal.
We added a binary missingness indicator for each column with more than 5% missing values. The diffusion model was then trained to jointly generate both the value and the missingness indicator, preserving the not-missing-at-random structure in the synthetic data.
Evaluating synthetic data is harder than generating it. "It looks similar" is not a metric. We used three evaluation dimensions:
For each feature, we compared the marginal distribution of real vs. synthetic data using Kolmogorov-Smirnov tests for continuous features and chi-squared tests for categorical features. We also computed pairwise correlations on both datasets and measured the Frobenius norm of the difference between the correlation matrices.
We trained a downstream classifier on synthetic data only, then evaluated it on a held-out real test set. We compared its performance to the same classifier trained on real data. The gap between the two is the "fidelity loss" of the synthetic data.
For our use case, a TSTR AUC within 0.03 of the real-data AUC was the acceptance threshold. We also checked that the TSTR model made similar types of errors (similar confusion matrix structure) to the real-data model.
We ran a membership inference attack on the synthetic data using a shadow model approach. The attack accuracy should be close to 0.5 (random chance) if the synthetic data is private. We also computed the nearest-neighbor distance ratio (NNDR) between synthetic and real records: if synthetic records are consistently very close to real records in feature space, the synthetic data is effectively memorizing the training set.
Clinical researchers and compliance teams are not ML engineers. The evaluation metrics above meant nothing to them. What they cared about was:
We translated the evaluation into two slides. The first showed two survival curves (one from real data, one from synthetic) overlaid on the same plot. Nearly identical curves were the signal they trusted. The second showed that the nearest synthetic record to any real patient differed by more than a defined clinical threshold on key variables, which was the privacy assurance they needed.
Their feedback on this framing directly shaped which evaluation metrics we prioritized in subsequent iterations.
Synthetic clinical data with diffusion models is a practical solution to a real bottleneck in clinical ML. The hard parts are not the model training; they are correctly handling missingness, defining a rigorous evaluation framework, and translating statistical results into clinical language that stakeholders can act on.
If you are working on a project with similar constraints, I am happy to discuss the approach in more detail.