ML engineering

Train on data you can actually share.

When production rows cannot leave their boundary, generate a statistically faithful training set, then prove what it keeps and what it does not reveal.

Product

Generate, then prove — in the product.

Watch the agentic dashboard synthesize rows and score them. This is the loop your team runs before a training job.

Profile this dataset and generate 10k synthetic rows.
Profiling 24 columns… detected 3 correlations. Generating with schema-aware model.
dataxid_generate · running
Correlation matrix24×24
0.94Fidelity
24/24Columns
10kRows
Why not

Mocks and safe subsets are not a training set.

Hand-built tables lack joint distributions. Stripped “safe” extracts hide the rare classes your model must learn. Synthetic rows without a fidelity report are hard to defend to reviewers.

  • Random or rule-based fixtures do not preserve the relationships features rely on
  • Dropping sensitive columns still leaves you short on volume and minority signal
  • A generation without typed scores is a claim, not evidence
Outcome

Train when the real table can't leave.

Consent, residency, or vendor rules block the rows your model needs. Encode locally, generate a shareable training set, and keep raw values off the wire.

  • Only abstract embeddings cross the API boundary — not source columns
  • Partner teams and sandboxes get rows they can use without a policy exception
The SDK encodes raw data on your machine, sends only 64-float embeddings to DataXID for training and generation, then decodes the returned embeddings back into synthetic rows locally.

Your environment

  • Raw data → encode
  • decode → synthetic rows

The wire

  • 64 floats per row
  • No values, no identifiers

DataXID

  • Train on embeddings
  • Generate embeddings
privacy.py
config = dataxid.ModelConfig(
    embedding_dim=64,
    model_size="large",
    privacy=dataxid.Privacy(
        enabled=True,
        noise=0.2,
    ),
)

model = dataxid.Model.create(
    data=df,
    config=config,
)
Outcome

Rebalance the classes the model never sees.

Safe subsets hide rare events. Steer the distribution after generation — upsample a minority class or close a parity gap before you fit.

  • Override categorical mixes instead of hoping the natural draw is enough
  • Correct statistical parity gaps across sensitive attributes when the job requires it

Rebalance a category

Override the natural distribution of any categorical column.

rebalance.py
distribution = dataxid.Distribution(
    column="gender",
    probabilities={"M": 0.5, "F": 0.5},
)

synthetic = model.generate(
    n_samples=1000,
    distribution=distribution,
)

Correct for bias

Reduce statistical parity gaps across sensitive attributes.

bias.py
bias = dataxid.Bias(
    target="income",
    sensitive=["gender", "race"],
)

synthetic = model.generate(
    n_samples=1000,
    bias=bias,
)
Outcome

Prove fidelity before you fit.

Synthetic rows without a report are hard to defend. Run SynthEval for typed fidelity and privacy scores — measure, don't assert — then start training.

  • Distribution and relationship fidelity with a per-column breakdown
  • Privacy checks you can read in code, not only in a slide
Evaluation Results
Example scores from SynthEval.scores (holdout-based privacy). Visual report shots above use a separate Census Income run.
Also via

Same engine, from Python.

When the dashboard is not the right surface, the Python SDK runs the same path — local encode, generate, evaluate.

pip install dataxid
scores.py
se = SynthEval(
    original=train,
    synthetic=synthetic,
    holdout=holdout,
)

scores = se.scores
scores.fidelity
scores.fidelity_detail.univariate
scores.fidelity_detail.bivariate
scores.privacy.dcr_share
scores.privacy.ims_training
scores.privacy.nndr_ratio

Start with your own dataset.

Generate a shareable training set on the free tier.