Skip to main content

ML Models

The ML Models page shows the machine-learning models that power predictions across your workspace - and lets you check their status, see when they last trained, and trigger training manually.

Navigate to Settings → ML Models from the sidebar.

The models​

Six workspace-level models are listed:

ModelWhat it does
Send-Time OptimizationPredicts the best hour and day to send messages for maximum engagement
Personalized PathsPredicts which experiment path will convert best for each user
Churn PredictionIdentifies users at risk of becoming inactive
Channel PreferencePredicts the best messaging channel for each user
Predicted CLVForecasts each customer's expected spend over the next 365 days, plus how likely they are to still be active (predicted customer lifetime value)

Each model trains per workspace, on your workspace's own event data - not on pooled data from other customers.

Reading a model card​

Each card shows:

  • Status - one of:

    StatusMeaning
    Not TrainedNo model exists yet for this workspace
    Training...A training job is currently running
    TrainedA model exists and is serving predictions
    FailedThe last training attempt failed; the card shows the error message

  • Version - the current model version, shown once a model exists.

  • Last Trained - relative time of the last successful training run, or Never.

  • Accuracy - the model's evaluation metric from its last training run, shown when available. For Personalized Paths it is measured on the most recent fifth of decisions, which the model did not train on, so it reflects how the model does on new decisions rather than on the ones it learned from. The card also shows the holdout AUC and how many paths were kept; a path whose holdout AUC is below the floor is not used, and the journey falls back to its plain split for it.

Statuses refresh automatically every 30 seconds while the page is open.

How the numbers are measured​

A model is never scored on the rows it learned from. Before training, part of the data is set aside; the model trains on the rest and is then asked to predict the part it has not seen. The figures on the card come from that held-back part only. After measuring, the model is refit on everything, so the model that serves predictions uses all your data.

  • Accuracy is the share of held-back people the model got right. 87% means 87 of every 100 held-back people were placed in the correct group (for example, will go quiet / will stay active).
  • AUC is a ranking score from 0.5 to 1. Take one person who did the thing and one who did not, at random: AUC is how often the model scores the first one higher. 0.5 is a coin flip, 1.0 is a perfect ordering. It is shown beside accuracy because accuracy alone can look good when almost everyone belongs to the same group.
  • Paths kept (Personalized Paths) is how many experiment paths cleared the floor of 0.55 AUC. A path below it is not used; the journey falls back to its plain split for that path.

Two things to keep in mind:

  • Churn Prediction predicts the future, not the present: its inputs are what a person had done up to a cutoff 30 days ago, and the answer is whether they went silent in the 30 days after it. Personalized Paths is measured on its most recent decisions, so the test is "would it have chosen well last time".
  • A small workspace gives a small held-back set, so the figure moves a few points between retrains. Treat it as a range, not a fixed grade.

Improved routing and Next Best Message​

Joryio learns from all your messages together, not one message at a time. That is what lets a brand-new message get a sensible prediction from its first send, and it is what the AI Decision node uses to route each person. You do not choose a model or train anything: it learns on its own, every night, from your matured results.

Improved routing in AI Decision nodes. A small comparison group (one person in five) keeps the old routing, so the improvement is measured rather than assumed. The card shows conversions per decision for the improved routing and for the comparison group, each with a likely range, and says when the improved routing is ahead. Until enough decisions have matured it says "too early to call".

Next Best Message, shadow mode. When a journey has a Next Best Message node, this card shows what the node would have sent against what the journey actually sent: how often they agreed on the channel and on the template, the most-picked templates, and purchases in the window when they agreed versus when they did not. It is a plumbing check, not a controlled result. Once a node is switched to send mode, a second table on the same card compares the arm that got the pick with the holdout, with a likely range each and the "no conversion loss" verdict. See Next Best Message nodes.

Held messages. On journey steps you allowed to wait, Joryio holds a low-value message when the person just received another one and sends it later or drops it; a comparison group always sends. The card shows purchases per person for both groups with a likely range and says whether holding cost anything. See Held by Joryio. The verdict also watches unsubscribes and complaints: it passes only when purchases held up and the held group did not unsubscribe or complain more than a small allowed margin over the comparison group.

If you ever want a journey or a workspace back on the old routing, ask Joryio: the switch is a support action, so nothing on your side can leave a journey half-configured.

Models activate as data accumulates​

Models cannot train on an empty workspace. Every model needs enough event history - and enough variety in it - before it produces anything:

  • If there is not enough data, training fails with a message like "Not enough data to train - continue using the platform to generate training data."
  • If the data lacks variety (for example, churn prediction needs both active and inactive users), training fails with an explanatory error.

This is expected for new workspaces. Keep tracking events and sending messages; models activate and improve as real traffic accumulates. A Failed or Not Trained status early on is not a defect.

Training cadence​

Once your workspace has event activity, models retrain automatically on a fixed schedule (all times UTC):

ModelSchedule
Personalized PathsDaily at 03:30
Predicted CLVDaily
Churn PredictionWeekly, Monday at 03:00
Send-Time OptimizationWeekly, Tuesday at 03:00
Channel PreferenceWeekly, Wednesday at 03:00

After each scheduled retrain, user-level scores are refreshed so segments built on them stay current.

Scheduled runs cover workspaces with event activity in the last 90 days; a dormant workspace is skipped until events flow again.

Training manually​

You don't have to wait for the schedule. Click Train Now on any card and confirm to start a training run immediately - useful after a burst of new data, or to retry after a failure.

  • Training runs in the background and may take a few minutes.
  • The button is disabled while that model is already training.
  • If training fails, the card shows the error so you can tell whether it's a data problem (wait for more traffic) or something to report.

Where the predictions are used​

  • Send-Time Optimization - powers send-time optimization when scheduling messages.
  • Personalized Paths - drives the journey Experiment node in personalized mode, picking the best path per user. Until this model is trained, the experiment falls back to a plain random split.
  • Churn Prediction, Engagement Scoring, Channel Preference - their per-user scores are available as segment filters (engagement score, churn risk, channel preference), so you can target audiences on them directly.
  • Predicted CLV - forecasts each customer's likely spend over the next year; exposed as e-commerce segment filters (Predicted CLV amount and Predicted CLV tier). See RFM Analysis.

Event-specific likelihood models (for example "likely to purchase in 30 days") are configured separately - see Predictive Events.

Troubleshooting​

  • Everything says Not Trained - normal for a new workspace. Models need real event traffic first; either wait for the scheduled runs or click Train Now once you have meaningful data.
  • Failed with a data-variety error - the model needs contrasting examples (e.g., both engaged and inactive users). This resolves on its own as your audience's behavior diversifies.
  • Failed to load model status - the page could not reach the ML service; the statuses shown may be stale. Refresh the page.

Notes​

  • Viewing this page requires settings access in your role.
  • Models are stored and served per workspace; switching workspaces shows that workspace's own model statuses.