Modeling waits until the formats agree
Every modeling project starts the same way. The data that should become one table exists as a pile of files that disagree about formats, column names, and units, and someone has to reconcile them before anything can be trained. The usual answer is a throwaway pandas script: written under time pressure, debugged against one dataset, deleted once the merge finally holds.
Hand-kept Excel files whose column names and units vary by author.
Time-stamped streams at their own sampling rates, with gaps where the rig was down.
One row per specimen, keyed by sample IDs that almost match the process data.
Vendor CSVs with their own delimiters, headers, and decimal conventions.
The settings behind each run, kept as key-value text.
Free-form notes that hold the reason a run looks strange.
A pandas script you read before it runs
The Smart Data Loader is an AI assistant node in STOCHOS Flow. You describe the merge in plain language: which files belong together, which ID joins them, which columns become the inputs and which one is the target. It answers with a pandas script, and nothing runs until you have read it. The code stays editable in the node, so a wrong join is a one-line fix, not another prompt.
The assistant stops at the table. Prediction belongs to the DIM-GP models downstream, which train on the merged result. Standard steps stay visual, and when one source is odd enough to need custom logic, you write Python for that step and leave the rest of the workflow alone.
Common questions
01What if the generated script gets the merge wrong?
Nothing runs until you have seen it. The output is ordinary pandas, so a wrong join key or a misread header is fixed by editing the code directly, the same way you would fix a script of your own.
02Does the assistant train the model too?
No. The assistant builds and configures the workflow, and the Smart Data Loader stops at the merged table. Prediction belongs to the DIM-GP models in the modeling nodes that train on that table.
03Does my data leave our infrastructure?
Not in a local setup. STOCHOS Flow is local-first: core features work without AI agents, and the assistant can run against a local LLM via Ollama. Cloud LLMs are optional, not required.
STOCHOS Flow
The visual workflow environment the Smart Data Loader is part of.
How STOCHOS works
The DIM-GP model that trains on the table the merge produces.
Build an ML workflow without an ML team
What the rest of the pipeline looks like once the data is in shape.
News and Guides
Every guide and insight we have published, in one place.
