The cleaning steps live in someone's head
Lab datasets rarely arrive ready for modeling. Units differ between instruments, values are missing, a pressure column holds negative readings that are physically impossible, and two sensors report nearly the same signal in separate columns. Cleaning all of that eats days, and the sequence of fixes usually exists nowhere except in the memory of whoever did it last.
That has two costs. The work repeats every time new data lands, and nobody can verify what was done: which rows were dropped, which gaps were filled, which columns were rescaled before the model ever saw them.
The recipe becomes a pipeline you can rerun
Inside STOCHOS Flow, the Smart Preprocessor profiles the dataset first (column types, ranges, gaps, correlations), then combines presets with instructions you write the way you would brief a colleague:
> one-hot encode the categorical columns > flag negative pressure values > drop features correlated above 0.95 > interpolate the gaps > scale the rest
From those instructions it generates a preprocessing pipeline in sklearn and pandas, as code you inspect before it runs. The assistant only prepares the data; prediction stays with the DIM-GP models downstream. In the episode demo, a raw thermal-spray dataset went in and a documented, model-ready table came out, every transformation visible as a pipeline step.
Because the recipe is code, it reruns unchanged on the next batch. The cleaning knowledge stops being tribal and becomes part of the project file.
Common questions
01How do I know what it changed in my data?
The steps are generated as sklearn and pandas code, and you review that code before anything runs. Every transformation is a named pipeline step, not a hidden operation inside a model.
02Does the assistant need a cloud LLM?
No. STOCHOS Flow is local-first: AI assistance runs on a local LLM via Ollama, with cloud models optional where you want them. Your data stays on your infrastructure.
03What happens when the next batch arrives?
You rerun the same pipeline. The recipe is saved with the workflow, so new data goes through identical steps. If the data has changed, you edit the instructions and regenerate instead of starting over.
STOCHOS Flow
The workflow environment the Smart Preprocessor is part of, from data loading to deployment.
How STOCHOS works
The DIM-GP model that trains on your data once it is clean.
Keep up with a high-throughput lab
When automation moves the bottleneck from the bench to planning, a model proposes the next plate.
News and Guides
Every guide and insight we have published, in one place.
