Hello world — WEKA-MCP

Ask a language model to classify a hundred and fifty flowers and it will do something remarkable: it will produce an answer that looks exactly like the right one. Confusion matrix, accuracy figure, a tidy decision tree. It will be shaped like statistics without being statistics. Nothing was fitted. Nothing was cross-validated. The numbers came from the same place the prose did.

This is not a flaw you prompt your way out of. It is what the tool is. So instead of asking a model to be a machine-learning library, we gave it one.

Why WEKA

WEKA came out of the University of Waikato and has been in continuous use since 1993 — older than most of the ML ecosystem people reach for today. It is unfashionable in the way that load-bearing infrastructure usually is.

That age is the point. Every classifier in it has been run against real datasets by real researchers for three decades. When WEKA reports 96% accuracy on a J48 tree, that number carries the weight of a very long argument that has already been had. We wanted a system where the model does the explaining and something older and more boring does the computing.

WEKA also has a property that turns out to matter enormously for this: it is legible. A J48 decision tree is a tree you can read. A confusion matrix is a grid you can point at. When a language model relays those results, it is relaying an artifact that a human can independently check — which is exactly the property that hallucinated statistics lack.

The boring part is the whole product

The architecture has two services, and the interesting thing about it is how little the interesting one does.

weka-api is Java 17 wrapping WEKA 3.9.6. It does the training, the evaluation, the clustering, the diagnostics. weka-mcp is a TypeScript MCP server that contains no machine-learning logic at all. Every tool call becomes a REST call and nothing more.

That split was a deliberate constraint rather than a convenience. The moment the translation layer starts computing anything — rounding a metric, reshaping a result, helpfully summarising a matrix — it becomes a second place where numbers can be wrong, and the guarantee collapses. Because the forwarder forwards, the ML behaves exactly like WEKA does. It is WEKA. Anything you can verify in the WEKA GUI, you can verify here.

What "typed" is doing in "46 typed tools"

Each tool declares its inputs as a schema. A model calling weka_train cannot invent a classifier that doesn't exist or pass a fold count as a string — the call is rejected before it reaches the JVM. The schema is a contract with the model, and it is the reason the boundary holds.

Hello, iris

The traditional first run is Fisher's iris dataset — 150 flowers, four measurements each, three species. It ships as iris.arff, and ARFF is WEKA's native format: a header that declares each attribute's type, then rows.

@relation iris

@attribute sepallength real
@attribute sepalwidth  real
@attribute petallength real
@attribute petalwidth  real
@attribute class       {Iris-setosa, Iris-versicolor, Iris-virginica}

@data
5.1,3.5,1.4,0.2,Iris-setosa
4.9,3.0,1.4,0.2,Iris-setosa

Declaring types up front, decades before it was fashionable. Three tool calls take you from that file to a prediction: weka_upload_dataset, then weka_train with the J48 classifier, then weka_predict.

What comes back from the training call is the part worth dwelling on:

petalwidth <= 0.6
│  ├─ Iris-setosa (50.0)
│  └─ petalwidth > 1.7 → Iris-virginica (46.0/1.0)

One measurement — petal width — separates setosa from everything else, perfectly, in all fifty cases. That is not a fact the model asserted. It is a fact the tree discovered in the data, and you can walk down the branches and check it yourself. The (46.0/1.0) is telling you something honest too: forty-six instances reached that leaf and one of them was misclassified.

The model's job, at that point, is the thing models are genuinely good at: reading the tree aloud and telling you what it means.

What we got wrong first

Two things, both about boundaries rather than machine learning.

The first was transport state. The HTTP transport creates a fresh server per request, which means it must run stateless — no session id. We initially let the SDK issue one. Every initialize succeeded and every single call after it failed with "Server not initialized", because the session it referred to had already been discarded. A stateless transport that pretends to have sessions fails on the second request, never the first, which is the most annoying place for a bug to live.

The second was timeouts. Container Apps drops a request at roughly 240 seconds, so the client timeout sits deliberately below it at 210. Training runs that need longer than that simply cannot complete over ingress. This is a real limit, not a tuning problem — and it is the honest reason the hosted demo is for trying things rather than for serious jobs. Big training belongs on a local install, where nothing is standing between you and the JVM.

Try it

There is a hosted server running in open demo mode. No key, no account, no header — paste the endpoint into ChatGPT or Claude and all 46 tools appear. It is a shared public sandbox, so treat anything you upload as public, and expect the first call after an idle spell to take ten to thirty seconds while the JVM wakes up.

If you would rather your data stayed yours, bin/setup builds the whole stack locally in one command.

Use the hosted server Run it locally

More posts to come on the diagnostics tools — threshold curves, calibration, and the quietly useful business of finding out where a model is wrong rather than how often.