LodeCraft insight · Model challenge
5 ways an exploration prospectivity model can fool you.
A practical field guide for challenging leakage, spatial optimism, invented negatives, out-of-domain confidence and scores that hide weak evidence.
Publication details & scope
Direct answer
A convincing map can be technically impressive and still point a team in the wrong direction.#
Before a prospectivity result influences field work, tenure, drilling or partnership decisions, ask whether it learned the intended geology, whether the test was genuinely independent and whether the score is supported by evidence in the area where it is being used.
01 · Leakage
Did information about the answer enter the model indirectly?#
Known occurrences can leak through derived layers, sampling decisions, distance features, preprocessing, threshold selection or repeated comparison against the same holdout. The model may appear to discover a geological relationship while reproducing a clue tied to the examples.
Challenge it: document when every dataset, transformation and label entered the workflow. Keep genuinely held-out occurrences closed until the model and output rules are frozen.
02 · Spatial optimism
Were the test areas far enough from the training areas?#
Neighbouring cells often share geology, survey artefacts and spatial context. Randomly separating nearby cells can let the model see almost the same ground during training and testing, producing performance that does not represent transfer into a genuinely new area.
Challenge it: use spatially separated validation suited to the decision area, and show how performance changes as geographic separation increases.
03 · Invented negatives
Was unexplored ground treated as though it were proven barren?#
Most exploration datasets contain confirmed examples and a large amount of unlabelled ground. Unlabelled does not mean negative. If unknown areas are treated as barren without qualification, the model learns a convenient fiction shaped by exploration history and access.
Challenge it: state how negatives or background samples were constructed, test alternative sampling strategies and retain uncertainty where the label is unknown.
04 · Out-of-domain confidence
Is the model scoring conditions it has not seen before?#
A numerical score can look precise even where survey coverage, lithology, regolith, scale or data relationships differ from the training evidence. The problem is not only error; it is confidence without relevant support.
Challenge it: publish coverage, novelty and support surfaces with the rank. Allow the workflow to abstain where evidence is too weak or unfamiliar.
05 · One score hiding a weak case
What disagreement and missing evidence disappeared inside the final composite?#
A combined rank can conceal whether evidence families agree, whether one layer dominates, whether the result is stable and which plausible explanations conflict with it. Agreement among models using overlapping inputs is not independent confirmation.
Challenge it: inspect component evidence, disagreement, sensitivity, data gaps and alternative explanations beside the final priority.
Seven questions to take into a model review#
Use this as your own preparation record. Selections last while this page is open; they do not validate project readiness.
The useful output is not certainty. It is a better next investigation.#
A responsible prospectivity study narrows attention while preserving the evidence, uncertainty and conditions under which the result could fail. That gives an accountable geological team something it can challenge rather than a heatmap it is expected to trust.
See how MindSync keeps holdouts, ordinal ranks and claim boundaries visible in the LodeCraft case studies, or explore the LodeCraft exploration analysis engagement for a defined project question.