How to evaluate any market signal before you trust it

Someone sends you a screenshot. An arrow, a score, a headline that says "unusual activity detected." Your first question should not be "is this true." It should be "what exactly is this claiming, and how would I know if it's wrong." That is the whole job. Here is the short version: figure out whether the signal is telling you about size or direction, check how much data it was built on, find out what would happen if you did nothing, see if it was tested on days it never saw before, and ask who gets paid if you believe it. Do that for any tool — a newsletter, an app alert, a friend's tip, or us — and most of the noise falls away on its own. We built this checklist because we use it on ourselves. If you want the long version of how we apply it to our own product, including the parts that did not hold up, that is laid out in full in we backtested our own signal — here is what failed. For now, let us walk through the framework itself.

#Ask what the signal is actually claiming

This sounds obvious but it is the step almost everyone skips. A signal can claim one of two very different things: direction (this will go up, this will go down) or magnitude (something bigger than usual is about to happen, in either direction). These are not the same claim, and mixing them up is where most money gets lost.

Take Palantir (PLTR) on a day it showed a Momentu score of 59 out of 100, trading volume — how many shares changed hands — running at 1.20 times its own normal 20-day pace, and 35 news articles in 48 hours. That tells you attention around the stock is elevated compared to its own recent past. It does not tell you whether that attention resolves upward or downward. The social reading that day was 67% bullish, which sounds encouraging, but bullish chatter and actual price direction are two different things measured by two different people for two different reasons. We go deeper on exactly this confusion in attention vs direction: the distinction that costs people money, because it is the single most common way a magnitude signal gets misread as a direction signal.

So the first question for any tool: does it say "this specific thing will happen" or "something is more likely to happen than usual"? Write down the exact sentence the tool is claiming. If you cannot write that sentence in one line, the tool has not earned your trust yet.

#Find out how much data it is standing on

A pattern spotted in ten examples is a story. A pattern checked against ten thousand examples is starting to be evidence. Sample size — how many separate cases the claim was tested on — is the difference between those two things, and it is almost never mentioned in the marketing.

Here is ours, stated plainly: our volume-based signal was tested across roughly 50 stocks over one year, which works out to about 10,500 individual observations. That is a real sample, large enough to say something, though still limited to one year and one set of stocks, which matters for the next question below. If a signal's promoters cannot tell you the sample size, or dodge the question with something like "thousands of successful trades," treat that as a red flag rather than reassurance. "Thousands" with no denominator tells you nothing about the hit rate.

#Check the base rate before you get excited

The base rate is simply: how often does this thing happen anyway, with no signal at all? If a coin comes up heads 52% of the time and someone sells you a "heads predictor" that is right 54% of the time, you have bought almost nothing. The improvement over doing nothing is the only number that matters, and it is usually smaller than the pitch implies.

Our own honest base rate: in that one-year test, unusual volume preceded price moves that were, on average, about 18% larger than normal for that asset. It did not beat the market on which way the move went. Repeat that back to yourself slowly — bigger moves, not correctly guessed direction. That is a real, useful, and modest finding. It is also the entire reason we do not tell anyone to buy or sell anything based on it. Size of the punch is not the same as knowing who throws it.

#Was it tested on data it had never seen?

This is the step that separates a genuine finding from a trick of hindsight. "In-sample" testing means checking a rule against the same data you used to build the rule — which is a bit like grading your own exam after writing the answer key from the exam itself. "Out-of-sample" testing means checking it against fresh data the rule never touched while being built. Only the second kind tells you anything about the future.

Ask any signal provider directly: was this backtested on the same period used to design it, or on a separate period held back specifically to check it? If they cannot answer, assume the worst case, because the worst case is depressingly common in this industry.

#Watch for overfitting — when a rule is fitted to noise, not signal

Overfitting is what happens when you tune a rule so precisely to past data that it starts memorizing coincidences instead of finding a real pattern. Financial markets are especially prone to producing convincing-looking coincidences, because with enough stocks and enough days, some strange correlation will always turn up by pure chance — copper prices moving with cloud cover in Ohio, say. A simple test: does the rule still make sense if you shift it slightly? If a signal only works with volume exactly 1.20 times normal, for exactly this list of 50 stocks, over exactly this 12-month window, and breaks the moment you nudge any of those numbers, that is a strong overfitting smell. A robust pattern should survive small changes to its own definition.

#Ask what would prove it wrong

A claim that cannot fail is not a claim, it is a slogan. "The market will eventually reward good companies" cannot be disproven on any useful timeline, so it is not testable, so it is not evidence of anything. A useful signal should be able to state its own failure condition in advance.

Ours is stated plainly: if unusual volume stopped preceding bigger-than-normal moves, or if it started reliably predicting direction instead of just size, that would change what we tell you. We have gone looking for that failure ourselves rather than waiting for a reader to find it, and the detailed results — including the specific stretches where the size effect weakened — are in we backtested our own signal — here is what failed. If a tool's creators have never published a version of that article about their own product, ask yourself why.

#Ask who profits if you believe it

This is the least technical question and often the most revealing. Does the signal provider make money from your subscription regardless of outcome, or from your trading activity, or from you buying a specific asset they also hold? A brokerage-run "hot stocks" list, funded by trading commissions, has a structural incentive for you to trade more, whether or not that trading serves you. A newsletter that gets paid in advertising from the companies it covers has an incentive you should know about before you read the next issue.

We are a subscription service. We make the same amount whether you read the daily radar and do nothing, or whether you trade five times that day. That does not make us right, but it does mean our incentive is to be useful and accurate over years, not to generate activity today. Judge any source, including us, by whether its business model rewards being correct or rewards being exciting.

#Compare against the asset's own history, not against everything else

One more check worth running on any signal: is it measuring an asset against some universal standard, or against that specific asset's own normal behavior? Chevron (CVX) trading at 0.81 times its own 20-day average volume is a quieter-than-usual day for Chevron specifically — it says nothing about whether that is high or low compared to, say, JD.com, which showed volume at 1.05 times its own norm on one article of news in 48 hours. A giant company like LLY, trading around $1,232 a share with volume at 1.10 times its own normal pace, should never get flagged simply because its dollar volume dwarfs a smaller stock's. We built our entire measurement approach around this idea, and explain the reasoning in full in why we measure every asset against itself — and why it matters. Any signal that ranks assets by raw size rather than by change-from-their-own-baseline is comparing apples to a fruit bowl.

#Put the framework to work on a real example

Try this on a live case. Copper recently showed a Momentu score of 60, with volume at 0.97 times its own normal pace — essentially ordinary — alongside 100% bullish social sentiment. Ask the six questions: is this a direction or magnitude claim (magnitude, and a mild one, since volume is near normal)? What is the sample size behind that scoring method (the same ~10,500-observation base described above)? What is the base rate (moves are, on average, only modestly larger than usual after unusual volume, and this reading is not even unusual)? Was it tested out of sample? What would disprove it? Who benefits if you believe it? Running through these takes about ninety seconds and tells you more than the headline score ever will. For a fuller walkthrough of what unusual volume can and cannot tell you across different situations, see unusual volume in stocks: the complete guide.

None of this is about distrust for its own sake. It is about knowing which claims can survive being questioned. If you want to see this framework applied fresh every morning to real numbers, our daily radar is free to try — no pressure, just the same honest math, applied out loud.

Momentu is an informational tool, not financial advice or a buy/sell recommendation, and it does not consider your personal circumstances. Any decision you make is your own.