We scored 1,581 crypto predictions from YouTube. The bullish ones were worse than random.

What was measured

The market backdrop — this matters for reading everything below

Extraction quality — the part that decides whether any of this is real

The first extraction pass over-collects. Reading its output showed why: it labelled “Ethereum gained 5%” as a bullish call. That is a fact about the past, not a prediction. So a second, independent pass re-read every extracted call against its source block and classified what kind of statement it actually was:

The second pass also caught 277 calls (11.7%) where pass one read the direction backwards, and 39 (1.6%) where the sentiment belonged to a different asset named nearby. Everything below uses only forecast + position + valuation statements with direction and asset confirmed: 1,581 of 2,368 (67%). The 787 descriptions are excluded — scoring them would have inflated every number on this page. Provenance: 76% of quotes match the source text character-for-character; the rest differ only by whitespace normalisation. 9 of 4,411 blocks (0.2%) never returned a parseable classification and are simply absent.

The two baselines

A hit rate means nothing on its own, so every result is scored against two controls:

• •Always-long — what you would have scored calling “bullish” on every one of the same (asset, date) pairs. Beating it means the direction mix added something.
• •Random-timing — for each asset, the unconditional probability that a randomly chosen day inside the feed’s own coverage window produced the called outcome over that horizon. Beating it means the calls landed on better-than-random moments. This is the one that tests for skill.

Sample discipline: a video that says “bullish BTC” eight times is one opinion, so the primary sample collapses to one view per (video, asset) — 1,581 calls become 673 views.

Headline scoreboard — 673 verified views

Read it this way: the feed beat buy-and-hold (+10.3pp at the final horizon) purely because a minority of views were bearish in a falling market. And it lost to a coin flip on the same assets (−24.4pp). Both are true at once. Applying a 2% deadband (ignore moves smaller than 2%, the same rule our live track record uses) changes no verdict.

Split by direction — and the correction that matters

Bullish calls were anti-predictive — not merely wrong because the market fell, but wrong relative to picking random dates on the same asset. 84% of all views were bullish. The bearish rows look like skill. They are not, and this is the single most important correction in the study:

Did the alt calls at least beat holding BTC?

Explicit price targets: distance decides everything

289 numeric targets were usable (plausible magnitude, direction-consistent). 76 were reached at some point — 26%. That headline hides everything:

Targets within 5% of spot always print — they are not forecasts, they are the current price with a round number on it. Every target asking for more than a 3x has so far failed, one hundred out of one hundred. Only 121 of 289 targets came with any stated horizon at all, so most can never be marked wrong — several are explicitly multi-decade:

By channel — the scoreboard nobody publishes

Only Perico is positive against baseline at both horizons — and it is the only channel whose calls are majority bearish. At n=20 that is suggestive, not proven. The bullish-share column explains most of the table: the more permanently bullish a channel was, the worse it scored.

The calls that actually worked — and the worst

Trader XO’s late-January 2025 short book (SUI +83%, SOL +69%, XRP +65%) is the best cluster in the dataset — it called the alt top within weeks. Worth stating plainly though: the same 27 January 2025 video was bullish BTC (−38%) and bullish ETH (−41%). The shorts were excellent and the longs were not, on the same day. Right on the alts they disliked, wrong on the majors they liked — that pattern is the whole dataset in miniature.

Is there tradeable intel in here? Four hypotheses, tested

Each tested against the same random-timing baseline, so “the market fell” is already priced out of every number below.

Caveats, stated plainly

1. “To today” is one endpoint, and a bad one. The measurement date sits near the low of a 50–75% drawdown. The 30d and 90d columns are the regime-robust reads; the to-today column mostly measures how long-biased a speaker was.
2. 84% of the sample is bullish. The bearish rows rest on 92 primary views.
3. Extraction is model-assisted. Two independent passes, every call carries a verbatim quote and a video URL, 76% of quotes match character-for-character. Auditable, not infallible.
4. Three assets are 80% of the sample (BTC 324, ETH 120, SOL 93). Per-asset rows below n≈20 are anecdote.
5. Survivorship in the source. The feed only summarises videos it ingested; any ingestion bias is inherited here. Coverage is visibly uneven (3 summaries in July 2025, 146 in March 2026).

What this audit changed in the product

The honest summary is that there is product intel here and very little trading intel. The feed cannot tell you where the market is going. It can tell you, with receipts, who was right — and nobody in this category publishes that. So:

• •Every prediction our own AI publishes is already scored on an append-only public track record — the same scoring rules this audit used. A per-channel scoreboard for the feed is next.
• •The summariser is being fixed to separate “what happened” from “what they expect” — 28% of extracted calls were past price action written in forecast-like language, and the share varies by channel (9% to 39%), which is a summarisation artefact.
• •New summaries will emit structured calls (asset, direction, horizon, target, quote) at generation time, so scoring becomes live and exact instead of a two-pass forensic exercise.
• •Channels will be labelled by measured reliability — not hidden, labelled. The traders-vs-narrators split is stable enough to show.
• •First-mention attention now feeds our intel engine as a de-rating input — an asset hitting peak narrative attention is a risk flag, never a buy trigger.

And three things the data says not to do: no sentiment trading signal (it echoes at p<0.001, leads at p=0.27), no inverting bullish calls into a short strategy (the anti-predictive bullish result and the bearish edge are the same one-cycle fact seen from two sides), and no quoting the bearish edge without the quarter table above.

Method, reproducibility, and the dataset

Pipeline: paginate the summary corpus to exhaustion → survey which of the CoinGecko top-1000 assets appear → deterministic candidate extraction over 4,411 paragraph blocks → LLM pass one extracts directional calls → prices attached from Binance 1h klines and Hyperliquid candles → independent LLM pass two re-reads every call against its source block (forecast vs description, direction check, asset check) → scoreboard. Every row carries the verbatim quote and video URL, so any single number on this page can be checked by hand.