Defense Twitter has a grifter problem. People post “$LMT to the moon 🚀🚀🚀” and rack up engagement whether they’re right or not. I wanted to know who’s actually good at this and who’s full of it.
Sentinel scrapes tweets about defense contractors, waits 24 hours, then checks what actually happened. Did the stock go up like they said? By how much? Was there real news behind the move, or were they just yelling into the void? Each claim gets labeled exaggerated, accurate, or understated based on whether the hype matched reality.
The tricky part is filtering out bots. Half of fintwit is automated accounts reposting headlines or copy-pasting from each other. Sentinel uses Claude to classify accounts before scoring them, so the grifter leaderboard only counts real people making real predictions.
For price data, I pull the stock price at tweet time (averaged over 30 minutes to smooth out noise) and then the first available candle 24 hours later. A Friday afternoon tweet gets compared against Monday’s open, not Friday’s close, because that’s the actual information gap a reader would have experienced. News articles from the same window get scanned for catalysts like contract awards or earnings, which helps distinguish lucky calls from informed ones.
The exaggeration score is a number from 0 to 1 based on whether they called the direction wrong, whether their language intensity exceeded the actual move, and whether there was any news to justify the hype. Those roll up into per-account grifter scores over time.
I built two labelers: a naive one that just matches keywords, and an improved version that handles negation and sarcasm and calibrates thresholds per-ticker based on historical volatility. Both feed into ML models (TF-IDF logistic regression and fine-tuned BERTweet) that can flag likely exaggerations before the 24-hour window even closes.