HN.zip

Show HN: StackScope – I crawled over 40k indie launches to see what they ship

Hey all, I built StackScope, a crawler/catalogue that looks at new product launches and shows what they were built with.It watches launches from Product Hunt, Show HN, and PeerPush, then crawls the public site behind each one. The goal is to show what people actually launched with: hosting, frameworks, analytics, DNS, security headers, legal pages, AI-builder signals, and other public clues.I started building it because most stack-detection sites look at the web as a whole. I was more interested in the current indie launch scene: what people are choosing right now, at the point they first put something in public.A few implementation details: it runs on .NET, uses Playwright for rendered pages, and has a first-party fingerprint catalogue rather than one copied from Wappalyzer/etc. robots.txt is honoured, and the bot identifies itself.Frustratingly, I am still waiting for verified bot status from Cloudflare and currently that knocks out about 10% of all sites.There is also a private readiness check: paste a URL, get the same style of report, fix things, and recrawl. No account or email needed.I'd be interested in feedback on the usefulness of this, the methodology, and any obvious false positives.Jonathan.

15 points by datafreak_ - 3 comments

3 Comments

addedlovely [3 hidden]5 mins ago
Nice one.

I've been doing the same bit wider scope, for the whole Crux list, pruned to apex domains, and looking for CMS signals - how's your throughput?

I'm not doing any headless browser stuff, or many requests, so hyper optimised for speed.

I do grab robots.txt - didn't really see much in llms.txt or humans.txt in the wild, does yours?

addedlovely [3 hidden]5 mins ago
Ohh Cloudflare verified bot status, interesting I'll check that out.

I'm seeing about 6.6% block rate, but that does climb over time.

idid [3 hidden]5 mins ago
One extension, beyond stack: market category/domain/application - or any combo that tells me what the product does.

Fab project otherwise!