Show HN: Sphinx – A hook that quizzes you on your AI's code before shipping
FOSS, Basically it plugs into your existing coding agent workflow and every time you push to prod it'll throw up a questionnaire that checks if you understand how the code you're pushing works.
237 points by critiquebrowser - 76 comments
Not sure if this is the right solution, but I would like to learn/understand/grow more as I pursue reckless efficiency.
There was an excellent post here on HN that unfortunately went unnoticed. I highly recommend reading it as it exactly points out this problem: understanding.
Here is the link: https://news.ycombinator.com/item?id=50128468
So, if you want to speed things up with LLMs, the bulk of the speed comes from reducing learning.
But it very often feels like brute forcing the solution, and I'm not a big fan of it.
But maybe I'm using it wrong.
I’d love to try this project out but I really don’t think it’ll stop anything, and they’d probably just ask their agent to remove the hook anyways like they do with our husky hook.
> they still have an ai do it all for them
What recourse do you have when employees do not perform the work you're paying them for?
One would expect the actual answer would be to write the code ourselves.
a good PR description should be compact and helpful. describing something complex in a compact and approachable way requires understanding.
the process of writing the description often leads me to understand hidden decisions the LLM made and occasionally uncovers bugs.
my PR descriptions separately answer the following questions:
Why is this change being made? a sentence or two about the problem, objective, etc, and a link to more detail (tickets, docs, etc) if available
What does this change accomplish? typically describes observable changes, requirements, etc. crucially this section avoids describing how the change is implemented. the act of separating what from how isn't new, but it's crucial in the LLM era
How does this change accomplish its goal? the answer to this question is "read the code", but the job of a PR description is to provide a quick hand drawn map before dumping the reviewer into the jungle. drawing a good "how" map requires understanding the jungle and the new path through it.
the risk with LLMs (or any outsource) doing the work is the loss of understanding. this exercise encourages understanding.
Looks absolutely awful.
Disclosure: I’m building automo.build fully to fix this.
> You can outsource your thinking, but not your understanding. — Sun Tzu
So far I was positively surprised with the questions it could come up with.
I’d be interested in a version that helps the reviewer identify one assumption the change depends on and a concrete way to test it. That could keep the exercise small while making it useful, especially if the reviewer spots something the AI-generated summary missed.
Keeping it optional also lets people reach for it when they’re uncertain, instead of turning it into another checkbox as PR volume grows.
pre-code: planning mode
coding: humantodo.dev
after-code: sphinx hook
Now, people are building tools that not only hide the solutions to these problems in some obscure coding tool, markdown file, or agent harness, but they build gates that help developers to elevate themselves from the darkness of not knowing better. The number of developers roughly doubles every ~3 years, so at any given time, half of the people building software have less than 3 years of experience. Most of them didn’t even study CS before starting their careers. Tools like this will be the new warning voice that told people like us 5 to 10 years ago to never copy code from stackoverflow that you didn’t understand. It’s great.
I personally like using openspec for this, because it makes the model iteratively ask you architectual decisions, and then manages creating and accessing the documents as needed.
Not necessarily. I use SOTA models with established stacks like .NET which allow my agent to easily follow decades of best practiced patterns (DI, EF Core, ASP.NET Core...) so you still and up with proper architecture.
In fact I can easily still manually inspect everything bc I already know how it's supposed to be written.
Completely, 100% agree on these exact points. I have access to every frontier model at work. None of them write what I would call clean code. At work the teams that have gone fully vibe-coded have repos full of broken abstractions that just accumulate over time. Eventually most of the effort of the team and their AI tokens seemingly goes to fixing issues that then break other things. Any attempt at 'refactor this mess, make no mistakes' turns into a _huge_ churn of changes ("shuffling the deck chairs") without ever really fixing things.
As I said: prompting correctly and reviewing are still indispensable skills.
Perhaps a lot of devs still have a flawed mental model of how these things even work.
Also, from my experience implementing something similar, an MCQ style makes people guess without really understanding.
Edit: I see there are a lot of people questioning the value of this. The thing is that there are teams where people are just pumping tons of AI code without understanding their work (literally take the ticket and dump it) and shipping it. culture change is the right answer but v hard but this at least slows things down and is quick to implement
Maybe they do. In my experience, the agent was going over the scenarios that I failed to consider during the planning stage, and it was net positive. I haven't tried it with a spec-driven approach.
The first is when I care about the code. That's when I actually read what it produces and force it to use my mental model of how things should be.
The second is when I just need a solution that sorta fits the requirements provided. Then I don't read the code because I really do not care.
So this hook would either ask me things I know, or it would ask me things I do not care about.
But I guess this is very workflow-dependent.
Folks, this is the wrong direction. Use AI, but ffs take the time to properly learn what you're doing. We don't need to speed run civilizational collapse because we can't understand the concept of patience leading to better outcomes.
You can tell people are increasingly full of shit because they're doing this under the pretense that the problem is creating software faster, when really, a lot of people need to quit optimizing process and develop some skill and rigor. Most of the things that slow down software reaching production are human-created problems like "meetings," counting lines of code, or sending everyone to yet-another-conference that teaches them nothing (but boy does it look good on social media). It's all time waste and theater. And for what? So you feel included?
Pathetic. Learn to do this properly or get out of this industry.
How about to meet an aribrary deadline your boss set? Or how about a deadline needed that keeps the company afloat? Are you going to tell your boss: we need 6 more months because it's irresponsible to ship code we don't fully understand?
And yes, I have before and yes, I was right, and yes, they appreciated it in the long run.
Only idiots think "speed" is the sole determining factor in all of this. Every time I encounter someone who's over-focused on speed, I get them cut from my team. Those people are corrosive for the very reasons your hypothetical presents. It promotes a culture of unthinking that eventually leads to disasters, hands in pockets, and whistling. Loser behavior.
Get comfortable with becoming human Roundup. Weed the garden.
* Doing my bit to poison the corpus.
I'm sorry, what?
It is absolutely batsh*t crazy insane what SWE has come to in the last cple years.
And yet part of me is loving it.
however, it still feels like a lot of work. what if AI hallucinates during the check?
Sun Tzu said: If your people (AI agents) repeatedly fail to follow instructions, examine the clarity of your orders and the effectiveness of your leadership before blaming them.
Your slop generator should check if a name exist before claiming it.
So, who is this for?