John Carmack's arguments against building a custom XR OS at Meta
https://xcancel.com/ID_AA_Carmack/status/1961172409920491849
171 points by OlympicMarmoto - 158 commentshttps://xcancel.com/ID_AA_Carmack/status/1961172409920491849
171 points by OlympicMarmoto - 158 comments
I've only seen John Carmack's public interactions, but they've all been professional and kind.
It's depressing to imagine HR getting involved because someone's feelings had been hurt by an objective discussion from a person like John Carmack.
I'm having flashbacks to the times in my career when coworkers tried to weaponize HR to push their agenda. Every effort was eventually dismissed by HR, but there is a chilling effect on everyone when you realize that someone at the company is trying to put your job at stake because they didn't like something you said. The next time around, the people targeted are much more hesitant to speak up.
impact is facebook for “how useful is this to the company” and its an explicit axis of judgement.
There weird hagiographies need to go. Carmack is absolutely not known to be kind. I have no idea what happened here but the idea that's he's this kindly old grandpa who could never, ever be rude or unprofessional is really out there.
These days, you get a medium-level description and a Linux driver of questionable quality. Part of this is just laziness, but mostly this is a function of complexity. Modern hardware is just so complicated it would take a long time to completely document, and even longer to write a driver for.
The people who develop OSes are cut from a different cloth and are not under the usual economic pressures.
Not the parent, but of course they're wasting their time... That's the point of a hobby OS.
I'm working on a hobby OS, and I have no illusions that it's most likely fewer than 10 people will ever run it, and less than 100 will hear about it, but it lets me explore some interesting (to me) ideas, and forces me to learn a little more about random pieces of computing. If I ran on GCP, I'd want the reboot button to work. That sounds useful.
On the topic, I don't see why anyone would want to build a general purpose OS. There's enough already and even with the shrinking of hardware variety, there's a lot of stuff to support to make a general purpose OS work on enough hardware for people to consider using it. You can take Linux or a BSD and hack it up pretty good to explore a lot of OS ideas. Chances are you're going to borrow some of their drivers anyway, and then you'll end up with at least some similarity... may as well start there and save a lot of time. (My hobby OS has a custom kernel and custom drivers, but I only support a bare minimum of devices... (pc) console i/o, one real NIC, and virtio-net... that's all I need; I might add support for more NICs and more consoles later)
That's what's claimed. That's what people say, yet it's just an excuse. I've heard the same sort of excuse people have, after they write a massive codebase, then say "Oops, sorry, didn't get around to documenting it".
And no, hardware is not more difficult than software to document.
If the system is complex, there's more need to document, just as with a huge codebase. On their end, they have new employees to train up, and they have to manage testing. So any excuse that silicon vendors have to deal with such immense complexity? My violin plays for them.
It’s not first party documentation that’s the problem. The problem is that they don’t share that documentation, so in order to get documentation for an “unsupported” OS a 3rd party needs to reverse engineer it.
If this is true, it will be interesting if/how this changes due to AI. Even AI's staunchest detractors agree that AI is quite good at writing at least passable documentation, which is infinitely better than zero documentation.
(And I feel bad saying this since Meta obviously did waste eleventy billion on their ridiculous Second Life recreation project ...)
So please don't mock the spend. Big spends fail sometimes, and at least people were paid to do the work.
On the other hand, Meta's experiment is primarily CEO-driven. The outcome is predetermined, changing direction is not possible. Sure, clever engineers get to draw the rest of the owl, but that's not very useful when it turns out that everyone needs a horse instead.
They are spending a fortune, but rather than getting 900 crappy ideas to throw away and 100 great ones to pick from for continued development, they are developing 1 technological marvel nobody is interested in.
If they'd spent the money researching nuclear fusion or space flight or a new way to develop microprocessors, I would be cheering their efforts even if they had failed in the end.
What is an easy gate task to get into “reverse engineering some drivers for some OS”?
Second thought: I don’t even know how to write a driver or a kernel, so I better start from there.
Mind you, this XROS idea came after Oculus reorged into FB proper. It felt to me like there were FB teams (or individuals) that wanted get on the ARVR train. Carmack was absolutely right, and after the reorg his influence slowly waned for the worse.
That jives with my sense that META is a mediocre company
"I think that your team shouldn't even exist" doesn't mean "I want your team to no longer exist.".
When I was on nuclear submarines we'd call what you are advocating "keep us in the dark and feed us bullshit."
There's nothing wrong with well-founded and thoughtful criticism. On the other hand, it is very easy for this to turn into personal attacks or bullying - even if it wasn't intended to be.
If you're not careful you'll end up with juniors copying the style and phrasing of less-carefully-worded messages of their tech demigod, and you end up with a huge hostile workplace behaviour cesspit.
It's the same reason why Linus Torvalds took a break to reflect on his communication style: no matter how strongly you feel about a topic, you can't let your emotions end up harming the community.
So yes, I can totally see poorly-worded critiques leading to HR complaints. Having to think twice about the impact of the things you write is an essential part of being at a high level in a company, you simply can't afford to be careless anymore.
It's of course impossible to conclude that this is what happened in this specific case without further details, but it definitely wouldn't be the first time something like this happened with a tech legend.
https://en.wikipedia.org/wiki/Singularity_(operating_system)
https://www.microsoft.com/en-us/research/project/singularity...
https://www.zdnet.com/article/whatever-happened-to-microsoft...
This gives you best of both worlds - carefully designed system for the hardware with near optimal performance, and still with the ability to take advantage of the full linux kernel for management, monitoring, debugging, etc.
You can always mmap /dev/mem to get at physical memory.
The OS does process scheduling, program management, etc. Ok, you don’t want a VR headset to run certain things slowly or crash. But some Linux distributions are battle-tested and stable, and fast, so can’t you write ordinary programs that are fast and reliable (e.g. the camera movement and passthrough use RTLinux and have a failsafe that has been formally verified or extensively tested) and that’s enough?
Everyone wants to make an OS because that's super cool and technical and hard. I mean, that's just resume gold.
Using Linux is boring and easy. Yawwwwn. But nobody makes an OS from scratch, only crazy greybeard developers do that!
The problem is, you're not crazy greybeard developers working out of your basement for the advancement of humanity. No. Youre paid employees of a mega corporation. You have no principles, no vision. You're not Linus Trovalds.
I think it was insane to start a new OS effort written in C/C++. We have plenty of OSes written in C/C++! We know how that story ends. If you're going to spend the effort, at least try a new language that could enable a better security model.
Still a very interesting project, but that feels like a similar story, for limited use cases (a smart thermostat/speaker with specific hardware) it works, but for wider use cases with heterogeneus hardware and complex interfaces (actual screen, peripherals) it didn't work.
The only thing I can imagine that would be more invasive would require a brain implant.
https://en.wikipedia.org/wiki/HarmonyOS_NEXT https://www.usenix.org/conference/osdi24/presentation/chen-h...
It makes absolutely zero financial sense to create a new general purpose operating system.
That's billions of lines of code. With a B. And that's just the code - getting it to work with hardware?
Do YOU want to talk to 10,000 hardware vendors and get them on board? No! Nobody does! That's just money burning!
But, there are valid political reasons for creating a new general purpose OS.
MS is a state backed company. Very natural that China went the same path.
But ultimately it just makes sense to adapt existing kernels / OS (say, arch) and adapt it to your needs. It can be hair wrenchingly frustrating, and requires the company to be willing to upstream changes and it still takes years, but the alternative is decades, because what sounds good and well designed on paper just melts when it hits the real world, and linux has already gone through those decades of pain.
The driver ecosystem is the moat. Linux finally overcame it decades later
5 Millions alone for the AMD graphic driver.
* Those that are are only loaded when needed
It's not that bad
It's probably not that hard to write bare metal code for a modern CPU that runs and crashes. It's obviously insurmountably hard to compete with Android in features with scratch built bare metal code. An "OS" can be anything between the two. But it's very easy to imagine an "XR OS" project snowballing quickly into the latter, and Carmack's concerns would be spot on(as always is, and as proven). Is it then an inherent difficulty in "designing a new operating system", or is it technically something else?
None of the code they wrote couldn't have just been written as a kernel module in Linux. It would've also been so much easier due to all the documentation and general knowledge people have about Linux both within the company and outside the company.
I mean, I'd give a fair shake to an OS from the SQLite team [1].
1. https://sqlite.org/codeofethics.html
Actually, I don't know how you join the Ita now that you mention it.
But I have wondered why one of these companies with billions of dollars to burn hasn't tried to create something new as a strategic initiative. Yes, there wouldn't be any ROI for years, and yes, the first several products on the platform would probably be better off on something more traditional.
But the long term value could potentially be astronomical.
Just another case of quarterly-report-driven decision making, I suppose. Sigh.
See Google's Fuschia: https://en.wikipedia.org/wiki/Fuchsia_(operating_system)
> But the long term value could potentially be astronomical.
Such as what?
Historically? The internet, the concept of a graphical user interface, the mouse, the smartphone, the LCD display, the laser printer...
It's about clever people trying weird stuff, and occasionally ending up with a world-changing idea. Asking for examples of to-be-discovered innovations is, by definition, an impossibility.
If you're competing against nothing, then I see it: it opens up a wide variety of product possibilities. But linux exists. Why not spend 1/1000th the time to adapt linux?
That's not even counting the rather substantial risk that your new OS will never approach the capabilities of linux, and may very well never become generally usable at all.
Option A: spend years and millions on a project that may never be as good as existing solutions, diverting attention and resources from actual products, or...
Option B: work on products now, using an existing, high-quality, extensible, adaptable OS, with very friendly licensing terms, for which numerous experts exist, with a proven track record of maintenance, a working plan for sustainability, a large & healthy developer community exists, etc.
It's hard to imagine how it wouldn't be a complete waste of time.
Google has Fuchsia - is about 10 years in development. Recently was a target for layoffs
They have; Taligent comes to mind. You may not have heard of that -- or more likely, you have but just forgot about it -- but it's a good object lesson (no pun intended) in why a successful new OS is hard to just conjure into existence. There has to be a crying, desperate need for it, not just a vague sense that This Time We'll Get It Right.
You could probably cite OS/2 Warp as a less-obscure example of the same phenomenon.
So someone at Meta was so sensitive that being told their behemoth of a project was ill advised ended up getting reported to HR?
Roll call!
I’ve seen this firsthand. These giant tech companies try to just jump into a massive new project thinking that because they have built such an impressive website and have so much experience at scale they should just be able to handle building a new OS.
In my case it wasn’t even a new OS it was just building around an existing platform and even that was massively problematic.
The companies that build them from scratch have had it as one of their core competencies pretty much from the start.
I’m unsurprised meta had issues like this.
Yes.
For example any of the systems listed in Carmack’s post. Or perhaps Serenity OS, RedoxOS, etc.
The technical justification for Meta writing their own OS is that they'd get to make design decisions that suited them at a very deep level, not that they could do the work equivalent of contributing a few drivers to an existing choice.
Most opinions of this man exists in a vacuum space isolated from the real world software industry. Building an OS from scratch is one of those examples.
It’s never seems like there’s a significant reason behind them other than………”I made dat :P”
Building a hobby OS taught me how little is just "software". The CPU sets the rules. Page tables exist because the MMU says so. Syscalls are privilege flips. Task switches are register loads and TLB churn. Drivers are interrupt choreography. The OS to me is just policy wrapped around fixed machinery.
We also need to be clear what an OS is. Is it "darwin" or "macOS" - they have different scopes.
Things I'd want from an OS for an XR device.
1. Fast boot. I don't want to have to wait 2-3-4-5 minutes to reboot for those times I need to reboot.
I feel like Nintendo figured this out? It updates the OS in the background somehow and reboot is nearly instant.
2. Zero jank. I'm on XR, if the OS janks in any way people will get sick AND perceive the product as sucking. At least I do. iOS is smooth, Androind is jank AF.
Do any of the existing OSes provide this? Sure, maybe take an existing OS an modify it, assuming you can.
https://en.m.wikipedia.org/wiki/Nintendo_Switch_system_softw...
Android suffers from being Java at the core, with all the baggage that brings with it.
You can't do it without going through their fucking app, that asks for every permissions under the sun, including GPS positioning for some reason. After finally getting this app working and pairing it with my headset, I could finally realize the controller was just dead and their was nothing to do.
Another point I would add in support of that meme comment, is Google's recent rug-pull of Android not allowing sideloading apps from unsigned developers anymore starting this autumn, after over a decade of conquering the market with their "go with us, we're the open alternative to iOS" marketing.
The conclusion is to just never EVER trust big-tech/VC/PE companies, even when they do nice things, since they're 100% just playing the long game, getting buddy-buddy with you waiting till they smothered the competition with their warchest, and then the inevitable rug-pull comes once you're tied to their ecosystem and you have nowhere else to go.
Avoid these scumbags, go FOSS form the start, go TempleOS. /s but not really
I'm not familiar of any platform that calls for collective ownership of the means of production, land redistribution and other far left positions.
The world they live in is also a place where the majority group consists of those unlike themselves is always smarter and correcter than them. They are not used to the world where smart minorities are more correct than the mainstream. This makes them very confused and unresponsive when that situation manifests before them.
One could argue that such wrong-ist people deserve representations and joy of expressions as anybody else do. But it would be so hard to do so without turning the whole thing into an all-loser mud throwing festivals. Even LLMs might not be of substantial help.
Pretty sure that's false: I remember seeing the recent commit that made Grok misbehave.
> Here's an experiment for you: open X in a new private tab and count political posts, see how many of them are far right. Bias is evident.
Go to Bluesky and count far left posts. The result will be similar, because political bubbles form by themselves.
Not updated in two years.
> Go to Bluesky...
At least people are not fantasizing about "race wars" or discussing the "jewsish question" there. Two subjects I saw multiple times on X's front page.
"Far left posts" are like, what, people should have access to healthcare? People should pay more taxes?
This is madness. The safe space culture has really gone too far.
If a professional can't give critical feedback in a professional setting without being rude or belittling others, then they need to improve their communication skills.
I've had it happen to me too, but my response was to resign on the spot (I was already not satisfied with the company).
The "toxic behaviour" I had done? I reverted a commit on the master branch that didn't compile, and sent a slack to the Dev who had committed it saying "hi! There appears to have been a mistake in your latest commit, could you please check it out and fix it? I've reverted it in the meantime since I need to deploy this other feature"
The dev responded by force pushing the code that did not compile to master and contacted HR.
I decided there was greener grass on other pastures. I was right.
Being "reported to HR" doesn't mean "almost got fired". It likely meant a meeting where someone explained "hey, the way you communicated that caused some upset, let's discuss better ways to handle that situation next time." Very often in larger companies, complaints about things like "this bigwig from this other group jumped all over us" are automatically sent through HR because HR has staff whose job just is resolving conflicts between people and keeping things peaceful.
Everything is ASAP. They are super excited about everything. And nothing you do is wrong, it just could be improved or they like it but don't love it.
You don't know if something is important, basically.
Just like Louis CK said, "if you used 'amazing' on chicken nuggets, what are you going to say when your first child is born?". But in reverse.
Personally, I'd rather work with someone who would tell me my work is terrible if it is.
In Germany, you can't even legally say somebody did a bad job at your company in a recommendation letter. Companies created a whole subtext to workaround that, it's crazy.
Some things are just bad. You should be able to say it is. Not by saying it could be better. Not by using euphemism. It's just something that needs to go to the trash.
In fact, I don't trust people who can't receive this information, even if not packaged with tact (which you should attempt to, but life happens). If you can't handle people not being perfectly polite every time, I can't help but feel I won't be able to count on you when things get hard.
That must be the French in me talking.
Cool off.
If you're in the middle of trying to write a new operating system, then it's probably not helpful to have John Carmack standing over you repeatedly telling you that you shouldn't be doing it. In this case Carmack gets the last laugh. Then again, it is easy to get the last laugh by predicting that a project will fail, given that most projects do.
He was the CTO of Oculus. Surely it is appropriate for the CTO to give advice on any big technical decisions, if not outright have veto power.
if youre apple, it does make sense to do stuff from scratch. i think in a way, software guys wind up building their own prisons. an api is created to solve problem X given world Y, but world Y+1 has a different set of problems - problems that may no longer be adequately addressed given the api invented for X.
people talk about "rewrite everything in rust" - I say, why stop there? lets go down to the metal. make every byte, every instruction, every syscall a commodity. imagine if we could go all the way back to bare metal programming, simply by virtue of the LLM auto-coding the bootloader, scheduler, process manager, all in-situ.
the software world is full of circularities like that. we went from Mainframe -> local -> mainframe, why not baremetal -> hosted -> baremetal?
It's also very hard to do so.