Sorry to break it to you, but the article doesn't describe that at all. In fact, the reason why a DB has such great performance isn't mentioned at all. Learn what a datapath architecture is and how a DB kernel works if you are interested in that topic. And then there is how an optimizer works, how the prepare time works, how metadata is handled, etc...

Postgres can do that as well.

Counterpoint, Meta is currently (and for the last decade) trying to rewrite MySQL so it is basically Postgres. They could just change their code so it works with Postgres and retrain their ops on Postgres. But for some reason they think its easier to just rewrite MySQL. Now, that is almost certainly more about office politics than technical matters...but it could also be the case that they have so much code that only works with MySQL that it is true (seriously doubtful).

You are just mislabling good architecture as 'premature optimization'. So I will give you another platitude... "There is nothing so permanent as a temporary software solution"

Wow, I'm sorry you have to work with such coworkers. For reference, joins are just an expensive use case. DBs do them about 10x faster that you can do them by hand. But if you need a join, you probably should either a) do it periodically and cache the result (making your data inconsistent) or b) just do it in a DB. Confusing caching the result with doing the join efficiently is an amazing misunderstanding of basic Computer Science.

microservices are about the fact that administrative overhead for a software system increases exponentially w.r.t. the complexity of the system. Or to put it another way, microservices are a way to make a complex system without having the architecture explode in size. They have nothing to do with making more efficient software systems. They are about making complex systems that trade dev costs for operational costs.

"All Unix filesystems will ensure the move operation is atomic."

This is false, but most fs will. However, there is a lot of fs calls you have to make that you probably don't know about to make the fs operations atomic.

PS The way you propose is probably the hardest way to do an atomic FS operation. It will have the highest probably of failure and have the longest period of operations and service disruption. There is good reason we move rows one at a time or in batches sized to match OS buffers.

Serious question, why are people here acting as if formatted files are somehow more reliable than a DB? That just simply isn't true. For most of software development's history, using flat files for persistence of data was the wrong thing to do with good reason. Flat files can easily be corrupted, and that happens much more often than a DB gets corrupted. The reason you might think otherwise is just sampling bias.

I do believe that you are missing a healthy dose of sarcasm. Such as faking downloads to give yourself inflated statistics so that your employer will trust untested and AI-written garbage.

That said, there really are good use cases for readable formatted files. For example configuration files that are checked into source control are far more trackable than a SQLite database for the same purpose. For another example, the files are convenient for a lot of data transfer purposes.

But for updateable data? You need a really good reason not to simply use a database. I've encountered such edge cases. But I've encountered a lot more people who thought that they had an edge case, than really did.

This wouldn't stop a lot of supply chain attacks. Attacks aren't identified immediately. Often they are only identified months later. And in that period, plenty of zero days are fixed. So this technique not only doesn't fix the problem, it introduces others. Also, again, this only happens to Python because of design flaws in the package managers themselves. Fix the package managers and this all goes away.

You don't follow politics in CA very closely if you think that. The way it works in CA is that the party makes sure that only 1 candidate runs in the Dem primary. Then they gerrymander the districts to make sure that they know which party will win in which district. The result of this is that the party insiders choose the politicians, not the voters.

PS Nobody in their right mind thinks the Dems support civil liberties. You just wish that was true and/or live in a bubble.

According to the Princeton Gerrymandering project, California's districts are better than average, with some bias. You can see a map of the entire U.S. on their front page.

https://gerrymander.princeton.edu/

Before the recent wave of gerrymandering started by Texas, California had an independent, non-partisan redistricting committee.

Could you provide a source for the claim that before 2025, there was significant gerrymandering in California?

As I said about civil liberties, there is a perception that Democrats are the lesser of two evils, given the realignment of the parties around segregation and civil rights in the 1960s. The Dixiecrats who were in favor of segregation left the Democratic party, while Republicans who favored racial integration joined the Democratic party. Then the Republican presidential campaigns of Goldwater, Nixon, and Regan shifted the party line to appeal more to the former Dixiecrats in the South. I'm agnostic about which party is better on civil liberties in 2025; I'd be interested in any research on the topic.

When you hit China, stop digging...you clearly have never lived in CA and know nothing of its politics.