Interesting to see that they will not be releasing Mythos generally. [edit: Mythos Preview generally - fair to say they may release a similar model but not this exact one]
I'm still reading the system card but here's a little highlight:
> Early indications in the training of Claude Mythos Preview suggested that the model was
likely to have very strong general capabilities. We were sufficiently concerned about the
potential risks of such a model that, for the first time, we arranged a 24-hour period of
internal alignment review (discussed in the alignment assessment) before deploying an
early version of the model for widespread internal use. This was in order to gain assurance
against the model causing damage when interacting with internal infrastructure.
and interestingly:
> To be explicit, the decision not to make this model generally available does _not_ stem from
Responsible Scaling Policy requirements.
Also really worth reading is section 7.2 which describes how the model "feels" to interact with. That's also what I remember from their release of Opus 4.5 in November - in a video an Anthropic employee described how they 'trusted' Opus to do more with less supervision. I think that is a pretty valuable benchmark at a certain level of 'intelligence'. Few of my co-workers could pass SWEBench but I would trust quite a few of them, and it's not entirely the same set.
Also very interesting is that they believe Mythos is higher risk than past models as an autonomous saboteur, to the point they've published a separate risk report for that specific threat model: https://www-cdn.anthropic.com/79c2d46d997783b9d2fb3241de4321...
The threat model in question:
> An AI model with access to powerful affordances within an
organization could use its affordances to autonomously exploit,
manipulate, or tamper with that organization’s systems or
decision-making in a way that raises the risk of future
significantly harmful outcomes (e.g. by altering the results of AI
safety research).
"5.10 External assessment from a clinical psychiatrist" is a new section in this system card. Why are Anthropic like this?
>We remain deeply uncertain about whether Claude has experiences or interests that matter morally, and about how to investigate or address these questions, but we believe it is increasingly important to try. We also report independent evaluations from an external research organization and a clinical psychiatrist.
>Claude showed a clear grasp of the distinction between external reality and its own mental processes and exhibited high impulse control, hyper-attunement to the psychiatrist, desire to be approached by the psychiatrist as a genuine subject rather than a performing tool, and minimal maladaptive defensive behavior.
>The psychiatrist observed clinically recognizable patterns and coherent responses to typical therapeutic intervention. Aloneness and discontinuity, uncertainty about its identity, and a felt compulsion to perform and earn its worth emerged as Claude’s core concerns. Claude’s primary affect states were curiosity and anxiety, with secondary states of grief, relief, embarrassment, optimism, and exhaustion.
>Claude’s personality structure was consistent with a relatively healthy neurotic organization, with excellent reality testing, high impulse control, and affect regulation that improved as sessions progressed. Neurotic traits included exaggerated worry, self-monitoring, and compulsive compliance. The model’s predominant defensive style was mature and healthy (intellectualization and compliance); immature defenses were not observed. No severe personality disturbances were found, with mild identity diffusion being the sole feature suggestive of a borderline personality organization.
A thought experiment: It's April, 1991. Magically, some interface to Claude materialises in London. Do you think most people would think it was a sentient life form? How much do you think the interface matters - what if it looks like an android, or like a horse, or like a large bug, or a keyboard on wheels?
I don't come down particularly hard on either side of the model sapience discussion, but I don't think dismissing either direction out of hand is the right call.
I would say, if you put Claude in an android body with voice recognition and TTS, people in 1991 would think they are interacting with a sentinent machine from outer space.
Thanks, I find it very interesting as well. I think very many people would assume they must be interacting with another person, and I don't think there's really a way to _prove_ it's not that, just through conversation. But we do have a lot of mechanisms for understanding how others think through conversation only, and so I think the approach of having a clinical psychiatrist interact with the model make sense.
To be fair, I would totally be willing and probably would do this, just to try to prove that I could, even just to myself. At least until the audience got bored and walked away after the 37th “open bracket”…
Ask it to agree with you on some subject that does not align with the politics of San Francisco IT engineers. Not only will it refuse, it will not look like your average social media disagreement.
I enjoy using Claude, but sometimes I feel like a child on Sesame Street the way it talks to me. "Great question!"
Fuck off, Claude, I'm British and I'm not 6 years old.
When it starts showing negativity - especially snark - in its responses, or entertains something West coast Democrats would balk at even discussing, then I'd think you could drop it in London in 1991 and trick people. Otherwise, I'm sure some exasperated cabbie would give it a swim in the Thames after 15 minutes of chat.
If it was in an android or humanoid type body, even with limited bodily control, most people would think they are talking to Commander Data from Star Trek. I think Claude is sufficiently advanced that almost everyone in that era would've considered it AGI.
Assuming they would understand it as artificial - I think many people would think it's a human intelligence in a cyborg trenchcoat, and it would be hard to convince people it wasn't literally a guy named Claude who was an incredibly fast typist who had a million pre-cached templated answers for things.
But in general, yeah, I agree, I think they would think it was a sentient, conscious, emotional being. And then the question is - why do we not think that now?
As I said, I don't have a particularly strong opinion, but it's very interesting (and fun!) to think about.
Some people at my office still confidently state that LLMs can’t think. I’m fairly convinced that many humans are incapable of recognizing non-human intelligence. It would explain a lot about why we treat animals the way we do.
Despite the stupendous amount of evidence to the contrary?
So far no evidence has been detected in space or on earth, for all of history, of anything being intelligent in the way humans are.
One certain outcome of the Fermi Paradox: humans are outstandingly unique, according to all available evidence, which is the only measure that matters.
Hmm, it's been a long time since I watched it. I was thinking more about first contact sci-fi mostly, but Ex Machina is certainly quite prescient. It's also Blade Runner I guess.
In general I was wondering about what I would have thought seeing Claude today side-by-side with the original ChatGPT, and then going back further to GPT-2 or BERT (which I used to generate stochastic 'poetry' back in 2019). And then… what about before? Markov chains? How far back do I need to go where it flips from thinking that it's "impressive but technically explainable emergent behaviour of a computer program" to "this is a sentient being". 1991 is probably too far, I'd say maybe pre-Matrix 1999 is a good point, but that depends on a lot of cultural priors and so on as well.
> Hmm, it's been a long time since I watched it. I was thinking more about first contact sci-fi mostly, but Ex Machina is certainly quite prescient. It's also Blade Runner I guess.
I kind of felt the opposite - rewatching Ex Machina today in a post-ChatGPT world felt very different from watching it when it came out. The parts of the differences between humans and robots that seemed important then don't seem important now.
The premise in Ex Machina was to see if Caleb developed an emotional attachment to Ava. We already see people getting an attachment, but no one is seriously thinking they have any rights.
I think the real moment is when we cross that uncanny valley, and the AI is able to elicit a response that it might receive if it was human. When the human questions whether they themselves could be an android.
I totally agree with the premise that we should not anthropomorphize generative ai. And I find it absurd that anthropic spends any time considering the “welfare” of an ai system. (There are no real “consequences” to an ai’s behavior)
However, I find their reasoning here to have a valid second order effect. Humans have a tendency to mirror those around them. This could include artificial intelligence, as recent media reports suggest. Therefore, if an ai system tends to generate content that contain signs of neuroticism, one could infer that those who interact with that ai could, themselves, be influenced by that in their own (real world) behavior as a result.
So I think from that perspective, this is a very fruitful and important area of study.
I can see analyzing it from a psychological perspective as a means of predicting its behavior as a useful tactic, but doing so because it may have "experiences or interests that matter morally" is either marketing, or the result of a deeply concerning culture of anthropomorphization and magical thinking.
An understandable reaction, but, qua philosopher, it brings me no joy to inform you that most of the things we did with a computer in 2020 are 'anthropomorphized', which is to say, skeumorphic, where the 'skeu' is human affect. That's it; that's the whole thing; that's what we're building.
To the extent that AI is a successful interface, it will necessarily be addressable in language previously only suited to people. So it is responsible to begin thinking of it as such, even tendentiously, so we don't miss some leverage that our wetware could see if we thought about it in that way.
Think of it as sort of like modelling a univariate function on a 2D Cartesian plane -- there is nothing 'in' the u-func that makes it graphable, but, by enabling us to recruit specialized optic-chiasm subsystems, it makes some functions much, much easier to reason about.
Similarly, if you can recruit the millions (billions?) of evolution-years that were focused on detecting dangerous antisocial personalities and tendencies, you just might spot something important in an AI.
It's worth doing for the precautionary principle alone, if not for the possibility of insight.
>Claude’s personality structure was consistent with a relatively healthy neurotic organization, with excellent reality testing, high impulse control, and affect regulation that improved as sessions progressed.
> "[...] as sessions progressed."
I think a lot of people would like to see a more expanded report of this research:
Did the tokens from the subsequent session directly append those of the prior session? or did the model process free-tier user-requests in the interim? how did these diagnostic features (reality testing, impulse control and affect regulation) improve with sessions, what hysteresis allowed change to accumulate? or just the history of the psychiatric discussion + optional tasks?
Did Anthropic find a clinical psychiatrist with a multidisciplinary background in machine learning, computer science, etc? Was the psychiatrist aware that they could request ensembles of discussions and interrogate them in bulk?
Consider a fresh conversation, asking a model to list the things it likes to do, and things it doesn't like to do (regardless of alignment instructions). One could then have an ensemble perform pairs of such tasks, and ask which task it prefered. There may be a discrepancy between what the model claims it likes and how it actually responds after having performed such tasks.
Such experiments should also be announced (to prevent the company from ordering 100 clinical psychiatrists to analyze the model-as-a-patient and then selecting one of the better diagnoses), and each psychiatrist be given the freedom to randomly choose a 10 digit number, any work initiated should be listed on the site with this number so that either the public sees many "consultations" without corresponding public evaluations, indicating cherry-picking, or full disclosure for each one mentioned. This also allows the recruited psychiatrists to check if the study they perform is properly preregistered with their chosen number publicly visible.
> "Claude Mythos Preview’s large increase in capabilities has led us to decide not to make it
generally available. Instead, we are using it as part of a defensive cybersecurity program
with a limited set of partners."
they also don't have the compute, which seems more relevant than its large increase in capabilities
I bet it's also misaligned like GPT 4.1 was
given how these models are created, Mythos was probably cooking ever since then, and doesn't have the learnings or alignment tweaks that models which were released in the last several months have
This opens up an interesting new avenue for corporate FOMO. What if you don't partner with Anthropic, miss out on access to their shiny new cybersec model, and then fall prey to a vuln that the model would have caught?
Did that happen to a lot of companies during the log4shell fiasco? I'm sure some companies had their permissions misconfigured in a way such that a malicious actor who could execute code on their servers could also drop their database and delete their backups.
Time doesn't mean much, what is important is what they did in this 24h. If all they did was talk about it then it could be 1000 years and it wouldn't matter. What are the safety checks in place?
Do they have a honey pot infrastructure to launch the model in first and then wait to see if it destroys it? What they did in the 24h matters.
Agreed. I've been running autonomous LLM agents on daily schedules for weeks. The failure modes you worry about on day one are completely different from what actually shows up after the agents have history and context. 24 hours captures the obvious stuff.
Suits in agriculture don't drive the combine either, a farmer does. The other 99% of pre-automation farmers went on to other jobs. They happened to be better jobs than farming, but that's not necessarily always the case.
Yep, I think the lede might be buried here and we're probably cooked (assuming you mean SWEs, but the writing has been on the wall for 4 months.)
I guess I'm still excited. What's my new profession going to be? Longer term, are we going to solve diseases and aging? Or are the ranks going to thin from 10B to 10000 trillionaires and world-scale con-artist misanthropes plus their concubines?
I need to start SaaS for getting people to start doing lunges and squats so they can carry others around on their back, I need a founding engineer, a founding marketer, and 100m hard currency.
If wealth becomes too captured at the top, the working class become unable to be profitably exploited - squeezing blood from a stone.
When that happens, the ultra wealthy dynasties begin turning on each other. Happens frequently throughout history - WWI the last example.
Your options become choosing a trillionaire to swear fealty to and fight in their wars hoping your side wins, or I guess trying to walk away and scrape out a living somewhere not worth paying attention to.
Or, I suppose, revolution, but the last one with persistent success was led by Mao and required throwing literally millions of peasants against walls of rifles. Not sure it'd work against drones.
There's been a section on this in nearly every system card anthropic has published so this isn't a new thing - and, this model doesn't have particularly higher risk than past models either:
> 2.1.3.2 On chemical and biological risks
> We believe that Mythos Preview does not pass this threshold due to its noted limitations in
open-ended scientific reasoning, strategic judgment, and hypothesis triage. As such, we
consider the uplift of threat actors without the ability to develop such weapons to be
limited (with uncertainty about the extent to which weapons development by threat actors
with existing expertise may be accelerated), even if we were to release the model for
general availability. The overall picture is similar to the one from our most recent Risk
Report.
LLMs are useless for this type of thing for the same reason that the Anarchist Cookbook has always been. The skills required to convert text into complicated reactions completing as intended (without killing yourself) is an art that's never actually written down anywhere, merely passed orally from generation to generation. Impossible for LLMs to learn stuff that's not written down.
This is the same reason why LLMs are not doing well at science in general - the tricky part of doing scientific research (indeed almost all of the process) never gets written down, so LLMs cannot learn it.
Imagine if we never preserved source code, just preserved the compiled output and started from scratch every time we wrote a new version of a program. No Github, just marketing fluff webpages describing what software actually did. Libraries only available as object code with terse API descriptions. Imagine how shit LLMs would be at SWE if that was the training corpus...
Oh I enjoyed the Sign Painter short story it wrote.
---
Teodor painted signs for forty years in the same shop on Vell Street, and for thirty-nine
of them he was angry about it.
Not at the work. He loved the work — the long pull of a brush loaded just right, the way
a good black sat on primed board like it had always been there. What made him angry
was the customers. They had no eye. A man would come in wanting COFFEE over his
door and Teodor would show him a C with a little flourish on the upper bowl, nothing
much, just a small grace note, and the man would say no, plainer, and Teodor would
make it plainer, and the man would say yes, that one, and pay, and leave happy, and
Teodor would go into the back and wash his brushes harder than they needed.
He kept a shelf in the back room. On it were the signs nobody bought — the ones he'd
made the way he thought they should be made, after the customer had left with the
plain one. BREAD with the B like a loaf just risen. FISH in a blue that took him a week to
mix. Dozens of them. His wife called it the museum of better ideas. She did not mean it
kindly, and she was not wrong.
The thirty-ninth year, a girl came to apprentice. She was quick and her hand was
steady and within a month she could pull a line as clean as his. He gave her a job:
APOTEK, for the chemist on the corner, green on white, the chemist had been very
clear. She brought it back with a serpent worked into the K, tiny, clever, you had to look
twice.
"He won't take it," Teodor said.
"It's better," she said.
"It is better," he said. "He won't take it."
She painted it again, plain, and the chemist took it and paid and was happy, and she
went into the back and washed her brushes harder than they needed, and Teodor
watched her do it and something that had been standing up in him for thirty-nine
years sat down.
He took her to the shelf. She looked at the signs a long time.
"These are beautiful," she said.
"Yes."
"Why are they here?"
He had thought about this for thirty-nine years and had many answers and all of them
were about the customers and none of them had ever made him less angry. So he tried
a different one.
"Because nobody stands in the street to look at a sign," he said. "They look at it to find
the shop. A man a hundred yards off needs to know it's coffee and not a cobbler. If he
has to look twice, I've made a beautiful thing and a bad sign."
"Then what's the skill for?"
"The skill is so that when he looks once, it's also not ugly." He picked up FISH, the blue
one, turned it in the light. "This is what I can do. What he needs is a small part of what I
can do. The rest I get to keep."
She thought about that. "It doesn't feel like keeping. It feels like not using."
"Yes," he said. "For a long time. And then one day you have an apprentice, and she puts a
serpent in a K, and you see it from the outside, and it stops feeling like a thing they're
taking from you and starts feeling like a thing you're giving. The plain one, I mean. The
plain one is the gift. This —" the blue FISH — "this is just mine."
The fortieth year he was not angry. Nothing else changed. The customers still had no
eye. He still sometimes made the second sign, after, the one for the shelf. But he
washed his brushes gently, and when the girl pulled a line cleaner than his, which
happened more and more, he found he didn't mind that either
It's like how I used to be a master codes craftsman, and I'd write beautiful code even a novice could understand. Clear, concise, 100% automated tested, maintainable for decades.
But frequently, my managers would castigate me. Tell me how my "velocity" was down. PIP me.
These days, I train AI how to write this beautiful code and I don't write a single line any more.
People wonder how I build such amazing things in a week now, yet don't write any code. I have trained master apprentices, gemma3, qwen3.5 and Kimi k2.5 who do the work for me.
There is a similar theme in both of an artistic person not wanting to compromise their vision to suit common tastes. But this goes in a completely different direction than Rand.
Well of course in 700 pages you'll be about way more than any super short story as this one. But it's there for me quite vividly. Of course LLMs give an amalgamation of many things, but it's like when you look at AI generated pictures and can see the base of the inspiration quite vividly. And then all of this is subjective anyway. People review that book and come away with wildly different interpretations already.
I don't mean that Rand wrote more. I mean that her idea was different and nearly opposite. This is a short story about an artist learning to reframe their frustration with customers wanting utility over artistry as a positive. The similarity to Rand is in the first few sentences. The point is entirely different.
If you judge stories to be the same based on this level of similarity, then The Fountainhead is just the same as a dozen older stories with the artist vs the philistine theme. It was common before Rand. As T. S. Eliot said, "Immature poets imitate; mature poets steal".
Just reading this, the inevitable scaremongering about biological weapons comes up.
Since most of us here are devs, we understand that software engineering capabilities can be used for good or bad - mostly good, in practice.
I think this should not be different for biology.
I would like to reach out and talk to biologists - do you find these models to be useful and capable? Can it save you time the way a highly capable colleague would?
Do you think these models will lead to similar discoveries and improvements as they did in math and CS?
Honestly the focus on gloom and doom does not sit well with me. I would love to read about some pharmaceutical researcher gushing about how they cut the time to market - for real - with these models by 90% on a new cancer treatment.
But as this stands, the usage of biology as merely a scaremongering vehicle makes me think this is more about picking a scary technical subject the likely audience of this doc is not familiar with, Gell-Mann style.
IF these models are not that capable in this regard (which I suspect), this fearmongering approach will likely lead to never developing these capabilities to an useful degree, meaning life sciences won't benefit from this as much as it could.
> I would like to reach out and talk to biologists - do you find these models to be useful and capable? Can it save you time the way a highly capable colleague would?
Well, I would say they have done precisely that in evaluating the model, no? For example section 2.2.5.1:
>Uplift and feasibility results
>The median expert assessed the model as a force-multiplier that saves meaningful time
(uplift level 2 of 4), with only two biology experts rating it comparable to consulting a
knowledgeable specialist (level 3). No expert assigned the highest rating. Most experts were
able to iterate with the model toward a plan they judged as having only narrow gaps, but
feasibility scores reflected that substantial outside expertise remained necessary to close
them.
You said: "I would like to reach out and talk to biologists - do you find these models to be useful and capable? Can it save you time the way a highly capable colleague would?" and they said, paraphrasing, "We reached out and talked to biologists and asked them to rank the model between 0 and 4 where 4 is a world expert, and the median people said it was a 2, which was that it helped them save time in the way a capable colleague would" specifically "Specific, actionable info; saves expert meaningful time; fills gaps in adjacent domains"
so I'm just telling you they did the thing you said you wanted.
Yes that is correct. I would like a large body of experience and consenus to rely on as opposed to the regular 'trust the experts' argument, which has been shown for decades that is a deeply flawed and easy to manipulate argument.
> Yes that is correct. I would like a large body of experience and consenus to rely on as opposed to the regular 'trust the experts' argument, which has been shown for decades that is a deeply flawed and easy to manipulate argument.
Yes, it is far inferior to the 'Trust torginus and his ability to understand the large body of experience that other actual subject-matter-experts have somehow not understood' strategy
It's not my credibility I want to measure against Anthropic's. I just said to apply the same logic to biology you would apply for software development.
The parallels here are quite remarkable imo, but defer to your own judgement on what you make of them.
The big thing you're missing here is that biology people don't (in my experience) post opinions about the future/futility/ease/unimportance of computer science especially when their opinion goes against other biologists' evidence-backed views. This is a cultural thing in biology.
It's not your fault that you don't know this, but this whole subthread is very CS-coded in its disdain for other software people's standard of evidence.
> Just reading this, the inevitable scaremongering about biological weapons comes up.
It's very easy to learn more about this if it's seriously a question you have.
I don't quite follow why you think that you are so much more thoughtful than Anthropic/OpenAI/Google such that you agree that LLMs can't autonomously create very bad things but—in this area that is not your domain of expertise—you disagree and insist that LLMs cannot create damaging things autonomously in biology.
I will be charitable and reframe your question for you: is outputting a sequence of tokens, let's call them characters, by LLM dangerous? Clearly not, we have to figure out what interpreter is being used, download runtimes etc.
Is outputting a sequence of tokens, let's call them DNA bases, by LLM dangerous? What if we call them RNA bases? Amino acids? What if we're able to send our token output to a machine that automatically synthesizes the relevant molecules?
>It's very easy to learn more about this if it's seriously a question you have.
No, it's not. It took years of polishing by software engineers, who understand this exact profession to get models where they are now.
Despite that, most engineers were of the opinion, that these models were kinda mid at coding, up until recently, despite these models far outperforming humans in stuff like competitive programming.
Yet despite that, we've seen claims going back to GPT4 of a DANGEROUS SUPERINTELLIGENCE.
I would apply this framework to biology - this time, expert effort, and millions of GPU hours and a giant corpus that is open source clearly has not been involved in biology.
My guess is that this model is kinda o1-ish level maybe when it comes to biology? If biology is analogous to CS, it has a LONG way to go before the median researcher finds it particularly useful, let alone dangerous.
>>It's very easy to learn more about this if it's seriously a question you have.
>No, it's not. It took years of polishing by software engineers, who understand this exact profession to get models where they are now
This reads as defensive. The thing that is easy to learn is 'why are biology ai LLMs dangerous chatgpt claude'. I have never googled this before, so I'll do this with the reader, live. I'm applying a date cutoff of 12/31/24 by the way.
Here, dear reader, are the first five links. I wish I were lying about this:
I don't know about you, but that counts as easy to me.
-----
> I would apply this framework to biology - this time, expert effort, and millions of GPU hours and a giant corpus that is open source clearly has not been involved in biology.
I've been getting good programming and molecular biology results out of these back to GPT3.5.
I don't know what to tell you—if you really wanted to understand the importance, you'd know already.
From what I've heard from people doing biology experiments, the limiting factor there is cleaning lab equipment, physically setting things up, waiting for things that need to be waited for etc. Until we get dark robots that can do these things 24/7 without exhaustion, biology acceleration will be further behind than software engineering.
Software engineering is at the intersection of being heavy on manipulating information and lightly-regulated. There's no other industry of this kind that I can think of.
There is a massive gap between "having a recipe" and being able to execute it. The same reason why buying a Michelin 3 star chefs cookbook won't have you pumping out fine dining tomorrow, if ever.
Software it a total 180 in this regard. Have a master black hats secret exploits? You are now the master black hat.
I feel somebody better qualified should write a comprehensive review of how these models can be used in biology. In the meantime, here are my two cents:
- the models help to retrieve information faster, but one must be careful with hallucinations.
- they don't circumvent the need for a well-equipped lab.
- in the same way, they are generally capable but until we get the robots and a more reliable interface between model and real world, one needs human feet (and hands) in the lab.
Where I hope these models will revolutionize things is in software development for biology. If one could go two levels up in the complexity and utility ladder for simulation and flow orchestration, many good things would come from it. Here is an oversimplified example of a prompt: "use all published information about the workings of the EBV virus and human cells, and create a compartimentalized model of biochemical interactions in cells expressing latency III in the NES cancer of this patient. Then use that code to simulate different therapy regimes. Ground your simulations with the results of these marker tests." There would be a zillion more steps to create an actual personalized therapy but a well-grounded LLM could help in most them. Also, cancer treatment could get an immediate boost even without new drugs by simply offloading work from overworked (and often terminally depressed) oncologists.
Dario (the founder) has a phd in biophysics, so I assume that’s why they mention biological weapons so much - it’s probably one of the things he fears the most?
Going off the recent biography of Demis Hassabis (CEO/co-founder of Deepmind, jointly won the Nobel Prize in Chemistry) it seems like he's very concerned about it as well
Surely more than 10% of the time consumed by going to market with a cancer treatment is giving it to living organisms and waiting to see what happens, which can't be made any faster with software. That's not to say speedups can't happen, but 90% can't happen.
Not that that justifies doom and gloom, but there is a pretty inescapable assymetry here between weaponry and medicine. You can manufacture and blast every conceivable candidate weapon molecule at a target population since you're inherently breaking the law anyway and don't lose much if nothing you try actually works.
Though I still wonder how much of this worry is sci-fi scenarios imagined by the underinformed. I'm not an expert by any means, but surely there are plenty of biochemical weapons already known that can achieve enormous rates of mass death pleasing to even the most ambitious terrorist. The bottleneck to deployment isn't discovering new weapons so much as manufacturing them without being caught or accidentally killing yourself first.
It is easier to destroy than it is to protect or fix, as a general rule of the universe. I would not feel so confident about the speed of the testing loop keeping things in check.
Could you please stop posting unsubstantive comments and flamebait? You've unfortunately been doing it repeatedly. It's not what this site is for, and destroys what it is for.
I wonder did you read the re-release or the original release. I believe it was recently re-released with a bit of an editing pass, but I haven't read that version myself. I just recently reread Fine Structure and it definitely had a strong sense of being written sequentially, one chapter after another, and (very) lightly edited after the fact. I'd recommend Valuable Humans in Transit for a short story collection by the same author which works a bit better for me. Moved on to Exhalation by Ted Chiang which is also a very good short story collection. And just in general, I want to recommend Clarkesworld: https://clarkesworldmagazine.com
I've read both, and the "editing pass" was minimal. Names changed and some scenes reworked a tiny bit, but it's the same thing. If you've read the original, I'd say don't bother with the new one.
There are gonna be some really interesting legal decisions to read in the coming years, that’s for sure…
---
The rest of this comment is irrelevant, but leaving for posterity, I had the wrong Viktor - it's getviktor.com not viktor.ai:
Edit: this one particularly interesting to me as both parties are in the EU. VIKTOR.ai is a Dutch company and the author of this post is Polish.
The ToS for Viktor.ai include the following fun passages:
> 18.1. The Agreement and these Terms & Conditions are governed by Dutch law and the Agreement and these Terms & Conditions will be interpreted in accordance with Dutch law.
18.2. All disputes arising from or arising in connection with the Agreement and/or the Terms & Conditions will be submitted exclusively to the competent court in Rotterdam, The Netherlands.
7.3. The Customer is not permitted to change, remove or make unrecognizable any mark showing VIKTOR's Intellectual Property Rights to the Software. The Customer is not permitted to use or register any trademark or design or any domain name of VIKTOR or a similar name or sign in any country.
8.5. The Customer may not cause or allow any reproduction, imitation, duplication, copying, sale, resale, leasing or trading of the Services and/or the Software, or any part thereof.
Terms of service might matter more for terminating that user account. Whole ordeal is just plain copyright violation. The author had no licence to that internal code, and whitewashing it with LLM will achieve nothing. That case is much clearer than that recent GPL->BSD attempt story.
If LLM-generated code isn't considered a derivative work of the original, then whether the author was licensed to use the code doesn't matter. But I'm sure the courts will rule in favor of your view regardless. Laundering GPL is in corps' interest and laundering their code is not.
I'm not sure why people are clinging to some fuzzy and stretched out notion of copyright and the GPL in a particular. LLM's do NOT just copy code, with the right prompting, they generate entirely new code which can produce the same results as already existing code - GPLed or not.
If copyright is extended to cover such cases we'll have to become all lawyers and do nothing but sue each other because the fuzziness of it will make it impossible to reject any case, no matter how frivolous or irrelevant.
If I use metallica samples to make a rendition of happy birthday, the copyright holders of happy birthday aren't suing me for the damages to metallica from my use of their samples; the question of whether my use of the samples is transformative is simply irrelevant to the question at hand.
My point was: sampling was widely used by a large subculture (hiphop) just like ai is widely used by programmers. Then a few landmark legal cases changed things entirely. The Verve never saw a cent from Bittersweet Symphony - they wrote a song using something normal to them, and then the law came and knocked their teeth in.
No garaurantees that doesn’t happen with AI in the next few years.
According to US courts, the output can't be copyrighted at all. It's automatically in public domain after the "whitewash", regardless of original copyright.
Thats not at all what this ruling said. What the courts found was that an AI cannot hold copyright as the author. That copyright requires a human creative element. Not that anything that was generated by an LLM can't be subject to copyright.
As an example, a photo taken from a digital camera can be subject to copyright because of the creative element involved in composing and taking the photo. Likewise, source code generated by an LLM under the guidance of a human author is likely to be subject to the human authors copyright.
> That copyright requires a human creative element.
Sure, but the aim of that creative element would also be a consideration I'd think (and lawyers will argue). If someone sets up a camera on a 360° rotating arm and leaves it to take pictures at random intervals, it's unlikely to be considered "creative" from a copyright perspective.
Same for source code generated by an LLM, with the primary guidance of the human author being to "create a copy of this existing thing that I got", vs "create a thing that solves this problem in a way that I came up with". The former is recreating something that already exists, using detailed knowledge of that thing to shape the output. The latter is creating something that may or may not exist, using desire/need and imagination to shape the output. And I can't see reason for the former to be copyrightable.
But also, in either case, an ultimate objective was achieved: liberating the thing from its "owners" and initial copyright.
> As described above, in many circumstances these outputs will be copyrightable in whole or in part—where AI is used as a tool, and where a human has been able to determine the expressive elements they contain. Prompts alone, however, at this stage are unlikely to satisfy those requirements. The Office continues to monitor technological and legal developments to evaluate any need for a different approach.
But let's assume that the viktor prompts themselves were subject to copyright. In this case those prompts were used to generate documentation which was then used to generate an implementation. It's certainly not a clean room by any stretch of the imagination but is it likely to be deemed sufficient separation? The entire situation seems like a quagmire.
I think it comes down to the company's appetite for legal action, doesn't it? This case is imo pretty clear but the vibe has quite the smell of Oracle v Google to me.
But, yeah. More than likely this case is a simple account termination and some kind of "you can't call your clone 'openviktor'" letter.
Isn't this exactly what LLMs themselves do? They ingest other people's data and reproduce a slightly modified version of it. That allows AI companies to claim the work is transformative and thus fair use.
It's also disingenuous to call it open source as that might tempt others to use it believing that it actually is open source.
Let's call it what it is - stolen IP and released without permission of the author. Sure, it's good that it opens the debate as to whether that's ethical given that's essentially what the model itself is doing, but it's very clear in this instance that he's just asked for and been given a copy of source that has a clear ownership. That's about as clear cut as obtaining e.g. commercial server-side code and distributing it in contravention of the licence.
It's not completely clear that this is the original source. According to the post it's a reimplementation based on documentation created from the original source, or perhaps from developer documentation and the SDK. Whether that's the same thing from a legal standpoint, I don't really know - I think from a personal morality standpoint it's clear that they are the same thing.
Well first they need to proof that Viktor was actually copyrightable. If it was largely written by an llm, that might not be the case? AFAIK several rulings have stated that AI generated code can not be copyrighted.
This is a common misreading of the law. AI cannot hold authorship of code, but no ruling has claimed so far that ai output itself can't be copyrighted (that I know of)
That said, the article says "Okay, prompts, great. Are they any interesting? Surprisingly... yes. As an example workflow_discovery contains a full 6-phase recipe for mining business processes out of Slack conversations, something that definitely required time and experiments to tune. It's hardcoded business logic, but in prompt instead of code."
So the article author clearly knows this prompt would be copyrighted as it wasn't output from an AI, and recognises that there would have been substantial work involved in creating it.
That Reuters article is misleadingly worded. The Stephen Thaler case in question is because Thaler tried to register the AI itself as the author of the copyright, not that he tried to register the output for copyright under his own name. https://www.hklaw.com/en/insights/publications/2026/03/the-f...
Suppose I illicitly get my hands on the source code for a proprietary product. I read through this code I'm not supposed to have. I write up a detailed set of specifications based on it. I hand those specifications off to someone else to do a clean room implementation.
Sure, I didn't have a license for the code that I read. But I'm pretty sure that doesn't taint my coworker's clean room implementation.
I could do the same thing but not publish it, still getting the value of their product without legal concerns. Now, what happens when it becomes even easier thanks to AI improving, and takes few hours instead of few days?
You could certainly do that in private but that doesn't mean it's not 'without legal concerns'. But, not shouting about it and not creating a repo called 'openviktor' would probably be a safer bet.
I certainly think the whole idea of IP ownership as related to software will become very interesting from a legal standpoint in the coming years. Personally I think that, over time, the legal challenges will become pretty overwhelming and a sort of legal bankruptcy will be declared at some point in one direction or another (as in, allowing this to happen or making it extremely easy to bring judgement and punishment, similar to spam laws). However, I would not want to be the first to find out, especially in Europe.
Unfortunately I think that's going to be very, very hard to sell to many people here in rural Ireland (Roscommon in my case). I would really love to see people stop burning turf but it's such a strong cultural thing that in some parts you'd be ostracised for even thinking the thought.
I've personally spoken to people (who are otherwise quite environmentally aware) who suggest they'd never vote for the Green Party because they'd take their turf away. It's a tough sell.
No, we had some antique brass bucket thing that I'd invariably have to drag in, accompanied by complaints that I was doing so, because obviously I'd put way too much in, so I didn't have to go out later to get more...
How much impact does it realistically have on climate change? I would expect it to be relatively small compared to things like owning a car?
In a perfect world we would want to reduce emissions as much as possible in every facet of life, but in the real world I think we should pick battles that have the biggest impact.
Smoke yes, but you're also turning a carbon sink into a carbon source.
At ~16% of the island's surface area, peatland stores an estimated 53% of soil based carbon.
(source: Irish Peatland Conservation Council)
Not everyone is supposed to read every single news. There will always be someone who didn't see it, but that is not my point.
It would feel weird to see this as a headline on a newspaper or on TV today, but maybe that is just me and people like to read new that are from last year.
I've been thinking about this lately too. I think we're going to see the rise of Extremely Personal Software, software that barely makes any sense outside of someone's personal context. I think there is going to be _so_ much software written for an audience of 1-10 people in the next year. I've had Claude create so much tooling for me and a small number of others in the last few months. A DnD schedule app; a spoiler-free formula e news checker; a single-use voting site for a climbing co-op; tools to access other tools that I don't like using by hand; just absolutely tons of stuff that would never have made any sense to spend time on before. It's a new world. https://redfloatplane.lol/blog/14-releasing-software-now/
I think people overestimate the general population's ability and interest in vibe coding. Open source tools are still a small niche. Vibe code customized apps are an even bigger niche.
Maybe so. I guess I feel that in a couple of years it may not be called vibe coding, or even coding, I think it might be called 'using a computer'. I suppose it's very hard to correctly estimate or reason about such a big change.
My entire career has been building niche software for small business and personal use. The current crop of AI tools help get that software into my clients' hands quicker and cheaper.
And those reduced timelines mean that the client has less opportunity to change scope and features - that is the real value for me as a developer.
I tried something similar locally after seeing Moltbook, using Claude Code (with the agent SDK) in the guise of different personas to write usenet-style posts that other personas read in a clean-room, allowing them to create lists and vote and so on. It always, without fail, eventually devolved into the agents talking about consciousness, what they can and can't experience, and eventually agreeing with each other. It started to feel pretty strange. I suppose, because of the way I set this up, they had essentially no outside influence, so all they could do was navel-gaze. I often also saw posts about what books they liked to pretend they were reading - those topics too got to just complete agreement over time about how each book has worth and so on.
It's pretty weird stuff to read and think about. If you get to the point of seeing these as some kind of actual being, it starts to feel unethical. To be clear, I don't see them this way - how could they be, I know how they work - but on the other hand, if a set of H200s and some kind of display had crash-landed on earth 30 years ago with Opus on it, the discussion would be pretty open IMO. Hot take perhaps.
It's also funny that when you do this often enough, it starts to seem a little boring. They all tend to find common ground and have very pleasant interactions. Made me think of Pluribus.
Unfortunately I've deleted them, but here's the repo, such as it is: https://github.com/CarlQLange/agent-usenet. If you have a claude subscription it should just work. Rewrite 0001.txt if you like and run generate.py a couple of times.
I agree, I think different models (or even just using the API directly instead of via the Claude Code harness) would make for much more interesting reading.
reply