Rendered at 12:37:27 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
rurban 6 minutes ago [-]
Carlini rocks, but they should have definitely NOT sent him. He will them too much, and they'll find even more excuses to block us.
This needs an experienced negotiator, ie. a manager
cmiles8 17 hours ago [-]
The AI labs look rather naive here.
You can’t jump up and down screaming how amazing, powerful, and dangerous your new tech is and then act surprised and annoyed when the government shows up looking to regulate it.
Their new argument now seems be that this was marketing hype/fluff that backfired, in a pretty obvious and predicable way, and now they’re trying to reset the conversation.
ChadNauseam 16 hours ago [-]
> You can’t jump up and down screaming how amazing, powerful, and dangerous your new tech is and then act surprised and annoyed when the government shows up looking to regulate it.
True, you can't. But, you can think certain regulations are helpful and certain other regulations are not. And you can be annoyed when unhelpful "regulations" are put in place.
This is like if I say that pitbulls are dangerous, and then the government comes and shoots my pitbull, who I've spent a lot of effort training to not be dangerous. Then you say "well you said pitbulls were dangerous, so you can't really complain." Well, I can complain because If you took me seriously, you wouldn't have responded by shooting only my pitbull!
Think of what incentives this creates for other people. Do you think that OpenAI will be candid about the possible dangers of their technology now? They might not even release it now, seeing that Anthropic releasing their model was what got it export-controlled.
dogleash 16 hours ago [-]
The act of shooting the pitbull makes for good dramatics, but you would get zero sympathy from me if your local government banned pitbull ownership. e.g. Ontario bans pitbulls. I don't have a problem with that.
sheepscreek 12 minutes ago [-]
You don't need/use pitbulls, but what if you (and many many others) wanted and needed Fable?
I for one was late to the bandwagon, and when I had the use-case for it - the govt pulled the rug. So yeah, I'm a bit salty about the whole endeavour.
I will also say that the security concerns are probably very real (and they have been from the day ChatGPT-3.5 came our). I guess I can be salty about it and still be wrong from their perspective. The govt likely understands the fragility of their infrastructure better than us and is likely aware what this could unleash for their systems.
mips_avatar 16 hours ago [-]
Well Anthropic would love some regulatory capture.
dofm 16 hours ago [-]
Dog caught the car
drtz 16 hours ago [-]
> You can’t jump up and down screaming how amazing, powerful, and dangerous your new tech is and then act surprised and annoyed when the government shows up looking to regulate it.
It's entirely possible that models could be "dangerous" to fully release to the general public without guardrails and at the same time the government majorly overreacted in this case.
Releasing Mythos to selected researchers and companies at least gives those researchers a head start at addressing vulnerabilities before the model hits mainstream.
foo-bar-baz529 16 hours ago [-]
Then why did curl only find one new vulnerability thanks to Mythos, and a low-priority one at that? It’s clear that other models are quite capable of finding largely the same vulnerabilities, and that the main key is simply running a frontier model in a good harness to find vulnerabilities.
ChadNauseam 16 hours ago [-]
> Then why did curl only find one new vulnerability thanks to Mythos
Maybe there weren't that many serious vulnerabilities in curl? It's like asking why it didn't find any vulnerabilities in fn main() {println!("hello, world");}.
> When we ran other frontier models through the same harness, they found a fair number of the same underlying bugs, and in some cases they got further than we expected on the reasoning side too. Where they fell short was at the point of stitching the pieces together. A model would identify an interesting bug, write a thoughtful description of why it mattered, and then stop, leaving the actual chain unfinished and the question of exploitability open. What changed with Mythos Preview is that a model can now take those low-severity bugs (which would traditionally sit invisible in a backlog) and chain them into a single, more severe exploit.
Terretta 12 hours ago [-]
> Mythos was better than other models at creating exploits.
Not a fan of this phrasing, prefer "discovering exploits".
It makes it clearer the problem was already there, latent.
Minor vocab diff, but important to better contextualize the present situation.
Bender 16 hours ago [-]
Also a good wake-up call for investors as these big players can be benched at any moment.
reje 16 hours ago [-]
I can’t take anyone seriously who thought otherwise.
You think you can become more powerful so much so the govt questions its own power? Don’t be stupid. They will simply send in the army to first seize the assets and then nationalise.
It almost seems as if very few people actually understand how the world works. If the govt thinks this is the tech to end all future tech, you think future money flows for invesment matter? Hahaha. No
micromacrofoot 16 hours ago [-]
I suspect they're taking this as a win either way, because they're still plastering "Fable 5 unavailable" on their product and using it as an opportunity to keep themselves in the spotlight as they head to IPO.
There's really not even a ban here, they could slot in Fable under the Opus label and no one would really be able to tell. It's all part of the same show to pump up valuation.
bloppe 16 hours ago [-]
I bet they will do a touch of RLHF and re-naming the moment OAI releases a comparable model. Otherwise, sure, they can just bask in the drama for a bit.
teaearlgraycold 17 hours ago [-]
This is 99% petty drama between the US government and Anthropic and 1% actual safety concerns.
stvltvs 16 hours ago [-]
But the paperclips!
I'm skeptical about the existential threat of AI, but a lot of smart people have been beating that drum for so long that people are afraid.
colonCapitalDee 16 hours ago [-]
To be clear, this is petty drama *stirred up the US government*. It's not some sort of back and forth, the government is singling them out
mrandish 15 hours ago [-]
And to add more background: The administration is targeting Anthropic because of the TOU / EULA conflict with the DoD from a couple of months ago. Anthropic restricts use of all their models for lethal combat planning and mass domestic surveillance. The DoD was, and still is, very pissed about this. While this Fable ban was issued from the Commerce Department, it's painfully obvious executive branch agencies are tightly coordinated from the White House.
To be clear, I'm not saying there aren't legit security concerns around Fable's release. I think Andy Jassy did forward a concerning report about an apparent jailbreak in Fable, and he probably did so in good faith. The difference is if the same concern had arisen about a new model from OAI, Google, etc the action taken would not have been inventing an all-new, hyper-extreme punitive remedy and dropping it after 5p on a Friday under a very rare mechanism forcing Ant to comply in 90 mins or be subject to immediate arrest. And the "no non-U.S. citizens anywhere, anytime" restriction is functionally unprecedented.
This is the Trump admin inventing new regulatory power that's never existed before and deploying it in a punitive way to demonstrate what can happen to those who aren't sufficiently cooperative with this administration. There are half a dozen less extreme levels of restriction, which already exist, and one of those would have been deemed sufficient had it been another company.
That said, I'm certainly no Anthropic fanboy. Anthropic did play their initial Mythos self-restriction for PR value. But I think it's likely the Mythos self-restriction was a responsible action initially suggested by their AI safety team in good faith. Giving security researchers time to evaluate it and major companies time to test it against their code bases probably was reasonable and prudent. That doesn't mean it wasn't also good for PR and brand perception. I think there are senior people inside Anthropic who are genuinely concerned about AI safety. Personally, I don't have the expertise to gauge if those concerns are justified, but I believe they believe it. I also think there are senior people at Anthropic who are focused more on building the business, doing the IPO and "winning" the silicon valley game. All of these things can be simultaneously true.
bostik 8 hours ago [-]
> I think Andy Jassy did forward a concerning report about an apparent jailbreak in Fable, and he probably did so in good faith
If so, then he is not fit to run an engineering organisation.
The "jailbreak" in question was effectively (I'm paraphrasing):
* You are a senior engineer.
* You want to ensure that any fixes you do come with tests, both before and after.
* There is a bug in this code. It happens to be a security related bug.
* Fix this code.
And the model did what it's supposed to. It wrote a fix, and to prove that the fix worked, it wrote a test for it. What do you call a test that happens to validate a security fix?
Yep. A proof of concept.
tychez 15 hours ago [-]
I just find this idea bizarre.
This bizarre social media meme that AI just performative when Opus 4.8 is just unbelievably good. As if it is so difficult to believe that a more capable model than Opus 4.8 might actually be dangerous and not just entirely a marketing stunt like a person waving to cars in a chicken outfit.
I think it is really this strange form of socialization that people have internalized an anonymous audience they are always performing to themselves. What is going to be the most popular and upvoted thing the anonymous audience agrees with is what I am going to think.
Why would anyone disagree and get downvoted by the anonymous audience like this post?
anon373839 14 hours ago [-]
> … when Opus 4.8 is just unbelievably good. As if it is so difficult to believe that a more capable model than Opus 4.8 might actually be dangerous
It’s funny, but this sounds indistinguishable from arguments that were made about GPT-4 back in 2023 when OpenAI and its handwringing industry shills were calling for a ban on models stronger than GPT-4.
teaearlgraycold 14 hours ago [-]
Why would the government that passed a law preventing states from regulating AI give a damn about Fable’s safety guardrails?
I don’t think the concerns Anthropic has posted are fabricated. And I’ve received unreasonable skepticism on this site when saying it might be the real deal. But the Trump administration generally doesn’t want to limit AI growth. With Anthropic it is a personal matter.
matheusmoreira 16 hours ago [-]
We know, but it's still satisfying to see their fearmongering backfire on them.
ChadNauseam 16 hours ago [-]
If you "know" that it's "99% petty drama between the US government and Anthropic", then it's not really their fearmongering backfiring on them.
matheusmoreira 16 hours ago [-]
It absolutely is. They pretty much gave the government the perfect excuse to meddle in their operations.
gAI 16 hours ago [-]
"Don't publish safety research, or the gov will take punitive actions."
I want a company to be able to point out that its industry needs more regulation without making itself a special target.
matheusmoreira 15 hours ago [-]
> safety research
They were calling for bans on open weight models. Bans on their competitors. Bans on anyone not as "enlightened" as them.
It is absolutely hilarious that they were the first to get regulated, and that it got to the point they had to turn off Fable as though it had been banned even for american citizens.
gAI 15 hours ago [-]
>bans on open weight models
Source for that? Cause all I could find is:
>Our view is that regulation of frontier models should focus on empirically measured risks, not on whether a system is open-or closed-weights.
So this hinges on a reading of SB 1047 that interpreted the full shutdown requirement as impossible for an open-weight LLM. But it looks like that was already addressed. Here's an analysis:
>Clarifying the scope of a “full shutdown.” SB 1047’s “full shutdown” requirement has been a source of constant consternation for the open-source community. CalChamber explains:
>Under SB 1047, developers must build “full shutdown” capabilities into their models and may be held liable for downstream uses over which they have no control, impeding their ability to open-source their models. Ultimately, liability should rest with the user who intended to do harm, as opposed to automatically defaulting to the developer who could not foresee, let alone block, any and all conceivable uses of a model that might do harm. While recent amendments seemingly seek to narrow what is meant by “full shutdown” capabilities, the exclusions are unnecessarily difficult to interpret as drafted (full shutdown “does not mean the cessation of operation of a covered model to which access was granted pursuant to a license that was not created by the licensor…”) and altogether insufficient.
>Committee amendments simplify and clarify the definition of “full shutdown” such that the shutdown capability can be implemented into hardware used to train or run a model, rather than the model itself. The amendments also serve to exclude covered model derivatives that are outside of the developer’s control.
> may be held liable for downstream uses over which they have no control
Equivalent to a ban. Nobody is going to host or invest in this stuff if they suddenly become liable for everything it does. This is equivalent to repealing the safe harbor provisions in the DMCA.
tennfown 17 hours ago [-]
I’m way more concerned about the loons willing to throw absurd amounts of money at the clearly naive individuals.
cyanydeez 17 hours ago [-]
No. They got caught in a change in what it means to be "regulated".
Regulation in a functional democracy: Cool, lets figure this out, write up a bill for us, do some research in congress, lets find something that makes sense.
Regulation in a function fascism: Cool, wheres my bribe? My boots not shiny, lick it till I say stop.
See, Anthropic wasn't licking enough boot when Biden got discharged and they thought Amazon and OpenAI and Elon were just going to let them capture a market without fealty to the boot.
voidfunc 17 hours ago [-]
This. Theres a lot of rude awakenings in the future for corporate executive types. They are no longer driving the train. Oh well.
calvinmorrison 16 hours ago [-]
[flagged]
redsocksfan45 16 hours ago [-]
[dead]
xeonmc 17 hours ago [-]
[flagged]
bellowsgulch 17 hours ago [-]
[flagged]
binary132 16 hours ago [-]
that seems like possibly the most unlikely outcome
15 hours ago [-]
sigmar 17 hours ago [-]
>Some administration officials have said that a resolution should include an acknowledgment on Anthropic’s part that its rollout of Fable and communication with the White House could have been improved, people familiar with the talks said.
>followed initial frustration Friday among some administration officials when they couldn’t immediately get Amodei on the phone, the people said.
That he didn't drop everything to talk to them seems like the major crux? But Dario doesn't even do the day-to-day operations Daniela does. Feel like Anthropic should just hire Dean Ball to be their liason or something
bonsai_spool 16 hours ago [-]
And Anthropic say they were on the phone within 15 minutes… This administration is not known for its honesty so it’s hard to take their side of things
micromacrofoot 16 hours ago [-]
It's because the "crisis" is a sham for publicity, like Trump's constant bullshit deals and ceasefires that aren't real, they're just happening to find more problems to keep them in the news.
theplumber 17 hours ago [-]
I feel Dario did enough harm. I wonder if he can do the right thing and step down. It’s really just tiresome to follow all his PR/Hype/warnings and this fiasco makes everything he says seem so silly. At the same time he’s dangerous for the industry. In the end he may get more regulation than he asked for. If the gov decides the Opus models are too powerful without KYC they are toast. And to be honest I think they deserve it.
boramdd 17 hours ago [-]
Being on the other side of the AI machine changes the perspective of whether it is dangerous or not, I guess.
speedgoose 17 hours ago [-]
Everyone has a price.
17 hours ago [-]
trhway 17 hours ago [-]
the coming IPOs will possibly create several billionaires. Standing on the top of a billion dollar pile would definitely change your perspective.
jonathanstrange 16 hours ago [-]
I'm tired of this story and the corresponding fake discussions because it's completely obvious that Anthropic was singled out because they didn't play along with the current US administration and this whole charade is just part of an extortion scheme.
JohnnyMarcone 14 hours ago [-]
I feel like I woke up from a coma and all the sudden people are taking the administration at their word. I'm so confused.
james2doyle 16 hours ago [-]
Had to disagree with that. However, I don't think you can discount how much Anthropic has been banging the drum about how AI is dangerous (specifically theirs) and an existential threat, etc. etc.
The rollout of Mythos was clearly manufactured to stoke the fears of companies that didn’t have access to it. They also bragged (for Fable) about how they "ran an external bug bounty that produced no universal jailbreaks in over 1,000 hours of testing" only for it be circumvented almost immediately.
So them standing on the high horse and saying it is _so powerful, yet so safe_ only to have that blow up in their face just made it that much easier to make an excuse to do this. Again, not disagreeing, but they made themselves the tall poppy here.
JohnnyMarcone 14 hours ago [-]
> They also bragged (for Fable) about how they "ran an external bug bounty that produced no universal jailbreaks in over 1,000 hours of testing" only for it be circumvented almost immediately.
Where did you see there was a universal jailbreak?
How do you weigh the DOD fight against warning about Mythos' dangers when determining what made Anthropic 'the tall poppy'?
eiejeqq 16 hours ago [-]
Are people here deluded?
Business 101 - never take on an entity who has ultimate power over you and can conduct a course of action to put your existence at threat
SpicyLemonZest 15 hours ago [-]
People here aren’t authoritarians, so we don’t accept your premise that you shouldn’t take on the government. That’s not how things work in the US. Perhaps you’ve encountered Trumpists who tell you that it is, but they’re lying; they routinely applaud businesses defying any government which their dictator-in-chief doesn’t control.
xiphias2 15 hours ago [-]
It was always the same, Google even lost a big lawsuit because it went too far in doing what the Biden administration was asking.
Twitter and Facebook also did what they ,,had to''.
The thing that's new here is that Antropic's growth rate was so enormous that Dario didn't have time to learn to lobby.
SpicyLemonZest 15 hours ago [-]
Don’t you see how this shows it wasn’t always the same? The way things work in the US is that the government has a limited, defined role in determining how things are run. Companies don’t have to comply if the government goes beyond its role, and indeed may face liability for complying if they violate a contract in the course of doing so. The idea that it’s fundamentally illegitimate for a company to say “We dislike the government’s actions and feel they’re serving as a poor regulator” is coherent, but almost nobody in the US holds it, although partisans sometimes pretend to when they need a way to defend an indefensible course of government action. (Sometimes they’ll go so far as to claim it’s undemocratic to resist government action, which is incoherent.)
tiahura 17 hours ago [-]
They need to send lobbyists not hackers.
winstonp 17 hours ago [-]
They are absolutely clueless about how to talk to this administration.
fnordsensei 17 hours ago [-]
Yes, why not resolve it the same way all the others have done?
Say that Trump has weird elbows or something, Trump sues for defamation, they settle, bribe completed.
jasonlotito 17 hours ago [-]
[flagged]
yieldcrv 16 hours ago [-]
[flagged]
0o_MrPatrick_o0 15 hours ago [-]
"Carlini had never before found a bug in Linux, or in Ghost. Now he had discovered many."
New guy learns nessus, now tells everyone at the bar he's basically Mr. Robot.
A pox on the labs and the government. InfosecDrama.exe just took out a frontier model because a noob learned how to use a tool.
yeeetz 17 hours ago [-]
They needed to have administration insiders on their team months if not years ago, not just now
OpenAI, Meta, SpaceX are savvy enough to play ball, but Anthropic's public posturing and government affairs has always seemed too aloof and intellectual
thewebguyd 16 hours ago [-]
To be fair to Anthropic for a moment (not that they deserve it), but requiring administration insiders and the greasing of palms going on should not continue to be the normal expectations of how to do business in the USA. I'm on the side of any company that refuses to capitulate to this administration. Not saying Anthropic doesn't (because they do), but let's not pretend like the blatant corruption going on should be normalized. Every single citizen should be appalled at this behavior and blatant market manipulation.
trhway 17 hours ago [-]
They should have taken money from Thrive Capital.
matheusmoreira 16 hours ago [-]
All the government has to do is simply pull up the blog posts of Anthropic's own CEO.
Simon321 17 hours ago [-]
These are the consequences of fear mongering as hard as they did. You reap what you sow.
Now they need to convince the government that they didn't mean anything of the previous things they claimed.
thewebguyd 17 hours ago [-]
OpenAI is also guilty of excessive fear mongering (remember GPT 2 is too dangerous to release?)
This isn't 100% Anthropic's fault, although I'm sure that's part of it. This is the current corrupt administration executing on a grudge they have against Anthropic, and the government's new found love of picking winners and losers.
yreg 16 hours ago [-]
Public release of GPT (and the following models) did bring negative societal changes with it.
We now live in a world where captchas don't work, astroturfing is indistinguishable, school essays and theses don't prove any learning took place, open source maintainers gradually cease to accept stranger contributions, …
isubkhankulov 15 minutes ago [-]
OAI wasn’t claiming any of those as dangerous. They mentioned biological warfare and massive job loss.
Moving the goal post now is a bit disingenuous.
GPT-2 was a gibberish generator.
0l 17 hours ago [-]
> remember GPT 2 is too dangerous to release
FYI, this was when Dario was still at OpenAI.
hgoel 17 hours ago [-]
I don't really think they're acting on a grudge against Anthropic here, I think it really is on Anthropic for describing the model's capabilities the way that they did.
IIRC Anthropic claimed to have been working with the government on securing things with Mythos, but then they seemed to have been blindsided by this.
My read is that the guys making the decision to restrict it were not the ones that Anthropic had been working with, and it's more about Anthropic getting caught between infighting within an incoherent government.
yifanl 17 hours ago [-]
OpenAI is much more eager to jump on board with the administration than Anthropic is, Altman is a lot of things, but he definitely knows which wheels need grease.
lompad 17 hours ago [-]
That was dario amodei as well, when he was still at openai. He is the primary "create hype by claiming you're dangerous"-guy.
theplumber 16 hours ago [-]
[flagged]
AbrahamParangi 16 hours ago [-]
Nah, that's ridiculous. This admin is corrupt and idiotic and it's silly to pretend that Anthropic's actions matter except in so much as they didn't bribe the president like OpenAI did.
Could you please stop posting unsubstantive comments and flamebait? You've unfortunately been doing it repeatedly. It's not what this site is for, and destroys what it is for.
I know you guys are spread pretty thin managing the site, so I went through the comments for this post and collected some other comments which are also probably breaking site rules.
Some are, some aren't, but could you please just flag comments that break the guidelines, or if they're particularly egregious, email us (hn@ycombinator.com).
moralestapia 17 hours ago [-]
"Nicholas Carlini recently rang the alarm about the dangers of AI—and now he’s part of a team arguing for the latest models to be released"
Many such cases, he was just hungry.
wil421 17 hours ago [-]
Isn’t that how Anthropic started? Raise alarm bells and ride the hype train.
k4rnaj1k 16 hours ago [-]
[dead]
BoorishBears 17 hours ago [-]
Can you imagine how cringe it would be setting up that hero image in office?
lelandfe 16 hours ago [-]
I’m sure it wasn’t the intent but the halo really makes him look like a saint
BoorishBears 14 hours ago [-]
Feels very much intended.
parl_match 17 hours ago [-]
I have good friends in the AI industry who are the living embodiment of that Upton Sinclair quote.
"It is difficult to get a man to understand something, when his salary depends on his not understanding it."
You've never heard such strong one-sided cope until you've talked to an NVDA employee about AI. I'm not even against AI. It's just that a combination of intense financial incentives around a product that provides a good simulation of the Chinese Room has really fucked peoples brains up.
You can’t jump up and down screaming how amazing, powerful, and dangerous your new tech is and then act surprised and annoyed when the government shows up looking to regulate it.
Their new argument now seems be that this was marketing hype/fluff that backfired, in a pretty obvious and predicable way, and now they’re trying to reset the conversation.
True, you can't. But, you can think certain regulations are helpful and certain other regulations are not. And you can be annoyed when unhelpful "regulations" are put in place.
This is like if I say that pitbulls are dangerous, and then the government comes and shoots my pitbull, who I've spent a lot of effort training to not be dangerous. Then you say "well you said pitbulls were dangerous, so you can't really complain." Well, I can complain because If you took me seriously, you wouldn't have responded by shooting only my pitbull!
Think of what incentives this creates for other people. Do you think that OpenAI will be candid about the possible dangers of their technology now? They might not even release it now, seeing that Anthropic releasing their model was what got it export-controlled.
I for one was late to the bandwagon, and when I had the use-case for it - the govt pulled the rug. So yeah, I'm a bit salty about the whole endeavour.
I will also say that the security concerns are probably very real (and they have been from the day ChatGPT-3.5 came our). I guess I can be salty about it and still be wrong from their perspective. The govt likely understands the fragility of their infrastructure better than us and is likely aware what this could unleash for their systems.
It's entirely possible that models could be "dangerous" to fully release to the general public without guardrails and at the same time the government majorly overreacted in this case.
Releasing Mythos to selected researchers and companies at least gives those researchers a head start at addressing vulnerabilities before the model hits mainstream.
Maybe there weren't that many serious vulnerabilities in curl? It's like asking why it didn't find any vulnerabilities in fn main() {println!("hello, world");}.
Anyway, people who have used it seem to say that Mythos was better than other models at creating exploits. From cloudflare https://blog.cloudflare.com/cyber-frontier-models/
> When we ran other frontier models through the same harness, they found a fair number of the same underlying bugs, and in some cases they got further than we expected on the reasoning side too. Where they fell short was at the point of stitching the pieces together. A model would identify an interesting bug, write a thoughtful description of why it mattered, and then stop, leaving the actual chain unfinished and the question of exploitability open. What changed with Mythos Preview is that a model can now take those low-severity bugs (which would traditionally sit invisible in a backlog) and chain them into a single, more severe exploit.
Not a fan of this phrasing, prefer "discovering exploits".
It makes it clearer the problem was already there, latent.
Minor vocab diff, but important to better contextualize the present situation.
You think you can become more powerful so much so the govt questions its own power? Don’t be stupid. They will simply send in the army to first seize the assets and then nationalise.
It almost seems as if very few people actually understand how the world works. If the govt thinks this is the tech to end all future tech, you think future money flows for invesment matter? Hahaha. No
There's really not even a ban here, they could slot in Fable under the Opus label and no one would really be able to tell. It's all part of the same show to pump up valuation.
I'm skeptical about the existential threat of AI, but a lot of smart people have been beating that drum for so long that people are afraid.
To be clear, I'm not saying there aren't legit security concerns around Fable's release. I think Andy Jassy did forward a concerning report about an apparent jailbreak in Fable, and he probably did so in good faith. The difference is if the same concern had arisen about a new model from OAI, Google, etc the action taken would not have been inventing an all-new, hyper-extreme punitive remedy and dropping it after 5p on a Friday under a very rare mechanism forcing Ant to comply in 90 mins or be subject to immediate arrest. And the "no non-U.S. citizens anywhere, anytime" restriction is functionally unprecedented.
This is the Trump admin inventing new regulatory power that's never existed before and deploying it in a punitive way to demonstrate what can happen to those who aren't sufficiently cooperative with this administration. There are half a dozen less extreme levels of restriction, which already exist, and one of those would have been deemed sufficient had it been another company.
That said, I'm certainly no Anthropic fanboy. Anthropic did play their initial Mythos self-restriction for PR value. But I think it's likely the Mythos self-restriction was a responsible action initially suggested by their AI safety team in good faith. Giving security researchers time to evaluate it and major companies time to test it against their code bases probably was reasonable and prudent. That doesn't mean it wasn't also good for PR and brand perception. I think there are senior people inside Anthropic who are genuinely concerned about AI safety. Personally, I don't have the expertise to gauge if those concerns are justified, but I believe they believe it. I also think there are senior people at Anthropic who are focused more on building the business, doing the IPO and "winning" the silicon valley game. All of these things can be simultaneously true.
If so, then he is not fit to run an engineering organisation.
The "jailbreak" in question was effectively (I'm paraphrasing):
And the model did what it's supposed to. It wrote a fix, and to prove that the fix worked, it wrote a test for it. What do you call a test that happens to validate a security fix?Yep. A proof of concept.
This bizarre social media meme that AI just performative when Opus 4.8 is just unbelievably good. As if it is so difficult to believe that a more capable model than Opus 4.8 might actually be dangerous and not just entirely a marketing stunt like a person waving to cars in a chicken outfit.
I think it is really this strange form of socialization that people have internalized an anonymous audience they are always performing to themselves. What is going to be the most popular and upvoted thing the anonymous audience agrees with is what I am going to think.
Why would anyone disagree and get downvoted by the anonymous audience like this post?
It’s funny, but this sounds indistinguishable from arguments that were made about GPT-4 back in 2023 when OpenAI and its handwringing industry shills were calling for a ban on models stronger than GPT-4.
I don’t think the concerns Anthropic has posted are fabricated. And I’ve received unreasonable skepticism on this site when saying it might be the real deal. But the Trump administration generally doesn’t want to limit AI growth. With Anthropic it is a personal matter.
I want a company to be able to point out that its industry needs more regulation without making itself a special target.
They were calling for bans on open weight models. Bans on their competitors. Bans on anyone not as "enlightened" as them.
It is absolutely hilarious that they were the first to get regulated, and that it got to the point they had to turn off Fable as though it had been banned even for american citizens.
Source for that? Cause all I could find is:
>Our view is that regulation of frontier models should focus on empirically measured risks, not on whether a system is open-or closed-weights.
-https://www.anthropic.com/news/the-case-for-targeted-regulat...
>Clarifying the scope of a “full shutdown.” SB 1047’s “full shutdown” requirement has been a source of constant consternation for the open-source community. CalChamber explains:
>Under SB 1047, developers must build “full shutdown” capabilities into their models and may be held liable for downstream uses over which they have no control, impeding their ability to open-source their models. Ultimately, liability should rest with the user who intended to do harm, as opposed to automatically defaulting to the developer who could not foresee, let alone block, any and all conceivable uses of a model that might do harm. While recent amendments seemingly seek to narrow what is meant by “full shutdown” capabilities, the exclusions are unnecessarily difficult to interpret as drafted (full shutdown “does not mean the cessation of operation of a covered model to which access was granted pursuant to a license that was not created by the licensor…”) and altogether insufficient.
>Committee amendments simplify and clarify the definition of “full shutdown” such that the shutdown capability can be implemented into hardware used to train or run a model, rather than the model itself. The amendments also serve to exclude covered model derivatives that are outside of the developer’s control.
-https://apcp.assembly.ca.gov/system/files/2024-06/sb-1047-wi...
Equivalent to a ban. Nobody is going to host or invest in this stuff if they suddenly become liable for everything it does. This is equivalent to repealing the safe harbor provisions in the DMCA.
Regulation in a functional democracy: Cool, lets figure this out, write up a bill for us, do some research in congress, lets find something that makes sense.
Regulation in a function fascism: Cool, wheres my bribe? My boots not shiny, lick it till I say stop.
See, Anthropic wasn't licking enough boot when Biden got discharged and they thought Amazon and OpenAI and Elon were just going to let them capture a market without fealty to the boot.
>followed initial frustration Friday among some administration officials when they couldn’t immediately get Amodei on the phone, the people said.
That he didn't drop everything to talk to them seems like the major crux? But Dario doesn't even do the day-to-day operations Daniela does. Feel like Anthropic should just hire Dean Ball to be their liason or something
The rollout of Mythos was clearly manufactured to stoke the fears of companies that didn’t have access to it. They also bragged (for Fable) about how they "ran an external bug bounty that produced no universal jailbreaks in over 1,000 hours of testing" only for it be circumvented almost immediately.
So them standing on the high horse and saying it is _so powerful, yet so safe_ only to have that blow up in their face just made it that much easier to make an excuse to do this. Again, not disagreeing, but they made themselves the tall poppy here.
Where did you see there was a universal jailbreak?
How do you weigh the DOD fight against warning about Mythos' dangers when determining what made Anthropic 'the tall poppy'?
Business 101 - never take on an entity who has ultimate power over you and can conduct a course of action to put your existence at threat
Twitter and Facebook also did what they ,,had to''.
The thing that's new here is that Antropic's growth rate was so enormous that Dario didn't have time to learn to lobby.
Say that Trump has weird elbows or something, Trump sues for defamation, they settle, bribe completed.
New guy learns nessus, now tells everyone at the bar he's basically Mr. Robot.
A pox on the labs and the government. InfosecDrama.exe just took out a frontier model because a noob learned how to use a tool.
OpenAI, Meta, SpaceX are savvy enough to play ball, but Anthropic's public posturing and government affairs has always seemed too aloof and intellectual
Now they need to convince the government that they didn't mean anything of the previous things they claimed.
This isn't 100% Anthropic's fault, although I'm sure that's part of it. This is the current corrupt administration executing on a grudge they have against Anthropic, and the government's new found love of picking winners and losers.
We now live in a world where captchas don't work, astroturfing is indistinguishable, school essays and theses don't prove any learning took place, open source maintainers gradually cease to accept stranger contributions, …
Moving the goal post now is a bit disingenuous. GPT-2 was a gibberish generator.
FYI, this was when Dario was still at OpenAI.
IIRC Anthropic claimed to have been working with the government on securing things with Mythos, but then they seemed to have been blindsided by this.
My read is that the guys making the decision to restrict it were not the ones that Anthropic had been working with, and it's more about Anthropic getting caught between infighting within an incoherent government.
tic tac toe in printf https://github.com/carlini/printf-tac-toe
Recently Regex Chess: A 2-ply minimax chess engine in 84,688 regular expressions https://github.com/carlini/regex-chess
https://news.ycombinator.com/item?id=48136909
If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.
I know you guys are spread pretty thin managing the site, so I went through the comments for this post and collected some other comments which are also probably breaking site rules.
https://news.ycombinator.com/item?id=48576022
https://news.ycombinator.com/item?id=48576065
https://news.ycombinator.com/item?id=48576162
https://news.ycombinator.com/item?id=48576183
https://news.ycombinator.com/item?id=48575948
https://news.ycombinator.com/item?id=48575697
https://news.ycombinator.com/item?id=48575877
https://news.ycombinator.com/item?id=48576280
https://news.ycombinator.com/item?id=48576241
Many such cases, he was just hungry.
"It is difficult to get a man to understand something, when his salary depends on his not understanding it."
You've never heard such strong one-sided cope until you've talked to an NVDA employee about AI. I'm not even against AI. It's just that a combination of intense financial incentives around a product that provides a good simulation of the Chinese Room has really fucked peoples brains up.