A community dedicated to the profession of IT Systems Administration
No generic Lemmy issue posts please! Posts about Lemmy belong in one of these communities:
!lemmy@lemmy.ml
!lemmyworld@lemmy.world
!lemmy_support@lemmy.ml
!support@lemmy.world
Inb4 an out of court settlement happens in about a year and a half with CS not admitting fault but paying an undisclosed amount to Delta.
The Atlanta-based judge also let Delta pursue a computer trespass claim, and a narrowed claim that CrowdStrike fraudulently promised not to introduce an "unauthorized back door" into the carrier's computers.
Also, this will be interesting.
unauthorized back door
Isn't autoupdating software by definition an authorized backdoor by virtue of enabling it? The whole premise of CrowdStrike is continuous updates for attacks they see in the wild on other companies' systems.
Also if anything CrowdStrike did the opposite of a backdoor since everyone needed to find their BitLocker keys to get back in and clean this mess. It locked out the front and back door.
There was an additional auto update function that wasn't disclosed. Delta had disabled the auto update because, like many large companies, they prefer to deploy changes incrementally so that an issue doesn't blow-up all their systems at once.
So...
Isn't autoupdating software by definition an authorized backdoor by virtue of enabling it?
Yes. Which is why they contend disabling it makes it unauthorized.
That's not how that works. CS didn't have at the time, an option to disable channel file updates. It's how their edr works. Delta's mssp or secops group, %100 knew this as it's in CS own documentation. They really don't have a foot to stand on here, but CS will pay it to make it go away.
CS didn't have at the time, an option to disable channel file updates
Yes, that's the crux of the accusation. Given the large number of people who seemed to be under the impression that selecting a staggered release cadence would protect them from a faulty update, it's not unreasonable to think that people were caught off guard by a second autoupdate system that they couldn't configure that could also tank their system.
Before this, you could throttle the rollout for channel files. You could knock it down to 1 a minute if you wanted.
Channel files were not something that CS admins didn't know about.
I wouldn't call an auto update mechanism an unauthorised backdoor, it is required behaviour for that kind of software.
It's absolutely not required behavior! Software for servers has very different requirements from software for end users, and if you have a lot of them you also want to manage your end user machines differently.
Updates can go wrong, and if you roll out a bad update to everything at once you can crash everything and lose a lot of money. As aptly demonstrated by cloudstrike.
That's why Delta and many other companies disabled the auto update functions: so they could control the rollout cadence.
They reasonably believed that disabling autoupdates disabled them. They didn't expect a second autoupdate system that wasn't documented, wasn't controlled by the autoupdate system settings and couldn't be disabled.
It's not a second auto update. It's %100 documented in the software and you can %100 throttle it. Channel files are heavily discussed when you roll out CS.
https://www.crowdstrike.com/en-us/blog/falcon-content-update-preliminary-post-incident-report/
Might want to let crowdstrike know.
Rapid Response Content Deployment
Implement a staggered deployment strategy for Rapid Response Content in which updates are gradually deployed to larger portions of the sensor base, starting with a canary deployment.
Improve monitoring for both sensor and system performance, collecting feedback during Rapid Response Content deployment to guide a phased rollout.
Provide customers with greater control over the delivery of Rapid Response Content updates by allowing granular selection of when and where these updates are deployed.
Provide content update details via release notes, which customers can subscribe to.
https://www.theregister.com/2024/07/23/crowdstrike_lessons_to_learn/
Maybe you're thinking of changes that they made as a result of the incident?
No channel files where %100 there. It's in the general GUI settings. You could throttle channel files. Now after this your able to do General availability, Early availability or pausing them.
Yes crowd strike is a huge security risk
Did any of the passengers on those 7,000 flights get any compensation? Or just a "so sorry, out of our control?"
Looks like delta tried to dismiss the passengers claims, but they are being sued for it.
Can I sue them for the 13 people that threw shit at me because I couldn't get them their stupid addiction tickets?
... their stupid addiction tickets?
What are addiction tickets?
Lottery tickets, usually scratch offs
I feel sorry fot you but I must ask, was it literal shit?
No, just random items. Whatever was close. I got hit in the face with a can of pepsi. That wasn't fun.
Holy shit. No permanent damage?
Busted open my eyebrow but that was about it. I'm not trying to come off all tough guy but in comparison to other things I've gone through, it really wasn't a big deal.
I hope you're on my team when SHTF, then
I'm on an island in the Atlantic Ocean that is not part of America and barely part of Canada. You got a trek to get over here and on my side lol
is Stupid Addiction a band name?
I work for a company that formerly used CrowdStrike. Since the event we no longer do.
Its okay they gave ubereats vouchers. That should cover everything
*clownstrike
When this happened it was pretty clear (to me, I was at least) that Delta didn't have an actual BCP. This is their CYA lawsuit and should be throw out - their negligence predates CloudStrike's incompetence.
That notwithstanding, iirc the update was pushed to all of their "rings" even the n+2 or whatever.
Cs clearly fucked up and I don't see why they shouldn't be penalized for it.