- A faulty update from cyber security firm CrowdStrike sent some 8.5 million PCs worldwide into the “blue screen of death” last Friday. Many of the computers controlled banking, retail, travel and emergency services systems.
- Experts say it was the worst IT outage in history, with losses incurred by airlines and others likely to surpass $1 billion.
- NZ’s National Cyber Security Centre warns scammers are trying to exploit the incident.
Texas firm CrowdStrike has revealed how it came to release a flawed update. The cyber security firm’s Australian president said it would be hard to avoid affected businesses seeking compensation or litigation.
Close to a week after it caused global chaos, CrowdStrike has revealed a quality control issue.
The US cyber security firm said a bug in a quality-control tool it uses to check system updates for mistakes allowed a critical flaw to be pushed to users’ machines.
The faulty update caused the “blue screen of death” and hit banking, travel, retail and some emergency services worldwide.
CrowdStrike said it now plans more pre-release testing - including a “canary” approach that will mean future updates get gradually rolled out to larger and larger groups of users. Customers will also be given “granular selection of when and where updates are deployed.”
Lovina McMurchy, an executive with Wellington-based cyber security firm Kry10 said the US firm was simply falling into line with industry norms.
“Some of the Crowdstrike issues were related to a software bug. However, some of them were about not using standard best practices such as staggered deployment of changes and giving customers control over when to accept updates.”
Meanwhile, CrowdStrike’s Australian president Michael Sentonas has apologised for the cyber security company’s role in causing an outage that crippled global IT systems - and conceded it would be hard to avoid affected businesses seeking compensation or litigation.
“Those conversations have to happen and will happen,” Sentonas told AAP on Tuesday.
“That phase will come after we get our customers remediated.”
New Zealand law firm Russell McVeagh said in a note to clients that legal class actions against CrowdStrike were “a real possibility”. Action by shareholders against affected firms who had a lack of readiness was also a possibility.
While the full cost of the outage is difficult to quantify, some experts have estimated the impact to businesses globally at over $1b.
In New Zealand, issues were reported with ASB, ANZ and Kiwibank debit and credit cards.
Jetstar cancelled flights and Woolworths closed stores. Their immediate focus was on mopping up after Crowdstrike’s mess.
Emergency Management and Recovery Minister Mark Mitchell said earlier this week he had not received any information to indicate ongoing issues caused by the CrowdStrike fault.
At this early stage the Government had not identified any need to talk to CrowdStrike about compensation, Mitchell said.
Shares tank, potential EU fine could add to the pressure
CrowdStrike shares were down another 4% in late trading on the Nasdaq. The firm has now lost around a quarter of its value since last Friday, wiping about US$20b (NZ$33.7b) from its market capitalisation.
Fast Company says there could be further pressure on the stock, and given CrowdStrike’s outage could have on some level involved breaches or issues related to personal data, it may come under the crosshairs of European regulators.
Those regulators can impose fines of up to 4% of annual revenues on companies that violate General Data Protection Regulation rules.
Last year, CrowdStrike’s global revenue was just over US$3b, implying a potential EU fine of up to US$160m.
The publication said CrowdStrike also faced a potential lawsuit “avalanche” in the US, with San Francisco-based Lieff Cabraser Heimann & Bernstein among firms collecting information from clients about business losses to “hold Crowdstrike accountable for its disruption of global business and the consequences thereof for all internet users”.
‘Rough deal’
McMurchy - who previously held senior roles for Microsoft and Amazon in the US - anticipated in a guest column shortly after the CrowdStrike outage that internal gaps in testing and other quality processes caused the outage.
She said that was symptomatic of a wider issue with Big Tech rushing updates, and customers wearing the risk.
“When it comes to quality and security, they are incentivised to do just enough to squeak by as customers can’t always see the impact of shortcuts in these areas.
“Part of the reason for this is that software is sold under ‘buyer beware’ terms and conditions.
“The software maker decides on the trade-off between new features and quality and, if the customer chooses to purchase, then they inherently take on the risk of those trade-offs.
“This is a rough deal given many customers are less technically able to assess those trade-offs than the makers of the software.”
Today, McMurchy added, “The lesson here is that the move towards more autonomous software services needs to be backed by more resilient systems and stronger quality processes. In future we may even see the emergence of more software certifcation in cyber security such as that used in safety-critical systems like cars, planes and factories.”
Chris Keall is an Auckland-based member of the Herald’s business team. He joined the Herald in 2018 and is the technology editor and a senior business writer.
Take your Radio, Podcasts and Music with you
