andrewducker | A modest proposal on avoiding massive international computer failures

I propose that if you supply software to more than X customers (say 10,000 but I'm totally open to discussion on this point) then you must have insurance that covers you for legal liability for damages.

Which is to say, if your software causes you to be sued then your legal insurance will cover it - but they get to set your premiums. And if there's one thing that insurance companies are very good at, it's pricing risk.

So you can shop around for a good deal, they will absolutely want to look at your processes regularly to make sure you're not being a bunch of careless idiots, and everyone wins except for the for the shareholders/business owners/senior management who thought that cutting testing and good practices to the bone was a good idea.

Flat | Top-Level Comments Only

From:

simont

Sell software to more than X customers, I hope you mean?

Someone who provides software free (even if it's only "as in beer") has no income to pay the premium from, and not very much control over how popular it gets. So if this rule doesn't exempt them, they'll have no choice but to avoid getting into that situation in the first place.

From:

andrewducker

Yes, fair point.

From:

bens_dad

I have seen suggestions that a current/recent EU proposal (product liabilities directive ?) on software will cause problems for suppliers of open source, even though open source itself is exempt.

Actually,how does Andrew's insurance proposal compare with the EUs ?

Plus there is the issue of removing access to software on public repositories. A year or two back there was a significant issue where a developer removed his code and lots of dependent software broke.

From:

hairyears

It would be ironic if the not-so-unintended consequence of legislative (and insurer-led) pressure ended the widespread use of free software.

I can foresee a future in which you (or the court-appointed receivers following your bankruptcy) are forced to sell Putty to Larry Ellison, after a campaign of well-funded lawsuits.

Even failed lawsuits using a sensibly-drafted law would would work - 'winning' your case in court can be costly - but you would lose, repeatedly, if a Machiavellian lobbying campaign convinced legislators to create an author liability for 'malicious use of inadequately secured software'.

Edited (Even nastier ideas kept on plopping-up) Date: 2024-07-22 06:54 am (UTC)

From:

simont

Being forced to sell PuTTY to someone is a particularly weird idea because I don't really know what it would mean to "sell" it in the first place. Supposing some corporation, or rich individual, wanted to buy it from me (for any reason, not just your lawsuit-bankruptcy scenario), what concrete aspect of intellectual property or other right do they imagine they'd get?

I can't reassign the copyright. Not all of it is mine – it's shared between me and many past contributors.

I can't sell the trademark rights, because there is no legal trademark.

I can't relicense it under a proprietary licence, because that would need the consent of all the copyright holders, and I'm not even in touch with them all.

It's not a question of nobody having offered me a big enough incentive yet. I can't do any of those things. They are not within my legal capacity.

Other people already have the right to make and distribute modified versions; they wouldn't need to buy that right from me. Many people have already done it. Most users don't switch over to the new thing as a result. But I don't think the reason why not has anything to do with intellectual property rights. It has to do with trust in the maintainers: do you trust the person who's already been maintaining the thing for 20 years, or some new person who's just picked it up and has no track record? Or some company who obviously has their own agenda?

And that's a thing that can't be bought and sold in the first place – no legal or financial transaction between me and Hypothetical Buyer can make millions of users change who they trust. A new maintainer has to earn trust on their own merits.

Which, in a roundabout way, brings me to my own suggestion in another comment, of independent inquiries into disasters as an alternative to lawsuit insurance. The real question after a disaster is "Is it likely to happen again? Should I lose my trust in the maintainers, and therefore, ditch this software and migrate to something else?" So what's needed is a way to let the public make an informed decision about that trust.

From:

hairyears

That's the logical view:the corporate view is "Whatever it is, we can own it" and everything you've said sounds like diffuse and unstructured ownership.

In other words, an acquisition target with a manageable legal risk.

I would quite like potential 'owners' to be proven wrong at vast expense: but arrogance, propertarian greed, and a seven-figure legal budget has been the modern definition of 'right' since the Enclosures.

From:

hairyears

As for how it could work in practice - an enclosure of the commons in open-source software - we would have to look at a legislative or commercial compulsion that offers an opportunity for monopolistic rents.

An obligation from insurers, legislators (or banking regulators), compelling the commercial world to use software packaged in a way that allows its sale or rental in a wrapper that provides locked-down source, standardised testing documentation, and indemnity insurance for its use, would be the obvious way to go.

Even (or especially) if it's unworkable, it will be profitable for someone to 'own' that packaging.

The worst-case outcome is that monopolistic operating system vendors gain the ability - or the right - to charge for the packaging and indemnity insurance of all and any software distributed though their 'app store'; and users face strict liability for the consequences of running uncertified code with bugs and vulnerabilities.

A handful of vendors might be able to package, certify, and insure their own software outside the OS app-store walled garden: Sage and Sony and Oracle and Cap Gemini consulting spring to mind.

This would supply the necessary fig-leaf of an open and competitive market, and broaden the funding base for lobbying and PR work to tell the masses (and, more importantly, the legislators) that it's working in a 'free market'.

Dystopian, much?

Edited (Premature 'post' ) Date: 2024-07-27 08:18 am (UTC)

From:

simont

I agree that's a way dystopian corporate overlords might arrange to effectively 'own' my free software, and everybody else's. But they didn't do it by buying it from me! Here, they're making a fork and compelling users to switch over to it by directly imposing requirements on those users. They would have no need to get my permission, or to offer me any kind of incentive.

This isn't PuTTY being bought by a corporation, as you originally suggested. It's PuTTY being nationalised without compensation, by a totalitarian government. (It's just that the "government" is in charge of some computing platform with an app store, rather than a geographic nation.)

From:

simont

Separately from that nitpick, a random thought I had a few days ago along the same lines of "I wonder what rule we could change":

When a sufficiently bad bug or vulnerability happens, the vendor is required to submit to an independent inquiry into how it wasn't caught. An external team of investigators comes in and looks around the inside of your company and how it works; you must let them see everything they need to see; they piece together the story, and produce a report of their findings, in a similar style to RAIB / MAIB / etc. A Software Accident Investigation Branch.

The report is public. It goes up on the SAIB website. So all the details of your SW development process are published to the world (or at least all the details that the investigators deemed relevant to the disaster). Not just to the insurance company; not just to the victims; not just to your current customers. To everyone, whether they had a prior relationship to your company or not. And they're the details of what really happens, as best the investigators can determine – not the process as you optimistically wrote it down in the internal wiki, which 99% of the time nobody bothers to follow in detail.

If the report concludes "despite appearances, this actually was a mistake that could have happened to anyone, they just got really unlucky" then fair enough, no harm done. But if it reads like a damning indictment of every development practice the company has, then all their customers start leaving for fear of the next similar disaster. And prospective customers (who will now adopt a due-diligence standard practice of checking the SAIB database before they do business with any new vendor) will steer clear.

Again, this has to exclude hobbyists writing free software, because otherwise you just make a huge incentive for them never to do that in the first place, and also, because it's not clear how the SAIB could investigate them successfully. But if a hobbyist free software author wants to be trusted as a source of safe or secure software, then it's on them to voluntarily hold themself to a similar standard, by e.g. publishing honest and unbiased root cause analyses of any big vulnerabilities they do have, so that users can make an informed decision about whether to carry on trusting them.

(I'm not sure whether it applies to corporate open-source activity. That's a weird in-between case.)

From:

simont

Oh, and one other exemption to the exemption: if a free software project goes out of its way to prevent people using alternatives to it, then perhaps I change my mind and bring it under this regulation anyway, on the grounds that I don't at all mind making a big incentive for free software authors not to do that!

(I'm thinking of things like the way systemd seems to have deliberately tied itself more and more tightly to the Linux kernel, trying to position itself as the only possible choice of userland init software, while simultaneously taking over the functionality of many previously separate pieces of software such as NTP and DNS daemons.)

From:

hairyears

An interesting corollary: many software bugs are an unfortunate discovery of errors in the operating system.

Microsoft will, of course, spend enough in court to win the legal argument that they are not in any way at fault, and pass the costs and the blame to the software authors and owners.

But... Would these court cases and enquiries take place in camera? The secrets would get out - not least, because the broke and bankrupt losers would have nothing left to lose from breaking NDA's - and this would become an avalanche of speculative litigation from any chancer who spots similar 'bugs' in unrelated software.

From:

splodgenoodles

Henry VIII and the meat laws, I reckon!

From:

armiphlage

Details? Googling is failing me.

From:

splodgenoodles

Unfortunately it appears I was talking through my hat.

I can't find anything either. I was told that he made laws about meat safety standards that meant people could not sell rotten meat.

*sigh*

Let's just say I wish it were true.

From:

armiphlage

... talking through your hat? In the sense of blustering? Sorry, not a term generally used in Canadian English.

https://english.stackexchange.com/questions/509356/where-does-talking-through-your-hat-come-from ?

I think you were correct, just with the wrong number; it seems Henry III is the one that created the "Assisa panis et cervisiae" laws about food safety, pricing, and quality:

https://books.google.ca/books?id=tKZFAAAAcAAJ&pg=PA22&redir_esc=y#v=onepage&q&f=false

https://www.btb.termiumplus.gc.ca/tpv2alpha/alpha-fra.html?lang=fra&i=1&index=enb&srchtxt=CERVISIAE

From:

splodgenoodles

Not quite blustering. Perhaps spouting opinions without a basis in fact.

Excellent links, thankyou! I will drop the V in any further conversation on the matter. :-D

From:

bens_dad

I think this is a great idea.
‐------------‐----------‐----
This might reduce the sale of buggy software and make prices reflect actual costs of quality software.

I trust that the insurance industry can figure out how to apportion blame? The current incident appears to be an interaction between MS and a security product ...

What happens to open source; do insurance companies and developers/packagers provide support packages with insurance included ?

The insurance world has been pulling out of some sectors recently (professional indemnity and car clubs to name two).
It is possible that this proposal would make some software unavailable :-(

From:

drplokta

It would massively increase the cost of software, of course, right down to that 99p iPhone app whose sole developer would now have to hire a security team in order to get the mandatory insurance. And you would probably see a lot of great software that you could never buy, because having sold their permitted 9,999 copies they were shutting up shop (and ceasing to support their 9,999 customers) rather than increase the price tenfold and make life far harder for themselves.

From:

drplokta

And so I think a better idea is to make the customer liable for doing appropriate due diligence. If you’re buying a 99p game for your phone, the appropriate level is none. If you’re buying software that keeps a national payment system running, then you must require the supplier to do the things that you’re proposing insurance companies would require.

From:

andrewducker

It is precisely to avoid every single customer of Crowdstrike having to work out how much risk there is in Crowdstrike, when that isn't their primary expertise, that I think that the company providing the software should be the one who should bear the cost and then share it between the 10,000 customers.

An insurer can hire specialists to say "iPhone app, no special permissions, that'll be £20 to cover costs. An antivirus solution that injects into the kennel, we're going to need to see your processes and then charge accordingly."

From:

drplokta

It’s not the risk in CrowdStrike specifically that the customer needs to work out, it’s the consequences of their own organisation suffering a massive outage. That informs the level of due diligence that is appropriate.

Edited Date: 2024-07-21 05:20 pm (UTC)

From:

andrewducker

Yes, but the insurance companies societies understanding the risk of your software causing a massive software outage is definitely easier than every company, down to my local GP, having to carry out an audit of how cowboy their antivirus providers are. I mean, I don't understand how they would even begin to do that.

From:

movingfinger

How in the name of goodness is every customer of software supposed to do this "due diligence" and become an instant security expert? You know hairdressers, gig drivers, small- and medium-sized business owners, independent workers, and people who have no training to understand software are "customers," right? There is a huge, wide exposure to risk and loss between "99p app" (which can be used to hack your phone and empty your bank accounts) and "software that keeps a national payment system running."

From:

andrewducker

If you had to provide insurance to an iOS app with no special permissions, how much would you need to charge for it?

What damages would you be assuming that they could do that the insurance might need to cover?

(Assuming that you were spreading risk across 10,000 of them)

From:

drplokta

Worst case is that it bricks all 10,000 phones, taking irreplaceable photos and videos with it. Call it £20 million of potential liability for an app with 10,000 users. I’d want to charge quite a bit to cover that.

From:

andrewducker

How does an iOS app with no special permissions brick an iPhone?

I mean, I'm not an expert in iPhones and their sandbox model, but I'm pretty sure that's the kind of thing that they aren't supposed to be able to do.

Feels like that's in the "What if the app causes spontaneous human combustion?" range of risk.

From:

bens_dad

If it tickles some bug in iOS, is your insurance company going to argue with Apple's lawyers or just pay up ?

From:

andrewducker

That's the advantage of insurance companies - they tend to be very large and have their own lawyers and to be able to pay for expertise.

From:

jducoeur

I think you're overestimating the willingness of insurance companies to engage with highly-variable risks like this. It is pretty normal for insurance companies to just nope out of providing insurance if the market they're covering isn't sufficiently large and homogeneous. (And governments often want to meddle in the details, often rendering the market unviable.)

From:

calimac

Is there evidence that "cutting testing and good practices to the bone" was the cause of this bug, and that it was not just a bug, as could happen to anyone no matter how careful they are?

From:

andrewducker

1) The file that went out crashed every machine it went onto. Which means that file wasn't tested.
2) It was rolled out globally, rather than a phased rollout to 1% (wait a day), 5% (wait a day), then full rollout (or something similar). Which is incredibly bad practice for something like this.

So they're definitely not following best practice or doing proper testing.

From:

calimac

Ah. Thank you for a solid professional evaluation.

From:

andrewducker

Just to follow up now I'm not on my phone:
If it had crashed 0.1% of the machines they sent it to then it could absolutely be something that got through testing. Testing absolutely fails, and it's never going to be perfect or cover every eventuality for anything complex. But, so far as I understand, it broke every machine it went out to. Which means it couldn't possibly have been tested. I *suspect* that people tested a pre-release version of it, and then some part of the release process broke it, which nobody spotted because the actual files that were released weren't then tested.

Really, though, it's the lack of a phased rollout that's damning. Both Firefox and Chrome do staggered rollouts, where it takes a week or two to reach all users (unless there's a critical reason to roll things out faster), because they recognise that they're going into a heterogenous environment and something might come up that means that they have to pause the rollout before it knocks out a large number of users. For something like Crowdstrike, which is doing brain surgery on the operating system, not doing so is reckless.