Bad Software Examples: How Much Can Poor Code Hurt You?

Bad software exists; everyone knows that. In the imperfect world, a set of a few coincidences, e.g., human errors, faulty code, or unforeseen circumstances, can cause a huge failure even in pretty good systems. Today let’s go through real-world examples where catastrophic software failures or errors caused huge losses and even cost a human life.

UK Post Office Software Bug Led to Convicting 736 Innocent Employees

The UK Post Office has been using software called Horizon for 20 years. It had bugs that caused it to report that accounts under the employees’ control were missing money. It looked like an employee stole thousands. As a result 736 post office operators were convicted. People lost jobs, families, and one woman was sent to prison while pregnant. One man committed suicide after the system showed his account was missing £100,000.

The whole situation is controversial because there is evidence that the legal department knew about system issues before the convictions were made. The Post Office started offering compensation and says that will replace the Horizon system with a cloud-based solution.

TUI Airline Miscalculated Flight Loads

In 2020, three flight loads were miscalculated. TUI check-in software treated travelers identified as “Miss” as children. As the passengers’ weight is used to estimate thrust during the take off, it led to an unfortunate miscalculation. Children are counted as 35kg and adults as 69kg. Lower calculated weight means lower thrust during take off. With an unfavorable passenger list, such a case can lead to a disaster. Fortunately, the final thrust value was within the safety limit, and everyone traveled without issues.

Citibank UX Caused a $500 Million Failure

Source: Court filing

Have you heard about Oracle FLEXCUBE? It’s a banking system used by Citibank. In 2020, employees wanted to send around $7.8 million in interest payments. By filling not enough fields in the form, almost $900 million was sent. The interesting fact is that transactions of this size need to be approved by 3 people, and in practice, all of them thought that the form was filled out correctly. Let’s not dive into the legal details, but as a result, Citibank hasn’t received back around $500 million.

Hawaii Missile False Alarm

In 2018, Hawaiian emergency alerting systems issued alerts about incoming ballistic missiles. Such an event caused widespread panic, some people hid their children in sewers, and others recorded their final messages to their families. The whole mobile network got overloaded, people were not able to call 911. It took 38 minutes to send a message that there was no danger and call-off the alarm.

The whole situation was thoroughly analyzed, and among the causes, multiple issues were identified. Among them were poor UI and human communication errors. The employee who started the alarm was fired. The whole alarm procedure was changed, so it now requires confirmation from 2 people to start the alarm.

Uber Sued for $45 Million Because of a Notification Showing After Log-Out

The Uber application had a bug; it was showing notifications even when the application was logged out. Sounds dangerous? Not really. In practice, a French businessman was cheating on his wife and notifications about his rides were sent to his wife’s phone. Why? Because he used Uber on her phone before but has logged out. The software bug concerned only the iPhone version and was fixed already. The couple has divorced, and the Frenchman sued Uber for $45 million.

Revolut Lost $20 Million

In early 2022, more than $20 million was stolen from Revolut. It appeared that due to differences between U.S. and European systems, some transactions were refunded using Revoluts money after being declined. The refunded amounts were withdrawn from ATMs. The software bug existed probably since 2021 and was patched in the spring of 2022 when Revolut’s partner notified that company funds were missing. The vulnerability was exploited by various malicious actors, and more than $20 million was stolen this way.

Nest Thermostat Update Left Users in the Cold Because of Software Bugs

Do you own a smart home? Google produces the Nest smart thermostat. Around the winter of 2016, a software fault caused its battery to drain and in the result to turn off the heating. Winter without heating? It can cause a lot of problems, for some even more, since some users were traveling and had the thermostat set to avoid freezing pipes.

That was not the only historical fault in Nest software. When you’re using IoT or Smart home devices, you need to keep in mind that updates or infrastructure outages can influence what works at your home.

Knight Capital Group's $440M Loss Due to Bad Trades

Knight Capital Group was leveraging an automated trading software. Due to multiple bugs and human operator mistakes, the system bought hundreds of millions of shares in 45 minutes. It appears that the new code release was not deployed to one of the company servers, and at the same time, the new release reused the old flag with other meaning. The flag was activated on all servers, with new and old code, and that led to the execution of old, unused test functions, which spawned all those orders.

The company lost $440 million due to those operations, and its stock price collapsed. That resulted in its acquisition by a competitor within the next year.

Equifax's Massive Data Breach

That's one of the largest stories from last year. Equifax was hacked, and attackers gained access to data related to hundreds of millions of people. Why has that happened? Again, due to multiple causes. Systems weren’t patched against the known vulnerability, although administrators were told so.

What is more, multiple other bad security practices were exposed, like inadequate internal systems separation or plain text passwords stored in the system. Hackers were able to access data for months before they got detected. After that event, Equifax spent $1.4 billion to improve security.

Toyota Software Glitches Killed 89 People

Toyota had to recall more than 8 million cars due to software errors. Some vehicles were accelerating, even when the gas pedal was not touched. Investigation showed that systems were badly designed, and had poor quality and had various software bugs, including memory corruption, buffer overflow, unsafe casting, race conditions, and others. The whole story took years in practice. Toyota claimed first that the problem was caused by floor mats. They got fined $1.2 billion for concealing safety defects. The most important acceleration related piece of code appeared to have huge cyclomatic complexity, in practice making it untestable.

Conclusions

There are a lot of such stories, and we could go on and on with various top software failures. What can we learn from them? Software is everywhere. It is in different parts of our life — homes, cars, healthcare, and work. Bad quality and bugs can destroy lifes, kill people, or cause huge financial losses. This clearly shows how important is the responsible software team, how important are the security and quality practices and how important is the UI and the UX!

Any negligence, like skipping vulnerable libraries, web servers, or operating systems updates, can lead, when combined with other factors, to massive data breaches. Nowadays, the software development process should include various procedures and practices, allowing to prevent all those tragic situations. How? For example, it should include computer systems security audits, UX tests, and proper test code coverage, among others. However, we need to remember that even if we have all of that, humans still make mistakes. As shown in the examples, the biggest software failures are the result of a set of different overlapping factors. A single human decision shouldn’t cause an issue, but only if the whole development and operation process is good.