Building Green Software by Anne Currie, Sarah Hsu and Sara Bergman, published by O'Reilly is available here under a CC BY-NC-ND Creative Commons license i.e. you can read it and quote it for non commercial purposes as long as you attribute the source (O'Reilly's book) and don't use it to produce derivative works.

You can buy the book from good bookstores including Amazon in all regions (currently on offer in the UK). or read it on the O'Reilly site if you are an O'Reilly subscriber.

Efficiency is everything. Or is it?

“I feel the need - the need for speed” - Maverick

Whenever you bring up the subject of software efficiency, the first question that springs to mind for most developers is what’s the fastest and therefore most efficient coding language. Is it Rust, C, or Go? Maybe it’ll surprise us all and actually be Java! (Dear reader, it isn’t. Although they are doing work there. One day it, or something similar, could be).

Controversially, much as we love those languages we’re going to argue that, for most folk, programming in them is not the optimum way to build green software. In fact, we’re going to attempt to convince you that although code efficiency is a tool in your toolbox, it is not the first one most engineers should reach for. 

For most of us, writing more streamlined code is almost certainly a less effective way to reduce our carbon footprint than a combination of operational efficiency improvements, architecting for demand shifting and shaping, and choosing the right platform. 

There are circumstances where active code efficiency on your part is vital to greeness. If you are writing code that will be deployed at hyper scale it must be optimized. You are presumably already doing that in order to manage your costs and SLAs, and reading books specifically designed to help you with that skilled and context-specific task. However, even if you’re not running systems at scale, as a producer of code you still have a critical role to play in making the tech industry green, and this chapter is here to talk about what it is. 

Before we start, though, let’s take a step back and look at the very idea of code efficiency because it is both key to the climate crisis and a potentially distracting and expensive rabbit hole. 

What is code efficiency?

Efficient code is code that doesn’t do more work than necessary to achieve a defined functional result. But how much is too much? 

Everything about efficiency depends on your context and that includes the time, skills, and experience of your development team. The good news is there is almost no limit to the machine efficiency you can theoretically achieve. The bad news is there is almost no limit to the time and skills required to achieve it. Every product we build is thus a compromise between machine and developer productivity. 

Why is most code deliberately inefficient?

For the past 30 years, our industry has been striving to wrap operations inside APIs in order to make them more useful to a wider range of developers. This is the concept of maximizing code reuse and it is not a mistake. It is the world we are operating in and it is the fundamental code efficiency tradeoff.

The principle of code reuse is that someone else writes and tests a tricky library or service and you, together with thousands or millions of others, use it. It might be open or closed source; free or a part of a paid service; part of a language’s standard libraries; or a service hosted in the cloud. They all have a potential penalty in terms of performance and greenness but save you a great deal of time as a developer. 

In order to be widely applicable, these APIs do more than you would need to if you just wrote the most crafted, minimal, and targeted code yourself to accomplish your specific job. Whenever you use an API or call a service, you are leveraging the specialized work of others and simplifying your life and application. However, the generalized code you are calling will be creating more work for the hardware, network, and electricity grid you are running on. 

<Sidebar>The most extreme version of targeted code would be writing assembly language for every different chip you run on, which some people do when they need their code to be very, very fast but are prepared for their developers not to be.</Sidebar> 

Does that mean we should stop using APIs in the form of libraries and services? For most of us the answer is absolutely not. You don’t have the time or skills to craft super efficient custom code. It would slow your development teams down so much your company would go out of business. 

Should we use APIs better, though? Yes. As well as demanding they are as efficient as possible.

Using APIs better means being more thoughtful with a greater awareness of the cost. They are not free. At the code and the architectural level, call outs to libraries or services hit energy use, performance, and hosting bills. Some architectures (such as excessive, poorly designed microservices endlessly calling one another) and code designs (maybe calling an API 10 times in a frequently called function when once would do) create way more work than they need to.

How much API use is enough depends on the skills of your team and how much time it is worth investing in tuning your code. That is something only you know, but there are benefits to being efficient beyond just being greener. Efficient code is faster than inefficient code. In fact, performance can usually be used as a proxy measurement for efficiency. 

<Sidebar>That is not a given in every case. Green, efficient, and performant are not synonyms and we are about to tell you why. (Spoiler alert: as always, it depends on the context and motivation). Nonetheless, using them as proxies for one another is usually a useful rule of thumb.</Sidebar>

The benefits of efficiency  

One end result of code efficiency is it reduces the number and size of machines required to host your services for your target: 

• number of users,

• reliability and performance level.

This can be described as maximizing hardware productivity, and it’s a very green concept. Fewer servers used means less electricity is required to power them and your system embodies less carbon (every machine embodies the carbon emitted during its manufacture and disposal). From a sustainability perspective, less is more.

Back in the olden days of the 1980’s and 1990’s, hardware productivity was important to businesses because machines were slow, expensive to buy, and costly to house in on prem data centers (DCs) - if you could even fit another machine into your DC - so we had no choice but to address the problem. How did we tackle it back then?

• Services were often coded in highly efficient languages like C. That made executables small and minimized the number of CPU cycles per operation.

• Services didn’t exchange too many messages with other machines (or even other processes!) because that was slow and CPU intensive. 

• They didn’t query data on disk all the time, because that was truly glacial.

• Most code was custom written from scratch and tightly targeted. 

Those steps worked. The early services were small and nimble enough to run on the dodgy hardware and networks we had and the early internet was born. 

If we want to be green, can't we just do all that again?

Don’t look back

Unfortunately, it’s not so easy. Those efficient services were extremely slow to develop; specialized; and often struggled to scale the way we need in a modern global world. We can’t return to that simpler past even if we wanted to. 

The environment we have to operate in today is a lot more complex: 

• Machines and networks did as the late great Gordon Moore instructed with his eponymous law and doubled in capacity every eighteen months (ish). They are now at least three orders of magnitude faster than in the 90’s. 

• Third-party services became more innovative, valuable, and eventually crucial. 

• User numbers grew and those users expected more functionality and far speedier evolution. 

At the same time, security threats became scarier and more common. Back then we had no clue what level of attack was coming.

To handle both the new opportunities and the new threats our old goal of machine productivity changed. It was replaced by one that was better placed to leverage the galloping improvements in hardware, software, and networking to securely deliver what customers wanted. 

Our new target became developer productivity.

Developer productivity

The tech industry has used three decades of Moore’s law wins to make developers’ lives easier with better and better APIs and services. For good or bad, we’re not likely to go back on that decision. 

Modern developers have a lot to do and they are expected to deliver it securely and a heck of a lot faster than we used to. Those techies also became more expensive and in higher demand, making developer productivity improvements vital. 

Most of these improvements were achieved using abstraction layers that allowed access to third party functionality and hid code complexity in exchange for additional CPU-cycles. Often lots and lots of CPU cycles. In the 1990’s, that wouldn’t have been possible because the machines were not up to it. However, they now are and that’s what we do with them. We used those increases in hardware productivity to improve our developer productivity.

Modern enterprises love to (or at least aspire to) build new features at a decent rate and rely on good developer productivity to do so. They are not going to relinquish it without a fight. If we advocate ideas for building green software that are not aligned with that goal, we have as much chance of getting them adopted as any of the three of us do of becoming the World Heavyweight Champion. To be clear, that’s not a great deal of chance (even Sara, who could probably beat Sarah and Anne to a pulp).

So, the question this chapter needs to answer is not just how do we make code more efficient but how do we do it without returning tech teams to the slow development speeds, inflexibility, and lack of scale of the last century. If we want to succeed, we need to align building green software with developer productivity. 

But how?

Before we answer that question, let's first look at what we all used to do in the olden days. At heart, that’s how the small number of folk who have no choice but to build efficient code do so today. It hasn’t disappeared as a skillset, but it has become a comparatively niche one. It is usually practiced only for code for which its price must be paid, i.e. code with extremely high performance requirements, like network data planes; or where minimizing electricity and hardware costs is critical to the business case, like the services of public cloud providers; or where it could pay off big time, like open source libraries in heavy use; or where there are extreme hardware limitations, like embedded systems in medical devices.

Today, any ordinary engineer is unlikely to give up the handiness of for-loops for rolling out the loops, even though it could result in fewer instruction jumps for the CPU. The impact on their productivity wouldn’t be worth it for them or their business and they would probably get themselves sacked for trying it. 

Nevertheless, let's take a look at what highly efficient code looks like.

Background: hyper efficient code

There are still teams in the world building highly optimized and fast-running software. In fact, much of that code underpins modern society.

Fundamental to such code is the minimization of processor use. That means running as close to and in sync with the operating system as possible with minimal intermediation. That means compiled code with no runtime environment. 

Languages that facilitate efficiency famously include C, C++, and Rust, but there is still plenty of code out there written in assembly language. The main reason these languages are used is they involve a minimal number of instructions per unit of work done.

For superspeed, in addition some folk use stripped down operating systems, specialized hardware, or code in the kernel. Really zippy products also use latest generation chips, which tend to be the quickest. In reality, however, all of these options have trade offs including: development time; longevity and wide applicability of the resulting code; and access to the specialist skills required. Even the concept of using uninterpreted languages has pros and cons. 

According to ex-Azure networking development expert Jon Berger, “People who write ultimately efficient code have an instruction-level grasp of modern chips. They’re the kind of engineers who understand how L1, L2, and L3 caches interact and how to use them. That stuff can make a big difference but it’s a full time specialist job.” 

Thankfully for developer productivity, it’s a specialist job that most of us don’t need to do.

The goodies

Some examples of highly efficient software include: 

• The TCP/IP stack running on your Android or Apple phone.

• A hypervisor like KVM or Hyper-V or the open-source distro Xen.

• The code underpinning a cloud provider’s virtual infrastructure.

• The standard libraries underpinning a modern language like Rust. 

Compilers are a meta example of efficient code - they work out how to optimize your executable for you. The Rust compiler is an open source example that you can take a look at to learn more about this if you want. Remember that, for almost all of us, a compiler will tune low level stuff like For Loops for performance and efficiency much better than we can. As Jon Berger says, “Don’t second guess the compiler.” 

He also points out that, “Compiler technology is progressing fast. Specialist Python compilers now exist to take that language and compile it to machine code, allowing it to match the performance of C. Compilers enhanced by large language models may transform the entire field of efficient code production.” 

In the next few years (months?) it is possible that large language models (LLMs) will lead to a revolution that will enable an even more human-readable language than Python, say English, to be compiled to machine code that is as efficient as handcrafted assembly. The limiting factor then will be debugging that code, but those LLMs will probably help with that too. Perhaps the days of humans creating efficient code are on their way out. 

However, in the meantime for the kind of use cases listed above where performance is critical, it is still necessary for maintainers to put in the enormous amount of effort required to optimize their code and the resulting efficiency pays off for all of us.

BTW to say this is a specialist job with perhaps a short remaining shelf-life is not gatekeeping. It’s just a tip that there’s a lot of work involved with less pay-off than there used to be. Nonetheless, we believe anyone willing to spend that effort can become a specialist. There are some lovely open source projects where you can learn these skills from experts and contribute to highly efficient code that is intended for world-changing effects. For example, the OCaml and Rust communities are both welcoming. Because that open source code is intended to be used at scale, your efforts there are more likely to have an impact than any changes you make to your own codebase.

<Sidebar>The less good?

Inevitably, efficiency and performance specialists are not always working for the benefit of humankind. For example, their software may only exist to give a handful of individuals some advantage. The textbook example is high frequency trading where speedy software teamed with loads and loads of expensive hardware lets a company respond to public financial information more quickly than the other guy (i.e. you). Such systems are carbon intensive, demonstrating that fast does not always mean green.

High frequency trading is a great illustration of how high performance, code efficiency, and being sustainable are not necessarily identical but, again, in most cases performance is a good metric for efficiency.

</Sidebar>

Performance vs efficiency vs greeness

As we mentioned earlier, efficient code is usually greener and faster than inefficient code but green, efficient, and performant are not always synonyms. 

• Code that is slow can still be green. In fact, it can be greener than more efficiently written code if it is carbon aware by design, i.e. its operation can be delayed to wait for the availability of low carbon electricity.

• Code that runs quickly can be resource intensive. Those high frequency trading applications use huge quantities of custom hardware and electricity to deliver that speed. Similarly, some research supercomputers use massive parallelisation to go faster, and quantum computers look set to use huge quantities of power. Being green is not their goal. 

• Sometimes energy efficiency doesn’t matter to greeness. There are times when too much electricity has been generated and it has to be thrown away before it burns out the grid. This is particularly common with renewable generation, which can be hard to plan for. Using that surplus power inefficiently might still be green because at least it would be being used.

So, is slow code a problem that needs to be fixed? It depends. 

Synchronous calls

Inefficient coding languages aren’t the only potential brake on the performance of software. It might be what your application is talking to that’s holding it back. 

Highly performant software never lets its entire execution get stuck behind a comparatively slow call. That might be a synchronous call to the local OS or, increasingly worse, a call to another process on the same machine; to local hardware e.g a disk; or to something across a network on another machine. It never even thinks about talking to an external service on the web via HTTP. However, the actual climate impact of your application waiting on a synchronous call depends on how it is operated. 

“Hang on a minute! Why should waiting ever be ungreen? Surely, if your application is stopped it isn’t using electricity. The application might be slower than it could be but why should that hurt the environment?”

Unfortunately, even when no application is running on it, a server that’s turned on still draws power (perhaps up to 60%!) In addition, electricity use is not the only way carbon is effectively emitted by software. The embodied carbon from a server’s manufacture has already been released into the atmosphere but we want to get the best return we can from that sacrifice. Therefore, we need every machine not just to last as long as possible but also to live its fullest life. To have it sitting idle, even for a moment, is a waste of that embodied carbon because those moments add up.

So, waiting for synchronous calls can be bad for the carbon efficiency of an application. However, there are several quite different things we can do about that.

1. By far the optimal, though hardest, is via your software’s design. In your code you can use techniques such as multithreading so that whenever one part of your application is waiting on a synchronous call another part can step in and keep making use of the server’s resources.

2. An easier solution is to run on a platform that handles threading for you or at least helps. Python, for example, offers some thread management. Serverless platforms take this off your hands entirely. We’ll talk more about platforms in the next section.

3. However, the easiest solution is operational. It is to run in a multi-tenant environment where all physical resources are shared such that if one application pauses another gets a chance to run. Cloud VMs are like this unless you choose dedicated instances (so don’t) or you can achieve similar effects on-prem by using an orchestrator with a task scheduler like Kubernetes or Hashicorp’s Nomad.

If you had all the time in the world, then for top greeness you would choose option 1 and write a custom, highly multithreaded application in a language like Rust to maximize the use of resources on your machine, but that would be difficult and take ages. How do you think Anne got her white hair? It would be bad for your developer productivity, and because it requires unfashionable skills It would reduce the available talent pool for expanding your team. 

Option 3, using a multi-tenant environment, isn’t perfect but it’s a decent compromise. Multi-tenancy adds isolation overheads for security, which means it’s not great for machine productivity. However, multi-tenant environments are better for developer productivity and the more managed they are the more machine-optimized they are likely to be. The best keep their isolation overhead as low as possible and if you choose well, they will keep getting more and more efficient under your feet without you having to do anything. Modern multi-tenant platforms like serverless WebAssembly (WASM), for example, are trying to minimize the isolation overheads to get the best of both worlds - machine productivity and developer productivity.

Fundamentally, the approach you should take to handling synchronous calls depends on what is important to your business. You are never going to be able to sell your CTO on a maximally green option that involves an inappropriate degree of effort. You may, however, be able to sell them on a good enough operational approach that could even save developer time. 

Multi-tenancy is an example of how machine productivity and developer productivity can be aligned, but it is an illustration of something else as well. Most multi-tenant platforms are intended to operate at, or at least be deployed at, scale. If their maintainers can cause millions of machines to be used more efficiently, that is far more impactful than any individual developer painstakingly improving their own non-scaled code to get another 1% performance improvement. Efficient multi-tenancy must be encouraged.

Efficient code examples

At this point in the book, we would normally give you some code examples. Don’t hold your breath because we aren’t going to. 

The reason we won’t underlines everything we have just said: efficient code is highly context specific. There is no point in us including a sample of machine code optimized for a particular CPU and hardware setup because it would be of no use to pretty much anyone. (Don’t worry, we can give useful examples and case studies in the later chapters).

In most cases, the most efficient code you will ever write is that which follows the best practices of your chosen language or platform. The compiler or runtime that comes with that will then optimize it for you.

If you need to make your application as a whole more efficient then do some performance profiling. Performance profilers are tools designed to help you analyze applications and improve poorly performing, inefficient bits, and there are a number of commercial ones out there. Sometimes you’ll have to instrument your code (basically put timers in there so you can see how long parts take to run) but often not. 

You can use that profiling data to find and fix bottlenecks. Common ones include:

• Expensive library calls (you might need to call the API differently or call a different API or library).

• Poorly implemented loops or algorithms (aka bugs. Test and fix your implementation or use a good library instead).

• Excessive API calls (could they be combined into a smaller number?)

Low level code efficiency beyond sensible performance testing and bug fixing is only a useful goal in a few circumstances. Most enterprise developers shouldn’t be writing their code in C or even its newer equivalents like Rust just to be green. 

So, what should the majority of us be doing?

Pick the right platform

For most of us, being green is more about high level choices than low level ones, and the best place to think about building sustainable software is at the design stage. In particular, it’s imperative you choose the right software platform - i.e. the environment in which your code will be executed together with its surrounding ecosystem of resources. 

Whenever you start building software, you make a platform choice. Maybe it’ll be Java, Ruby on Rails, Amazon Web Services or Google Cloud Serverless. Your decision may be based on cost, popularity, or team familiarity. Given the climate crisis, it is now vital that you also pick a platform that is evolving, and will keep evolving, in a green direction. 

Unfortunately, just looking at a platform’s marketing materials, or even their code, is not enough to tell you that. You need to verify the platform’s other users have similar requirements to you and their needs are likely to evolve in the same direction. That is a better indicator than just looking at what the platform says it will do in future, because that could be marketing spin. Someone will have to hold the platform to their promises, and that means a good-sized, or at least noisy, community of climate-conscious users. 

In other words, you need to become a green software consumer:

• Check that your chosen platform has an existing community of users pushing for them to become ever greener with specific targets by joining that community and listening to what it is saying.

• Follow that platform’s guidelines on how to use it best from a green perspective and ask them about it to demonstrate demand.

Use green design patterns

When it comes to greeness, there are architectural patterns you need to use and others you need to avoid. 

Some design choices mean your software will use more resources than it needs to and, unfortunately, many of those are common. Dodging them requires some thinking in advance. If you just wing it, you may well end up with a carbon inefficient design.

Jon Berger says, “You have to understand your performance. Where are you choosing to invest your CPU? Design for performance and then test to find out what’s actually happening.”

Avoid too many layers

API layers are great. They simplify our lives and let us make use of all those fancy third-party tools. Unfortunately, they also add processing to every operation that passes across those boundaries. Avoiding unnecessarily CPU intensive architectures is key to sustainability and one common issue is a wasteful use of too many layers. Are you duplicating anywhere?

For example, it is important that you are not doubling up on the work done by your platform. Make sure you are using it as it was intended (you should have already checked those intentions were green).

Be mindful with microservices

Microservices underpin modern architectural best practice because they allow you to make use of tools and services and promote a better division of labor within your teams (arguably humanity’s most effective tool of all). However, they can be done well or badly and a dodgy microservice design is particularly CPU intensive. 

One of the eco risks with microservices is the high level of traffic between them, which is often transmitted in a multi-layered, CPU-intensive fashion. 

<Sidebar>Another issue is the potential platform overhead associated with each microservice.</Sidebar>

Communications inside single services tend to require little in the way of message wrapping and unwrapping. They are fast and lightweight and don’t burn much electricity.  Therefore, you can pass around internal messages without worrying too much about sustainability. However, once you split your system out into many services, perhaps running on different machines, then things get a lot more CPU-heavy.

Note that when it comes to networking, speed of transmission and energy use are usually correlated, so monitoring network performance can give you a useful rule of thumb for measuring climate impact. Some context:

• It might take 100 times longer (and more energy) to send a message from one machine to another than it does to just pass a message internally from one component to another.

• Many services use text-based RESTful, or even SOAP, messages to communicate. RESTful messages are cross-platform and easy to use, read and debug but slow to transmit and receive. In contrast, Remote Procedure Call (RPC) messages, paired with binary message protocols are not human readable and are therefore harder to debug and use but require less energy to transmit and receive. It’s around 20 times faster to send a message via an RPC method, of which a popular example is gRPC, than it is to send RESTful messages.

In terms of energy use, you could therefore mitigate a microservice approach by:

• Trying to send fewer and larger messages.

• Sending messages more efficiently using RPC rather than json-based communication.

HOWEVER (and that however is in capitals for a reason) using stuff like RPC and binary encoding is much harder. Using those techniques will impact developer productivity, which means it is often not the right idea for your team. Sending fewer messages, for example, might damage the division of labor approach you have taken to ownership of your services. RPC might limit your access to great third party services.

If the extra work will put you out of business or that added efficiency will never pay off at your scale, you need an alternative approach:

• Do not wing it. Carefully plan your microservice architecture and the calls it needs to make and don’t be excessive. A well designed system will be more efficient and less wasteful. Try reading O’Reilly’s excellent “Building Microservices” by Sam Newman and follow his advice. Dig out that most useful of objects, a whiteboard, and discuss your ideas.

• Alternatively, choose a platform that is committed to optimizing communications for performance and use it how it was intended (you’ll still need to do some reading, but you are demonstrably a reader so that’s fine).

The problem with service meshes

Unfortunately, one of the most useful services to use as part of a microservice architecture is also the most potentially energy-wasteful way of sending messages. Service meshes can be a big problem. They can add a lot of additional processing to every message, particularly if they use a so-called sidecar approach which adds a paired communications service (i.e. an extra communication step) to every single one of your microservices. 

If you want to use a service mesh, your best option is to select one whose community is committed to extremely high performance because, as we mentioned above, performance and energy use are correlated for service meshes. 

The monolithic question

If you are wondering if this all means that monoliths are greener than microservices the answer is yes and no. As with all aspects of sustainability, there are tradeoffs. Monoliths generate less expensive inter-service traffic and can be designed to be super-efficient. However, as well as their other drawbacks, they are often more difficult to operate in a green way, which we will discuss in the next chapter. 

<Sidebar>In 2023, Amazon provided an excellent example of the tradeoffs of efficient design. It came to light that the company hadn’t chosen to optimize one aspect of its Prime Video service until it decided to monitor every real-time stream rather than a few sample streams. At that point, they rejigged their serverless architecture to involve larger chunks of code. In doing so, Amazon made a strategic choice that backs up our statement above: more carefully designed architectures made up of bigger chunks of code are more efficient, but those systems are expensive to build and harder to iterate on. 

Like Amazon, you may not want to invest the money until you have established enough demand and you understand that demand. Then absolutely do, because at that point commercials and greenness are aligned.

The irony of the fuss around the Amazon story is that, in the future, serverless platforms (and there are many) could be the way most enterprises get their hands on an efficient-enough commodity coding platform. We hope serverless platforms get there because that’s the holy grail of green software and why serverless comes up so often in green code discussions. 

</sidebar>

For sustainability there is seldom an obvious answer. There are pros and cons to everything, and you need to ponder the potential downsides of your chosen approach and mitigate them.

Replace inefficient services and libraries

Even if it is a bit late to replace your entire development platform, it is never too late to swap out libraries or services for more efficient or lightweight ones. Again, this is more of a shopping job than a code-writing one and probably starts with measurement (which we cover in chapter 9).

Fundamentally, use performance profiling. Are you calling a service or library that is slow and CPU-intensive? Check if there is an off-the-shelf alternative with a green-aligned userbase that you can upgrade to.

This advice might sound trite or obvious, but it is incredibly important. The revolution in our industry in the past decade has been the ready availability of high quality libraries and services. We need the smart teams and maintainers behind them to turn their attention to being as carbon efficient and carbon aware as possible. That will make a huge difference to the world, but they will only do it if users ask for it. 

The good news is that asking for stuff is easy. So much less stressful than re-writing everything in C, believe us.

Don’t do or save too much

Being green is highly aligned with a Lean software mindset - an approach that emphasizes the continuous delivery of high-quality software while minimizing waste. One of the aims of lean is to eliminate wasted time, effort, resources, and features that do not provide value to the customer. Don’t implement features or save data that you don’t have to yet. 

<Sidebar> This doesn’t include thinking. You can use a whiteboard and think as much as you want but don’t build stuff in advance of need.</sidebar>

Being lean is the very essence of building green software. The most efficient code is no code at all. For now, everything we do that generates carbon must be balanced against the value people get from it. The easiest way to cut carbon is to remove functionality that is seldom used and delete data that is hardly ever queried or, even better, never add it to start with. 

Don’t add features ‘just in case’. As well as being ungreen, the resulting under-maintained code is often the source of security holes. Delete it or don’t add it in the first place. Again, the easiest time to eliminate waste is as early as possible, so make sure your product management team is on board with your green software mission.

The most subtle way for your system to creep on its energy use is via excessive stored data. In a heating world, data storage policies must become frugal. Databases should be optimized (data stored is minimized, queries are tuned). By default, data should auto-delete after a period, and if you do have to keep it for a long time (perhaps for regulatory reasons) use cheap and secure long term storage options like tape. That consumes a fraction of the power of easy access locations like SSDs. 

It’s painful to tidy up but you’ll feel better for it and the planet will thank you. Excess data is a subtle hit on every interaction. It makes queries take longer and use more power. Delete it. If you can’t bear that, move it to tape.

Leverage client devices

As we discussed in the introduction, the embodied carbon in user devices like smartphones is huge (the carbon cost of manufacturing and destroying them dwarfs the carbon emitted as a result of powering them over their entire lives). So, we need to use them to their fullest and make them last as long as possible. Anything we can do to get them to do more, and the servers in data centers to do less, helps justify the emissions associated with the creation of those carbon-intense gadgets. 

In addition, device batteries can play a part in balancing grids and supporting low carbon power because the electricity they have saved is easy to demand shift. Phones can be, and often are, charged using cheaper green power. It’s a tiny amount of carbon in the grand scheme of things but it puts us in the right mindset. 

As well as being greener, pushing intelligence and self-reliance to devices makes systems more resilient to network issues, which many experts fear will be increasingly associated with a more unstable climate.

Manage machine learning

Depending on where you are in the machine learning life cycle the same tricks apply as for other software we have mentioned. Two key areas are a little different though and deserve a special mention: data collection and training.

As we build larger AI models, they need larger datasets. Large data sets are attractive for several reasons. We want to prevent overfitting and capture the input data fairly in order to ensure our model doesn't suffer from reduced accuracy as a side effect of skewed data. Large data sets can also later prove themselves to be valuable in scenarios we did not consider in the initial planning phase. The problem is, that kind of thinking has lean waste written all over it.

Unfortunately, sustainability research and public thought in this area are still lacking, especially considering how much ML we are all doing right now. However, fear not. As we will discuss in chapter 8, there are tools you can add to your toolbelt for greener data collection. Open source and already collected data sets also exist out there for you to take advantage of. Yes, the storage cost will be the same but you won’t have to gather the actual data yourself.

The next big topic is ML training. Here, research shows that reducing model size and training time can be of significant help in making your training greener. Train ML models faster and more efficiently by shrinking the model size using techniques such as federated learning, pruning, compression, distillation, or quantization. 

For certain scenarios, there are also already pre-trained models which might serve your use case well like open source models for natural languages - after all, natural languages don’t change THAT quickly. 

The growth in Edge computing and IoT means we are seeing more and more devices with limited capabilities, and smaller models will also be the way to go here. Another perk of Edge computing is reducing energy consumption by providing local computing and storage for data. 

Lastly and most importantly, ML training has the great benefit of very rarely being urgent, so make sure to make your training carbon-aware. Never train using carbon intensive electricity. Wait for the clean stuff.

<Sidebar>Case Study: Microsoft Teams and the 2020 Pandemic

In 2021, Anne was interested in how the internet had survived the sudden increase in demand caused by lockdowns and did some research on the subject (which is another book in itself. Just let her get this one done first. However, we’ll cover some of this in Chapter 7 - Networking). 

In short, the reason the tech response to the Pandemic was fascinating is it provided real world examples not only of demand shifting and shaping, which we’ll discuss in chapter 7, but also of how to improve code efficiency. 

One lockdown response very relevant to this chapter was the code-focussed one from the team supporting Microsoft’s video conference product: Teams. 

It’s almost a cliché that you should never prematurely optimize and, in terms of developer productivity, we agree. Following that approach, however, usually means you’ll have underutilized capacity in your systems. During the Covid pandemic of 2020, MS Teams was faced with unprecedented demand and no availability of more machines to host on. To keep the show on the road, they were therefore forced to exploit the slack in their existing systems.

The Teams team had no option but to wring more machine performance out of their architecture. I.e. they needed to improve its efficiency. For example, they switched from text-based to binary-encoded data formats in their caches, which reduced both network traffic and storage requirements. 

As is often the case, this greater efficiency added complexity to their systems, which made them less reliable, and the team was then forced to address this by implementing a lot more testing and monitoring.

What do we learn here? The pandemic gave Microsoft no alternative but to be more efficient, which will help them meet their green hosting targets. Hurray! However, the lesson is also that even for Microsoft, a company that has huge scale, a long term view on stuff, and very deep pockets, efficiency improvements were apparently not a no-brainer. They had costs in developer time, which is presumably why they hadn’t made them already.  

This true tale underlines the fact that efficiency is great, but expensive. Most folk only do it when they have to. I.e. when the balance of risk changes.</Sidebar>

The big problem with efficiency

“Why didn’t they ask Jevons?”

Before we leave this chapter, we have to mention the English 19th century economist William Stanley Jevons because the first issue that always crops up whenever you discuss efficiency is Jevons paradox. The second is the drag of efficiency on progress.

We need efficiency and therefore genuine worries about the concept have to be addressed. These two concerns don’t just apply to code efficiency but we might as well talk about them here. 

Issue 1: Jevons paradox

The Jevons paradox is William Jevons’ historical observation that when we get more efficient at doing something, it gets cheaper and in the long run we end up doing much more of it. The argument is thus that if we improve the energy efficiency of data centers we’ll want more of them and the result will be we consume a greater amount of energy than when we started.

Issue 2: The productivity hit

The second concern, that efficiency is a drag on progress, is based on another historical observation: that freely abundant energy has been vital to the progress of humanity so far and will probably remain so. Using it frugally adds work to, and therefore slows down, every human endeavor and will hold us back in solving the very problem we need to fix - climate change. 

These two objections are not merely crackpot ravings. They have genuine merit and are something we have thought about long and hard. They deserve a book in themselves. 

• Basically, Jevons paradox is a way of saying that efficiency ultimately drives abundance. 

• The drag on productivity argument, however, appears to imply that efficiency is bad for abundance. 

We could just tell you these statements are mutually contradictory and so we should ignore them and leave it there, but that would be a facile bit of avoidance. We should be roundly told off for it. The reality is they are not contradictory. They are both true, but address different contexts and timescales. 

Jevons Paradox is telling us that getting better (in this case, more efficient) at something can be very positive for the adoption of that thing. We agree.

The impact on progress observation is saying that high efficiency is really hard. It takes a long time and tons of investment to achieve. Again, we agree. In fact, high efficiency may never be commercially viable for small, or even average-sized, businesses, and it is usually only worthwhile for goods or services for which there is loads of untapped demand. 

However, I refer back to Jevons: if you can invest the time and effort and the demand is there, then efficiency should pay off big time. The paradox of Jevons Paradox is that it isn’t a paradox at all. It’s a statement of the obvious and it doesn’t conflict with our main point in this chapter - gaining efficiency is costly. There are always benefits, but they are not sufficient in every case to outweigh the costs.

So is efficiency good or bad for green?

In the context of humanity and climate change, in the long run we want to have abundant energy at all times. It is probably fundamental to humanity’s continued progress (or even existence in its current form). We just don’t want to get that energy from where we do now - burning fossil fuels. 

Once the energy transition from high-carbon energy sources to low-carbon ones is complete, the aim is that energy abundance is once more the result. The energy might be from modular nuclear, space-based solar power, renewables paired with super grids or new forms of battery, something we haven’t even thought of yet, or all of the above. At that point, efficiency will merely be a nice to have. Fill yer boots. 

“Hurray! We don’t need to read this book after all. That was efficient.”

Hold your horses! Unfortunately, we’re not there yet. We’re at the start of the energy transition, not the end. 

The reality is we have been in this transitory state for decades and we’ll be in it for many, many decades more, and throughout that period we’ll be ramping up these new forms of power. However, they will not yet have had time to become efficient commodities themselves. Until they do, we’ll need to meet them halfway by being more efficient users. 

In the real world, the downside of efficiency is it requires a lot of investment. The upside is, as Jevons pointed out, in the long term and the right circumstances that investment will more than pay off. Fortunately, if it’s unlikely to ever pay off for you there are ways to get your suppliers to make that investment instead - mostly by demanding it.

“But Jevons says if we get more efficient at using fossil fuel we’ll use more of it!” 

And historically he has often been proved right. However, this chapter isn’t about being efficient at using fossil fuels. It’s about making tech more efficient and context-aware at using electricity that comes mostly from renewables. 

Underneath our feet, the world will move from generating electricity with bad, but already efficient, coal and gas to increasingly-efficient solar, wind, and nuclear. At the same time, batteries and other forms of storage will get better. In the short term, we just need to give humanity a hand to make this shift by asking for less to do the same job. We also need to be ready to make the best use of cheap but variable wind and solar.

In twenty years time, when the grid is clean, we will probably find that as a result of all the great work we did to get better at using electricity we use way more of it. We contend that that will be success, not failure - we are unashamed tech utopians. 

The lowdown

The conclusion on code efficiency is that it's not obvious you should be investing in it directly (beyond performance monitoring and bug fixing) unless you are developing high performance or high scale code or a software platform. In that case, you absolutely must be and, hopefully, you already are and it will pay off for you. However, those engineers are not in the majority. 

Code efficiency has pros.

• It reduces the amount of energy required to execute a program or run a system. So, lower carbon emissions and energy bills.

• For devices that rely on batteries, such as smartphones or laptops, efficient code extends battery life and reduces the electricity needed to recharge them.

• It produces code that can run on fewer machines and older, less powerful kit - minimizing embodied carbon and e-waste.

On the other hand:

• Writing efficient code is hard and takes ages. That increases development time and cost, which lowers developer productivity. It may never be worthwhile for products without enough scale or even for early stage scaled products where you are trying to avoid premature optimization.

• Efficient code is difficult to maintain, requiring specialized knowledge and making it harder to evolve the code in the future.

Whether writing efficient code is worth it for you depends on your circumstances, but as a developer it isn’t your only option to cut carbon emissions from your software.

• You can be lean and frugal in the functionality you add and the data you save - and delete stuff that isn’t used.

• You can leverage the power of code reuse by choosing efficient software platforms that are aligned with a green future and state your preference as a user for them to keep evolving fast in that direction. Then use them as they were designed to be used. The hypercloud services have done better than most but they need to be held to their commitments and made more accountable. The open source community needs to get in the game. In our opinion, this is the long term solution for wide adoption of code efficiency.

• You can make design choices that reduce waste. For example, following best practice for your use case in your microservice design, or not exchanging an excessive number of small inter-service messages over a CPU-intensive network layer like a heavyweight (slow) service mesh. 

• You can write code that works well in efficient multi-tenant environments.

• You can use performance profiling to identify egregious bugs or badly behaved libraries in your system.

• You can choose or switch to more efficient services and libraries.

• You can design for richer clients running on user devices such as phones.

The good news is that as a DevOps engineer or SRE, you have even more options for cutting carbon emissions that are aligned with devops productivity. We’ll cover those in the next chapter - Operational Efficiency - so hang on to your baseball caps/beanies/bucket hats (delete as age appropriate).

Back to the index