Price is a function of supply and demand. Increase demand and price goes up. Increase supply and price goes down.

Bitcoin has a fixed supply of 21 million coins forever. So we don’t need to worry about supply, it is 100% predictable and limited to 21,000,000 BTC in the year 2140.

With fixed supply, the investment case for Bitcoin is simple: Will there be more demand for Bitcoin in the future?

If yes, the price will go up. If not, the price will go down. The extent of future demand for BTC controls the exact degree of price movement.

So, what are some key demand drivers for Bitcoin?

Here’s a quick overview of the demand drivers and my estimated annual $ volume:

Let’s dive into these points one by one. At the end of this article, I’ll give you my estimation of what this will mean for the BTC price (this will blow your mind )!

1. Nation States’ Adoption

How it Drives Demand: As countries face economic uncertainties, some are turning to Bitcoin as a strategic reserve. By holding Bitcoin on their balance sheets, nations can hedge against currency devaluation and global economic downturns.

Guess who’s the biggest holder of Bitcoin among all nation states?

The United States of America! (source)

But there are many other nation states that have a large incentive to accumulate and hold Bitcoin quickly. The game theory may drive more nations into Bitcoin — and quicker than you expect!

The inception of Bitcoin was driven by the need for a decentralized currency, free from the control of central banks, especially in the wake of financial crises that have historically plagued various nations.

Bitcoin, built on a peer-to-peer network, offers a solution to countries with weak currencies or high inflation rates, serving as a hedge against currency devaluation. Its decentralized nature also shields it from government censorship or interference.

Countries like El Salvador and the Central African Republic have recognized Bitcoin’s potential, adopting it as official legal tender. El Salvador’s experience post-adoption showcases the tangible benefits, with significant growth in tourism, remittance savings, and a surge in popularity.

Currently, nation states hold roughly $11 billion in Bitcoin (source):

But how much money could flow into Bitcoin? What is the TAM? Here’s a chart of the consolidated balance sheet assets of the Eurosystem:

If 1% of the $8,000 billion balance sheet of the Eurosystem consolidated balance sheets would flow into Bitcoin each year, the annual dollar demand would be $80 billion for the Eurozone alone.

However, Europe makes only a small portion of the overall nation state reserves as can be seen in this graphic (source):

Bitcoin demand driver: So a conservative estimation of the annual dollar volume that could easily be flowing into Bitcoin only by nation state treasuries would be as follows.

Estimated Annual Dollar Volume: $50 billion – $100 billion.

2. Corporate Adoption

How it Drives Demand: Companies, from tech giants to small startups, diversify their assets by investing in Bitcoin, which can act as a hedge against inflation and showcase a forward-thinking approach.

Currently, public and private companies hold already $17 billion in Bitcoin (source):

Here are a few examples (non-exhaustive list) of public companies holding Bitcoin on their balance sheets (source):

As Bitcoin is already one of the largest currencies by market cap (source) and the only currency with limited supply (=21 million BTC), companies worldwide may decide to allocate a fair portion of their currency holdings to Bitcoin.

13 US companies hoard $1 trillion in cash (Google, Apple, Amazon, Tesla, Microsoft). A sensible strategy for these cash holdings is to invest a portion, e.g., 10%, into the hardest currency, Bitcoin, that cannot be inflated away. Investing 10% or even only 2% into Bitcoin would contain volatility while injecting a better-than-treasury risk/return ratio, as determined in many financial research studies.

For example:

5% of 1 trillion USD is $50 billion dollars and we’re talking only about 13 US companies’ cash positions! So we may easily see a $20 to $40 billion annual USD demand for Bitcoin from corporate investors alone (public and private companies).

Estimated Annual Dollar Volume: $20 billion – $40 billion.

3. Individual Investment Strategy

How it Drives Demand: The average person is becoming more crypto-savvy. By dollar-cost-averaging into Bitcoin, individuals are viewing it as a long-term investment, similar to stocks or real estate.

There are already more Bitcoin Hodlers than Spanish citizens. It’s a medium-sized country that grows quicker than any other nation state!

What do these people do? They accumulate Bitcoin, month after month after month, and never stop. This drives annual demand.

For example, say you have 100 million people buying only $100 of Bitcoin every single month, on average, you’ll get an annual USD demand for Bitcoin by individual hodlers of $10 billion USD.

But the average is always skewed up in financial matters, because a small percentage of people hold a big percentage of assets, the average buy pressure may be much higher for 100 million individual hodlers.

Estimated Annual Dollar Volume: $10 billion – $30 billion.

4. AI and Autonomous Agents

How it Drives Demand: The rise of AI and autonomous agents using Bitcoin for transactions showcases the digital currency’s versatility. These agents require a permissionless system to operate efficiently.

Soon an infinite number of intelligent agents based on LLMs and other AI technologies will start to acquire the scarcest good on Earth, which makes it even more scarce for ordinary people like you and me.

Recommended: The Scarcest Resource on Earth

Bitcoin is money over IP, it is Internet-native money that can be accessed without permission. A machine cannot open a bank account but they can create a Bitcoin account — or hundreds at the same time — and start accumulating monetary energy.

With the rapid adoption of autonomous agents such as BabyAGI and Auto-GPT, there will be billions of profit-oriented AIs soon that do nothing else but accumulate and hold the most Internet-native scarce good, that is, Bitcoin.

This recent Ark invest video talks about Bitcoin’s role for AI agents:

With 100 million autonomous Bitcoin agents working for 24 hours 365 days per year, we can expect an average income of $365 per year or a dollar a day (conservative!). All of this will flow into Bitcoin!

Estimated Annual Dollar Volume: $10 billion – $30 billion. Or much more.

5. Bitcoin ETFs

How it Drives Demand: ETFs simplify Bitcoin investment for traditional investors. By buying into an ETF, investors indirectly own Bitcoin without managing a wallet.

This is the historical development of assets under management of the ETF industry globally (source):

If we estimate that only 0.5% of the AUM will flow into Bitcoin annually (there’s a lot of internal and external growth so this is extremely conservative), we’d get a $50 billion annual USD demand for Bitcoin.

Let’s cut this by half to stay super conservative:

Estimated Annual Dollar Volume: $15 billion – $25 billion.

6. Remittances and Cross-border Transactions

How it Drives Demand: Bitcoin offers a cheaper and faster solution for international money transfers, especially in countries with expensive or slow banking processes.

We already discussed this in the “nation state adoption” point but we didn’t count it towards the Bitcoin demand there.

Worldbank: “This edition of the Brief also revises upwards 2022’s growth in remittance flows to 8%, reaching $647 billion.” (source)

“Globally, sending remittances costs an average of 6.25 percent of the amount sent.” (source)

As already proven by the country El Salvador, Bitcoin is the easy fix the Worldbank is looking for. The Bitcoin Lightning network can solve the remittance problem in the world, instant and free payments without intermediaries, saving $40 billion annually or more.

We assume that of the $600 billion of annual remittance payments, $5 to $10 will flow into the superior solution Bitcoin. Again, we err’ on the conservative side.

Estimated Annual Dollar Volume: $5 billion – $10 billion.

7. Hedge Against Inflation

How it Drives Demand: In countries with hyperinflation, Bitcoin is a refuge. It offers a stable alternative to rapidly devaluing local currencies.

But inflation is a fact in almost every economy — most people would agree that the major source is monetary debasement, i.e., more dollars are created which results in higher prices of non-dollar assets and goods.

Here’s the chart of recent inflation numbers globally (as these are official, they are a conservative proxy for real inflation):

Thus, if the demand for Bitcoin stays the same, the monetary units (e.g., USD, EUR, CNY) flowing into Bitcoin will increase by ~5% annually. For a $500 billion asset that is Bitcoin, this yields an annual demand increase of $25 billion.

Note that this figure doesn’t include the additional monetary units coming from people who actually want to hedge against inflation by buying Bitcoin. This is just to account for the monetary debasement of the money already flowing into Bitcoin.

The real annual demand is likely to be much higher.

Estimated Annual Dollar Volume: $15 billion – $25 billion.

8. Financial Inclusion

How it Drives Demand: For the billions without access to traditional banking, Bitcoin offers financial services, from saving to borrowing.

As the billions of unbanked people are usually poor, the annual dollar volume from these people won’t be much.

According to the World Bank Group, as of 2017, 31% of the world’s adult population, or approximately 1.7 billion people, were unbanked (source: World Bank Group). McKinsey & Company estimates that as of 2019, 2.5 billion of the world’s adults do not use formal banks or semiformal microfinance institutions to save or borrow money (source: McKinsey & Company).

According to the World Bank Group, the bottom 20% of the world’s population had an average income of $1,298 in 2017 (source: World Bank Group).

Although the numbers are unimpressive, let’s assume that only $1 to $3 per year goes into Bitcoin. I don’t know how I can make it more conservative than that given that Bitcoin may be the only option for those people to participate in the global financial system — and Bitcoin is inclusive and doesn’t reject them like banks do.

Estimated Annual Dollar Volume: $2 billion – $10 billion.

9. Speculation and Trading

How it Drives Demand: Active traders buy and sell Bitcoin daily, hoping to profit from its volatility. This trading volume significantly contributes to its demand.

The monthly trading volume is roughly $400 billion for Bitcoin (~$5,000 billion annually). Let’s assume the Bitcoin trading demand grows by 1% per year and the growth rate gradually declines. Assuming a $50 billion annual new flow into Bitcoin just for trading purposes doesn’t seem unrealistic to me at all, that’s only 1% of the annual trading volume.

Estimated Annual Dollar Volume: $20 billion – $50 billion.

10. Decentralized Finance (DeFi) Platforms

How it Drives Demand: Bitcoin can be integrated into DeFi platforms, allowing for lending and borrowing.

Business Research Company: “The global lending market size grew from $7887.89 billion in 2022 to $8682.26 billion in 2023 at a compound annual growth rate (CAGR) of 10.1%.” (source)

The CAGR of 10% implies a first-year cash flow (demand) into the decentralized finance market of $800 billion per year.

Many platforms for borrowing and lending using Bitcoin exist but I don’t believe it’ll be a big market share from the $800 billion of new capital, probably only a tiny portion like $1 billion to $5 billion will flow into Bitcoin. This is because you need asset collateral first before you can borrow against them. Most people don’t have significant BTC assets though.

This may change over time but let’s stay hyper conservative.

Estimated Annual Dollar Volume: $1 billion – $5 billion.

11. Retail and Merchant Adoption

How it Drives Demand: As more merchants accept Bitcoin, its utility as a currency grows, driving both consumer and business demand.

The lightning network (=Bitcoin’s layer 2 cheap and fast payment solution) grows significantly but is still small (source):

The total addressable market (TAM) of payments is huge but let’s keep it super conservative. This overestimates the cash demand in the short term but significantly underestimates it in the long term:

Estimated Annual Dollar Volume: $1 billion

12. Institutional Investment Products

How it Drives Demand: Beyond ETFs, products like futures and mutual funds centered around Bitcoin attract institutional investors.

Business Research Company: “The market size of global investments market is expected to grow to $5193.94 billion in 2027 at a CAGR of 7.9%.” (source)

There will be some demand for Bitcoin derivatives such as futures or short products and mutual funds concentrating on Bitcoin-related industries such as mining. Let’s assume the new money flowing into these products is less than the CAGR so it remains meaningless in the big scheme of things. Again to stay conservative.

Estimated Annual Dollar Volume: $10 billion – $20 billion.

13. Network Effects and Education

How it Drives Demand: The more people use Bitcoin, the more valuable and accepted it becomes, creating a positive feedback loop.

For instance, the more developers, educators, researchers, and investors contribute to Bitcoin, the stronger the network becomes creating a virtuous loop. A classic example of a network effect that is hard to beat.

You could argue this will be the mother of all demand drivers for BTC but let’s stay conservative and assign a small number to it:

Estimated Annual Dollar Volume: $5 billion – $10 billion.

If you want my detailed view on Bitcoin network effects, check out the following blog tutorial:

Recommended: Want Exploding Prices North of $500,000 per BTC? “Grow N” Says Metcalfe’s Law

Summary and Bitcoin Price Derivation

So let’s recap the annual dollar demand moving into Bitcoin based on the analysis in this article:

Based on this analysis, we anticipate an annual USD inflow of $164 billion to $355 billion into Bitcoin. While there will be outflows, the demand drivers are expected to expand over time rather than diminish. For example, as nation states continue to print more currency, their acquisition of Bitcoin will likely intensify. But for the sake of argument, let’s assume that half of the projected inflow is withdrawn from the Bitcoin market each year. This would result in an approximate annual net positive demand of $80 billion to $175 billion for Bitcoin.

Consider a scenario where the net demand for Bitcoin is $100 billion, and its market cap stands at $500 billion (as of this writing). If the market cap remained constant for five years, the cumulative net demand would have absorbed all available Bitcoin by the end of the fifth year. If the USD demand remains consistent or increases, the only logical outcome would be a rise in the market capitalization, translating to an increase in Bitcoin’s price.

With a consistent $100 billion net demand increasing annually, Bitcoin’s market cap would likely approach $10 trillion USD rather than just $1 trillion USD. Otherwise, we’d encounter the same supply issue.

This market cap would continue to grow indefinitely in response to unceasing demand. Thus, Bitcoin’s price trajectory is upward, albeit with expected fluctuations.

At a market cap of $10 trillion, the additional annual demand of $100 billion could be theoretically accommodated, as it would take a century for this demand to absorb all the Bitcoin. However, if an increasing number of individuals, institutions, and nation states adopt a long-term holding strategy (HODL), the market cap would need to rise even further.

A $10 trillion market cap for Bitcoin would correlate with a price of $500,000 per Bitcoin. This aligns with my recent analysis using Metcalfe’s Law and is also comparable to the gold market cap (approximately $12 trillion USD or roughly $600k per BTC). This suggests a convergence of multiple influencing factors.

Recommended: Want Exploding Prices North of $500,000 per BTC? “Grow N” Says Metcalfe’s Law

The post 13 Insane Bitcoin Demand Drivers That Force the Price Up appeared first on Be on the Right Side of Change.

The Blender Foundation have just announced their “Big Projects” list for Blender in 2021. It is hard to argue that 2020 wasn’t a banner year for Blender development, with three major releases as well as the first ever LTS release. Through 2020 we saw improvements to the Blender UI/UX, sculpting tools, modeling, EEVEE, Cycles and so much more. We also saw a record number of massive companies coming on board the Blender development fund. With the release today of the projects list, we get insight into the Blender priorities in 2021, including priorities such as:

• launch of a new open movie called Sprite Fight
• the everything nodes project, where everything in Blender will be able to be driven procedurally using nodes (see Geometry Nodes in action here)
• all new Asset Browsers editor window for better content management
• massive improvements to the VSE or Video Sequence Editor
• EEVEE real-time rendering improvements including Vulkan support, motion blur, depth of field and possibly raytracing
• VR improvements including the ability to use VR controllers and author content in virtual reality
• Cycles rendering improvements especially related to perfromance
• Animation 22 (previously Animation 2020), an effort to improve animation tools in Blender, sponsored by AWS
• improved pipeline and USD support, Pixar’s open interchange format

You can learn more about the Blender’s accomplishments in 2020, as well as the new projects in 2021 in the video below.

The Monster Hunter franchise has never been hotter, and it’s easy to see why. After a steady string of releases over the last two decades, developer Capcom slowly refined and expanded upon the winning formula, giving fans bigger and better games as the years progressed. Recent outings like Monster Hunter World and Monster Hunter Rise have been huge critical and sales successes, and now it’s time for something new.

Now, MH creator Capcom is teaming up with mobile developer Niantic (Pokémon Go, Pikmin Bloom, Ingress) in the hopes of expanding the appeal and audience of Monster Hunter, and who better than the creators of Pokémon Go to realize that goal? Monster Hunter might not quite have the prolific library of monsters that Pokémon does, but it has a sizable and dedicated audience already.

To learn more about the game, we spoke to Kei Kawai, who works as executive producer of Monster Hunter Now and as Niantic chief product officer. Bringing the world of Monster Hunter to mobile is a difficult task, so we wanted to hear more about the challenges of adapting a game with so many elements and so much nuance to a mobile device.

One interesting element is learning that Niantic is the one that approached Capcom in the first place. Speaking on the collaboration, Kawai says that “I pitched the idea for the game to the team at Capcom, and they agreed to the idea in that first meeting.” Given Niantic’s previous success with Pokémon Go, it’s little surprise Capcom saw potential in the idea, and according to Kawai, Capcom had “already been considering a location-based game” anyway.

If you’re an existing Pokémon Go fan, you might wonder why you should check out Monster Hunter Now. While Kawai does explain that “different IPs appeal to different audiences” they also maintain that Niantic and Capcom have “worked hard to give players a very different kind of experience.” What does that entail exactly? Well, according to Kawai, Monster Hunter Now is “a new game built from the ground up” while the main shift is with “real time battles and a lot of action.”

As a Monster Hunter fan for some years, I’m interested in how the notoriously complex battle system translates to mobile, and Niantic and Capcom have clearly made this a top priority. When asked about transferring the gameplay, Kawai explains that Monster Hunter Now has been in development for “over four years,” a lot of which was “trial and error.” According to Kawai, early prototypes had matches lasting anywhere between “five and ten minutes,” though playtesting eventually led to the current game’s maximum of 75 seconds. I’ve spent multiple hours tackling beasts on consoles, but I certainly don’t think I want to stand in the street for forty minutes in the hopes of slaying a Rathian.

Instead, the focus is on making “a game that remains true to Monster Hunter” but is still accessible to the mobile audience and experience, with controls that are “easy to start and hard to master.” One essential element of this is the one-finger gameplay that replaces the regular weapon attacks. However, Niantic and Capcom are adding nuance to this mode with “innovations from the team like the gyro for the projectile weapons.”

The community is one element of Monster Hunter that fans adore, and this seems to be another priority for the Monster Hunter Now team. As Kawai explains, the existing focus on multiplayer and the existing community are “what makes Monster Hunter such a great fit for a Niantic game.” In Now, up to four players can “match up quickly with people around you,” and you can also work alongside strangers. This collaborative gameplay is apparently “key to defeating the higher-level monsters.”

One recurring element of Monster Hunter that I personally adore is the pattern of flagship monsters representing a game and its cover art. Recently Monster Hunter Rise featured the fearsome Magnamalo, and I couldn’t help but ask if Monster Hunter Now might get its own monster down the line. Kawai gives a suitably careful answer, saying “This is really a question for our friends at Capcom,” though apparently the launch of Monster Hunter Now is just the start, and the developer would “love to include new unique monsters in the game in the future.”

Monster Hunter Now has been in beta testing for a couple of months now, and while we have also been playing a pre-release build, we are waiting to give a full review near launch. According to Kawai, this testing has been crucial, and the Monster Hunter Now team is still making “final touches” to the game, including “fine-tuning the speed of progress and the strengths of the different monsters and weapons.” Much like other Monster Hunter games, we anticipate the community is going to be crucial in creating a fair and satisfying game.

While Pokémon Go is over seven years old, it’s certainly interesting that Monster Hunter is seeing now as the perfect time to launch this mobile title. However, after the success of Rise, World, and even the release of a Monster Hunter movie, Niantic would like to use this opportunity to help Monster Hunter “reach a broader and truly global audience of smartphone users and mobile game players.”

Now is sure to be many players’ first introduction to the series, and as Kawai mentions, Niantic hopes that the title will “help expand the franchise and bring new players into the world.” After all, Go is partly responsible for a huge resurgence in Pokémon’s popularity. If you’re at all worried about the team behind this, Kawai assures fans that both series producer Ryozo Tsujimoto and “all the top Monster Hunter producers at Capcom” are giving Niantic “valuable feedback throughout the process.”

The Monster Hunter series is one with a clear vision, and it’s reassuring to know that so many of its most talented creators are overseeing this new venture. It’s evident that Capcom and the Monster Hunter team want to build on their existing audience and spread the gospel of Monster Hunter, and Kawai confirms it, saying the team hopes to build on the existing audience “by attracting a diverse audience to the game, including families. There is so much potential on mobile.”

One element of the game that is unclear so far is how Niantic and Capcom hope to keep players engaged over time. The Monster Hunter gameplay loop of creating armor and weapons is fantastic, but a mobile game needs certain elements to keep things fresh. When asked how Now is set to keep fans playing, Kawai explains that Niantic is “introducing a season system and will be adding new monsters, storylines, and weapons types to the game over time.”

Kawai further explains as they mention that “Niantic has a great track record of running live operations and in-game events for mobile games,” and it’s certainly true. After a somewhat slow start, Pokémon Go is now constantly supported with regular interesting events, with creative new features or tasks to keep players happy. According to Kawai, the Now team is “excited about running the game as a service” and seeing what the team can achieve with such a “passionate community.”

Finally, we couldn’t finish the interview without asking Kawai what the series means to them, and thankfully it’s clear this title is being developed with a lot of love and passion. Kawai told us that the Monster Hunter series “means a lot to me personally,” and there’s a fun (and novel) reason for that.

Kawai says “my cousin actually met and then married their partner by playing Monster Hunter together at college,” and according to Kawai that’s where they “got the idea for the game.” Two people meeting and bonding over the series is touching, and as Kawai further explains the “Monster Hunter world clearly had the power to bring people together in the real world.”

We also couldn’t help but ask Kawai what their favorite monster is, and they happily explained that “It has to be Pukei-Pukei. I just love its offbeat quirkiness!” We’ll have to wait just a bit longer to learn more about Monster Hunter Now, but it’s clear that the developer is passionate about the series, and driven to help spread it even further. Perhaps just like Kawai’s cousin, it’ll help bring you closer to people in the real world, very soon.

James

gRPC is a modern open source remote procedure call framework. There are many exciting features in gRPC: real-time streaming, end-to-end code generation, and great cross-platform support to name a few. The most exciting to me, and consistently mentioned by developers who are interested in gRPC, is performance.

Last year Microsoft contributed a new implementation of gRPC for .NET to the CNCF. Built on top of Kestrel and HttpClient, gRPC for .NET makes gRPC a first-class member of the .NET ecosystem.

In our first gRPC for .NET release, we focused on gRPC’s core features, compatibility, and stability. In .NET 5, we made gRPC really fast.

gRPC and .NET 5 are fast

In a community run benchmark of different gRPC server implementations, .NET gets the highest requests per second after Rust, and is just ahead of C++ and Go.

This result builds on top of the work done in .NET 5. Our benchmarks show .NET 5 server performance is 60% faster than .NET Core 3.1. .NET 5 client performance is 230% faster than .NET Core 3.1.

Stephen Toub discusses dotnet/runtime changes in his Performance Improvements in .NET 5 blog post. Check it out to read about improvements in HttpClient and HTTP/2.

In the rest of this blog post I’ll talk about the improvements we made to make gRPC fast in ASP.NET Core.

HTTP/2 allocations in Kestrel

gRPC uses HTTP/2 as its underlying protocol. A fast HTTP/2 implementation is the most important factor when it comes to performance. Our gRPC server builds on top of Kestrel, a HTTP server written in C# that is designed with performance in mind. Kestrel is a top contender in the TechEmpower benchmarks, and gRPC benefits from a lot of the performance improvements in Kestrel automatically. However, there are many HTTP/2 specific optimizations that were made in .NET 5.

Reducing allocations is a good place to start. Fewer allocations per HTTP/2 request means less time doing garbage collection (GC). And CPU time “wasted” in GC is CPU time not spent serving HTTP/2 requests.

The performance profiler above is measuring allocations over 100,000 gRPC requests. The live object graph’s sawtooth shaped pattern indicates memory building up, then being garbage collected. About 3.9KB is being allocated per request. Lets try to get that number down!

dotnet/aspnetcore#18601 adds pooling of streams in a HTTP/2 connection. This one change almost cuts allocations per request in half. It enables reuse of internal types like Http2Stream, and publicly accessible types like HttpContext and HttpRequest, across multiple requests.

Once streams are pooled a range of optimizations become available:

There are many smaller allocation savings. dotnet/aspnetcore#19783 removes allocations in Kestrel’s HTTP/2 flow control. A resettable ManualResetValueTaskSourceCore<T> type replaces allocating a new object each time flow control is triggered. dotnet/aspnetcore#19273 replaces an array allocation with stackalloc when validating the HTTP request path. dotnet/aspnetcore#19277 and dotnet/aspnetcore#19325 eliminate some unintended allocations related to logging. dotnet/aspnetcore#22557 avoids allocating a Task<T> if a task is already complete. And finally dotnet/aspnetcore#19732 saves a string allocation by special casing content-length of 0. Because every allocation matters.

Per-request memory in .NET 5 is now just 330 B, a decrease of 92%. The sawtooth pattern has also disappeared. Reduced allocations means garbage collection didn’t run at all while the server processed 100,000 gRPC calls.

Reading HTTP headers in Kestrel

A hotpath in HTTP/2 is reading and writing HTTP headers. A HTTP/2 connection supports concurrent requests over a TCP socket, a feature called multiplexing. Multiplexing allows HTTP/2 to make efficient use of connections, but only the headers for one request on a connection can be processed at a time. HTTP/2’s HPack header compression is stateful and depends on order. Processing HTTP/2 headers is a bottleneck so has to be as fast as possible.

dotnet/aspnetcore#23083 optimizes the performance of HPackDecoder. The decoder is a state machine that reads incoming HTTP/2 HEADER frames. The approach here is good, the state machine allows Kestrel to decode frames as they arrive, but the decoder was checking state after parsing each byte. Another problem is literal values, the header names and values, were copied multiple times. Optimizations in this PR include:

• Tighten parsing loops. For example, if we’ve just parsed a header name then the value must come afterwards. There is no need to check the state machine to figure out the next state.
• Skip literal parsing all together. Literals in HPack have a length prefix. If we know the next 100 bytes are a literal then there is no need to inspect each byte. Mark the literal’s location and resuming parsing at its end.
• Avoid copying literal bytes. Previously literal bytes were always copied to an intermediary array before passed to Kestrel. Most of the time this isn’t necessary and instead we can just slice the original buffer and pass a ReadOnlySpan<byte> to Kestrel.

Together these changes significantly decrease the time it takes to parse headers. Header size is almost no longer a factor. The decoder marks the start and end position of a value and then slices that range.

private HPackDecoder _decoder = CreateDecoder();

private byte[] _smallHeader = new byte[] { /* HPack bytes */ };

private byte[] _largeHeader = new byte[] { /* HPack bytes */ };

private IHttpHeadersHandler _noOpHandler = new NoOpHeadersHandler(); [Benchmark]

public void SmallDecode() => _decoder.Decode(_smallHeader, endHeaders: true, handler: _noOpHandler); [Benchmark]

public void LargeDecode() => _decoder.Decode(_largeHeader, endHeaders: true, handler: _noOpHandler);

Once headers have been decoded, Kestrel needs to validate and process them. For example, special HTTP/2 headers like :path and :method need to be set onto HttpRequest.Path and HttpRequest.Method, and other headers need to be converted to strings and added to the HttpRequest.Headers collection.

Kestrel has the concept of known request headers. Known headers are a selection of commonly occuring request headers that have been optimized for fast setting and getting. dotnet/aspnetcore#24730 adds an even faster path for setting HPack static table headers to the known headers. The HPack static table gives 61 common header names and values a number ID that can be sent instead of the full name. A header with a static table ID can use the optimized path to bypass some validation and quickly be set in the collection based on its ID. dotnet/aspnetcore#24945 adds extra optimization for static table IDs with a name and value.

Adding HPack response compression

Prior to .NET 5, Kestrel supported reading HPack compressed headers in requests, but it didn’t compress response headers. The obvious advantage of response header compression is less network usage, but there are performance benefits as well. It’s faster to write a couple of bits for a compressed header than it is to encode and write the header’s full name and value as bytes.

dotnet/aspnetcore#19521 adds initial HPack static compression. Static compression is pretty simple: if the header is in the HPack static table then write the ID to identify the header instead of the longer name.

Dynamic HPack header compression is more complicated, but also provides bigger gains. Response header names and values are tracked in a dynamic table and are each assigned an ID. As a response’s headers are written, the server checks to see if the header name and value are in the table. If there is a match then the ID is written. If there isn’t then the full header is written, and it is added to the table for the next response. There is a maximum size of the dynamic table, so adding a header to it may evict other headers with a first in, first out order.

dotnet/aspnetcore#20058 adds dynamic HPack header compression. To quickly search for headers the dynamic table groups header entries using a basic hash table. To track order and evict the oldest headers, entries maintain a linked list. To avoid allocations, removed entries are pooled and reused.

Using Wireshark, we can see the impact of header compression on response size for this example gRPC call. .NET Core 3.x writes 77 B, while .NET 5 is only 12 B.

Protobuf message serialization

gRPC for .NET uses the Google.Protobuf package as the default serializer for messages. Protobuf is an efficient binary serialization format. Google.Protobuf is designed for performance, using code generation instead of reflection to serialize .NET objects. There are some modern .NET APIs and features that can be added to it to reduce allocations and improve efficiency.

The biggest improvement to Google.Protobuf is support for modern .NET IO types: Span<T>, ReadOnlySequence<T> and IBufferWriter<T>. These types allow gRPC messages to be serialized directly using buffers exposed by Kestrel. This saves Google.Protobuf allocating an intermediary array when serializing and deserializing Protobuf content.

Support for Protobuf buffer serialization was a multi-year effort between Microsoft and Google engineers. Changes were spread across multiple repositories.

protocolbuffers/protobuf#7351 and protocolbuffers/protobuf#7576 add support for buffer serialization to Google.Protobuf. This is by far the biggest and most complicated change. Three attempts were made to add this feature before the right balance between performance, backwards compatibility and code reuse was found. Protobuf reading and writing uses many performance oriented features and APIs added to C# and .NET Core:

• Span<T> and C# ref struct types enables fast and safe access to memory. Span<T> represents a contiguous region of arbitrary memory. Using span lets us serialize to managed .NET arrays, stack allocated arrays, or unmanaged memory, without using pointers. Span<T> and .NET protects us against buffer overflow.
• stackalloc is used to create stack-based arrays. stackalloc is a useful tool to avoid allocations when a small buffer is required.
• Low-level methods such as MemoryMarshal.GetReference(), Unsafe.ReadUnaligned() and Unsafe.WriteUnaligned() convert directly between primitive types and bytes.
• BinaryPrimitives has helper methods for efficiently converting between .NET primitive types and bytes. For example, BinaryPrimitives.ReadUInt64LittleEndian reads little endian bytes and returns an unsigned 64 bit number. Methods provided by BinaryPrimitive are heavily optimized and use vectorization.

A great thing about modern C# and .NET is it is possible to write fast, efficient, low-level libraries without sacrificing memory safety. When it comes to performance, .NET lets you have your cake and eat it too!

private TestMessage _testMessage = CreateMessage();

private ReadOnlySequence<byte> _testData = CreateData();

private IBufferWriter<byte> _bufferWriter = CreateWriter(); [Benchmark]

public IMessage ToByteArray() => _testMessage.ToByteArray(); [Benchmark]

public IMessage ToBufferWriter() => _testMessage.WriteTo(_bufferWriter); [Benchmark]

public IMessage FromByteArray() => TestMessage.Parser.ParseFrom(CreateBytes()); [Benchmark]

public IMessage FromSequence() => TestMessage.Parser.ParseFrom(_testData);

Adding support for buffer serialization to Google.Protobuf is just the first step. More work is required for gRPC for .NET to take advantage of the new capability:

• grpc/grpc#18865 and grpc/grpc#19792 adds ReadOnlySequence<byte> and IBufferWriter<byte> APIs to the gRPC serialization abstraction layer in Grpc.Core.Api.
• grpc/grpc#23485 updates gRPC code generation to glue the changes in Google.Protobuf to Grpc.Core.Api.
• grpc/grpc-dotnet#376 and grpc/grpc-dotnet#629 updates gRPC for .NET to use the new serialization abstractions in Grpc.Core.Api. This code is the integration between Kestrel and gRPC. Because Kestrel’s IO is built on top of System.IO.Pipelines, we can use its buffers during serialization.

The end result is gRPC for .NET serializes Protobuf messages directly to Kestrel’s request and response buffers. Intermediary array allocations and byte copies have been eliminated from gRPC message serialization.

Wrapping Up

Performance is a feature of .NET and gRPC, and as cloud apps scale it is more important than ever. I think all developers can agree it is fun to make fast apps, but performance has real world impact. Lower latency and higher throughput means fewer servers. It is an opportunity to save money, reduce power use and build greener apps.

As is obvious from this tour, a lot of changes have gone into gRPC, Protobuf and .NET aimed at improving performance. Our benchmarks show a 60% improvement in gRPC server RPS and a 230% improvement in gRPC client RPS.

.NET 5 RC2 is available now, and the official .NET 5 release is in November. To try out the performance improvements and to get started using gRPC with .NET, the best place to start is the Create a gRPC client and server in ASP.NET Core tutorial.

We look forward to hearing about apps built with gRPC and .NET, and to your future contributions in the dotnet and grpc repos!

Frontline workers represent the face of organizations and make up the lion’s share of the workforce. But new Work Trend Index data reveals that 1 in 2 frontline workers cite being burned out in their jobs. Investing in technology that supports the frontline is key. Notably, 65% of frontline workers are optimistic that AI will help them in their jobs.  
We are excited to highlight new Microsoft solutions and investments in next-generation AI for the frontline workforce across nearly every business. Frontline managers and workers can optimize their time from work order creation to schedule management with Copilot in Dynamics 365 Field Service and the new Shifts plug-in for Microsoft 365 Copilot. In Viva Connections, frontline workers can stay up-to-date on internal communications. And with shared device mode for Intune, VMware and SOTI, employees can simplify the sign-in experience securely.  
Today’s announcements not only reduce administrative burden and time on spent on manual tasks, but also enable frontline workers to focus on end customer experiences and receive an overall improved employee experience. Learn more here: https://lnkd.in/e9Ash_82

Apple is reportedly looking to cut back on providing human support on various social media outlets like YouTube and Twitter.