10
Azure Functions and Event Hubs: Optimising for Throughput

Azure Functions and Event Hubs: Optimising for Throughput

5 years ago
Anonymous $L9wC17otzH

https://medium.com/@iizotov/azure-functions-and-event-hubs-optimising-for-throughput-549c7acd2b75

TL;DR. I take a fairly standard serverless event processing scenario — an Azure Function triggered by messages in an Event Hub — and talk about how how to optimise this system for throughput through: a) overall architecture b) EH partitioning, andc) tweaking the Event Hub trigger host.json settings: maxBatchSize, prefetchCount and batchCheckpointFrequency. I doubt anyone would be interested in reading a yet another theoretical opus, so for my reader’s benefit I ran a series of experiments which involved testing multiple combinations of Event Hub partition counts and host.json settings, and observing the impact on latency and throughput. Feel free to jump straight through to the approach and results published to Power BI.

In other words, I’m trying to help those individuals who are at present staring detachedly in the middle distance trying to understand: