Scale without delay
The fund's risk platform had to process billions of JSON objects daily while ensuring every update was instantly queryable — without data loss or performance lag.
CASE STUDY · CAPITAL MARKETS
From 100K+ lines of microservices to simple SQL.
How a major hedge fund redefined real-time risk analytics — processing 5 billion JSON objects daily with sub-second latency and a radically simplified architecture.
Real-time risk, measured.
- 5B+
- JSON objects ingested per day
- 1.3M
- records / minute sustained ingest
- Sub-second
- query latency during ingestion
- 100K+
- lines of microservice code retired
What drove the change
Delivering real-time, scalable risk analytics across billions of daily data points.
Faster metric innovation
Each new risk metric required custom microservices, slowing development. The goal was to replace this complexity with a more agile, SQL-driven process.
Flexible scalability
As data volumes exploded, the system struggled to maintain performance. A new architecture was required to scale seamlessly under continuous growth.
Instant, complex analytics
Traders and analysts needed to run complex calculations against constantly changing datasets, demanding sub-second responses for better decision-making.
The previous architecture
A microservice-heavy setup struggling to keep up with real-time financial data demands.
The platform relied on a large Cassandra cluster serving as a simple key lookup system, designed for high-volume data ingestion from Kafka reaching up to two million wide records per minute. Each analytical function existed as a standalone microservice containing its own processing logic. To support this, a large pool of application servers hosted the microservices, but the architecture quickly became complex and expensive to maintain.
Bottlenecks in the legacy stack
Ingestion bottlenecks
Data ingestion from Kafka into Cassandra required constant tuning to handle surges, creating operational overhead.
Limited analytics
Microservices could only process small portions of data, restricting the scope and depth of real-time insights.
High development overhead
Every new metric demanded a new microservice, driving up maintenance and slowing innovation.
Infrastructure costs
The need for heavy-weight nodes to host containerized microservices led to significant infrastructure spending.
Real-time reinvented
Unified ingestion and analytics with GPU-accelerated performance.
With Kinetica, the hedge fund achieved seamless, high-speed ingestion and analytics in one platform. Kinetica's native Kafka integration easily surpassed ingestion SLAs, averaging 1.3 million records per minute on a small AWS cluster. Complex queries could now run directly against live datasets, enabling analysts to generate richer, real-time insights.
THE HEADLINE — Over 100,000 lines of microservice
code were replaced with simple, flexible SQL queries — drastically reducing complexity and
accelerating innovation.
Why Kinetica outperformed other technologies.
Evaluated against modern analytical databases — ClickHouse, TimescaleDB, and StarRocks — on the fund's real ingestion and query workload.
| Capability | Kinetica | Other analytical databases |
|---|---|---|
| Ingestion speed |
1.3M+ records/min effortlessly
|
Unable to meet required ingestion speeds; slower than Cassandra
|
| Query performance during ingestion |
Sub-second latency
|
Poor, especially for complex queries with JOINs
|
| Kafka integration |
Native Kafka consumption supported
|
No native Kafka consumption capability
|
| Development overhead |
Simplified to SQL queries
|
Higher; often requires manual tuning or complex queries
|
| Infrastructure load |
Lean AWS footprint
|
Larger infrastructure footprint
|
| Monitoring effort |
Auto-optimized ingestion pipeline
|
Manual tuning often required
|