Stream Smarter and at Lower Costs with Decodable’s Scale to Zero

At Decodable, we're committed to empowering data teams to work both efficiently and reliably. Today, we’re introducing Scale to Zero—a feature that brings cost efficiency to our real-time data platform. With a simple configuration, your jobs automatically pause and resume without any modification to the underlying streaming jobs. This integration of batch-like control into continuous streaming processing jobs enables true stream-batch unification, ensuring resources are utilized only when necessary.

Why Scale to Zero?

Decodable’s real-time data platform used to be  “always-on” to ensure low-latency data processing. But we understand that not every workload demands constant processing. Many of our customers deal with sporadic or batch-based event sources, making it inefficient to keep jobs running idly. The result? Wasted resources and unnecessary costs.

Scale to Zero solves this challenge by empowering teams to automatically pause jobs during idle periods and resume them later, thereby  providing a seamless, efficient, and cost-effective streaming experience. 

How it works 

Scale to Zero introduces intelligent job lifecycle management: 

• Automatic Pausing: Based on user-defined and system conditions, jobs pause when no meaningful data is processed. Examples include: 
  
  • Metrics-based: when fewer than a given number of records are processed in a configurable time window.
  • Duration-based: when the job has been running for over a given number of hours.
• Automatic Resuming: Paused jobs resume automatically after a configurable trigger, ensuring jobs reactivate promptly to process fresh events. 

Check out this example for more details. We’ve also put together a brief demo video showcasing the feature in action:

Benefits at a Glance

1. Cost Savings: Stop paying for idle resources. Scale to Zero pauses compute when no work is being done.
2. Streamlined Operations: Define flexible pause/resume criteria to align with your operational and business needs.
3. Sustainability: Use resources only when necessary, reducing waste and energy consumption. 
4. Seamless Resumption: Jobs resume instantly, ensuring no delays when real-time processing needs to restart.

Looking Forward

In its first iteration, Scale to Zero focuses on pausing jobs for SQL pipelines and connections only. We are also exploring more ways to express the pause and resume conditions to expand the applicability of this feature across more job types. 

Ready to Scale Smarter?

Scale to Zero is available now. Whether your workloads are event-driven or batch-oriented, this feature empowers you to optimize costs while maintaining the responsiveness of your data pipelines. Get started today by logging into your Decodable account, and check out our documentation for more details. For questions, join the conversation in our community Slack.

With Scale to Zero, Decodable is making real-time data processing not just powerful but practical and affordable for every use case. Experience the future of streaming efficiency today!