When we launched Digital Tap AI four months ago, we had a focused mission: eliminate idle cluster waste across the four major data platforms — Databricks, EMR, Synapse, and Dataproc. Today, with v2.0, we're expanding that mission dramatically.
Digital Tap AI v2.0 is our biggest release ever: 304 files, 27,850 lines of new code, 13 new agents, and first-class support for Kubernetes.
Why We Expanded Beyond the Big 4
Our customers kept telling us the same thing: "We love what you do for our Databricks clusters, but we have the same problem with our Kubernetes workloads." And they were right.
The $44.5 billion idle compute waste problem isn't limited to data platforms. It exists everywhere compute runs — and increasingly, that means Kubernetes. Over 60% of enterprises now run production workloads on K8s, and studies show that the average Kubernetes cluster runs at just 20-35% utilization.
We couldn't ignore that. So we built native support from the ground up.
Kubernetes as a First-Class Citizen
This isn't a bolt-on integration. We built six dedicated Kubernetes optimizers:
- Pod Right-Sizing — Analyzes actual resource usage vs. requests/limits and recommends precise adjustments. Most pods over-request CPU by 3-5x.
- Node Right-Sizing — Identifies over-provisioned nodes and recommends optimal instance types based on workload patterns.
- Bin Packing — Optimizes pod placement to maximize node utilization, reducing the total number of nodes needed.
- Namespace Optimization — Cost allocation and optimization at the namespace level, giving teams visibility into their actual spend.
- Autoscale Tuning — Intelligently tunes HPA and VPA parameters based on observed traffic patterns.
- Spot Detection — Identifies workloads suitable for spot/preemptible instances and manages the migration automatically.
We support vanilla Kubernetes, Amazon EKS (with Karpenter integration), and Google GKE (with Autopilot recommendations).
Carbon and Water-Aware Scheduling
This is the feature we're most excited about. Our new Smart Scheduler doesn't just optimize for cost — it optimizes for environmental impact.
The average hyperscale data center consumes 3-5 million gallons of water per day for cooling. When your jobs run in water-stressed regions during peak grid carbon intensity, the environmental cost is enormous.
The Smart Scheduler considers three factors when routing jobs:
- Carbon intensity — Real-time grid carbon data from electricityMap. Route batch jobs to low-carbon regions.
- Water Usage Effectiveness (WUE) — Regional data on data center water consumption. Avoid water-stressed regions for non-urgent workloads.
- Cost — Still the primary factor for most customers, but now with full visibility into the environmental trade-offs.
Every job now generates an ESG report showing gallons of water saved, CO2 avoided, and the sustainability score compared to a naive placement. This data feeds directly into corporate ESG reporting frameworks.
27 Autonomous Agents Working Together
With v2.0, we've grown from 14 agents to 27+. Each agent is now an independently scalable microservice in our Horizontal Agent Runner. The new agents include:
- CarbonAwareScheduler — Routes jobs based on real-time carbon and water data
- QueryOptimizer — Detects slow queries, bad joins, and recommends Spark config changes
- SustainabilityReporter — Generates ESG compliance reports automatically
- StorageOptimizer — Tiered storage management, orphan cleanup, small file compaction
- NetworkCostAnalyzer — Cross-AZ and cross-region data transfer cost optimization
- IdleResourceReaper — Aggressive idle resource cleanup beyond just clusters
- PredictiveScaler — ML-powered demand forecasting for proactive scaling
- CostAnomalyDetector — Real-time spending anomaly detection with alerting
- RightSizingAdvisor — Continuous right-sizing recommendations across all platforms
- 4 Kubernetes-specific agents — RightSizer, SpotManager, CostAllocator, AutoscaleTuner
The Path to Universal Platform Support
v2.0 establishes our platform adapter architecture — a standardized interface that makes adding new platforms straightforward. Every platform adapter implements the same metric collection, optimization, and action interfaces.
This architecture means we can now move faster. Our roadmap for the next two quarters:
- Q2 2026 — Snowflake (warehouse optimization, credit tracking)
- Q3 2026 — Apache Kafka (broker right-sizing, partition optimization) and Apache Airflow (DAG optimization, worker scaling)
- Q4 2026 — Amazon Redshift, Apache Flink, and Amazon SageMaker
Our goal: by end of 2026, Digital Tap AI should optimize every compute workload in your organization, regardless of where it runs.
What's Next
We're just getting started. v2.0 lays the foundation for a future where cost optimization and sustainability aren't competing priorities — they're the same thing. Every dollar of compute waste is also wasted energy, wasted water, and unnecessary carbon.
If you're ready to see what 27 autonomous agents can do for your infrastructure, sign up for free or talk to our team.
Ready to optimize?
Get Started Free →