AI Model Wiki

    Skip to main content
    22 min
    Lesson 2 of 4

    Workload Distribution and Cost Optimization

    From the course Multi-Cloud AI Strategy

    Strategically distribute AI workloads across clouds for optimal cost and performance.

    Model Routing Strategy

    Intelligent Model Selector

    class MultiCloudModelRouter:
        def __init__(self):
            self.providers = {
                "aws-claude": {"cost_per_token": 0.003, "latency_ms": 400},
                "azure-gpt4o": {"cost_per_token": 0.005, "latency_ms": 350},
                "gcp-gemini": {"cost_per_token": 0.0025, "latency_ms": 300}
            }
        
        def select_provider(self, task_type, priority="cost"):
            if priority == "cost":
                return min(self.providers, 
                          key=lambda p: self.providers[p]["cost_per_token"])
            
            elif priority == "latency":
                return min(self.providers,
                          key=lambda p: self.providers[p]["latency_ms"])
            
            elif priority == "capability":
                # Route by task type
                routing = {
                    "vision": "gcp-gemini",        # Best multimodal
                    "reasoning": "azure-gpt4o",    # Best reasoning
                    "code": "aws-claude",           # Strong code generation
                    "general": self.cheapest()
                }
                return routing.get(task_type, self.cheapest())
        
        def cheapest(self):
            return min(self.providers, 
                       key=lambda p: self.providers[p]["cost_per_token"])
    

    Cost Tracking Across Clouds

    class MultiCloudCostTracker:
        def get_monthly_ai_spend(self):
            costs = {}
            
            # AWS Cost Explorer
            ce = boto3.client('ce')
            response = ce.get_cost_and_usage(
                TimePeriod={"Start": "2025-01-01", "End": "2025-02-01"},
                Granularity="MONTHLY",
                Filter={"Dimensions": {"Key": "SERVICE", "Values": ["Amazon Bedrock"]}}
            )
            costs["aws"] = float(response["ResultsByTime"][0]["Total"]["BlendedCost"]["Amount"])
            
            # GCP Billing API
            costs["gcp"] = self.get_gcp_costs("aiplatform.googleapis.com")
            
            # Azure Cost Management
            costs["azure"] = self.get_azure_costs("microsoft.cognitiveservices")
            
            return costs
    

    Key Takeaways

    • •Intelligent model routing directs tasks to the most cost-effective or capable provider
    • •Task-based routing leverages provider strengths for vision, reasoning, and code
    • •Cross-cloud cost tracking provides unified visibility into AI spending
    • •Dynamic routing based on real-time pricing and latency optimizes continuously
    0 of 4 lessons marked complete

    Progress is kept in this browser only.