Monitor usage, manage costs, and implement responsible AI controls in Azure OpenAI deployments.

Monitoring

Azure Monitor Integration

# Key metrics to monitor
# - SuccessRate: % of successful API calls
# - TokenTransaction: Total tokens consumed
# - ProcessedPromptTokens: Input tokens
# - GeneratedCompletionTokens: Output tokens
# - ContentFilteredRequests: Blocked requests

# Set up alert for high error rate
az monitor metrics alert create \
    --name "OpenAI_High_Error_Rate" \
    --resource-group myRG \
    --scopes /subscriptions/.../openai/myOpenAI \
    --condition "avg SuccessRate < 95" \
    --evaluation-frequency 5m \
    --window-size 15m

Diagnostic Logging

# Enable request/response logging
az monitor diagnostic-settings create \
    --name "OpenAI_Diag" \
    --resource /subscriptions/.../openai/myOpenAI \
    --logs '[{"category":"RequestResponse","enabled":true}]' \
    --workspace /subscriptions/.../workspaces/myWorkspace

Cost Management

Token-Based Optimization

import tiktoken

def count_tokens(text, model="gpt-4o"):
    encoding = tiktoken.encoding_for_model(model)
    return len(encoding.encode(text))

def optimize_prompt(prompt, max_tokens=2000):
    token_count = count_tokens(prompt)
    if token_count > max_tokens:
        # Truncate from middle to preserve context
        words = prompt.split()
        truncated = words[:max_tokens // 2] + ["..."] + words[-max_tokens // 2:]
        return " ".join(truncated)
    return prompt

Responsible AI Controls

Content Filters Configuration

# Annotate model invocations for content analysis
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": prompt}]
)

# Check content filter results
if response.choices[0].finish_reason == "content_filter":
    filter_results = response.choices[0].content_filter_results
    print(f"Blocked: {filter_results}")
    # Log and handle appropriately

Prompt Shields

• Detect prompt injection attacks
• Block jailbreak attempts automatically
• Monitor for indirect injection in RAG systems
• Groundedness detection for hallucination