sf-ai-agentforce-observability: Agentforce Session Tracing Extraction & Analysis

Expert in extracting and analyzing Agentforce session tracing data from Salesforce Data Cloud. Supports high-volume data extraction (1-10M records/day), Parquet storage, and Polars-based analysis for debugging agent behavior.

Core Responsibilities

1. Session Extraction: Extract STDM (Session Tracing Data Model) data via Data Cloud Query API
2. Data Storage: Write to Parquet format with PyArrow for efficient storage
3. Analysis: Polars-based lazy evaluation for memory-efficient analysis
4. Debugging: Session timeline reconstruction for troubleshooting agent issues
5. Cross-Skill Integration: Works with sf-connected-apps for auth, sf-ai-agentscript for fixes

Document Map

Quick Links:

• Data Model Overview
• CLI Quick Reference
• Analysis Examples
• Cross-Skill Integration

CRITICAL: Prerequisites Checklist

Before extracting session data, verify:

Auth Setup (via sf-connected-apps)

# 1. Generate certificate
openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 \
  -keyout ~/.sf/jwt/myorg.key \
  -out ~/.sf/jwt/myorg.crt \
  -subj "/CN=DataCloudAuth"

# 2. Create External Client App (use sf-connected-apps skill)
Skill(skill="sf-connected-apps", args="Create ECA with JWT Bearer for Data Cloud")

# Required scopes: cdp_query_api, cdp_profile_api

See docs/auth-setup.md for detailed instructions.

Session Tracing Data Model (STDM)

The STDM consists of 4 Data Model Objects (DMOs) in a hierarchical structure:

ssot__AIAgentSession__dlm (SESSION)
├── ssot__Id__c                          # Session ID
├── ssot__AIAgentApiName__c              # Agent API name
├── ssot__StartTimestamp__c              # Session start
├── ssot__EndTimestamp__c                # Session end
├── ssot__AIAgentSessionEndType__c       # End type (Completed, Abandoned, etc.)
├── ssot__RelatedMessagingSessionId__c   # Linked messaging session
└── ssot__OrganizationId__c              # Org ID

    └── ssot__AIAgentInteraction__dlm (TURN/SESSION_END)  [1:N]
        ├── ssot__Id__c                          # Interaction ID
        ├── ssot__aiAgentSessionId__c            # FK to Session
        ├── ssot__InteractionType__c             # TURN or SESSION_END
        ├── ssot__TopicApiName__c                # Topic that handled this turn
        ├── ssot__StartTimestamp__c              # Turn start
        └── ssot__EndTimestamp__c                # Turn end

            ├── ssot__AIAgentInteractionStep__dlm (STEP)  [1:N]
            │   ├── ssot__Id__c                          # Step ID
            │   ├── ssot__AIAgentInteractionId__c        # FK to Interaction
            │   ├── ssot__AIAgentInteractionStepType__c  # LLM_STEP or ACTION_STEP
            │   ├── ssot__Name__c                        # Action/step name
            │   ├── ssot__InputValueText__c              # Input to step
            │   ├── ssot__OutputValueText__c             # Output from step
            │   ├── ssot__PreStepVariableText__c         # Variables before
            │   ├── ssot__PostStepVariableText__c        # Variables after
            │   └── ssot__GenerationId__c                # LLM generation ID

            └── ssot__AIAgentMoment__dlm (MESSAGE)  [1:N]
                ├── ssot__Id__c                              # Message ID
                ├── ssot__AIAgentInteractionId__c            # FK to Interaction
                ├── ssot__ContentText__c                     # Message content
                ├── ssot__AIAgentInteractionMessageType__c   # INPUT or OUTPUT
                └── ssot__MessageSentTimestamp__c            # Timestamp

See resources/data-model-reference.md for full field documentation.

Workflow (5-Phase Pattern)

Phase 1: Requirements Gathering

Use AskUserQuestion to gather:

Phase 2: Auth Configuration

Verify JWT auth is configured:

from scripts.auth import DataCloudAuth

auth = DataCloudAuth(
    org_alias="myorg",
    consumer_key="YOUR_CONSUMER_KEY"
)

# Test authentication
token = auth.get_token()
print(f"Auth successful: {token[:20]}...")

If auth fails, invoke:

Skill(skill="sf-connected-apps", args="Setup JWT Bearer for Data Cloud")

Phase 3: Extraction

Basic Extraction (last 7 days):

python3 scripts/cli.py extract \
  --org prod \
  --days 7 \
  --output ./stdm_data

Filtered Extraction:

python3 scripts/cli.py extract \
  --org prod \
  --since 2026-01-01 \
  --until 2026-01-28 \
  --agent Customer_Support_Agent \
  --output ./stdm_data

Session Tree (specific session):

python3 scripts/cli.py extract-tree \
  --org prod \
  --session-id "a0x..." \
  --output ./debug_session

Phase 4: Analysis

Session Summary:

from scripts.analyzer import STDMAnalyzer
from pathlib import Path

analyzer = STDMAnalyzer(Path("./stdm_data"))

# High-level summary
summary = analyzer.session_summary()
print(summary)

# Step distribution by agent
steps = analyzer.step_distribution(agent_name="Customer_Support_Agent")
print(steps)

# Topic routing analysis
topics = analyzer.topic_analysis()
print(topics)

Debug Specific Session:

python3 scripts/cli.py debug-session \
  --data-dir ./stdm_data \
  --session-id "a0x..."

Phase 5: Integration & Next Steps

Based on analysis findings:

CLI Quick Reference

Extraction Commands

Analysis Commands

Common Flags

See docs/cli-reference.md for complete documentation.

Analysis Examples

Session Summary

📊 SESSION SUMMARY
════════════════════════════════════════════════════════════════

Period: 2026-01-21 to 2026-01-28
Total Sessions: 15,234
Unique Agents: 3

SESSIONS BY AGENT
────────────────────────────────────────────────────────────────
Agent                          │ Sessions │ Avg Turns │ Avg Duration
───────────────────────────────┼──────────┼───────────┼─────────────
Customer_Support_Agent         │   8,502  │    4.2    │     3m 15s
Order_Tracking_Agent           │   4,128  │    2.8    │     1m 45s
Product_FAQ_Agent              │   2,604  │    1.9    │       45s

END TYPE DISTRIBUTION
────────────────────────────────────────────────────────────────
✅ Completed:    12,890 (84.6%)
🔄 Escalated:     1,523 (10.0%)
❌ Abandoned:       821 (5.4%)

Debug Session Timeline

🔍 SESSION DEBUG: a0x1234567890ABC
════════════════════════════════════════════════════════════════

Agent: Customer_Support_Agent
Started: 2026-01-28 10:15:23 UTC
Duration: 4m 32s
End Type: Completed
Turns: 5

TIMELINE
────────────────────────────────────────────────────────────────
10:15:23 │ [INPUT]  "I need help with my order #12345"
10:15:24 │ [TOPIC]  → Order_Tracking (confidence: 0.95)
10:15:24 │ [STEP]   LLM_STEP: Identify intent
10:15:25 │ [STEP]   ACTION_STEP: Get_Order_Status
         │          Input: {"orderId": "12345"}
         │          Output: {"status": "Shipped", "eta": "2026-01-30"}
10:15:26 │ [OUTPUT] "Your order #12345 has shipped and will arrive by Jan 30."

10:16:01 │ [INPUT]  "Can I change the delivery address?"
10:16:02 │ [TOPIC]  → Order_Tracking (same topic)
10:16:02 │ [STEP]   LLM_STEP: Clarify request
10:16:03 │ [STEP]   ACTION_STEP: Check_Modification_Eligibility
         │          Input: {"orderId": "12345", "type": "address_change"}
         │          Output: {"eligible": false, "reason": "Already shipped"}
10:16:04 │ [OUTPUT] "I'm sorry, the order has already shipped..."

Cross-Skill Integration

Prerequisite Skills

Follow-up Skills

Commonly Used With

Key Insights

Common Issues & Fixes

See resources/troubleshooting.md for detailed solutions.

Output Directory Structure

After extraction:

stdm_data/
├── sessions/
│   └── date=2026-01-28/
│       └── part-0000.parquet
├── interactions/
│   └── date=2026-01-28/
│       └── part-0000.parquet
├── steps/
│   └── date=2026-01-28/
│       └── part-0000.parquet
├── messages/
│   └── date=2026-01-28/
│       └── part-0000.parquet
└── metadata/
    ├── extraction.json      # Extraction parameters
    └── watermark.json       # For incremental extraction

Dependencies

Python 3.10+ with:

polars>=1.0.0           # DataFrame library (lazy evaluation)
pyarrow>=15.0.0         # Parquet support
pyjwt>=2.8.0            # JWT generation
cryptography>=42.0.0    # Certificate handling
httpx>=0.27.0           # HTTP client
rich>=13.0.0            # CLI progress bars
click>=8.1.0            # CLI framework
pydantic>=2.6.0         # Data validation

Install: pip install -r requirements.txt

License

MIT License. See LICENSE file. Copyright (c) 2024-2026 Jag Valaiyapathy