Master Application Flow Guide

The Master Application Flow is a comprehensive, production-ready workflow system that combines all subsystems with extensive error handling, fallback mechanisms, and interconnected nodes.

Overview

This workflow represents a complete software application with:

• 20+ interconnected steps
• Complete error handling with 5 fallback strategies
• Memory integration throughout
• Security and compliance features
• Performance optimization
• Observability and monitoring

Architecture Components

Core Orchestration Layer

MasterOrchestrator - Entry point for all requests

• Receives API requests at /master/execute
• Searches memory for similar past executions
• Routes requests to appropriate subsystems
• Manages execution context and state

RequestValidator - Validates all incoming requests

• Validates request structure using Zod schemas
• Checks subsystem and operation validity
• Performs business rule validations
• Emits validation results

SecurityChecker - Security and compliance

• Authentication verification
• Authorization checks based on roles
• Rate limiting (100 requests per minute)
• Suspicious activity detection using memory

Routing Layer

SubsystemRouter - Routes to appropriate subsystems

• Routes to 8+ subsystems (order, payment, user, approval, integration, analytics, data-pipeline, notification)
• Uses memory insights for routing decisions
• Maintains routing history

Subsystem Processors

OrderProcessor - Complete order lifecycle

• Validates order structure
• Checks inventory availability
• Processes payment
• Fulfills order
• Stores completion in memory

PaymentProcessor - Payment processing

• Supports multiple payment methods (credit card, PayPal, bank transfer)
• Validates payment data
• Processes transactions
• Handles refunds
• Stores transactions in memory

UserManager - User operations

• Authentication
• Authorization
• User creation and updates
• Profile management
• Preference loading from memory

Error Handling Layer

ErrorDetector - Detects errors from all sources

• Monitors all workflow steps
• Captures error context
• Classifies errors by type and severity

ErrorRetrier - Retries with exponential backoff

• Automatic retry mechanism
• Exponential backoff (1s, 2s, 4s, max 10s)
• Max 3 attempts
• Tracks retry history

ErrorResolver - Resolves errors and stores learnings

• Determines resolution strategy
• Stores resolutions in memory
• Learns from past resolutions

FallbackHandler - Comprehensive fallback system

• 5 fallback strategies:
  1. Retry - For transient errors
  2. Alternative - Use cache or defaults
  3. Skip - For non-critical errors
  4. Escalate - For critical errors
  5. Graceful Degradation - Reduced functionality

Supporting Systems

NotificationManager - Multi-channel notifications

• Email notifications
• SMS notifications
• Push notifications
• Channel selection based on priority

CacheManager - Performance optimization

• Cache get/set operations
• TTL management
• Cache hit/miss tracking

AuditLogger - Complete audit trail

• Logs all critical operations
• Maintains audit history
• Tracks user actions

WorkflowMonitor - Real-time monitoring

• Tracks execution metrics
• Monitors error rates
• Generates alerts
• Performance analytics

WorkflowCompleter - Finalization

• Generates workflow summaries
• Stores summaries in memory
• Finalizes execution state

Event Flow

The workflow uses an event-driven architecture with 50+ event topics:

master.request-received
  → request.validated
  → security.verified
  → master.routed
  → subsystem.{subsystem}.trigger
  → [subsystem-specific events]
  → notification.sent
  → monitoring.metrics-updated
  → workflow.completed

Error Flow

[Any Step] → error.occurred
  → error.detected
  → error.classified
  → fallback.activated
  → [Fallback Strategy]
    → retry.initiated / fallback.alternative / fallback.skip / fallback.escalate / fallback.graceful-degradation

State Management

All steps share state through Motia’s state system:

// Store state
state.set('key', value)

// Retrieve state
const value = state.get('key')

// Common state keys:
- master-${executionId} - Master execution context
- validation-${executionId} - Validation results
- security-${executionId} - Security check results
- routing-${executionId} - Routing decisions
- order-${orderId} - Order data
- payment-${paymentId} - Payment data
- cache-${key} - Cache entries
- audit-logs - Audit log history
- workflow-monitoring - Monitoring metrics

Memory Integration

Memory is integrated at 10+ points:

1. Master Orchestrator - Searches for similar past executions
2. Security Checker - Detects suspicious activity patterns
3. Subsystem Router - Uses memory for routing decisions
4. Order Processor - Stores order completion in memory
5. Payment Processor - Stores payment transactions
6. User Manager - Loads user preferences from memory
7. Approval Router - Finds best approvers from past decisions
8. Error Resolver - Stores resolution strategies for learning
9. Fallback Handler - Stores fallback decisions
10. Workflow Completer - Stores workflow summaries

Usage Examples

Complete Order Flow

curl -X POST http://localhost:3000/master/execute \
  -H "Content-Type: application/json" \
  -d '{
    "operation": "create",
    "subsystem": "order",
    "data": {
      "customerId": "user-123",
      "items": [
        {"id": "prod-1", "price": 49.99, "quantity": 2},
        {"id": "prod-2", "price": 29.99, "quantity": 1}
      ],
      "paymentMethod": "credit-card"
    },
    "userId": "user-123",
    "priority": "normal",
    "fallbackEnabled": true
  }'

Payment Processing

curl -X POST http://localhost:3000/master/execute \
  -H "Content-Type: application/json" \
  -d '{
    "operation": "process",
    "subsystem": "payment",
    "data": {
      "amount": 199.99,
      "paymentMethod": "paypal",
      "customerId": "user-123",
      "orderId": "ORD-12345"
    },
    "userId": "user-123"
  }'

User Management

curl -X POST http://localhost:3000/master/execute \
  -H "Content-Type: application/json" \
  -d '{
    "operation": "profile",
    "subsystem": "user",
    "data": {},
    "userId": "user-123"
  }'

Fallback Strategies

1. Retry Strategy

• When: Timeout, network, server errors
• How: Exponential backoff (1s, 2s, 4s, max 10s)
• Max Attempts: 3
• Success Rate: 70% on retry

2. Alternative Strategy

• When: Not-found, validation errors
• Options:
  • Use cache instead of API
  • Use default values instead of validation
• Automatic: Yes

3. Skip Strategy

• When: Low-severity, non-critical errors
• Action: Skip step, continue workflow
• Use Case: Non-essential validations

4. Graceful Degradation

• When: Rate limits, partial failures
• Action: Continue with reduced functionality
• Features Disabled: Real-time updates, advanced features

5. Escalation Strategy

• When: Critical errors, permission issues, max retries reached
• Action: Escalate to admin/human
• Levels: Supervisor → Admin → Critical Alert

Best Practices

1. Always enable fallbacks - Set fallbackEnabled: true for production
2. Use appropriate priorities - Set priority based on business needs
3. Include userId - Always provide userId for security and memory integration
4. Validate data - Ensure data structure matches subsystem requirements
5. Monitor metrics - Check workflow monitoring for performance insights
6. Review audit logs - Regularly review audit logs for compliance

Troubleshooting

High Error Rate

• Check error analytics in WorkflowMonitor
• Review fallback activation patterns
• Verify subsystem availability

Slow Performance

• Check cache hit rates
• Review parallel processing opportunities
• Optimize memory queries

Security Failures

• Verify authentication tokens
• Check rate limit thresholds
• Review suspicious activity patterns

Next Steps

• Examples - See complete examples
• API Reference - Complete API documentation
• Architecture - System architecture details