Tides Editor User Guide

Overview

The Tides Editor (Sequence Editor) is a visual tool for designing time-based document generation sequences. It allows you to orchestrate multiple DataFlood models to generate related documents over time with parent-child relationships and realistic timing patterns.

Key Concepts

Sequences

A sequence defines:

  • Time range for generation
  • Multiple document types (steps)
  • Generation intervals and timing
  • Relationships between documents
  • Transaction patterns

Steps

Each step represents:

  • A DataFlood model to use
  • When to generate documents
  • How many to generate
  • Generation probability
  • Custom properties

Transactions

Transactions create relationships:

  • Parent-child document links
  • Field inheritance
  • Triggered generation
  • Correlated timing

Interface Components

Timeline View (Top)

Visual representation of the sequence:

  • Time axis with intervals
  • Step blocks showing active periods
  • Transaction arrows between steps
  • Generation density visualization

Step Configuration Panel (Left)

List of sequence steps:

  • Step name and ID
  • Associated model
  • Generation settings
  • Timing configuration

Transaction Panel (Right)

Parent-child relationships:

  • Transaction definitions
  • Linking strategies
  • Field mappings
  • Trigger conditions

Properties Panel (Bottom)

Sequence configuration:

  • Start/end times
  • Interval settings
  • Global parameters
  • Output options

Creating a Sequence

Starting a New Sequence

  1. Create New Tides

    File → New Sequence
or click "New Tides" tab

  2. Set Basic Configuration

    • Name: Descriptive sequence name
    • Start Time: Beginning of generation
    • End Time: End of generation
    • Interval: Generation frequency (ms)

Adding Steps

Method 1: Drag and Drop

  1. Drag model from project tree
  2. Drop onto timeline
  3. Position at desired time
  4. Configure generation settings

Method 2: Add Step Button

  1. Click "Add Step" in toolbar
  2. Select model file
  3. Configure step properties:
    Step ID: unique_identifier
Model Path: ./models/schema.json
Start Offset: 0 (ms from sequence start)
End Offset: 60000 (ms from sequence start)

Method 3: Context Menu

  1. Right-click on timeline
  2. Select "Add Step Here"
  3. Choose model and configure

Step Configuration

Basic Settings

  • Step ID: Unique identifier for referencing
  • Model Path: Path to DataFlood schema
  • Description: Optional description

Timing Configuration

  • Start Time: When to begin generating
  • End Time: When to stop generating
  • Start Offset: Milliseconds from sequence start
  • End Offset: Milliseconds from sequence start

Generation Settings

  • Documents Per Interval: Count per time interval
  • Generation Probability: 0.0-1.0 chance to generate
  • Weight: Relative frequency vs other steps
  • Entropy Override: Custom randomness level

Advanced Options

  • Tags: Metadata tags for filtering
  • Custom Properties: Additional fields to inject
  • Conditional Generation: Rules for when to generate

Configuring Transactions

Creating Parent-Child Relationships

  1. Add Transaction

    Click "Add Transaction" button
or drag arrow between steps on timeline

  2. Configure Transaction

    {
  "transactionId": "order_to_payment",
  "parentStepId": "orders",
  "childSteps": [
    {
      "stepId": "payments",
      "minCount": 1,
      "maxCount": 1,
      "additionalDelayMs": 5000
    }
  ],
  "triggerProbability": 0.95
}

Linking Strategies

InjectParentId Adds parent reference to child:

{
  "linkingStrategy": "InjectParentId",
  "parentIdField": "orderId",
  "parentReferenceField": "orderRef"
}

EmbedParentDocument Includes full parent in child:

{
  "linkingStrategy": "EmbedParentDocument",
  "embedField": "parentDocument"
}

FieldInheritance Copies specific fields:

{
  "linkingStrategy": "FieldInheritance",
  "inheritFields": ["customerId", "region", "tier"]
}

CorrelationMetadata Adds correlation tracking:

{
  "linkingStrategy": "CorrelationMetadata",
  "correlationField": "correlationId"
}

Timeline Operations

Zooming and Panning

  • Zoom In/Out: Ctrl+Scroll or zoom buttons
  • Pan: Click and drag timeline
  • Fit to View: Double-click timeline
  • Reset View: Home key

Step Manipulation

  • Move Step: Drag step block
  • Resize Step: Drag step edges
  • Copy Step: Ctrl+drag
  • Delete Step: Select and press Delete

Visual Indicators

  • Green: Active generation period
  • Yellow: Conditional generation
  • Red: Validation errors
  • Blue Arrow: Transaction relationship
  • Dotted Line: Conditional trigger

Advanced Features

Multi-Model Orchestration

Configure multiple models to work together:

Example: E-commerce Flow
1. Products (continuous, low rate)
2. Customers (burst at start)
3. Browsing (continuous, medium rate)
4. Cart Items (triggered by browsing)
5. Orders (triggered by cart)
6. Payments (triggered by orders)
7. Shipping (triggered by payment)

Weighted Generation

Control relative generation rates:

{
  "steps": [
    {
      "stepId": "premium_customers",
      "weight": 1.0,
      "documentsPerInterval": 1
    },
    {
      "stepId": "standard_customers", 
      "weight": 4.0,
      "documentsPerInterval": 2
    }
  ]
}

Time-Based Patterns

Create realistic timing patterns:

Business Hours Pattern

{
  "steps": [
    {
      "stepId": "day_shift",
      "startTime": "09:00:00",
      "endTime": "17:00:00",
      "generationProbability": 0.9
    },
    {
      "stepId": "night_shift",
      "startTime": "17:00:00",
      "endTime": "09:00:00",
      "generationProbability": 0.3
    }
  ]
}

Burst Pattern

{
  "intervalMs": 100,
  "addJitter": true,
  "jitterMs": 50,
  "burstProbability": 0.1,
  "burstMultiplier": 10
}

Custom Properties

Inject dynamic values into documents:

{
  "customProperties": {
    "timestamp": "{{now}}",
    "sequenceId": "{{sequence.id}}",
    "environment": "production",
    "region": "{{random:us-east,us-west,eu-west}}",
    "correlationId": "{{uuid}}"
  }
}

Testing and Preview

Test Generation

  1. Preview Mode

    • Click "Preview" button
    • Set preview duration (e.g., 1 minute)
    • Review sample generation

  2. Dry Run

    • Click "Test Run"
    • Generates without saving
    • Shows statistics and timing

  3. Validation

    • Click "Validate"
    • Checks all model paths
    • Verifies relationships
    • Reports issues

Generation Statistics

View generation metrics:

  • Documents per step
  • Transaction success rate
  • Timing distribution
  • Error summary

Export and Execution

Export Configuration

  1. Save Sequence

    File → Save Sequence
Choose .tide or .json format

  2. Export for CLI

    File → Export → For DataFlood CLI
Creates compatible configuration

  3. Export for API

    File → Export → For FloodGate API
Includes API-specific settings

Direct Execution

  1. Generate from Editor

    • Click "Generate" button
    • Set output options
    • Monitor progress
    • Save results

  2. Generation Options

    • Max Documents: Limit total generation
    • Output Format: JSON, CSV, JSONL
    • Include Metadata: Add generation info
    • Seed: For reproducibility

Examples

Example 1: Banking Transaction Flow

{
  "name": "Banking Daily Flow",
  "startTime": "2024-01-01T00:00:00Z",
  "endTime": "2024-01-01T23:59:59Z",
  "intervalMs": 1000,
  "steps": [
    {
      "stepId": "accounts",
      "modelPath": "./models/account.json",
      "documentsPerInterval": 1,
      "generationProbability": 0.01
    },
    {
      "stepId": "transactions",
      "modelPath": "./models/transaction.json",
      "documentsPerInterval": 5,
      "generationProbability": 0.8
    },
    {
      "stepId": "settlements",
      "modelPath": "./models/settlement.json",
      "documentsPerInterval": 1,
      "generationProbability": 0.0
    }
  ],
  "transactions": [
    {
      "parentStepId": "transactions",
      "childSteps": [{
        "stepId": "settlements",
        "minCount": 1,
        "maxCount": 1,
        "additionalDelayMs": 60000
      }],
      "triggerProbability": 0.1
    }
  ]
}

Example 2: IoT Sensor Network

{
  "name": "Sensor Data Stream",
  "intervalMs": 5000,
  "steps": [
    {
      "stepId": "temperature_sensors",
      "modelPath": "./models/temp_sensor.json",
      "documentsPerInterval": 10,
      "customProperties": {
        "sensorType": "temperature",
        "unit": "celsius"
      }
    },
    {
      "stepId": "motion_sensors",
      "modelPath": "./models/motion_sensor.json",
      "documentsPerInterval": 3,
      "generationProbability": 0.3
    },
    {
      "stepId": "alerts",
      "modelPath": "./models/alert.json",
      "generationProbability": 0.0
    }
  ],
  "transactions": [
    {
      "parentStepId": "temperature_sensors",
      "childSteps": [{
        "stepId": "alerts"
      }],
      "triggerProbability": 0.05,
      "triggerCondition": "parent.value > 30"
    }
  ]
}

Best Practices

Sequence Design

  • Start simple, add complexity gradually
  • Test with short time ranges first
  • Use meaningful step IDs
  • Document transaction purposes

Performance Optimization

  • Limit concurrent steps
  • Use appropriate intervals
  • Set reasonable document counts
  • Test with small batches

Relationship Modeling

  • Keep parent-child relationships logical
  • Use appropriate linking strategies
  • Test field inheritance
  • Verify transaction triggers

Testing Strategy

  • Validate before generating
  • Preview with small counts
  • Check timing patterns
  • Verify relationships

Keyboard Shortcuts

Shortcut Action
Ctrl+N New sequence
Ctrl+S Save sequence
Ctrl+Z Undo
Ctrl+Y Redo
Space Play/pause preview
Delete Delete selected step
Ctrl+D Duplicate step
Ctrl+G Generate
F5 Validate
Home Reset timeline view
+/- Zoom in/out

Troubleshooting

Models Not Found

  • Verify model paths are correct
  • Use relative paths from sequence file
  • Check file permissions

Transaction Failures

  • Ensure parent step generates documents
  • Verify child step configuration
  • Check trigger probability
  • Review linking strategy

Timing Issues

  • Verify start/end times
  • Check interval settings
  • Review step overlaps
  • Test with longer intervals

Performance Problems

  • Reduce documents per interval
  • Increase interval duration
  • Limit concurrent steps
  • Use CLI for large generations

Next Steps