Troubleshooting Guide

This guide covers common issues and solutions for all DataFlood Suite components.

General Issues

.NET Runtime Issues

Problem: "dotnet: command not found"

Solution:

1. Install .NET 9.0 SDK from Microsoft
2. Verify installation:

   dotnet --version

3. Ensure PATH includes dotnet:

   # Linux/macOS
   export PATH=$PATH:/usr/local/share/dotnet
   
   # Windows
   # Add to System PATH via Environment Variables

Problem: "The framework 'Microsoft.NETCore.App', version '9.0.0' was not found"

Solution:

1. Update to .NET 9.0:

   # Check current version
   dotnet --list-runtimes
   
   # Install .NET 9.0
   # Download from Microsoft website

File and Permission Issues

Problem: "Access to the path is denied"

Solution:

1. Check file permissions:

   # Linux/macOS
   ls -la filename
   chmod 644 filename  # For read/write
   
   # Windows
   # Right-click → Properties → Security

2. Run with appropriate permissions
3. Ensure output directory exists and is writable

Problem: "File not found" when file exists

Solution:

1. Use absolute paths instead of relative
2. Check working directory:

   pwd  # Linux/macOS
   cd   # Windows

3. Verify file extension (hidden extensions on Windows)
4. Check for special characters in path

DataFlood CLI Issues

Schema Generation Problems

Problem: "No JSON or CSV files found in directory"

Solution:

1. Verify directory contains .json or .csv files:

   ls *.json *.csv

2. Check file extensions are correct
3. Ensure files are valid JSON/CSV
4. Try with a single file first

Problem: "Invalid JSON in file"

Solution:

1. Validate JSON syntax:

   # Using jq
   cat file.json | jq .
   
   # Or online validator

2. Common JSON issues:
   • Missing commas between elements
   • Trailing commas (not allowed)
   • Unmatched brackets/braces
   • Unquoted keys

Problem: Generated schema has no statistical models

Solution:

1. Provide more sample data (minimum 10-20 samples)
2. Ensure sample data has variety
3. Check data types are consistent
4. Use import in DataFloodEditor for better control

Document Generation Problems

Problem: "Schema validation failed"

Solution:

1. Validate schema structure:

   DataFlood generate schema.json --count 1

2. Check for:
   • Valid property types
   • Logical constraints (min < max)
   • Valid regex patterns
   • Compatible format specifications

Problem: "Out of memory" with large generation

Solution:

1. Reduce document count:

   # Instead of
   DataFlood generate schema.json --count 100000
   
   # Use batches
   DataFlood generate schema.json --count 10000 --seed 1
   DataFlood generate schema.json --count 10000 --seed 2

2. Use --separate flag for individual files
3. Generate CSV instead of JSON (more memory efficient)
4. Increase heap size if needed

Problem: Generated values don't match expectations

Solution:

1. Check entropy settings:

   # More predictable
   DataFlood generate schema.json --entropy 1.0
   
   # More random
   DataFlood generate schema.json --entropy 4.0

2. Verify statistical models in schema
3. Test with smaller count first
4. Review constraints and patterns

Sequence Execution Problems

Problem: "Model file not found" in sequence

Solution:

1. Use relative paths from sequence file location
2. Or use absolute paths
3. Verify all model files exist:

   # List all model paths in sequence
   grep modelPath sequence.json

4. Check file permissions

Problem: "Invalid sequence configuration"

Solution:

1. Validate sequence structure
2. Ensure all step IDs are unique
3. Verify parent steps exist for transactions
4. Check time ranges are logical

DataFloodEditor Issues

Startup Problems

Problem: Application won't start

Solution:

1. Check .NET 9.0 is installed
2. Look for error logs:

   # Windows
   %APPDATA%\DataFloodEditor\logs
   
   # macOS
   ~/Library/Application Support/DataFloodEditor/logs
   
   # Linux
   ~/.config/DataFloodEditor/logs

3. Try running from terminal for error messages:

   DataFloodEditor

4. Check display requirements (GUI needs display server)

Problem: "Unable to load DLL 'libSkiaSharp'"

Solution:

1. Install SkiaSharp dependencies:

   # Ubuntu/Debian
   sudo apt-get install libfontconfig1
   
   # macOS
   brew install fontconfig
   
   # Windows - usually works out of box

Editor Issues

Problem: Can't add properties to schema

Solution:

1. Ensure parent is selected (object or array)
2. Check property name is unique
3. Verify schema is not read-only
4. Try right-click → Add Property

Problem: Drag and drop not working

Solution:

1. Ensure source and target are compatible
2. Check if property is locked
3. Try copy-paste instead (Ctrl+C, Ctrl+V)
4. Restart application if frozen

Problem: Generation from editor fails

Solution:

1. Validate schema first (F5)
2. Reduce document count (GUI limit: 10,000)
3. Check all required fields have models
4. Verify constraints are satisfiable

Import/Export Issues

Problem: CSV import creates wrong types

Solution:

1. Ensure CSV has header row
2. Provide more sample rows (>10)
3. Check for consistent data in columns
4. Manually adjust types after import

Problem: Can't save project

Solution:

1. Check disk space
2. Verify write permissions
3. Try Save As to different location
4. Check for invalid characters in filename

FloodGate API Issues

Startup Problems

Problem: "Port 5000 is already in use"

Solution:

1. Change port in configuration:

   # floodgate.config.yaml
   server:
     port: 5001

2. Or find and stop process using port:

   # Linux/macOS
   lsof -i :5000
   kill -9 <PID>
   
   # Windows
   netstat -ano | findstr :5000
   taskkill /PID <PID> /F

Problem: API returns 404 for all endpoints

Solution:

1. Verify API is running:

   curl http://localhost:5000/api/documentgenerator/health

2. Check URL format (include /api/ prefix)
3. Verify Swagger UI loads: http://localhost:5000/swagger
4. Check logs for startup errors

Request/Response Issues

Problem: "Request body too large"

Solution:

1. Increase limit in configuration:

   server:
     maxRequestBodySize: 104857600  # 100MB

2. Or reduce schema/document size
3. Use file upload endpoint instead of inline

Problem: "Timeout" on large generation

Solution:

1. Increase timeout:

   limits:
     requestTimeout: 600  # 10 minutes

2. Use smaller batches
3. Use async/download endpoints
4. Consider using CLI for very large batches

Problem: CORS errors from browser

Solution:

1. Enable CORS in configuration:

   api:
     enableCors: true
     corsOrigins:
       - "http://localhost:3000"
       - "https://yourapp.com"

2. Or use proxy in development
3. Check browser console for specific error

Serving Mode Issues

Problem: "Model not found" in serving mode

Solution:

1. Check file exists in Serving/Models:

   ls Serving/Models/

2. Include or omit .json extension consistently
3. Check for case sensitivity
4. Verify file is valid JSON

Problem: Can't list serving models

Solution:

1. Create Serving directories:

   mkdir -p Serving/Models
   mkdir -p Serving/Sequences

2. Add at least one model file
3. Check permissions on directories

Performance Issues

Slow Generation

Problem: Generation takes too long

Solutions:

1. Reduce complexity:

   • Simplify schema structure
   • Reduce nesting depth
   • Use fewer properties

2. Optimize models:

   • Reduce histogram bins
   • Simplify string patterns
   • Lower entropy values

3. Batch strategies:

   # Instead of one large batch
   DataFlood generate schema.json --count 100000
   
   # Use parallel smaller batches
   for i in {1..10}; do
     DataFlood generate schema.json --count 10000 --seed $i &
   done

Memory Issues

Problem: High memory usage

Solutions:

1. Use streaming:

   # FloodGate download endpoint
   curl -X POST "http://localhost:5000/api/serving/generate/Model/download" \
     -d '{"count": 100000}' \
     --output data.jsonl

2. Generate separate files:

   DataFlood generate schema.json --separate --count 1000

3. Use CSV format (more memory efficient than JSON)

4. Increase heap size:

   # Set max heap size
   export DOTNET_GCHeapHardLimit=2147483648  # 2GB

Validation Errors

Schema Validation

Problem: "Invalid schema structure"

Common causes and solutions:

1. Missing required fields:

   // Bad
   {
     "properties": { ... }
   }
   
   // Good
   {
     "type": "object",
     "properties": { ... }
   }

2. Invalid constraints:

   // Bad - min > max
   {
     "type": "integer",
     "minimum": 100,
     "maximum": 50
   }
   
   // Good
   {
     "type": "integer",
     "minimum": 50,
     "maximum": 100
   }

3. Invalid patterns:

   // Bad - invalid regex
   {
     "type": "string",
     "pattern": "[invalid("
   }
   
   // Good
   {
     "type": "string",
     "pattern": "^[A-Z][a-z]+$"
   }

Getting Additional Help

Diagnostic Steps

1. Enable verbose logging:

   # DataFlood CLI
   export DATAFLOOD_LOG_LEVEL=Debug
   
   # FloodGate API
   # In floodgate.config.yaml
   logging:
     level: Debug

2. Check system resources:

   # Memory
   free -h  # Linux
   vm_stat  # macOS
   
   # Disk space
   df -h
   
   # Process info
   top

3. Capture error details:

   • Full error message
   • Stack trace if available
   • Steps to reproduce
   • Sample files (if shareable)

Log Locations

• DataFlood CLI: Console output or test-run folder
• DataFloodEditor:
  • Windows: %APPDATA%\DataFloodEditor\logs
  • macOS: ~/Library/Application Support/DataFloodEditor/logs
  • Linux: ~/.config/DataFloodEditor/logs
• FloodGate API: Console output or configured log file

Common Log Messages

Message	Meaning	Action
"Schema validation failed"	Invalid schema structure	Check schema syntax and constraints
"Insufficient samples"	Too few examples for statistics	Provide more sample data
"Constraint conflict"	Incompatible constraints	Review min/max, patterns
"Memory limit exceeded"	Out of memory	Reduce batch size
"Timeout exceeded"	Operation took too long	Increase timeout or reduce complexity

Prevention Tips

1. Always validate before generating
2. Start with small batches for testing
3. Keep backups of working schemas
4. Use version control for schemas
5. Document custom patterns and models
6. Monitor resource usage during generation
7. Test sequences with short time ranges first