EdenGuideReference

Templates

Templates in Eden provide a powerful mechanism for defining reusable data transformations, formatting rules, and processing logic. They serve as the foundation for consistent data manipulation across your organization, ensuring that complex transformations can be defined once and reused wherever needed.

The Power of TemplatesCopied!

Templates address several critical needs in data integration and processing:

  1. Data Standardization

    • Ensure consistent formatting across systems

    • Normalize variations in data representation

    • Apply business rules uniformly

  2. Complex Transformations

    • Convert between data formats (JSON, XML, CSV)

    • Reshape data structures (nesting, flattening)

    • Apply mathematical and statistical operations

  3. Code Reuse

    • Define transformations once, use them many times

    • Build libraries of common operations

    • Create hierarchies of templates for complex logic

  4. Governance and Compliance

    • Centralize transformation logic for auditing

    • Version control critical business rules

    • Ensure regulatory compliance in data handling

Eden templates use a flexible, expression-based language that combines declarative patterns with procedural capabilities, making them suitable for both simple mapping operations and complex business logic.

Template StructureCopied!

A typical template in Eden consists of:

  • Metadata: Name, description, version information

  • Input Schema: Defines the expected structure of input data

  • Output Schema: Defines the structure of processed results

  • Transformation Logic: The rules for converting input to output

  • Parameters: Configurable values that modify template behavior

  • Dependencies: References to other templates or resources

Templates support a variety of transformation approaches:

  • Mapping: Direct field-to-field transformations

  • Conditional Logic: If-then-else rules for processing

  • Aggregations: Summarizing or grouping data

  • Custom Functions: For complex calculations

  • External Calls: Integrating with endpoints or services

Managing TemplatesCopied!

  • Create a Template: POST /templates

    • Defines a new reusable transformation

    • Requires specifying the template structure, input/output schemas, and transformation logic

    • Validates the template syntax and structure before creation

    • Templates are immediately available for use after creation

    • Example request:

      POST /templates{  "name": "CustomerDataNormalizer",  "description": "Standardizes customer data across systems",  "version": "1.0",  "inputSchema": {    "type": "object",    "properties": {      "customerData": {        "type": "object",        "properties": {          "first_name": { "type": "string" },          "last_name": { "type": "string" },          "email": { "type": "string" },          "phone": { "type": "string" }        }      }    }  },  "outputSchema": {    "type": "object",    "properties": {      "customer": {        "type": "object",        "properties": {          "firstName": { "type": "string" },          "lastName": { "type": "string" },          "contactInfo": {            "type": "object",            "properties": {              "emailAddress": { "type": "string" },              "phoneNumber": { "type": "string" }            }          }        }      }    }  },  "transformationRules": [    {      "target": "customer.firstName",      "source": "customerData.first_name",      "transformers": ["trim", "capitalize"]    },    {      "target": "customer.lastName",      "source": "customerData.last_name",      "transformers": ["trim", "capitalize"]    },    {      "target": "customer.contactInfo.emailAddress",      "source": "customerData.email",      "transformers": ["trim", "lowercase"]    },    {      "target": "customer.contactInfo.phoneNumber",      "source": "customerData.phone",      "transformers": ["formatPhoneNumber"]    }  ],  "functions": {    "formatPhoneNumber": "function(phone) { /* phone formatting logic */ }"  }}

  • Get Template Details: GET /templates/{template}

    • Retrieves the complete definition of a template

    • Includes all metadata, schemas, and transformation logic

    • Provides usage statistics and version history

    • Useful for understanding or extending existing templates

    • Example response includes all template components plus:

      {  "id": "tmpl_cust_norm_01",  "name": "CustomerDataNormalizer",  "created": "2025-03-15T10:22:33Z",  "createdBy": "jane.doe@example.com",  "lastModified": "2025-04-02T14:17:05Z",  "lastModifiedBy": "john.smith@example.com",  "usage": {    "directReferences": 12,    "invocationsLast30Days": 4572,    "averageProcessingTime": 42  },  "versions": [    {      "version": "1.0",      "created": "2025-03-15T10:22:33Z",      "active": true    },    {      "version": "0.9",      "created": "2025-02-28T15:44:12Z",      "active": false    }  ]}

  • Update a Template: PATCH /templates/{template}

    • Modifies an existing template

    • Can update metadata, schemas, or transformation logic

    • Supports versioning to maintain compatibility

    • Changes are validated before being applied

    • Example update request:

      PATCH /templates/tmpl_cust_norm_01{  "description": "Enhanced customer data normalizer with international support",  "version": "1.1",  "transformationRules": [    {      "target": "customer.contactInfo.country",      "source": "customerData.country",      "transformers": ["normalizeCountryCode"]    }  ],  "functions": {    "normalizeCountryCode": "function(country) { /* country normalization logic */ }"  }}

  • Delete a Template: DELETE /templates/{template}

    • Permanently removes a template from the system

    • Requires confirmation for templates in active use

    • All versions of the template are removed

    • References in workflows must be updated

Using TemplatesCopied!

Templates become truly valuable when applied to actual data processing needs.

  • Run a Template: POST /templates/{template}

    • Executes the template with provided input data

    • Applies all transformation rules to generate output

    • Returns the processed result

    • Can be used for both synchronous and asynchronous processing

    • Example request:

      POST /templates/tmpl_cust_norm_01{  "customerData": {    "first_name": "jane ",    "last_name": "doe",    "email": "JANE.DOE@EXAMPLE.COM",    "phone": "555-123-4567"  }}

    • Example response:

      {  "customer": {    "firstName": "Jane",    "lastName": "Doe",    "contactInfo": {      "emailAddress": "jane.doe@example.com",      "phoneNumber": "+1 (555) 123-4567"    }  }}

  • Render a Template: POST /templates/{template}/render

    • Similar to running a template but provides additional context

    • Returns not just the output but execution details

    • Includes processing time, transformation steps, and validation results

    • Useful for debugging and understanding template behavior

    • Example response:

      {  "success": true,  "processingTimeMs": 35,  "inputValidation": {    "valid": true  },  "output": {    "customer": {      "firstName": "Jane",      "lastName": "Doe",      "contactInfo": {        "emailAddress": "jane.doe@example.com",        "phoneNumber": "+1 (555) 123-4567"      }    }  },  "transformationSteps": [    {      "rule": "customer.firstName",      "input": "jane ",      "transformations": [        { "name": "trim", "output": "jane" },        { "name": "capitalize", "output": "Jane" }      ]    },    // Additional transformation steps...  ],  "outputValidation": {    "valid": true  }}

API ReferenceCopied!

Create Template

  • URL: /templates

  • Method: POST

  • Headers Required: Authorization with JWT token

  • Request Body: Template constructor details

  • Success Response: 200 OK with template identifier

  • Example Response:

    {  "id": "tmpl_cust_norm_01",  "name": "CustomerDataNormalizer",  "version": "1.0",  "message": "Template created successfully"}

Get Template

  • URL: /templates/{template}

  • Method: GET

  • Headers Required: Authorization with JWT token

  • URL Parameters:

    • template: The ID of the template

  • Success Response: 200 OK with template details

  • Example Response: See the template details example above

Run Template

  • URL: /templates/{template}

  • Method: POST

  • Headers Required: Authorization with JWT token

  • URL Parameters:

    • template: The ID of the template

  • Request Body: Input data for the template

  • Success Response: 200 OK with transformed data

  • Example Response: See the template execution example above

Update Template

  • URL: /templates/{template}

  • Method: PATCH

  • Headers Required: Authorization with JWT token

  • URL Parameters:

    • template: The ID of the template

  • Request Body: Updated template details

  • Success Response: 200 OK with confirmation message

  • Example Response:

    {  "id": "tmpl_cust_norm_01",  "name": "CustomerDataNormalizer",  "version": "1.1",  "message": "Template updated successfully",  "updatedFields": ["description", "version", "transformationRules", "functions"]}

Delete Template

  • URL: /templates/{template}

  • Method: DELETE

  • Headers Required: Authorization with JWT token

  • URL Parameters:

    • template: The ID of the template

  • Success Response: 200 OK with confirmation message

  • Example Response:

    {  "message": "Template deleted successfully",  "id": "tmpl_cust_norm_01",  "affectedResources": {    "workflows": ["customer_onboarding", "data_synchronization"],    "templates": ["customer_full_profile"]  }}

Render Template

  • URL: /templates/{template}/render

  • Method: POST

  • Headers Required: Authorization with JWT token

  • URL Parameters:

    • template: The ID of the template

  • Request Body: Input data for the template

  • Success Response: 200 OK with detailed execution results

  • Example Response: See the template rendering example above

Template Advanced FeaturesCopied!

Eden templates support several advanced capabilities:

  1. Conditional Processing

    • Apply different transformations based on data values

    • Handle missing or null fields gracefully

    • Implement complex business rules

    • Example conditional rule:

      {  "condition": "customerData.type === 'business'",  "rules": [    {      "target": "customer.accountType",      "value": "corporate"    },    {      "target": "customer.taxInfo.requiresW9",      "value": true    }  ],  "else": [    {      "target": "customer.accountType",      "value": "personal"    }  ]}

  2. Nested Templates

    • Compose complex transformations from simpler ones

    • Create template hierarchies for organization

    • Reuse common transformation patterns

    • Example nested template usage:

      {  "target": "customer.address",  "template": "AddressFormatter",  "input": {    "addressData": "customerData.address"  }}

  3. Error Handling

    • Define custom error responses

    • Implement fallback transformations

    • Add validation with detailed error messages

    • Example error handling:

      {  "target": "customer.creditScore",  "source": "customerData.creditScore",  "validation": {    "type": "number",    "minimum": 300,    "maximum": 850  },  "onError": {    "action": "setDefault",    "value": null,    "logLevel": "warning",    "message": "Invalid credit score provided"  }}

  4. Performance Optimizations

    • Lazy evaluation of expensive operations

    • Caching of frequently accessed values

    • Parallel processing for independent transformations

    • Example optimization configuration:

      {  "performanceSettings": {    "caching": {      "enabled": true,      "ttl": 3600    },    "parallelization": {      "enabled": true,      "maxWorkers": 4    },    "lazyEvaluation": true  }}

  5. Custom Functions

    • Define specialized transformation logic

    • Implement complex calculations

    • Create domain-specific operations

    • Example custom function:

      "functions": {  "calculateTax": "function(amount, rate) { return amount * (rate / 100); }",  "generateReference": "function() { return 'REF-' + Date.now() + '-' + Math.floor(Math.random() * 1000); }"}

Common Template PatternsCopied!

Data Mapping Template

{
  "name": "CRM to ERP Customer Mapping",
  "description": "Maps customer data from CRM format to ERP format",
  "version": "1.0",
  "inputSchema": {
    "type": "object",
    "properties": {
      "crmCustomer": {
        "type": "object",
        "properties": {
          "id": { "type": "string" },
          "firstName": { "type": "string" },
          "lastName": { "type": "string" },
          "email": { "type": "string" },
          "phone": { "type": "string" },
          "companyName": { "type": "string" },
          "type": { "type": "string", "enum": ["individual", "business"] }
        }
      }
    }
  },
  "outputSchema": {
    "type": "object",
    "properties": {
      "erpCustomer": {
        "type": "object",
        "properties": {
          "customerCode": { "type": "string" },
          "customerName": { "type": "string" },
          "contactEmail": { "type": "string" },
          "contactPhone": { "type": "string" },
          "customerType": { "type": "string" },
          "taxCode": { "type": "string" }
        }
      }
    }
  },
  "transformationRules": [
    {
      "target": "erpCustomer.customerCode",
      "source": "crmCustomer.id",
      "transformers": ["prefixWithCRM"]
    },
    {
      "target": "erpCustomer.customerName",
      "condition": "crmCustomer.type === 'individual'",
      "expression": "crmCustomer.firstName + ' ' + crmCustomer.lastName"
    },
    {
      "target": "erpCustomer.customerName",
      "condition": "crmCustomer.type === 'business'",
      "source": "crmCustomer.companyName"
    },
    {
      "target": "erpCustomer.contactEmail",
      "source": "crmCustomer.email",
      "transformers": ["lowercase"]
    },
    {
      "target": "erpCustomer.contactPhone",
      "source": "crmCustomer.phone",
      "transformers": ["standardizePhoneNumber"]
    },
    {
      "target": "erpCustomer.customerType",
      "mapping": {
        "individual": "PERSON",
        "business": "COMPANY"
      },
      "source": "crmCustomer.type"
    },
    {
      "target": "erpCustomer.taxCode",
      "condition": "crmCustomer.type === 'business'",
      "value": "BIZ"
    },
    {
      "target": "erpCustomer.taxCode",
      "condition": "crmCustomer.type === 'individual'",
      "value": "IND"
    }
  ],
  "functions": {
    "prefixWithCRM": "function(id) { return 'CRM-' + id; }",
    "standardizePhoneNumber": "function(phone) { /* Phone standardization logic */ }"
  }
}

Data Validation Template

{
  "name": "Order Validation",
  "description": "Validates order data before processing",
  "version": "1.0",
  "inputSchema": {
    "type": "object",
    "properties": {
      "order": {
        "type": "object",
        "properties": {
          "orderId": { "type": "string" },
          "customerId": { "type": "string" },
          "items": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "productId": { "type": "string" },
                "quantity": { "type": "number" },
                "price": { "type": "number" }
              }
            }
          },
          "shippingAddress": {
            "type": "object",
            "properties": {
              "street": { "type": "string" },
              "city": { "type": "string" },
              "state": { "type": "string" },
              "zipCode": { "type": "string" },
              "country": { "type": "string" }
            }
          },
          "paymentMethod": { "type": "string" }
        }
      }
    }
  },
  "outputSchema": {
    "type": "object",
    "properties": {
      "validationResult": {
        "type": "object",
        "properties": {
          "valid": { "type": "boolean" },
          "errors": {
            "type": "array",
            "items": {
              "type": "object",
              "properties": {
                "field": { "type": "string" },
                "message": { "type": "string" },
                "code": { "type": "string" }
              }
            }
          }
        }
      },
      "processedOrder": {
        "type": "object"
      }
    }
  },
  "transformationRules": [
    {
      "initialize": {
        "validationResult.valid": true,
        "validationResult.errors": [],
        "processedOrder": "$.order"
      }
    },
    {
      "validation": {
        "field": "order.customerId",
        "rules": [
          {
            "type": "required",
            "message": "Customer ID is required",
            "code": "MISSING_CUSTOMER"
          },
          {
            "type": "pattern",
            "pattern": "^CUST-[0-9]{6}$",
            "message": "Customer ID format is invalid",
            "code": "INVALID_CUSTOMER_FORMAT"
          }
        ]
      }
    },
    {
      "validation": {
        "field": "order.items",
        "rules": [
          {
            "type": "minItems",
            "min": 1,
            "message": "Order must contain at least one item",
            "code": "EMPTY_ORDER"
          }
        ]
      }
    },
    {
      "forEach": {
        "array": "order.items",
        "as": "item",
        "index": "i",
        "validation": {
          "field": "item.quantity",
          "rules": [
            {
              "type": "minimum",
              "min": 1,
              "message": "Item quantity must be at least 1",
              "code": "INVALID_QUANTITY",
              "context": { "itemIndex": "i" }
            }
          ]
        }
      }
    },
    {
      "validation": {
        "field": "order.shippingAddress",
        "rules": [
          {
            "type": "required",
            "message": "Shipping address is required",
            "code": "MISSING_ADDRESS"
          }
        ]
      }
    },
    {
      "validation": {
        "field": "order.shippingAddress.zipCode",
        "rules": [
          {
            "type": "custom",
            "function": "validateZipCode",
            "args": ["order.shippingAddress.country", "order.shippingAddress.zipCode"],
            "message": "Zip code is invalid for the specified country",
            "code": "INVALID_ZIP_CODE"
          }
        ]
      }
    },
    {
      "validation": {
        "field": "order.paymentMethod",
        "rules": [
          {
            "type": "enum",
            "values": ["credit_card", "paypal", "bank_transfer", "cash_on_delivery"],
            "message": "Payment method is not supported",
            "code": "UNSUPPORTED_PAYMENT"
          }
        ]
      }
    },
    {
      "target": "processedOrder.totalAmount",
      "expression": "calculateTotalAmount(order.items)"
    },
    {
      "target": "processedOrder.processingTime",
      "expression": "Date.now()"
    }
  ],
  "functions": {
    "validateZipCode": "function(country, zipCode) { /* Country-specific zip code validation */ }",
    "calculateTotalAmount": "function(items) { return items.reduce((sum, item) => sum + (item.quantity * item.price), 0); }"
  }
}

Data Transformation Template

{
  "name": "PaymentDataEnrichment",
  "description": "Enriches payment data with additional information",
  "version": "1.0",
  "inputSchema": {
    "type": "object",
    "properties": {
      "payment": {
        "type": "object",
        "properties": {
          "id": { "type": "string" },
          "amount": { "type": "number" },
          "currency": { "type": "string" },
          "method": { "type": "string" },
          "timestamp": { "type": "string", "format": "date-time" }
        }
      }
    }
  },
  "outputSchema": {
    "type": "object",
    "properties": {
      "enrichedPayment": {
        "type": "object",
        "properties": {
          "id": { "type": "string" },
          "amount": { "type": "number" },
          "formattedAmount": { "type": "string" },
          "currency": { "type": "string" },
          "currencySymbol": { "type": "string" },
          "method": { "type": "string" },
          "methodDetails": { "type": "object" },
          "timestamp": { "type": "string" },
          "localTimestamp": { "type": "string" },
          "formattedDate": { "type": "string" },
          "status": { "type": "string" },
          "riskScore": { "type": "number" },
          "metadata": { "type": "object" }
        }
      }
    }
  },
  "transformationRules": [
    {
      "copy": {
        "from": "payment",
        "to": "enrichedPayment",
        "fields": ["id", "amount", "currency", "method", "timestamp"]
      }
    },
    {
      "target": "enrichedPayment.formattedAmount",
      "expression": "formatCurrency(payment.amount, payment.currency)"
    },
    {
      "target": "enrichedPayment.currencySymbol",
      "function": "getCurrencySymbol",
      "args": ["payment.currency"]
    },
    {
      "target": "enrichedPayment.methodDetails",
      "template": "PaymentMethodDetails",
      "input": {
        "method": "payment.method",
        "paymentId": "payment.id"
      }
    },
    {
      "target": "enrichedPayment.localTimestamp",
      "function": "convertToLocalTime",
      "args": ["payment.timestamp", "America/New_York"]
    },
    {
      "target": "enrichedPayment.formattedDate",
      "function": "formatDate",
      "args": ["payment.timestamp", "MMMM D, YYYY h:mm A"]
    },
    {
      "target": "enrichedPayment.status",
      "value": "processed"
    },
    {
      "target": "enrichedPayment.riskScore",
      "function": "calculateRiskScore",
      "args": ["payment"]
    },
    {
      "target": "enrichedPayment.metadata",
      "function": "generateMetadata",
      "args": ["payment"]
    }
  ],
  "functions": {
    "formatCurrency": "function(amount, currency) { /* Format currency with proper thousands separators and decimal places */ }",
    "getCurrencySymbol": "function(currency) { /* Return symbol for currency code */ }",
    "convertToLocalTime": "function(timestamp, timezone) { /* Convert UTC timestamp to local timezone */ }",
    "formatDate": "function(timestamp, format) { /* Format date according to specified format */ }",
    "calculateRiskScore": "function(payment) { /* Calculate risk score based on payment attributes */ }",
    "generateMetadata": "function(payment) { /* Generate additional metadata for payment */ }"
  }
}

Template Design Best PracticesCopied!

  1. Modularity

    • Build small, focused templates that do one thing well

    • Compose complex transformations from simpler ones

    • Avoid monolithic templates that are difficult to maintain

    • Example: Create separate templates for address validation, format conversion, and business rule application, then combine them in workflows

  2. Naming and Documentation

    • Use clear, descriptive template names

    • Document expected inputs and outputs thoroughly

    • Include examples of use in template descriptions

    • Add comments to complex transformation rules

    • Example naming convention:

      [Domain][Entity][Action]CustomerAddressValidationOrderDataTransformationProductCatalogFormatting

  3. Versioning

    • Create new versions for significant changes

    • Maintain backward compatibility when possible

    • Deprecate old versions gracefully

    • Document changes between versions

    • Example versioning approach:

      v1.0 - Initial versionv1.1 - Added support for international addressesv1.2 - Performance improvementsv2.0 - Major restructuring (breaking changes)

  4. Testing

    • Create test cases that cover edge conditions

    • Validate performance for large datasets

    • Test templates in isolation before integration

    • Document expected behavior with example inputs/outputs

    • Example testing strategy:

      1. Unit tests for individual transformations2. Integration tests with sample data3. Performance tests with production-like volumes4. Edge case tests for error handling

  5. Reusability

    • Design templates with reuse in mind

    • Parameterize behavior where appropriate

    • Share common templates across the organization

    • Create template libraries for standard operations

    • Example reusability enhancement:

      {  "parameters": {    "dateFormat": {      "description": "Format string for date formatting",      "default": "YYYY-MM-DD",      "type": "string"    },    "timeZone": {      "description": "Timezone for date conversions",      "default": "UTC",      "type": "string"    }  }}

Template Performance OptimizationCopied!

When working with large datasets or high-volume processing, consider these performance optimization techniques:

  1. Minimize Template Nesting

    • Deep nesting increases overhead

    • Flatten structures where possible

    • Combine templates that are always used together

  2. Optimize Custom Functions

    • Keep functions simple and focused

    • Cache expensive calculations

    • Minimize dependencies on external services

    • Use efficient algorithms and data structures

  3. Batch Processing

    • Process records in batches rather than individually

    • Consider streaming for very large datasets

    • Use pagination for large result sets

  4. Use Built-in Transformers

    • Prefer built-in transformers over custom functions

    • Built-ins are optimized for performance

    • Common operations have dedicated transformers

  5. Implement Caching

    • Cache reference data

    • Cache frequently used values

    • Set appropriate cache invalidation policies