Run History Logs
Tracking execution details for replication and transformation workflows is essential for maintaining data integrity, diagnosing failures, and optimizing performance.
Edilitics' Run History Logs provide end-to-end execution telemetry for every replication and transformation job, enabling users to analyze workflow performance, troubleshoot failures, and manage compute consumption efficiently.
This guide explains how Run History Logs enhance workflow visibility, operational resilience, and governance compliance in Edilitics.
Comprehensive Run History: Tracking Workflow Execution
Run History Logs in Edilitics provide a full execution record of all replication and transformation jobs. Users can leverage these logs to:
✔ Monitor execution progress to ensure jobs complete as expected.
✔ Analyze failures with detailed diagnostics and corrective insights.
✔ Optimize workflow schedules based on execution trends.
✔ Track compute allocation and prevent inefficient resource consumption.
✔ Maintain an immutable execution record for governance and compliance.
Each workflow execution is assigned:
A Run Status – Indicates the current execution state.
A Run Type – Specifies whether the workflow was scheduled, manually triggered, or executed as part of a recurring schedule.
Run Status Classification
Each workflow execution (run) is assigned a status based on its lifecycle progression and outcome.
Replication & Transformation Flow Statuses
| Status | Description |
|---|---|
| Scheduled | The workflow is scheduled to run at a future date/time, as per user-defined scheduling settings. |
| Pending | The workflow is awaiting resource allocation before execution can begin. It may experience delays due to insufficient compute availability or resource queuing. |
| Running | The workflow is actively executing, processing data per the configured replication or transformation logic. |
| Success | The workflow completed successfully, processing all tasks without errors. |
| Failed | The workflow failed completely, preventing execution from reaching completion. Error logs capture the failure root cause. |
| Success with Exceptions (Replication Flows Only) | The workflow partially succeeded—some tables or records failed to transfer while others completed successfully. |
💡 Why Pending Status Matters: If a workflow remains in Pending for an extended period, users should check compute availability and adjust their resource allocation strategy if necessary.
Run Type Classification: How Workflows Are Initiated
Each workflow run is also tagged based on how it was triggered:
| Run Type | Description |
|---|---|
| Quick Run | A one-time execution, manually triggered to run only once. |
| Triggered | A manual execution, initiated by a user outside of its scheduled run. |
| Auto | A scheduled execution, automatically triggered as part of a daily or custom cron schedule. |
This classification allows users to distinguish between scheduled jobs, one-time test executions, and manually forced runs for better workload visibility.
Granular Run Details: Analyzing Workflow Execution
Users can access detailed execution logs by navigating to the Run History section in Edilitics. These logs provide:
Replication Logs
📌 Table-Level Analysis – Logs contain execution details per table, including:
- Processing time per table
- Compute resources consumed for data extraction & loading
📌 Error Logs – For replication failures, logs capture:
- Specific tables that failed
- Detailed failure diagnostics for quick troubleshooting
📌 Incremental vs. Full Load Tracking – Tracks whether:
- The job executed a full dataset sync
- The job processed only incremental data changes
Transformation Logs
📌 Operation-Level Analysis – Logs include execution details for each transformation step, such as:
- Processing time per transformation operation
- Compute resources consumed per step
📌 Error Logs – Captures issues such as:
- Transformation logic failures
- Schema mismatches or data type conflicts
Enterprise Admin Access: Workspace-Wide Run History & Compute Metrics
For Enterprise accounts, Run History Logs and compute usage metrics are centrally accessible to Admins under the Usage Summary section.
Global Visibility for Admins
-
Admins have workspace-wide access to all Run History Logs & execution metrics.
-
They can track compute usage across all flows and users to ensure resources are allocated efficiently.
Exclusive Access to Top 10 Longest-Running Flows
-
Admins receive a dedicated view of the top 10 replication and transformation flows by execution time.
-
This helps identify inefficient workflows and allows Admins to request optimizations from users when necessary to reduce excessive compute consumption.
Compute Purchase & Budget Control
- Since only Admins can purchase additional compute, they require a global overview of resource consumption to ensure workflows don’t halt due to compute depletion.
Automated Admin Notifications for Compute Depletion
- Admins receive real-time alerts when compute resources approach depletion thresholds:
| Notification Trigger | Alert Type |
|---|---|
| Compute balance ≤ 75% | In-app notification, Slack, and email notification |
| Compute balance ≤ 90% | In-app notification, Slack, and email notification |
| Compute fully depleted (0%) | Immediate in-app alert, Slack, and email |
💡 Why This Matters: These alerts enable Admins to take proactive actions—whether by purchasing more compute before workflows are interrupted or optimizing high-consumption jobs.
Aggregated Run Summaries: Macro-Level Performance Insights
Edilitics provides aggregated execution statistics to help teams analyze trends and manage workloads efficiently.
| Metric | Description |
|---|---|
| Total Runs | The cumulative count of executions across all workflows. |
| Success & Failure Rates | The proportion of successful vs. failed runs. |
| Total Compute Consumption | The total compute resources consumed across all runs. |
| Average Compute Usage per Run | Helps teams analyze resource efficiency over time. |
💡 Best Practice: Regularly reviewing aggregated execution insights helps teams optimize workflows, control costs, and improve operational efficiency.
Leveraging Run History Logs for Data Reliability
Edilitics' Run History Logs provide end-to-end workflow visibility, enabling teams to:
✔ Monitor execution performance – Analyze workflow trends, success rates, and compute consumption.
✔ Diagnose failures effectively – Use detailed logs to accelerate error resolution and maintain data integrity.
✔ Ensure operational continuity – Track workflow execution history to prevent disruptions caused by missed schedules or insufficient compute resources.
By leveraging Run History Logs, organizations can enhance workflow management, maintain data accuracy, and ensure scalable analytics pipelines. 🚀
Need Assistance? Edilitics Support is Here for You!