Data Transformation with Edilitics | Structured, Secure, and Scalable Workflows#

The Transform module in Edilitics empowers teams to reshape, enrich, and operationalize data directly from integrated sources—without relying on external tools or manual processing. Whether you're preparing datasets for reporting, applying business logic, or generating feature-ready data for ML models, Edilitics ensures transformations are fast, governed, and reusable.

With support for no-code operations and Polars-based custom scripts, Edilitics enables highly flexible workflows while preserving consistency, schema control, and enterprise-grade governance.

Why Transformation Matters in Analytics Pipelines#

Data transformation is critical to deriving meaningful insights and operationalizing raw data—but legacy approaches often introduce friction:

• ❌ Manual Scripting and Dependency Chains

  Custom scripts across notebooks and BI tools increase maintenance and onboarding effort.

• ❌ Data Inconsistencies and Schema Mismatches

  Poor schema validation can break downstream logic or dashboards.

• ❌ Scattered Logic and Lack of Reusability

  Business logic embedded in dashboards or codebases leads to duplication and inconsistent outputs.

• ❌ Lack of Auditability

  Changes to transformation logic go undocumented, risking compliance and oversight.

How Edilitics Solves It#

The Transform module provides a governed, centralized environment to author, schedule, and monitor transformations:

• 🧩 No-Code Operations

  Apply transformations like filtering, joins, calculated fields, pivoting, and casting using point-and-click logic.

• 🐍 Polars-Based Custom Scripting

  Embed complex logic using Python’s Polars DataFrame library—built for performance and analytical expressiveness.

• 📜 Schema Preservation

  Transformation logic is applied only after schema validation using sample previews.

• 🗓️ Built-in Scheduling and Fail-Safes

  Configure timing, pause on errors, and auto-resume with Scheduler & Advanced Settings.

• 🧾 Execution Monitoring & Run Logs

  Track outcomes, runtimes, and data changes with full traceability (Run History Logs).

Supported Transformation Workflows#

Setting Up a Transformation Workflow in Edilitics#

Create or Reuse Integrations#

• Ensure a data source and destination are connected via the Integrate module.

• These may be the same system or different platforms.

Access the Transform Module#

• Go to the Transform tab from the main dashboard.

• Click “New Transformation” to begin.

Select Source and Destination#

• Choose your data source and destination integrations.

• Only integrations that support transformations will appear.

📌 If the source and destination differ, Edilitics auto-validates type compatibility during mapping.

Choose Base Table and Preview Data#

• Select a base table from the source to build the transformation logic on.

• Edilitics loads a sample of up to 5,000 rows for you to explore and validate.

🔒 Sample data is encrypted, temporary, and deleted automatically after transformation setup or after 24 hours if unsaved.

Apply Transformation Logic#

✅ No-Code Operations#

Choose from over a dozen prebuilt operations like:

• Filter, Sort, Rename

• GroupBy + Aggregation

• Add Calculated Column

• Cast Types, Merge Tables

• Join or Union

📖 See full operation list: Introduction to No-Code Operations

🐍 Custom Python (Polars) Scripts#

For complex transformations, add a custom step and use Polars syntax to define logic.

📌 Each script is sandboxed, validated, and version-tracked per execution.

🔁 Hybrid Workflows#

Combine both no-code and scripted steps in any sequence. Operations are executed in order—modifications to earlier steps require deleting downstream steps to preserve integrity.

Add Workflow Metadata#

• Name your transformation clearly (e.g., Monthly_PNL_Preparation)

• Add a short description for team visibility and governance.

Schedule the Transformation#

Control when your transformation executes:

Advanced Settings#

Configure operational safeguards:

• Pause on Failure – Auto-suspend after consecutive failed runs

• Schema Change Detection – Pause or continue if source schema changes

• Table Lock Awareness – Retry or skip execution if upstream tables are unavailable

All of the above are configurable in the Scheduler & Advanced Settings interface.

Monitoring & Managing Transformations#

🧾 Run History and Logs#

Track each execution across metrics:

• Status (Success, Failed, Skipped)

• Runtime duration

• Row-level stats and logs

• Execution timestamp and triggering user

(See Run History)

⚙️ Edit and Maintain Workflows#

• Add new operations only at the end

• Delete middle steps only by removing downstream steps

• Edit metadata or schedule from the dashboard

🛠️ Manual Execution or Pause#

• Run transformations on-demand

• Pause during source maintenance or validation periods

👥 Collaboration and Access Controls#

• Share transformation workflows with team members

• Editors can trigger or monitor

• Owners/admins can modify logic, delete, or reshare (Manage Access)

Security, Governance, and Auditability#

• 🔐 Secure Execution – All credentials encrypted at rest and decrypted only during runtime

• 📜 Audit Logging – Every run, edit, and trigger is recorded with user metadata

• ✅ Enterprise Compliance – Supports SOC 2, GDPR, and internal data governance practices

• 📈 Compute Reporting – Track runtime cost and usage in the Compute Dashboard

Built for Structured, Scalable, and Secure Transformations#

Edilitics helps you move from ad hoc scripts and manual notebooks to scalable, governed workflows—where every transformation is auditable, testable, and reusable.

With point-and-click operations, built-in validation, and support for Python-powered logic, the Transform module ensures your data is always analytics-ready, pipeline-safe, and audit-compliant.