Concat

The CONCAT feature in the Edilitics Transform module empowers users to seamlessly integrate tables across different databases without the need for coding. This robust tool supports vertical, horizontal, and diagonal concatenation, providing flexible and comprehensive data integration capabilities.

Step-by-Step Guide to Utilizing CONCAT#

Select the Type of CONCAT#

Choose between Vertical, Horizontal, and Diagonal CONCAT based on your data integration requirements.

Vertical CONCAT

• Ideal for appending rows from one table to another, useful for combining datasets with the same structure but different records.

• Validation: All column names and data types in both tables must match precisely.

Horizontal CONCAT

• Joins tables by adding columns side by side, beneficial for combining different attributes from related datasets.

• Validation: Both tables must have an identical number of rows.

Diagonal CONCAT

• Merges tables along both rows and columns, particularly useful for complex datasets requiring intricate integration.

• Validation: Both tables must have the same number of rows and columns with matching data types.

Select Database#

Choose a table from the current database you are working with, or select a different database that has been integrated.

Select Table#

Choose the table you wish to concatenate based on the selected type.

Handle Duplicate Values#

Choose how to manage duplicate values in the concatenated dataset:

Keep First

• Definition

  Retains only the first occurrence of each duplicate entry and removes subsequent duplicates.

• How it Works

  The dataset is processed in its existing order, and when duplicates are encountered, only the first one is kept.

Keep Last

• Definition

  Retains the most recent occurrence of each duplicate entry while removing earlier instances.

• How it Works

  The dataset is processed in its existing order, and when duplicates are encountered, only the last one is kept.

Drop All

• Definition

  Completely removes all occurrences of duplicate entries, leaving only unique values in the dataset.

• How it Works

  If a value appears more than once, all instances of that value are removed, meaning neither the original nor the duplicate remains.

Perform Checks and Adjustments#

Edilitics will perform multiple checks based on the type of CONCAT selected:

• Compatibility Check:

  Ensures tables can be concatenated.

• Column Mismatch Notification:

  Alerts users of any column mismatches and provides options to drop or adjust columns based on the type of join.

Submit#

Submit the CONCAT operation to integrate the selected tables.

Practical Applications#

Retail#

• Objective: Consolidate sales data from multiple regions.

• Scenario:

  • Vertical CONCAT: Merge sales records from different regional databases into a single table to analyze overall sales performance.

  • Handling Duplicates: Keep all records to ensure no sales data is lost.

Healthcare#

• Objective: Integrate patient data from various departments.

• Scenario:

  • Horizontal CONCAT: Join patient records from different departments (e.g., Cardiology, Neurology) to create a comprehensive patient profile.

  • Handling Duplicates: Drop duplicates to maintain a single record per patient.

Finance#

• Objective: Consolidate financial statements.

• Scenario:

  • Vertical CONCAT: Append quarterly financial reports into a single annual report.

  • Handling Duplicates: Keep first to retain the initial entry of financial transactions.

Manufacturing#

• Objective: Integrate production data with supply chain information.

• Scenario:

  • Diagonal CONCAT: Merge production logs with supply chain records to monitor end-to-end processes.

  • Handling Duplicates: Drop all to eliminate redundant records and focus on unique transactions.

Education#

• Objective: Combine student performance data from multiple semesters.

• Scenario:

  • Vertical CONCAT: Combine student grades from different semesters to track academic progress over time.

  • Handling Duplicates: Keep last to update the most recent performance records.

The CONCAT feature in Edilitics offers a robust, no-code solution for advanced data integration. With options for vertical, horizontal, and diagonal concatenation, users can flexibly merge tables across different databases. The ability to manage duplicates and handle column mismatches further enhances the feature's versatility, ensuring data integrity and making data integration both intuitive and accessible for all users.