Add a manual check for referential integrity

[npatki]

Problem Description

For multi-table datasets, it's very important for the synthetic data to maintain referential integrity, meaning that a foreign key column should contain no unknown/broken references. Currently, the Diagnostic Evaluation runs a check for referential integrity.

However, it would be useful to add a function that runs through a few rows and provides another form of manual confirmation that referential integrity is met. This is useful for educational/illustrative purposes.

Expected behavior

Add a function called print_referential_integrity that manually checks a few rows and prints what it has checked.

from sdv.evaluation.utils import print_referential_integrity

print_referential_integrity(
  metadata=metadata,
  synthetic_data=synthetic_data,
  table_name='transaction',
  foreign_key_column_name='user_id',
  num_rows=10
)

Picking random transaction row: 1468732365
✅ Found user row! user_id: 86-25-19730100-15 

Picking random transaction row: 1988317645
✅ Found user row! user_id: 0-35-79964375-26 

Picking random transaction row: 1507984147
❌ Unable to find the linked user row

Parameters:

• (required) metadata: The metadata object
• (required) synthetic_data: A dictionary that maps each table name to a dataframe containing the synthetic data for it
• (required) table_name: A string containing the table name that has the foreign key to check
• (required) foreign_key_column_name: A string with the column of the foreign key to check
• num_rows: An int containing the number of columns to check; defaults to 10.

Returns: Nothing.

Behavior: This function should run a basic lookup and take the following steps:

1. Pick a random row (without replacement) from the table name provided. Print "Picking random <table_name> row". If the table name has a primary key, then print out the primary key's value of the chosen row.
2. For the given foreign key, perform a lookup to see if it exists in in the parent table (refer to the metadata to find the parent table).
   • If it is found, print: " ✅ Found <parent_table> row! <parent_primary_key_name>: <parent_primary_key_value>"
   • If it is not found, print: "❌ Unable to find the linked <parent_table> row"

Additional context

For a comprehensive calculation of referential integrity across all rows and all foreign keys across a multi-table schema, we recommend running the diagnostic evaluation instead of using this function. This function is for eduational/illustrative purposes only.