Problem Description
The HMASynthesizer that is part of SDV Community is meant to be a general education resource for showcasing synthetic data for very small, simple datasets. It not really designed to handle complex data schemas, such as schemas with more than 1 level of depth and more than 5 tables. For this, it is recommended to purchase SDV Enterprise, which provides access to synthesizers that scale better by orders of magnitude (eg. what takes a couple hours for HMA takes a few minutes with SDV Enterprise).
Currently if you try to use HMASynthesizer with a complex schema, then SDV generates a PerformanceAlert explaining the details. For example:
PerformanceAlert: Using the HMASynthesizer on this metadata schema is not recommended. To model this data, HMA will generate a large number of columns. (10659 columns)
Table Name # Columns in Metadata Est # Columns
bill_of_materials 1 1
comp_adaptor 14 14
...
We recommend simplifying your metadata schema using 'sdv.utils.poc.simplify_schema'.
If this is not possible, please visit datacebo.com and reach out to us for enterprise solutions.
But it is easy for this to go unnoticed. Users may try fitting it only to find that it takes an unreasonable amount of time, when the solution all along is to upgrade to SDV Enterprise.
Furthermore, it is not always feasible to simplify the schema, especially for complex datasets where all tables are important/necessary.
Expected behavior
Instead of showing a PerformanceAlert, we should just turn it into an error outright. The error message should describe that SDV Community is not meant for complex schemas (re-iterating the above).
synthesizer = HMASynthesizer(metadata)
SynthesizerInputError: HMASynthesizer is not designed to handle a schema with many tables and connections.
Please use SDV Enterprise to model this schema.
SDV Enterprise provides access to synthesizers that can easily scale with the amount of data and complexity of your schema.
For more information, visit datacebo.com
Additional context
The trigger for this error should be the same.
Problem Description
The HMASynthesizer that is part of SDV Community is meant to be a general education resource for showcasing synthetic data for very small, simple datasets. It not really designed to handle complex data schemas, such as schemas with more than 1 level of depth and more than 5 tables. For this, it is recommended to purchase SDV Enterprise, which provides access to synthesizers that scale better by orders of magnitude (eg. what takes a couple hours for HMA takes a few minutes with SDV Enterprise).
Currently if you try to use HMASynthesizer with a complex schema, then SDV generates a
PerformanceAlertexplaining the details. For example:But it is easy for this to go unnoticed. Users may try fitting it only to find that it takes an unreasonable amount of time, when the solution all along is to upgrade to SDV Enterprise.
Furthermore, it is not always feasible to simplify the schema, especially for complex datasets where all tables are important/necessary.
Expected behavior
Instead of showing a PerformanceAlert, we should just turn it into an error outright. The error message should describe that SDV Community is not meant for complex schemas (re-iterating the above).
Additional context
The trigger for this error should be the same.