Here I think we have the option to create a generic File Source type and use a Path parameter in source config to determine how to handle ingestion. Basically, something like:
if path startswith "s3://" do s3
else if path startswith "gs://" do gcs
else search local
Here I think we have the option to create a generic
FileSource type and use aPathparameter in source config to determine how to handle ingestion. Basically, something like: