Data Ingestion is not just data acquisition, It’s about prepping the data for curation
Data Lakes require huge amounts of data to be processed, in some cases into the Petabytes, requiring thousands of pipelines to be created. Traditional ETL based tools are time consuming & expensive to use. Modak’s unique & proprietary technology dramatically reduces the time, complexity & risk to automatically generate data pipelines at scale, reducing the time to create a new pipeline from hours / days to less than a minute.
Modak uses metaprogramming approach to generate the code for ingestion pipelines, using the metadata captured by Data Spiders.