FAQs on Azure Data Factory Config Driven Framework

The configuration-driven approach moves the drivers of logic in our code from the code itself to a table reserved only for things we can configure. With config driven approach, you don’t need to modify the code every time to apply changes; you can plug in a new configuration to apply the changes.

• Development speed
  Minimize effort to load & transform data
  With loads of information available at your disposal, it’s easy to use and learn.
• Governance
  Building a new framework will help enforce the best practices and enterprise-level standards for better security and data governance policies.
• Identify changes quickly
  With the framework in place, developers can have source control and quickly identify the changes in config files.
• Reusable
  The framework is reusable; it’s not specified to one team; different teams can use the framework and scale on an enterprise level.”

Yes, it can handle ETL/ELT data migration with low code; AnblicksCDF was developed to address the present-day data integration challenges such as increased development time, operational inefficiencies, maintenance cost, and performance issues.

The tool is config-driven instead of code-driven. It has built-in support for RDMS, APIs, flat files, and semi-structured data sources like JSON or XML.

ADFCDF can migrate from OnPrem to Cloud, data stage, or legacy technology to Azure migration, version, and JDK upgrades.

ADFCDF is a config-driven framework. It supports all relational databases and Files(XML, JSON, Flat Files) (Oracle, SQL). Cloud Related databases such as AWS, Oracle, Azure, etc.

Migration time would depend on the data size and the server capacity. ADFCDF can run all the tasks parallelly, which helps reduce the migration time.

Objects such as Database tables, Stored Procedures, Views, and Functions are configurable in the metadata table. It’s about time to add x number of objects into the metadata table.

Our certified Azure experts won’t be touching the existing target database. We point the ADFCDF to a new database that replicates the existing database, and customizations can be done based on your need. So there would be no impact on the existing systems, and both systems (old and new) can run parallelly. Once the replication is done without any issues, we can switch to the new system.

• Cost-effective solution with reduced maintenance effort
• Add new data sources, pipelines with little configuration or no input
• Improved performance with configurable Task parallelization
• Insightful, proactive operational dashboard to mitigate failures

With comprehensive logging and error handling, users can monitor the task details to ensure smooth BAU operations.

Azure Data Factory Config Driven Framework is a comprehensive error handling function that will capture all the failed tasks in a log table with an error message. Once the error is resolved, the task will restart from any point of failure without manual intervention.

We are Microsoft’s Gold Partner offering certified Azure professionals. Our best practices & reusable frameworks help you optimize Azure data factory development and execution configurations. We help you migrate from OnPrem to Cloud, data stage, or legacy technology to Azure migration, version, and JDK upgrades. Our deep Azure ETL design and big data expertise accelerate all your data and ETL/ELT initiatives with low maintenance and minimum costs.

ADFCDF’s configurable workflow will allow users to build custom programs based on the requirements. For Eg: We helped a client build a custom program that will automate the conversion of XML/JSON files to CSV.

Users can decrease their ETL development time halfway thanks to the metadata-driven framework and task reusability feature. However, when it comes to metadata-driven ETL development, there is no set formula to follow and no one-size-fits-all approach. This is why it’s critical to work with a development team like Anblicks that knows exactly what they’re doing.

We integrated the accelerator with PowerBI, a data visualization application to create interactive and out-of-the-box operational dashboards for efficient troubleshooting and error monitoring. With the help of the log table, users can create their operational dashboards.

Yes, Users can integrate ADFCDF with other data visualization applications like Kibana, Tableau, Looker, etc.

With our comprehensive logging and error handling mechanism, users can see the metrics in real-time.

Yes, you can add java, python, Unix, etc.

As a Microsoft gold partner, we aim at providing customers with the ability to solve disparate data integration challenges and transform their business operations with data-driven decision-making. Our certified Azure architects bring deep Azure design and big data expertise to help you accelerate all your data and ETL/ELT initiatives.

Anblicks config-driven framework is a Low Code Enterprise ETL/ELT Framework powered by Azure ETL data factory. Its comprehensive logging and error handling mechanism allows parallel task execution to reduce development time. The out of the box operational dashboard helps in efficient troubleshooting and monitoring of failed tasks.

Please refer to the below picture to learn more about the features and benefits of Azure Data Factory CDF.