How IDMC in Serverless Mode Drives Productivity and Scale for Azure Users

High-performance data integration is the key to delivering a seamless flow of high-quality data for various analytics and artificial intelligence (AI) use cases in any modern business. But an increasingly complex data landscape — increasingly high data volumes, in even more formats, from multiple cloud and on-premises sources — stretches data management infrastructure and capabilities to the maximum.

To cope with these challenges and roadblocks that negatively impact the success of analytics and AI initiatives, disruptive innovations have occurred in various aspects of modern data management. For example, data storage has evolved from Excel sheets to data lake houses, and data integration has progressed from complex pro-code to low-code and now to no-code data pipelines that virtually anyone can use. These innovations have improved productivity, optimized costs and maximized performance across the data management cycle.

Among these game changers is serverless, a cloud deployment mode that not only elevates data engineer productivity but also optimizes costs and elevates flexibility in an unpredictable business environment, at any scale. 

Serverless is a cloud architecture that allows data engineers to build and run applications without worrying about setting up or managing the underlying servers, virtual machines (VMs) or containers.

Serverless is a “mode” of runtime deployment or functioning. When you choose serverless mode, you opt to free yourself from the administration and logistics of infrastructure management, forecasting data management workloads or firefighting spikes/dips in a dynamic market. Instead, you can shift your attention to more strategic solutions-focused work.

Higher Data Engineer Productivity

Servers are still used for running applications, but you, as the data engineer, no longer need to interact with, manage, optimize or control them and can focus instead on designing optimized data pipelines required to generate insights for critical decision-making.

Cost Optimization

Data teams are under constant pressure to optimize or reduce data management costs, while also improving and innovating data outcomes. Going serverless can help because it:

• Eliminates the effort and cost of maintaining infrastructure for data integration with no clusters or software to manage while providing a fully scale-out environment with built-in elasticity.
• Optimizes costs by being responsive to unpredictable data volumes and compute loads during high growth stages.
• Allows you to only pay for resources and compute power used with consumption-based pricing, unlike traditional models where you may pay for idle capacity. 

Scalability, Flexibility and Responsiveness

As a cloud-native architecture, serverless enables you to take advantage of the full flexibility of the cloud. Advanced serverless deployments offer:

• Built-in elasticity: Auto-scaling resources up for new jobs or down to zero during idle time, based on the current workload, eliminates the need for complex demand forecasting and resource planning.
• On-demand execution: Applications run when needed, eliminating the expense of paying for idle time.
• Auto-tuning: Built-in resiliency ensures high availability, automatic recovery and fault tolerance. 

The Informatica Intelligent Data Management Cloud (IDMC) for Microsoft Azure Native ISV Service already offers users the benefits of a secure agent-based cloud data integration runtime environment model on Azure.

Now, you can opt to deploy a serverless runtime environment model from within your Azure environment, which will free you from managing the secure agent and reap all the additional benefits of serverless.

With IDMC serverless mode, users can experience serverless processing and elastic scaling for enterprise workloads, while also enjoying the cloud-native microservices architecture that connects data consumers to data sources for AI-powered data integration, multi-cloud/multi-hybrid capabilities and a low-code/no-code execution. 

Figure 1: Serverless runtime created from Azure portal with management options highlighted.

With IDMC serverless, which natively integrates with Azure, you can create advanced serverless environments without ever leaving your Azure console and run event-driven code without ever needing to manage the underlying infrastructure.

Create a serverless runtime environment from the Azure portal and achieve unified and trusted data integration outcomes with all the infrastructure you need within the Azure portal itself. You can:

• Find, cleanse and ingest data into Microsoft Azure.
• Connect seamlessly with a wide range of data sources across multi-cloud platforms and on-premises systems for complex data integration use cases.
• Use drag and drop connectors to connect from source to targets for data transformation. Serverless eliminates the need for in-depth coding knowledge or hand-coding for data transformation.

In advanced serverless execution mode on Microsoft Azure, your applications and services will remain the same as when you deploy the secure agent, except you no longer need to install or manage the secure agent. IDMC serverless mode eliminates infrastructure maintenance and secure agent administration overhead while executing the data processing on Informatica’s secure infrastructure. With no further need to manage hardware or software, data engineers can focus on business logic and the faster, more efficient deployment of new data pipelines.

Unlike point solution providers, IDMC also seamlessly connects to related data management components such as data governance, master data management and security, so that system integration and technical debt never become a challenge as you scale data volumes and expand use cases.

Set up and subscribe to Informatica Intelligent Data Management Cloud – An Azure Native ISV Service from Azure Marketplace. Follow the documentation to create an Informatica organization and runtime environments for deploying your integration mappings.

Read the Microsoft blog to learn more.