Informatica Cloud - Copy data from SQL to Azure ADLS

Today we will learn on how to load data from SQL database to Azure ADLS as file

  1. Purchase and enable Microsoft Azure data lake (ADLS) Gen 2 connector for Informatica 
  2.  Register your informatica app with Azure AD
    1. This app will act as a service principal 
    2. Note down the client secret and client ID
    3. Give appropriate permissions to Informatica app (in Azure)
      • Storage blob data contributor
      • Internet access to Informatica to talk to Azure ADLS
  3. Create an Informatica connection to Azure ADLS Gen 2
    1. Enter the following information:
      • Client ID
      • Client Secret
      • Tenant ID
      • Account Name (Azure Storage account name)
      • File System Name (Container name within a storage account)
      • Directory Path: /   (leave as default)

        4. Go to Informatica Cloud - Data Integration and create a new mapping task
    1. Drag source 
      1. Choose the connection in source as SQL (you must have created a SQL connection in Informatica to your database. if not, create a connection to SQL and choose here)
      2. Source type: Single object
      3. Object: <choose your SQL table name by clicking the select button>
     5.  Drag the target and point source to target

     6. Configure the target:
    • Connection: <ADLS connection name>
    • Target type: Single Object
    • Object: <click select and choose "Create new at run time">
      • Formatting Type: select "Flat"
    • Operation: Insert  
    • Note: Insert will always overwrite the above file. like delete and create new file with same name
    • In Target Field Mapping, map the source fields to the appropriate target fields





      7. You are done with the Informatica mapping now. 
      8.  Save and Run the informatica mapping.
      9. Once the run is successful, go to the appropriate ADLS storage account > Container
      10. Check that if informatica has created a new file and copied all the data from SQL in that file

    No comments:

    Post a Comment