Data Solutions

My personal journey in this world full of data and continuous efforts to provide solutions to some of the technical challenges i've encountered so far.

Search This Blog

Home
Contact Me
About Me

Showing posts from December, 2019

View all

Posts

Informatica Cloud - Incremental Load (Data Ingestion)

Posted by Tushar Bhalla December 14, 2019

Informatica Informatica cloud

+

0

Get link
Facebook
Twitter
Pinterest
Email
Other Apps

Contact Form

Name

Email *

Message *

Labels

.Net
AWS
AX 2012
Azure
Business Central
C#
DevOps
GCP
Hadoop
Hive

Informatica
Microstrategy
NAV
Power BI
python
Spark
SQL
SSIS
Tableau

Show more Show less

Move file from one S3 folder to another using AWS Glue

Today we will learn on how to move file from one S3 location to another using AWS Glue Steps: Create a new Glue Python Shell Job Import boto3 library This library will be used to call S3 and transfer file from one location to another Write the below code to transfer the file Change the bucket name to your S3 bucket Change the source and target file path Run the job Check whether the file has been transferred to the target location or not. You are done! :)

Upload a file to Azure Blob Storage (ADLS Gen 2) using Python

Today we will learn on how to upload a csv file to Azure Blob Storage (ADLS) using Python Steps: Create a file to be uploaded to Azure Storage Create a python script Install Azure package from pip pip install azure-storage-file-datalake Import the azure module import os, uuid, sys from azure.storage.filedatalake import DataLakeServiceClient from azure.core._match_conditions import MatchConditions from azure.storage.filedatalake._models import ContentSettings Create a connection to your Azure storage in the python script service_client = DataLakeServiceClient(account_url="{}://{}.dfs.core.windows.net".format("https", 'Storage_account_name'), credential='Storage_Account_Key') Specify the container name file_system_client = service_client.get_file_system_client(file_system="your_container_name") Specify the directory in your Azure storage directory_client = file_system_client.get_directory_client("my-directory") Create a blank txt f

Report Abuse

Powered by Blogger

(C) TUSHAR BHALLA