Data copy from ADLS Gen2 and AWS S3 accounts into Google Storage bucket

  1. Azure: Root Management group → Management groups → Subscriptions → Resource groups → Resources
  2. AWS: Root → Organizations → Member Accounts → Resources
  3. GCP: Organization → Folders → Projects → Resources
  1. Storage Transfer Service — just for data copy, not for transformation
  2. Data Fusion — for building pipelines to copy and transform the data
  1. Create ADLS Gen2 storage account and upload a file into that.
  2. Create AWS S3 bucket and upload a file into that.
  3. Create Google Storage account.
  4. Create and execute Transfer service for copying the data from ADLS Gen 2 and AWS S3 storage accounts into Google Storage account.
  5. Create and execute Data Fusion pipeline for copying the data from ADLS Gen 2 and AWS S3 storage accounts into Google Storage account.

--

--

--

Databases enthusiast

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

What does the interpreter actually do?

From Analog to Tech

A layout of a project plan, hand written on a large notepad

How to ping from an Azure App service with TCPPING

Lessons learned for performance engineering in hybrid cloud (Azure, AWS, Google …)

Driving a Forklift with the New Input System in Unity 2021

SAP Tutorial: Complete CAP Java Part 5

5 Common Coding Bootcamp Myths Busted

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Bharath N

Bharath N

Databases enthusiast

More from Medium

Cloud Computing: No More a Costly Affair for Financial Services Organizations

How to calculate CIDR manually for IPV4

CIDR

Using Athena Views As A Source In Glue

How to Implement an ERP Solution that Will Turn Your Business into a Blooming Garden