Using AzCopy Command Line Tool

00:00

In this next lesson, we're gonna take a look at another tool called a Z copy and how we can use it to upload data into our storage account services.

00:10

The objectives include we're gonna take a overview of the ese copy utility. We'll look at some authentication options and then finally jump out to a demo and take a look at a couple scenarios where we can upload our data.

00:22

So first, what is a Z copy? It is a command line tool that you can use to copy, block or file data into our storage accounts. So note that this cannot be used for our table or accuse services. This is just four blobs containers and our files.

00:39

Ese copy is Justin Execute Herbal, and it's available for Windows or Mac in a zip file or Lennix as a tar file. So there's nothing to install. It's literally just a command line utility that you can run without any installation.

00:52

In fact, we've already seen it at work. Storage Explorer actually uses a Z copy underneath the covers for its data transfer operations, and here's a quick screen shot from our previous demo where we're uploading those log fouls. And if you notice here on the right, there is a link here to copy the easy command to the clipboard.

01:08

That Storage Explorer just used to upload our data.

01:15

So just like in Storage Explorer, we had a couple of different options to authenticate to the service to get access to our storage account.

01:22

And that holds true for easy copy as well. Let's take a look at its authentication options.

01:26

Ese copy can use either azure active directory or a shared access signature or s A S.

01:33

And as we saw in our last demo when we were using Storage Explorer, it's generating those shared access signatures for us on the fly and then using ese copy in the background to perform the data operations.

01:45

Ese copy can only be used with our container blob storage or our file share services, but each one uses different authentication for a container storage. We can use azure active directory or shared access signatures.

01:57

But if we're gonna be working with our foul share storage, it only uses shared access signatures, and we'll take a look at each of these in our demo.

02:06

The benefits to using azure active Directory is, you only need to provide your credentials once. Once you log in, you'll have access to all your services and be able to use the command line tool to upload or download data,

02:16

whereas with using a shared access signature or S A S, you have to append the token each time. And if the token you have will perform the operations you need, you'll have to go generate another one.

02:28

You'll also need to be aware of our back authorization, and this is something we looked at in our last module. Just because I'm the owner of a storage account doesn't necessarily mean I'll have access to perform data operations inside the container or the foul share. And that's gonna hold true here, since we can use azure active directory to connect to our container storage.

02:47

And we'll see this in the demo where initially I won't have permissions.

02:52

We'll have to go in and add them in order for me to upload data into our container.

02:59

Next, let's take a look at what the Azure 80 authentication looks like.

03:02

First inside the command line will run the A Z copy utility and specify the log in parameter after it.

03:08

This will put up some text into our command window, saying we need to go to Microsoft dot com slash device, log in and enter this code in order to authenticate. Once you navigate to that page and enter in the code,

03:21

it's going to ask for you to pick an account to sign into your azure storage. Izzy copy on the remote device. Once I select my name and complete the sign in,

03:30

we go back to the command line is going to say Loggins succeeded at the bottom.

03:35

And finally, once we're done uploading or downloading data, we simply run ese copy again and specify. Log out.

03:44

Next, Let's take a look at our shared access signature authentication

03:46

here. First, I have an example of what Storage Explorer generated in order to perform its data operations,

03:53

and here in the pink is three shared access signature. It generated for me to perform the action from our last demo. So I know this is a lot of information right here to look at will bring it down a little bit better so you can see how the shared access signature integrates.

04:08

This is a simpler example where we're just running the easy copy command line utility were saying, We're going to copy files.

04:14

I'm specifying the log file on my local system and then pasting in the shared access signature. Once that starts running and authenticates weaken, see it, upload the files here and give a status every couple of seconds of how maney it's done and the percentage complete. And then the number of transfers completed and the total bytes transferred.

04:33

That does it for some of our slides and our demo. We're going to log into a Z copy using Azure Active Directory, and then we'll upload files using by active director credentials.

04:44

Well, then generate a SAS token and upload files into a different container using it.

04:50

Let's jump over to our demo now.

04:55

So here in the demo, I first wanted to start off at the getting started with a Z copy documentation page. If you scroll down here to download ese copy, here are the individual download leaks for Windows, Linux and Mac OS to download the command line utility.

05:09

I've already done this and extracted it out to my demo folder,

05:13

so let's switch over to our azure portal inside RJB t 2020 storage account. Now let's go into containers.

05:19

I've created two new containers here for a demo. First the easy copy. Azure A D

05:26

ese. Copy s A s.

05:28

I just want to go into these real quick and show you that there is no files in them or blobs right now as we're gonna be uploading them in our demo.

05:35

First, let's go back to our A Z copy. Azure A D. I'll click the three dots to bring up context menu and go into container properties.

05:43

And I'm going to copy this your l here to the container and save it for later.

05:47

Next. Let's jump over to my Windows terminal here

05:50

first. I just want to show I already have the easy copy utility extracted here in my folder.

05:56

So, first, let's get started by logging into azure active directory.

06:00

Well, Renee, Z copy with the log in parameter,

06:02

I will select and copy our authentication code. Here.

06:08

Let's switch back to the browser. I'll open a new tab and navigate to Microsoft dot com slash device log in

06:15

here. I'll paste in the code from our windows terminal.

06:18

I'll select my account to log into a Z copy says it's successful. I'll go ahead and close out. This tab will switch back to my Windows terminal

06:29

and we can see Log in. Has succeeded.

06:30

Saleh were logged in. Let me clear this screen.

06:33

Let's go ahead and try to copy data into our new container.

06:38

Here I'll run the A Z copy utility. I'll add the copy parameter

06:44

and in double quotes. I'll put the path out to our Log Files folder, and I'll put a wildcard here at the end to say I want to upload each file in this folder into the root of the container.

06:55

I'll paste in the URL to the container and go ahead and press enter

07:00

and we'll see. We're actually going to get a failure. Here, let me scroll up and show you the air message

07:05

says for 03 This request is not authorized before in this operation.

07:11

So this is what I mentioned back in the slides. Even though I'm an owner on this container, I don't have permission to upload using Azure Active Directory.

07:20

Let's go back to our container and go into access control.

07:25

If I take a look under role assignments and scroll down. My account has storage Blob data Reader, but it doesn't have contributor or owner, which is required to upload data into the container.

07:36

So for this specific container, let's click on the plus sign was Click on the add role assignment,

07:44

and I'm going to select the Storage Blob Data Contributor role.

07:48

We'll search for my name,

07:50

select my user account and click on Save.

07:55

And if we let this refresh and scroll down, we can see on this particular container. I now have date a contributor role for it

08:01

so it can take up to five minutes for this are back role to take effects. I'm gonna pause the video and we'll come back and try it again in a couple of minutes.

08:11

All right, I've waited a good 45 minutes here. Let's go and clear this screen. I'll bring back our old command here using the up arrow, and we'll try it again.

08:22

You can see the authorization is successful this time now that I have contributor access onto the container,

08:28

so this is going to start scrolling through. It's going to initiate our connection and start uploading our files

08:33

and let's go and speed this up and finish the data Upload.

08:37

There we go. All 50 of my log files are uploaded. And just under a minute.

08:41

And if we switch back into the overview of our container and let it refresh, we can see Oliver. Objects are now uploaded into the container.

08:50

So next we need to upload data into our A Z copy s A s container using a shared access signature.

08:58

Let's go back into the overview of containers and go under settings and shared access. Signature.

09:03

Since I'm just uploading data into our blob's service, I'm gonna uncheck these other options, but leave everything else at the default.

09:09

Let's go ahead and generate our S A S and connection string, and I'm gonna go down here and copy the Blob's service s a s, your l

09:16

Let's switch back to our Windows terminal and I'm actually going to log out from our active directory just so I can prove that we're not using those credentials to upload. It's good and clear the screen

09:26

analysts attempt to upload data into this other container using our shared access signature. Again, I'm going to run a Z copy with a copy parameter

09:35

going to specify are Log files folder and all the files in it.

09:39

And next I'm going to paste in the full year l inside of double quotes.

09:45

Let's go ahead and try to run this

09:46

and you see, I'm gonna get error is going to say I can't transfer individual files into the root of containers. I need to specify the actual container or folder that I'm gonna upload it in.

09:56

So this is easily remedied. If I up arrow and I go to the end of our blob, that core dot windows dot net

10:03

and after the slash, I put the name of our container, which is a Z copy S A s.

10:09

So all I'm doing here is specifying the actual container. I want to upload these in, Not there, brute of the container service.

10:16

Let's go ahead and try to run that again. And this time we can see it's successful. It's going to start uploading data just like it did in our previous example. Using azure active directory credentials.

10:26

I'm going to speed this up so we can get to the end of the upload

10:30

and it's complete. It uploaded 50 more items just under a minute.

10:33

So let's go back to our storage account.

10:35

Let me go back to overview

10:39

and containers.

10:39

Let's go into a Z copy s a s.

10:43

And here we can see all of our log files.

10:46

So again, in this demo, we used both azure active directory

10:50

and generated shared access signatures in order to use in order to authenticate against our container service to upload data using the A Z copy command line utility

11:00

that does it for a demo. Let's jump back to the slides and wrap this up.

11:05

That does it for a demo. I wanted to finish up with a quick quiz. Question. What authentication type can ese copy Use for file storage, and this is gonna be uploading to our foul share.

11:16

Ese copy. Can Onley use the shared access signature for file share storage containers can use both azure active directory or shared access signatures

11:28

that does. For this list where we had an overview of the ese copy tool, we took a look at our different authentication options

11:35

and we uploaded and downloaded data