Amazon Web Services (AWS) Command Line Arguments (CLI)

In this post I am explaining about AWS CLI. Here we come to know about how to access the amazon account using CLI.

            We can access AWS account from Window's  machine and Linux machine.

Windows

To access AWS account in window we should  install aws cli.

For that we can download from the link (https://aws.amazon.com/cli/) or we can use https://s3.amazonaws.com/aws-cli/AWSCLI64.msi

if we run this .msi file, AWS CLI install in our local windows machine.

Then open command prompt:

Type "aws configure"

Then it will ask flllowing configurations 

AWS Access Key ID [None]: xxxxxxxxxxxx

AWS Secret Access Key [None]: ###########################

Default region name [None]: @@@@@

Default output format [None]: ENTER

Finally you  installed AWS CLI in your windows machine.

For testing you can use ”aws s3 ls”. It will give all s3 bucket list.

AWS EC2 machine:

In AWS ec2 machine it will give default in the position /usr/bin/aws.

So no need to install again in this machine. U can access directly from anywhere.

Here I explained some queries, we can use them for the fast processing.

“aws s3 ls “

It will give us list of all buckets.

“aws s3 ls s3://somutest/”

It will give us list of all documents and folders in this bucket.

“aws s3 cp . s3://somutest/”

It will transfer all documents from present local folder (should type .dot to represent present location) to s3. To transfer only one file we should mention file name like “aws cp abc.txt to s3://somutest/”

Here fallowed by some commands which are mostly used in real life.

aws s3 cp s3://somutest .

$ aws s3 sync . s3://my-bucket/MyFolder --acl public-read

Creating Buckets

$ aws s3 mb s3://bucket-name

Removing Buckets

$ aws s3 rb s3://bucket-name

$ aws s3 rb s3://bucket-name --force (Non empty bucket)

Listing Buckets

$ aws s3 ls

$ aws s3 ls s3://bucket-name/MyFolder

Managing objects

aws s3 mv s3://mybucket . --recursive

$ aws s3 cp file.txt s3://bucket-name/ --grants read=uri=http://acs.amazonaws.com/groups/global/AllUsers full=emailaddress=user@example.com

$ aws s3 sync <source> <target> [--options]

aws s3 sync . s3://somutest (It will upload all documents from local that folder to s3 bucket)

// Delete local file

$ rm ./MyFile1.txt

// Attempt sync without --delete option - nothing happens

$ aws s3 sync . s3://my-bucket/MyFolder

// Sync with deletion - object is deleted from bucket

$ aws s3 sync . s3://my-bucket/MyFolder --delete

delete: s3://my-bucket/MyFolder/MyFile1.txt

// Delete object from bucket

$ aws s3 rm s3://my-bucket/MyFolder/MySubdirectory/MyFile3.txt

delete: s3://my-bucket/MyFolder/MySubdirectory/MyFile3.txt

// Sync with deletion - local file is deleted

$ aws s3 sync s3://my-bucket/MyFolder . --delete

delete: MySubdirectory\MyFile3.txt

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

Local directory contains 3 files:

MyFile1.txt

MyFile2.rtf

MyFile88.txt

'''

$ aws s3 sync . s3://my-bucket/MyFolder --exclude '*.txt'

upload: MyFile2.rtf to s3://my-bucket/MyFolder/MyFile2.rtf

'''

$ aws s3 sync . s3://my-bucket/MyFolder --exclude '*.txt' --include 'MyFile*.txt'

upload: MyFile1.txt to s3://my-bucket/MyFolder/MyFile1.txt

upload: MyFile88.txt to s3://my-bucket/MyFolder/MyFile88.txt

upload: MyFile2.rtf to s3://my-bucket/MyFolder/MyFile2.rtf

'''

$ aws s3 sync . s3://my-bucket/MyFolder --exclude '*.txt' --include 'MyFile*.txt' --exclude 'MyFile?.txt'

upload: MyFile2.rtf to s3://my-bucket/MyFolder/MyFile2.rtf

upload: MyFile88.txt to s3://my-bucket/MyFolder/MyFile88.txt

@@@@@@@@@@@@@@@@@@@@@@@@@@@

// Copy MyFile.txt in current directory to s3://my-bucket/MyFolder

$ aws s3 cp MyFile.txt s3://my-bucket/MyFolder/

// Move all .jpg files in s3://my-bucket/MyFolder to ./MyDirectory

$ aws s3 mv s3://my-bucket/MyFolder ./MyDirectory --exclude '*' --include '*.jpg' --recursive

// List the contents of my-bucket

$ aws s3 ls s3://my-bucket

// List the contents of MyFolder in my-bucket

$ aws s3 ls s3://my-bucket/MyFolder

// Delete s3://my-bucket/MyFolder/MyFile.txt

$ aws s3 rm s3://my-bucket/MyFolder/MyFile.txt

// Delete s3://my-bucket/MyFolder and all of its contents

$ aws s3 rm s3://my-bucket/MyFolder --recursive

Aws EMR  Cluster

aws emr create-cluster \

  --ami-version 3.8.0 \

  --instance-type m1.xlarge \

  --instance-count 1 \

  --name "cascading-kinesis-example" \

  --visible-to-all-users \

  --enable-debugging \

  --auto-terminate \           

  --no-termination-protected \

  --log-uri s3n://quanttestbucket/logs/ \

  --service-role EMR_DefaultRole --ec2-attributes InstanceProfile=EMR_EC2_DefaultRole \