AWS S3 CLI Uploads: cp vs sync vs mv Explained

Simple explanation

What is S3 in this context? Amazon S3 is object storage: a place to put files (called objects) organised inside containers called buckets. There is no real folder hierarchy. Each object has a key (its full name, including any slashes). A key like reports/2026/january.pdf looks like a folder path but is just a flat string. The part before the filename, reports/2026/, is called a prefix.

What does s3://bucket-name/prefix/file.pdf mean? This is an S3 URI. s3:// is the scheme. bucket-name is the bucket. prefix/file.pdf is the object key. When you upload a local file, you provide a local path on one side and an S3 URI on the other. The CLI figures out the direction from the URI scheme.

Why use aws s3 instead of aws s3api? The AWS APIs are powerful but verbose: s3api requires you to manage multipart uploads, handle pagination, and specify every option explicitly. The aws s3 high-level commands wrap that complexity. They handle recursive operations, parallel transfers, progress bars, and multipart uploads automatically. For everyday uploading, downloading, and syncing, aws s3 is the right tier.

Before you start

You need four things in place before any aws s3 command will work:

1. AWS CLI installed and configured. Follow the AWS CLI installation guide to get set up. Run aws --version to confirm it is installed.
2. Credentials available. Your CLI must know which AWS identity to use: a named profile, environment variables, or an attached IAM role. See the AWS CLI overview for how profiles and credential chains work. If you are working inside AWS, AWS CloudShell comes pre-authenticated.
3. Target bucket exists. You cannot upload to a bucket that does not exist. Create the bucket first with the console, Terraform, or aws s3api create-bucket.
4. IAM permission to upload. Your identity needs at minimum s3:PutObject on the target bucket/prefix. If uploads fail with AccessDenied, see the S3 access denied errors guide.

cp vs sync vs mv at a glance

How uploads work

The AWS CLI uses S3 URIs to determine direction:

• Local to S3: aws s3 cp ./file.pdf s3://my-bucket/file.pdf uploads from your machine.
• S3 to Local: aws s3 cp s3://my-bucket/file.pdf ./file.pdf downloads to your machine.
• S3 to S3: aws s3 cp s3://bucket-a/file.pdf s3://bucket-b/file.pdf copies entirely within S3’s infrastructure, not through your machine. No data transfer cost within the same region.

The key difference between cp and sync is state awareness. cp is stateless: it copies whatever you point it at, every time. sync checks the destination first. It compares object size and last-modified timestamp (not a checksum by default) and skips any file that already matches. This makes sync the better choice for anything you run more than once.

aws s3 cp

Use cp when you need to copy a specific file once, for example uploading an artifact, downloading a config, or making a point-in-time copy between two S3 locations.

# Upload a single file to S3
aws s3 cp report.pdf s3://my-company-assets/reports/report.pdf

# Download a file from S3 to the current directory
aws s3 cp s3://my-company-assets/reports/report.pdf ./report.pdf

# Copy between two S3 locations (transfer happens inside S3, not through your machine)
aws s3 cp s3://my-company-assets/reports/report.pdf \
          s3://my-company-archive/reports/report.pdf

# Upload an entire directory recursively
aws s3 cp ./logs/ s3://my-company-logs/app/ --recursive

# Upload with a specific storage class
aws s3 cp archive.tar.gz s3://my-company-archive/ --storage-class GLACIER

# Upload with custom metadata
aws s3 cp report.pdf s3://my-company-assets/reports/report.pdf \
  --metadata '{"author":"jane.smith","department":"finance"}'

aws s3 sync

sync is the better command any time you are transferring a directory and do not want to re-upload files that have not changed. It is the standard tool for deploying static sites, running nightly backups, and mirroring folders.

# Deploy a local build directory to S3 (only upload changed files)
aws s3 sync ./dist/ s3://my-company-site/

# Preview what sync would do without transferring anything
aws s3 sync ./dist/ s3://my-company-site/ --dryrun

# Sync and remove objects from S3 that no longer exist locally
aws s3 sync ./dist/ s3://my-company-site/ --delete

# Sync S3 back down to a local directory (useful for disaster recovery)
aws s3 sync s3://my-company-assets/backups/ ./local-restore/

# Sync only .html and .css files, exclude everything else
aws s3 sync ./dist/ s3://my-company-site/ \
  --exclude "*" \
  --include "*.html" \
  --include "*.css"

The --delete flag makes the S3 bucket a mirror of your local source. In a CI/CD context this is usually what you want: old files get removed when you delete them locally. Outside of deployments, use it carefully. It permanently removes S3 objects without a recycle bin.

aws s3 mv

aws s3 mv is copy-then-delete, not a true atomic move. It copies the object to the destination first, then deletes the source. If the copy succeeds but the delete fails, you end up with both copies. There is no rollback.

# Rename or move an object within the same bucket
aws s3 mv s3://my-company-assets/incoming/report.pdf \
          s3://my-company-assets/processed/report.pdf

# Move a local file to S3 (uploads it, then deletes the local copy)
aws s3 mv ./report.pdf s3://my-company-assets/reports/report.pdf

When to use it: Moving processed files along a workflow inside S3, for example from an incoming/ prefix to processed/ after a Lambda function handles them.

When not to use it: For large-scale reorganisation, prefer sync with --delete, which handles partial failures more gracefully. Avoid mv on critical files where losing the source before confirming the destination copy would be a problem.

Filtering with —exclude and —include

Both cp --recursive and sync support --exclude and --include to filter which files are transferred. The flags accept glob patterns and are evaluated left to right.

# Sync but exclude all log files
aws s3 sync ./dist/ s3://my-company-site/ --exclude "*.log"

# Sync only JavaScript files (exclude everything first, then re-include *.js)
aws s3 sync ./dist/ s3://my-company-site/ \
  --exclude "*" \
  --include "*.js"

# Exclude a subdirectory
aws s3 sync ./project/ s3://my-company-assets/project/ \
  --exclude ".git/*"

# Exclude multiple patterns
aws s3 sync ./dist/ s3://my-company-site/ \
  --exclude "*.log" \
  --exclude "*.tmp" \
  --exclude "node_modules/*"

Order matters. Filters are applied in the sequence you write them. --exclude "*" --include "*.js" means: exclude everything, then re-include JS files, so only .js files transfer. Reversing the order to --include "*.js" --exclude "*" would cause the final --exclude "*" to override the include and nothing would transfer at all.

Multipart uploads

For files over 8 MB, the AWS CLI automatically switches to multipart upload. The file is split into chunks that upload in parallel, then assembled at S3. If one chunk fails, only that chunk retries; not the whole file. For large files this is significantly faster than a single stream.

You can tune the thresholds in ~/.aws/config:

[profile default]
s3 =
  multipart_threshold = 64MB
  multipart_chunksize = 16MB
  max_concurrent_requests = 10

Raising multipart_threshold means smaller files use a single stream (simpler, slightly fewer API calls). Raising max_concurrent_requests speeds up large directory syncs by running more parallel transfers.

Output flags and scripting

By default, cp and sync print a progress bar to stderr and a summary of transferred files. In scripts and CI pipelines you usually want cleaner output.

# Suppress the progress bar but still print file-transfer lines
aws s3 sync ./dist/ s3://my-company-site/ --no-progress

# Print only errors (no progress bar, no per-file output)
aws s3 sync ./dist/ s3://my-company-site/ --only-show-errors

# Complete silence (no output at all, check the exit code instead)
aws s3 sync ./dist/ s3://my-company-site/ --quiet

These three flags are distinct:

• --no-progress removes the animated progress bar but keeps the per-file transfer lines. Good for log files where you want a record of what transferred.
• --only-show-errors suppresses everything except error messages. Useful for a clean CI log that only gets noisy on failure.
• --quiet suppresses all output including errors to stdout/stderr. Use this only when you are capturing the exit code and handling errors programmatically.

When to use each command

Public access warning