s4cmd
s4cmd is a Python-based, multi-threaded S3 CLI that's significantly faster than s3cmd for bulk operations. It works with Filebase via standard environment variables and the --endpoint-url flag.
Install
pip install s4cmd
Configure
s4cmd reads AWS-style credentials from environment variables or ~/.aws/credentials:
export AWS_ACCESS_KEY_ID="YOUR_FILEBASE_KEY"
export AWS_SECRET_ACCESS_KEY="YOUR_FILEBASE_SECRET"
Common commands
# List buckets
s4cmd --endpoint-url=https://s3.filebase.io ls
# Upload a file
s4cmd --endpoint-url=https://s3.filebase.io put ./photo.jpg s3://my-bucket/
# Download
s4cmd --endpoint-url=https://s3.filebase.io get s3://my-bucket/photo.jpg ./
# Sync directory
s4cmd --endpoint-url=https://s3.filebase.io sync ./local/ s3://my-bucket/folder/
# Bulk delete with prefix
s4cmd --endpoint-url=https://s3.filebase.io del s3://my-bucket/old/ --recursive
To save typing the endpoint, use a wrapper:
alias s4=' s4cmd --endpoint-url=https://s3.filebase.io '
Multi-threaded transfers
s4cmd's main appeal is parallelism. Tune the number of threads with --num-threads:
s4cmd --num-threads 16 --endpoint-url=https://s3.filebase.io \
sync ./big-folder/ s3://my-bucket/folder/
For thousands of small files, this can be 10× faster than s3cmd.