How to sync a folder to AWS S3 | Next.js Blog Example with Markdown

I recently have so sync a static website to AWS S3. I was looking how to sync a folder to aws s3 by a simple comand. On this topic i will not cover how to create a S3 bucket. I will only focus on how to sync the folder on linux.

install pip3

You will need pip3 to install awsCli. it's not install by default on ubuntu. I'm not sure if pip2 will work for that. pip require python, but python3 is already installed on recent linux distributions. Install pip using package manager.

$ sudo apt-get -y install python3-pip

install aws cli

When pip is install, you can install aws cli.

$ pip3 install awscli --upgrade --user

Add was to the path

After installation, awscli will not be find on your path. You need to add it manualy. Open the .bashrc or the .zshrc if you are using zsh like me, and add the following export PATH command at the end of the file. add the following command on the .bashrc or the .zshrc at the end of the file

export PATH=~/.local/bin:$PATH

You need to reload the profile for take effect. Or close and reopen the terminal, it's work too.

$ source ~/.bash_profile

Now you can use aws command on your terminal. But it's not connected yet to your Amazone account.

Generate Iam key

Before configure awscli with your amazon account, you need to be sure to have a valid access key. For do that, connect to AWS IAM web interface, and select your user.

Then go on Security credential tab. Under the Access Key section, if you don't have a key already created, you should create one. Keep the file generated. It's contain your ID and Secret key, it's will be provided only one time. So if you loose it you will need to create a new one.

Add aws profile

Once you have your key, you can use them for link awscli to your account. You need your ID Key, Secret Key and region, even if S3 don't use region. For the format, keep json.

AWS Access Key ID [None]: <public name generated>
$ aws configure
AWS Secret Access Key [None]: <Secrete key provided>
Default region name [None]: us-west-2
Default output format [None]: json

Sync the folder

Now you can use aws for sync local folder with S3 bucket. The command use following syntax

aws s3 sync <options> <source folder> <destination folder>

as example for us case:

$ aws s3 sync . s3://bucket_name

For sync s3 to local, simply switch source folder and destination folder.

Conclusion

Now we can sync a folder with S3 bucket. As we can see, the final command is quite simple. you can explore different option available on the documentation. You can integrate this command on bash script for automatize the synchronization.