How to create a new bucket and add files to a specific folder using local stack (running in Docker)

First copy the contents of this docker-compose.yml file listed in the localstack repository into a local directory (make sure the file name is not changed).

Run up the container with docker-compose up. This will get localstack running locally.

At the time of writing this post (Oct 2020), the S3 service is available on port 4566 (i.e. http://localhost:4566)

Install the aws / localstack wrapper

Install awscli-local from here as a wrapper around localstack. This will allow you to use awslocal in your command line, which is very close to the aws command usage.

Create a new bucket

Create a new folder inside your newly created bucket

The easiest way I found to create a new folder is to use the put-object api.

UPDATE: You don’t need to create a folder like this ^ before adding a file. Everything in S3 is an object, and if you create a folder like this before creating a file, when you listObject on the bucket, you will see 2 items which might not be ideal to your use-case. You can add a file to a folder directly(without creating it first) using the following command:

Add a new file to it

We’re going to copy over a file into the nested here folder you just created.

Verify that your new bucket exists

Verify that the file you just added exists

Add tags to your file

Check the tags on a file

How to automatically create buckets and folders

If you’re using localstack in your project, and would like to create a certain bucket, or folders within that bucket when you start up localstack, you will need to modify the docker-compose.yml file I’ve listed above to include these values:

With DATA_DIR=/tmp/localstack/data , we’re asking localstack to use tmp/localstack/data as the directory to store data inside the container. Then, we create a volume to sync all the data between the docker container and our local machine using ./stubs/s3:/tmp/localstack . If Docker doesn’t automatically create this /stubs/s3 folder for you, you might need to create it yourself.

Now, with the above changes to the docker-compose.yml file, run up the container using docker-compose up. With the localstack container up, create a new bucket called test:

Now, create a folder called /data inside the bucket.

You will notice you have a new file within the /stubs/s3/data folder called recorded_pi_calls.json which contains 2 new lines, which looks something like this:

These are the lines that are responsible for creating the resources (buckets and folders that you need).

Important: Make sure you commit the recorded_api_calls.json file at this point to version control, so if others in your team (or you cloning this repo on another computer) will have access to these. In the same vein, make sure you’re not committing unnecessary changes to this file that do not reflect the state of the S3 buckets / folders you want when you spin up localstack.

I like making things and solving problems.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store