Compare commits

...

11 Commits

Author SHA1 Message Date
Fábio D. Batista d576a7f752
Adds links to the badges 2020-12-02 18:32:38 -03:00
Fábio D. Batista a6d878638b
Updates badges 2020-12-02 17:40:09 -03:00
Fábio D. Batista 7bdb1a17b4
Fixes `S3_SYNC_FLAGS` documentation 2020-12-02 17:32:26 -03:00
Fábio D. Batista 082e5188aa
Merge pull request #14 from jmahowald/master
allow us to not delete files
2020-12-02 16:41:37 -03:00
Fábio D. Batista 014b843a2c
Apply suggestions from code review
Co-authored-by: Mirsal <mirsal@mirsal.fr>
2020-12-02 16:40:07 -03:00
Josh Mahowald 05dfe25557 allow us to not delete files 2020-02-13 19:04:19 -06:00
Fábio D. Batista 957697dc1a
Merge pull request #13 from tlex/master
Updates to latest alpine
2019-08-27 11:43:43 -03:00
Alex Thomae f3d579a0e9
Updates to latest alpine
* updates to alpine:3.10 from alpine:3.6
* updates from python2 to python3
* ensures that pip doesn't create a cache directory
2019-08-27 10:09:54 +02:00
Fábio D. Batista f19b3fa1ee
Added badges to the README 2019-08-07 10:46:45 -03:00
Fábio D. Batista 4d18ac3fc8
Merge pull request #12 from Arizona-Tecnologia/configurable-url
Allow configuration of the endpoint url
2019-08-07 10:40:57 -03:00
Mauricio H Nagaoka 9cb9e30ad7 Allow configuration of the endpoint url 2019-08-06 16:31:40 -03:00
3 changed files with 36 additions and 6 deletions

View File

@ -1,10 +1,11 @@
FROM alpine:3.6
MAINTAINER Elementar Sistemas <contato@elementarsistemas.com.br>
FROM alpine:3.10
label maintainer="Elementar Sistemas <contato@elementarsistemas.com.br>"
RUN apk --no-cache add bash py-pip && pip install awscli
RUN apk --no-cache add bash py3-pip && pip3 install --no-cache-dir awscli
ADD watch /watch
VOLUME /data
ENV S3_SYNC_FLAGS "--delete"
ENTRYPOINT [ "./watch" ]
CMD ["/data"]

View File

@ -1,5 +1,11 @@
# docker-s3-volume
[![Docker Build Status](https://img.shields.io/docker/build/elementar/s3-volume?style=plastic)](https://hub.docker.com/r/elementar/s3-volume)
[![Docker Layers Count](https://img.shields.io/microbadger/layers/elementar/s3-volume?style=plastic)](https://hub.docker.com/r/elementar/s3-volume)
[![Docker Version](https://img.shields.io/docker/v/elementar/s3-volume?style=plastic)](https://hub.docker.com/r/elementar/s3-volume)
[![Docker Pull Count](https://img.shields.io/docker/pulls/elementar/s3-volume?style=plastic)](https://hub.docker.com/r/elementar/s3-volume)
[![Docker Stars](https://img.shields.io/docker/stars/elementar/s3-volume?style=plastic)](https://hub.docker.com/r/elementar/s3-volume)
Creates a Docker container that is restored and backed up to a directory on s3.
You could use this to run short lived processes that work with and persist data to and from S3.
@ -49,6 +55,15 @@ Any environment variable available to the `aws-cli` command can be used. see
http://docs.aws.amazon.com/cli/latest/userguide/cli-environment.html for more
information.
### Configuring an endpoint URL
If you are using an S3-compatible service (such as Oracle OCI Object Storage), you may want to set the service's endpoint URL:
```bash
docker run -d --name my-data-container -e ENDPOINT_URL=... \
elementar/s3-volume /data s3://mybucket/someprefix
```
### Forcing a sync
A final sync will always be performed on container shutdown. A sync can be
@ -69,6 +84,14 @@ docker run -d --name my-data-container \
elementar/s3-volume --force-restore /data s3://mybucket/someprefix
```
### Deletion and sync
By default if there are files that are deleted in your local file system, those will be deleted remotely. If you wish to turn this off, set the environment variable `S3_SYNC_FLAGS` to an empty string:
```bash
docker run -d -e S3_SYNC_FLAGS="" elementar/s3-volume /data s3://mybucket/someprefix
```
### Using Compose and named volumes
Most of the time, you will use this image to sync data for another container.

12
watch
View File

@ -40,6 +40,12 @@ PROGNAME=$0
LOCAL=$1
REMOTE=$2
if [ "$ENDPOINT_URL" ]; then
AWS="aws --endpoint-url $ENDPOINT_URL"
else
AWS=aws
fi
function restore {
if [ "$(ls -A $LOCAL)" ]; then
if [[ ${FORCE_RESTORE:false} == 'true' ]]; then
@ -48,14 +54,14 @@ function restore {
fi
echo "restoring $REMOTE => $LOCAL"
if ! aws s3 sync "$REMOTE" "$LOCAL"; then
if ! $AWS s3 sync "$REMOTE" "$LOCAL"; then
error_exit "restore failed"
fi
}
function backup {
echo "backup $LOCAL => $REMOTE"
if ! aws s3 sync "$LOCAL" "$REMOTE" --delete; then
if ! $AWS s3 sync "$LOCAL" "$REMOTE" $S3_SYNC_FLAGS; then
echo "backup failed" 1>&2
return 1
fi
@ -63,7 +69,7 @@ function backup {
function final_backup {
echo "backup $LOCAL => $REMOTE"
while ! aws s3 sync "$LOCAL" "$REMOTE" --delete; do
while ! $AWS s3 sync "$LOCAL" "$REMOTE" $S3_SYNC_FLAGS; do
echo "backup failed, will retry" 1>&2
sleep 1
done