feat: replace shell script with python implementation

- add requirements.txt
- allow disabling a stack
- update config example
- update README
This commit is contained in:
Sebastian Mark 2023-12-19 10:55:34 +01:00
parent 77a3ae1ed6
commit ef0620ee41
6 changed files with 92 additions and 49 deletions

2
.gitignore vendored
View file

@ -1 +1 @@
stacklist
config.yml

View file

@ -1,19 +1,27 @@
# Docker Compose Ops (dc-ops)
# Docker Compose GitOps (dc-ops)
`dc-ops` is a shell script designed to operations automate the local building and updating of several Docker services.
It aims to simplify continuous delivery (CD) processes and infrastructure management in a Docker environment.
`dc-ops` is a simple python script that automates the update and deployment of multiple Docker Compose applications.
## How does it work?
It reads a list of git repositories from a YAML configuration file (`config.yml`), pulls the latest changes for each repo, and runs `docker compose up` for each specified `docker-compose.yml` file.
* The script iterates through each line of the `stacklist` file (skipping comments) or through all cli parameters
* It fetches the latest changes from the remote git repository and checks if there are new commits
* If there is a change it pulls the updates from the remote git repository, otherwise the entry is skipped
* Following this, it runs `docker compose up` which builds, (re)creates and starts the containers
## Features
- Automatically checks for updates in git repositories
- Supports multiple Docker Compose files per repository
- Can be configured to skip certain repositories
The `stacklist` file can list either a directory containing a `docker-compose.yml` file or a precise `docker-compose` file (see `stacklist.example`).
Alternatively, the stacklist can also be passed as a list of parameters.
## Requirements
See `requirements.txt`
## Usage
1. Install the required Python packages: `pip install -r requirements.txt`
2. Adapt `config.yml.example` to your needs and save it as `config.yml`
3. Run `dc-ops` from the shell or from a cron job
* `./dc-ops /path/to/repo /path/to/another-repo /path/with/docker-compose-dev.yml` or
* Update `stacklist` file and run `./dc-ops` (or create a crontab entry)
## Detailed process
For each enabled stack in the config file, the following process will be executed:
1. Checking directory existence
2. Fetching latest changes from remote repository
3. If there is a new commit, it will pull the changes
4. Running Docker Compose with the defined or default compose-file(s)

11
config.yml.example Normal file
View file

@ -0,0 +1,11 @@
stacks:
- dir: /path/to/first-repo
- dir: /path/to/second-repo
compose-files:
- docker-compose-dev.yml
- dir: /path/to/third-repo
compose-files:
- subdirA/docker-compose.yml
- subdirB/docker-compose.yml
- dir: /path/to/fourth-repo
enabled: false

98
dc-ops
View file

@ -1,44 +1,70 @@
#! /bin/bash
#!/usr/bin/env python3
# -*- encoding: utf-8; py-indent-offset: 4 -*-
## Author: Sebastian Mark
## CC-BY-SA (https://creativecommons.org/licenses/by-sa/4.0/deed.de)
## for civil use only
# Author: Sebastian Mark
# CC-BY-SA (https://creativecommons.org/licenses/by-sa/4.0/deed.de)
# for civil use only
## see README.md
import logging as log
import subprocess
from pathlib import Path
h1() { echo "* $*"; }
msg() { echo "$*"; }
import yaml
from git import GitCommandError, NoSuchPathError, Repo
BASEDIR=$(dirname "$0")
# read config file
configfile = Path(__file__).with_name("config.yml")
with configfile.open("r") as f:
cfg = yaml.safe_load(f.read())
# use list from file or passed parameters
STACKLIST=$(cat "$BASEDIR/stacklist" 2>/dev/null)
[[ $# -gt 0 ]] && STACKLIST=$*
# init logging
log.basicConfig(format="%(message)s", level=log.INFO)
# iterate list
for LINE in $STACKLIST; do
grep -q "^[[:space:]]*#" <<<"$LINE" && continue # skip comments
# determine if passed a directory or a file
STACKDIR=$LINE
if [[ -f $LINE ]]; then
STACKDIR=$(dirname "$LINE")
COMPOSEFILE=$LINE
fi
h1 "processing $STACKDIR"
# skip if directroy not found
cd "$STACKDIR" || continue
# fetch from repo and check for new commits
git fetch --quiet || continue
if [[ $(git rev-parse HEAD) == $(git rev-parse "@{u}") ]]; then
msg "no changes - skipping"
# iterate all stacks
for stack in cfg["stacks"]:
# skip disabled stacks
if "enabled" in stack and not stack["enabled"]:
continue
fi
# pull new commits and run docker compose
git pull || continue
docker compose --file "${COMPOSEFILE:=docker-compose.yml}" up --build --detach --remove-orphans
done
# header
stackdir = stack["dir"]
log.info(f"* processing {stackdir}")
# create repo instance if it exists
try:
repo = Repo(stackdir)
except NoSuchPathError:
log.error("directory not found")
continue
# try to fetch latest changes
try:
repo.git.fetch()
except GitCommandError as e:
log.error(str(e))
continue
# check for new commits
if repo.rev_parse("HEAD") == repo.rev_parse(f"origin/{repo.active_branch}"):
log.info("no changes - skipping")
continue
# pull remote changes to local branch
try:
log.info(repo.git.pull())
except GitCommandError as e:
log.error(str(e))
continue
# run `docker compose` for all compose-files
# (or just for the directory if no compose-file defined)
composefiles = stack.get("compose-files", ["docker-compose.yml"])
for composefile in composefiles:
try:
subprocess.run(
f"docker compose --file {stackdir}/{composefile} up --build --detach --remove-orphans", # noqa
shell=True,
text=True,
)
except subprocess.CalledProcessError:
pass

2
requirements.txt Normal file
View file

@ -0,0 +1,2 @@
gitpython
pyyaml

View file

@ -1,4 +0,0 @@
/path/to/repo
/path/to/another-repo
/path/with/docker-compose.yml
/another-path/with/docker-compose-dev.yml