Development Conventions
A few conventions should be followed in a development chain to facilitate collaboration, maintainability, and clarity. Generally speaking, I follow the points below in my development.
- Commit Message
- Development workflow
- CI/CD
Commit Message
The Conventional Commits is a regular specification I used in my development for providing an easy set of rules for creating an explicit commit history.
<type>[optional scope]: <description>
[optional body]
[optional footer(s)]
The commit types which is based on Angular Convention are listed in the below table:
Type | Description |
---|---|
build | Build: Changes that affect the build system or external dependencies (e.g., webpack). |
chore | Chore: Miscellaneous tasks that do not modify src or test files (e.g., updates). |
ci | Continuous Integration: Changes related to CI configurations and scripts. |
docs | Documentation: Changes or additions to documentation. |
feat | Feature: Introduces a new feature to the codebase. |
fix | Fix: Patches a bug in the codebase. |
refactor | Refactor: Code changes that neither fix a bug nor add a feature, improving structure. |
revert | Revert: Reverts a previous commit. |
style | Style: Code style changes that do not affect the meaning of the code (e.g., formatting). |
test | Test: Adding or updating tests. |
perf | Performance: Changes that improve performance. |
WIP | Work In Progress: Indicates that the commit is a work in progress and not yet complete. |
I am irregularly using this way, so we’re not gonna expand here.
Recommended Tools
Two type of tools are recommended for standardisation of commit message.
- Interactive prompt of a commit message
- npm version: Commitizen CLI
- pip version: Commitizen
- Automatically generate a commit message
Both tools have their respective strengths and weaknesses; it depends on how you choose. From my perspective, I recommend you use the first type of tool if you are an experienced developer who knows how to write a meaningful and readable message for each commit.
Otherwise, I would say that opencommit
is the best tool for you. Likely, the AI model could not provide an authentic reason why you did this change in the commit message, but it is still better than a rubbish message like ‘update code.’.
Development Workflow

From my experience, I prefer to use trunk-base development as my default development workflow rather than standard Gitflow, because it is more DevOps-friendly to prevent merge-hell from a long-term branch, and feedback the code quality quickly.
However, I am not really sure how to cherry-pick a hotfix commit from the trunk to an old version branch that has different code from the latest version. Therefore, I have read some articles and followed the below workflow in my projects.
The gitflow diagram is shown below:
CI/CD
Git Hooks
Pre-commit is a multi-language package manager for pre-commit hooks. It is used locally in my projects for standardization of code quality checking progress before submission of code to the remote repository.
All developers could simply install it and set it up by following official guide locally, and the configuration I used in the projects is shown below.
repos:
- repo: local
hooks:
- id: trufflehog
name: TruffleHog
description: Detect secrets in your data.
entry:
bash -c 'trufflehog git file://. --since-commit HEAD --results=verified,unknown --fail'
# For running trufflehog in docker, use the following entry instead:
# entry: bash -c 'docker run --rm -v "$(pwd):/workdir" -i --rm trufflesecurity/trufflehog:latest git file:///workdir --since-commit HEAD --results=verified,unknown --fail' language: system
stages: ["commit", "push"]
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v3.2.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-toml
- id: check-added-large-files
- id: mixed-line-ending
args: ["--fix=lf"]
- id: pretty-format-json
args: ["--autofix"]
- repo: local
hooks:
- id: prettier
name: Run Prettier
entry: npx prettier --write
language: system
files: \.json$|\.md$|\.yaml$|\.yml$
types: [file]
pass_filenames: true
- repo: https://github.com/psf/black
rev: 24.10.0
hooks:
- id: black
- repo: local
hooks:
- id: run-ut
name: Run Pytest Tests with Coverage
entry: poetry run pytest --cov=ures --cov-report=term-missing
language: system
types: [python]
always_run: true
pass_filenames: false
Continuous Integration
Github Action is a native CI/CD framework in GitHub. For personal use, GitHub Action could satisfy almost all users’ requirements and provide 2000 Action minutes with plentiful 3rd plugins in the market.
Credential Check
It leverages the power of trufflehog, and example action yaml shown below:
name: Token Secret Check
on: [push, pull_request]
jobs:
secretCheck:
runs-on: ubuntu-24.04
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Secret Scanning
uses: trufflesecurity/trufflehog@main
with:
extra_args: --results=verified,unknown
Linters
It leverages the power of [Super-linters], and example action yaml shown below:
name: Linter
on: [push, pull_request]
jobs:
superLint:
name: Liters
runs-on: ubuntu-20.04
permissions:
contents: read
packages: read
# To report GitHub Actions status checks
statuses: write
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
# super-linter needs the full git history to get the
# list of files that changed across commits
fetch-depth: 0
- name: Configure 1Password Service Account
uses: 1password/load-secrets-action/configure@v2
with:
# Persist the 1Password Service Account Authorization token
# for next steps.
# Keep in mind that every single step in the job is now
# able to access the token.
service-account-token: ${{ secrets.OP_SERVICE_ACCOUNT_TOKEN }}
- name: Load Docker credentials
id: load-docker-credentials
uses: 1password/load-secrets-action@v2
with:
export-env: false
env:
GITHUB_TOKEN: "op://DevOps/Git PAT - StoneHome - Action/credential"
- name: Super-linter
uses: super-linter/super-linter/slim@v7.2.1 # x-release-please-version
env:
# To report GitHub Actions status checks
GITHUB_TOKEN: ${{ steps.load-docker-credentials.outputs.GITHUB_TOKEN }}
VALIDATE_YAML: true
VALIDATE_PYTHON_BLACK: true
VALIDATE_JUPYTER_NBQA_BLACK: true
VALIDATE_JSON: true
VALIDATE_JSON_PRETTIER: true
VALIDATE_GITLEAKS: true
VALIDATE_ENV: true
VALIDATE_DOCKERFILE_HADOLINT: true
VALIDATE_GIT_COMMITLINT: true
VALIDATE_GIT_MERGE_CONFLICT_MARKERS: true
Unit Test
name: Code Testing
on: [push, pull_request]
jobs:
ut:
runs-on: ubuntu-24.04
strategy:
matrix:
python-version: [3.11, 3.12] # Specify the Python versions you want to test against
poetry-version: ["1.8.5"]
steps:
# 1. Checkout the repository
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0
# 2. Set up Python environment
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
# 3. Run poetry image
- name: Install poetry
uses: abatilo/actions-poetry@v2
with:
poetry-version: ${{ matrix.poetry-version }}
# 4. Setup a local virtual environment (if no poetry.toml file)
- name: Setup a local virtual environment (if no poetry.toml file)
run: |
poetry config virtualenvs.create true --local
poetry config virtualenvs.in-project true --local
# 5. Cache the virtual environment based on the dependencies lock file
- uses: actions/cache@v3
name: Define a cache for the virtual environment based on the dependencies lock file
with:
path: ./.venv
key: venv-${{ hashFiles('poetry.lock') }}
# 6. Install dependencies
- name: Install the project dependencies
run: poetry install --with test
# 7. Run the tests
- name: Run the tests
run: poetry run pytest -v
Continuous Delivery
In order to control the frequency of releases, we use a manual trigger approach for each release, and the version number is automatically detected by [[2024-12-29 - Git - versioning tool - Semantic-release|Semantic-Release tool]]. The action yaml is shown below:
name: Release
on: workflow_dispatch
permissions:
contents: read # for checkout
jobs:
release:
permissions:
contents: write # to be able to publish a GitHub release
issues: write # to be able to comment on released issues
pull-requests: write # to be able to comment on released pull requests
name: release
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
- uses: actions/setup-node@v4
with:
node-version: 18
- name: Configure 1Password Service Account
uses: 1password/load-secrets-action/configure@v2
with:
- name: Load GitHub credentials
id: load-credentials
uses: 1password/load-secrets-action@v2
with:
export-env: false
env:
GITHUB_TOKEN: "op://DevOps/Git PAT - StoneHome - Action/credential"
- run: npx semantic-release@21.0.2
env:
GITHUB_TOKEN: ${{ steps.load-credentials.outputs.GITHUB_TOKEN }}
Post-actions of Delivery
Here, we introduce two methods to implement these post-actions effectively:
Triggering via Internal Events Since the release pipeline generates a new version in GitHub, you can trigger the post-action using the internal
publishing published
event. This approach allows your post-actions to respond automatically whenever a new release is published.Extending the Release Process Alternatively, you can add additional steps directly after the release process by modifying the existing release action. This method involves customizing the release workflow to include your specific post-release tasks.
Create a release blog
poetry
as your dependency manager. You should use poetry run python <script>
instead of python <script>
to prevent the ModuleNotFound
error.name: Release Blog
on:
release:
types: [published, edited]
jobs:
post-release:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Fetch release information
id: release
run: |
echo "RELEASE_SUBJECT=${{ github.event.release.name }}" >> $GITHUB_ENV
echo "RELEASE_BODY<<EOF" >> $GITHUB_ENV
echo "${{ github.event.release.body }}" >> $GITHUB_ENV
echo "EOF" >> $GITHUB_ENV
echo "RELEASE_URL=${{ github.event.release.html_url }}" >> $GITHUB_ENV
- name: Configure 1Password Service Account
uses: 1password/load-secrets-action/configure@v2
with:
service-account-token: ${{ secrets.OP_SERVICE_ACCOUNT_TOKEN }}
- name: Load Docker credentials
id: load-credentials
uses: 1password/load-secrets-action@v2
with:
export-env: true
env:
GITHUB_TOKEN: "op://DevOps/Git PAT - Personal - GitHub Action/credential"
- name: Checkout target repository
uses: actions/checkout@v4
with:
repository: stonebo/stone-journey.github.io
token: ${{ env.GITHUB_TOKEN }}
path: hugo_blog
ref: trunk
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: "3.12"
- name: Install poetry
uses: abatilo/actions-poetry@v2
with:
poetry-version: "1.8.5"
- name: Setup a local virtual environment (if no poetry.toml file)
run: |
poetry config virtualenvs.create true --local
poetry config virtualenvs.in-project true --local
- name: Install dependency and generate new file
run: |
export PYTHONPATH="${PYTHONPATH}:${{ github.workspace }}"
python -m pip install --upgrade pip
poetry install
poetry run python .github/scripts/release_blog.py -p "URes" -v "${{ env.RELEASE_SUBJECT }}" -n "${{ env.RELEASE_BODY }}" -u "${{ env.RELEASE_URL }}" -o "hugo_blog"
- name: Commit changes
run: |
cd hugo_blog
git config --global user.email "github@stone-bo.com"
git config --global user.name "Post Release Bot"
git add ./content/posts/project
git commit -m "docs(project): add new blog for ${{ env.RELEASE_SUBJECT }} of ures"
git push
env:
GITHUB_TOKEN: ${{ env.GITHUB_TOKEN }}
Package Delivery
The package delivery is triggered by the same event as section. Due to vary types of package, workflow may be significantly different between each other. Therefore, the package delivery workflow is defined by package management tools. For example, poetry in python, maven for java, etc.
As poetry is my major package management tool, I use it as an example here.
name: Poetry Publish
on:
release:
types: [published]
jobs:
publish:
runs-on: ubuntu-24.04
steps:
# 1. Checkout the repository
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0
# 2. Set up Python environment
- name: Set up Python ${{ inputs.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python-version }}
# 3. Run poetry image
- name: Install poetry
uses: abatilo/actions-poetry@v2
with:
poetry-version: ${{ inputs.poetry-version }}
# 4. Setup a local virtual environment (if no poetry.toml file)
- name: Setup a local virtual environment (if no poetry.toml file)
run: |
poetry config virtualenvs.create true --local
poetry config virtualenvs.in-project true --local
# 5. Cache the virtual environment based on the dependencies lock file
- uses: actions/cache@v3
name: Define a cache for the virtual environment based on the dependencies lock file
with:
path: ./.venv
key: venv-${{ hashFiles('poetry.lock') }}
# 6. load Secrets from 1password
- name: Configure 1Password Service Account
uses: 1password/load-secrets-action/configure@v2
with:
service-account-token: ${{ secrets.onepass_token }}
- name: Load Docker credentials
id: load-docker-credentials
uses: 1password/load-secrets-action@v2
with:
export-env: true
env:
GITHUB_TOKEN: "op://DevOps/Git PAT - StoneHome - Action/credential"
PYPI_TOKEN: "op://DevOps/PyPi Token/credential"
# 7. create a branch
- name: Create Release Branch
shell: bash
run: |
BRANCH_NAME="versions/${{ inputs.version-id }}"
git config user.name "Publish Action"
git config user.email "github@stone-bo.com"
git checkout -b "$BRANCH_NAME"
git push --set-upstream origin "$BRANCH_NAME"
echo "Created branch: $BRANCH_NAME"
# 8. Update version in pyproject.toml to match the tag version
- name: Update version in pyproject.toml
shell: bash
run: |
NEW_VERSION=$(echo ${{ inputs.version-id }} | sed 's/^v//') # Remove 'v' prefix if it exists
poetry version "$NEW_VERSION"
git add pyproject.toml
git commit -m "chore(poetry): bumping the version to ${{ inputs.version-id }}"
git push
# 9. Build package
- name: Build package
run: poetry build
# 10. Publish the package
- name: Publish package to PyPI
run: poetry publish -u __token__ -p ${{ env.PYPI_TOKEN }}
Reusable Workflow
The reusing workflow is a convenient function for free your from copying and pasting of workflows between repositories. You and anyone with access to the reusable workflow can then call the reusable workflow from another workflow.
The reusable workflow aligns with the code provided above and is stored in the repository: link.
GitHub Repository Template
The template repository provides a simple way to create a new repository that comes pre-configured with GitHub actions.