A few conventions should be followed in a development chain to facilitate collaboration, maintainability, and clarity. Generally speaking, I follow the points below in my development.

  • Commit Message
  • Development workflow
  • CI/CD

Commit Message

The Conventional Commits is a regular specification I used in my development for providing an easy set of rules for creating an explicit commit history.

<type>[optional scope]: <description>

[optional body]

[optional footer(s)]

The commit types which is based on Angular Convention are listed in the below table:

TypeDescription
buildBuild: Changes that affect the build system or external dependencies (e.g., webpack).
choreChore: Miscellaneous tasks that do not modify src or test files (e.g., updates).
ciContinuous Integration: Changes related to CI configurations and scripts.
docsDocumentation: Changes or additions to documentation.
featFeature: Introduces a new feature to the codebase.
fixFix: Patches a bug in the codebase.
refactorRefactor: Code changes that neither fix a bug nor add a feature, improving structure.
revertRevert: Reverts a previous commit.
styleStyle: Code style changes that do not affect the meaning of the code (e.g., formatting).
testTest: Adding or updating tests.
perfPerformance: Changes that improve performance.
WIPWork In Progress: Indicates that the commit is a work in progress and not yet complete.

gitmoji is an alternative approach
gitmoji is an alternative approach to present commit type combined with emoji icons. This could provide more types than the conventional commit approach, but users are also confused by overwhelming icons before they remember all the meanings of icons.

I am irregularly using this way, so we’re not gonna expand here.

Two type of tools are recommended for standardisation of commit message.

  1. Interactive prompt of a commit message
    1. npm version: Commitizen CLI
    2. pip version: Commitizen
  2. Automatically generate a commit message
    1. opencommit

Both tools have their respective strengths and weaknesses; it depends on how you choose. From my perspective, I recommend you use the first type of tool if you are an experienced developer who knows how to write a meaningful and readable message for each commit.

Otherwise, I would say that opencommit is the best tool for you. Likely, the AI model could not provide an authentic reason why you did this change in the commit message, but it is still better than a rubbish message like ‘update code.’.

Development Workflow

Only Pull Request is allowed to merge code from a branch to the trunk. Direct merging may bypass the CI check.
Development Workflow

From my experience, I prefer to use trunk-base development as my default development workflow rather than standard Gitflow, because it is more DevOps-friendly to prevent merge-hell from a long-term branch, and feedback the code quality quickly.

However, I am not really sure how to cherry-pick a hotfix commit from the trunk to an old version branch that has different code from the latest version. Therefore, I have read some articles and followed the below workflow in my projects.

The gitflow diagram is shown below:

%%{init: { 'logLevel': 'debug', 'theme': 'dark', 'gitGraph': {'mainBranchName': 'trunk', 'parallelCommits': false}} }%% gitGraph commit id: "Initial commit" type: HIGHLIGHT branch feature/feature-1 checkout feature/feature-1 commit id: "Feature 1 commit 1" commit id: "Feature 1 commit 2" checkout trunk merge feature/feature-1 id: "feature-1" branch feature/feature-2 checkout feature/feature-2 commit id: "Feature 2 commit 1" commit id: "Feature 2 commit 2" checkout trunk merge feature/feature-2 id: "feature-2" branch release/v1.1.x commit id: "v1.0.0" tag: "v1.0.0" checkout trunk commit branch hotfix/v1.1.1 checkout hotfix/v1.1.1 commit commit checkout trunk merge hotfix/v1.1.1 commit id:"hotfix" checkout trunk commit commit id: "break change" branch release/v2.0.x commit id: "v2.0.0" tag: "v2.0.0" checkout release/v1.1.x cherry-pick id:"hotfix" tag:"v1.1.1" checkout release/v1.1.x branch hotfix/v1.1.2 checkout hotfix/v1.1.2 commit id:"fix issue" checkout release/v1.1.x merge hotfix/v1.1.2 tag: "v1.1.2" checkout trunk

CI/CD

Git Hooks

Pre-commit is a multi-language package manager for pre-commit hooks. It is used locally in my projects for standardization of code quality checking progress before submission of code to the remote repository.

All developers could simply install it and set it up by following official guide locally, and the configuration I used in the projects is shown below.

black is the uncompromising Python code formatter.
trufflehog is an analysis tool for credential discovery.
prettier is a language formatter used for a consistent coding style overall project. It supports bunch of languages, but I recently use it for JSON/YAML/Markdown

repos:
  - repo: local
    hooks:
      - id: trufflehog
        name: TruffleHog
        description: Detect secrets in your data.
        entry:
          bash -c 'trufflehog git file://. --since-commit HEAD --results=verified,unknown --fail'
          # For running trufflehog in docker, use the following entry instead:
          # entry: bash -c 'docker run --rm -v "$(pwd):/workdir" -i --rm trufflesecurity/trufflehog:latest git file:///workdir --since-commit HEAD --results=verified,unknown --fail'        language: system
        stages: ["commit", "push"]
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v3.2.0
    hooks:
      - id: trailing-whitespace
      - id: end-of-file-fixer
      - id: check-yaml
      - id: check-toml
      - id: check-added-large-files
      - id: mixed-line-ending
        args: ["--fix=lf"]
      - id: pretty-format-json
        args: ["--autofix"]
  - repo: local
    hooks:
      - id: prettier
        name: Run Prettier
        entry: npx prettier --write
        language: system
        files: \.json$|\.md$|\.yaml$|\.yml$
        types: [file]
        pass_filenames: true
  - repo: https://github.com/psf/black
    rev: 24.10.0
    hooks:
      - id: black
  - repo: local
    hooks:
      - id: run-ut
        name: Run Pytest Tests with Coverage
        entry: poetry run pytest --cov=ures --cov-report=term-missing
        language: system
        types: [python]
        always_run: true
        pass_filenames: false

Continuous Integration

I prefer GitLab or Jenkins if it is used by a team.
Due to all credentials being managed in 1Password, the action loads all tokens from it.

Github Action is a native CI/CD framework in GitHub. For personal use, GitHub Action could satisfy almost all users’ requirements and provide 2000 Action minutes with plentiful 3rd plugins in the market.

Credential Check

It leverages the power of trufflehog, and example action yaml shown below:

name: Token Secret Check

on: [push, pull_request]

jobs:
  secretCheck:
    runs-on: ubuntu-24.04
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - name: Secret Scanning
        uses: trufflesecurity/trufflehog@main
        with:
          extra_args: --results=verified,unknown

Linters

It leverages the power of [Super-linters], and example action yaml shown below:

name: Linter

on: [push, pull_request]

jobs:
  superLint:
    name: Liters
    runs-on: ubuntu-20.04
    permissions:
      contents: read
      packages: read
      # To report GitHub Actions status checks
      statuses: write

    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          # super-linter needs the full git history to get the
          # list of files that changed across commits
          fetch-depth: 0

      - name: Configure 1Password Service Account
        uses: 1password/load-secrets-action/configure@v2
        with:
          # Persist the 1Password Service Account Authorization token
          # for next steps.
          # Keep in mind that every single step in the job is now
          # able to access the token.
          service-account-token: ${{ secrets.OP_SERVICE_ACCOUNT_TOKEN }}

      - name: Load Docker credentials
        id: load-docker-credentials
        uses: 1password/load-secrets-action@v2
        with:
          export-env: false
        env:
          GITHUB_TOKEN: "op://DevOps/Git PAT - StoneHome - Action/credential"

      - name: Super-linter
        uses: super-linter/super-linter/slim@v7.2.1 # x-release-please-version
        env:
          # To report GitHub Actions status checks
          GITHUB_TOKEN: ${{ steps.load-docker-credentials.outputs.GITHUB_TOKEN }}
          VALIDATE_YAML: true
          VALIDATE_PYTHON_BLACK: true
          VALIDATE_JUPYTER_NBQA_BLACK: true
          VALIDATE_JSON: true
          VALIDATE_JSON_PRETTIER: true
          VALIDATE_GITLEAKS: true
          VALIDATE_ENV: true
          VALIDATE_DOCKERFILE_HADOLINT: true
          VALIDATE_GIT_COMMITLINT: true
          VALIDATE_GIT_MERGE_CONFLICT_MARKERS: true

Unit Test

The below example is only used for Python projects.
name: Code Testing

on: [push, pull_request]

jobs:
  ut:
    runs-on: ubuntu-24.04
    strategy:
      matrix:
        python-version: [3.11, 3.12] # Specify the Python versions you want to test against
        poetry-version: ["1.8.5"]
    steps:
      # 1. Checkout the repository
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0
      # 2. Set up Python environment
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
      # 3. Run poetry image
      - name: Install poetry
        uses: abatilo/actions-poetry@v2
        with:
          poetry-version: ${{ matrix.poetry-version }}
      # 4. Setup a local virtual environment (if no poetry.toml file)
      - name: Setup a local virtual environment (if no poetry.toml file)
        run: |
          poetry config virtualenvs.create true --local
          poetry config virtualenvs.in-project true --local
      # 5. Cache the virtual environment based on the dependencies lock file
      - uses: actions/cache@v3
        name: Define a cache for the virtual environment based on the dependencies lock file
        with:
          path: ./.venv
          key: venv-${{ hashFiles('poetry.lock') }}
      # 6. Install dependencies
      - name: Install the project dependencies
        run: poetry install --with test
      # 7. Run the tests
      - name: Run the tests
        run: poetry run pytest -v

Continuous Delivery

In order to control the frequency of releases, we use a manual trigger approach for each release, and the version number is automatically detected by [[2024-12-29 - Git - versioning tool - Semantic-release|Semantic-Release tool]]. The action yaml is shown below:

name: Release
on: workflow_dispatch
permissions:
  contents: read # for checkout
jobs:
  release:
    permissions:
      contents: write # to be able to publish a GitHub release
      issues: write # to be able to comment on released issues
      pull-requests: write # to be able to comment on released pull requests
    name: release
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
      - uses: actions/setup-node@v4
        with:
          node-version: 18
      - name: Configure 1Password Service Account
        uses: 1password/load-secrets-action/configure@v2
        with:
      - name: Load GitHub credentials
        id: load-credentials
        uses: 1password/load-secrets-action@v2
        with:
          export-env: false
        env:
          GITHUB_TOKEN: "op://DevOps/Git PAT - StoneHome - Action/credential"
      - run: npx semantic-release@21.0.2
        env:
          GITHUB_TOKEN: ${{ steps.load-credentials.outputs.GITHUB_TOKEN }}

Post-actions of Delivery

Post-actions are crucial for enhancing the reusability of previous steps, except for package distribution, since each language has unique artifacts for storing published packages. For instance, you might need to publish to PyPI for Python releases or to Docker Hub for Docker image releases.

Here, we introduce two methods to implement these post-actions effectively:

  • Triggering via Internal Events Since the release pipeline generates a new version in GitHub, you can trigger the post-action using the internal publishing published event. This approach allows your post-actions to respond automatically whenever a new release is published.

  • Extending the Release Process Alternatively, you can add additional steps directly after the release process by modifying the existing release action. This method involves customizing the release workflow to include your specific post-release tasks.

Create a release blog

Here may have a pitfall you will meet during creating this action when you use poetry as your dependency manager. You should use poetry run python <script> instead of python <script> to prevent the ModuleNotFound error.
In order to manage and observe all release information of all my projects, each release triggers a post-action pipeline, called release-blog, to create a markdown blog and push it to my personal blog repository. It is implemented by the following code:

name: Release Blog

on:
  release:
    types: [published, edited]

jobs:
  post-release:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Fetch release information
        id: release
        run: |
          echo "RELEASE_SUBJECT=${{ github.event.release.name }}" >> $GITHUB_ENV
          echo "RELEASE_BODY<<EOF" >> $GITHUB_ENV
          echo "${{ github.event.release.body }}" >> $GITHUB_ENV
          echo "EOF" >> $GITHUB_ENV
          echo "RELEASE_URL=${{ github.event.release.html_url }}" >> $GITHUB_ENV

      - name: Configure 1Password Service Account
        uses: 1password/load-secrets-action/configure@v2
        with:
          service-account-token: ${{ secrets.OP_SERVICE_ACCOUNT_TOKEN }}

      - name: Load Docker credentials
        id: load-credentials
        uses: 1password/load-secrets-action@v2
        with:
          export-env: true
        env:
          GITHUB_TOKEN: "op://DevOps/Git PAT - Personal - GitHub Action/credential"

      - name: Checkout target repository
        uses: actions/checkout@v4
        with:
          repository: stonebo/stone-journey.github.io
          token: ${{ env.GITHUB_TOKEN }}
          path: hugo_blog
          ref: trunk

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: "3.12"

      - name: Install poetry
        uses: abatilo/actions-poetry@v2
        with:
          poetry-version: "1.8.5"

      - name: Setup a local virtual environment (if no poetry.toml file)
        run: |
          poetry config virtualenvs.create true --local
          poetry config virtualenvs.in-project true --local

      - name: Install dependency and generate new file
        run: |
          export PYTHONPATH="${PYTHONPATH}:${{ github.workspace }}"
          python -m pip install --upgrade pip
          poetry install
          poetry run python .github/scripts/release_blog.py -p "URes" -v "${{ env.RELEASE_SUBJECT }}" -n "${{ env.RELEASE_BODY }}" -u "${{ env.RELEASE_URL }}" -o "hugo_blog"

      - name: Commit changes
        run: |
          cd hugo_blog
          git config --global user.email "github@stone-bo.com"
          git config --global user.name "Post Release Bot"
          git add ./content/posts/project
          git commit -m "docs(project): add new blog for ${{ env.RELEASE_SUBJECT }} of ures"
          git push
        env:
          GITHUB_TOKEN: ${{ env.GITHUB_TOKEN }}
Package Delivery

The package delivery is triggered by the same event as section. Due to vary types of package, workflow may be significantly different between each other. Therefore, the package delivery workflow is defined by package management tools. For example, poetry in python, maven for java, etc.

As poetry is my major package management tool, I use it as an example here.

name: Poetry Publish

on:
  release:
    types: [published]

jobs:
  publish:
    runs-on: ubuntu-24.04
    steps:
      # 1. Checkout the repository
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0
      # 2. Set up Python environment
      - name: Set up Python ${{ inputs.python-version }}
        uses: actions/setup-python@v5
        with:
          python-version: ${{ inputs.python-version }}
      # 3. Run poetry image
      - name: Install poetry
        uses: abatilo/actions-poetry@v2
        with:
          poetry-version: ${{ inputs.poetry-version }}
      # 4. Setup a local virtual environment (if no poetry.toml file)
      - name: Setup a local virtual environment (if no poetry.toml file)
        run: |
          poetry config virtualenvs.create true --local
          poetry config virtualenvs.in-project true --local
      # 5. Cache the virtual environment based on the dependencies lock file
      - uses: actions/cache@v3
        name: Define a cache for the virtual environment based on the dependencies lock file
        with:
          path: ./.venv
          key: venv-${{ hashFiles('poetry.lock') }}
      # 6. load Secrets from 1password
      - name: Configure 1Password Service Account
        uses: 1password/load-secrets-action/configure@v2
        with:
          service-account-token: ${{ secrets.onepass_token }}
      - name: Load Docker credentials
        id: load-docker-credentials
        uses: 1password/load-secrets-action@v2
        with:
          export-env: true
        env:
          GITHUB_TOKEN: "op://DevOps/Git PAT - StoneHome - Action/credential"
          PYPI_TOKEN: "op://DevOps/PyPi Token/credential"

	  # 7. create a branch
	  - name: Create Release Branch
        shell: bash
        run: |
          BRANCH_NAME="versions/${{ inputs.version-id }}"
          git config user.name "Publish Action"
          git config user.email "github@stone-bo.com"
          git checkout -b "$BRANCH_NAME"
          git push --set-upstream origin "$BRANCH_NAME"
          echo "Created branch: $BRANCH_NAME"
      # 8. Update version in pyproject.toml to match the tag version
      - name: Update version in pyproject.toml
        shell: bash
        run: |
          NEW_VERSION=$(echo ${{ inputs.version-id }} | sed 's/^v//') # Remove 'v' prefix if it exists
          poetry version "$NEW_VERSION"
          git add pyproject.toml
          git commit -m "chore(poetry): bumping the version to ${{ inputs.version-id }}"
          git push
      # 9. Build package
      - name: Build package
        run: poetry build
      # 10. Publish the package
      - name: Publish package to PyPI
        run: poetry publish -u __token__ -p ${{ env.PYPI_TOKEN }}

Reusable Workflow

The reusing workflow is a convenient function for free your from copying and pasting of workflows between repositories. You and anyone with access to the reusable workflow can then call the reusable workflow from another workflow.

The reusable workflow aligns with the code provided above and is stored in the repository: link.

GitHub Repository Template

The template repository provides a simple way to create a new repository that comes pre-configured with GitHub actions.