Files
library-charts/.github/workflows/common_library_tests.yaml
Stavros Kois 69b13f07d6 chore(common): check helm version before everything (#683)
**Description**
<!--
Please include a summary of the change and which issue is fixed. Please
also include relevant motivation and context. List any dependencies that
are required for this change.
-->
⚒️ Fixes  # <!--(issue)-->

Adds a pre-flight check for helm version.
By default the restriction is set to `>3.9.4` which is know to not work
with parts of this library.

If the annotation `truecharts.org/min_helm_version` is found then the
restriction becomes
`>=` + the version defined in the annotation.

Currently we can't really "unit test" this, because there is no way to
manipulate annotations in the test suites.
But we are at least testing that it does not "fail", because if it
would, all unittests would fail.

---

Additionally updated the upper bound of helm version we run the tests
on, and also updated the unittest plugin version.
Adjusted some "matchers" in the unittest files to follow the semantic
changes in from the unittest plugins.

**⚙️ Type of change**

- [ ] ⚙️ Feature/App addition
- [ ] 🪛 Bugfix
- [ ] ⚠️ Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] 🔃 Refactor of current code

**🧪 How Has This Been Tested?**
<!--
Please describe the tests that you ran to verify your changes. Provide
instructions so we can reproduce. Please also list any relevant details
for your test configuration
-->

**📃 Notes:**
<!-- Please enter any other relevant information here -->

**✔️ Checklist:**

- [x] ⚖️ My code follows the style guidelines of this project
- [x] 👀 I have performed a self-review of my own code
- [x] #️⃣ I have commented my code, particularly in hard-to-understand
areas
- [ ] 📄 I have made corresponding changes to the documentation
- [x] ⚠️ My changes generate no new warnings
- [ ] 🧪 I have added tests to this description that prove my fix is
effective or that my feature works
- [x] ⬆️ I increased versions for any altered app according to semantic
versioning

** App addition**

If this PR is an app addition please make sure you have done the
following.

- [ ] 🪞 I have opened a PR on
[truecharts/containers](https://github.com/truecharts/containers) adding
the container to TrueCharts mirror repo.
- [ ] 🖼️ I have added an icon in the Chart's root directory called
`icon.png`

---

_Please don't blindly check all the boxes. Read them and only check
those that apply.
Those checkboxes are there for the reviewer to see what is this all
about and
the status of this PR with a quick glance._
2024-01-21 15:33:41 +02:00

416 lines
14 KiB
YAML

name: Common Library Tests
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
on:
push:
branches:
- main
pull_request:
branches:
- main
workflow_dispatch:
env:
DATREE_TOKEN: ${{ secrets.DATREE_TOKEN }}
jobs:
lint:
name: Lint Common
runs-on: ubuntu-22.04
strategy:
fail-fast: false
matrix:
helm-version:
- v3.12.3
- v3.14.0
steps:
- name: Checkout
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
with:
fetch-depth: 1
- name: Install Helm
uses: azure/setup-helm@5119fcb9089d432beecbf79bb2c7915207344b78 # tag=v3
with:
version: ${{ matrix.helm-version }}
- uses: actions/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c # v5
with:
python-version: "3.10"
- name: Set up chart-testing
uses: helm/chart-testing-action@e6669bcd63d7cb57cb4380c33043eebe5d111992 # v2.6.1
- name: Run chart-testing (lint)
id: lint
run: |
ct lint --config .github/ct-install-config/ct-lint.yaml \
--lint-conf .github/ct-install-config/lint-conf.yaml \
--charts library/common-test \
--debug
unittest:
needs:
- lint
name: Unit Tests
runs-on: ubuntu-22.04
env:
helmUnitVersion: 0.4.1
strategy:
fail-fast: false
matrix:
helm-version:
- v3.12.3
- v3.14.0
steps:
- name: Checkout
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
with:
fetch-depth: 1
- name: Install Helm
uses: azure/setup-helm@5119fcb9089d432beecbf79bb2c7915207344b78 # v3
with:
version: ${{ matrix.helm-version }}
- name: Cache helm plugins
uses: actions/cache@13aacd865c20de90d75de3b17ebe84f7a17d57d2 # v4
with:
path: |
/home/runner/.local/share/helm/plugins/helm-unittest
/tmp/_dist/
key: helm-${{ matrix.helm-version }}-unittest-v${{ env.helmUnitVersion }}
restore-keys: |
helm-${{ matrix.helm-version }}-unittest-v${{ env.helmUnitVersion }}
- name: Run Unittests
shell: bash
run: |
(helm unittest -h > /dev/null) || helm plugin install https://github.com/helm-unittest/helm-unittest --version v${helmUnitVersion} || (sleep 10 && helm plugin install https://github.com/helm-unittest/helm-unittest --version v${helmUnitVersion}) || echo "finished unittest reinstall tries"
# Run tests
cd library/common-test/
helm dependency update
helm unittest -f "tests/**/*.yaml" .
install:
needs:
- lint
name: Install Charts
runs-on: ubuntu-22.04
strategy:
fail-fast: false
matrix:
# We run tests on k3s version of latest SCALE release and SCALE nightly
k3s-version:
- v1.26
- v1.27
- v1.28
# We run tests on Helm version of latest SCALE, fluxcd and latest helm release
helm-version:
- v3.12.3
- v3.14.0
values:
- basic-values.yaml
- configmap-values.yaml
- secrets-values.yaml
- imagePullSecret-values.yaml
- daemonset-values.yaml
- job-values.yaml
- cron-values.yaml
- statefulset-values.yaml
- persistence-values.yaml
- extra-containers-values.yaml
- rbac-values.yaml
- ingress-values.yaml
- networkPolicy-values.yaml
# TODO: broken
# - codeserver-values.yaml
- netshoot-values.yaml
- metrics-values.yaml
- cnpg-values.yaml
- cnpg-multi-values.yaml
- register-operator-values.yaml
- portal-svc-values.yaml
- portal-ingress-values.yaml
steps:
- name: Checkout
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
with:
fetch-depth: 1
- name: Install Helm
uses: azure/setup-helm@5119fcb9089d432beecbf79bb2c7915207344b78 # v3
with:
version: ${{ matrix.helm-version }}
- uses: actions/setup-python@0a5c61591373683505ea898e09a3ea4f39ef2b9c # v5
with:
python-version: "3.10"
- name: Set up chart-testing
uses: helm/chart-testing-action@e6669bcd63d7cb57cb4380c33043eebe5d111992 # v2.6.1
- name: Create k3d cluster - Attempt 1/3
continue-on-error: true
id: createc1
uses: nolar/setup-k3d-k3s@293b8e5822a20bc0d5bcdd4826f1a665e72aba96 # tag=v1.0.9
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
version: ${{ matrix.k3s-version }}
# Flags found here https://github.com/k3d-io/k3d
k3d-args: --k3s-arg --disable=metrics-server@server:*
- name: Wait 10 second to retry
if: steps.createc1.outcome=='failure'
run: |
sleep 10
- name: Create k3d cluster - Attempt 2/3
continue-on-error: true
if: steps.createc1.outcome=='failure'
id: createc2
uses: nolar/setup-k3d-k3s@293b8e5822a20bc0d5bcdd4826f1a665e72aba96 # tag=v1.0.9
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
version: ${{ matrix.k3s-version }}
# Flags found here https://github.com/k3d-io/k3d
k3d-args: --k3s-arg --disable=metrics-server@server:*
- name: Wait 10 second to retry
if: steps.createc2.outcome=='failure'
run: |
sleep 10
- name: Create k3d cluster - Attempt 3/3
id: createc3
if: steps.createc2.outcome=='failure'
uses: nolar/setup-k3d-k3s@293b8e5822a20bc0d5bcdd4826f1a665e72aba96 # tag=v1.0.9
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
version: ${{ matrix.k3s-version }}
# Flags found here https://github.com/k3d-io/k3d
k3d-args: --k3s-arg --disable=metrics-server@server:*
# Install Kail to grab logs from tests, as there are cases ct-install fail to output logs
- name: Install Kail
run: |
export KAIL_VERSION=v0.16.1
wget https://github.com/boz/kail/releases/download/${KAIL_VERSION}/kail_${KAIL_VERSION}_linux_amd64.tar.gz
tar -xvzf kail_${KAIL_VERSION}_linux_amd64.tar.gz
chmod +x kail
- name: Prep Helm
run: |
helm repo add truecharts-deps https://deps.truecharts.org
helm repo update
- name: Add Dependencies
run: |
if [[ "${{ matrix.values }}" =~ (metrics|cnpg).*-values.yaml ]]; then
helm install prometheus-operator oci://tccr.io/truecharts/prometheus-operator --namespace prometheus-operator --create-namespace --wait
fi
if [[ "${{ matrix.values }}" =~ cnpg.*-values.yaml ]]; then
helm install cloudnative-pg oci://tccr.io/truecharts/cloudnative-pg --namespace cloudnative-pg --create-namespace --wait
fi
if [[ "${{ matrix.values }}" =~ ingress.*-values.yaml ]]; then
helm install cert-manager oci://tccr.io/truecharts/cert-manager --namespace cert-manager --create-namespace --wait
helm install traefik oci://tccr.io/truecharts/traefik --namespace traefik --create-namespace --wait \
--set service.main.type=ClusterIP --set service.tcp.type=ClusterIP
fi
- name: Run chart-testing (install)
run: |
# Move all ci values on a temp location (or skip if already moved from another matrix job)
mv library/common-test/ci library/common-test/runtests || echo "Nothing to move"
# Move one values.yaml to the correct location to run the test
mv -f library/common-test/runtests/${{ matrix.values }} library/common-test/values.yaml
# Stat kail on the background to grab logs from tests
./kail --ignore-ns kube-system --ignore-ns cert-manager --ignore-ns metallb-system --ignore-ns prometheus-operator >> /tmp/output.log &
# Actually run the test
ct install --config .github/ct-install-config/ct-install.yaml \
--charts library/common-test \
--debug || (echo -e "\n\n--===PODLOGS===--\n\n" && \
cat /tmp/output.log && \
rm -f /tmp/output.log && exit 1)
kill $!
echo -e "\n\n--===PODLOGS===--\n\n"
cat /tmp/output.log
rm -f /tmp/output.log
# security:
# needs:
# - lint
# name: Security Scans
# runs-on: ubuntu-22.04
# strategy:
# fail-fast: false
# matrix:
# # We run tests on k3s version of latest SCALE release and SCALE nightly
# k3s-version:
# - v1.26
# - v1.27
# - v1.28
# # We run tests on Helm version of latest SCALE release
# helm-version:
# - v3.12.3
# - v3.14.0
# values:
# - basic-values.yaml
# - configmap-values.yaml
# - secrets-values.yaml
# - imagePullSecret-values.yaml
# - daemonset-values.yaml
# - job-values.yaml
# - cron-values.yaml
# - statefulset-values.yaml
# - persistence-values.yaml
# - extra-containers-values.yaml
# - rbac-values.yaml
# - networkPolicy-values.yaml
# - register-operator-values.yaml
# # Runs as root, so test results become obviously red
# # - codeserver-values.yaml
# # - netshoot-values.yaml
#
#
# steps:
# - name: Install Helm
# uses: azure/setup-helm@5119fcb9089d432beecbf79bb2c7915207344b78 # v3
# with:
# version: ${{ matrix.helm-version }}
#
# - name: Checkout
# uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
# with:
# fetch-depth: 1
#
# - name: build helm deps
# run: |
# cd library/common-test
# helm dependency update
# cd -
# - name: Add namespace
# run: |
# echo "namespace: common-test" >> library/common-test/ci/${{ matrix.values }}
#
# - name: Run Security Scan
# uses: datreeio/action-datree@main
# with:
# path: 'library/common-test'
# cliArguments: '--ignore-missing-schemas --policy CommonBasic'
# isHelmChart: true
# helmArguments: '--values library/common-test/ci/${{ matrix.values }}'
#
# security-man:
# needs:
# - lint
# name: Security Scans (manifests)
# runs-on: ubuntu-22.04
# strategy:
# fail-fast: false
# matrix:
# # We run tests on k3s version of latest SCALE release and SCALE nightly
# k3s-version:
# - v1.26
# - v1.27
# - v1.28
# # We run tests on Helm version of latest SCALE release
# helm-version:
# - v3.12.3
# - v3.14.0
# values:
# - ingress-values.yaml
# - metrics-values.yaml
# # - cnpg-values.yaml
# # - cnpg-multi-values.yaml
#
# steps:
# - name: Install Helm
# uses: azure/setup-helm@5119fcb9089d432beecbf79bb2c7915207344b78 # v3
# with:
# version: ${{ matrix.helm-version }}
#
# - name: Prep Helm
# run: |
# helm repo add truecharts-deps https://deps.truecharts.org
# helm repo update
# - name: Add Dependencies
# run: |
# helm install prometheus-operator oci://tccr.io/truecharts/prometheus-operator --namespace prometheus-operator --create-namespace --wait
# helm install cloudnative-pg oci://tccr.io/truecharts/cloudnative-pg --namespace cloudnative-pg --create-namespace --wait
# helm install traefik oci://tccr.io/truecharts/traefik --namespace traefik --create-namespace --wait \
# --set service.main.type=ClusterIP --set service.tcp.type=ClusterIP
#
# - name: Checkout
# uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
# with:
# fetch-depth: 1
#
# - name: build helm deps
# run: |
# cd library/common-test
# helm dependency update
# cd -
#
# - name: Run Security Scan
# uses: datreeio/action-datree@main
# with:
# path: 'library/common-test'
# cliArguments: '--ignore-missing-schemas --policy ManifestManager'
# isHelmChart: true
# helmArguments: '--values library/common-test/ci/${{ matrix.values }}'
#
# security-vpn:
# needs:
# - lint
# name: Security Scans (vpn)
# runs-on: ubuntu-22.04
# strategy:
# fail-fast: false
# matrix:
# # We run tests on k3s version of latest SCALE release and SCALE nightly
# k3s-version:
# - v1.26
# - v1.27
# - v1.28
# # We run tests on Helm version of latest SCALE release
# helm-version:
# - v3.12.3
# - v3.14.0
# values:
# - vpn-gluetun-values.yaml
# - vpn-tailscale-values.yaml
# # Is deprecated and runs as root, skipping for now
# # - vpn-openvpn-values.yaml
# - vpn-wireguard-values.yaml
#
# steps:
# - name: Install Helm
# uses: azure/setup-helm@5119fcb9089d432beecbf79bb2c7915207344b78 # v3
# with:
# version: ${{ matrix.helm-version }}
#
# - name: Checkout
# uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
# with:
# fetch-depth: 1
#
# - name: build helm deps
# run: |
# cd library/common-test
# helm dependency update
# cd -
#
# - name: Run Security Scan
# uses: datreeio/action-datree@main
# with:
# path: 'library/common-test'
# cliArguments: '--ignore-missing-schemas --policy WithVPN'
# isHelmChart: true
# helmArguments: '--values library/common-test/ci/${{ matrix.values }}'