Skip to content

Commit

Permalink
workflows/scheduled: improve coverage of scheduled online checks
Browse files Browse the repository at this point in the history
Let's test our formulae more systematically. We do this by
indexing all formulae so that:
- formulae 1 to 50 are tested on the first day of the year
- formulae 51 to 100 are tested on the second day of the year
- formulae 101 to 150 are tested on the third day of the year
- and so on.

This works fine as long as we have fewer than 365 * TEST_COUNT formulae
(currently 18250).

While we're here:
- scope token permissions more tightly
- remove the `os` key from the matrix, since there is only one entry
- remove unused references to `macos-latest`
- always run the formula source test as long as the upstream repository
  has not been archived
- remove the unnecessary `${{  }}` in `if:` properties
- error out with a comment posted if `create_matrix` fails
  • Loading branch information
carlocab committed Oct 1, 2024
1 parent 78ce796 commit b197de5
Showing 1 changed file with 52 additions and 19 deletions.
71 changes: 52 additions & 19 deletions .github/workflows/scheduled.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,14 @@ concurrency:
group: scheduled
cancel-in-progress: true

permissions:
issues: write
permissions: {}

env:
GH_NO_UPDATE_NOTIFIER: 1
GH_PROMPT_DISABLED: 1
GH_REPO: ${{ github.repository }}
REPORTING_ISSUE: 139929
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}

jobs:
create_matrix:
Expand All @@ -31,7 +37,6 @@ jobs:
json: ${{ steps.matrix.outputs.json }}
env:
TEST_COUNT: 50
TAP: homebrew/core
steps:
- name: Set up Homebrew
id: set-up-homebrew
Expand All @@ -43,33 +48,63 @@ jobs:

- name: Generate matrix
id: matrix
working-directory: ${{ steps.set-up-homebrew.outputs.repository-path }}
run: |
formula="$(find "$(brew --repo "${TAP}")/Formula" -type f | shuf -n "${TEST_COUNT}" | xargs -I{} basename {} .rb)"
# shellcheck disable=SC2086
# Index all formulae so that we test:
# - formulae 1 to 50 on the first day of the year
# - formulae 51 to 100 on the second day of the year
# - formulae 101 to 150 on the third day of the year
# - ...
# This works fine as long as we have fewer than 365 * TEST_COUNT formulae.
mapfile -t formulae < <(find Formula -type f -exec basename {} .rb ';' | sort)
formulae_count="${#formulae[@]}"
if (( formulae_count > 365 * TEST_COUNT )); then
echo "::error ::Too many formulae (${formulae_count})! Adjust TEST_COUNT to a number greater than ${TEST_COUNT}."
exit 1
fi
day="$(date +%j)"
start_index="$(( (day - 1) * TEST_COUNT % formulae_count ))"
json="$(
brew info --json=v2 $formula |
jq --compact-output '[.formulae[] | select(.deprecated == false and .disabled == false) | .name]'
brew info --json=v2 "${formulae[@]:${start_index}:${TEST_COUNT}}" |
jq --compact-output '[.formulae[] | select(.deprecated or .disabled | not) | .name]'
)"
echo "json=${json}" >> "$GITHUB_OUTPUT"
comment_on_failure:
needs: create_matrix
if: needs.create_matrix.result == 'failure'
runs-on: ubuntu-latest
permissions:
issues: write
env:
GH_TOKEN: ${{ github.token }}
steps:
- name: Post comment on failure
run: |
gh issue comment "$REPORTING_ISSUE" \
--body "\`create_matrix\` job failed. Check $RUN_URL" \
--repo "$GITHUB_REPOSITORY"
audit_online:
if: startsWith( github.repository, 'Homebrew/' )
runs-on: ${{ matrix.os }}
runs-on: ubuntu-latest
container:
image: ghcr.io/homebrew/ubuntu22.04:master
permissions:
issues: write
needs: create_matrix
name: "Online check (${{ matrix.os }}): ${{ matrix.formula }}"
name: "Online check: ${{ matrix.formula }}"
env:
HOMEBREW_GITHUB_API_TOKEN: "${{ github.token }}"
GH_TOKEN: "${{ github.token }}"
REPORTING_ISSUE: 139929
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
FORMULA: ${{ matrix.formula }}
strategy:
fail-fast: false
matrix:
formula: ${{ fromJson(needs.create_matrix.outputs.json) }}
os: [ubuntu-22.04]
steps:
- name: Set up Homebrew
id: set-up-homebrew
Expand All @@ -81,35 +116,33 @@ jobs:

- name: Check formula source is not archived.
id: archived
if: matrix.os != 'macos-latest'
run: brew audit --online --skip-style --only github_repository_archived,gitlab_repository_archived "$FORMULA"

- name: Report online issues
if: ${{ failure() && steps.archived.conclusion == 'failure' }}
if: failure() && steps.archived.conclusion == 'failure'
run: |
gh issue comment "$REPORTING_ISSUE" \
--body "$FORMULA should be archived. Check $RUN_URL" \
--repo "$GITHUB_REPOSITORY"
- name: Check formula for unavailable homepage.
id: homepage
if: matrix.os != 'macos-latest'
run: brew audit --online --skip-style --only homepage "$FORMULA"

- name: Report homepage issues
if: ${{ failure() && steps.homepage.conclusion == 'failure' }}
if: failure() && steps.homepage.conclusion == 'failure'
run: |
gh issue comment "$REPORTING_ISSUE" \
--body "$FORMULA has homepage issues. Check $RUN_URL" \
--repo "$GITHUB_REPOSITORY"
- name: Check formula for missing sources.
id: fetch
if: matrix.os != 'macos-latest'
run: brew fetch -s "$FORMULA"
if: always() && steps.archived.conclusion != 'failure'
run: brew fetch --build-from-source "$FORMULA"

- name: Report fetch issues
if: ${{ failure() && steps.fetch.conclusion == 'failure' }}
if: failure() && steps.fetch.conclusion == 'failure'
run: |
gh issue comment "$REPORTING_ISSUE" \
--body "$FORMULA source has problems. Check $RUN_URL" \
Expand Down

0 comments on commit b197de5

Please sign in to comment.