Releases: facebook/openbmc
Releases · facebook/openbmc
flashy-e1fa335
flashy_arm64 build Summary: Compile arm64 version of flashy Test Plan: Checkout D61731216, build flashy ephemeral: ``` $ fbpkg build -E openbmc.utils.flashy --yes --configerator-path ~/configerator/ ``` Ensure arm64 flashy binary is present ``` $ cd $(mktemp -d) && fbpkg fetch openbmc.utils.flashy:c84f8ad ... $ file flashy_arm64 flashy_arm64: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, Go BuildID=1AdKZP4L9rfwwmxcE3C3/pY_LLowZsUX8QlcymBEz/yAXuELHk-8td4irRAKp1/M9d2EazNavpmMXz27q3x, stripped ``` Reviewed By: doranand Differential Revision: D61730815 fbshipit-source-id: 7e5328f2f3604733f1577803444d105876e18afb
flashy-f5510f3
Restore fw_printenv command Summary: D60115096 introduced a potential bug whereby `fw_printenv` check is expecintg stdout from the previous command, which is now replaced by an `ls` command Restore the previous command and clearly differentiate between the two errors Test Plan: Unit tests ``` $ ./tools/flashy/scripts/run_unit_tests.sh ... ... ... 142/142 unit tests passed ``` Reviewed By: kawmarco Differential Revision: D60234128 fbshipit-source-id: 50ebd779d47082ca8487bbf53bb68d5db292c883
flashy-87c3594
Fix missing MTD error handling - add specific error Summary: Currently, upgrades that fail because of a blank /proc/mtd are being classed as `UNSAFE_TO_REBOOT` errors. This is because to classify as a `BAD_FLASH_CHIP` error, Flashy is grepping for the particular string *'Cannot access MTD device'* whereas the error on the BMC is returning: ``` root@rsw006-oob:~# cat /proc/mtd dev: size erasesize name root@rsw006-oob:~# fw_printenv bootargs Cannot open /dev/mtd1: No such file or directory ``` Let's fix this by skipping the string grepping logic completely, as we already know that we cannot see MTD at this stage Also added as separate error class, as missing MTD is a specific issue that we see quite frequently and good to classify it separately Test Plan: Unit tests ``` $ ./tools/flashy/scripts/run_unit_tests.sh ... ... ... 142/142 unit tests passed ``` --- Will handle exceptions in oobgrader and test in following diff Reviewed By: doranand Differential Revision: D60115096 fbshipit-source-id: 77e0cb6aa18fdb71e4c711d3951c7282aab23b6a
flashy-5074d45
sleep briefly while flashing Summary: Go idle periodically when flashing the image, to allow other work on the system to happen, for example SPI I/O. For a 32MB flash image, this would result in an extra 8s of idle time during wipe plus 8s during flashing. For S434121. Test Plan: ``` 0 ~/local/openbmc/tools/flashy $ ./build.sh && ./build_dev.sh && go test ./... ? github.com/facebook/openbmc/tools/flashy/flash_procedure [no test files] ? github.com/facebook/openbmc/tools/flashy/lib/logger [no test files] ? github.com/facebook/openbmc/tools/flashy/tests [no test files] ? github.com/facebook/openbmc/tools/flashy/utilities [no test files] ok github.com/facebook/openbmc/tools/flashy 2.024s ok github.com/facebook/openbmc/tools/flashy/checks_and_remediations/bletchley (cached) ok github.com/facebook/openbmc/tools/flashy/checks_and_remediations/common (cached) ok github.com/facebook/openbmc/tools/flashy/checks_and_remediations/galaxy100 (cached) ok github.com/facebook/openbmc/tools/flashy/checks_and_remediations/grandteton (cached) ok github.com/facebook/openbmc/tools/flashy/checks_and_remediations/wedge100 (cached) ok github.com/facebook/openbmc/tools/flashy/checks_and_remediations/yamp (cached) ok github.com/facebook/openbmc/tools/flashy/install 0.007s ok github.com/facebook/openbmc/tools/flashy/lib/fileutils (cached) ok github.com/facebook/openbmc/tools/flashy/lib/flash 0.007s ok github.com/facebook/openbmc/tools/flashy/lib/flash/flashcp 0.508s ok github.com/facebook/openbmc/tools/flashy/lib/flash/flashutils (cached) ok github.com/facebook/openbmc/tools/flashy/lib/flash/flashutils/devices (cached) ok github.com/facebook/openbmc/tools/flashy/lib/step (cached) ok github.com/facebook/openbmc/tools/flashy/lib/utils (cached) ok github.com/facebook/openbmc/tools/flashy/lib/validate (cached) ok github.com/facebook/openbmc/tools/flashy/lib/validate/image (cached) ok github.com/facebook/openbmc/tools/flashy/lib/validate/partition (cached) 0 ~/local/openbmc/tools/flashy $ ``` Build ephemeral fbpkg and flash a wedge100 using it: ``` Host Workflow ID Progress Status Result ---------------------------------- ------------------------------------ ---------- ----------------------- ---------------------- fboss8382003-oob.snc1.facebook.com b4600065-e674-4696-9378-5c313ae45819 finished WorkflowStatus.FINISHED FinishStatus.SUCCEEDED ``` -> https://fburl.com/scuba/openbmc_upgrades/yksvon8b Reviewed By: kawmarco Differential Revision: D59927810 fbshipit-source-id: 7db5bb534575d360816a2ec27bea361936fe16dd
flashy-652f5d7
use consistent watchdog logic Summary: When it's impossible to extend the watchdog timeout because the character device is held open by another process, we have a backup plan: wait 10s before each step when erasing/writing flash chips so that whatever thread manages the watchdog has an opportunity to pet it. For some reason this logic was skipped on LF OpenBMC. Re-enable it because regardless of whether healthd / fscd / systemd is managing the watchdog we should do the same thing. Test Plan: ``` 0 ~/local/openbmc/tools/flashy $ ./build.sh && ./build_dev.sh && go test ./... ? github.com/facebook/openbmc/tools/flashy/flash_procedure [no test files] ? github.com/facebook/openbmc/tools/flashy/lib/logger [no test files] ? github.com/facebook/openbmc/tools/flashy/tests [no test files] ? github.com/facebook/openbmc/tools/flashy/utilities [no test files] ok github.com/facebook/openbmc/tools/flashy 2.116s ok github.com/facebook/openbmc/tools/flashy/checks_and_remediations/bletchley 0.006s ok github.com/facebook/openbmc/tools/flashy/checks_and_remediations/common 0.250s ok github.com/facebook/openbmc/tools/flashy/checks_and_remediations/galaxy100 0.009s ok github.com/facebook/openbmc/tools/flashy/checks_and_remediations/grandteton 0.009s ok github.com/facebook/openbmc/tools/flashy/checks_and_remediations/wedge100 0.009s ok github.com/facebook/openbmc/tools/flashy/checks_and_remediations/yamp 0.009s ok github.com/facebook/openbmc/tools/flashy/install 0.009s ok github.com/facebook/openbmc/tools/flashy/lib/fileutils (cached) ok github.com/facebook/openbmc/tools/flashy/lib/flash 0.008s ok github.com/facebook/openbmc/tools/flashy/lib/flash/flashcp 0.008s ok github.com/facebook/openbmc/tools/flashy/lib/flash/flashutils 0.010s ok github.com/facebook/openbmc/tools/flashy/lib/flash/flashutils/devices 0.009s ok github.com/facebook/openbmc/tools/flashy/lib/step 0.008s ok github.com/facebook/openbmc/tools/flashy/lib/utils 0.443s ok github.com/facebook/openbmc/tools/flashy/lib/validate 0.008s ok github.com/facebook/openbmc/tools/flashy/lib/validate/image 0.008s ok github.com/facebook/openbmc/tools/flashy/lib/validate/partition 0.022s ``` Build ephemeral fbpkg with these changes and force-flash a bletchley that's in repair (first chip only): ``` 1 ~ $ oobgrader --host macbmc1r0036p0009-oob.03.pci1 --wait --flashy-tag 9640644 --force --allow-downgrade ... Host Workflow ID Progress Status Result ----------------------------- ------------------------------------ ---------- ----------------------- ---------------------- macbmc1r0036p0009-oob.03.pci1 041870c1-4e8b-4194-9d69-ef27dfb42286 finished WorkflowStatus.FINISHED FinishStatus.SUCCEEDED ``` -> https://fburl.com/scuba/openbmc_upgrades/5ogr4a5q Reviewed By: williamspatrick Differential Revision: D59917900 fbshipit-source-id: 7cd7050c75226b1badc98fb043fd9f1f98b6f3a0
flashy-37525d4
note tahan, morgan800cc as a BMC-lite Summary: Otherwise we will accidentally start the watchdog and reset while wiping the second chip: https://www.internalfb.com/intern/provisioning/asset_details/911771887375906 XXX This needs to be table driven off platforms.thrift. Test Plan: ``` 0 ~/local/openbmc/tools/flashy $ ./build.sh && ./build_dev.sh && go test ./... && echo it works ok github.com/facebook/openbmc/tools/flashy (cached) ok github.com/facebook/openbmc/tools/flashy/checks_and_remediations/bletchley 0.006s ? github.com/facebook/openbmc/tools/flashy/flash_procedure [no test files] ? github.com/facebook/openbmc/tools/flashy/lib/logger [no test files] ok github.com/facebook/openbmc/tools/flashy/checks_and_remediations/common (cached) ok github.com/facebook/openbmc/tools/flashy/checks_and_remediations/galaxy100 (cached) ok github.com/facebook/openbmc/tools/flashy/checks_and_remediations/grandteton (cached) ok github.com/facebook/openbmc/tools/flashy/checks_and_remediations/wedge100 (cached) ok github.com/facebook/openbmc/tools/flashy/checks_and_remediations/yamp (cached) ok github.com/facebook/openbmc/tools/flashy/install (cached) ? github.com/facebook/openbmc/tools/flashy/tests [no test files] ? github.com/facebook/openbmc/tools/flashy/utilities [no test files] ok github.com/facebook/openbmc/tools/flashy/lib/fileutils 0.010s ok github.com/facebook/openbmc/tools/flashy/lib/flash (cached) ok github.com/facebook/openbmc/tools/flashy/lib/flash/flashcp (cached) ok github.com/facebook/openbmc/tools/flashy/lib/flash/flashutils 0.006s ok github.com/facebook/openbmc/tools/flashy/lib/flash/flashutils/devices (cached) ok github.com/facebook/openbmc/tools/flashy/lib/step (cached) ok github.com/facebook/openbmc/tools/flashy/lib/utils 0.441s ok github.com/facebook/openbmc/tools/flashy/lib/validate 0.007s ok github.com/facebook/openbmc/tools/flashy/lib/validate/image (cached) ok github.com/facebook/openbmc/tools/flashy/lib/validate/partition 0.017s it works 0 ~/local/openbmc/tools/flashy $ ``` Reviewed By: gepoggio Differential Revision: D59759896 fbshipit-source-id: b5c742f8370dc5f79da403d41be3c096b534ca0e
flashy-54734f8
Ensure BMC image is compatible with MB CPLD version
Summary: grandteton versions v2024.11.1 require MB CPLD version `20010` or greater. Added `checks_and_remediations` step to fail the upgrade if the MB CPLD version is incompatible with the image version.
Test Plan:
Unit tests, push flashy/bmc images to grandteton units e.g:
```
$ scripts/run_flashy_remote.sh --device mtd:flash1 --imagepath /tmp/tmp.H8gzWMVfip/openbmc.image.grandteton\:v2024.11.1/flash-grandteton --host twshared39604-oob.02.nao6.facebook.com --dry-run
```
Run step on a BMC with image version = v2024.11.1 and old MB CPLD (should fail):
```
# /run/flashy/flashy --install && /run/flashy/checks_and_remediations/grandteton/00_ensure_compatible_mb_cpld_version --imagepath /run/upgrade/image --device mtd:flash1
...
{"message":"S365492: Image version ('grandteton-v2024.11.1') requires MB CPLD version 20010 or greater (current MB CPLD version is 20006)"}
```
Run step on a BMC with image version = v2024.11.1 and new MB CPLD (should succeed):
```
/run/flashy/flashy --install && /run/flashy/checks_and_remediations/grandteton/00_ensure_compatible_mb_cpld_version --imagepath /run/upgrade/image --device mtd:flash1
...
2024/04/09 08:33:48 MB CPLD version 20010 is compatible with image version grandteton-v2024.11.1
```
Run step on a BMC with image version < v2024.11.1 and old MB CPLD (should succeed):
```
# /run/flashy/flashy --install && /run/flashy/checks_and_remediations/grandteton/00_ensure_compatible_mb_cpld_version --imagepath /run/upgrade/image --device mtd:flash1
...
2024/04/09 08:35:46 MB CPLD version 20006 is compatible with image version grandteton-v2023.39.2
```
Run step on a BMC with image version < v2024.11.1 and new MB CPLD (should succeed):
```
# /run/flashy/flashy --install && /run/flashy/checks_and_remediations/grandteton/00_ensure_compatible_mb_cpld_version --imagepath /run/upgrade/image --device mtd:flash1
...
2024/04/09 08:40:29 MB CPLD version 20010 is compatible with image version grandteton-v2023.39.2
```
Reviewed By: doranand
Differential Revision: D55921434
fbshipit-source-id: 8672d76c8f4ef18978cda8826f73395b1309b50c
flashy-eaa1917
javaisland: add flashy support Summary: Add the flashy support for JavaIsland platform. Test Plan: Built flash with tools/flashy/build.sh Reviewed By: williamspatrick Differential Revision: D55333620 fbshipit-source-id: 86363e06b8d98f020f0ebe25a91e0950c5cb164b
flashy-37fefba
Allow for unexpcted version names in checks & remediations Summary: Upgrading from a wiped bletchley fails, due to image version name validation in flashy checks and remediations ``` 2024-03-13 02:41:18.275510 - Unsafe to reboot error: "Unable to parse version info: No match for regex 'bletchley-v(?P<year>[0-9]+).(?P<week>[0-9]+)' for input 'bletchley-9e5605ae7d-dirty'". ``` We should allow for Flashy to upgrade from versions without a `vYYYY.WW.N` version tag, such as the OSS image build or locally built versions, and only proceed to the year check if it's an actual date tagged version Test Plan: Test upgrade from same device: ``` $ oobgrader --host macbmc5r0034p0049-oob.02.pci1.facebook.com --fbpkg-name openbmc.image.bletchley:weeklybuild --flashy-tag e44d3bf --wait Host Workflow ID Progress Status Result ------------------------------------------ ------------------------------------ ---------- ----------------------- ---------------------- macbmc5r0034p0049-oob.02.pci1.facebook.com 44c2f034-d1ae-4bc3-b692-40c17113f1ac finished WorkflowStatus.FINISHED FinishStatus.SUCCEEDED ``` Test upgrade on wedge100 to ensure normal behaviour ``` $ oobgrader --host fboss8382003.snc1 --primary-only --wait --force --flashy-tag e44d3bf --allow-downgrade Host Workflow ID Progress Status Result --------------------- ------------------------------------ ---------- ----------------------- ---------------------- fboss8382003-oob.snc1 647a8025-e0c2-4977-8208-dbede581f229 finished WorkflowStatus.FINISHED FinishStatus.SUCCEEDED ``` --- Unit tests: ``` ~/local/openbmc/tools/flashy (flashy-reg)]$ ./scripts/run_unit_tests.sh 141/141 unit tests passed ``` Reviewed By: doranand Differential Revision: D54872760 fbshipit-source-id: 0952d175e12fb9c9e958db9563267e40cadaab1f
flashy-48e2599
minerva: flashy: add flash procedure Summary: Copy flash procedure from bletchley for minerva and harma update. Test Plan: Build flashy. Reviewed By: kawmarco Differential Revision: D54686413 fbshipit-source-id: 16b557f53714a2ee145730cf14d48975e4d65800