Add tests for OS update and boot slot switching #4108
Conversation
Add test that OS update works - use the whole stack using CLI to update to the latest stable version (unless executed manually on the latest stable release, this version should never be the same as the currently tested one). With this test in place, we can also test command for switching the slots, so add an extra test for that too. Fixes #4103
📝 WalkthroughWalkthroughA new smoke test suite for OS update functionality has been added using pytest. The suite includes tests for system initialization, performing an OS update to the latest stable version, and switching the boot slot. The tests use shell commands, handle reboots, and verify system states and versions through JSON parsing and retries. Changes
Sequence Diagram(s)sequenceDiagram
participant Tester
participant Shell
participant System
participant RemoteServer
Tester->>Shell: Wait for containers (homeassistant, hassio_supervisor)
Shell->>System: Query container status
Shell->>System: Query system status (ready, supervisor version)
Tester->>RemoteServer: Fetch latest OS version (JSON)
Tester->>Shell: Run OS update command to target version
Shell->>System: Perform OS update
System-->>Shell: Reboot
Shell->>System: Wait for readiness after reboot
Tester->>Shell: Switch boot slot
Shell->>System: Switch slot and reboot
Shell->>System: Wait for readiness and verify slot version
Assessment against linked issues
Assessment against linked issues: Out-of-scope changesNo out-of-scope changes were found. ✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🧹 Nitpick comments (5)
tests/smoke_test/test_os_update.py (5)
6-6: Remove unused import.The
ExecutionErrorimport is never used in the code.-from labgrid.driver import ExecutionError
8-8: Consider removing unused logger.The logger is defined but never used throughout the test file. Consider removing it or adding appropriate logging statements for better test observability.
31-31: Improve JSON parsing robustness.The JSON parsing could be more robust. Consider using a try-catch block for better error handling.
- supervisor_info = json.loads(supervisor_info) if supervisor_info.startswith("{") else None + try: + supervisor_info = json.loads(supervisor_info) if supervisor_info.startswith("{") else None + except json.JSONDecodeError: + supervisor_info = None
87-87: Remove unnecessary f-string prefix.The f-string has no placeholders, so the
fprefix is unnecessary.- shell.run_check(f"ha os boot-slot other --no-progress || true") + shell.run_check("ha os boot-slot other --no-progress || true")
44-78: Consider adding logging for better test observability.Since this is a smoke test that involves system updates and reboots, adding logging statements would help with debugging test failures and understanding test progress.
def test_os_update(shell, shell_json, target): + _LOGGER.info("Starting OS update test") # fetch version info and OTA URL shell.run_check("ha su reload --no-progress") # update OS to latest stable - in tests it should never be the same version stable_version = shell_json("curl -sSL https://version.home-assistant.io/stable.json")["hassos"]["ova"] + _LOGGER.info(f"Target stable version: {stable_version}")
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Cache: Disabled due to data retention organization setting
Knowledge Base: Disabled due to data retention organization setting
📒 Files selected for processing (1)
tests/smoke_test/test_os_update.py(1 hunks)
🧰 Additional context used
🪛 Ruff (0.11.9)
tests/smoke_test/test_os_update.py
6-6: labgrid.driver.ExecutionError imported but unused
Remove unused import: labgrid.driver.ExecutionError
(F401)
87-87: f-string without any placeholders
Remove extraneous f prefix
(F541)
| while True: | ||
| output = "\n".join(shell.run_check(f"ha os update --no-progress --version {stable_version} || true", timeout=120)) | ||
| if "Don't have an URL for OTA updates" in output: | ||
| shell.run_check("ha su reload --no-progress") |
There was a problem hiding this comment.
There is the /reload_updates API endpoint nowadays which is a bit more lightweight. But I think we have no CLI for that 😢
Add test that OS update works - use the whole stack using CLI to update to the latest stable version (unless executed manually on the latest stable release, this version should never be the same as the currently tested one). With this test in place, we can also test command for switching the slots, so add an extra test for that too. Fixes #4103 (cherry picked from commit 90d3614)
Add test that OS update works - use the whole stack using CLI to update to the latest stable version (unless executed manually on the latest stable release, this version should never be the same as the currently tested one).
With this test in place, we can also test command for switching the slots, so add an extra test for that too.
Fixes #4103
Summary by CodeRabbit