Testing guide

Tell us you are on version 4

Issues are tracked in the App-Auto-Patch/App-Auto-Patch repository, which also covers the original 3.x script. When you file one, choose the form for App Auto-Patch 4 (the native app). It marks the issue as version 4, so the development team sees it, and it asks for the exact version and build from Settings → About. If you used another form, say that you are using version 4 and a maintainer will add the label.

A structured walk-through for testers evaluating AppAutoPatch (AAP) 4.x. Each scenario lists a goal, preconditions, numbered steps, and what “pass” looks like. Work through them in order on a clean Mac and you’ll exercise every major surface in the app. If any test produces unexpected behavior, stop, generate a diagnostic bundle, and file an issue (see Reporting an issue).

Before you start

• macOS 14 (Sonoma) or later. macOS 15 (Sequoia) is the primary development platform.
• Apple Silicon or Intel.
• Admin rights (you’ll be approving a root LaunchDaemon).
• Recommended: a Mac without a previous AAP install. If you’ve run AAP before, see Resetting state between test runs first.
• The signed .pkg from the Download page.

Reporting an issue

The fastest path to a fix is a diagnostic bundle:

1. Open AppAutoPatch.
2. Settings → About → Diagnostics → Generate Diagnostic Bundle.
3. Drag the resulting .zip from your Desktop into a new issue in App-Auto-Patch/App-Auto-Patch using the Bug report template. The bundle contains logs, the redacted preferences plist, helper state, and the most recent patching receipts. Webhook URLs are redacted before the bundle is written; nothing leaves your Mac without your consent.

---

1. Installation & first launch

1a. Fresh install via .pkg

Goal: confirm the installer completes without errors and seeds the expected directory layout.

1. Double-click AppAutoPatch-<version>.pkg.
2. Click through Continue → Install → authenticate as admin.
3. Wait for the “Installation was successful” screen. Pass: the package finishes without warning, and /Library/Application Support/AppAutoPatch/ exists with a jobs/ subfolder.

1b. First-run setup wizard

Goal: the guided 3-step setup appears on first launch and walks you through helper approval, Installomator deploy, and initial discovery.

1. Open /Applications/App Auto-Patch.app.
2. The Status pane should show “Welcome to App Auto-Patch” with three numbered steps.
3. Step 1: click Register Helper (or Open Login Items & Extensions if prompted), then approve AppAutoPatch in System Settings.
4. Return to AAP — Step 1 should auto-advance to a green check.
5. Step 2 (Installomator) auto-progresses while the helper deploys Installomator. This typically takes 10–30 seconds the first time.
6. Step 3: click Discover Apps.
7. After discovery completes, click Done. Pass: the dashboard appears with discovered apps; subsequent launches go directly to the dashboard, no wizard.

1c. Auto-skip on already-managed Macs

Goal: confirm the wizard auto-completes when prerequisites are already in place.

1. With the helper approved and Installomator deployed, run defaults delete xyz.techitout.appAutoPatch FirstRunSetupCompleted in Terminal.
2. Quit and relaunch AAP. Pass: the dashboard opens directly. No wizard appears.

---

2. Discovery & manual updates

2a. Discover Applications

1. From the dashboard, click Discover Applications.
2. Wait for the scan to complete. Pass: the list of installed third-party apps populates with Current versions but no Latest versions yet.

2b. Check For Updates

1. Click Check For Updates.
2. Wait — this typically takes 1–3 minutes depending on the size of your Installomator label set. Pass: each row shows both Current and Latest. Apps needing updates have an orange download icon; up-to-date apps show a green check; apps that failed the check show a red icon.

2c. Update a single app

Goal: per-row patching works.

1. Pick an app with an available update (orange icon).
2. Click the orange download icon.
3. Watch the in-progress banner (“Updating labelname (1 of 1)…”). Pass: the app’s Current version updates to match Latest; the icon flips to green. A receipt is written (verify in 8a below).

2d. Update All

1. Click Update All in the dashboard header.
2. The deferral prompt appears. Click Install Now. Pass: every app with a pending update is patched in sequence.

2e. Updates-available notification

Goal: confirm the new “App Auto-Patch: N updates available” banner fires when a Check For Updates run surfaces fresh pending updates, and is suppressed in the foreground. Preconditions: notification permission was granted on a previous launch (any background-patch run is enough), and at least one new app update is available since the last check.

1. Send AAP to the background (Cmd+H, or focus another app).
2. Click Check For Updates from the menu bar popover, or relaunch AAP and let the dashboard’s onAppear chain trigger one.
3. Wait for the check to finish.
4. Click Check For Updates again with no new updates pending.
5. Bring AAP forward (Cmd+Tab) and run Check For Updates once more. Pass: step 3 produces exactly one banner notification titled “App Auto-Patch: N update(s) available” with body “Open App Auto-Patch to install or defer.” Step 4 produces no notification (count didn’t grow). Step 5 also produces no notification because the dashboard is foreground. Clicking the notification opens App Auto-Patch’s deferral prompt. Disable test: Settings → Advanced → Background Patching → Show a notification when new updates become available → off, Save Settings, then repeat steps 1–3. The toast should not appear.

---

3. Deferrals & deadlines

Clicking Update All opens the deferral prompt whenever there are pending updates. Two thresholds escalate the flow:

• A soft focus deadline tightens the prompt without forcing.
• A hard deadline force-installs with no prompt. Both can fire by count (deferrals so far) or by days (a date seeded on the first deferral of the cycle).

3a. DeferralWatcher resurrects the prompt automatically

Goal: confirm DeferralWatcher re-engages the prompt when NextAutoLaunch elapses, without the user having to click Update All again.

1. From the dashboard, click Update All with at least one app pending.
2. In the prompt, pick the shortest defer option available.
3. Click Defer.
4. Leave AAP open and wait out the deferral. Pass: within a few seconds of the deadline elapsing, the prompt re-appears on its own. Logs include “DeferralWatcher: Armed watchpoint” when the deferral is saved, then “Deadline trigger fired” at that time. If a check or patch is still running, the log says it is retrying in 15s rather than waiting out the one-minute timer.

3b. Hard count deadline forces install

Goal: confirm DeadlineCountHard triggers a forced install with no prompt.

1. defaults write xyz.techitout.appAutoPatch DeadlineCountHard -int 2.
2. Defer twice via Update All — or fast-forward the counter: defaults write xyz.techitout.appAutoPatch HardDeferralCount -int 2.
3. Click Update All. Pass: AAP skips the prompt and patches immediately. Logs include “StatusViewModel: Hard deadline reached — installing immediately.”

3c. Hard day deadline forces install

Goal: confirm DeadlineDaysHard works end-to-end (date seeded on first deferral, checked against Date()).

1. defaults write xyz.techitout.appAutoPatch DeadlineDaysHard -int 1.
2. Click Update All → Defer once. The prompt’s footer should read “Auto-installs by <date 1 day out>.”
3. Fast-forward the persisted date so it’s already in the past: defaults write xyz.techitout.appAutoPatch HardDeferralDate "$(date -u -v-1d +%Y-%m-%dT%H:%M:%SZ)"
4. Click Update All. Pass: AAP force-installs without showing the prompt. Same hard­ deadline log line as 3b.

3d. A reached focus limit uses the normal dialog

Goal: once either focus limit is met, reminders continue with the normal dialog. There is no separate “strongly recommended” copy.

1. defaults write xyz.techitout.appAutoPatch DeadlineCountFocus -int 1.
2. defaults write xyz.techitout.appAutoPatch FocusDeferralCount -int 1.
3. Click Update All. Pass: the prompt headline is “Updates Available” and the button is “Defer”. The timer picker still shows the full DeferralTimerMenu. HardDeferralCount increases by 1 if you defer. FocusDeferralCount stays at 1.

3e. Automatic focus deferral

Goal: Focus, Do Not Disturb, or a display-sleep assertion defers silently until a focus limit is reached. A manual Update All still shows the dialog.

1. defaults write xyz.techitout.appAutoPatch DeadlineCountFocus -int 2.
2. defaults write xyz.techitout.appAutoPatch DeferralTimerFocus -int 1.
3. defaults delete xyz.techitout.appAutoPatch FocusDeferralNotificationsEnabled.
4. Start caffeinate -d in Terminal, or turn on Focus.
5. With a pending update, set NextAutoLaunch a few seconds in the past: defaults write xyz.techitout.appAutoPatch NextAutoLaunch "$(date -u -v-1M +%Y-%m-%dT%H:%M:%SZ)" Pass: no dialog and no toast. The menu bar deadline row reads “Focus deferral — checking again in…” with a moon icon. The log says “Automatic focus deferral”. FocusDeferralCount is 1 and HardDeferralCount is unchanged. NextAutoLaunch is about 1 minute ahead. Click Update All while caffeinate -d is still running: the normal dialog appears, and deferring it increases HardDeferralCount only. Repeat the past-NextAutoLaunch step once more. The second automatic deferral sets FocusDeferralCount to 2. Do it a third time: the normal dialog and toast appear even though Focus is still active. Toast override: defaults write xyz.techitout.appAutoPatch FocusDeferralNotificationsEnabled -bool true posts the “Your deferral has ended” toast on an automatic deferral. The dialog stays hidden. Clicking the toast opens the normal dialog. Ignore list: defaults write xyz.techitout.appAutoPatch IgnoreDNDApps -string "caffeinate" makes caffeinate -d no longer count. A real Focus mode still defers.

3f. Aggressive patch-day deferral capping

Goal: confirm AggressivePatchDayDeferral shrinks the picker options as PatchingEndTime approaches, and bypasses the prompt entirely once the window has passed.

1. Enable the feature: defaults write xyz.techitout.appAutoPatch AggressivePatchDayDeferral -bool true
2. Trigger a hard deadline (per 3b or 3c).
3. Set PatchingEndTime to 45 minutes from now (e.g. if it’s 14:00, use 14:45): defaults write xyz.techitout.appAutoPatch PatchingEndTime "HH:MM"
4. Click Update All. Pass (30–60 min remaining bracket): the picker shows only options ≤15 minutes. Click Defer — if all your configured options are larger, a synthesised 5-minute entry appears.
5. Now set PatchingEndTime to 1 minute from now and click Update All again. Pass (past window): the deferral prompt is bypassed entirely and patching starts immediately — same as a hard-deadline force-install. Clean up: defaults delete xyz.techitout.appAutoPatch AggressivePatchDayDeferral PatchingEndTime

3g. Deferral audit trail

Goal: confirm user-initiated deferrals, focus suppressions, and forced installs are recorded on disk and visible in the dashboard.

1. With a pending update, click Update All → Defer.
2. In Terminal: ls /Library/Application\ Support/AppAutoPatch/deferral-events/ Pass: a JSON file is present. Expand that app’s row in the dashboard — a Deferral Events section appears below the receipt history showing event type (userDeferred), escalation, minutes, and timestamp. After running test 3b or 3c (forced install), a .deadlineForced event appears for those labels. After 3e (focus suppression), a .focusSuppressed event appears.
3. Generate a diagnostic bundle (Settings → About → Generate Diagnostic Bundle) and open the zip — confirm a deferral-events/ folder is present alongside receipts/.

3h. A successful patch ends the cycle

Goal: confirm resetDeferralCounts() runs after a successful patch and clears every cycle key.

1. After any of 3a–3g, run a successful Update All → Install Now (or let the hard deadline force-install).
2. After the run completes, in Terminal: defaults read xyz.techitout.appAutoPatch | grep -E 'FocusDeferralCount|HardDeferralCount|FocusDeferralDate|HardDeferralDate|NextAutoLaunch|JitterOffset' Pass: FocusDeferralCount and HardDeferralCount are 0. None of FocusDeferralDate, HardDeferralDate, NextAutoLaunch, or BackgroundPatchStartupJitterOffsetMinutes appear (all cleared by resetDeferralCounts()). The next round of pending updates starts a fresh deferral cycle with a new random jitter offset.

---

4. Background patching

AAP runs in one of two scheduler modes, controlled by the BackgroundPatchClosedAppsScheduler preference:

• app (default) — in-process timer in the GUI app.
• helper — runs inside the persistent root LaunchDaemon.

4a. Update staging

Goal: detected updates are downloaded before a closed-app patch, and the install reuses that file.

1. Leave WorkflowStageUpdates at its default (true), or confirm Settings → Advanced → Background Patching → Download installers before installing is on.
2. Make sure at least one closed app has a pending update.
3. Let a background pass run (in-app or helper daemon). Pass: when a closed app is pending, the log shows Patching closed apps before the open-apps notification and the “open apps need updating” notification arrives only after Closed-app pass finished; delivering the open-apps notification. If every pending app was closed, that notification is not posted. While the pass runs, the dashboard banner reads “Background patching in progress…” and does not revert to “Patching in progress…” or “Updating <app>…”. The log also shows Staging N update(s) before the open-apps notification when an open app still needs a toast, and that toast arrives only after Staging complete. The log also shows Staging N detected update(s) before the closed-app pass (or Staging N update(s) before installing) before Installomator runs, then Using staged installer. A successful install logs Removed staged installer for that label only. Updating one app from its row leaves other labels’ files in /private/tmp/AAPStage. Set WorkflowStageUpdates to false and confirm the next pass does not stage and the open-apps notification is not delayed.

4b. In-app scheduler smoke test

1. Settings → Advanced → Background Patching.
2. Confirm Enabled is on, Scheduler is In-app, and the interval is 1 hour (the test minimum).
3. Make sure at least one closed app on the dashboard has a pending update.
4. Quit any running instance of that app and leave AAP running.
5. Wait one hour (or set the interval lower for testing — the floor the scheduler enforces is 1 hour, so plan around that). Pass: the app gets patched silently; a notification appears titled “AppAutoPatch updated apps”.

4c. Helper-daemon scheduler smoke test

Goal: background passes survive a full quit of the GUI app.

1. Settings → Advanced → Background Patching → Scheduler → Helper daemon.
2. Click Save Settings.
3. Quit AAP entirely (menu bar popover → Quit AppAutoPatch).
4. Wait out the configured interval. Pass: when you next launch AAP, you should see logs in the in-app Logs pane confirming a HelperBackgroundPatchScheduler pass ran. If you re-enabled the GUI’s helper observer (the default), a single completion toast appears within 60 seconds of relaunch.

4d. AC-power gate

1. Confirm BackgroundPatchClosedAppsRequireOnAC is true (the default).
2. Disconnect AC, close an app with a pending update, and refresh AAP.
3. Refresh again without changing the pending count. Pass: the log shows “On battery and require-AC is enabled; skipping run.” No patches happen until you reconnect AC. Exactly one persistent App updates are waiting notification asks the user to connect power; step 3 does not duplicate it. Reconnect AC and confirm that the notification clears and patching resumes.

4e. Notification routing

1. Install the example managed notification profile and grant notification access to both App Auto-Patch identities.
2. Trigger a background-patch start/completion and an updates-available or AC-power-required notification.
3. Set TemporaryNotificationTypes to ["updatesAvailable"] and PersistentNotificationTypes to ["backgroundPatchCompleted"], then repeat. Pass: informational background notifications are temporary banners by default and action-required notifications are persistent alerts. Step 3 reverses those two selected routes. If the same type appears in both arrays, it uses the persistent identity.

4f. Allow / block list

1. Add a single label (e.g. firefox) to the Block list in Background Patching settings.
2. Force a tick (set interval to 1 hour and wait, or restart AAP). Pass: Firefox is skipped even if it’s eligible. Other apps still patch.

4g. Captive portal guard

Goal: confirm the scheduler skips network phases when internet is unavailable or behind a captive portal.

1. Block captive.apple.com at the hosts level: sudo sh -c 'echo "0.0.0.0 captive.apple.com" >> /etc/hosts'
2. Force a background-scheduler tick. Pass: log includes “No internet / captive portal detected — skipping network phases this tick.” No update-check or label-refresh activity is logged. Remove the hosts entry when done: sudo sed -i '' '/captive.apple.com/d' /etc/hosts

---

5. Self-update

Prerequisite: an appautopatch Installomator label exists in your deployed Installomator instance, OR you set SelfUpdateLabel to a label you control.

1. Settings → Advanced → Self-Update → set frequency to Daily and enable.
2. Restart AAP.
3. Wait roughly 5 minutes (the scheduler’s startup grace period). Pass (interactive mode): a dashboard banner appears titled “AppAutoPatch update available” with Update Now and Later buttons. Clicking Update Now triggers the same XPC patch path used by every other app; AAP quits and is replaced by the new version. Pass (silent mode, InteractiveMode == 0): AAP installs the new version automatically without surfacing a banner.

---

6. Menu bar & window lifecycle

6a. Menu bar persistence

1. Open AAP, then close the dashboard window (red close button or Cmd+W). Pass: the dashboard closes; the menu bar item stays visible. The Dock icon disappears (app drops to .accessory).

6b. Cmd+Q behavior

1. With AAP open, press Cmd+Q. Pass: the dashboard window closes; the menu bar item remains. The process keeps running. The only way to fully terminate is the menu bar popover’s Quit AppAutoPatch button.

6c. Reopen from menu bar

1. Click the menu bar item.
2. Click Open Dashboard. Pass: the dashboard window reappears with the Dock icon back.

---

7. Settings & MDM behavior

7a. Local Settings persist

1. Change any preference (e.g. add a label to Ignored Labels).
2. Click Save Settings.
3. Quit AAP completely (menu bar → Quit) and relaunch. Pass: your change is still there.

7b. Managed preferences override

Goal: confirm that an MDM-delivered key is honored and the corresponding Settings field is disabled.

1. Deploy a configuration profile setting IgnoredLabels to a known list, using the manifest in Profile Manifests/.
2. Open AAP → Settings → Advanced. Pass: the Ignored Labels field shows the managed value and is disabled. A “Provided by your organization through managed configuration” caption appears on each managed section.

7c. 3.5.0 profile carryover

1. Apply an existing AAP 3.5.0 configuration profile that uses the IgnoredLabels key (and any others — RequiredLabels, OptionalLabels, InteractiveMode, etc.).
2. Open AAP. Pass: every key is honored. No edits to the profile were needed.

---

8. Receipt history

8a. Per-app history

Prerequisite: at least one patch run has completed (test 2c satisfies this).

1. On the dashboard, click the chevron on the right of any patched app’s row. Pass: a Recent patches panel appears showing up to 5 receipts with timestamp, version, and status icon. Failed runs show their Installomator exit code in red.

8b. Empty history

1. Expand the row of an app you’ve never patched. Pass: the panel reads “No patches recorded for this label yet.”

---

9. Diagnostic bundle

1. Settings → About → Diagnostics → Generate Diagnostic Bundle.
2. After ~5 seconds, click Reveal in Finder.
3. Open the resulting zip. Pass: the zip contains

• metadata.txt (app version, helper status, cdhash summary)
• prefs/user.plist and (if applicable) prefs/managed.plist — open them and confirm WebhookURLSlack / WebhookURLTeams are <redacted>, and DiscoveredAppsData is absent.
• logs/aap.log and (if rotated) logs-archive/
• helper-state.json (if helper-daemon mode has run at least once)
• receipts/ (if any patching has happened)
• deferral-events/ (if any deferrals, focus suppressions, or forced installs have occurred)

---

10. Helper recovery

10a. Manual Reset Helper

1. Settings → About → Reset Helper → Reset. Pass: status flips to Re-registered. Approve in System Settings… or Helper re-registered and is running. depending on whether macOS needs re-approval.

10b. Auto-reset on upgrade (cdhash mismatch)

1. Install AAP version A; let it register the helper.
2. Install AAP version B (different build, hence different cdhash) on top.
3. Launch AAP. Pass: the dashboard briefly shows “Refreshing root helper…” and auto-runs the reset. The status banner clears once the helper is re-bound. Logs include “Helper binary cdhash changed since last registration”.

10c. Operationally-broken auto-reset

1. Manually break the helper (e.g. revoke its approval in System Settings → Login Items & Extensions).
2. Quit and relaunch AAP. Pass: AAP detects the helper isn’t operational and silently attempts a reset, capped at one attempt per session and once per hour.

---

Resetting state between test runs

If you want to start fresh, run the following:

# Stop and unregister the helper
sudo launchctl bootout system/xyz.techitout.appAutoPatch.roothelper 2>/dev/null
sudo rm -f /Library/LaunchDaemons/xyz.techitout.appAutoPatch.roothelper.plist
# Wipe app state
sudo rm -rf "/Library/Application Support/AppAutoPatch"
# Wipe local prefs
defaults delete xyz.techitout.appAutoPatch 2>/dev/null
# Optionally remove the .app (handles both new and pre-rename paths)
sudo rm -rf "/Applications/App Auto-Patch.app" "/Applications/AppAutoPatch.app"

Then re-install the .pkg from scratch and start at test 1a again.

What we’re hoping you’ll catch

• Behaviors that surprise you in any direction. “It worked, but I didn’t expect that” is just as useful as a hard failure.
• Wording on banners / notifications that’s confusing or misleading.
• Edge cases the matrix above doesn’t cover, especially:
  • Multi-user Macs (Fast User Switching, multiple console users)
  • Locked-screen / lid-closed behavior of the background scheduler
  • Recovery after a forced power-off mid-patch
  • Macs with restrictive endpoint security tools (CrowdStrike, SentinelOne, Sophos, etc.) that may interfere with the helper
• Anything an MDM-managed Mac does differently from an unmanaged one. Thank you for testing!