Skip to main content

Keep flows working when the app changes

Automation built on what the screen looks like has a standing dependency on what the screen looks like. When the app you automate ships an update — or you change the emulator's resolution — some things break and some do not. Knowing which is which saves rebuilding what was never broken.

What breaks, in order of likelihood​

ThingSurvives a UI change?Why
Image templatesNo — first to breakMatching is pixel-based. A restyled button is a different image
Typed coordinatesNoNothing anchors them to the thing you meant
Drawn regionsUsuallyStored as percentages of the screen, so they survive a resolution change — but not a moved element
OCR content typesUsuallyNumber stays Number even if the font changes
Flow structure, variables, sub flow callsYesBound by id and by name, not by appearance

The practical consequence: a UI update is normally a template problem, not a flow problem. Do not rebuild the graph.

Fix it in this order​

1. Confirm it is what you think​

Run Test Node on the failing detection and read the real confidence. This distinguishes the two cases that feel identical:

  • Confidence far below the threshold → the template no longer matches. Re-capture.
  • Confidence fine, but the click misses → the template matches, the position moved. Check the search region.

Guessing at thresholds without this number is how a working template gets thrown away.

2. Re-capture the template tightly​

Crop just the distinctive part — the glyph, the icon, the label. Background that surrounds it is exactly what changes when the app is restyled, and every pixel of it you include is a pixel that can break you.

TEMPLATE_QUALITY_TOO_LOW means the crop is too small or too plain to match reliably; go slightly bigger, or pick a more distinctive element.

3. Re-check the search region, not just the template​

If the element moved, the template is fine and the search region is wrong. Draw regions on Live Preview rather than typing coordinates — and note that a region also makes matching faster, so this is the cheapest run-time speed-up available.

4. Only then touch the threshold​

Lowering a threshold to make a bad template pass buys a false positive later. Fix the template first; adjust the threshold when Test Node shows a genuinely good match sitting just under the line.

Resolution changes are a separate failure​

If you changed the emulator's resolution or zoom, templates break wholesale — matching is pixel-based. Two options:

  • Re-capture at the new resolution. Reliable, and what most people should do.
  • Enable multi-scale detection in the flow's feature gates. Costs more per match; useful when one flow must serve differently-sized instances.

Also check the queue: an entry stores the flow/instance resolution pair you confirmed. Change either side and your old "Add anyway" no longer applies — that is QUEUE_RESOLUTION_MISMATCH, and it is the system refusing to assume you meant it.

Keeping every instance on the documented 1280×720 · DPI 240 · 30 FPS baseline is what makes one template set work everywhere. See Configure MuMu.

Build so the next change costs less​

These are design choices that turn a breakage into a shrug.

Put every reusable rectangle in the Region Library. Then a moved element is one edit, not five. Five nodes that each redrew the same rectangle are five separate repairs.

Never trust a screen you did not verify. A flow that acts and moves on cannot tell you the app changed; it just does the wrong thing quietly. Act → settle → verify. See Lesson 02.

Own a "get to a known screen" routine. A Sub Flow that presses BACK a couple of times and confirms it is home makes every other flow safe to start. When the app changes you fix that one routine instead of every flow's opening moves — see Lesson 09.

Prefer detection over memorised position. Point a Click's coordinate source at a Find Image node instead of a fixed region. Then the tap follows the element when it moves, and you change nothing.

Give every failure route somewhere honest to go. "The button is not there" should reach a Stop with a message, not fall off the end of the graph.

After an app update: a 10-minute pass​

  1. Run each flow once with Test Flow on a real instance.
  2. For each failure, run Test Node on the reported node and read the confidence.
  3. Re-capture the templates that dropped; leave the ones that did not.
  4. Re-check search regions for anything that moved.
  5. Re-run. Fix the "get to a known screen" routine first if the opening moves broke — everything downstream depends on it.
  6. Only then look at thresholds.

What matters is whether the pixels changed, not what the app's version number says.