The first few days after you start monitoring a site, everything looks fine. Then the first false alarm arrives: a clock. A minute later, a stock counter. Then a "related articles" block.
After three alarms you stop trusting the system. This post explains where those three alarms come from, and how to keep the monitor useful without silencing it.
Why dynamic content breaks visual comparison
Visual comparison finds every difference between two moments. Not text changes specifically — position changes, colour shifts, everything.
That is correct behaviour. It is only a problem if what you are comparing is a page that was never supposed to be static.
A page with a clock changes every second. The system reports it as a change, because technically it is one.
So the problem is not that the system is misbehaving. The problem is that you chose a page that was never static to compare against itself.
The usual dynamic regions
Time and dates. Clocks, countdowns, "posted on" timestamps. The most common, and the first one to fix.
Counters. View counts, comment counts, stock levels.
Personalisation. "Welcome back, [name]", or content that varies by location. These are hard to spot, because you are not logged in and you see the same page every time.
Ads and third-party embeds. Rotating banners, "random" content widgets, social embeds.
Latest posts / related content. These change every time you publish anything.
Anything session-dependent. Cookie banners, subscription state, the logged-in menu.
The fix: masking
Masking means "do not include this region in the comparison". You mark the rectangle around the changing element and tell the system not to look there.
Three approaches in practice:
1. Fixed-region masking. Mark the bounding box and leave it out. Simple, obvious, and sufficient in most cases.
The catch: if the element moves over time, the mask drifts. Add a new element and you have to update the mask.
2. Variable-text masking. Mask a word or pattern anywhere on the page — for example, any date-like string.
The catch: a bad pattern hides a real change along with the noise.
3. Element-identity masking. When the element has an identifier, track it and mask by identity rather than by position. Mostly possible with third-party widgets.
The catch: the most setup of the three.
Choosing between them
| Situation | Approach |
|---|---|
| A single clock | Fixed region |
| A few counters | Fixed region + variable text |
| Third-party widget | Element identity, if possible |
| Personalised content | By identifier |
If one page needs more than about five masks, that is a signal: the wrong page was chosen. Monitoring a modern product listing with a mask on every dynamic element is worse than monitoring a static page, and tells you less.
This is a common mistake. "I will monitor this page and mask the dynamic bits" means monitoring the most dynamic page on the site; the system produces noise and gets ignored.
The risk of masking
Being honest here: masking also prevents you from seeing changes you did not want to hide.
A page has both a clock and a price. You mask the clock; the mask drifts and covers the price; now you miss price changes.
That is a real risk, and there are two ways to manage it:
- Review masks whenever the page's content changes. When the content is updated, the mask map is updated with it.
- Run an unmasked scan periodically. Every so often, remove the masks and compare everything. See what is there, then put the masks back.
The second is easy to skip, and it is exactly where a real break stays silent.
Waiting for stability
Masking does not catch the moment the page settles. Scan timing matters too.
Do not compare before the page has finished loading. An image captured mid-load differs from the same page a second later, with nothing actually changed.
Wait long enough, but not forever. Too short captures unloaded images. Too long slows every scan and loads the site unnecessarily.
Do not compare the same clock time twice. Two screenshots an hour apart have twelve hours between them. A banner that appeared overnight produces a false alarm against a noon capture.
Why this happens everywhere
Because modern websites are not static. Ten years ago a page's appearance was mostly fixed; today every page is fed by a data source.
That does not make visual testing invalid. It means the decision to monitor a page is editorial, not technical. "Is this page worth monitoring?" comes before "is this change important?" — and the answer is often no.
The practical rule: there must be something on the page whose change matters to you. A page with nothing worth watching generates masks rather than information.
This is the same rule as which pages to watch in client monitoring, applied to the testing side. For the mechanics of the comparison itself, see what visual regression testing is.