Monitoring a page for price changes: massive diffs happen often

Hi !

I’m new to Distill and try it in order to monitor a page for price changes: Star Wars: Assaut sur l'Empire - Figurines sur un Plateau - Jeux de figurines - Boutique Philibert (yes, I like boardgames and Star Wars :slight_smile: )

I’ve setup a simple monitor for that whole page, and tried it both directly from Firefox or from your cloud. It works relatively well, reporting small changes, except once in a while (at least once a day, maybe more) when it reports a full removal and replacement of all elements, though visually nothing changed, and reverses it on the next check an hour later:

How can I fix this ?

Hope someone can help,
Paul-Henri

@phlampe what does the page look like when the full removal is captured? is it because of a partially loaded page being captured or an entirely different kind of change, say an error page being rendered? using the “explore diff”'s split view can provide a better picture here.

the most common solution in such scenarios is to monitor a more specific part of the page instead of monitoring the full page. this is recommended in most cases unless monitoring the full page is a must.

another slightly advanced solution is to validate the content before it is monitored. the validation can raise an error based on certain conditions (like page doesn’t have expected content). in case of error, that particular check fails and is logged as an error and the content is not captured.

let me what you think or have any questions. cheers!

Hello Ajit !

Thanks a lot for answering my question.

Here’s an example of what happened during one day:

. 10:32 AM: all is good, the page looks perfectly normal. It seems a good reference

(next screenshots in next posts, I’m limited to 1 per post)

And at the same, when I checked the page in another tab, nothing seemed that different from the initial version of 10:32.

I hope this helps.
I’ll get another screenshot of the side by side diff screen next time it happens.
Paul-Henri

. 01:18 PM massive changes: 3020 additions, 2170 removals. Page looks all green

. 02:19 PM it reverts to the previous state: 2170 additions, 3020 removals. Page looks all red.

Cool, it just happened again. I got more screenshots.

Screenshot of side by side diff, top:

Screenshot of side by side diff, middle, where it switches between additions and removals:

Screenshot of side by side diff, almost at the bottom of the page:

in these all red content, 2170 additions were detected. they might become visible after scrolling down.

can you view diff in text mode to quickly find what the additions were in this case?

here are two recommendations to try:

  1. when monitoring the full page, try to disable javascript so that dynamic rendering doesn’t happen.

  2. or, monitor a more specific part of the page so that only change within the specified selection is monitored.

OK, I’ll try that and keep you posted.

Thanks a lot !
Paul-Henri

1 Like