How to prevent distill from being discovered and blocked as an auto refresher by the hosting website?

Help and Support
1

Hi everyone,

I’m using an app that needs to refresh a web page every few seconds. However, I’m worried that the hosting website will block my app as an auto refresher. I’m looking for any advice on how to prevent this from happening.

I’ve tried using a random interval between refreshes.

Is this method correct?

Any help would be greatly appreciated.

Thanks

2

Hello @norine.dobiesz - when monitoring websites, it is easy for any website to log every visit and see access patterns. The best way is to use Distill responsibly and be within websites’ access limits. Cheers!

3

I think I’m running into the same issue. I used to be able to access the webpage I’m monitoring but now it won’t load. How do you know what a websites specific “access limits” are?

4

@espanaorbust Good question—this can be tricky since most websites don’t explicitly publish their exact “access limits.”

In general:

  • Websites detect patterns like very frequent requests, repetitive intervals, or high volume from a single IP

  • If your checks are too frequent, they may rate-limit or block access

Using random intervals can sometimes help, but it’s not a guaranteed solution, as sites use more advanced detection methods.

A safer approach is to:

  • Keep a reasonable check interval (avoid very aggressive refresh rates)

  • Monitor only the specific section you need instead of the full page

  • Use Cloud monitors if local IP blocking is an issue

If you can share:

  • The website URL you are monitoring

  • The specific section you want to track

we can take a closer look and guide you on the best way to monitor it (or let you know if there are limitations).