Monitoring websites with dynamic filtering

1

Hello,

I would like to seek assistance regarding website monitoring capabilities. Specifically, I am trying to understand if it is possible to monitor websites with interactive filtering features:

  1. Can we monitor a website that dynamically filters content through selection options?
  2. Is monitoring possible for a website that implements pull-down menu navigation?
2

monitoring pages that require requires user action can be tricky to configure. distill offers two features that makes it easy to configure the steps to automate the interactions.

  1. visual recorder: it is an interactive remote browser that lets a user interact with any page. it automatically captures the actions that can be saved and replayed later. this recording is called a macro.
  2. the macro is then used with a monitor. the monitor uses the macro to replay the recorded actions before checking the pages content for changes.

check out the following article is a quick start guide on creating and using macros:

if you have any questions or need help with a specific use case, feel free to share them here. cheers!

1 Like
3

Hi @ajitk

Thank you for introducing the macro functionality.
I have two questions:

  1. Is it possible to bypass the robot verification panel selection shown in the image below?

    image
    image175Ă—216 58 KB

  2. For websites like the one shown in the image, we need to select dates from a calendar. Since the dates shift daily and pull-down menu selections for month/year need to be changed, I wasn’t able to replicate this using macros.

    image
    image425Ă—527 47.3 KB

4

  1. the captcha can sometimes be bypassed by using premium proxies. but it is not guaranteed to work in all cases. using local monitors are the ultimate solutions in those cases. did you encounter this while recording a macro?

  2. this is an interesting use case. do you always need to select the next date? does the page indicate “the current date” in any way?

1 Like
5

@ajitk
Thank you.

  1. I would like to know the reason why you recommend a local monitor. Does using a local monitor prevent robot verification from appearing? Also, I am currently using a local monitor. When recording a macro, reCAPTCHA appears.

    image
    image2334Ă—1062 217 KB

  2. This needs to change the date daily, and when the month changes, the dropdown needs to be changed, so I couldn’t reproduce it with a macro.

6
  1. in most cases, captchas are shown less frequently on a local device because of ip address, cookies etc. is solving the captcha mandatory in this page’s case? websites can choose to make this a mandatory or an optional step.
  2. it will be possible if the calendar displays the current date in some way. is there any visual cue?

note that the visual recorder uses a remote browser for recording the steps even when using the browser extension or the desktop app.

7

HI @ajitk ,

  1. Solving CAPTCHA is mandatory. I am testing with the Distill app installed on a local device, but CAPTCHA still appears. If passing the CAPTCHA leaves information in the local browser’s cookies, would the following approach be effective?
  1. Start creating a macro.
  2. Open the relevant website within the macro operation screen and manually pass the CAPTCHA.
  3. Delete all recorded steps related to the manual CAPTCHA operation.
  4. Perform the necessary operations for monitoring while CAPTCHA is not displayed, and record the macro steps.

Regarding point 2, I will confirm.
By the way, where is the remote browser located?

8

this won’t work if the captcha is displayed every time the page is accessed.

US is the default location that can be customized by changing the proxy server.

9

@ajitk
Thank you for the response.

  1. I understand that bypassing the CAPTCHA is nearly impossible under the current circumstances.
  2. I also understand the explanation regarding the location settings.