What do I do if I encounter blocking or captchas?

1

While trying to setup a cloud monitor, I encountered an error.

There was no challenge in the setup, but on actual run it gave me a “SELECTION_EMPTY” error and upon checking the error details, I see a snapshot of a bot detection page as shown below:

image
image1846×390 30.4 KB

Please suggest a solution

2

Hi,

This is a fairly common challenge and I have 2 possible recommendations:

  1. Try and see if a local monitor helps address the challenge(Local Monitor vs. Cloud Monitor | Distill). This should usually work well, if the frequency of checks is not too high.
  2. For monitors in the cloud, try using higher grade proxies: Using proxy servers for checks | Distill

Feel free to reach out, if you still continue to face any challenges.

1 Like
3

Thank you, for your detailed response.

4

What do I do I encounter a Captcha or No. to be put in from the getgo (see two screenshots attached)?

Bulgaria
Bulgaria1471×742 179 KB

https://legalacts.justice.bg/

resp. this site

India
India1963×967 131 KB

5

Hi @alextaxanalysts,

Unfortunately, at this point there is no solution to automate pages which require you to input captchas in the page.

Regards,
Surya

6

Hello,

i get an error message in my watchlist, because the website i want to follow requires a CAPTCHA part of the times it refreshes. What can i do about this?

7

Hello @thomas97

Firstly, welcome to Distill’s forums

Can you provide additional details:

  1. Webpage you are trying to monitor
  2. Local vs. Cloud monitoring
  3. Frequency of checks

The general guidelines to handle the scenario have been listed above in this thread.

Cheers,
Surya

8

Hello! New member here and just purchased the $15/month membership. I’m having an issue while trying to track the out of stock text for a website. On my own PC it works just fine until maybe a 4 or 5 hours have passed and then it requires a puzzle piece verification, or a image select verification to make sure you’re human. At this point it fails. This was why i purchased membership, since using proxy’s its much less likely to happen. I set it up on the proxy to do the same just like my PC, and its still happening, but of course more randomly since its different IP’s (proxy). I feel like this is happening far too often though, is there no way to fix this? Image below shows how often its happening. Also here is the code it runs:

{
  "selections": [
    {
      "frames": [
        {
          "index": 0,
          "excludes": [],
          "includes": [
            {
              "expr": "(//main[@id='main']/div//div//div//h3)[1]",
              "type": "xpath",
              "fields": [
                {
                  "name": "text",
                  "type": "builtin"
                }
              ]
            }
          ]
        }
      ],
      "dynamic": true,
      "delay": 2
    }
  ],
  "regexp": null,
  "ignoreEmptyText": true,
  "includeStyle": false,
  "viewport": {},
  "dataAttr": "text"
}

selectempty

9

Hello @vyserage - welcome to the community forum. So far it looks like 30% success rate using cloud. Are you using any premium proxy or the default shared pool? If you can share more details about the website being monitored, I can check it out and share recommendations about the proxy to be used. Thanks!

10

Hi! Im using the resedential US proxys, and here is a pic of what im trying to monitor, marked in orange-the number amount so in this case ( 1 - 4 of 4 ):

edit: Based in prior replies in this thread, it would seem there is no way to solve the human verify issue. The reason mine get so many errors is because the page its loading is searching for the 1 -4 of 4 text i told it to but it loaded a verify human page instead. I might as well just use the free membership and use my own local monitor and just have to set the checks to super low, unfortunately :frowning:
That or deal with manually verifying the human checks every so often on my local PC. How unfortunate :frowning:

11

Is there a way to integrate this into Distill?

12

Did anyone see my post?

13

Hi @alextaxanalysts

At this point, we do not have a direct integration with these tools & its not on our immediate roadmap.
Almost all real-life scenarios are addressed either via premium proxies or by using a local monitor.
Can you share the specific scenarios where you see value for the integration?

Thanks,
Surya

14

Well, please see this captcha in the following website

https://juportal.be/zoekmachine/zoekformulier

How can a local monitor overcome this issue?

15

I haven’t heard anything. Any response?

16

When there is no way to work around a captcha, it is not possible to monitor that page.

17

but in these instances there would be value for the tool described above. The matter of the conversation was: is there a way to work around captchas? I provided one. Answer was: Premium proxies (what’s that anyways) and a local monitor help. - Ok. I provided a link and asked whether a local monitor can help. An answer is still open whether a local monitor can help with the site provided below

https://juportal.be/zoekmachine/zoekformulier

18

When there is no way to work around a captcha even when using a local monitor, it is not possible to monitor that page.