Distill web monitor not pulling from webpage

hello. i recently set up a distill web monitor for the following website (https://news.treeofalpha.com/) for updates to certain elements on the page; however, the web monitor appears to not be capturing the periodic updates now. can you help me figure out why?

Hello @joe14, what does the log of checks look like?

The log of checks only says last checked on January 20, which is an error. I have been using this monitor for a few weeks now.

I keep getting error notifications or when i make ignore empty text “false,” i do not receive an error message, otherwise it does not pick up updates in the specific selected boxes i chose from the webpage selector area and i get an error message. what do you think the problem is?

The check log shows the error and its details. Can you please share a screenshot of the following to show what is happening:

  1. Check log. If there any error, then the error details will be good.
  2. Change history if there is any

ok, please see below. for whatever reason distill is not picking up the text contained in the element i selected in the webpage selector. the change log isnt even recording the activity for whatever reason. please let me know if you need anything else!

the change logs are not logging any activity, but are just reading error. there is no change history. see below:

The one glaring issue I see is that ignoreEmptyText has probably been set to false in the config. Did you do this to help solve a SELECTION_EMPTY error?

That error indicates that the selectors for that page does not match content in the page. The solution in most cases is to update or recreate a selector that matches elements in the page.

What are you trying to monitor in that page? I can help with the selectors.

the issue is not an error tho, its that distill isnt pulling the updated info for whatever reason.

im trying to monitor the updated headlines in the latest news page. i attached a screenshot below. the headlines are constantly updated but for whatever reason distill doesnt pull the headline. if you look at the webpage you ll see what i mean.


how can we fix this?

@joe14, I tried monitoring the headlines in the page using the following config:

{
  "selections": [
    {
      "frames": [
        {
          "index": 0,
          "excludes": [],
          "includes": [
            {
              "type": "xpath",
              "expr": "//h2",
              "fields": [
                {
                  "name": "text",
                  "type": "builtin"
                }
              ]
            }
          ]
        }
      ],
      "dynamic": true,
      "delay": 2
    }
  ],
  "ignoreEmptyText": true,
  "includeStyle": false,
  "dataAttr": "text"
}

Note that it is monitoring all h2 elements in the page. Here is what the change history looks like:

One thing I noticed is that sometimes that page fails to load in a tab that is not active (a background tab). If the page doesn’t load properly, then the selectors will not find the content to be monitored.

As a quick test, can you try to run this monitor in the cloud and see if works for you?

A note regarding Watchlist and the check log:

Distill can check monitors for changes using different devices. The visibility of the log of check depends on the Watchlist being used.

A webapp’s Watchlist looks like this:

An extension’s watchlist looks like this:

Notice the differences between the two (names and URLs).

An extension’s Watchlist’s URL will start with extension, chrome-extension or moz-extension depending on the browser used.

One can access the extension’s Watchlist by clicking “Distill’s icon in the browser toolbar > Go to Watchlist”:

The screenshot in #7 most likely belongs to the webpp (because of missing logs from the extension). Try using the extension’s Watchlist instead. It should show the correct check log for a local monitor with more details.

1 Like

thank so much ajitk. you are right. its the way the website loads in the background that makes the selectors not able to pick up the changes. when i switch it to cloud - distill servers, the problem generally goes away.

i unfortunately want updates less than 2 minutes apart (as it looks like thats the quickest interval distill offers to check webpages via cloud servers). is there any program that would allow me to use cloud distill servers on say 1 minutes intervals? thanks!

@joe14 Distill’s upcoming desktop app can help in cases where the extension is not able to load a page in a background tab. It is currently in development and preview build is available to paid customers. Will you like to give it a try? :slight_smile:

great. and yes, i very much would! how can i become part of the program?

1 Like

Cool. Just sent a DM with links to download the desktop app for Windows and MacOS. Let me know how it works.

1 Like

thanks for sending. i downloaded the app. what setting would you recommend for me to be able to constantly monitor the page for intervals less than 5 minutes apart?

You can use a local device (anyone that is not cloud) to monitor the page at a lower interval.

BTW, did the desktop app work well for monitoring the headlines in the news feed?

hello! the desktop app worked well for the past few days but now is having some hiccups. I keep getting a TIMEOUT error in the check log so the monitor is not running like it was before. Before this error though, it had been working really well. Any tips on how I could fix this one?

By the way, this app is great. You have a committed user in me now (once this issue is worked out ofc)!

1 Like

Thanks for the feedback and glad to have you as a committed user! :smiley:

Couple of questions regarding the timeout errors:

  1. Did all checks error out with timeout error or did some succeed?
  2. Does restarting the app affect the behavior in any way?

after restarting the app several times, its back to working really well. this really is great. when will the alpha launch happen?!

It is currently undergoing private alpha testing and you are already a part of it. We will start public beta testing once we have a more stable version of the app ready.