XML: Structural error in xml file, 1 open items. -- JSON: The returned JSON does not match the expected structure (Invalid JSON.). (code: PE231)

I am getting a strange error and not sure where to start, here is the python code, it is based on the sample script provided which runs fine and imports into PRTG. But I get the above error when I run my script. If I run from the command line I get this error:

Traceback (most recent call last):
  File "c:\Program Files (x86)\PRTG Network Monitor\Custom Sensors\python\prtg_twilio.py", line 9, in <module>
    from prtg.sensor.result import CustomSensorResult
  File "c:\Program Files (x86)\PRTG Network Monitor\Python34\lib\site-packages\prtg\sensor\result.py", line 52
    _channels: list = []


Script:

# -*- coding: utf-8 -*-

import json
import sys
from twilio.rest import Client
import datetime
from datetime import timedelta

from prtg.sensor.result import CustomSensorResult
from prtg.sensor.units import ValueUnit

if __name__ == "__main__":

    account_sid = 'xxx'
    auth_token = 'xxx'
    client = Client(account_sid, auth_token)

    calls = client.calls.list(limit=200)
    total_calls = []
    no_answer = []

    for record in calls:
        d1 = datetime.datetime(datetime.date.today().year, datetime.date.today().month, datetime.date.today().day, 0, 0,
                               0, 0, datetime.timezone(-timedelta(hours=7)))
        if record.date_created > d1 and record.direction == 'outbound-api':
            total_calls.append(record)
            print(record.date_created)
            if record.status == "no-answer":
                no_answer.append(record)

    try:
        data = json.loads(sys.argv[1])

        csr = CustomSensorResult(text="This sensor runs on %s" % data["host"])

        csr.add_primary_channel(name="No Answer Calls Today",
                                value=int(len(no_answer)),
                                unit=ValueUnit.COUNT,
                                is_float=False,
                                is_limit_mode=True,
                                limit_min_error=0,
                                limit_max_error=5,
                                limit_error_msg="No Answer calls too high")

        csr.add_channel(name="Total Calls Today",
                        value=int(len(total_calls)),
                        unit=ValueUnit.COUNT)

        print(csr.json_result)
    except Exception as e:
        csr = CustomSensorResult(text="Python Script execution error")
        csr.error = "Python Script execution error: %s" % str(e)
        print(csr.json_result)

Article Comments

Hi there,

It seems like that your script has an issue with the module "CustomSensorResult". Please ensure that this module is available on the corresponding Probe.
In addition, you can enable the option " Write Results To Disk" in your Sensor settings. This will create additional log files in the "Logs > Sensors" folder in the PRTG Data Directory of the corresponding Probe (https://helpdesk.paessler.com/en/support/solutions/articles/7600006351163).


Jul, 2020 - Permalink

The library is available we only have the one probe and server on all on the same box. It seemed to find the file but not like the code. Which is odd. The example script runs fine. I will post a saved output shortly


Jul, 2020 - Permalink

Hi there,

Please note that this script will executed in 32bit mode. This is due to the Probe service process which is also a 32bit process. We made the experience that some modules are not available in 32bit.


Jul, 2020 - Permalink

Here is the log, but it looks like it is running python from the /python dir, not the /python34 dir: 7/9/2020 5:11:03 PM Script File: prtg_twilio.py 7/9/2020 5:11:03 PM Script Parameters: {"tlsexplicit_pop3":"","fastcount":"0","esxpassword":"","hostv6":"","blockedsens":"","secretaccesskeyid":"","tlsexplicit_default":"","isexesensor":"0","windowsloginpassword":"","privatekey":"","secret":"","resttoken":"","reboot":"44020.7072764931","pop3password":"","pythonscript":"prtg_twilio.py","tlsexplicit_port":"","inerror":"1","smspassword":"","params":"","host":"www.twilio.com","mailpassword":"","tlsexplicit_ftp":"","timeout":"299","elevationpass":"","momopersistent":"","notonpod":"0","linuxloginpassword":"","smtppassword2":"","smtppassword":"","lastuptime":"0","writeresult":"1","reqmsginterval":"300","socksproxypass":"","prtg_linuxpassword":"","tlsexplicit_imap":"","sshelevationpass":"","uptimecount":"0","canlinux":"0","proxypassword":"","proxypass":"","mutexname":"","snmpencpass":"","authuserpwd":"","usednstime":"0","restpassword":"","httppassword":"","addomainpass":"","authtoken":"","oauthrefreshtoken":"","evapassword":"","oauthtoken":"","snmpcommv1":"","sensorid":"7374","ipversion":"0","accesskey":"","xmlhttppassword":"","tlsexplicit_smtp":"","password":"","snmpcommv2":"","snmpauthpass":"","awssk":"","dbpassword":"","pingdompassword":"","prtg_windowspassword":"","usewindowsauthentication":"0","simulate":"0","lastmsg":"#Y2 Structural error in xml file, 3 open items. @#O231[Invalid JSON.]","imappassword":"","ftppassword":""} 7/9/2020 5:11:03 PM Script Parameters (escaped): {\"tlsexplicit_pop3\":\"\",\"fastcount\":\"0\",\"esxpassword\":\"\",\"hostv6\":\"\",\"blockedsens\":\"\",\"secretaccesskeyid\":\"\",\"tlsexplicit_default\":\"\",\"isexesensor\":\"0\",\"windowsloginpassword\":\"\",\"privatekey\":\"\",\"secret\":\"\",\"resttoken\":\"\",\"reboot\":\"44020.7072764931\",\"pop3password\":\"\",\"pythonscript\":\"prtg_twilio.py\",\"tlsexplicit_port\":\"\",\"inerror\":\"1\",\"smspassword\":\"\",\"params\":\"\",\"host\":\"www.twilio.com\",\"mailpassword\":\"\",\"tlsexplicit_ftp\":\"\",\"timeout\":\"299\",\"elevationpass\":\"\",\"momopersistent\":\"\",\"notonpod\":\"0\",\"linuxloginpassword\":\"\",\"smtppassword2\":\"\",\"smtppassword\":\"\",\"lastuptime\":\"0\",\"writeresult\":\"1\",\"reqmsginterval\":\"300\",\"socksproxypass\":\"\",\"prtg_linuxpassword\":\"\",\"tlsexplicit_imap\":\"\",\"sshelevationpass\":\"\",\"uptimecount\":\"0\",\"canlinux\":\"0\",\"proxypassword\":\"\",\"proxypass\":\"\",\"mutexname\":\"\",\"snmpencpass\":\"\",\"authuserpwd\":\"\",\"usednstime\":\"0\",\"restpassword\":\"\",\"httppassword\":\"\",\"addomainpass\":\"\",\"authtoken\":\"\",\"oauthrefreshtoken\":\"\",\"evapassword\":\"\",\"oauthtoken\":\"\",\"snmpcommv1\":\"\",\"sensorid\":\"7374\",\"ipversion\":\"0\",\"accesskey\":\"\",\"xmlhttppassword\":\"\",\"tlsexplicit_smtp\":\"\",\"password\":\"\",\"snmpcommv2\":\"\",\"snmpauthpass\":\"\",\"awssk\":\"\",\"dbpassword\":\"\",\"pingdompassword\":\"\",\"prtg_windowspassword\":\"\",\"usewindowsauthentication\":\"0\",\"simulate\":\"0\",\"lastmsg\":\"#Y2 Structural error in xml file, 3 open items. @#O231[Invalid JSON.]\",\"imappassword\":\"\",\"ftppassword\":\"\"} 7/9/2020 5:11:03 PM Script Path: C:\Program Files (x86)\PRTG Network Monitor\custom sensors\python\prtg_twilio.py 7/9/2020 5:11:03 PM Command Line: "C:\Program Files (x86)\PRTG Network Monitor\python\python.exe" -E "C:\Program Files (x86)\PRTG Network Monitor\custom sensors\python\prtg_twilio.py" "{\"tlsexplicit_pop3\":\"\",\"fastcount\":\"0\",\"esxpassword\":\"\",\"hostv6\":\"\",\"blockedsens\":\"\",\"secretaccesskeyid\":\"\",\"tlsexplicit_default\":\"\",\"isexesensor\":\"0\",\"windowsloginpassword\":\"\",\"privatekey\":\"\",\"secret\":\"\",\"resttoken\":\"\",\"reboot\":\"44020.7072764931\",\"pop3password\":\"\",\"pythonscript\":\"prtg_twilio.py\",\"tlsexplicit_port\":\"\",\"inerror\":\"1\",\"smspassword\":\"\",\"params\":\"\",\"host\":\"www.twilio.com\",\"mailpassword\":\"\",\"tlsexplicit_ftp\":\"\",\"timeout\":\"299\",\"elevationpass\":\"\",\"momopersistent\":\"\",\"notonpod\":\"0\",\"linuxloginpassword\":\"\",\"smtppassword2\":\"\",\"smtppassword\":\"\",\"lastuptime\":\"0\",\"writeresult\":\"1\",\"reqmsginterval\":\"300\",\"socksproxypass\":\"\",\"prtg_linuxpassword\":\"\",\"tlsexplicit_imap\":\"\",\"sshelevationpass\":\"\",\"uptimecount\":\"0\",\"canlinux\":\"0\",\"proxypassword\":\"\",\"proxypass\":\"\",\"mutexname\":\"\",\"snmpencpass\":\"\",\"authuserpwd\":\"\",\"usednstime\":\"0\",\"restpassword\":\"\",\"httppassword\":\"\",\"addomainpass\":\"\",\"authtoken\":\"\",\"oauthrefreshtoken\":\"\",\"evapassword\":\"\",\"oauthtoken\":\"\",\"snmpcommv1\":\"\",\"sensorid\":\"7374\",\"ipversion\":\"0\",\"accesskey\":\"\",\"xmlhttppassword\":\"\",\"tlsexplicit_smtp\":\"\",\"password\":\"\",\"snmpcommv2\":\"\",\"snmpauthpass\":\"\",\"awssk\":\"\",\"dbpassword\":\"\",\"pingdompassword\":\"\",\"prtg_windowspassword\":\"\",\"usewindowsauthentication\":\"0\",\"simulate\":\"0\",\"lastmsg\":\"#Y2 Structural error in xml file, 3 open items. @#O231[Invalid JSON.]\",\"imappassword\":\"\",\"ftppassword\":\"\"}" 7/9/2020 5:11:03 PM Script Output (OEMCP Encoding): Traceback (most recent call last):[CR][LF] File "C:\Program Files (x86)\PRTG Network Monitor\custom sensors\python\prtg_twilio.py", line 5, in <module>[CR][LF] from twilio.rest import Client[CR][LF] File "C:\Program Files (x86)\PRTG Network Monitor\python\lib\twilio\rest\__init__.py", line 18, in <module>[CR][LF] from twilio.http.http_client import TwilioHttpClient[CR][LF] File "C:\Program Files (x86)\PRTG Network Monitor\python\lib\twilio\http\http_client.py", line 1, in <module>[CR][LF] from requests import Request, Session, hooks[CR][LF]ModuleNotFoundError: No module named 'requests'[CR][LF] 7/9/2020 5:11:03 PM Exit Code: 1 7/9/2020 5:11:03 PM RawStream Size: 565 7/9/2020 5:11:03 PM OutputStream Size: 565 7/9/2020 5:11:03 PM Script Output (UTF8 Encoding): Traceback (most recent call last):[CR][LF] File "C:\Program Files (x86)\PRTG Network Monitor\custom sensors\python\prtg_twilio.py", line 5, in <module>[CR][LF] from twilio.rest import Client[CR][LF] File "C:\Program Files (x86)\PRTG Network Monitor\python\lib\twilio\rest\__init__.py", line 18, in <module>[CR][LF] from twilio.http.http_client import TwilioHttpClient[CR][LF] File "C:\Program Files (x86)\PRTG Network Monitor\python\lib\twilio\http\http_client.py", line 1, in <module>[CR][LF] from requests import Request, Session, hooks[CR][LF]ModuleNotFoundError: No module named 'requests'[CR][LF]


Jul, 2020 - Permalink

Hi there,

It's still a module which causes the issue:

ModuleNotFoundError: No module named 'requests'

Unfortunately, we don't assist by coding custom scripts. For more information, why we don't offer any customization help for PRTG, please follow this link.


Jul, 2020 - Permalink

But that same module works fine if outside prtg. The only thing I did different was import the prtg library and format the return by replacing some values in the CSR field.


Jul, 2020 - Permalink

To add modules to the PRTG integrated Python Interpreter they have to reside in the interpreter's site-packages directory. The same can be found at <PRTG Install Directory>\Python34\Lib\site-packages.
After adding the package there, you should be able to import the same in a script run by PRTG.


Jul, 2020 - Permalink

Hello there.

As of PRTG version 22.1.74 it is possible to install/add modules to PRTG's python. If my script needs the requests module for example, I can do so by heading to C:\Program Files (x86)\PRTG Network Monitor\python\ (PRTG's install path) and issuing:

python.exe -m pip install requests

Afterwards requests can be used in scripts. This has to be done on a per-probe basis.

Best regards,
Luciano Lingnau, Paessler Team


Feb, 2022 - Permalink

Hi Luciano,

We're running 22.2.77.2204 x64 here and I'm still getting ModuleNotFoundError: No module named 'requests' in my logs, even after following your install step.

(BTW the script runs fine when executed directly in a PowerShell window).

I've performed the ./python.exe -m pip install requests step from both a user and admin-level P$ window:

PS C:\Program Files (x86)\PRTG Network Monitor\python> ./python.exe -m pip install requests
Requirement already satisfied: requests in c:\users\myusername\appdata\roaming\python\python39\site-packages (2.28.1)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\myusername\appdata\roaming\python\python39\site-packages (from requests) (2022.6.15)
Requirement already satisfied: charset-normalizer<3,>=2 in c:\users\myusername\appdata\roaming\python\python39\site-packages (from requests) (2.1.1)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\users\myusername\appdata\roaming\python\python39\site-packages (from requests) (1.26.11)
Requirement already satisfied: idna<4,>=2.5 in c:\users\myusername\appdata\roaming\python\python39\site-packages (from requests) (3.3)

[notice] A new release of pip available: 22.2 -> 22.2.2
[notice] To update, run: python.exe -m pip install --upgrade pip
PS C:\Program Files (x86)\PRTG Network Monitor\python> ./python.exe -m pip3 install requests
C:\Program Files (x86)\PRTG Network Monitor\python\python.exe: No module named pip3
PS C:\Program Files (x86)\PRTG Network Monitor\python>


You wrote "This has to be done on a per-probe basis", but installing requests in this manner appears to have a global effect. How do I make 'requests' available to this sensor?


- Greig.


Aug, 2022 - Permalink

Aha!

For the benefit of anyone who finds this thread in future, the solution is that you need to be logged into Windows as THE Administrator account to install the required modules. Being logged in as AN administrator is insufficient.

Oh, and you also need to be on a release later than 22.2.77.2204:

PRTG Version 22.2.77.2204 broke compatibility with pip.exe install (to install Python packages). With this PRTG release we have removed support for this command to prevent unexpected errors. As of now the only supported way of installing a Python package to use Python Script Advanced sensor is python.exe -m pip install. Please also see our Knowledge Base article for more information about Python installation: https://helpdesk.paessler.com/en/support/solutions/articles/90686.


- Greig.


Sep, 2022 - Permalink