sticky hbmonitor3 available


Steve KC1AWV
 

I think I've finally gotten hbmonitor ported to Python 3. All are welcome to test it out, and use this message thread to question or comment. You can find the github repo here: https://github.com/kc1awv/hbmonitor3

A couple things to note:
  • Module is renamed to monitor.py
  • Log file is kept in the ./log directory - this was for ease of testing, you can put the log anywhere you want. Just change the line in the config file

Right now, "it works for me, here." Hopefully I can get some feedback from others to see if it works there, too.

I also tried to keep the style and formatting as close to the original HBmonitor as much as I could. Some changes made by me were for my own ease of readability.

Steve KC1AWV


Steve KC1AWV
 

Quick-and-dirty setup:

  • Make sure to have python3 and python3-pip available, and hblink3 running
  • git clone https://github.com/kc1awv/hbmonitor3.git
  • cd hbmonitor3
  • pip3 install -r requirements.txt
  • cp config_SAMPLE.py config.py
  • Edit config.py with your favorite editor to your liking
  • python3 monitor.py
  • Browse to http://[your-ip-address]:8080
Steve KC1AWV


EA5GVK Joaquin
 

Many thx. Perfect


El jue., 8 ago. 2019 21:41, Steve KC1AWV <smiller@...> escribió:
Quick-and-dirty setup:

  • Make sure to have python3 and python3-pip available, and hblink3 running
  • git clone https://github.com/kc1awv/hbmonitor3.git
  • cd hbmonitor3
  • pip3 install -r requirements.txt
  • cp config_SAMPLE.py config.py
  • Edit config.py with your favorite editor to your liking
  • python3 monitor.py
  • Browse to http://[your-ip-address]:8080
Steve KC1AWV


Eric-K6KWB
 
Edited

Steve, (KC1AWV)
 
How d? can you give some guidance which need to be change here below please? I made a domain name kf6itc-hblink.ddns.net (149.28.89.46 ip of hblink) but doesn't seems working. I downloaded this HBlink from Steve N4IRS wiki page, the HBlink3 appears to be working, just nice to have a dashboard to it : ) Thank you in advanced Steve.
 
best, Eric
 
 
REPORT_NAME     = 'system.domain.name'  
CONFIG_INC      = True                                      
BRIDGES_INC     = True                  
HBLINK_IP       = '127.0.0.1'          
HBLINK_PORT     = 4321                  
FREQUENCY       = 10                   
WEB_SERVER_PORT = 8080                  
CLIENT_TIMEOUT  = 300                  

# Files and stuff for loading alias files for mapping numbers to names
PATH            = './'                          
PEER_FILE       = 'peer_ids.json'                
SUBSCRIBER_FILE = 'subscriber_ids.json'          
TGID_FILE       = 'talkgroup_ids.json'           
LOCAL_SUB_FILE  = 'local_subscriber_ids.json'  
LOCAL_PEER_FILE = 'local_peer_ids.json'         
FILE_RELOAD     = 7                             
PEER_URL        = 'https://www.radioid.net/static/rptrs.json'
SUBSCRIBER_URL  = 'https://www.radioid.net/static/users.json'

# Settings for log files
LOG_PATH        = '/var/log/link/'             # MUST END IN '/'
LOG_NAME        = 'webtables.log'


Steve KC1AWV
 

As long as you have kept the defaults for the reporting stanza in hblink.cfg then the only change that you would need to make is on the first line, REPORT_NAME. That's the line that shows up underneath the title HBlink Monitoring Server on the web page, and I believe that most users just change that to their domain name.

If you run the program, then you should be getting log output on the console. Copy and paste that output and attach it as a file here so we can all look at it.

Steve KC1AWV


Eric-K6KWB
 

Right it does shows up, let me play some more. Will report back.

Thanks, Eric

INFO 2019-08-08 20:40:07,881 (GLOBAL) HBlink TCP reporting server configured
INFO 2019-08-08 20:40:07,883 (GLOBAL) HBlink 'HBlink.py' -- SYSTEM STARTING...
INFO 2019-08-08 20:40:07,884 (REPEATER-1) Sending login request to master 64.94.238.196:62031
INFO 2019-08-08 20:40:07,935 (REPEATER-1) Repeater Login ACK Received with 32bit ID: 1367091373
INFO 2019-08-08 20:40:07,988 (REPEATER-1) Repeater Authentication Accepted
INFO 2019-08-08 20:40:07,989 (REPEATER-1) Repeater Configuration Sent
INFO 2019-08-08 20:40:08,042 (REPEATER-1) Repeater Configuration Accepted
INFO 2019-08-08 20:40:08,043 (REPEATER-1) Connection to Master Completed
INFO 2019-08-08 20:40:13,631 (MASTER-1) Repeater Logging in with Radio ID: 310754, 47.157.82.87:53857
INFO 2019-08-08 20:40:13,632 (MASTER-1) Sent Challenge Response to 310754 for login: 1272138098
INFO 2019-08-08 20:40:13,641 (MASTER-1) Peer 310754 has completed the login exchange successfully
INFO 2019-08-08 20:40:13,654 (MASTER-1) Peer b'KF6ITC  ' (310754) has sent repeater configuration
 


Eric-K6KWB
 

Got this error down below? Please advise.

root@HBlink:/opt/HBmonitor# python web_tables.py
Traceback (most recent call last):
  File "web_tables.py", line 611, in <module>
    datefmt='%Y-%m-%d %H:%M:%S'
  File "/usr/lib/python2.7/logging/__init__.py", line 1554, in basicConfig
    hdlr = FileHandler(filename, mode)
  File "/usr/lib/python2.7/logging/__init__.py", line 920, in __init__
    StreamHandler.__init__(self, self._open())
  File "/usr/lib/python2.7/logging/__init__.py", line 950, in _open
    stream = open(self.baseFilename, self.mode)
IOError: [Errno 2] No such file or directory: '/var/log/link/webtables.log'  <<<<<
root@HBlink:/opt/HBmonitor#
 


Steve KC1AWV
 

It looks like you're actually using the original HBmonitor, not hbmonitor3.

Try running:

mkdir /var/log/link
touch /var/log/link/webtables.log

Steve KC1AWV


On Thu, Aug 8, 2019 at 5:05 PM Eric Dayrit <kf6itc@...> wrote:
Got this error down below? Please advise.

root@HBlink:/opt/HBmonitor# python web_tables.py
Traceback (most recent call last):
  File "web_tables.py", line 611, in <module>
    datefmt='%Y-%m-%d %H:%M:%S'
  File "/usr/lib/python2.7/logging/__init__.py", line 1554, in basicConfig
    hdlr = FileHandler(filename, mode)
  File "/usr/lib/python2.7/logging/__init__.py", line 920, in __init__
    StreamHandler.__init__(self, self._open())
  File "/usr/lib/python2.7/logging/__init__.py", line 950, in _open
    stream = open(self.baseFilename, self.mode)
IOError: [Errno 2] No such file or directory: '/var/log/link/webtables.log'  <<<<<
root@HBlink:/opt/HBmonitor#
 



--
Steve Miller
KC1AWV


Eric-K6KWB
 

yup this came from wiki page. Let me try that.


Eric-K6KWB
 

That fixed it, I'll finish up when I get home. Once again we appreciated what you guys do.
Thanks Steve, later.

best and 73,
Eric


On Thu, Aug 8, 2019 at 02:07 PM, Steve KC1AWV wrote:
touch /var/log/link/webtables.log
Copyright (c) 2016, 2017, 2018, 2019
        The Regents of the K0USY Group. All rights reserved.
 
2019-08-08 21:13:32,746 INFO ID ALIAS MAPPER: 'peer_ids.json' is current, not downloaded
2019-08-08 21:13:32,746 INFO ID ALIAS MAPPER: 'subscriber_ids.json' is current, not downloaded
2019-08-08 21:13:32,944 INFO ID ALIAS MAPPER: peer_ids dictionary is available
2019-08-08 21:13:35,569 INFO ID ALIAS MAPPER: subscriber_ids dictionary is available
2019-08-08 21:13:35,789 INFO CLIENT TIMEOUT: List does not exist, skipping. If this message persists, contact the            developer
2019-08-08 21:13:35,790 INFO reportClient object for connecting to HBlink.py created at: <__main__.reportClientFa           ctory instance at 0x7f56f7f468c0>
2019-08-08 21:13:35,791 INFO Initiating Connection to Server.
2019-08-08 21:13:35,801 INFO Connected.
2019-08-08 21:13:35,801 INFO Resetting reconnection delay

INFO 2019-08-08 21:13:35,799 (REPORT) HBlink reporting client connected: IPv4Address(type='TCP', host='127.0.0.1'                , port=41312)
INFO 2019-08-08 21:15:43,949 (REPORT) HBlink reporting client disconnected: IPv4Address(type='TCP', host='127.0.0                .1', port=41312)
 


laussana@...
 

Hi Steve
Besides the above I assume you need to install Jinja2 and autobahn as the other required components  are already there in HBlink3 ? Still learning how to put these together ..
Thank you
Jorge hk4cze


Steve KC1AWV
 

The requirements.txt is a copy of the original HBmonitor with one change to use dmr_utils3. Yes, there may be overlap with some of the modules if you install hbmonitor3 on the same machine as hblink3. But, the requirements still exist should someone want to have the monitor installed on a separate machine from hblink3.

Please remember, hbmonitor3 is a port and not a complete rewrite. A lot of the thinking of the original designer still exists.

Steve KC1AWV


On Thu, Aug 8, 2019 at 6:15 PM laussana via Groups.Io <laussana=icloud.com@groups.io> wrote:
Hi Steve
Besides the above I assume you need to install Jinja2 and autobahn as the other required components  are already there in HBlink3 ? Still learning how to put these together ..
Thank you
Jorge hk4cze



--
Steve Miller
KC1AWV


Carlos Minguela <minguela@...>
 

Works great!!!! Thanks!!!!!


 

I'm getting the dreaded dict error

Traceback (most recent call last):
  File "monitor.py", line 640, in <module>
    peer_ids = mk_full_id_dict(PATH, PEER_FILE, 'peer')
  File "/usr/local/lib/python3.5/dist-packages/dmr_utils3/utils.py", line 112, in mk_full_id_dict
    records = jload(_handle)['results']
KeyError: 'results'
 


Steve KC1AWV
 

Could be an issue with the peer_ids.json file. Stop monitor.py and delete both subcriber_ids.json and peer_ids.json files and then start monitor.py again. This will (hopefully) force a download of the files and clear that up.

Steve KC1AWV


 

Ya have done that, no dice.

 

From: HBlink@DVSwitch.groups.io <HBlink@DVSwitch.groups.io> On Behalf Of Steve KC1AWV
Sent: Thursday, August 8, 2019 3:38 PM
To: HBlink@DVSwitch.groups.io
Subject: Re: [HBlink] hbmonitor3 available

 

Could be an issue with the peer_ids.json file. Stop monitor.py and delete both subcriber_ids.json and peer_ids.json files and then start monitor.py again. This will (hopefully) force a download of the files and clear that up.

Steve KC1AWV


 

2019-08-08 15:40:31,219 INFO monitor.py starting up

2019-08-08 15:40:31,219 INFO

 

        Copyright (c) 2016, 2017, 2018, 2019

        The Regents of the K0USY Group. All rights reserved.

 

        Python 3 port:

        2019 Steve Miller, KC1AWV <smiller@...>

 

2019-08-08 15:40:32,250 INFO ID ALIAS MAPPER: 'peer_ids.json' successfully downloaded

2019-08-08 15:40:43,940 INFO ID ALIAS MAPPER: 'subscriber_ids.json' successfully downloaded

Traceback (most recent call last):

  File "monitor.py", line 640, in <module>

    peer_ids = mk_full_id_dict(PATH, PEER_FILE, 'peer')

  File "/usr/local/lib/python3.5/dist-packages/dmr_utils3/utils.py", line 112, in mk_full_id_dict

    records = jload(_handle)['results']

KeyError: 'results'

 

From: HBlink@DVSwitch.groups.io <HBlink@DVSwitch.groups.io> On Behalf Of Steve KC1AWV
Sent: Thursday, August 8, 2019 3:38 PM
To: HBlink@DVSwitch.groups.io
Subject: Re: [HBlink] hbmonitor3 available

 

Could be an issue with the peer_ids.json file. Stop monitor.py and delete both subcriber_ids.json and peer_ids.json files and then start monitor.py again. This will (hopefully) force a download of the files and clear that up.

Steve KC1AWV


Steve KC1AWV
 

Driving home right now, so give me a couple of hours and I will try to take another look at it. Unless somebody else has an idea in the meantime.


On Thu, Aug 8, 2019, 6:41 PM Ted Freitas (KE6YJC) via Groups.Io <ted.freitas=me.com@groups.io> wrote:

2019-08-08 15:40:31,219 INFO monitor.py starting up

2019-08-08 15:40:31,219 INFO

 

        Copyright (c) 2016, 2017, 2018, 2019

        The Regents of the K0USY Group. All rights reserved.

 

        Python 3 port:

        2019 Steve Miller, KC1AWV <smiller@...>

 

2019-08-08 15:40:32,250 INFO ID ALIAS MAPPER: 'peer_ids.json' successfully downloaded

2019-08-08 15:40:43,940 INFO ID ALIAS MAPPER: 'subscriber_ids.json' successfully downloaded

Traceback (most recent call last):

  File "monitor.py", line 640, in <module>

    peer_ids = mk_full_id_dict(PATH, PEER_FILE, 'peer')

  File "/usr/local/lib/python3.5/dist-packages/dmr_utils3/utils.py", line 112, in mk_full_id_dict

    records = jload(_handle)['results']

KeyError: 'results'

 

From: HBlink@DVSwitch.groups.io <HBlink@DVSwitch.groups.io> On Behalf Of Steve KC1AWV
Sent: Thursday, August 8, 2019 3:38 PM
To: HBlink@DVSwitch.groups.io
Subject: Re: [HBlink] hbmonitor3 available

 

Could be an issue with the peer_ids.json file. Stop monitor.py and delete both subcriber_ids.json and peer_ids.json files and then start monitor.py again. This will (hopefully) force a download of the files and clear that up.

Steve KC1AWV


 

I was able to download a subset of data and get it to work, it’s because some people don’t have a first or last name and it does not know what to do with a null value.

 

Also see this other error:

 

2019-08-08 15:46:34,364 INFO registered client tcp4:10.41.1.50:53347

Unhandled error in Deferred:

 

Traceback (most recent call last):

  File "/usr/local/lib/python3.5/dist-packages/autobahn/websocket/protocol.py", line 1211, in consumeData

    self.processHandshake()

  File "/usr/local/lib/python3.5/dist-packages/autobahn/websocket/protocol.py", line 2813, in processHandshake

    txaio.add_callbacks(f, self.succeedHandshake, forward_error)

  File "/usr/local/lib/python3.5/dist-packages/txaio/tx.py", line 506, in add_callbacks

    future.addCallbacks(callback, errback)

  File "/usr/local/lib/python3.5/dist-packages/twisted/internet/defer.py", line 311, in addCallbacks

    self._runCallbacks()

--- <exception caught here> ---

  File "/usr/local/lib/python3.5/dist-packages/twisted/internet/defer.py", line 654, in _runCallbacks

    current.result = callback(current.result, *args, **kw)

  File "/usr/local/lib/python3.5/dist-packages/autobahn/websocket/protocol.py", line 3006, in succeedHandshake

    self._onOpen()

  File "/usr/local/lib/python3.5/dist-packages/autobahn/twisted/websocket.py", line 145, in _onOpen

    self.onOpen()

  File "monitor.py", line 566, in onOpen

    self.sendMessage('l' + _message)

  File "/usr/local/lib/python3.5/dist-packages/autobahn/websocket/protocol.py", line 2172, in sendMessage

    assert(type(payload) == bytes)

builtins.AssertionError:

 

2019-08-08 15:46:35,904 INFO BRIDGE EVENT: 'GROUP VOICE,START,RX,BM3103-OBP,2946484089,3103,3145763,1,3100'

2019-08-08 15:46:36,780 INFO BRIDGE EVENT: 'GROUP VOICE,END,RX,BM3103-OBP,2946484089,3103,3145763,1,3100,0.88'

2019-08-08 15:46:42,629 INFO BRIDGE EVENT: 'GROUP VOICE,END,RX,BM3103-OBP,1868919524,3103,1106916,1,31666,41.55'

2019-08-08 15:46:47,193 INFO BRIDGE EVENT: 'GROUP VOICE,START,RX,BM3103-OBP,2354101450,3103,3106184,1,31666'

2019-08-08 15:46:59,072 INFO BRIDGE EVENT: 'GROUP VOICE,END,RX,BM3103-OBP,1703658193,3103,3112133,1,91,41.10'

2019-08-08 15:47:03,257 INFO BRIDGE EVENT: 'GROUP VOICE,START,RX,BM3103-OBP,352058796,3103,5151098,1,91'

2019-08-08 15:47:09,139 INFO Lost connection.  Reason: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionDone'>: Connection was closed cleanly.

]

2019-08-08 15:47:12,303 INFO Initiating Connection to Server.

2019-08-08 15:47:12,304 INFO Connected.

2019-08-08 15:47:12,304 INFO Resetting reconnection delay

 

From: HBlink@DVSwitch.groups.io <HBlink@DVSwitch.groups.io> On Behalf Of Steve KC1AWV
Sent: Thursday, August 8, 2019 3:46 PM
To: HBlink@dvswitch.groups.io
Subject: Re: [HBlink] hbmonitor3 available

 

Driving home right now, so give me a couple of hours and I will try to take another look at it. Unless somebody else has an idea in the meantime.

 

On Thu, Aug 8, 2019, 6:41 PM Ted Freitas (KE6YJC) via Groups.Io <ted.freitas=me.com@groups.io> wrote:

2019-08-08 15:40:31,219 INFO monitor.py starting up

2019-08-08 15:40:31,219 INFO

 

        Copyright (c) 2016, 2017, 2018, 2019

        The Regents of the K0USY Group. All rights reserved.

 

        Python 3 port:

        2019 Steve Miller, KC1AWV <smiller@...>

 

2019-08-08 15:40:32,250 INFO ID ALIAS MAPPER: 'peer_ids.json' successfully downloaded

2019-08-08 15:40:43,940 INFO ID ALIAS MAPPER: 'subscriber_ids.json' successfully downloaded

Traceback (most recent call last):

  File "monitor.py", line 640, in <module>

    peer_ids = mk_full_id_dict(PATH, PEER_FILE, 'peer')

  File "/usr/local/lib/python3.5/dist-packages/dmr_utils3/utils.py", line 112, in mk_full_id_dict

    records = jload(_handle)['results']

KeyError: 'results'

 

From: HBlink@DVSwitch.groups.io <HBlink@DVSwitch.groups.io> On Behalf Of Steve KC1AWV
Sent: Thursday, August 8, 2019 3:38 PM
To: HBlink@DVSwitch.groups.io
Subject: Re: [HBlink] hbmonitor3 available

 

Could be an issue with the peer_ids.json file. Stop monitor.py and delete both subcriber_ids.json and peer_ids.json files and then start monitor.py again. This will (hopefully) force a download of the files and clear that up.

Steve KC1AWV


Steve KC1AWV
 

The speech to text on the phone can't read that log file very well but if it's on line 566, check the stability update that I did to monitor.py. basically change the word messages on that line to be bmessages.


On Thu, Aug 8, 2019, 6:47 PM Ted Freitas (KE6YJC) via Groups.Io <ted.freitas=me.com@groups.io> wrote:

I was able to download a subset of data and get it to work, it’s because some people don’t have a first or last name and it does not know what to do with a null value.

 

Also see this other error:

 

2019-08-08 15:46:34,364 INFO registered client tcp4:10.41.1.50:53347

Unhandled error in Deferred:

 

Traceback (most recent call last):

  File "/usr/local/lib/python3.5/dist-packages/autobahn/websocket/protocol.py", line 1211, in consumeData

    self.processHandshake()

  File "/usr/local/lib/python3.5/dist-packages/autobahn/websocket/protocol.py", line 2813, in processHandshake

    txaio.add_callbacks(f, self.succeedHandshake, forward_error)

  File "/usr/local/lib/python3.5/dist-packages/txaio/tx.py", line 506, in add_callbacks

    future.addCallbacks(callback, errback)

  File "/usr/local/lib/python3.5/dist-packages/twisted/internet/defer.py", line 311, in addCallbacks

    self._runCallbacks()

--- <exception caught here> ---

  File "/usr/local/lib/python3.5/dist-packages/twisted/internet/defer.py", line 654, in _runCallbacks

    current.result = callback(current.result, *args, **kw)

  File "/usr/local/lib/python3.5/dist-packages/autobahn/websocket/protocol.py", line 3006, in succeedHandshake

    self._onOpen()

  File "/usr/local/lib/python3.5/dist-packages/autobahn/twisted/websocket.py", line 145, in _onOpen

    self.onOpen()

  File "monitor.py", line 566, in onOpen

    self.sendMessage('l' + _message)

  File "/usr/local/lib/python3.5/dist-packages/autobahn/websocket/protocol.py", line 2172, in sendMessage

    assert(type(payload) == bytes)

builtins.AssertionError:

 

2019-08-08 15:46:35,904 INFO BRIDGE EVENT: 'GROUP VOICE,START,RX,BM3103-OBP,2946484089,3103,3145763,1,3100'

2019-08-08 15:46:36,780 INFO BRIDGE EVENT: 'GROUP VOICE,END,RX,BM3103-OBP,2946484089,3103,3145763,1,3100,0.88'

2019-08-08 15:46:42,629 INFO BRIDGE EVENT: 'GROUP VOICE,END,RX,BM3103-OBP,1868919524,3103,1106916,1,31666,41.55'

2019-08-08 15:46:47,193 INFO BRIDGE EVENT: 'GROUP VOICE,START,RX,BM3103-OBP,2354101450,3103,3106184,1,31666'

2019-08-08 15:46:59,072 INFO BRIDGE EVENT: 'GROUP VOICE,END,RX,BM3103-OBP,1703658193,3103,3112133,1,91,41.10'

2019-08-08 15:47:03,257 INFO BRIDGE EVENT: 'GROUP VOICE,START,RX,BM3103-OBP,352058796,3103,5151098,1,91'

2019-08-08 15:47:09,139 INFO Lost connection.  Reason: [Failure instance: Traceback (failure with no frames): <class 'twisted.internet.error.ConnectionDone'>: Connection was closed cleanly.

]

2019-08-08 15:47:12,303 INFO Initiating Connection to Server.

2019-08-08 15:47:12,304 INFO Connected.

2019-08-08 15:47:12,304 INFO Resetting reconnection delay

 

From: HBlink@DVSwitch.groups.io <HBlink@DVSwitch.groups.io> On Behalf Of Steve KC1AWV
Sent: Thursday, August 8, 2019 3:46 PM
To: HBlink@dvswitch.groups.io
Subject: Re: [HBlink] hbmonitor3 available

 

Driving home right now, so give me a couple of hours and I will try to take another look at it. Unless somebody else has an idea in the meantime.

 

On Thu, Aug 8, 2019, 6:41 PM Ted Freitas (KE6YJC) via Groups.Io <ted.freitas=me.com@groups.io> wrote:

2019-08-08 15:40:31,219 INFO monitor.py starting up

2019-08-08 15:40:31,219 INFO

 

        Copyright (c) 2016, 2017, 2018, 2019

        The Regents of the K0USY Group. All rights reserved.

 

        Python 3 port:

        2019 Steve Miller, KC1AWV <smiller@...>

 

2019-08-08 15:40:32,250 INFO ID ALIAS MAPPER: 'peer_ids.json' successfully downloaded

2019-08-08 15:40:43,940 INFO ID ALIAS MAPPER: 'subscriber_ids.json' successfully downloaded

Traceback (most recent call last):

  File "monitor.py", line 640, in <module>

    peer_ids = mk_full_id_dict(PATH, PEER_FILE, 'peer')

  File "/usr/local/lib/python3.5/dist-packages/dmr_utils3/utils.py", line 112, in mk_full_id_dict

    records = jload(_handle)['results']

KeyError: 'results'

 

From: HBlink@DVSwitch.groups.io <HBlink@DVSwitch.groups.io> On Behalf Of Steve KC1AWV
Sent: Thursday, August 8, 2019 3:38 PM
To: HBlink@DVSwitch.groups.io
Subject: Re: [HBlink] hbmonitor3 available

 

Could be an issue with the peer_ids.json file. Stop monitor.py and delete both subcriber_ids.json and peer_ids.json files and then start monitor.py again. This will (hopefully) force a download of the files and clear that up.

Steve KC1AWV