Slow and unstable connection to panel
Hello, I'm struggling with slow connection to the Panel. When I try to connect I get "Connecting to server, attempt number : ...1" and good if I will connect over 20-30 counts. Eventually, connect will happen but become unstable. Clicking on any button may cause re-connection again (but may be not).
What is even more strange, Panel has tend to be "warming up". If I more often manually re-connect to panel and more frequent clicking on buttons, panel becomes more responsive and stable...
I run Elastix 2.4 and recently upgraded to FOP2 2.29 latest but this problem has been persistent all way down to Elastix 2.4 and FOP2 2.0. At the very initial stage it worked well but then something happened.
FOP1 worked just fine.
I carefully reviewed all related posts:
- port 4445 is open and listening
- FOP1 is disabled
- Firewall (iptables) switched off
- excessive events in manager.conf fired off
- workstation is connected to the server over direct Ethernet cable (only switch between, no active devices)
- same problem in Chrome and Firefox
- I use HTTPS, probably because it's default for Elastix I don't know how to bypass HTTPS and connect via HTTP, it simply doesn't work.
- Chrome extension seems work smooth
Any ideas?
What is even more strange, Panel has tend to be "warming up". If I more often manually re-connect to panel and more frequent clicking on buttons, panel becomes more responsive and stable...
I run Elastix 2.4 and recently upgraded to FOP2 2.29 latest but this problem has been persistent all way down to Elastix 2.4 and FOP2 2.0. At the very initial stage it worked well but then something happened.
FOP1 worked just fine.
I carefully reviewed all related posts:
- port 4445 is open and listening
- FOP1 is disabled
- Firewall (iptables) switched off
- excessive events in manager.conf fired off
- workstation is connected to the server over direct Ethernet cable (only switch between, no active devices)
- same problem in Chrome and Firefox
- I use HTTPS, probably because it's default for Elastix I don't know how to bypass HTTPS and connect via HTTP, it simply doesn't work.
- Chrome extension seems work smooth
Any ideas?
Comments
FOP2 knows every Websocket protocol, and it also uses flash xmlsockets as a fallback mechanism (and it was initially the only protocol before Websocket even existed). So, if you do not have ssl certificates configured for the fop2 server, it will offer only normal websocket , the browser will fail to negotiate secure web sockets, then it will attempt normal websocket and browser will make it fail, and finally it will attempt flash xmlsockets and it will work (but it takes several attemps/fallbacks until the connection is finally done).
What this means is that in order to have a fast negotiation of a web socket connection with FOP2 when using https, you *must* configure the ssl certificates in fop2.cfg. Secure web socket was added in FOP 2.27, but usually when you upgrade, your original fop2.cfg file is preserved, and the new one (with new options) is saved as fop2.cfg.new. (So you must edit fop2.cfg by hand and add the proper options).
If you come from Elastix, then you might have installed the Elastix version of FOP2 also, that version has configuration files in a different place than the standard FOP2 install.. so, check the contents of the file /etc/sysconfig/fop2. If in that file you have -c /etc/asterisk/fop2 as one of the options, the the config file is in /etc/asterisk/fop2 otherwise it is in /usr/local/fop2. Open the file (fop2.cfg) and add the ssl certtificate configurations (they must match exactly the certificates that are set in your web browser), in an Elastix install the default values are:
[fixed]
ssl_certificate_file=/etc/pki/tls/certs/localhost.crt
ssl_certificate_key_file=/etc/pki/tls/private/localhost.key
[/fixed]
With that in place, after restarting FOP2, the initial connection to it over https should be fast and not take more than one cycle/attempt.
Now, this does not affect at all the 'stability' you mention. That is something entirely different and I do not know what can cause that, except for some kind of network issue between the browser and the server. FOP2 uses different keepalive mechanisms besides the actual protocol keepalives, and it will reconnect if those fail.
Well, I re-intsalled FOP2 step-by-step from your site so not sure if something left from Elastix.
In /etc/sysconfig/fop2 I have only OPTIONS="-d"
In /etc/asterisk/fop2 I have nothing
In /usr/local/fop2 I have normal fop2.cfg; both strings are in place.
I double checked; both certificates are in place.
We have Certificate Server (from Microsoft) in our environment so I issued and installed certificates.
It seems they work in Elastix. The only issue that my certificate chain doesn't have intermediate server and I don't have published CRL. Both Chrome and Firefox consider this as unsecure but skip usual additional clicks to open "unsecure" sites.
Any chance to check if my certificates are okey for FOP2? In some Microsoft products. lack of CRL is an issue.
Could you advice, why I can't connect over HTTP at all?
It says "this page attempts to load unsecured scripts" If I click "load unsecured scripts" it triggers long reconncetion cycle. Could it say something?
https://your.server:4445
That should prompt you for the insecure certificate, that you must allow. After that, the background websocket connection when accessing fop2 normally should work.
in any case, you can open the javascript console in chrome while attempting connection, and you will see every connection attempt, the method, and fallbacks..
Could you look into???
So, I have following errors:
Failed to load resource: the server responded with a status of 404 (Not Found)
fail fop2-variables, default to port 4445 wit no TLS
antes de preinit fail fop2variables
pre init
Client has HTML5 web sockets!
connectxml
intento conectar web socket en wss://my.server:4445
set session context
---- Now following connection attempts, counts of 11---
WebSocket connection to 'wss://my.server:4445/' failed: Error in connection establishment: net::ERR_TIMED_OUT
WebSocket Error
could not connect via wss, attempt ws
---- Now following connection attempts, counts of 5 ---
intento conectar web socket en ws://my.server:4445
Mixed Content: The page at 'https://my.server/fop2/?exten=1200&pass=xxxx' was loaded over HTTPS, but attempted to connect to the insecure WebSocket endpoint 'ws://my.server:4445/'. This request has been blocked; this endpoint must be available over WSS.
WebSocket Error
could not connect via ws, attempt flash xmlsockets
---- Now following connection attempts, counts of 10 ---
connectxml
embed flash
Attempt flash xmlsocket connection on port 4445
2
Connection successful flash xmlsockets general
-- Now there are a lot of stuff and ---
GET https://my.server/fop2/fop2-variablesGENERAL.txt 404 (Not Found)
fail fop2-variables, default to port 4445 wit no TLS
antes de preinit fail fop2variables
pre init
Client has HTML5 web sockets!
connectxml
intento conectar web socket en wss://my.server:4445
set session context
Best regards,
[fixed]
callback (ignoring): /etc/pki/tls/certs/localhost.crt: failed to use local certificate chain (cert_file or cert)
[/fixed]
FOP2 knows how to handle certificate files in PEM format (the most common, that is base64 encoded).
nikriaz had a custom certificate in another (binary) format (DEM). FOP2 was not able to handle that certificate so it did not negotiate SSL. Converting it to PEM format made it work.
The command to convert a certificate from DEM to PEM:
Then changing the file to read on fop2.cfg
[fixed]
ssl_certificate_file=/etc/pki/tls/certs/localhost.pem
[/fixed]
(As I did not want to alter the original certificate that might be used by other software and perhaps only in that format).
With the file in the correct format FOP2 knows how to handle it.