FOP2 Stops responding after awhile
After a certain period of inactivity FOP 2.20 stops reponding. I can log into the FOP and everything shows up as it should but nothing changes(calls don't show up, etc). I've tried refreshing the screen, using chrome and IE instead of firefox, and logging in as different extensions.
If I restart the service (service fop2 restart). It works fine again. I'm currently trying to figure out how long the program works before I have this issue and I will post that as soon as I have a conclusive answer. It seems a bit random when it occurs. Sometimes it will be fine when I come in in the morning (15 hours of inactivity) and sometimes not. On the other hand I just left for a 45 minute lunch and when I came back I'm having the same issue. I'm also not sure if this only occurs when there is complete inactivity (no phone calls and no FOP2 access) or if it would happen if the phones were being used as well (machine is still in testing so not many calls)
I am running PIAF 1.7.5.5 Bronze (Asterisk 1.6.2.16.1) and I have disabled FOP1 by changing "FOPRUN" in the amportal.conf file. Using FOP2.20.
Does FOP2 log anywhere where I might be able to see what the heck is going on? Anybody have any ideas?
Thanks,
Tyler
If I restart the service (service fop2 restart). It works fine again. I'm currently trying to figure out how long the program works before I have this issue and I will post that as soon as I have a conclusive answer. It seems a bit random when it occurs. Sometimes it will be fine when I come in in the morning (15 hours of inactivity) and sometimes not. On the other hand I just left for a 45 minute lunch and when I came back I'm having the same issue. I'm also not sure if this only occurs when there is complete inactivity (no phone calls and no FOP2 access) or if it would happen if the phones were being used as well (machine is still in testing so not many calls)
I am running PIAF 1.7.5.5 Bronze (Asterisk 1.6.2.16.1) and I have disabled FOP1 by changing "FOPRUN" in the amportal.conf file. Using FOP2.20.
Does FOP2 log anywhere where I might be able to see what the heck is going on? Anybody have any ideas?
Thanks,
Tyler
Comments
Start fop2 with debug enabled under a gnu screen session:
screen -dmS fop2 /usr/local/fop2/fop2_server -X 511
(you will need to stop the service first "service fop2 stop")
When you notice the problem, attach to the screen session with:
screen -r fop2
And look at the output.... that will give us a hint on what is going on.
Best regards,
The following is what is currently up. Is there any way to go back further in the log? Or do you notice anything from that?
Thanks Again,
Tyler
Well, I do not see anything that can help. What I wanted to know if the AMI-Fop2 server connection was still up when you notice the problem. If it is not up, you will see some connection attempts logged, and you should not see anything like:
1
Those are actual AMI events that are being read/received by fop2 server. If the AMI connection stalls or freezes or dies, then fop2 will stop receiving status updates. But for what I see, if those are the latest lines in the screen session, then that did not happen, you might want to look for individual hangup events for the call you saw as frozen...
ctrl-a ]
and then the arrows, if I recall correctly.
I have the exact same issue with my Firefox/Ubuntu system.
XP virtual works fine though.
To do that you need to backup this file /var/www/html/fop2/index.html and then edit the following line in it:
Replace
With
I don't know if you need the rest of the meta line, but I made this change about 2 months ago and it's been working fine.
I keep an XP virtual open all day anyways and I'm the only one using Ubuntu, so I'll just live with it. Though it would be nice if it could be addressed.
Thank you for the online chat help!!
The panel hangs until a manual browser refresh with some browsers.
Tested with Chrome 11.0.696.57 beta and Firefox 4 on openSuse 11.4, IE8 and Firefox 3.6 on Windows 7 and IE7 on Vista.
Works with:
- Firefox 4 (openSuse 11.4)
- IE8 and Firefox 3.6 (Windows 7)
Works NOT with:
- Chrome 11.0.696.57 beta (openSuse 11.4)
- IE7 (Vista)
Best regards,
wav false
set session context
formato wav
set menu
Client has HTML5 web sockets!
setLang
set menu
set menu queue
no tiene permiso chat
get cookie extensionlist
get cookie queuelist
get cookie trunklist
get cookie conferencelist
get cookie parklist
Client has HTML5 web sockets!
set session extension
/fop2/fop2-variablesGENERAL.txtFailed to load resource: the server responded with a status of 404 (Not Found)
Second Init after loading variables
Attempt to connect to port 4445
12
intento conectar web socket en [url=ws://sip.paviliongift.com:4445]ws://sip.paviliongift.com:4445[/url]
on open
go
ws send <msg data="GENERAL|contexto|1|" />
ws connect 1
0,key=reTbUlZklQicDOALAMquHC7neiU en slot
0,version=2.20!!1 en slot
ws send <msg data="1|auth|149|79c9ab4cb9b0eee1b7ddaacb20c8398f" />
0,preferences=eyAiZGlzcGxheVF1ZXVlIjogIm1heCIsImR5bmFtaWNMaW5lRGlzcGxheSI6ICJvZmYiLCJsYW5ndWFnZSI6ICJlbiIsIm5vdGlmeUR1cmF0aW9uIjogIjYiLCJzb3VuZENoYXQiOiAiIiwic291bmRRdWV1ZSI6ICIiLCJzb3VuZFJpbmciOiAiIn0= en slot
llegaron las preferences { "displayQueue": "max","dynamicLineDisplay": "off","language": "en","notifyDuration": "6","soundChat": "","soundQueue": "","soundRing": ""}
Ponemos preferencias en valores default
0,vmailpath=/var/spool/asterisk/voicemail en slot
0,permit=YWxs en slot
set session phonebook
set session admin
2drawButton
set menu queue
no tiene licencia chat
ws send <msg data="1|initState||79c9ab4cb9b0eee1b7ddaacb20c8398f" />
0,demo=2 en slot 0
2,qualify=notok en slot 0
3,voicemail=2 en slot
3,voicemailcount=&vmail_count!0!1 en slot
3,qualify=ok en slot 0
4,status=unpaused en slot 0
4,rename=125 Ann Koehler en slot
4,qualify=ok en slot 0
5,qualify=ok en slot 0
6,qualify=ok en slot 0
8,status=unpaused en slot 0
8,rename=143 Daniel VerWeire en slot
8,voicemail=1 en slot
8,voicemailcount=&vmail_count!2!0 en slot
8,qualify=ok en slot 0
9,status=unpaused en slot 0
9,qualify=ok en slot 0
10,status=unpaused en slot 0
10,rename=152 Jerah Augello en slot
10,qualify=ok en slot 0
11,status=unpaused en slot 0
11,rename=153 Norma Pagano en slot
11,qualify=ok en slot 0
12,details= en slot 0
13,queuemembers=PGRpdiBjbGFzcz0nbWVtYmVycmVhZHkgbXljbGljaycgaWQ9J3FtIVFVRVVFLzI1MCFMb2NhbC8xNDlAZnJvbS1xdWV1ZS9uJz48L2Rpdj48c3Bhbj5TaGF3biBXaWx0c2U8L3NwYW4+PGJyIGNsYXNzPSdjbGVhcicvPg== en slot
14,queuemembers=PGRpdiBjbGFzcz0nbWVtYmVycmVhZHkgbXljbGljaycgaWQ9J3FtIVFVRVVFLzI1MSFMb2NhbC8xMjVAZnJvbS1xdWV1ZS9uJz48L2Rpdj48c3Bhbj5Bbm4gS29laGxlcjwvc3Bhbj48YnIgY2xhc3M9J2NsZWFyJy8+PGRpdiBjbGFzcz0nbWVtYmVycmVhZHkgbXljbGljaycgaWQ9J3FtIVFVRVVFLzI1MSFMb2NhbC8xNTJAZnJvbS1xdWV1ZS9uJz48L2Rpdj48c3Bhbj5KZXJhaCBBdWdlbGxvPC9zcGFuPjxiciBjbGFzcz0nY2xlYXInLz48ZGl2IGNsYXNzPSdtZW1iZXJyZWFkeSBteWNsaWNrJyBpZD0ncW0hUVVFVUUvMjUxIUxvY2FsLzE1M0Bmcm9tLXF1ZXVlL24nPjwvZGl2PjxzcGFuPk5vcm1hIFBhZ2Fubzwvc3Bhbj48YnIgY2xhc3M9J2NsZWFyJy8+ en slot
15,queuemembers= en slot
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
4,settimer=0@UP en slot 1
4,state=UP en slot 1
4,settext=125 device en slot 1
4,direction=outbound en slot 1
4,settext=915073764556 en slot 1
4,settext=ZultysMX25/15073764556 en slot 1
4,direction=outbound en slot 1
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
4,settimer=0@UP en slot 1
4,state=UP en slot 1
4,settext=915073764556 en slot 1
4,settext=915073764556 en slot 1
4,link=915073764556 en slot 1
4,settimer=0@STOP en slot 1
4,settext=&inactive_line!1 en slot 1
4,state=DOWN en slot 1
DOWN pero quedan sesiones, no pongo boton en free
4,state=DOWN en slot 0
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
4,qualify=ok en slot 0
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
If you want to fix it you must restart fop2 server, the problem seems to occur when asterisk is restarted while there are fop2 clients connected.
Brief explanation of what it might be happening:
Upon fop2 start, he order of the socket connections is this:
1) Connection to AMI (server)
2) [More AMI Connections on multi server]
3) Connection to Client 1
4) Connection to Client 2
5) etc...
When the connection to AMI is lost (asterisk restarted or crashed), and if you have clients connected to fop2, the order changes and you end up with this:
1) Connection to Client 1
2) Connection to Client 2
3) Connetion to AMI (server)
Now, if a new client connects, you will have 2 clients , 1 manager and a final client connection. In those cases , not sure if the first or the last connections after AMI will stop receiving updates.
1) Connection to Client 1
2) Connection to Client 2
3) Connetion to AMI (server)
4) Connection to Client 3
Restarting fop2 server will clear the issue as the socket order is restored. The next beta includes a fix that will make all clients receive updates (it will not clear the event cache after each AMI read cycle)
~ca
However, there are two issues that we've noticed. One is that sometimes parked callers aren't removed from fop (we have someone who's supposedly been parked for an hour and a half, but picking up the line gives "no call parked). Sometimes it works fine thought.
The other is that only our IAX trunk shows calls, the ZAP trunks we have constly remain green, and never show a call, despite sometimes being in use.
Thanks,
~ca
Is the beta stable enough? I'm not looking for half baked solutions.
There are lots of issues discussed in this post, so I am not sure what is the deal breaker for you. I am sorry if you think fop2 is a half baked solution. Asterisk has tons of nasty bugs too, and I do not think it is half baked either.
If you experience a flash client that stops updating, it is because asterisk crashes, not something else. If asterisk is stable, you will not experience the issue. If the connection with asterisk dies unexpectedly while you have flash clients connected into fop2, then you might experience the issue where some flash clients gets updates and some others do not get them. The problem is fixed if you restart fop2. If asterisk never crashes, or if you remember to restart fop2 server when you also restart asterisk, then there is no issue. Anyways, the beta fixes this, it "restarts itself" when asterisk crashes, and avoids the problem.
Now, if your deal breaker is something else, please let me know what it is so I can work on the fix.
Best regards,
Parking is complicated, the park bugs are related mostly to asterisk bugs or odd cases involving masquerading, local channels, etc. They are really hard to reproduce, and when users were able to capture manager logs of the issues, most of the times is because missing or nonsense manager events related to those parked calls (like parking channel A, and then getting an unpark event for channel
Are you using fop2admin? Did you setup the channel ranges in fop2 buttons? You must do it in order to see status.
Best regards,
The problem is FOP2 is not updating. Has nothing to do with Asterisk which is running properly while this is happening. If I am on a call via a trunk FOP2 still shows the extension and trunk as free unless I refresh my browser. Then when the call ends I need to refresh the browser to have it update the status again. That's a deal breaker if I can't get that working properly.
But sometimes it works. So it's intermittent and not likely a configuration issue.
Restarting fop2_server will fix the issue. The problem happens ONLY if asterisk is restarted while fop2_server is still running AND you have a flash client connected.
Imagine this scenario, you boot the machine:
asterisk starts
fop2 starts
some fop2 client connects via web
everything is fine and dandy
suddenly asterisk crashes
fop2_server is still running
fop2 client is still connected, and it will not see updates, as asterisk is GONE
safe_asterisk relaunches asterisk
at that point, your old connected flash client will not receive updates, but newer fop2 client connections will.
restarting fop2_server fixes the issue. It is a bug in the way fop2 handles asterisk disconnections, and the problem is addressed already in the beta, and will be rolled out on the next version. But it only occurs when asterisk is restarted or dies, without restarting fop2_server.
Try it, instead of hitting F5 in your browser when you notice the problem, just restart fop2 server
service fop2 restart
and the problem will be gone.
Best regards,
I hope you are not suggesting that requiring us to restart the service is your permanent solution. This happens quite often and your suggestion is a non-starter. You must not use FreePBX where Asterisk restarts happen every time you make a change via the GUI which happens all the time. What is the permanent solution and when can I expect it? Otherwise I may have to go back to FOP1
A permanent solution could be to have a stable asterisk that does not crash. Or remember to restart fop2 when you restart asterisk. Or if your asterisk still crashes and you are fine with the safe_asterisk script bringing it up again, just add a line to the safe_asterisk script to do a service fop2 restart there too. Something like this (untested)
Be creative. I tried to describe *when* the issue happens, so you know where to look for to address the problem. If you are unable to avoid asterisk crashes, then you might want to try updating the safe_asterisk script. However, I would not do this myself, I would rather fix asterisk or my asterisk configuration to avoid crashes or ami disconnections.
You know, if asterisk crashes, taking down all your active calls with it.... its much worst that than having some fop2 clients not updating after that crash, fop2 would be my last of concerns. I would try to get asterisk stabilized first. This *only* happens when you have a fop2 client connected to fop2 server, and then the AMI connection gets down or broken. It is not usual in normal circumstances. For me and for the majority of users this does not happen often or at all. It was very hard for me to reproduce the issue, fortunately I got help from users who let me look at their servers when the problem happened, and I was able to find the cause.
Anyways, the next fop2 version fixes this, as it is indeed a fop2 issue. You are welcome to try the beta if you want: http://www.fop2.com/downloads/fop2-2.21 ... 5-i386.tgz
You can also go back to FOP1 if it makes you feel more comfortable. Please excuse me, but I am still sensing a hostile attitude towards me and the software itself. It seems like you want to spread FUD more than anything else.
Can the beta just be installed over the original?
Or what is the correct process for re-installing on a working system?
thanks.
The suggestion of putting fop2 restart in the safe_asterisk restart script is a more practical solution assuming it is initiated everytime FreePBX is updated via the Apply changes GUI button (I'm not sure if it is). Again, I don't know why you are going off on a tangent trying to blame Asterisk. As I said repeatedly asterisk is NOT crashing. Asterisk is NOT crashing. Did I mention that asterisk is NOT crashing?
You seem to have no concept of how people use this. Most use FreePBX and for those people your solution of "just restart fop2 service because it rarely happens" is ridiculous. FreePBX is constantly being used and the way FreePBX works is every change restarts asterisk which kills fop2. That does not seem to sink in with you for some reason. Also a lot of administrators like the ones I deal with do not know linux. They can use a browser and configure Asterisk via FreePBX but asking them to open a command line and restart the FOP2 service is a completely foreign concept to them. I may as well be asking them to fly to the moon.
My attitude about it is irrelevant but welcome to the world of paid apps. I have never complained about FOP1 because it's completely free and has always worked well for me. FOP2 is intentionally limited to encourage people to buy a license and my expectations are much higher for a product like that. At a minimum I do NOT expect show stoppers like this. And in this case where there is a show stopper, I don't expect useless advice about how to get around it.
Frankly, if you just said, "yea it's a problem, these things happen sometimes, I think I fixed it in the current beta which may be buggy right now" and then some sort of estimate on a stable version is a better answer.
An idea until the next version is final...
Is there a way to make a button, or could you make a button on the Fop2 panel, that would restart the fop2 service?
That way it could be refreshed if there is an issue?
Another way to do it.... Is there any harm in running a cron script that runs:
/etc/init.d/fop2 restart
every 15 minutes?
thanks.