FOP2 Stops responding after awhile

tvittito

You might give chrome a try. It seemed to behave a little bit better in that for some reason. What asterisk version are you running?

nicolas

I use ubuntu/firefox, and I do not experience the issue at all. In order to fix bugs, the idea is for you to capture at least some debug output as explained in this and several other posts. It seems you have a problem with your client if it works for other machines in the system (client is firefox/flash). You might want to test at least with google chrome, that uses html5 websockets instead of flash. Or you can install firebug and open the javascript console and send me some of the output you see there in order to troubleshot. Best regards,

gregc

My XP virtual machine had a firefox update this morning and is now exhibiting the same issue. I'll do a capture as suggested.

gregc

Issue ended up being with FOP1 interfering. I have it disabled now and things look to be working normally.

Thank you for the online chat help!!

fme

Same problem here.
The panel hangs until a manual browser refresh with some browsers.

Tested with Chrome 11.0.696.57 beta and Firefox 4 on openSuse 11.4, IE8 and Firefox 3.6 on Windows 7 and IE7 on Vista.

Works with:

Firefox 4 (openSuse 11.4)
IE8 and Firefox 3.6 (Windows 7)

Works NOT with:

Chrome 11.0.696.57 beta (openSuse 11.4)
IE7 (Vista)

nicolas

Thanks for taking the time to make all those tests. But unfortunately none of them will help much if you do not at least open the javascript console on an affected browser and send me the output. Chrome has such a console.

Best regards,

swiltse

I'm experiencing similar issues with Chrome 11.0.696.65 not responding after a while on a WinXP machine. Here's the info from the console:

wav false
set session context
formato wav
set menu
Client has HTML5 web sockets!
setLang
set menu
set menu queue
no tiene permiso chat
get cookie extensionlist
get cookie queuelist
get cookie trunklist
get cookie conferencelist
get cookie parklist
Client has HTML5 web sockets!
set session extension
/fop2/fop2-variablesGENERAL.txtFailed to load resource: the server responded with a status of 404 (Not Found)
Second Init after loading variables
Attempt to connect to port 4445
12
intento conectar web socket en ws://sip.paviliongift.com:4445
on open
go
ws send <msg data="GENERAL|contexto|1|" />
ws connect 1
0,key=reTbUlZklQicDOALAMquHC7neiU en slot
0,version=2.20!!1 en slot
ws send <msg data="1|auth|149|79c9ab4cb9b0eee1b7ddaacb20c8398f" />
0,preferences=eyAiZGlzcGxheVF1ZXVlIjogIm1heCIsImR5bmFtaWNMaW5lRGlzcGxheSI6ICJvZmYiLCJsYW5ndWFnZSI6ICJlbiIsIm5vdGlmeUR1cmF0aW9uIjogIjYiLCJzb3VuZENoYXQiOiAiIiwic291bmRRdWV1ZSI6ICIiLCJzb3VuZFJpbmciOiAiIn0= en slot
llegaron las preferences { "displayQueue": "max","dynamicLineDisplay": "off","language": "en","notifyDuration": "6","soundChat": "","soundQueue": "","soundRing": ""}
Ponemos preferencias en valores default
0,vmailpath=/var/spool/asterisk/voicemail en slot
0,permit=YWxs en slot
set session phonebook
set session admin
2drawButton
set menu queue
no tiene licencia chat
ws send <msg data="1|initState||79c9ab4cb9b0eee1b7ddaacb20c8398f" />
0,demo=2 en slot 0
2,qualify=notok en slot 0
3,voicemail=2 en slot
3,voicemailcount=&vmail_count!0!1 en slot
3,qualify=ok en slot 0
4,status=unpaused en slot 0
4,rename=125 Ann Koehler en slot
4,qualify=ok en slot 0
5,qualify=ok en slot 0
6,qualify=ok en slot 0
8,status=unpaused en slot 0
8,rename=143 Daniel VerWeire en slot
8,voicemail=1 en slot
8,voicemailcount=&vmail_count!2!0 en slot
8,qualify=ok en slot 0
9,status=unpaused en slot 0
9,qualify=ok en slot 0
10,status=unpaused en slot 0
10,rename=152 Jerah Augello en slot
10,qualify=ok en slot 0
11,status=unpaused en slot 0
11,rename=153 Norma Pagano en slot
11,qualify=ok en slot 0
12,details= en slot 0
13,queuemembers=PGRpdiBjbGFzcz0nbWVtYmVycmVhZHkgbXljbGljaycgaWQ9J3FtIVFVRVVFLzI1MCFMb2NhbC8xNDlAZnJvbS1xdWV1ZS9uJz48L2Rpdj48c3Bhbj5TaGF3biBXaWx0c2U8L3NwYW4+PGJyIGNsYXNzPSdjbGVhcicvPg== en slot
14,queuemembers=PGRpdiBjbGFzcz0nbWVtYmVycmVhZHkgbXljbGljaycgaWQ9J3FtIVFVRVVFLzI1MSFMb2NhbC8xMjVAZnJvbS1xdWV1ZS9uJz48L2Rpdj48c3Bhbj5Bbm4gS29laGxlcjwvc3Bhbj48YnIgY2xhc3M9J2NsZWFyJy8+PGRpdiBjbGFzcz0nbWVtYmVycmVhZHkgbXljbGljaycgaWQ9J3FtIVFVRVVFLzI1MSFMb2NhbC8xNTJAZnJvbS1xdWV1ZS9uJz48L2Rpdj48c3Bhbj5KZXJhaCBBdWdlbGxvPC9zcGFuPjxiciBjbGFzcz0nY2xlYXInLz48ZGl2IGNsYXNzPSdtZW1iZXJyZWFkeSBteWNsaWNrJyBpZD0ncW0hUVVFVUUvMjUxIUxvY2FsLzE1M0Bmcm9tLXF1ZXVlL24nPjwvZGl2PjxzcGFuPk5vcm1hIFBhZ2Fubzwvc3Bhbj48YnIgY2xhc3M9J2NsZWFyJy8+ en slot
15,queuemembers= en slot
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
4,settimer=0@UP en slot 1
4,state=UP en slot 1
4,settext=125 device en slot 1
4,direction=outbound en slot 1
4,settext=915073764556 en slot 1
4,settext=ZultysMX25/15073764556 en slot 1
4,direction=outbound en slot 1
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
4,settimer=0@UP en slot 1
4,state=UP en slot 1
4,settext=915073764556 en slot 1
4,settext=915073764556 en slot 1
4,link=915073764556 en slot 1
4,settimer=0@STOP en slot 1
4,settext=&inactive_line!1 en slot 1
4,state=DOWN en slot 1
DOWN pero quedan sesiones, no pongo boton en free
4,state=DOWN en slot 0
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot
4,qualify=ok en slot 0
ws send <msg data="1|ping||" />
envio ping 2
0,pong=0 en slot

nicolas

For everyone experience this issue, I think I found the cause. I will post a new beta tomorrow that you will be able to try, the problem is that the beta has a ton of new major features , so expect a ton of new bugs.

If you want to fix it you must restart fop2 server, the problem seems to occur when asterisk is restarted while there are fop2 clients connected.

Brief explanation of what it might be happening:

Upon fop2 start, he order of the socket connections is this:

Connection to AMI (server)
[More AMI Connections on multi server]
Connection to Client 1
Connection to Client 2
etc...

When the connection to AMI is lost (asterisk restarted or crashed), and if you have clients connected to fop2, the order changes and you end up with this:

Connection to Client 1
Connection to Client 2
Connetion to AMI (server)

Now, if a new client connects, you will have 2 clients , 1 manager and a final client connection. In those cases , not sure if the first or the last connections after AMI will stop receiving updates.

Connection to Client 1
Connection to Client 2
Connetion to AMI (server)
Connection to Client 3

Restarting fop2 server will clear the issue as the socket order is restored. The next beta includes a fix that will make all clients receive updates (it will not clear the event cache after each AMI read cycle)

compassist

Hi, I'm having the same issue with fop2.20 (basic license). Could you link/email me a copy of the beta version to test out? Thanks,

~ca

nicolas

http://www.fop2.com/downloads/fop2-2.21 ... 5-i386.tgz

compassist

Thanks, that seems to have fixed the problem with fop2 stopping.

However, there are two issues that we've noticed. One is that sometimes parked callers aren't removed from fop (we have someone who's supposedly been parked for an hour and a half, but picking up the line gives "no call parked). Sometimes it works fine thought.

The other is that only our IAX trunk shows calls, the ZAP trunks we have constly remain green, and never show a call, despite sometimes being in use.

Thanks,

~ca

mustardman

What is the status of this? I'm having this problem too. It's basically a deal breaker if I can't get this working.

Is the beta stable enough? I'm not looking for half baked solutions.

nicolas

Hi,

What is the status of this? I'm having this problem too. It's basically a deal breaker if I can't get this working.

Is the beta stable enough? I'm not looking for half baked solutions.

There are lots of issues discussed in this post, so I am not sure what is the deal breaker for you. I am sorry if you think fop2 is a half baked solution. Asterisk has tons of nasty bugs too, and I do not think it is half baked either.

If you experience a flash client that stops updating, it is because asterisk crashes, not something else. If asterisk is stable, you will not experience the issue. If the connection with asterisk dies unexpectedly while you have flash clients connected into fop2, then you might experience the issue where some flash clients gets updates and some others do not get them. The problem is fixed if you restart fop2. If asterisk never crashes, or if you remember to restart fop2 server when you also restart asterisk, then there is no issue. Anyways, the beta fixes this, it "restarts itself" when asterisk crashes, and avoids the problem.

Now, if your deal breaker is something else, please let me know what it is so I can work on the fix.

Best regards,

nicolas

However, there are two issues that we've noticed. One is that sometimes parked callers aren't removed from fop (we have someone who's supposedly been parked for an hour and a half, but picking up the line gives "no call parked). Sometimes it works fine thought.

Parking is complicated, the park bugs are related mostly to asterisk bugs or odd cases involving masquerading, local channels, etc. They are really hard to reproduce, and when users were able to capture manager logs of the issues, most of the times is because missing or nonsense manager events related to those parked calls (like parking channel A, and then getting an unpark event for channel B). There is another case when using named parking lots (freepbx does not use them), where the unpark event does not have the parkinglot in the event itself, so fop2 tries to follow the channel life between events, but sometimes it gets lost on the masquerading madness event flow. I will probably a double check for paked calls when receiving events, polling for status and not relying on received events. So, if you want to find out why there is a ghost parked call (perhaps it is a fop2 bug too!), you must capture fop2_server debug level 1 until the problem happens, and then stop capturing and send me the capture log together with your button configuration file.

The other is that only our IAX trunk shows calls, the ZAP trunks we have constly remain green, and never show a call, despite sometimes being in use.

Are you using fop2admin? Did you setup the channel ranges in fop2 buttons? You must do it in order to see status.

Best regards,

mustardman

Hi,

[quote]What is the status of this? I'm having this problem too. It's basically a deal breaker if I can't get this working.

Is the beta stable enough? I'm not looking for half baked solutions.

There are lots of issues discussed in this post, so I am not sure what is the deal breaker for you. I am sorry if you think fop2 is a half baked solution. Asterisk has tons of nasty bugs too, and I do not think it is half baked either.

If you experience a flash client that stops updating, it is because asterisk crashes, not something else. If asterisk is stable, you will not experience the issue. If the connection with asterisk dies unexpectedly while you have flash clients connected into fop2, then you might experience the issue where some flash clients gets updates and some others do not get them. The problem is fixed if you restart fop2. If asterisk never crashes, or if you remember to restart fop2 server when you also restart asterisk, then there is no issue. Anyways, the beta fixes this, it "restarts itself" when asterisk crashes, and avoids the problem.

Now, if your deal breaker is something else, please let me know what it is so I can work on the fix.

Best regards,[/quote]

The problem is FOP2 is not updating. Has nothing to do with Asterisk which is running properly while this is happening. If I am on a call via a trunk FOP2 still shows the extension and trunk as free unless I refresh my browser. Then when the call ends I need to refresh the browser to have it update the status again. That's a deal breaker if I can't get that working properly.

But sometimes it works. So it's intermittent and not likely a configuration issue.

nicolas

Hi mustardman,

Restarting fop2_server will fix the issue. The problem happens ONLY if asterisk is restarted while fop2_server is still running AND you have a flash client connected.

Imagine this scenario, you boot the machine:

asterisk starts
fop2 starts
some fop2 client connects via web

everything is fine and dandy

suddenly asterisk crashes

fop2_server is still running
fop2 client is still connected, and it will not see updates, as asterisk is GONE

safe_asterisk relaunches asterisk

at that point, your old connected flash client will not receive updates, but newer fop2 client connections will.

restarting fop2_server fixes the issue. It is a bug in the way fop2 handles asterisk disconnections, and the problem is addressed already in the beta, and will be rolled out on the next version. But it only occurs when asterisk is restarted or dies, without restarting fop2_server.

Try it, instead of hitting F5 in your browser when you notice the problem, just restart fop2 server

service fop2 restart

and the problem will be gone.

Best regards,

mustardman

Hi mustardman,

Restarting fop2_server will fix the issue. The problem happens ONLY if asterisk is restarted while fop2_server is still running AND you have a flash client connected.

Imagine this scenario, you boot the machine:

asterisk starts
fop2 starts
some fop2 client connects via web

everything is fine and dandy

suddenly asterisk crashes

fop2_server is still running
fop2 client is still connected, and it will not see updates, as asterisk is GONE

safe_asterisk relaunches asterisk

at that point, your old connected flash client will not receive updates, but newer fop2 client connections will.

restarting fop2_server fixes the issue. It is a bug in the way fop2 handles asterisk disconnections, and the problem is addressed already in the beta, and will be rolled out on the next version. But it only occurs when asterisk is restarted or dies, without restarting fop2_server.

Try it, instead of hitting F5 in your browser when you notice the problem, just restart fop2 server

service fop2 restart

and the problem will be gone.

Best regards,

I hope you are not suggesting that requiring us to restart the service is your permanent solution. This happens quite often and your suggestion is a non-starter. You must not use FreePBX where Asterisk restarts happen every time you make a change via the GUI which happens all the time. What is the permanent solution and when can I expect it? Otherwise I may have to go back to FOP1

nicolas

Normal asterisk installs should not crash, nor asterisk restarted frequently. If you restart asterisk, you can also restart fop2. That was my suggestion.

A permanent solution could be to have a stable asterisk that does not crash. Or remember to restart fop2 when you restart asterisk. Or if your asterisk still crashes and you are fine with the safe_asterisk script bringing it up again, just add a line to the safe_asterisk script to do a service fop2 restart there too. Something like this (untested)

run_asterisk()
{
        while :; do 

                if test "x$TTY" != "x" ; then
                        cd /tmp
                        stty sane < /dev/${TTY}
                        nice -n $PRIORITY ${ASTSBINDIR}/asterisk -f ${CLIARGS} ${ASTARGS} > /dev/${TTY} 2>&1 < /dev/${TTY}
                else
                        cd /tmp
                        nice -n $PRIORITY ${ASTSBINDIR}/asterisk -f ${CLIARGS} ${ASTARGS}
                fi

                # Restart fop2 after asterisk dies or ends
                service fop2 restart

                EXITSTATUS=$?
                message "Asterisk ended with exit status $EXITSTATUS"
                if test "x$EXITSTATUS" = "x0" ; then
                        # Properly shutdown....

                        # Stop fop2 when asterisk is shutdown
                        service fop2 stop

                        message "Asterisk shutdown normally."
                        exit 0

Be creative. I tried to describe when the issue happens, so you know where to look for to address the problem. If you are unable to avoid asterisk crashes, then you might want to try updating the safe_asterisk script. However, I would not do this myself, I would rather fix asterisk or my asterisk configuration to avoid crashes or ami disconnections.

You know, if asterisk crashes, taking down all your active calls with it.... its much worst that than having some fop2 clients not updating after that crash, fop2 would be my last of concerns. I would try to get asterisk stabilized first. This only happens when you have a fop2 client connected to fop2 server, and then the AMI connection gets down or broken. It is not usual in normal circumstances. For me and for the majority of users this does not happen often or at all. It was very hard for me to reproduce the issue, fortunately I got help from users who let me look at their servers when the problem happened, and I was able to find the cause.

Anyways, the next fop2 version fixes this, as it is indeed a fop2 issue. You are welcome to try the beta if you want: http://www.fop2.com/downloads/fop2-2.21 ... 5-i386.tgz

You can also go back to FOP1 if it makes you feel more comfortable. Please excuse me, but I am still sensing a hostile attitude towards me and the software itself. It seems like you want to spread FUD more than anything else.

mattmc97

Nicolas,

Can the beta just be installed over the original?

Or what is the correct process for re-installing on a working system?

thanks.

mustardman

Normal asterisk installs should not crash, nor asterisk restarted frequently. If you restart asterisk, you can also restart fop2. That was my suggestion.

A permanent solution could be to have a stable asterisk that does not crash. Or remember to restart fop2 when you restart asterisk. Or if your asterisk still crashes and you are fine with the safe_asterisk script bringing it up again, just add a line to the safe_asterisk script to do a service fop2 restart there too. Something like this (untested)

run_asterisk()
{
        while :; do 

                if test "x$TTY" != "x" ; then
                        cd /tmp
                        stty sane < /dev/${TTY}
                        nice -n $PRIORITY ${ASTSBINDIR}/asterisk -f ${CLIARGS} ${ASTARGS} > /dev/${TTY} 2>&1 < /dev/${TTY}
                else
                        cd /tmp
                        nice -n $PRIORITY ${ASTSBINDIR}/asterisk -f ${CLIARGS} ${ASTARGS}
                fi

                # Restart fop2 after asterisk dies or ends
                service fop2 restart

                EXITSTATUS=$?
                message "Asterisk ended with exit status $EXITSTATUS"
                if test "x$EXITSTATUS" = "x0" ; then
                        # Properly shutdown....

                        # Stop fop2 when asterisk is shutdown
                        service fop2 stop

                        message "Asterisk shutdown normally."
                        exit 0

Be creative. I tried to describe when the issue happens, so you know where to look for to address the problem. If you are unable to avoid asterisk crashes, then you might want to try updating the safe_asterisk script. However, I would not do this myself, I would rather fix asterisk or my asterisk configuration to avoid crashes or ami disconnections.

You know, if asterisk crashes, taking down all your active calls with it.... its much worst that than having some fop2 clients not updating after that crash, fop2 would be my last of concerns. I would try to get asterisk stabilized first. This only happens when you have a fop2 client connected to fop2 server, and then the AMI connection gets down or broken. It is not usual in normal circumstances. For me and for the majority of users this does not happen often or at all. It was very hard for me to reproduce the issue, fortunately I got help from users who let me look at their servers when the problem happened, and I was able to find the cause.

Anyways, the next fop2 version fixes this, as it is indeed a fop2 issue. You are welcome to try the beta if you want: http://www.fop2.com/downloads/fop2-2.21 ... 5-i386.tgz

You can also go back to FOP1 if it makes you feel more comfortable. Please excuse me, but I am still sensing a hostile attitude towards me and the software itself. It seems like you want to spread FUD more than anything else.

The suggestion of putting fop2 restart in the safe_asterisk restart script is a more practical solution assuming it is initiated everytime FreePBX is updated via the Apply changes GUI button (I'm not sure if it is). Again, I don't know why you are going off on a tangent trying to blame Asterisk. As I said repeatedly asterisk is NOT crashing. Asterisk is NOT crashing. Did I mention that asterisk is NOT crashing?

You seem to have no concept of how people use this. Most use FreePBX and for those people your solution of "just restart fop2 service because it rarely happens" is ridiculous. FreePBX is constantly being used and the way FreePBX works is every change restarts asterisk which kills fop2. That does not seem to sink in with you for some reason. Also a lot of administrators like the ones I deal with do not know linux. They can use a browser and configure Asterisk via FreePBX but asking them to open a command line and restart the FOP2 service is a completely foreign concept to them. I may as well be asking them to fly to the moon.

My attitude about it is irrelevant but welcome to the world of paid apps. I have never complained about FOP1 because it's completely free and has always worked well for me. FOP2 is intentionally limited to encourage people to buy a license and my expectations are much higher for a product like that. At a minimum I do NOT expect show stoppers like this. And in this case where there is a show stopper, I don't expect useless advice about how to get around it.

Frankly, if you just said, "yea it's a problem, these things happen sometimes, I think I fixed it in the current beta which may be buggy right now" and then some sort of estimate on a stable version is a better answer.

FOP2 Community Forum

FOP2 Stops responding after awhile