On 02/05/2011 09:45 AM, Gerry Reno wrote:
On 02/05/2011 05:25 AM, Mike Gabriel wrote:
Hi Gerry,
On Sa 05 Feb 2011 00:09:52 CET Gerry Reno wrote:
What's interesting is that it presents the menu and the process is shown as "running" so then your "suspend" it and "resume" it and you still get the "remote proxy closed the connection" error and NX proxy shows "aborting" but it does resume the session. Very confusing to users but at least they get their session back.
By what I read the issue really occurs around the port forwarding request. I try to sketch the mechanism...
o client: sets up a forwarding tunnel to the server (for nxproxy traffic which in our discussed scenario) o server: thinks the tunnel is already/still set up (by the former request) and denies the tunnel
-> here PyHoca-GUI tries to push a cancel_port_forward request tothe SSH daemon which sometimes works but sometimes doesn't
o the x2goclient does not realize the failed tunnel (PyHoca-GUI btw. does and reports an error) o thus, x2goclient continues as planned o client: call of server-side x2goresume-session script (which fails because of the missing port forwarding tunnel) o client: from x2goresum-session script the x2goresume script is called (which updates the database and states that the new session is running although the session resuming failed)
The problem is that old port forwarding requests on the SSH server do not get properly cancelled by the clients. If a request does not get canceled the port is blocked/busy and another SSH client instance cannot request a port forwarding on this server port.
Hope that sheds more light on the situation... If you can find an SSHd option that could fix this behaviour this would be really great!!!
Greetings, Mike
Right now I'm working only with the browser-client. I'm intentionally crashing the browser and then trying to resume the detached session. So from the client perspective it's a whole new client process that doesn't know anything about an existing tunnel.
What I'm thinking is that when this happens the server is not receiving an ACK and is putting the connection into TIME_WAIT state whose interval is defined as 2*MSL where MSL is the packet lifetime. Most networks have MSL set to 30 seconds these days. I think it used to be 120 seconds. So it looks like TIME_WAIT will last from 1 to 4 minutes depending upon the server network.
So what happens is if the user just waits long enough then they can resume the detached session. If they keep trying at intervals less than TIME_WAIT then it seems they never get reconnected.
I'm not sure what type of experiments need to be constructed to account for all the scenarios that can happen between the different types of clients and the server. But we need to think about that. And then see if we can manually construct these scenarios just using terminals and try to tweak the settings until we have good behavior under each scenario. Then try to determine how to get the client, nxproxy and the server to implement those behaviors.
Regards, Gerry
Been working with this problem some more today.
Here is what I observed:
I detach the client by killing the browser. On the client:
$ ps -ef | grep x2go
greno 24577 24393 0 21:08 ? 00:00:00 x2goclient
--embed-into=54525953 --config=/home/greno/.x2go/ssh/gen/x2go.cfg-eB8dER
greno 24592 1 0 21:08 ? 00:00:00
/home/greno/.mozilla/firefox/dmbbyyox.default/extensions/x2goplugin@obviously-nice.de/x2goclient/x2goclient
--dialog ok --caption NX - 51 --message The remote proxy closed the
connection while negotiating?the session. This may be due to the
wrong authentication?credentials passed to the server.? --local
--parent 24590 --display :0.0On the server:
# netstat -a | grep 300
tcp 0 0 localhost:30005 *:*
LISTEN
tcp 0 0 localhost:30006 *:*
LISTEN
tcp6 0 0 ip6-localhost:30005 [::]:*
LISTEN
tcp6 0 0 ip6-localhost:30006 [::]:*
LISTEN Start a new browser-client session which aborts (twice on one try) with the remote proxy closed the connection error messages. On the client: $ ps -ef | grep x2go
greno 24577 24393 0 21:08 ? 00:00:01 x2goclient
--embed-into=54525953 --config=/home/greno/.x2go/ssh/gen/x2go.cfg-eB8dER
greno 24592 1 0 21:08 ? 00:00:00
/home/greno/.mozilla/firefox/dmbbyyox.default/extensions/x2goplugin@obviously-nice.de/x2goclient/x2goclient
--dialog ok --caption NX - 51 --message The remote proxy closed the
connection while negotiating?the session. This may be due to the
wrong authentication?credentials passed to the server.? --local
--parent 24590 --display :0.0
greno 24627 24577 0 21:11 ? 00:00:00 [x2goclient] <defunct>
greno 24638 24577 0 21:11 ? 00:00:00 [x2goclient] <defunct>On the server:
# netstat -a | grep 300
tcp 0 0 localhost:30005 *:*
LISTEN
tcp 0 0 localhost:30006 *:*
LISTEN
tcp 0 0 localhost:59060 localhost:30004
TIME_WAIT
tcp 0 0 localhost:59058 localhost:30004
TIME_WAIT
tcp6 0 0 ip6-localhost:30005 [::]:*
LISTEN
tcp6 0 0 ip6-localhost:30006 [::]:*
LISTEN Wait 5 minutes (ServerKeepAliveInterval)
Try to Reconnect: first we get abort status showing and the remote proxy closed the connection message but it is immediately followed by resuming and running status and the session is reconnected. On the client:
$ ps -ef | grep x2go
greno 24577 24393 0 21:08 ? 00:00:01 x2goclient
--embed-into=54525953 --config=/home/greno/.x2go/ssh/gen/x2go.cfg-eB8dER
greno 24592 1 0 21:08 ? 00:00:00
/home/greno/.mozilla/firefox/dmbbyyox.default/extensions/x2goplugin@obviously-nice.de/x2goclient/x2goclient
--dialog ok --caption NX - 51 --message The remote proxy closed the
connection while negotiating?the session. This may be due to the
wrong authentication?credentials passed to the server.? --local
--parent 24590 --display :0.0
greno 24627 24577 0 21:11 ? 00:00:00 [x2goclient] <defunct>
greno 24638 24577 0 21:11 ? 00:00:00 [x2goclient] <defunct>
greno 24672 24577 0 21:12 ? 00:00:00 [x2goclient] <defunct>On the server:
# netstat -a | grep 300
tcp 0 0 localhost:30005 *:*
LISTEN
tcp 0 0 localhost:30006 *:*
LISTEN
tcp 0 0 localhost:34849 localhost:30006
ESTABLISHED
tcp 0 0 localhost:47652 localhost:30004
ESTABLISHED
tcp 0 0 localhost:30006 localhost:34849
ESTABLISHED
tcp 0 0 localhost:30004 localhost:47652
ESTABLISHED
tcp 0 0 localhost:47649 localhost:30004
TIME_WAIT
tcp6 0 0 ip6-localhost:30005 [::]:*
LISTEN
tcp6 0 0 ip6-localhost:30006 [::]:*
LISTEN So when we reconnect instead of 2 TIME_WAITS we only get one and the other 30004 connection goes into ESTABLISHED state.
Also the client seems to be severely misbehaving in that it is sending TWO x2goresume-session requests back-to-back. This probably is hosing up the database as well as the connection processing. And that probably accounts for the double defunct processes we see on the client as well as the double TIME_WAIT connections we see on the server.
Regards, Gerry