[X2go-dev] remote proxy closed the connection

Gerry Reno greno at verizon.net
Sun Feb 6 03:46:26 CET 2011


On 02/05/2011 09:45 AM, Gerry Reno wrote:
> On 02/05/2011 05:25 AM, Mike Gabriel wrote:
>> Hi Gerry,
>>
>> On Sa 05 Feb 2011 00:09:52 CET Gerry Reno wrote:
>>
>>> What's interesting is that it presents the menu and the process is shown
>>> as "running" so then your "suspend" it and "resume" it and you still get
>>> the "remote proxy closed the connection" error and NX proxy shows
>>> "aborting" but it does resume the session.  Very confusing to users but
>>> at least they get their session back.
>>
>> By what I read the issue really occurs around the port forwarding
>> request. I try to sketch the mechanism...
>>
>>
>>   o client: sets up a forwarding tunnel to the server (for nxproxy
>> traffic
>>     which in our discussed scenario)
>>   o server: thinks the tunnel is already/still set up (by the former
>> request)
>>             and denies the tunnel
>>
>>     -> here PyHoca-GUI tries to push a cancel_port_forward request to
>> the SSH
>>        daemon which sometimes works but sometimes doesn't
>>
>>   o the x2goclient does not realize the failed tunnel (PyHoca-GUI btw.
>> does and
>>     reports an error)
>>   o thus, x2goclient continues as planned
>>   o client: call of server-side x2goresume-session script (which fails
>> because
>>     of the missing port forwarding tunnel)
>>   o client: from x2goresum-session script the x2goresume script is called
>>     (which updates the database and states that the new session is
>> running
>>     although the session resuming failed)
>>
>> The problem is that old port forwarding requests on the SSH server do
>> not get properly cancelled by the clients. If a request does not get
>> canceled the port is blocked/busy and another SSH client instance
>> cannot request a port forwarding on this server port.
>>
>> Hope that sheds more light on the situation... If you can find an SSHd
>> option that could fix this behaviour this would be really great!!!
>>
>> Greetings,
>> Mike
>>
>>
>
> Right now I'm working only with the browser-client.  I'm intentionally
> crashing the browser and then trying to resume the detached session.  So
> from the client perspective it's a whole new client process that doesn't
> know anything about an existing tunnel.
>
> What I'm thinking is that when this happens the server is not receiving
> an ACK and is putting the connection into TIME_WAIT state whose interval
> is defined as 2*MSL where MSL is the packet lifetime.  Most networks
> have MSL set to 30 seconds these days.  I think it used to be 120
> seconds.  So it looks like TIME_WAIT will last from 1 to 4 minutes
> depending upon the server network.
>
> So what happens is if the user just waits long enough then they can
> resume the detached session.  If they keep trying at intervals less than
> TIME_WAIT then it seems they never get reconnected.
>
> I'm not sure what type of experiments need to be constructed to account
> for all the scenarios that can happen between the different types of
> clients and the server.  But we need to think about that.  And then see
> if we can manually construct these scenarios just using terminals and
> try to tweak the settings until we have good behavior under each
> scenario.  Then try to determine how to get the client, nxproxy and the
> server to implement those behaviors.
>
>
> Regards,
> Gerry
>
>

Been working with this problem some more today.

Here is what I observed:

I detach the client by killing the browser.
On the client:

    $ ps -ef | grep x2go
    greno    24577 24393  0 21:08 ?        00:00:00 x2goclient
    --embed-into=54525953 --config=/home/greno/.x2go/ssh/gen/x2go.cfg-eB8dER
    greno    24592     1  0 21:08 ?        00:00:00
    /home/greno/.mozilla/firefox/dmbbyyox.default/extensions/x2goplugin at obviously-nice.de/x2goclient/x2goclient
    --dialog ok --caption NX - 51 --message The remote proxy closed the
    connection while negotiating?the session. This may be due to the
    wrong authentication?credentials passed to the server.? --local
    --parent 24590 --display :0.0

On the server:

    # netstat -a | grep 300
    tcp        0      0 localhost:30005         *:*                    
    LISTEN    
    tcp        0      0 localhost:30006         *:*                    
    LISTEN    
    tcp6       0      0 ip6-localhost:30005     [::]:*                 
    LISTEN    
    tcp6       0      0 ip6-localhost:30006     [::]:*                 
    LISTEN    


Start a new browser-client session which aborts (twice on one try) with
the remote proxy closed the connection error messages.
On the client:
$ ps -ef | grep x2go

    greno    24577 24393  0 21:08 ?        00:00:01 x2goclient
    --embed-into=54525953 --config=/home/greno/.x2go/ssh/gen/x2go.cfg-eB8dER
    greno    24592     1  0 21:08 ?        00:00:00
    /home/greno/.mozilla/firefox/dmbbyyox.default/extensions/x2goplugin at obviously-nice.de/x2goclient/x2goclient
    --dialog ok --caption NX - 51 --message The remote proxy closed the
    connection while negotiating?the session. This may be due to the
    wrong authentication?credentials passed to the server.? --local
    --parent 24590 --display :0.0
    greno    24627 24577  0 21:11 ?        00:00:00 [x2goclient] <defunct>
    greno    24638 24577  0 21:11 ?        00:00:00 [x2goclient] <defunct>

On the server:

    # netstat -a | grep 300
    tcp        0      0 localhost:30005         *:*                    
    LISTEN    
    tcp        0      0 localhost:30006         *:*                    
    LISTEN    
    tcp        0      0 localhost:59060         localhost:30004        
    TIME_WAIT 
    tcp        0      0 localhost:59058         localhost:30004        
    TIME_WAIT 
    tcp6       0      0 ip6-localhost:30005     [::]:*                 
    LISTEN    
    tcp6       0      0 ip6-localhost:30006     [::]:*                 
    LISTEN    


Wait 5 minutes (ServerKeepAliveInterval)

Try to Reconnect:  first we get abort status showing and the remote
proxy closed the connection message but it is immediately followed by
resuming and running status and the session is reconnected.
On the client:

    $ ps -ef | grep x2go
    greno    24577 24393  0 21:08 ?        00:00:01 x2goclient
    --embed-into=54525953 --config=/home/greno/.x2go/ssh/gen/x2go.cfg-eB8dER
    greno    24592     1  0 21:08 ?        00:00:00
    /home/greno/.mozilla/firefox/dmbbyyox.default/extensions/x2goplugin at obviously-nice.de/x2goclient/x2goclient
    --dialog ok --caption NX - 51 --message The remote proxy closed the
    connection while negotiating?the session. This may be due to the
    wrong authentication?credentials passed to the server.? --local
    --parent 24590 --display :0.0
    greno    24627 24577  0 21:11 ?        00:00:00 [x2goclient] <defunct>
    greno    24638 24577  0 21:11 ?        00:00:00 [x2goclient] <defunct>
    greno    24672 24577  0 21:12 ?        00:00:00 [x2goclient] <defunct>

On the server:

    # netstat -a | grep 300
    tcp        0      0 localhost:30005         *:*                    
    LISTEN    
    tcp        0      0 localhost:30006         *:*                    
    LISTEN    
    tcp        0      0 localhost:34849         localhost:30006        
    ESTABLISHED
    tcp        0      0 localhost:47652         localhost:30004        
    ESTABLISHED
    tcp        0      0 localhost:30006         localhost:34849        
    ESTABLISHED
    tcp        0      0 localhost:30004         localhost:47652        
    ESTABLISHED
    tcp        0      0 localhost:47649         localhost:30004        
    TIME_WAIT 
    tcp6       0      0 ip6-localhost:30005     [::]:*                 
    LISTEN    
    tcp6       0      0 ip6-localhost:30006     [::]:*                 
    LISTEN    


So when we reconnect instead of 2 TIME_WAITS we only get one and the
other 30004 connection goes into ESTABLISHED state.

Also the client seems to be severely misbehaving in that it is sending
TWO x2goresume-session requests back-to-back.  This probably is hosing
up the database as well as the connection processing.  And that probably
accounts for the double defunct processes we see on the client as well
as the double TIME_WAIT connections we see on the server.


Regards,
Gerry











More information about the x2go-dev mailing list