Hi,
I'm forced to work with some extreme bandwith restrictions and was looking into using more compression and/or the Modem / ISDN connection setting. (I'm down to old time modem speeds, 128 kbit or less) So far I could not find any documentation that shows what these really do. I normally use the ADSL one, which works. The Modem and ISDN setting seems broken (and has been for as long as I rememeber), they don't really reduce the bandwidth usage nor improve latency. When switching to Modem, the screen updates actually get much worse, I can watch the system update the screen in lines (not the whole screen is updated in one go), which looks like a bug. I have had success setting the image quality to a lower setting, the screen looks worse, which is a good indicator that it is working. I can't find any documentation on what the difference is between different compression methods. Can you point me in the right direction? If I can find the info, I would be up for updating the documentation so that others can find it better.
Thanks
Markus
-- Por sperto kaj lerno ne sufiĉas eterno.
Dear Nigra,
On Fri, 26 Apr 2019 18:48:34 -0700, Nigra Truo <nigratruo@gmail.com> wrote:
I'm forced to work with some extreme bandwith restrictions and was looking into using more compression and/or the Modem / ISDN connection setting. (I'm down to old time modem speeds, 128 kbit or less) So far I could not find any documentation that shows what these really do. I normally use the ADSL one, which works. The Modem and ISDN setting seems broken (and has been for as long as I rememeber), they don't really reduce the bandwidth usage nor improve latency. When switching to Modem, the screen updates actually get much worse, I can watch the system update the screen in lines (not the whole screen is updated in one go), which looks like a bug. I have had success setting the image quality to a lower setting, the screen looks worse, which is a good indicator that it is working. I can't find any documentation on what the difference is between different compression methods. Can you point me in the right direction? If I can find the info, I would be up for updating the documentation so that others can find it better.
I asked pretty much the same question here in 2017. The only response I got was from another user who, like you and me, was only able to make a few observations by trial and error testing: <http://lists.x2go.org/pipermail/x2go-user/2017-February/004254.html>
I agree that it would be nice to get a comprehensive overview of the different settings.
Regards, Tristan
--
Tristan MillerFree Software developer, ferret herder, logologist https://logological.org/
On Mon, Apr 29, 2019 at 3:42 PM Tristan Miller <psychonaut@nothingisreal.com> wrote:
Dear Nigra,
On Fri, 26 Apr 2019 18:48:34 -0700, Nigra Truo <nigratruo@gmail.com> wrote:
I'm forced to work with some extreme bandwith restrictions and was looking into using more compression and/or the Modem / ISDN connection setting. (I'm down to old time modem speeds, 128 kbit or less) So far I could not find any documentation that shows what these really do. I normally use the ADSL one, which works. The Modem and ISDN setting seems broken (and has been for as long as I rememeber), they don't really reduce the bandwidth usage nor improve latency. When switching to Modem, the screen updates actually get much worse, I can watch the system update the screen in lines (not the whole screen is updated in one go), which looks like a bug. I have had success setting the image quality to a lower setting, the screen looks worse, which is a good indicator that it is working. I can't find any documentation on what the difference is between different compression methods. Can you point me in the right direction? If I can find the info, I would be up for updating the documentation so that others can find it better.
The code sets tile size and defer parameters depending on the link speed, but you can also set these parameters on your own, see man nxagent:
defer=<int>
defer image updates (enabled for all connection typesexcept LAN), accepts values 0, 1 and 2
The default value can be set via the command line(-defer). The value provided as nx/nx option is set when resuming a session, thus it overrides the command line default.
tile=<string>
set the tile size in pixels (<W>x<H>) for bitmap datasent over the wire
The default value can be set via the command line(-tile). The value provided as nx/nx option is set when resuming a session, thus it overrides the command line default.
You can add the desired valued to the X2GO_NXOPTIONS= line in /etc/x2go/x2goagent.options, e.g. "menu=1,defer=2,tile=128x128"
- Each defer level adds the following rules to the previous ones:
- Level 0 Eager encoding.
- Level 1 No data is put or copied on pixmaps, marking them always
as corrupted and synchronizing them on demand, i.e. whena copy area to a window is requested, the source is syn-chronized before copying it.- Level 2 The put images over the windows are skipped marking the
destination as corrupted. The same happens for copy areaand composite operations, spreading the corrupted regionsof involved drawables.
Here are the defaults as found in the code:
case LINK_TYPE_MODEM:
deferLevel = 2;
tileWidth = 64;
tileHeight = 64;
case LINK_TYPE_ISDN:
deferLevel = 2;
tileWidth = 64;
tileHeight = 64;
case LINK_TYPE_ADSL:
deferLevel = 2;
tileWidth = 4096;
tileHeight = 4096;
case LINK_TYPE_WAN:
deferLevel = 1;
tileWidth = 4096;
tileHeight = 4096;
case LINK_TYPE_NONE:
case LINK_TYPE_LAN:
deferLevel = 0;
tileWidth = 4096;
tileHeight = 4096;
default:
deferLevel = 0;
tileWidth = 64;
tileHeight = 64;To my surprise I found another parameter today which is currently undocumented: -irlimit <kB/s>. This will limit the amount of data that is used to update drawables on the real display. It can only be passed via command line. To play with it add it to /etc/x2go/x2goagent.options.
I asked pretty much the same question here in 2017. The only response I got was from another user who, like you and me, was only able to make a few observations by trial and error testing: <http://lists.x2go.org/pipermail/x2go-user/2017-February/004254.html>
I agree that it would be nice to get a comprehensive overview of the different settings.
Well, I have done some research tonight:
First it is worth noting that what we are talking about here is the way NX will compress images travelling the wire. NX also (delta) compresses the message format that X11 uses. This is unaffected by the pack method setting. So if you only have an xterm handling some text you probalby do not see much difference after changing the pack method because there are not many images involved. Also note that NX uses a cache so image compression will help in low bandwidth situations, but maybe less than expected. That's because once the image has travelled the wire it will be cached on the other side and not travel the wire again.
x2goclient has three command line options that will influence the compression:
--link Set default link type (modem,isdn,adsl,wan or lan, default: adsl).
--pack Set default pack method (default: '16m-jpeg').
--quality
Set default image quality(0-9, default: 9).These can also be set in the Connection tab of the chosen session.
The values you use for these parameter are passed down to the nx software. Some of the parameters there are described in "man nxproxy" as follows:
link=<string>
An indication of the link speed that is going to beused between the proxies. Usually the compression and the other link parameters depend on this setting. The value can be either 'modem', 'isdn', 'adsl', 'wan', 'lan', 'local' or a bandwidth specification, like for example '56k', '1m', '100m', etc. data=<int> Enable or disable the ZLIB data compression. It is possible to specify a value between 0 and 9. Usually the value is chosen automatically based on the requested link setting. stream=<int> Enable or disable the ZLIB stream compression. The value, between 0 and 9, is usually determined according to the requested link setting. Not fully implemented in nx-X11 Agent, yet. pack=<string> Determine the method used to compress images.
Now, the pack and quality settings define the compression algorithm and the quality of the compressed images. In general there are several groups of compression methods. Most of them will use a user-provided number of colors from 8 to 16m. Generic form: <number-of-colors>[-<method>[-<quality>]] parameter: "<number of colors>", e.g "16m"
- none (no image compression, lossless)
- masked (reduce number of bits per pixel, lossy): Format of the pack
- jpeg (jpeg compression, lossy): <number of colors>-jpeg, e.g. "16m-jpeg"
- png (png compression, lossy): <number of colors>-png, e.g. "4k-png"
- rgb (zlib compression with Z_DEFAULT_STRATEGY, lossless) : "rgb" or "16ḿ-rgb" /(16m is the only number of colors supported)
- Run Length Encoding (zlib compression with Z_RLE strategy, lossless): "rle" or "16m-rle" (16m is the only number of colors supported)
- bitmap (remove each 4th byte, lossless): "bitmap" or "16m-bitmap" (16m is the only number of colors supported)
For the lossy compressors the string can also have a "-<quality>" suffix. This suffix will define the pack quality. The default is "9".
There are some hig level compression modes, too, that will map to one from the list above:
- "lossy" will map to "16m-jpeg"
- "lossless" will map to "16m-rle-9" for link=lan and "16m-bitmap" for all other link types.
- "adaptive" will choose lossy or lossless, depending on the characteristics of the image
If no pack method is provided it will be set automatically from the link parameter:
- modem: adaptive-3
- isdn: adaptive-5
- adsl: adaptive-7
- wan: adaptive-9
- lan: adaptive-9
Images, that have a width or height of 2 or less pixels or are not bigger than 512 bytes will not be compressed (but they are still cached).
Hope this helps,
Uli
"Ulrich" == Ulrich Sibiller <uli42@gmx.de> writes:
Ulrich> On Mon, Apr 29, 2019 at 3:42 PM Tristan Miller Ulrich> <psychonaut@nothingisreal.com> wrote:
Dear Nigra,
On Fri, 26 Apr 2019 18:48:34 -0700, Nigra Truo <nigratruo@gmail.com> wrote:
I'm forced to work with some extreme bandwith restrictions and was looking into using more compression and/or the Modem / ISDN connection setting. (I'm down to old time modem speeds, 128 kbit or less) So far I could not find any documentation that shows what these really do. I normally use the ADSL one, which works. The Modem and ISDN setting seems broken (and has been for as long as I rememeber), they don't really reduce the bandwidth usage nor improve latency. When switching to Modem, the screen updates actually get much worse, I can watch the system update the screen in lines (not the whole screen is updated in one go), which looks like a bug. I have had success setting the image quality to a lower setting, the screen looks worse, which is a good indicator that it is working. I can't find any documentation on what the difference is between different compression methods. Can you point me in the right direction? If I can find the info, I would be up for updating the documentation so that others can find it better.
Ulrich> The code sets tile size and defer parameters depending on the link Ulrich> speed, but you can also set these parameters on your own, see man Ulrich> nxagent:
Ulrich> defer=<int> Ulrich> defer image updates (enabled for all connection types Ulrich> except LAN), accepts values 0, 1 and 2
Ulrich> The default value can be set via the command line Ulrich> (-defer). The value provided as nx/nx option is set when resuming a Ulrich> session, thus it overrides the command line default.
Ulrich> tile=<string> Ulrich> set the tile size in pixels (<W>x<H>) for bitmap data Ulrich> sent over the wire
Ulrich> The default value can be set via the command line Ulrich> (-tile). The value provided as nx/nx option is set when resuming a Ulrich> session, thus it overrides the command line default.
Ulrich> You can add the desired valued to the X2GO_NXOPTIONS= line in Ulrich> /etc/x2go/x2goagent.options, e.g. "menu=1,defer=2,tile=128x128"
Ulrich> * Each defer level adds the following rules to the previous ones: Ulrich> * Ulrich> * Level 0 Eager encoding. Ulrich> * Ulrich> * Level 1 No data is put or copied on pixmaps, marking them always Ulrich> * as corrupted and synchronizing them on demand, i.e. when Ulrich> * a copy area to a window is requested, the source is syn- Ulrich> * chronized before copying it. Ulrich> * Ulrich> * Level 2 The put images over the windows are skipped marking the Ulrich> * destination as corrupted. The same happens for copy area Ulrich> * and composite operations, spreading the corrupted regions Ulrich> * of involved drawables.
Ulrich> Here are the defaults as found in the code:
Ulrich> case LINK_TYPE_MODEM: Ulrich> deferLevel = 2; Ulrich> tileWidth = 64; Ulrich> tileHeight = 64; Ulrich> case LINK_TYPE_ISDN: Ulrich> deferLevel = 2; Ulrich> tileWidth = 64; Ulrich> tileHeight = 64; Ulrich> case LINK_TYPE_ADSL: Ulrich> deferLevel = 2; Ulrich> tileWidth = 4096; Ulrich> tileHeight = 4096; Ulrich> case LINK_TYPE_WAN: Ulrich> deferLevel = 1; Ulrich> tileWidth = 4096; Ulrich> tileHeight = 4096; Ulrich> case LINK_TYPE_NONE: Ulrich> case LINK_TYPE_LAN: Ulrich> deferLevel = 0; Ulrich> tileWidth = 4096; Ulrich> tileHeight = 4096; Ulrich> default: Ulrich> deferLevel = 0; Ulrich> tileWidth = 64; Ulrich> tileHeight = 64;
Ulrich> To my surprise I found another parameter today which is currently Ulrich> undocumented: -irlimit <kB/s>. This will limit the amount of data that Ulrich> is used to update drawables on the real display. It can only be passed Ulrich> via command line. To play with it add it to Ulrich> /etc/x2go/x2goagent.options.
I asked pretty much the same question here in 2017. The only response I got was from another user who, like you and me, was only able to make a few observations by trial and error testing: <http://lists.x2go.org/pipermail/x2go-user/2017-February/004254.html>
I agree that it would be nice to get a comprehensive overview of the different settings.
Ulrich> Well, I have done some research tonight:
Ulrich> First it is worth noting that what we are talking about here is the Ulrich> way NX will compress images travelling the wire. NX also (delta) Ulrich> compresses the message format that X11 uses. This is unaffected by the Ulrich> pack method setting. So if you only have an xterm handling some text Ulrich> you probalby do not see much difference after changing the pack method Ulrich> because there are not many images involved. Also note that NX uses a Ulrich> cache so image compression will help in low bandwidth situations, but Ulrich> maybe less than expected. That's because once the image has travelled Ulrich> the wire it will be cached on the other side and not travel the wire Ulrich> again.
Ulrich> x2goclient has three command line options that will influence the compression:
Ulrich> --link Set default link type (modem,isdn,adsl,wan or lan, default: adsl).
Ulrich> --pack Set default pack method (default: '16m-jpeg').
Ulrich> --quality Ulrich> Set default image quality(0-9, default: 9).
Ulrich> These can also be set in the Connection tab of the chosen session.
Ulrich> The values you use for these parameter are passed down to the nx Ulrich> software. Some of the parameters there are described in "man nxproxy" Ulrich> as follows:
Ulrich> link=<string> Ulrich> An indication of the link speed that is going to be Ulrich> used between the proxies. Usually the compression and the other link Ulrich> parameters depend on this setting. The value can be either 'modem', Ulrich> 'isdn', 'adsl', 'wan', 'lan', 'local' or a bandwidth Ulrich> specification, like for example '56k', '1m', '100m', etc. Ulrich> data=<int> Ulrich> Enable or disable the ZLIB data compression. It is Ulrich> possible to specify a value between 0 and 9. Usually the value is Ulrich> chosen automatically based on the requested link setting. Ulrich> stream=<int> Ulrich> Enable or disable the ZLIB stream compression. The Ulrich> value, between 0 and 9, is usually determined according to the Ulrich> requested link setting. Not fully implemented in nx-X11 Agent, yet. Ulrich> pack=<string> Ulrich> Determine the method used to compress images.
Ulrich> Now, the pack and quality settings define the compression algorithm Ulrich> and the quality of the compressed images. In general there are several Ulrich> groups of compression methods. Most of them will use a user-provided Ulrich> number of colors from 8 to 16m. Generic form: Ulrich> <number-of-colors>[-<method>[-<quality>]] Ulrich> - none (no image compression, lossless) Ulrich> - masked (reduce number of bits per pixel, lossy): Format of the pack Ulrich> parameter: "<number of colors>", e.g "16m" Ulrich> - jpeg (jpeg compression, lossy): <number of colors>-jpeg, e.g. "16m-jpeg" Ulrich> - png (png compression, lossy): <number of colors>-png, e.g. "4k-png" Ulrich> - rgb (zlib compression with Z_DEFAULT_STRATEGY, lossless) : "rgb" or Ulrich> "16ḿ-rgb" /(16m is the only number of colors supported) Ulrich> - Run Length Encoding (zlib compression with Z_RLE strategy, Ulrich> lossless): "rle" or "16m-rle" (16m is the only number of colors Ulrich> supported) Ulrich> - bitmap (remove each 4th byte, lossless): "bitmap" or "16m-bitmap" Ulrich> (16m is the only number of colors supported)
Ulrich> For the lossy compressors the string can also have a "-<quality>" Ulrich> suffix. This suffix will define the pack quality. The default is "9".
Ulrich> There are some hig level compression modes, too, that will map to one Ulrich> from the list above: Ulrich> - "lossy" will map to "16m-jpeg" Ulrich> - "lossless" will map to "16m-rle-9" for link=lan and "16m-bitmap" for Ulrich> all other link types. Ulrich> - "adaptive" will choose lossy or lossless, depending on the Ulrich> characteristics of the image
Ulrich> If no pack method is provided it will be set automatically from the Ulrich> link parameter: Ulrich> - modem: adaptive-3 Ulrich> - isdn: adaptive-5 Ulrich> - adsl: adaptive-7 Ulrich> - wan: adaptive-9 Ulrich> - lan: adaptive-9
Ulrich> Images, that have a width or height of 2 or less pixels or are not Ulrich> bigger than 512 bytes will not be compressed (but they are still Ulrich> cached).
Ulrich> Hope this helps,
This is a huge help. And of course it probably helps alot more to size down your remote screen if at all possible. I currently run a 1920x1133 desktop and ... it's not ideal over a WAN link between offices with approximately 100ms of delay. I've tried various flavors, but it was never clear what would work best. Now I have some better ideas to try.
Right now, I can watch my screen re-draw stuff (xterms are fine, web browsing is terrible) when I do scrolling in firefox. I'm using CentOS 7.6 for my desktop under XFCE, with the remote display hosted on a Fedora 28 desktop. I know the problem is more with Gnome and X11 and the lack of NX developement these days, not the x2go developers. It's just annoying that remote desktops are still stuck so badly.
Maybe I just need to use x2go for specific applications, not my entire desktop, except when I access from home I have to bounce across the country both ways, which just sucks sucks sucks. :-)
Thanks, John
-
John Stoffel -
Nigra Truo -
Tristan Miller -
Ulrich Sibiller