Hi George,

1st, please use either "diff -c" or "git diff" to prepare the patch. You can even use "git diff" on a files/folders that are outisde of a git source tree.

2nd, I tried what I think was your patch. I've attached the change I used, it is outputted with "git diff". I used the x2goclient master branch (4.0.3.0.)

Unfortunately, it failed to fix the bug. I did not experience any regressions though.

I tested it on both a Windows XP 32-bit SP3 machine and a Windows 8.1 64-bit machine (with the latest required updates & optional updates from MS.)

3rd, the username I used was "δοκιμαστικό χρήστη". That is how Google Translate translated "test user" into Greek. Because it is not easy for me to type that name into the Windows login prompt, I simply ran commands like the following to test it: runas "/user:δοκιμαστικό χρήστη" "c:\x2gobuilds\x2goclient\dist\x2goclient.exe"

-Mike

On Mon, Sep 1, 2014 at 7:23 PM, Michael DePaulo <mikedep333@gmail.com> wrote:

On Mon, Sep 1, 2014 at 3:51 PM, Mike Gabriel <mike.gabriel@das-netzwerkteam.de> wrote:

...

Hi Mike#2,

On Mo 01 Sep 2014 19:28:50 CEST, George Trakatelis wrote:

...

A suggested patch is to substitute the Qt4 toAscii() function with toLocal8Bit() in two specific places of SshMasterConnection::run(). A git diff is as follows:

$ diff /c/a/sshmasterconnection.cpp /c/b/sshmasterconnection.cpp 521c521 < ssh_options_set ( my_ssh_session, SSH_OPTIONS_SSH_DIR, (mainWnd->getHomeDirectory()+"/ssh").toAscii());

...

ssh_options_set ( my_ssh_session, SSH_OPTIONS_SSH_DIR,

(mainWnd->getHomeDirectory()+"/ssh").toLocal8Bit());

630c630 < ssh_options_set ( my_ssh_session, SSH_OPTIONS_SSH_DIR, (mainWnd->getHomeDirectory()+"/ssh").toAscii());

...

ssh_options_set ( my_ssh_session, SSH_OPTIONS_SSH_DIR,

(mainWnd->getHomeDirectory()+"/ssh").toLocal8Bit());

The change affects only Windows builds and was tested in Windows 8.1 Pro and Windows 7 Professional.

G. Trakatelis

Can you review these changes and if appropriate commit them to the master branch of x2goclient.git?

Thanks! Mike

[...]

Sure. I'm busy today, but I'll do so within the next few/several days.

-Mike

On Sun, Sep 7, 2014 at 6:04 AM, George Trakatelis <trakatelis@uom.edu.gr> wrote: [...]

a. Installed German keyboard. Hit ';' to produce an accented letter and -to my surprise- got ö. So I thought ölexandr was the right name to test for username.

b. Created user ölexandr and logged in as that user. The patch did not work, as you mentioned.

c. As toLocal8Bit() returns the local 8-bit representation of a string, I changed the system locale for non-unicode programs to German. Now the patch worked!

I think the patch solves the problem for the Windows users who use English as a foreign language and prefer having (non-Ascii) usernames in their native language.

-George

PS. A better title for the bug would be

X2Go Client for Windows 4.0.2.1 cannot create C:\Users\<username>\ssh\known_hosts file when the local Windows account username has non-Ascii characters

Hi Mike#1 and George,

1. What do you think of using toUtf8() instead? I am hoping that will fix the bug regardless of the locale.

I tried changing it to that (bug566.utf8.test.patch), but it still did not fix this bug with the greek username on my system with the locale set to English. It did not introduce a regression for my ASCII user account at least.

If libssh needs to be recompiled for Unicode, I can do that. I just recompiled it for bug #590.

FYI: This is the API we are calling: http://api.libssh.org/stable/group__libssh__session.html#ga7a801b85800baa3f4...

2. in sshmasterconnection.cpp, we also set "SSH_OPTIONS_USER" . I think that we should also change its value from to.Ascii() to toLocal8Bit() or toUtf8(). bug566.test.v2.patch has this change for toLocal8Bit(), and bug566.utf8.test.patch has it for toUtf8().

-Mike#2

Yeah. I think that libssh isn't being compiled with Unicode support, or at least not under MingW 4.4. It's not 100% clear how to compile a C library with Unicode support under MinGW and CMake. For example, recent versions of MinGW let you specify the linker option -municode. But I cannot find a single release note that states when exactly that feature was added. I do not know if 4.4 has it or not.

I'm going to try to compile it with MinGW 4.8. I've been meaning to upgrade to X2Go to MinGW 4.8 and QT 4.8.6 anyway. (bug: #474)

On Sun, Sep 7, 2014 at 8:13 PM, George Trakatelis <trakatelis@uom.edu.gr> wrote:

Hi Mike,

1. If you recompile libssh 0.6.3 with Unicode support and provide me with a download link, I will happily test it on all Windows platforms (namely Windows 8, 7, and XP).

2. I agree.

-George

-----Original Message----- From: Michael DePaulo [mailto:mikedep333@gmail.com] Sent: Sunday, September 7, 2014 4:23 PM To: George Trakatelis; 566@bugs.x2go.org; Mike Gabriel Subject: Re: [X2Go-Dev] Bug#566: X2Go Client for Windows 4.0.2.1 cannot create C:\Users\<username>\ssh\known_hosts file when the local Windows account username has non-Ascii characters

[...]

Hi Mike#1 and George,

1. What do you think of using toUtf8() instead? I am hoping that will fix the bug regardless of the locale.

I tried changing it to that (bug566.utf8.test.patch), but it still did not fix this bug with the greek username on my system with the locale set to English. It did not introduce a regression for my ASCII user account at least.

If libssh needs to be recompiled for Unicode, I can do that. I just recompiled it for bug #590.

FYI: This is the API we are calling: http://api.libssh.org/stable/group__libssh__session.html#ga7a801b85800baa3f4...

2. in sshmasterconnection.cpp, we also set "SSH_OPTIONS_USER" . I think that we should also change its value from to.Ascii() to toLocal8Bit() or toUtf8(). bug566.test.v2.patch has this change for toLocal8Bit(), and bug566.utf8.test.patch has it for toUtf8().

-Mike#2

On Mon, Sep 8, 2014 at 4:58 AM, Mike Gabriel <mike.gabriel@das-netzwerkteam.de> wrote:

Hi Mike#2,

On So 07 Sep 2014 15:23:22 CEST, Michael DePaulo wrote:

...

On Sun, Sep 7, 2014 at 6:04 AM, George Trakatelis <trakatelis@uom.edu.gr> wrote: [...]

...

a. Installed German keyboard. Hit ';' to produce an accented letter and -to my surprise- got ö. So I thought ölexandr was the right name to test for username.

b. Created user ölexandr and logged in as that user. The patch did not work, as you mentioned.

c. As toLocal8Bit() returns the local 8-bit representation of a string, I changed the system locale for non-unicode programs to German. Now the patch worked!

I think the patch solves the problem for the Windows users who use English as a foreign language and prefer having (non-Ascii) usernames in their native language.

-George

PS. A better title for the bug would be

X2Go Client for Windows 4.0.2.1 cannot create C:\Users\<username>\ssh\known_hosts file when the local Windows account username has non-Ascii characters

Hi Mike#1 and George,

1. What do you think of using toUtf8() instead? I am hoping that will fix the bug regardless of the locale.

I tried changing it to that (bug566.utf8.test.patch), but it still did not fix this bug with the greek username on my system with the locale set to English. It did not introduce a regression for my ASCII user account at least.

If libssh needs to be recompiled for Unicode, I can do that. I just recompiled it for bug #590.

FYI: This is the API we are calling:

http://api.libssh.org/stable/group__libssh__session.html#ga7a801b85800baa3f4...

I am getting the suspicion, that this is a libssh issue.

X2Go Client and libssh should communicate via UTF-8 (because it is _the_ encoding that applications use internally, nowadays) and libssh should handle whatever conversion (probably to UTF-16 wide string?) is necessary for creating files, being aware of usernames, etc.

...

2. in sshmasterconnection.cpp, we also set "SSH_OPTIONS_USER" . I think that we should also change its value from to.Ascii() to toLocal8Bit() or toUtf8(). bug566.test.v2.patch has this change for toLocal8Bit(), and bug566.utf8.test.patch has it for toUtf8().

Same thing. SSH_OPTIONS_USER should be set as UTF-8 and libssh should convert to whatever encoding is needed.

Greets, Mike [...]

Hi Mike#1 and George,

I believe the best approach is to try to compile libssh with unicode support, and pass unicode values to it. There is no guarantee that a user's username (and home dir path) is in the same language as the language that is set for non-Unicode programs. After all, the "Language for non-Unicode programs" is called the "system locale", it applies to all users on the system.

If we have to for the 4.0.3.0 release of x2goclient, we should fix this bug for the system locale only, and then fix the bug for all languages/locales later.

See bug 474 for George's patch for compiling libssh with MinGW 4.8.2. It worked :)

I made some progress compiling libssh 0.6.3 with Unicode support.

With MinGW 4.4, when I specified the following values in CMAKE, libssh built successfully, but it did not seem to change anything. CMAKE_EXE_LINKER_FLAGS:STRING=-municode CMAKE_MODULE_LINKER_FLAGS:STRING=-municode CMAKE_SHARED_LINKER_FLAGS:STRING=-municode CMAKE_STATIC_LINKER_FLAGS:STRING=-municode

With MinGW 4.8.2, when I specify those values, libssh fails to build. I've attached the output.

This is a positive sign because it implies that MinGW 4.8.2 supports -municode, whereas MinGW 4.4 did not.

I plan to try to resolve this build failure later tonight.

-Mike#2

Control: severity 566 grave

Dear George,

Thank you for your research.

Over the next day or so, I will apply bug566.test.v2.patch, which will then be included in both the MinGW 4.4 and the MinGW 4.8 nightly builds. Once you (or someone else) confirms that the patch fixes this bug for said installations, I will mark this bug as fixed. I've already cloned this bug. The clone will be for handling languages other than the "Language for non-Unicode programs"; i.e., using Unicode.

I am marking this original bug as severity "grave" because it "makes the package in question unusable or mostly so"[1] for a significant number of users. Getting this original bug fixed for x2goclient 4.0.3.0 will be a high priority for me.

Also, I tried to shorten the title of this bug (and correct a typo: cannon -> cannot) but I ran into a bug with the debbugs software. I'm trying to correct the title.

-Mike

[1] http://bugs.x2go.org/Developer.html#severities

On Mon, Sep 15, 2014 at 1:26 AM, George Trakatelis <trakatelis@uom.edu.gr> wrote:

Dear Mike,

I have built libssh 0.6.3 in Visual Studio 2013 (with UNICODE and _UNICODE defined in all projects) and debugged a tiny test app with function calls analogous to those used in x2goclient. I'm afraid preprocessor definitions are not enough to make an application/program/library support Unicode. One could migrate existing programs following a list of guidelines

http://msdn.microsoft.com/en-us/library/cc194801.aspx

For example, a function that fails in libssh (used in ssh_path_expand_tilde(), which is always called to replace a ~ with the actual home folder path in a string) is strdup. To make libssh unicode-friendly strdup has to be replaced by _strdup

http://msdn.microsoft.com/en-us/library/y471khhc.aspx

_strdup would benefit from preprocessor definitions for Unicode support as it would be converted to the proper Unicode function.

Imho, the only option for now is patch "bug566.test.v2.patch" as it corrects the problem in the vast majority of Windows installations. Remember that foreign Windows installations set by default the "Language for non-Unicode programs" to that particular foreign language.

-George

-----Original Message----- From: Michael DePaulo [mailto:mikedep333@gmail.com] Sent: Tuesday, September 9, 2014 4:16 PM To: 566@bugs.x2go.org Cc: George Trakatelis; Mike Gabriel Subject: Re: [X2Go-Dev] Bug#566: X2Go Client for Windows 4.0.2.1 cannot create C:\Users\<username>\ssh\known_hosts file when the local Windows account username has non-Ascii characters

[...]

Hi Mike#1 and George,

I believe the best approach is to try to compile libssh with unicode support, and pass unicode values to it. There is no guarantee that a user's username (and home dir path) is in the same language as the language that is set for non-Unicode programs. After all, the "Language for non-Unicode programs" is called the "system locale", it applies to all users on the system.

If we have to for the 4.0.3.0 release of x2goclient, we should fix this bug for the system locale only, and then fix the bug for all languages/locales later.

See bug 474 for George's patch for compiling libssh with MinGW 4.8.2. It worked :)

I made some progress compiling libssh 0.6.3 with Unicode support.

With MinGW 4.4, when I specified the following values in CMAKE, libssh built successfully, but it did not seem to change anything. CMAKE_EXE_LINKER_FLAGS:STRING=-municode CMAKE_MODULE_LINKER_FLAGS:STRING=-municode CMAKE_SHARED_LINKER_FLAGS:STRING=-municode CMAKE_STATIC_LINKER_FLAGS:STRING=-municode

With MinGW 4.8.2, when I specify those values, libssh fails to build. I've attached the output.

This is a positive sign because it implies that MinGW 4.8.2 supports -municode, whereas MinGW 4.4 did not.

I plan to try to resolve this build failure later tonight.

-Mike#2

Hi George,

See inline responses.

On Tue, Sep 16, 2014 at 1:50 AM, Γεώργιος Τρακατέλης <trakatelis@uom.edu.gr> wrote:

Dear Mike,

Two things.

1. Please make a preview build of X2Go Client for Windows with the patch applied and inform me when you do so. I'd rather test a complete preview package of X2Go Client 4.0.3.0 for Windows.

Done: 4.0.3.0-pre01. The debug build is available too, and I manually updated the shortcuts in the debug build.[1] http://code.x2go.org/releases/binary-win32/x2goclient/previews/4.0.3.0/ I will now announce this on the mailing list.

2. Before writing this reply I had a recollection of initially having bumped onto a similar bug that was dismissed due to the fact that the submitter's system was not POSIX compliant. I searched for the bug but I couldn't find it.

Then I decided to give bug #566 a try on Linux, but Ubuntu 14.04 does not allow non-Ascii characters in user names. After that, I did a more thorough search and finally found the aforementioned bug. It is bug #397! I'm wondering why that bug was filed as an x2goserver bug instead of x2goclient.

Bug #397 refers to when the user's home folder on the server has non-Ascii characters. That is why the bug is filed against x2goserver.

Normally I would say that we should clone this bug (566) for X2Go Client for Linux. However, the bug report is already extremely long, I think we should just file a new one and link to this one. Could you please file one?

The good news: I managed to change my (linux test) home folder to /home/néstor using usermod (changing to /home/Γιώργος is impossible (Γιώργος=George in Greek)). Findings: x2goclient managed to create known_hosts but the nxproxy module failed "*** longjmp causes uninitialized stack frame ***: /user/lib/nx/bin/nxproxy terminated"

In bug #397, Néstor gives another reason for failure "Error: The remote NX proxy closed the connection." which implies x2goserver rejected the connection, so I stopped there. Could you possibly test it? (If you cannot type an é, just copy-paste néstor)

-George

1st of all, you were talking about a home dir with non-Ascii characters on the client machine, correct? If so, that test is not applicable to #397 for the aforementioned reason.

In order to properly test the fix for #397 on Ubuntu, we would have to build a nightly build of x2goserver 4.0.1.16 or 4.1.0.0 from source. It is not fixed in 4.0.1.15.

(AFAIK, there are no automated nightly x2goserver builds for Ubuntu. Only for Debian, Fedora and EPEL.[2])

Besides, our "fix" for #397 is not to make homedirs with non-Ascii characters "work". Our "fix" is to present a better error message and cleanly abort the session.

-Mike#2

[1] http://wiki.x2go.org/doku.php/wiki:development:build-howto-mswin:x2goclient#... [2] http://jenkins.x2go.org:8080/view/Server/

On Tue, Sep 16, 2014 at 12:09 PM, George Trakatelis <trakatelis@uom.edu.gr> wrote:

Hi Mike,

First of all, patch bug566.test.v2.patch definitely solves the problem as described in bug #566. I have tried successfully both the release and the debug builds you uploaded both in Windows 8.1 Pro and Windows 7 Professional SP1 with all the Microsoft updates installed.

Great :) I'll update debian/changelog. That update will mark this bug as pending for release.

Now, with regard to bug #397:

...

1st of all, you were talking about a home dir with non-Ascii characters on the client machine, correct?

Yes.

...

Bug #397 refers to when the user's home folder on the server has non-Ascii characters. That is why the bug is filed against x2goserver. Besides, our "fix" for #397 is not to make homedirs with non-Ascii characters "work". Our "fix" is to present a better error message and cleanly abort the session.

I see.

...

Normally I would say that we should clone this bug (566) for X2Go Client for Linux. However, the bug report is already extremely long, I think we should just file a new one and link to this one. Could you please file one?

I was only testing an analogous to bug #566 situation in Linux. To be honest, I had to get round Ubuntu's policy for POSIX user names by using usermod. That test revealed that nxproxy (in Ubuntu 14.04) crashes when path names with non-Ascii characters are used, but is it really a situation that deserves to be filed as a bug?

I mean it's common for Windows users to have user names in their native language. On the other hand, Ubuntu (and I suppose Unix in general) dictates exactly what characters are permitted in a user name. If one by-passes this rule, erroneous program behavior may be observed. Can we call it a bug of the program? I don’t know. You decide!

-George

The way I see it, it is an "issue", not a "bug". You could also call it a "limitation".

Still, we should show the user a better error message when a home dir with non-english characters is detected on Linux. So please report a bug about it.

-Mike#2