Subject: Re: [PATCH] Honour window_size_initial from _libssh2_channel_read

Re: [PATCH] Honour window_size_initial from _libssh2_channel_read

From: Salvador Fandino <sfandino_at_yahoo.com>
Date: Thu, 17 Oct 2013 17:00:04 +0200

On 10/17/2013 01:30 AM, Salvador Fandiņo wrote:
> On 10/16/2013 11:14 PM, Daniel Stenberg wrote:
>> On Wed, 16 Oct 2013, Salvador Fandino wrote:
>>
>>> See the commit message for the details.
>>
>> This is the first one in your series that I'm not totally convinced is
>> correct. When we do SFTP over big pipes with really large RTT, I fear
>> that having a "mere" 256KB in maximum window is a potential problem to
>> users.
>
> 256000 is also too low for my taste.
>
> But in any case, the correct way to solve that is not to hard-code some
> arbitrary size but to use a better value for
> LIBSSH2_CHANNEL_WINDOW_DEFAULT .
>
> OpenSSH uses 2MB for its window size, that would probably be a good
> field-tested default.
>
>> Have you done any performance tests with and without this patch?
>
> As stated on the patch, just over a LAN with out any noticeable speed
> degradation. I would try to write some script to simulate different
> network conditions and measure the transfer speed for a few window sizes.

I have been running some transfer test and measuring their speed.

My setup was composed of a quad-core Linux machine running Ubuntu 13.10
x86_64 with a LXC container inside. The data transfers were performed
from the container to the host (never crossing through a physical
network device).

Network delays were simulated using the tc tool. And ping was used to
verify that they worked as intended during the tests.

The operation performed was the equivalent to the following ssh command:

  $ ssh container "dd bs=16K count=8K if=/dev/zero" >/dev/null

Though, establishment and closing of the SSH connection was excluded
from the timings.

I run the tests several times transferring files of sizes up to 128MB
and the results were consistent between runs.

The results corresponding to the 128MB transfer are available here:

https://docs.google.com/spreadsheet/ccc?key=0Ao1yRmX6PQQzdG5wSFlrZl9HRWNET3ZyN0hnaGo5ZFE&usp=sharing

It clearly shows that 256KB is too small as the default window size.
Moving to a 512MB generates a great improvement and after the 1MB mark
the returns rapidly diminish. Other factors (TCP window size, probably)
become more limiting than the channel window size

For comparison I also performed the same transfers using OpenSSH. Its
speed is usually on par with that of libssh2 using a window size of 1MB
(even if it uses a 2MB window, maybe it is less aggressive sending the
window adjust msgs).

In summary, the conclusion I get from these results is that
LIBSSH2_CHANNEL_WINDOW_DEFAULT value should be increased to 1 or 2 MB.

The script used run the tests and generate the data is here:

https://github.com/salva/p5-Net-SSH-Any/blob/master/examples/window_size.pl

It requires the development versions of the Perl modules Net::SSH::Any
and Net::SSH2 available from the following URLs:

  https://github.com/salva/p5-Net-SSH-Any
  https://github.com/salva/p5-Net-SSH2

Net::SSH2 should be compiled against the git version of libssh2 with the
patch "Honour window_size_initial from _libssh2_channel_read" applied.

_______________________________________________
libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel
Received on 2013-10-17