Subject: RE: libssh2_sftp_write doesn't hang anymore

RE: libssh2_sftp_write doesn't hang anymore

From: Mark Riordan <mriordan_at_ipswitch.com>
Date: Thu, 11 Nov 2010 12:33:26 -0600

> Since we're now down to "just" being slow, you can try to add more
> output logs again to see if they reveal anything.

> - We need to figure out where the EAGAINs come from and make sure
> libssh2 acts correctly on them, but it a bit hard when the logging
> slows things down this much. Possibly you should make your example
> use the libssh2_trace_sethandler() function...

Last time, as I mentioned, uploads were quite slow regardless
of whether logging was turned on.
Now uploads are faster - 20 times faster - when logging
is turned on than when logging is disabled!
For that reason, I didn't bother trying to speed
up the logging. With the OS buffering writes to disk, I don't
think it would have made much of a difference anyway.

Without logging turned on, the process uses about 100% of
the CPU even though through is about 0.03 MB/sec
(compared with about 6 MB/sec expected throughput).

With logging turned on, throughput is about 0.7 MB/sec.

I have a feeling that a lot of retransmitting is still going on.
mitm-ssh reports a massive amount of retransmitting, but I think
it's somehow making things worse. The above timings, and below
log files, were directly to the SSH server.

I did use Wireshark on one of the uploads. I processed the
Wireshark dump with a script I wrote, and it indicated that
30517591 bytes were sent for the upload of a 120000-byte file.
Granted, there is some overhead to SSH, but not that much
overhead.

I have a libssh2 trace and corresponding Wireshark capture:
DebugFast2010-11-11.zip

From a run without tracing turned on (and therefore a run
which counter-intuitively was very slow) I have this much
larger Wireshark capture, presumably showing a lot of
retransmissions (though encrypted, of course):
NoDebugSlow2010-11-11.zip

I'm running out of space on my personal website, so rather than
put them on my website, I will get these files to Daniel
via some other way.

Mark

_______________________________________________
libssh2-devel http://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel
Received on 2010-11-11