Subject: SFTP read buffer size

SFTP read buffer size

From: Tamar Sery (BLOOMBERG/ 120 PARK) <tsery_at_bloomberg.net>
Date: Wed, 26 Aug 2020 19:51:43 -0000

Hi,
I'm trying to increase the download speed of files using libssh2_sftp_read .
I was able to see considerable speed increase by using a larger buffer, up to 30000.
However, any size over 30000 seem to have no affect on actual received packet size, all packets are exactly 30000 (except for the last chunk of the file, and an occasional 60000 sized packet).
Is there a way to increase the download speed, using a larger buffer or otherwise?

Code snippet:

while (true)
{
            char mem[79990];
            //loop until we fail
            int rc = libssh2_sftp_read(sftpHandle_, mem, sizeof(mem));
            std::cout << "Got data size="<<rc<<std::endl;

            if(rc > 0) {
                fileStream_.write(mem, rc);
            }
            else {
                break;
            }
}

Example result output:
2020-08-25 22:00:07.198 [T4]:Got data size=30000
2020-08-25 22:00:07.406 [T4]:Got data size=30000
2020-08-25 22:00:07.407 [T4]:Got data size=30000
2020-08-25 22:00:07.409 [T4]:Got data size=30000
2020-08-25 22:00:07.613 [T4]:Got data size=30000
2020-08-25 22:00:07.613 [T4]:Got data size=30000
2020-08-25 22:00:07.613 [T4]:Got data size=30000
2020-08-25 22:00:07.614 [T4]:Got data size=30000
2020-08-25 22:00:07.615 [T4]:Got data size=60000
2020-08-25 22:00:07.615 [T4]:Got data size=30000
2020-08-25 22:00:07.616 [T4]:Got data size=30000
2020-08-25 22:00:07.820 [T4]:Got data size=30000
2020-08-25 22:00:07.821 [T4]:Got data size=30000
2020-08-25 22:00:07.821 [T4]:Got data size=30000
.......
2020-08-25 22:00:10.529 [T4]:Got data size=30000
2020-08-25 22:00:10.730 [T4]:Got data size=30000
2020-08-25 22:00:10.730 [T4]:Got data size=31633
2020-08-25 22:00:10.936 [T4]:Got data size=0

libssh2 version: 1.7.0

_______________________________________________
libssh2-devel https://cool.haxx.se/cgi-bin/mailman/listinfo/libssh2-devel
Received on 2020-08-26