Subject: Re: [libssh2] libssh2_sftp_read* bug

Re: [libssh2] libssh2_sftp_read* bug

From: Lucas Newman <lucas_at_delicious-monster.com>
Date: Mon, 12 Feb 2007 22:02:30 -0800

I think I understand what is happening now. If a large read blocks in
the middle, the chunk of data obtained is processed in transport.c to
determine if a full packet was snagged. While processing half of a
packet, if the data is not a multiple of the crypto block size, the
extra bytes are discarded from the end of the chunk, and the remainder
of the packet is missing those bytes, hence the MAC failure.

To see this happening, add the following line to transport.c:441:
fprintf(stderr, "Bytes being discarded: %d\n", numdecrypt % session-
>remote.crypt->blocksize);

If you are able to read a whole packet at once, you will never discard
any bytes when decrypting, so that is why the behavior only appears
when doing large, blocking reads.

A solution would be to retain the extra bytes and just process them in
the next iteration.

-Lucas

On Feb 12, 2007, at 3:59 PM, Lucas Newman wrote:

> As a followup to the bug I reported yesterday, I have found some more
> issues in the packet handling with the latest code in CVS. For one,
> in the transportpacket structure, the total_num field is an integer,
> so if the packet is corrupted (this is happening with large SFTP reads
> – I'm still trying to determine why), this value can be negative.
>
> As a result, on transport.c:374, the comparison will not correctly
> detect the bad packet and libssh2 tries to allocate the memory of
> total_num, which as a negative number results in a large allocation
> that may crash. It would be best to make total_num unsigned, but if
> this isn't possible for some reason, then it should be explicitly cast
> as unsigned or checked for a negative value whenever it used.
>
> Regards,
> Lucas
>
> On Feb 11, 2007, at 11:25 PM, Lucas Newman wrote:
>
>> Hello,
>>
>> There appears to be a problem doing SFTP reads with the latest code
>> in
>> CVS. I used the samples "example/simple/sftp_nonblock.c" and
>> "example/
>> simple/sftp.c"for these results. Trying to read buffers larger
>> than a
>> kilobyte (1393 bytes seems to be the cutoff - memory smasher?) causes
>> MAC validation to fail. I double checked to make sure my buffers
>> were
>> big enough, so this seems to be a bug in libssh2. Here's the
>> relevant
>> log:
>>
>> ...
>> libssh2_sftp_open() is done, now receive data!
>> [libssh2] Connection: Setting blocking mode on channel 0/0 to 0
>> [libssh2] SFTP Subsystem: Reading 8192 bytes from SFTP handle
>> [libssh2] Connection: Setting blocking mode on channel 0/0 to 1
>> [libssh2] Connection: Writing 29 bytes on channel 0/0, stream #0
>> [libssh2] Connection: Sending 29 bytes on channel 0/0, stream_id=0
>> [libssh2] Connection: Setting blocking mode on channel 0/0 to 0
>> [libssh2] SFTP Subsystem: Waiting for packet: willnot block
>> [libssh2] Connection: Setting blocking mode on channel 0/0 to 0
>> [libssh2] Connection: Attempting to read 4 bytes from channel 0/0
>> stream #0
>> [libssh2] Connection: Setting blocking mode on channel 0/0 to 0
>> [libssh2] Connection: Setting blocking mode on channel 0/0 to 0
>> [libssh2] Failure Event: -30 - Timeout waiting for FXP packet
>> [libssh2] SFTP Subsystem: Asking for 103 packet
>> [libssh2] SFTP Subsystem: Asking for 101 packet
>> [libssh2] SFTP Subsystem: Waiting for packet: will block
>> [libssh2] Connection: Setting blocking mode on channel 0/0 to 1
>> [libssh2] Connection: Attempting to read 4 bytes from channel 0/0
>> stream #0
>> [libssh2] Connection: Setting blocking mode on channel 0/0 to 0
>> [libssh2] Connection: Setting blocking mode on channel 0/0 to 1
>> [libssh2] Transport: Packet type 94 received, length=8214
>> [libssh2] Failure Event: -4 - Invalid Message Authentication Code
>> received
>>
>> The subsystem then discards the packet and I get an error thrown
>> back. It is impossible to get a decent of transfer speed using 512 -
>> 1000 byte reads like the example. It looks like every read incurs a
>> command write and status read on the wire before the user's read, so
>> there is a ton of overhead from doing small reads.
>>
>> Any ideas?
>>
>>
>> -------------------------------------------------------------------------
>> Using Tomcat but need to do more? Need to support web services,
>> security?
>> Get stuff done quickly with pre-integrated technology to make your
>> job easier.
>> Download IBM WebSphere Application Server v.1.0.1 based on Apache
>> Geronimo
>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>> _______________________________________________
>> libssh2-devel mailing list
>> libssh2-devel_at_lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/libssh2-devel
>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services,
> security?
> Get stuff done quickly with pre-integrated technology to make your
> job easier.
> Download IBM WebSphere Application Server v.1.0.1 based on Apache
> Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> libssh2-devel mailing list
> libssh2-devel_at_lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/libssh2-devel

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier.
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
libssh2-devel mailing list
libssh2-devel_at_lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/libssh2-devel
Received on 2007-02-13