Subject: [libssh2] Crash while Key Exchange at libssh2 session startup time!!

[libssh2] Crash while Key Exchange at libssh2 session startup time!!

From: Satish Mittal <>
Date: Tue, 7 Nov 2006 22:12:41 +0530

Dear All,

I am using libssh2-0.14 on windows. From my application, when i call
libssh2_session_startup(), my application is randomly (but very frequently)
crashing, reporting 'out of virtual memory'.

The problem is during the DH key exchange phase, when the call to
libssh2_packet_read() is made,

        long buf_len, payload_len; //line 876
        unsigned long packet_length;
        unsigned long padding_length;

        if (should_block) {
            buf_len = libssh2_blocking_read(session, buf, 5);
        } else {
            buf_len = recv(session->socket_fd, buf, 1,
            if (buf_len <= 0) {
                return 0;
            buf_len += libssh2_blocking_read(session, buf, 5 -
buf_len); //line
        if (buf_len < 5) {
            /* Something bad happened */
            return -1; //line 891
        packet_length = libssh2_ntohu32(buf); //line893
        padding_length = buf[4];
    _libssh2_debug(session, LIBSSH2_DBG_TRANS, "Processing plaintext packet
%lu bytes long (with %lu bytes padding)", packet_length, padding_length);

        payload_len = packet_length - padding_length - 1; /*
padding_length(1) */ //line 899
        payload = LIBSSH2_ALLOC(session, payload_len);

Here payload_len is coming out to be a huge number (junk value on stack)
since packet_length in line 893 is junk. This is in turn because in line
887, the libssh2_blocking_read() returns -1. But since buf_len is declared
as an unsigned long, so -1 gets converted to a huge 32bit value, which then
eventually crashes the application.

I have tried editing line 876 to declare buf_len and payload_len as long,
instead of unsigned long. This then in turn returns -1 in line 891 and fails
the libssh2_session_startup() call.

Here is the call stack:
libssh2.dll!libssh2_default_alloc(unsigned int count=3435973631, void * *
abstract=0x0507db28) Line 69 + 0xc C
> libssh2.dll!libssh2_packet_read(_LIBSSH2_SESSION *
session=0x0507db28, int should_block=1) Line 899 + 0x13 C
        libssh2.dll!libssh2_packet_require_ex(_LIBSSH2_SESSION *
session=0x0507db28, unsigned char packet_type='', unsigned char * *
data=0x0630ef38, unsigned long * data_len=0x0630eeec, unsigned long
match_ofs=0, const unsigned char * match_buf=0x00000000, unsigned long
match_len=0) Line 1007 + 0xb C

* session=0x0507db28, bignum_st * g=0x04402410, bignum_st * p=0x0440ffa0,
int group_order=256, unsigned char packet_type_init='', unsigned char
packet_type_reply='', unsigned char * midhash=0x00000000, unsigned long
midhash_len=0) Line 142 + 0x1b C

* session=0x0507db28) Line 604 + 0x24 C
        libssh2.dll!libssh2_kex_exchange(_LIBSSH2_SESSION *
session=0x0507db28, int reexchange=0) Line 1320 + 0xf C
        libssh2.dll!libssh2_session_startup(_LIBSSH2_SESSION *
session=0x0507db28, int socket=2868) Line 342 + 0xb C

Has anyone seen this error before, or know of any hacks/fix to this issue? I
need to fix this urgently, in order to I can proceed ahead.
Any help will be appreciated.


"The happiest of people donĀ“t necessarily have the best of everything, they
just make the most of everything that comes along their way."

Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo

libssh2-devel mailing list
Received on 2006-11-07