Lotus Notes Authentication Buffer Overflow

Description

Lotus Notes and Domino servers support a proprietary protocol called NotesRPC, commonly known as the Notes protocol. This protocol is usually bound to TCP port 1352, but can also use NetBIOS, Netware SPX, Banyan Vines, and modem dialup for transport.

When a Notes client connects to a Notes server, it authenticates with the server to establish a session. This authentication consists of a series of exchanges in which the client and server present each other with challenges to verify their identity.

It is possible for an unauthenticated client to manipulate the data during this exchange to trigger a buffer overflow on the Notes server. This allows an attacker to overwrite large sections of the heap with arbitrary data. While our testing only covered TCP/IP, we believe it is possible for an attacker to trigger this overflow via other protocols, including dialup. It is theoretically possible for an attacker to supply the data in such a way as to compromise the Notes server's security.