openssh/rsa.c

121 lines
3.8 KiB
C
Raw Normal View History

1999-10-27 03:42:43 +00:00
/*
* Author: Tatu Ylonen <ylo@cs.hut.fi>
* Copyright (c) 1995 Tatu Ylonen <ylo@cs.hut.fi>, Espoo, Finland
* All rights reserved
*
- (djm) Merge OpenBSD changes: - markus@cvs.openbsd.org 2000/09/05 02:59:57 [session.c] print hostname (not hushlogin) - markus@cvs.openbsd.org 2000/09/05 13:18:48 [authfile.c ssh-add.c] enable ssh-add -d for DSA keys - markus@cvs.openbsd.org 2000/09/05 13:20:49 [sftp-server.c] cleanup - markus@cvs.openbsd.org 2000/09/06 03:46:41 [authfile.h] prototype - deraadt@cvs.openbsd.org 2000/09/07 14:27:56 [ALL] cleanup copyright notices on all files. I have attempted to be accurate with the details. everything is now under Tatu's licence (which I copied from his readme), and/or the core-sdi bsd-ish thing for deattack, or various openbsd developers under a 2-term bsd licence. We're not changing any rules, just being accurate. - markus@cvs.openbsd.org 2000/09/07 14:40:30 [channels.c channels.h clientloop.c serverloop.c ssh.c] cleanup window and packet sizes for ssh2 flow control; ok niels - markus@cvs.openbsd.org 2000/09/07 14:53:00 [scp.c] typo - markus@cvs.openbsd.org 2000/09/07 15:13:37 [auth-options.c auth-options.h auth-rh-rsa.c auth-rsa.c auth.c] [authfile.h canohost.c channels.h compat.c hostfile.h log.c match.h] [pty.c readconf.c] some more Copyright fixes - markus@cvs.openbsd.org 2000/09/08 03:02:51 [README.openssh2] bye bye - deraadt@cvs.openbsd.org 2000/09/11 18:38:33 [LICENCE cipher.c] a few more comments about it being ARC4 not RC4 - markus@cvs.openbsd.org 2000/09/12 14:53:11 [log-client.c log-server.c log.c ssh.1 ssh.c ssh.h sshd.8 sshd.c] multiple debug levels - markus@cvs.openbsd.org 2000/09/14 14:25:15 [clientloop.c] typo - deraadt@cvs.openbsd.org 2000/09/15 01:13:51 [ssh-agent.c] check return value for setenv(3) for failure, and deal appropriately
2000-09-16 02:29:08 +00:00
* As far as I am concerned, the code I have written for this software
* can be used freely for any purpose. Any derived versions of this
* software must be clearly marked as such, and if the derived work is
* incompatible with the protocol description in the RFC file, it must be
* called by a name other than "ssh" or "Secure Shell".
*
*
* Copyright (c) 1999 Niels Provos. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
- (djm) Merge OpenBSD changes: - markus@cvs.openbsd.org 2000/09/05 02:59:57 [session.c] print hostname (not hushlogin) - markus@cvs.openbsd.org 2000/09/05 13:18:48 [authfile.c ssh-add.c] enable ssh-add -d for DSA keys - markus@cvs.openbsd.org 2000/09/05 13:20:49 [sftp-server.c] cleanup - markus@cvs.openbsd.org 2000/09/06 03:46:41 [authfile.h] prototype - deraadt@cvs.openbsd.org 2000/09/07 14:27:56 [ALL] cleanup copyright notices on all files. I have attempted to be accurate with the details. everything is now under Tatu's licence (which I copied from his readme), and/or the core-sdi bsd-ish thing for deattack, or various openbsd developers under a 2-term bsd licence. We're not changing any rules, just being accurate. - markus@cvs.openbsd.org 2000/09/07 14:40:30 [channels.c channels.h clientloop.c serverloop.c ssh.c] cleanup window and packet sizes for ssh2 flow control; ok niels - markus@cvs.openbsd.org 2000/09/07 14:53:00 [scp.c] typo - markus@cvs.openbsd.org 2000/09/07 15:13:37 [auth-options.c auth-options.h auth-rh-rsa.c auth-rsa.c auth.c] [authfile.h canohost.c channels.h compat.c hostfile.h log.c match.h] [pty.c readconf.c] some more Copyright fixes - markus@cvs.openbsd.org 2000/09/08 03:02:51 [README.openssh2] bye bye - deraadt@cvs.openbsd.org 2000/09/11 18:38:33 [LICENCE cipher.c] a few more comments about it being ARC4 not RC4 - markus@cvs.openbsd.org 2000/09/12 14:53:11 [log-client.c log-server.c log.c ssh.1 ssh.c ssh.h sshd.8 sshd.c] multiple debug levels - markus@cvs.openbsd.org 2000/09/14 14:25:15 [clientloop.c] typo - deraadt@cvs.openbsd.org 2000/09/15 01:13:51 [ssh-agent.c] check return value for setenv(3) for failure, and deal appropriately
2000-09-16 02:29:08 +00:00
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
* IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
* IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
* NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
* DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
* THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
* THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*
*
* Description of the RSA algorithm can be found e.g. from the following
* sources:
*
* Bruce Schneier: Applied Cryptography. John Wiley & Sons, 1994.
*
* Jennifer Seberry and Josed Pieprzyk: Cryptography: An Introduction to
* Computer Security. Prentice-Hall, 1989.
*
* Man Young Rhee: Cryptography and Secure Data Communications. McGraw-Hill,
* 1994.
*
* R. Rivest, A. Shamir, and L. M. Adleman: Cryptographic Communications
* System and Method. US Patent 4,405,829, 1983.
*
* Hans Riesel: Prime Numbers and Computer Methods for Factorization.
* Birkhauser, 1994.
*
- (djm) Merge OpenBSD changes: - markus@cvs.openbsd.org 2000/09/05 02:59:57 [session.c] print hostname (not hushlogin) - markus@cvs.openbsd.org 2000/09/05 13:18:48 [authfile.c ssh-add.c] enable ssh-add -d for DSA keys - markus@cvs.openbsd.org 2000/09/05 13:20:49 [sftp-server.c] cleanup - markus@cvs.openbsd.org 2000/09/06 03:46:41 [authfile.h] prototype - deraadt@cvs.openbsd.org 2000/09/07 14:27:56 [ALL] cleanup copyright notices on all files. I have attempted to be accurate with the details. everything is now under Tatu's licence (which I copied from his readme), and/or the core-sdi bsd-ish thing for deattack, or various openbsd developers under a 2-term bsd licence. We're not changing any rules, just being accurate. - markus@cvs.openbsd.org 2000/09/07 14:40:30 [channels.c channels.h clientloop.c serverloop.c ssh.c] cleanup window and packet sizes for ssh2 flow control; ok niels - markus@cvs.openbsd.org 2000/09/07 14:53:00 [scp.c] typo - markus@cvs.openbsd.org 2000/09/07 15:13:37 [auth-options.c auth-options.h auth-rh-rsa.c auth-rsa.c auth.c] [authfile.h canohost.c channels.h compat.c hostfile.h log.c match.h] [pty.c readconf.c] some more Copyright fixes - markus@cvs.openbsd.org 2000/09/08 03:02:51 [README.openssh2] bye bye - deraadt@cvs.openbsd.org 2000/09/11 18:38:33 [LICENCE cipher.c] a few more comments about it being ARC4 not RC4 - markus@cvs.openbsd.org 2000/09/12 14:53:11 [log-client.c log-server.c log.c ssh.1 ssh.c ssh.h sshd.8 sshd.c] multiple debug levels - markus@cvs.openbsd.org 2000/09/14 14:25:15 [clientloop.c] typo - deraadt@cvs.openbsd.org 2000/09/15 01:13:51 [ssh-agent.c] check return value for setenv(3) for failure, and deal appropriately
2000-09-16 02:29:08 +00:00
* The RSA Frequently Asked Questions document by RSA Data Security,
* Inc., 1995.
*
- (djm) Merge OpenBSD changes: - markus@cvs.openbsd.org 2000/09/05 02:59:57 [session.c] print hostname (not hushlogin) - markus@cvs.openbsd.org 2000/09/05 13:18:48 [authfile.c ssh-add.c] enable ssh-add -d for DSA keys - markus@cvs.openbsd.org 2000/09/05 13:20:49 [sftp-server.c] cleanup - markus@cvs.openbsd.org 2000/09/06 03:46:41 [authfile.h] prototype - deraadt@cvs.openbsd.org 2000/09/07 14:27:56 [ALL] cleanup copyright notices on all files. I have attempted to be accurate with the details. everything is now under Tatu's licence (which I copied from his readme), and/or the core-sdi bsd-ish thing for deattack, or various openbsd developers under a 2-term bsd licence. We're not changing any rules, just being accurate. - markus@cvs.openbsd.org 2000/09/07 14:40:30 [channels.c channels.h clientloop.c serverloop.c ssh.c] cleanup window and packet sizes for ssh2 flow control; ok niels - markus@cvs.openbsd.org 2000/09/07 14:53:00 [scp.c] typo - markus@cvs.openbsd.org 2000/09/07 15:13:37 [auth-options.c auth-options.h auth-rh-rsa.c auth-rsa.c auth.c] [authfile.h canohost.c channels.h compat.c hostfile.h log.c match.h] [pty.c readconf.c] some more Copyright fixes - markus@cvs.openbsd.org 2000/09/08 03:02:51 [README.openssh2] bye bye - deraadt@cvs.openbsd.org 2000/09/11 18:38:33 [LICENCE cipher.c] a few more comments about it being ARC4 not RC4 - markus@cvs.openbsd.org 2000/09/12 14:53:11 [log-client.c log-server.c log.c ssh.1 ssh.c ssh.h sshd.8 sshd.c] multiple debug levels - markus@cvs.openbsd.org 2000/09/14 14:25:15 [clientloop.c] typo - deraadt@cvs.openbsd.org 2000/09/15 01:13:51 [ssh-agent.c] check return value for setenv(3) for failure, and deal appropriately
2000-09-16 02:29:08 +00:00
* RSA in 3 lines of perl by Adam Back <aba@atlax.ex.ac.uk>, 1995, as
* included below:
*
* [gone - had to be deleted - what a pity]
- (djm) Merge OpenBSD changes: - markus@cvs.openbsd.org 2000/09/05 02:59:57 [session.c] print hostname (not hushlogin) - markus@cvs.openbsd.org 2000/09/05 13:18:48 [authfile.c ssh-add.c] enable ssh-add -d for DSA keys - markus@cvs.openbsd.org 2000/09/05 13:20:49 [sftp-server.c] cleanup - markus@cvs.openbsd.org 2000/09/06 03:46:41 [authfile.h] prototype - deraadt@cvs.openbsd.org 2000/09/07 14:27:56 [ALL] cleanup copyright notices on all files. I have attempted to be accurate with the details. everything is now under Tatu's licence (which I copied from his readme), and/or the core-sdi bsd-ish thing for deattack, or various openbsd developers under a 2-term bsd licence. We're not changing any rules, just being accurate. - markus@cvs.openbsd.org 2000/09/07 14:40:30 [channels.c channels.h clientloop.c serverloop.c ssh.c] cleanup window and packet sizes for ssh2 flow control; ok niels - markus@cvs.openbsd.org 2000/09/07 14:53:00 [scp.c] typo - markus@cvs.openbsd.org 2000/09/07 15:13:37 [auth-options.c auth-options.h auth-rh-rsa.c auth-rsa.c auth.c] [authfile.h canohost.c channels.h compat.c hostfile.h log.c match.h] [pty.c readconf.c] some more Copyright fixes - markus@cvs.openbsd.org 2000/09/08 03:02:51 [README.openssh2] bye bye - deraadt@cvs.openbsd.org 2000/09/11 18:38:33 [LICENCE cipher.c] a few more comments about it being ARC4 not RC4 - markus@cvs.openbsd.org 2000/09/12 14:53:11 [log-client.c log-server.c log.c ssh.1 ssh.c ssh.h sshd.8 sshd.c] multiple debug levels - markus@cvs.openbsd.org 2000/09/14 14:25:15 [clientloop.c] typo - deraadt@cvs.openbsd.org 2000/09/15 01:13:51 [ssh-agent.c] check return value for setenv(3) for failure, and deal appropriately
2000-09-16 02:29:08 +00:00
*/
1999-10-27 03:42:43 +00:00
#include "includes.h"
RCSID("$OpenBSD: rsa.c,v 1.18 2000/12/19 23:17:57 markus Exp $");
1999-10-27 03:42:43 +00:00
#include "rsa.h"
#include "ssh.h"
#include "xmalloc.h"
void
rsa_public_encrypt(BIGNUM *out, BIGNUM *in, RSA *key)
1999-10-27 03:42:43 +00:00
{
u_char *inbuf, *outbuf;
int len, ilen, olen;
1999-10-27 03:42:43 +00:00
if (BN_num_bits(key->e) < 2 || !BN_is_odd(key->e))
fatal("rsa_public_encrypt() exponent too small or not odd");
1999-10-27 03:42:43 +00:00
olen = BN_num_bytes(key->n);
outbuf = xmalloc(olen);
1999-10-27 03:42:43 +00:00
ilen = BN_num_bytes(in);
inbuf = xmalloc(ilen);
BN_bn2bin(in, inbuf);
1999-10-27 03:42:43 +00:00
if ((len = RSA_public_encrypt(ilen, inbuf, outbuf, key,
- OpenBSD CVS updates to v1.2.3 [ssh.h atomicio.c] - int atomicio -> ssize_t (for alpha). ok deraadt@ [auth-rsa.c] - delay MD5 computation until client sends response, free() early, cleanup. [cipher.c] - void* -> unsigned char*, ok niels@ [hostfile.c] - remove unused variable 'len'. fix comments. - remove unused variable [log-client.c log-server.c] - rename a cpp symbol, to avoid param.h collision [packet.c] - missing xfree() - getsockname() requires initialized tolen; andy@guildsoftware.com - use getpeername() in packet_connection_is_on_socket(), fixes sshd -i; from Holger.Trapp@Informatik.TU-Chemnitz.DE [pty.c pty.h] - register cleanup for pty earlier. move code for pty-owner handling to pty.c ok provos@, dugsong@ [readconf.c] - turn off x11-fwd for the client, too. [rsa.c] - PKCS#1 padding [scp.c] - allow '.' in usernames; from jedgar@fxp.org [servconf.c] - typo: ignore_user_known_hosts int->flag; naddy@mips.rhein-neckar.de - sync with sshd_config [ssh-keygen.c] - enable ssh-keygen -l -f ~/.ssh/known_hosts, ok deraadt@ [ssh.1] - Change invalid 'CHAT' loglevel to 'VERBOSE' [ssh.c] - suppress AAAA query host when '-4' is used; from shin@nd.net.fujitsu.co.jp - turn off x11-fwd for the client, too. [sshconnect.c] - missing xfree() - retry rresvport_af(), too. from sumikawa@ebina.hitachi.co.jp. - read error vs. "Connection closed by remote host" [sshd.8] - ie. -> i.e., - do not link to a commercial page.. - sync with sshd_config [sshd.c] - no need for poll.h; from bright@wintelcom.net - log with level log() not fatal() if peer behaves badly. - don't panic if client behaves strange. ok deraadt@ - make no-port-forwarding for RSA keys deny both -L and -R style fwding - delay close() of pty until the pty has been chowned back to root - oops, fix comment, too. - missing xfree() - move XAUTHORITY to subdir. ok dugsong@. fixes debian bug #57907, too. (http://cgi.debian.org/cgi-bin/bugreport.cgi?archive=no&bug=57907) - register cleanup for pty earlier. move code for pty-owner handling to pty.c ok provos@, dugsong@ - create x11 cookie file - fix pr 1113, fclose() -> pclose(), todo: remote popen() - version 1.2.3 - Cleaned up
2000-03-09 10:27:49 +00:00
RSA_PKCS1_PADDING)) <= 0)
fatal("rsa_public_encrypt() failed");
1999-10-27 03:42:43 +00:00
BN_bin2bn(outbuf, len, out);
1999-10-27 03:42:43 +00:00
memset(outbuf, 0, olen);
memset(inbuf, 0, ilen);
xfree(outbuf);
xfree(inbuf);
1999-10-27 03:42:43 +00:00
}
void
rsa_private_decrypt(BIGNUM *out, BIGNUM *in, RSA *key)
{
u_char *inbuf, *outbuf;
int len, ilen, olen;
1999-10-27 03:42:43 +00:00
olen = BN_num_bytes(key->n);
outbuf = xmalloc(olen);
1999-10-27 03:42:43 +00:00
ilen = BN_num_bytes(in);
inbuf = xmalloc(ilen);
BN_bn2bin(in, inbuf);
1999-10-27 03:42:43 +00:00
if ((len = RSA_private_decrypt(ilen, inbuf, outbuf, key,
- OpenBSD CVS updates to v1.2.3 [ssh.h atomicio.c] - int atomicio -> ssize_t (for alpha). ok deraadt@ [auth-rsa.c] - delay MD5 computation until client sends response, free() early, cleanup. [cipher.c] - void* -> unsigned char*, ok niels@ [hostfile.c] - remove unused variable 'len'. fix comments. - remove unused variable [log-client.c log-server.c] - rename a cpp symbol, to avoid param.h collision [packet.c] - missing xfree() - getsockname() requires initialized tolen; andy@guildsoftware.com - use getpeername() in packet_connection_is_on_socket(), fixes sshd -i; from Holger.Trapp@Informatik.TU-Chemnitz.DE [pty.c pty.h] - register cleanup for pty earlier. move code for pty-owner handling to pty.c ok provos@, dugsong@ [readconf.c] - turn off x11-fwd for the client, too. [rsa.c] - PKCS#1 padding [scp.c] - allow '.' in usernames; from jedgar@fxp.org [servconf.c] - typo: ignore_user_known_hosts int->flag; naddy@mips.rhein-neckar.de - sync with sshd_config [ssh-keygen.c] - enable ssh-keygen -l -f ~/.ssh/known_hosts, ok deraadt@ [ssh.1] - Change invalid 'CHAT' loglevel to 'VERBOSE' [ssh.c] - suppress AAAA query host when '-4' is used; from shin@nd.net.fujitsu.co.jp - turn off x11-fwd for the client, too. [sshconnect.c] - missing xfree() - retry rresvport_af(), too. from sumikawa@ebina.hitachi.co.jp. - read error vs. "Connection closed by remote host" [sshd.8] - ie. -> i.e., - do not link to a commercial page.. - sync with sshd_config [sshd.c] - no need for poll.h; from bright@wintelcom.net - log with level log() not fatal() if peer behaves badly. - don't panic if client behaves strange. ok deraadt@ - make no-port-forwarding for RSA keys deny both -L and -R style fwding - delay close() of pty until the pty has been chowned back to root - oops, fix comment, too. - missing xfree() - move XAUTHORITY to subdir. ok dugsong@. fixes debian bug #57907, too. (http://cgi.debian.org/cgi-bin/bugreport.cgi?archive=no&bug=57907) - register cleanup for pty earlier. move code for pty-owner handling to pty.c ok provos@, dugsong@ - create x11 cookie file - fix pr 1113, fclose() -> pclose(), todo: remote popen() - version 1.2.3 - Cleaned up
2000-03-09 10:27:49 +00:00
RSA_PKCS1_PADDING)) <= 0)
fatal("rsa_private_decrypt() failed");
1999-10-27 03:42:43 +00:00
BN_bin2bn(outbuf, len, out);
1999-10-27 03:42:43 +00:00
memset(outbuf, 0, olen);
memset(inbuf, 0, ilen);
xfree(outbuf);
xfree(inbuf);
1999-10-27 03:42:43 +00:00
}