Using client/server encryption

This topic includes the following:

• Understanding cipher suites and protocols
• Setting up the server and client machines for encryption
• Verifying that encryption is requested

The xfServer encryption feature enables you to encrypt sensitive data as it is transferred across a network. On Unix and OpenVMS, xfServer interfaces with a third-party library, OpenSSL, to provide SSL support for secure data transport between client and server. On Windows, as of version 12, xfServer interfaces with Microsoft Secure Channel (Schannel).

To use encryption, the server and all clients must be version 9.3 or higher. To be able to specify the protocol version (-scl option), the server and all clients must be version 10.3.1b or higher.

Once encryption is enabled, all data packets to and from the server are encrypted.

Prior to version 11, xfServer offered two levels of encryption—slave and master. In version 11 and higher, only master is offered. As an alternative to slave encryption, you may want to consider having two xfServer services—one that uses encryption for secure communication and a second one for non-secure communication. Using encryption can affect performance because data must be encrypted and decrypted on both sides of the xfServer connection. You can improve performance by using Select statements to read the file on the server and select the records to send to the client; then, only the selected records need to be encrypted and decrypted, rather than the whole file. (See Synergex.SynergyDE.Select.Select for more information on the Select class.)

To implement encryption, you must start rsynd with the -encrypt option (/ENCRYPT on OpenVMS) or by selecting the Enable encryption option in the Synergy Configuration Program on Windows. You can also, optionally, specify a certificate file and the security compliance level (i.e., protocol).

See Setting up the server and client machines for encryption.  

(Windows and Unix) When rsynd starts up, if you have specified a .pem file, it is validated. If the .pem file is invalid or expired, rsynd logs an error and terminates. You can specify the -invcertOK option (available in SynConfig as the “Allow startup with invalid pem file certificate” check box) to allow rsynd to continue running with an invalid certificate; the error is still logged.

 

If you get the error “Cannot load random state,” it means there is not enough random data on the system to seed cryptographic algorithms. To correct this, define the SYNSSL_RAND environment variable on either the client or the server (depending on where the error occurred) to point to a file that can be used to gather random data.

Ensuring that specific files are encrypted

On Windows and Unix, you can further safeguard data by flagging individual files as requiring encryption with either the NETWORK_ENCRYPT option to the ISAMC routine or the -qfile=network_encrypt option to the isutl patch command (-p). Either method ensures that any client that accesses that file over a network must use encryption. (There is no prohibition against accessing the file locally.) If a file with the network encryption flag set is referenced with an OPEN statement, and encryption has not been enabled on the server, a “File requires network encryption” error ($ERR_NETCRYPT) occurs. For more information about setting the network encryption flag, see the ISAMC routine and isutl -p.

The network encryption flag applies to all remote access from a Synergy application. This means that attempting to access the file via a network path specification (e.g., a mapped drive on Windows) will generate the error $ERR_NETCRYPT. There is no prohibition to accessing the file locally.

Enforcing encryption from the client side

The /encrypt option to the OPEN statement causes the client to request data packet encryption from the server. If encryption is not enabled on the server, a “Client server error, host:xxx” ($ERR_CLNTERR) occurs. There is also an /scl runtime option for the OPEN statement that can be used to specify the desired protocol. See the OPTIONS qualifier for more information about /encrypt and /scl.

Understanding cipher suites and protocols

When enabling encryption for xfServer (or xfServerPlus), you can specify a security compliance level (-scl option), which defines the protocols to use. Each cipher suite supports one or more protocols. The cipher suite is determined by the cipher level, which is always “high.” These suites vary by operating system and the type and version of security service provider (OpenSSL or Schannel), and they often change as new, more secure ciphers are developed and older ones are found to be vulnerable and are retired. To see the specific suite and protocol being used, see Verifying that encryption is requested.

The recommended protocol is TLS 1.2 (or higher). xfServer also supports TLS 1.1 and TLS 1.3. Bear in mind that because protocol and cipher support varies by operating system, older systems may not support the newer protocols. If you have older clients connecting to xfServer, you may need to use level 2 (-scl=2), which supports TLS 1.1 and 1.2 and should work with most clients. Security compliance level 3 (-scl=3) is the current default; it supports a minimum of TLS 1.2 and may be required for compliance with certain industry security standards. Level 4 (-scl=4) restricts support to TLS 1.3, which is the highest protocol available as of this writing, but may not be available on all operating systems. If you are using a version 11 or higher xfServer with pre-version 11 clients, and you request level 4, rsynd will generate an error because pre-version 11 clients cannot support TLS 1.3; try level 3 instead.

Setting the security compliance level to zero (-scl=0) causes rsynd to always use the default security compliance level, whatever it may be. This means the protocols available may change when xfServer is upgraded. (As new threats and vulnerabilities are found, Synergex may need to update the default security compliance level to maintain a high level of security.) See the -scl option in rsynd program for the current default.

For more information about TLS protocols, see Section 3.1 in the NIST publication, “Guidelines for the Selection, Configuration, and Use of Transport Layer Security (TLS) Implementations”. It's very important that you maintain regular security updates to your OpenSSL software, the same as you do with your operating system.

You can also specify a security compliance level on the xfServer client with the /scl=level runtime option to the OPEN statement. The scl value on the client should be the same as that specified on the server because the client and server must have a cipher suite and protocol in common in order to establish secure communication. By setting an scl value on the client, you can ensure that the server that the client connects to is running at the desired security level. (See the OPTIONS qualifier for more information about using the /scl option with the OPEN statement.)

If the client is unable to negotiate a protocol with the server, the server will close the socket and log an error to the event log (Windows) or rsynd log (Unix and OpenVMS). On the client, the Synergy Runtime will generate the error “Client server error: host: name” ($ERR_CLNTERR). This error can be generated for any number of reasons; the informational text will tell you the specific problem. When the problem is an inability to negotiate a protocol, you’ll see “Error requesting secure connection: socket error: an EOF was observed that violates the protocol.”

Setting up the server and client machines for encryption

Follow these steps to set up your xfServer system to use encryption. (Note that as of version 12, Windows uses Schannel instead of OpenSSL for both the server and clients.)

1.

(Unix and OpenVMS) Install OpenSSL on your server machine. See OpenSSL requirements (Unix and OpenVMS) for details.

2.

(Unix and OpenVMS) Ensure that the OpenSSL shared libraries are in the correct location on the server or have been added to the correct path. The library path must be set before starting rsynd. On Unix, you can run the dltest utility to verify the libraries are in the correct location.

On Unix and OpenVMS, the OpenSSL libraries are installed in a standard location determined by the operating system. You don’t need to move them or set a path. If the libraries are not installed, you’ll see an error 523, $ERR_FNOTFOUND, “OpenSSL libcrypto.so not found.”

3.

Create a certificate file (.pem file). (On Windows, you can optionally use a Windows Certificate Store specification; see the -cert=filename argument in the rsynd syntax topic.) If you name this file rsynd.pem and put it in DBLDIR, it will be used by default when you start rsynd with encryption enabled. However, you may name the file anything you like and put it elsewhere if desired, and then specify it with the -cert option (/CERTIFICATE on OpenVMS).

On Windows and OpenVMS, the certificate file cannot include a passphrase. On Unix, a passphrase is permitted.

4.

(Unix) Install and configure OpenSSL on the client machines. The library path is used to find the OpenSSL libraries. If you used the setsde script to set up your Synergy environment, the path will be correctly set. You can run the dltest utility to see the shared library path and determine if Synergy can find the necessary DLLs.

5.

Start rsynd with the -encrypt option (/ENCRYPT on OpenVMS). Specify a non-default certificate filename and security compliance level if desired. See rsynd program for detailed syntax and examples.

Or, on Windows, start rsynd from the Synergy Configuration Program and select the Enable encryption option, and then specify the certificate file and security compliance level. See Modifying the SynSrv xfServer service.

Verifying that encryption is requested

To verify that a channel is encrypted, use the SLE (Socket Level Encryption) option to the GETFA routine; it returns 1 or 0.

To see the specific cipher and protocol being used, use the SLC (Socket Level Cipher) option to the GETFA routine; it returns a string identifying the cipher and protocol. For example,

AES256-GCM-SHA384    TLSv1.2 Kx=RSA     Au=RSA  Enc=AESGCM(256) Mac=AEAD

If the field argument is not large enough to hold the entire string (which could be up to 128 bytes), it will be truncated. If the channel is not encrypted, a blank string is returned. For more information on the SLE and SLC options, see the GETFA routine.

If you are using file level encryption, as described in Ensuring that specific files are encrypted, you can verify that a file is encrypted with the NETCRYPT request keyword to the %ISINFO function.