Using xfServerPlus encryption

This topic includes the following:

The xfServerPlus encryption feature enables you to encrypt sensitive data as it is transferred between an xfNetLink client and xfServerPlus. xfServerPlus interfaces with a third-party library (OpenSSL on Unix and OpenVMS; Schannel on Windows) to provide SSL support for secure data transport.

For Synergy and .NET clients, both xfServerPlus and xfNetLink must be version 9.3 or higher; for Java, the minimum client version is 9.5.1a. For all clients, to be able to specify the protocols to use (-scl option), both client and server must be version 10.3.1b or higher.

Once encryption is enabled, parameter and return value data for all methods is encrypted.

Tip

Prior to version 11, xfServerPlus offered two levels of encryption—slave and master. As of version 11, only master is offered, as slave was inherently not secure. As an alternative to slave encryption, you may want to consider having two xfServerPlus services—one that uses encryption for secure communication and a second one for non-secure communication.

Using encryption can affect performance because data must be encrypted and decrypted on both sides of the xfServerPlus—xfNetLink connection. The cipher negotiation between client and server also takes time. You will get better performance if your client makes a connection, performs all the necessary method calls, and then disconnects. If you connect, make a call, and disconnect, repeating this process for each call, performance may be degraded because the cipher negotiation has to be performed on each call.

If you are using a Java or .NET client, using pooling can further improve performance, because the connections (and, consequently, the cipher negotiations) are made when the pool is first populated, rather than with each call.

To implement encryption, you must start rsynd with the -encrypt option (/ENCRYPT on OpenVMS) or by selecting the Enable encryption option in the Synergy Configuration Program on Windows. You also specify a certificate file and the security compliance level (i.e., protocol). (As of Synergy/DE 11, the only available cipher level is “high,” so you do not need to specify it.) See Setting up the xfServerPlus machine for encryption for details. For a general description of cipher suites and protocols, see Understanding cipher suites and protocols.

When encryption is enabled, you’ll see an entry in the xfServerPlus log if XFPL_SESS_INFO or XFPL_FUNC_INFO is set to ALL. When encryption is enabled, a string of 10 asterisks will display in the log file in place of the parameter, return value, and packet data for encrypted methods. Consequently, you may want to enable encryption only after you have thoroughly tested your application and have no need to examine packet data.

Note

If you get the error message “Cannot load random state”, it means there is not enough random data on the system to seed cryptographic algorithms. To correct this, define the SYNSSL_RAND environment variable on either the server or the client (depending on where the error occurred) to point to a file that can be used to gather random data.

Setting up the xfServerPlus machine for encryption

1. (Unix and OpenVMS) Install the OpenSSL libraries.

For details on which version of OpenSSL is required for your operating system and where to get it, see OpenSSL requirements (Unix and OpenVMS).

2. (Unix and OpenVMS) Ensure the OpenSSL shared libraries are in the correct location or have been added to the correct path.

The library path must be set before starting rsynd. On Unix, you can run the dltest utility to verify the libraries are in the correct location.

On both Unix and OpenVMS, the OpenSSL libraries are installed in a standard location determined by the operating system. You don’t need to move them or set a path. If the libraries are not installed, you’ll see an error 523, $ERR_FNOTFOUND, “OpenSSL libcrypto.so not found.”

3. Create a certificate (.pem) file.

If you name this file rsynd.pem and put it in the DBLDIR directory, it will be used by default when you start rsynd with encryption enabled. However, you may name the file anything you like and put it elsewhere if desired, and then specify it with the -cert option (/CERTIFICATE on OpenVMS) or with the Synergy Configuration Program on Windows.

On Windows and OpenVMS, the certificate file cannot include a pass phrase. On Unix, a pass phrase is permitted.

On all operating systems, the file should contain both a CERTIFICATE section and an RSA PRIVATE KEY section.

Note

See Requesting a certificate from a certificate authority for instructions on creating a certificate request file and sending it to a public certificate authority (CA). The CA will then send back a certificate file. For testing purposes, a local CA with self-signed certificates may suffice. See Creating a local certificate authority for steps on how to create a local file.

 

Important

If your client is xfNetLink .NET, when creating the .pem file, the commonName (specified in the configuration file or when prompted) must match the xfServerPlus host name as it is passed in the connect() method. If it does not, you’ll get the error “The remote certificate is invalid according to the validation procedure” at runtime. See Synergex KnowledgeBase article 2375 for additional information.

4. Start rsynd with encryption enabled.

Start rsynd with the -encrypt option (/ENCRYPT on OpenVMS). Specify the certificate filename and security compliance level if desired. See rsynd program for detailed syntax and examples.

Or, on Windows, start rsynd from the Synergy Configuration Program and select the Enable encryption option, and then specify the certificate filename and security compliance level. For more information see Starting xfServerPlus from the Synergy Configuration Program.

Setting up the xfNetLink Synergy machine for encryption

Tip

From the xfNetLink Synergy client you can check the encryption setting on the xfServerPlus machine programmatically. The secure_status argument to %RX_START_REMOTE and %RX_DEBUG_START returns 0 (no encryption) or 2 (encryption).

Setting up the xfNetLink Java machine for encryption

Java encryption does not use the .pem file directly. Rather, it requires that the certificates be placed in a keystore file. A default keystore file, cacerts, is distributed with the JRE. You can use the genCert utility (see below) to add additional certificates to this file.

The genCert utility creates a copy of the distributed cacerts file and adds the certificates from the .pem file on the xfServerPlus machine to it. We recommend you use this new file and call it something other than “cacerts” because the cacerts file is replaced whenever Java is upgraded.

Once xfServerPlus has been started with encryption enabled, do the following on the xfNetLink Java machine:

1. Run the genCert utility to create the certificate file. See below for complete syntax. For example, 
java genCert -h myServer -p 3535
2. Set the xf_SSLCertFile and xf_SSLPassword settings in the xfNetLink Java properties file to specify the filename and password for the certificate file created in step 1. Alternatively, you may call the setSSLCertFile() and setSSLPassword() methods to specify this information at runtime. To override the default security compliance level, call the setSSLSecCompliance() method. (There is no properties file setting for security compliance level.) See Specifying encryption options and the Method reference.

genCert utility

The genCert utility, which is distributed in the xfNLJava directory, assists you in setting up encryption on your xfNetLink Java machine. xfServerPlus must be running with encryption enabled before you can run genCert. 

java genCert [-c filename] [-s password] [-n filename] -h host [-p port] [-?]

Arguments

-c filename

(optional) The name and path of the base Java certificate file. The default is java.home\lib\security\cacerts, where java.home is the JRE installation directory.

-s password

(optional) The password for the base certificate file. If not passed, the default is “changeit”. (Note that this is the password for the cacerts file distributed with the JRE; your system administrator may have changed it.)

-n filename

(optional) The name and path of the new certificate file to put the generated certificates in. If not passed, the default is newcerts, and the file is placed in the current working directory.

-h host

The name of the xfServerPlus machine to connect to. xfServerPlus must be running with encryption enabled.

-p port

(optional) The port that xfServerPlus is running on. If not passed, defaults to 2356.

-?

Display usage message.

Usage and examples

When you run the genCert utility, you will see output similar to that shown below. The utility outputs the parameters it is using, followed by the certificates found in the .pem file on the xfServerPlus machine. Notice that the certificates are numbered. At the end, you’ll be prompted to enter the number of the certificate that you want added to the keystore file (the default is 1), and the file will be created. The new certificate file has the same password as the original file. Type q at the prompt to quit without creating the new certificate file.

The example below uses the default cacerts file and password. It specifies a name for the new certificate file, and the xfServerPlus machine and port to connect to. 

            java genCert -n myCertFile -h myServer -p 3535 

Checking for valid certificate in base certificate file.
Base certificate file = c:\java\jdk6_12\jre\lib\security\cacerts
Base certificate password = changeit
New certificate file = myCertFile
xfServerPlus Host = myServer
xfServerPlus Port = 3535

Loading KeyStore c:\java\jdk6_12\jre\lib\security\cacerts...
Server sent 2 certificate(s):

1 Subject EMAILADDRESS= user@synergex.com, CN=myServer, OU=SDE, O=Synergex, 
L=Sacramento, ST=California, C=CA
Issuer EMAILADDRESS= user@synergex.com, CN=myServer, OU=SDE, O=Synergex, 
L=Sacramento, ST=California, C=US
sha1 c4 a3 f8 63 85 84 b6 cb b1 63 65 9e ca e5 bf 34 8f 5f e8
md5 26 87 6c 34 77 ac 8d e9 39 27 99 f1 0a fc 97 5f

2 Subject EMAILADDRESS= user@synergex.com, CN=myServer, OU=SDE, O=Synergex, 
L=Sacramento, ST=California, C=US
Issuer EMAILADDRESS= user@synergex.com, CN=myServer, OU=SDE, O=Synergex, 
L=Sacramento, ST=California, C=US
sha1 07 d2 b4 e7 d7 7a 9f 09 44 88 3c 50 1c 20 5f 90 0d cf dd
md5 55 1d f6 13 3b 6c cd e3 7c a6 60 75 93 7f e4 a8

Enter certificate to add to trusted keystore or 'q' to quit: [1]
1
Certificate added

Setting up the xfNetLink .NET machine for encryption

Note

For xfNetLink version 11.1.1f and later, you do not need to do anything to set up the xfNetLink .NET machine for encryption.

For version 11.1.1e and lower, you need to perform the steps below.

The necessary software is already installed, but you must ensure that SSL is enabled and that the desired cipher suites have been specified.

1. Run gpedit.msc from the Run dialog. This launches the Local Group Policy Editor.
2. In the left panel, navigate to Computer Configuration > Administrative Templates > Network > SSL Configuration Settings.
3. In the right panel, double-click on SSL Cipher Suite Order to display the SSL Cipher Suite Order dialog.
4. Select Enabled and click the Apply button.

You will see the enabled suites listed below in the SSL Cipher Suites field in the Options pane. When you run xfServerPlus with encryption enabled, one of these suites will be selected based on the specified level of encryption (high, medium, low). You can see which suite is selected in the xfNetLink .NET log file and the protocol used; see Using client-side logging.

Tip

Although only a few characters display, the field contains a great number of cipher suites by default. To see them all, copy the field contents and paste into a text editor. We strongly recommend you read Synergex KnowledgeBase article 2202 before attempting to edit this field.

5. When you are done, click OK in the SSL Cipher Suite Order dialog and then close the Local Group Policy Editor.

Ensuring that specific methods are encrypted

Although xfServerPlus encryption encrypts all data going to and from the server, you may want to flag some methods that pass sensitive data as requiring encryption. Doing so is not a requirement, but it can serve as a fail-safe: if the method is called when encryption is not enabled on the server, there will be an error.

xfNetLink Synergy

There are two ways to mark specific methods for encryption when you are using xfNetLink Synergy. 

xcall rxsubr(netid, "mymethodid/encrypt", arg1, arg2)

If you use the /encrypt switch and encryption is not enabled on the xfServerPlus machine, the error $ERR_XFSERVNOSEC, “Encryption not enabled on server,” will be returned.

xfNetLink Java and xfNetLink .NET

To mark specific methods for encryption when you are using xfNetLink Java or xfNetLink .NET, specify the encrypt=true property in the xfMethod attribute (or select the “Enable encryption” check box when defining the method in the MDU), and then generate classes and build the JAR file or assembly.

On Java, the method setEncryptedMethod(true) is added to each method that is marked for encryption in the SMC.

On .NET, the attribute xfAttr, with the member encrypt=true, is added to each method that is marked for encryption in the SMC. You can specify the security compliance level (protocol) to use with the scl parameter to the connect() method.

If a method is marked for encryption in the SMC and encryption is not enabled on the xfServerPlus machine, the error “Method requires encryption” is generated.