While configuring an API link from POM CTI server config and application trying to access an external API but fails with the following message ,
"HttpWebRequest The underlying connection was closed: An unexpected error occurred on a send"
Step 1: Try to simulate the issue in the lab environment. The error was not able to be simulated in a lab environment.
Step 2: Try to establish the connection to API via postman - successful
Try to call the API link from the browser - successful
Step 3: Create a custom application to trigger the API call from other servers and observe the behavior.
Simulate the UAT behavior from SIT servers.
Copy the same custom application for API call into 4 SIT servers
Server1: API call fails with error
Server2: API call is successful
Server3: API call fails with error
Server4: API call is successful
Step 4: Do a continuous netstat on the POM WAS server(where the API is deployed) while making a call to API.
The connection to the port does not establish, we see the request in "TIME_WAIT" mode.
Step 5:Do a trace on the server level with Wireshark application
From Server where API fails: no server hello seen
From Server where API is successful: Server hello seen
Possible Root Cause:
The client-server handshake is failing. https://www.thesslstore.com/blog/explaining-ssl-handshake/
SCHANNEL setting on Registry for ciphers:
Since there is a difference between 2 SIT servers, a possible guess could be due to server hardening.
So the server hardening script was run on [Server2: API call is successful]
After the server was hardened the API call failed on Server2
While comparing the cipher and SChannel keys from a hardened UAT server that can make a successful API call to the same POM WAS server, the difference was seen with the Diffie-Hellman key [enabled=0].
Adding the key to the client servers where the API call failed resolved the issue.
The difference in the key exchange algorithm between the server and client seems to have an impact. The dependency also will be the cipher order used in the server(POM WAS).