IVR

IVR: All hotlines are down as in no announcement in IVR because of exceed in tomcat heap size.

Issue scenario: In IVR, All hotlines are down as in no announcement in IVR because of exceed in tomcat heap size.

Analysis: These announcement call flows will be in IW and will be processed by the IW interpreter which is placed inside Tomcat.

Logs:

We can check tomcat catalina logs which will be usually inside the path: $/tomcat/logs/catalina

Also, we can check in IWIVRClient log

If we see "runoutofmemory" error/exception in catalina log then it is because Tomcat ran out of heap space.

Workaround:

We will need to increase the Tomcat memory size and restart the Tomcat server.

We can increase the Tomcat heap space by specifying something like below by double-clicking a file which will be of the name like  "$TomcatNamew.exe" found in "/Tomcat/bin/"

Usually maximum limit will be 4096 MB which is 4 GB.

Initial memory pool: 512 MB

Maximum memory pool: 4096 MB

Before making the above changes (primary) we need to make sure that IVR calls are routed to secondary. This can be done in AEP (voice portal).

Once changes are done then tomcat restart is required.

Post-service restart please check Catalina logs to make sure the changes are affected.

Then we can make IVR hotline calls and verify the respective IWInterpreter logs and IWIVRClient logs.

IVR/VIVR DB credential encryption issue for plaintext containing #

Issue Description:- While using the MqApi web url (http://<IP>/mqapi/mqapi/ID/getEncryptedData?input=<plaintext>  for IVR/VIVR Database credential encryption if the plain text has "#" Hash character then the application will ignore the characters post# while encrypting and it may lead to DB connectivity error OR DB lock .

 

Resolution :- Replace the "#" in PlainText with "%23" while encrypting .

 

Ex :- for encrypting PlainText key "Adity#a" pass "Adity%23" which passing input in encryption URL

 

Note:- we can decrypt the encrypted key using below web url call for verification

http://<IP>/mqapi/mqapi/ID/getDecryptedData?input=<EncryptedText>

IVR call drop immediately after dial (Avaya routing) or keep ringing.

a) Issue description:

While dialing the IVR hotline, the call is getting dropped immediately after the engage tone.

ComponentWhat to CheckWhere to Check
EPM/ AEP (Avaya Experience Portal)/ Avaya voice portalPort DistributionLogin to EPM/ AEP/ Avaya voice portal
Select Real-time Monitoring--> Port Distribution
Select the server and click OK.Check the State column, If it is out of service.
If yes, Avaya has to check.
EPM/ AEP (Avaya Experience Portal)/ Avaya voice portalEPM managerLogin to EPM/ AEP/ Avaya voice portal
Select System Management--> EPM mangerCheck the Values of column "Mode" it will be online and column  "state" should be running.If some other value check with AVAYA
EPM/ AEP (Avaya Experience Portal)/ Avaya voice portalMPP ManagerLogin to EPM/ AEP/ Avaya voice portal
Select System Management--> MPP mangerCheck the Values of column "Mode" it will be online and column  "state" should be running.If some other value check with AVAYA
EPM/ AEP (Avaya Experience Portal)/ Avaya voice portalActive callsLogin to EPM/ AEP/ Avaya voice portal
Select  Real-time Monitoring--> Active CallsCheck whether the call entry is there for the call you made.If it is not there call is not hitting MPP.Avaya to checkAlso check the call volume, whether it has reached max. (based on the port disctribution)
EPM/ AEP (Avaya Experience Portal)/ Avaya voice portalLog ViewerLogin to EPM/ AEP/ Avaya voice portal
Select  System Maintenance--> Log ViewerSelect the date range or number of days and click onCheck for the error in "Event Message" column for errors.
EPM/ AEP (Avaya Experience Portal)/ Avaya voice portalHotline/ DNIS(Incoming VDN)Dial both Hotline and DNIS

If DNIS is working and Hotline is not working . Avaya to check at the CM level and trunk capacity

IVR Application URLTo check if it is up and runningLogin to EPM/ AEP/ Avaya voice portal
Select  System Configuration--> Applicationsclick on the IVR Application name and click on verify in front of VoiceXML URL.It should open one more browser and should give output.If it keeps loading then we should check further

 

 

EPM Manger

MPP Manger

Active call:

Log viewer

c) Resolution:

If the issue is immediate call drop after dial, then mostly issue is due to Avaya.

If the issue is call keep ringing, then IVR Service(Tomcat/Websphere) can be restarted to resolve the issue. Also for time being IVR can be put for default call flow. (Can refer to the other FAQ "How to put IVR call to default callflow)

How to activate default IVR Call flow via EPM/AEP

  1. Login to EPM / Avaya Experience portal/ AEP

2. Under system configuration, select Application

Select the Application Eg:- CBGApplication

Here, either we can keep in Single-mode or load balancer.

Currently, the application is in load balancing mode as there are 2 URLs configured.

Select the application and edit the URL to the dummy URL

Eg: https://10.5.23**.12345:8443/Interaction/Start

Once the URL is changed, Click on verify and ensure the application is not reachable and then save the changes.

When both server application is not reachable, default call flow will be triggered.

Phrases are not played in IVR even though Phrase name is printed in IVR trace logs

Issue Resolution 1:

This issue occurs when wave file format differs. Wave file format must be u-Law, 8000 Hz, 64 kbps,mono for IVR to play

Issue Resolution 2:

Wav files are not present in the location specified

ADD/Manage MPP Error handlers via Avaya EPM

The MPP Error handlers are the flow file which will be used when MPP could not able to connect to IVR/WAS servers due to certain reasons. In such cases, we can control the call via using these error handler flow files which areADD AND MANAGE THE VXML ERROR HANDLER TO THE MPP configured in the Avaya EPM portal.

This document will guide the user to add and manage error event handlers in the EPM.

AEP Port appears as ANI on TMAC Client

TMAC show the AEP port number when application transfer the call to hunt group instead of VDN.

Details on VDN, Hunt group and reason for the issue is explained in the attached document.

Check Java Version

Run java -version in command prompt

Output:

G:\General>java -version
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) Client VM (build 25.161-b12, mixed mode, sharing)

How to configure WebLM License URL in Avaya runtimeconfig

Follow the steps specified in attached document to configure the WebLM license URL in Avaya OD application.

runtimeconfig has to be accessed with web server(tomcat/weblogic/websphere) port.
Avaya OD application validates the license at the beginning of the call.
 

STEP TO CONFIGURE LICENSE URL

Create a local Certificate Signing Request using java keytool

Self sign certificate has to be created first to generate the CSR file.

Keytool command to Create a local self-signed Certificate

keytool -genkey -alias tomcat -keyalg RSA -keystore projectname.jks

Alias name and jks name can be updated based on project requirement. It is provided by customer.

This command creates the JKS file.

keytool command to Create CSR file

keytool -certreq -keyalg RSA -alias tomcat -file certreq.csr -keystore projectname.jks

It creates the CSR based on the details available in JKS.

CSR has to be shared with customer to sign the certificate.

Below document contains more details with screen shot:

Steps to create CSR

Import certificate into keystore using java keytool

Use keytool command to import the root, intermediate and server certificate in java keystore JKS. Keystore has the certificates and keys.

keytool -import -trustcacerts -file certfilename -alias aliasname -keystore  keystorename

Example:

keytool -import -trustcacerts -file Root-CA.crt -alias root -keystore tomcat.jks 

It prompts for password and provide the password used while creating the key store.

Keystore can be JKS/P12(PKCS12)/other formats.

JKS is java keystore

P12(PKCS12) is language-neutral and it can be used by Java,.Net applications.

How to check Tomcat version

Run the following commands from your <application-install> or <tomcat-install> directory:

If you do not have your Java Home directory included in your PATH environment variable, you will need to include the path to your <java-home>/bin directory in front of the java command.

Windows:

java.exe -cp lib\catalina.jar org.apache.catalina.util.ServerInfo

The output should be similar to this:

Server version: SpringSource tc Runtime/2.0.4.RELEASE
Server built: August 3 2010 0710
Server number: 6.0.28.29
OS Name: Linux
OS Version: 2.6.18-194.11.1.el5
Architecture: i386
JVM Version: 1.6.0_21-b06
JVM Vendor: Sun Microsystems Inc.