Troubleshooting tools

Trace session functionality is an efficient tool that allows user sessions to be traced and simplifies troubleshooting for administrators.

The key features of trace session functionality are:

• A single page for session search and display;
• Merged SIP and billing logs to provide an entire overview of the session flow;
• General information about a session in the form of a diagram between the main participants (e.g., SIP phones and PortaSwitch) with the issue description and troubleshooting instructions for the most common issues; and
• The ability to switch to the advanced log view if deeper investigation is required (e.g., to review the call flow among nodes within the PortaSwitch cluster).

For instance, on a call with an administrator, John Doe (12103355787) wants to find out why he cannot call 18660333777. The administrator opens the trace session page and searches the session using the account number and time interval. On the log viewer page, he sees that John Doe is not allowed to call the destination because it is forbidden.

For backward compatibility, an administrator can access billing logs on the old web interface by clicking the Old UI icon. This feature drastically simplifies troubleshooting and reduces the load on your administrative staff. Please refer to the Troubleshooting section of the Unified PortaSwitch handbook collection for more troubleshooting tips.

Administrators can trace the events that have affected a customer’s balance, status, and invoice calculation using the Audit log in PortaBilling. The records have extended descriptions, e.g., “Unallocated payments changed from 10.00 to 70.00 (increased by 60.00). Payments source – from customer payment transaction (xDR id 2204).” This helps administrators find the root of the issue faster when dealing with inquiries from customers.

Let’s say, a customer John Doe receives an invoice for September with an amount of $100 due. John calls the customer service manager to dispute the September invoice. During the call, the customer service manager agrees to reduce the amount due by $30. In two weeks, John contacts the customer service manager again to clarify why he received a notification about an overdue invoice even though he paid in full.

To investigate the issue, the administrator opens Customer > Audit log and filters the records made in October. He finds the record that the invoice was adjusted by $30 and the balance after adjustment was $70. Also, he sees that a $60 payment was made, so the amount due is now $10. No payments were received after that and the invoice status was changed from “Partially paid” to “Overdue”. The administrator decides to check whether the invoice was adjusted correctly. He goes to the CRM system and finds a note about the disputed invoice, confirming that $30 is the adjustment amount that was actually agreed upon with the customer. The administrator calls back the customer to explain that his payment didn’t cover the adjusted invoice amount, so John still needs to pay $10 to cover the invoice in full.

By default, audit logs are stored for 180 days in the Elasticsearch storage. You can change the default storage period on the Configuration server.

1. Open ClusterSuite > Web Cluster > Global environment > Tasks group.
2. In the Log_Max_Age option, define a new period in days. The minimum number of days is 30.
3. Click Save > Verify > Check/Apply.

Benefits

• Administrators can troubleshoot invoice-related issues faster via the PortaBilling web interface.

Logs of periodic tasks such as invoice calculation, taxation, periodic payments, reports are available via the Grafana web interface. Grafana is an open-source platform for data visualization, monitoring, and analysis. With the integration with Grafana, service providers’ engineers can access the information from text log files, potentially located across several servers, via one web interface.

Grafana is a single entry-point for displaying the periodic task logs (taskstack.log/task_queue.log files) that are collected from all servers and stored in Elasticsearch in JSON format.

Currently, the integration with Grafana allows using a simple dashboard to cover the most common troubleshooting requests: filtering logs, e.g., by customer or environment, reviewing them, and downloading logs in .txt format to submit them to support.

Also, service providers can create their own dashboards, e.g., graphs demonstrating the performance of specific tasks.

Grafana must be deployed on premises. To configure Grafana on your PortaSwitch, contact PortaOne support.

Benefits

• Service providers’ engineers can troubleshoot issues related to periodic tasks faster.

The External System Provisioning Framework (ESPF) captures changes in customer/account configuration in PortaBilling and sends provisioning events to an external application (for example, an IPTV platform). After the external application updates the entity’s configuration, it returns the event provisioning status to PortaBilling via the ESPF API or replies with HTTP Status Codes (e.g., “404 Not Found”, or “500 Internal Server Error.”) An administrator can see the provisioning status of events (e.g., all, failed, queued, successful) and check event logs directly in the PortaBilling web interface for these entities:

• Account
• Customer
• Access policy
• Account Add-on Product
• Account commitment
• Connection
• CPE
• CPE profile
• DID number
• Node
• Product
• SIM card

Thus, if the provisioning to an external application fails, an administrator can check the web logs directly on the web interface, instead of accessing logs through a command line interface. This saves time and helps solve the issues faster. After solving the issues, the administrator can re-start the provisioning in one click, right from the web interface.

This is how it works:

Let’s say a service provider needs to provision the information about customers and accounts from PortaBilling to an external application (for example, the IPTV platform). There is an account, John Doe, in PortaBilling. John Doe is assigned a product – Standard TV. The administrator changes John Doe’s product from Standard TV to Premium TV in PortaBilling. These changes are recorded as a provisioning event and sent by the ESPF to the IPTV platform. The IPTV platform receives the provisioning event that contains John Doe’s unique account ID and the information that the product has changed. The IPTV platform then attempts to change John Doe’s product from Standard TV to Premium TV. The product change fails. The external application sends the failed provisioning status to PortaBilling.

The administrator checks the logs on the External provisioning panel on the PortaBilling web interface and sees that provisioning failed because incorrect credentials were used in the API request. The administrator reports the issue to their colleague, who sets the correct credentials on the IPTV platform side. Then the administrator clicks the Re-run button on the PortaBilling web interface to re-provision the event.

As a result, the IPTV platform successfully changes John Doe’s product from Standard TV to Premium TV and sends the successful provisioning status to PortaBilling.

Let’s say a customer service representative (CSR) uses the service provider’s CRM system, which then communicates with PortaBilling via API. The service provider has a new customer, Jerry Price.

The information about this customer needs to be provisioned to PortaBilling and to an external application (for example, the IPTV platform).

To do this, the CSR creates a new customer record for Jerry Price in the CRM system, which provisions this information to PortaBilling. The customer creation is recorded in PortaBilling as a provisioning event and is then sent by the ESPF to the IPTV platform. The IPTV platform fails to create a customer record because it ran out of disc space so it replies with a “500 Internal Server Error” message to PortaBilling.

The CRM system requests the customer provisioning status from PortaBilling via API using the GET/espf/v1/logs method and displays this status on the web interface. The CSR sees that the event provisioning failed and asks their colleague to check the logs on the IPTV platform side. The logs shows that the provisioning failed due to an issue with the disc space. Once the disc space is freed up, the CSR clicks the button on the CRM system’s web interface to re-provision the customer record via the API.

As a result, the customer record for Jerry Price is successfully created in the IPTV platform.

Find the detailed description on how to display the status of a provisioning event for ESPF in the “How to” chapter of the PortaSwitch Configuration server web reference section.

Benefits

• The administrator can control the provisioning of events from PortaBilling web interface to external applications.
• The administrator saves time that might be spent on troubleshooting.