Realtime Connection


About Real-time Integration:  #

In real-time mode, Wowizer seamlessly integrates with your Qlik Sense environment and continuously collects and analyzes logs and data in real-time. This approach provides immediate visibility into the performance, health, and activities of your Qlik Sense environment. Real-time monitoring allows you to detect issues as they occur, receive alerts, and take proactive actions to optimize performance and ensure smooth operation. It’s ideal for proactive monitoring, alerting, and rapid incident response. 

Realtime mode is a key benefit of Wowizer TPM, collecting incremental logs of all essential Qlik Sense components/services and Windows Event Logs. The typical Realtime data latency is 1 minute, ensuring that your logs are ready for analysis or triggering alerting conditions within 1 minute. 

Wowizer Log shipper / Adaptor :
#

Wowizer Realtime setup can be done in 2 ways:
– Automated procedure
– Manual procedure

Automated procedure #

Wowizer team has simplified the setup procedure by automating all steps required for setting up adaptor.
You can download the automated script from this link:

Before running this script, kindly complete one step of setting Virtual Proxy . Refer this link

After downloading the automation script, launch PowerShell with Administrator privileges and execute the script as below:
.\wowizer_realtime_setup_1.ps1 <INGESTION_ID> <QLIK_SERVER_IP>

INGESTION_ID : Refer this section below to know where you will find your unique Ingestion ID
QLIK_SERVER_IP : Your Qlik server IP where we running this setup.

Following this, proceed with the subsequent steps, which involve certain manual tasks related to the creation of a Virtual Proxy. You can jump to STEP-4



Manual procedure #

Filebeat and Winlogbeat are lightweight Opensource data shippers developed by Elastic, designed to collect, ship, and centralize log data. They play a crucial role in log aggregation and analysis, and here’s why they are needed and how they are used in Wowizer:

Filebeat:

  • Why Needed: Filebeat is essential for collecting log files from various sources, including application logs, system logs, and custom logs. It ensures that log data is reliably transported to a central repository for analysis.
  • Use in Wowizer: Wowizer uses Filebeat to collect log data from Qlik Sense environments, such as reload logs, audit logs, and application logs. Filebeat helps ensure that log information is gathered in real-time and sent to Wowizer for analysis.
  • Download pre-configured ‘Filebeat’ from here
    Keep this on your Qlik Server or node you want to link to Wowizer. (This is pre-configured version of the utility which has everything configured.)
     

Winlogbeat:

  • Why Needed: In Windows environments, events are recorded in the Windows Event Log. Winlogbeat is essential for capturing these events, which can contain critical information related to system health, security, and application performance.
  • Use in Wowizer: Wowizer utilizes Winlogbeat to gather Windows event logs from Qlik Sense servers. It allows Wowizer to correlate these logs with Qlik Sense activities, helping identify potential issues stemming from Windows updates, security breaches, or system errors.
  • Download Wowizer pre-configured Winlogbeat from here.
    Keep this on your Qlik Server or node you want to link to Wowizer. (This is pre-configured version of the utility which has everything configured.) 

In Wowizer, both Filebeat and Winlogbeat serve as critical components for real-time log collection and analysis. They ensure that log data from Qlik Sense environments and Windows servers is efficiently transported to Wowizer, enabling administrators to monitor, analyze, and respond to system events, performance anomalies, and security incidents effectively.

Configuration on Qlik server #

Once you have downloaded this utility, you need to create 2 Environment variables as below: 

QLIK_LOGS_ROOT_PATH :
This is the path of your Qlik installation folder up to the Log directory. 

WOW_Ingestion_ID :
This is the unique Tenant-ID assigned to you upon having a valid Realtime license.
You can find this as the ‘Unique Stream ID’ under the ‘Real Time logs’ section, as shown in the screenshot below. 

If you do not have a valid Realtime license, you won’t see this option in your account. However, you can still use the Static mode and continue troubleshooting within your specified data limits based on your account type.  

Once you have configured these two Environment variables, you are ready to start streaming Realtime logs. Proceed to the next steps to begin the actual log streaming to Wowizer from your Qlik Sense server.

Running Winlogbeat as a service :

To run Winlogbeat as a Windows service, follow these steps:

  • Open a PowerShell prompt as an Administrator (right-click the PowerShell icon and select “Run As Administrator”).
  • Navigate to the Winlogbeat directory that you extracted (as done in Step-2).
  • From the PowerShell prompt, run the following commands to install Winlogbeat as a Windows service:

    - cd 'C:\Program Files\winlogbeat_wowizer_v1' # Replace with the actual directory where you extracted Winlogbeat
    - .\install-service-winlogbeat.ps1
    - Start-Service winlogbeat


Note: If script execution is disabled on your system, you need to set the execution policy for the current session to allow the script to run. Please consult your administrator before proceeding. 

The command to set the execution policy is as follows:

PowerShell.exe -ExecutionPolicy UnRestricted -File .\install-service-winlogbeat.ps1 

Running Filebeat as a service 

To run Filebeat as a service, follow these steps:  

  • Open a PowerShell prompt as an Administrator (right-click the PowerShell icon and select “Run As Administrator”). 
  • Navigate to the Filebeat directory that you extracted (as done in Step-2).
  • From the PowerShell prompt, run the following commands to install Filebeat as a Windows service: 

    cd 'C:\Program Files\filebeat_wowizer_v1' (The directory where you extracted Filebeat) 
    - .\install-service-filebeat.ps1 
    - Start-Service filebeat 

Note: If script execution is disabled on your system, you need to set the execution policy for the current session to allow the script to run. Please consult your administrator before proceeding. 

The command to set the execution policy is as follows:

PowerShell.exe -ExecutionPolicy UnRestricted -File .\install-service-filebeat.ps1 

Wowizer HealthData Utility #

Wowizer’s health data detection feature is designed to collect essential system performance metrics in real-time, including CPU usage, RAM utilization, and available disk space.  
This real-time data provides valuable insights into the health and performance of your Qlik Sense environment, allowing you to monitor system resources and take proactive actions to optimize performance and ensure smooth operation. Having access to such real-time data is crucial for administrators and IT teams to maintain the reliability and efficiency of their Qlik Sense platform. 

 
Configuring health data detection in Qlik Sense typically involves two key steps, as mentioned: 

1. Create Virtual Proxy 

   – The first step is to create a virtual proxy in the Qlik Sense Management Console (QMC). This virtual proxy acts as an intermediary for routing health data requests. You can configure various settings within the virtual proxy, including authentication, security, and load balancing, to ensure that health data requests are properly handled. 

2. Start a Health Data Service 

   – The second step is to start a health data service that will collect, analyze, and report on the health of your Qlik Sense environment. This service can be configured to monitor various aspects of the system, such as server performance, resource utilization, user activity, and more. Once the service is running, it continuously collects data and sends it to WoWizer server. 

These two steps are essential for setting up health data detection and monitoring in Qlik Sense. The virtual proxy helps route health data requests to the appropriate service, while the health data service itself collects and organizes the data to ensure the smooth operation of your BI platform. This proactive approach allows you to identify and address issues before they impact your users and helps optimize the performance and reliability of your Qlik Sense environment. 

Steps to Create/Setup Virtual Proxy in Qlik Sense:  #

  • Access QMC: Open a web browser and enter the URL for your Qlik Sense Management Console (usually https://<YourServer>:<Port>/qmc/).  
     
  • Navigate to Virtual Proxies: In the QMC, go to the “Proxies” section in the left-hand navigation pane. 
    You can Scroll down and select “Configure System” → “Virtual Proxies” from QMC menu. 

  • Create a New Virtual Proxy: To create a new virtual proxy, simply click on the “Create” or “New” button. You can name it as “Health API” or any other desired name. 
  • Configure Identification
     
  • Enter a description for the virtual proxy.   
  • Assign a prefix for the proxy. 
  • Indicate the session inactivity timeout. 
  • Specify the session cookie header name. 


  • Load Balancing
  • Navigate to the “Load Balancing” option located in the right-side menu.. 
  • Next, add a new server node, specifically for your Qlik Sense server, commonly referred to as the “Central node”. 


  • Advanced: 
     
    Proceed by clicking on the “Advanced” option. Ensure that both the HTTPS and HTTP secure attributes are checked, and set the SameSite attribute to “None.” Additionally, include “Access-Control-Allow-Origin” in the list of additional response headers. 


  • Host Allow List

Within the “Advanced” section of the virtual proxy configuration, locate the “Host Allow List.” 

You should then include the domain or IP address of the Qlik Sense server in this list. For instance, if the connection is established via the server’s IP address, make sure to add that IP address to the allowed list. 

  • Apply Configuration: After configuring the virtual proxy settings, click on the “Apply” button to save the changes you’ve made. 
  • Link to Central Proxy
     
  • Navigate to “Proxies” menu item on the right side of the screen.
     

  • Press the “Link” button and add the Central proxy to the configuration.
     

Steps to Run HealthData Service:  #

Here are the steps to run the HealthData Service: 

Running HealthData Service: 

1. Download HealthData archive
– You can download Healthdata archive from https://github.com/wowizer/public/raw/wowizer-documents/HealthData/v2/HealthData_Wowizer_v2.zip 

 
2. Unzip HealthData_Wowizer_v2.zip
Extract the contents of the HealthData_Wowizer_v2.zip file onto your Qlik server. It’s recommended to extract it to the C drive.  
The extracted files will include: healthcheckstart.bat, healthcheckremove.bat, wowizer_healthcheck.bat, diskinfo.ps1, nssm.exe, and curl.exe. 

2. Update wowizer_healthcheck.bat

   – Open the wowizer_healthcheck.bat file using a text editor. 

3. Replace Qlik Sense Server IP

   – Replace “<YOUR QLIK HOST IP>” with the actual IP address or domain of your Qlik Sense server. For example: https://<Qlik host ip>/healthapi/engine/healthcheck/. 

4. Save and Close File

   – After updating the URL, save and close the wowizer_healthcheck.bat file. 

5. Open PowerShell as Administrator

   – Press Win + X and select “Windows PowerShell (Admin).” 

6. Navigate to HealthData_Wowizer_v2 Folder

   – Use the ‘cd’ command to navigate to the folder where you extracted HealthData_Wowizer_v2. 

7. Start HealthData Service

   – Run the following command to start the HealthData service: 

     “` 
.\healthcheckstart.bat 

     “` 

8. Check healthcheckdata.txt

   – After running the command, verify if the healthcheckdata.txt file is generated in the same folder. The presence of this file indicates that the service has started successfully. 

9. Configure Filebeat for Realtime Logs

   – Stop the Filebeat service using PowerShell as an Administrator: 

     “`
Stop-Service filebeat 

     “` 

10. Update filebeat.yml

    – Update the filebeat.yml file to include the path to healthcheckdata.txt. 

11. Start Filebeat Service

    – Start the Filebeat service again: 

      “` 

      Start-Service filebeat 

      “` 

12. Verify Realtime Logs

    – If no issues are reported, your data ingestion will commence within 2-5 minutes. You can monitor the logs in the Realtime Dashboard at https://tpm.wowizer.com

Please note that these instructions assume you have administrative access to your Qlik Sense server and can perform system-level actions. Additionally, ensure that all necessary dependencies and permissions are in place for this setup to work correctly. 

Wowizer App Sync
#

For Wowizer to stay informed about the latest published and new Qlik Sense apps, a lightweight utility app has been developed. This app compiles a list of all currently available apps and is regularly updated.

To ensure that Wowizer remains up-to-date with the latest apps, it’s crucial to schedule this utility app for nightly or off-peak-hour reloads. Although this app is lightweight, its frequent reloads help maintain synchronization between Wowizer and the Qlik Sense environment.

In situations where immediate updates are necessary during the day, triggering a reload of this utility app ensures that Wowizer’s analysis remains in sync with the evolving Qlik Sense ecosystem.

You can find the App QVF here .

Download the app and upload it through the QMC, and refer to the associated help article for any assistance needed.


Updated on December 20, 2023