. . .

[Analytics] Installation


Introduction

With Analytics you will get functionality to track different actions from users of your app. These tracking methods will be available in your SDKs after the Countly dependency is included. When the Analytics feature is activated, the Analytics module will automatically be added to your application. Using the Analytics module you have a complete dashboard only for analytics purposes.

Prerequisites

Open the following port in the firewall:

Port

Transport

Service

Host

Client

80/443

TCP

Webserver

Application Server

Outside World

Install MongoDB [optional]

If your are installing Analytics with a package, MongoDB has to be installed first. If MongoDB is already installed, you don't need to do the following step:

Analytics 1.6 requires MongoDB 3.4 or 3.6 .
Analytics 1.7 requires MongoDB 4.4 .

System requirements

The following instance configurations are recommended.

Size

CPU

Memory

Disk

Typical Cloud Instance Types

Small

(One Server)

2 core

4 GB RAM

min. 50 GB

AWS: t2.medium

Azure: Standard_A2_V2

Medium

4 core

8 GB RAM

min. 100 GB

AWS: c5.xlarge

Azure: Standard_A4_v2

Large

8 core

16 GB RAM

500 GB

AWS: c5.2xlarge

Azure: Standard_A8_v2

Installation

USERNAME, PASSWORD, and TOKEN values are provided per customer from ApiOmat, if needed.

The ApiOmat Analytics packages aren't available in our repository yet. Contact us, so that we can provide the latest version to you. Use the RPM-Package for CentOS and DEB for Debian and Ubuntu.

After you got the package and put it into a temporary folder, you need to follow the specific installation guide.

Debian/Ubuntu
sudo dpkg -i ApiomatAnalytics-<ANALYTICS_VERSION>.deb
sudo add-apt-repository universe
 
 
#install dependencies
sudo apt-get install -f -y --no-install-recommends
Centos/Oracle/RedHat
Centos/Oracle/RedHat
sudo yum --nogpgcheck localinstall -y apiomat-Analytics-<ANALYTICS_VERSION>.rpm

During installation you may notice that some of the required package dependenies can only be retrieved from third-party repositories. Examples:

Visit the given links and follow the "Install HowTo" guide from these pages. After successful installation of the third-party libs you just need to execute the installation of the analytics package again.

Suse
sudo zypper --no-gpg-checks install -y apiomat-Analytics-suse-<ANALYTICS_VERSION>.rpm
Docker Compose

Before you can run ApiOmat Analytics in Docker, you should check the requirements here.

The following script will start up the Analytics docker container.

# Docker login
docker login -u <USERNAME> -p <PASSWORD>
 
# Store the following file and execute:
docker-compose -f analytics.yml up

After running those startup commands, it could take a while until ApiOmat Analytics is started and reachable. It is completely started when you reach the ApiOmat Analytics website with your browser (e.g. over localhost:8001).

The analytics.yaml is a typical docker compose yaml file and could be configured like this:

analytics.yml
version: '3.5'
services:
aomanalytics:
image: apiomat/analytics:<VERSION>
environment:
analytics_aomDashboardUrl: http://localhost/dashboard_2.0/apiomat
analytics_mongodb_host: mongo
analytics_mongodb_db: countly
analytics_mongodb_port: 27017
ports:
- 8001:80
depends_on:
- mongo
mongo:
image: mongo:3.4
command: --smallfiles
Kubernetes

To obtain the latest manifest file, please contact the EASY ApiOmat support.

When installing installing from a package, you should then adjust the /opt/aomanalytics/application.yaml like it is described below and then start the ApiOmat initialization script with sudo bash /opt/aomanalytics/bin/aom_init.sh.

Configuration

Linux systems

Analytics Config

You have to configure the MongoDB settings and the aomDashboardUrl parameter which should be the ApiOmat Dashboard URL.

The application.yaml is placed in the /etc/apiomat/aom-analytics directory. To use the properties below, you simply have to uncomment those specific lines. It uses the yaml syntax.

application.yaml
production:
# analytics:
# aomDashboardUrl: "https://address.to.my.apiomat.dashboard"
# path: /
#### configure a single mongo instance
# mongodb:
# host: localhost
# db: countly
# port: 27017
##### define username, password and dbOptions only when mongo auth is configured
# username: test
# password: test123
# dbOptions:
# authSource: admin
#### configure a mongo replica set
# mongodb:
# replSetServers:
# - 192.168.3.1:27017
# - 192.168.3.2:27017
# replicaName: test
##### define username, password and dbOptions only when mongo auth is configured
# username: test
# password: test123
# dbOptions:
# authSource: admin
#
#### configure consul
# consul:
# host: 127.0.0.1
# port: 8500
# secure: false
# healthCheck:
# hostname: http://192.168.178.96
# port: 80
# timeout: 2
# interval: 5
# deregistercriticalserviceafter: 60

When coming from an earlier version (when updating), you have to adjust the old config.js files: simply overwrite both config.js with the config.aom.js files, which are placed in the same directory (/api and /frontend/express). After this overwriting, you can use the application.yaml to configure Analytics. For fresh 1.5 (or higher) installations, you don't need this step.

Configure Graylog (Optional step)

When not using docker, the graylog integration needs further configuration. You first have to install the plugin for the nodejs server manager 'pm2' and then set the hostname and port via the following commands:

configure graylog plugin
# install plugin
sudo pm2 install pm2-gelf-pro
 
 
# configure address
sudo pm2 set pm2-gelf-pro:graylogPort 12201
sudo pm2 set pm2-gelf-pro:graylogHost <GRAYLOG_HOST>
sudo pm2 set pm2-gelf-pro:gelfAdapterName udp
sudo pm2 set pm2-gelf-pro:graylogFields {"serviceName" : "analytics", "serviceVersion" : "<ANALYTICS_VERSION>"}
Docker

Environmental Variables

Required variables are printed in bold.

variable

description

example

analytics_path

URL path extension for analytics

/analytics

analytics_aomDashboardUrl

this is the public URL to your ApiOmat dashboard

http://mydashboard.address.com

analytics_mongodb_host

use this, when having a single mongo instance,

localhost

analytics_mongodb_replSetServers

use this and replicaName instead of "host" when configuring a mongo replicaset

192.168.3.1:27017,192.168.3.2:27017,192.168.3.3:27017

analytics_mongodb_replicaName

name of the replicaset

test

analytics_mongodb_db

name of the db to use,

countly

analytics_mongodb_port

mongodb port

27017

analytics_mongodb_username

if an authentication is configured for the mongodb, you have to set the username and password

superadmin

analytics_mongodb_password

password of the mongodb authentication

test123

analytics_mongodb_dbOptions_authSource

having an authentication configured, this is the name if the database

admin

consul_host

host address of consul

127.0.0.1

consul_port

port of consul

8500

consul_healthCheck_hostname

hostname of the analytics instance for consul to send health check requests

http://locahost

consul_healthCheck_port

port of analytics instance

80

consul_healthCheck_timeout

timeout for the health check request (in seconds)

2

consul_healthCheck_interval

interval for health checks (in seconds)

5

consul_healthCheck_deregistercriticalserviceafter

deregister analytics from consul, when it's unhealthy (in seconds)

60

graylog_host

host of graylog

locahost

graylog_port

port of graylog

12201

graylog_service_name

optional tag for "serviceName" that can be used for filtering in graylog

analytics (default)

graylog_service_version

optional tag for "serviceVersion" that can be used for filtering in graylog

1.6.2

ApiOmat Configuration

After the installation, you can set the analytics host property in yambas.conf/apiomat.yaml to enable the ApiOmat-Analytics feature for your installation.

The following snippet shows the three lines to add to your config:

apiomat.yaml
common:
analytics:
hosts:
internal: http://localhost:8001/ # internal host target of the installation
public: http://www.yourdomain.yourhost.tld/ # external reachable address of the analytics instance
 
## add Analytics to the default Modules:
yambas:
## ...
defaultModules: Basics,Analytics
yambas.conf
analyticsHostInternal=http://localhost:8001/ # internal host target of the installation
analyticsHostPublic=http://www.yourdomain.yourhost.tld/ # external reachable address of the analytics instance
 
## add Analytics to the default Modules:
defaultModules: Basics,Analytics

Activate Analytics

A restart of the Tomcat server will not initialize your analytics instance anymore (since 3.0.1).

You have to log into the Dashboard via the SuperAdmin account, go to the configuration page and click the "Activate Analytics" button .

This will initialize the connection between Analytics and Yambas.

Logging

Linux

Logging is done to a separate logfolder: /var/log/apiomat/aom-analytics which is accessible using the following commands:

Linux
# for the api server log
tail -f /var/log/apiomat/aom-analytics/api.log
# for the frontend log
tail -f /var/log/apiomat/aom-analytics/dashboard.log
Docker

Please replace <ANALYTICS_CONTAINER_ID> with your running container ID. You can list your running containers using 'docker container list' command.

Docker
docker logs -f <ANALYTICS_CONTAINER_ID>

Process management

When being inside the docker container or having it installed on a linux system, the following commands are available:

sudo countly start
sudo countly stop
sudo countly status

ApiOmat Analytics is using pm2 as process manager. This provides more useful features, that you can access with following commands:

pm2 usage
# Process Monitoring
sudo pm2 list # List all processes started with PM2
sudo pm2 show [countly-api|countly-dashboard] # Show all information about application
 
# Log management
sudo pm2 logs # Display logs of all apps
sudo pm2 logs [countly-api|countly-dashboard] # Display logs for a specific app

Troubleshooting

Analytics is based on Countly which is a bundle of different Analytics libraries and plugins. We recommend to carefully read the following sub sections and to come back whenever you experience these problems after configuration of Analytics on startup.

The errors listed here can be identified by reading the api log file if they occur (e.g. via sudo pm2 logs countly-api).

General Approach

The technology behind Analytics/Countly is Node.js. That means that in general dependency vulnerabilities and conflicts are handled via the npm command. All installation packages above provide you a Node.js version that contains the npm command.
So whenever you run into dependency or compilation failures and have the chance to execute the npm command, we recommend to do the following:

  • delete the directory /opt/aomanalytics/node_modules/

  • delete the file /opt/aomanalytics/package-lock.json

  • change directory to /opt/aomanalytics and run npm install

This will clean up the npm dependencies and resolve the most issues. However, if you still facing dependency or compilation problems or if you do not have the chance to execute npm install we recommend to read the next section.

Operating System specific Approaches

This section contains more precise solution approaches that depend on the operating system and put focus to the occurred error.

Debian/Ubuntu

Solution for: Cannot find libc.musl-x86_64.so.1

The library 'musl' was used to compile the necessary node modules and node plugins that are included in each installation package by default. Musl is included as dependency within the Debian and Ubuntu installation packages. So, normally the automatic installation script should already have been created the following symbolic link to this shared library. Please check that this link exists and can be used by the compiled node plugins:

Debian/Ubuntu
ln -sf /usr/lib/x86_64-linux-musl/libc.so /lib/libc.musl-x86_64.so.1
Centos/Oracle/RedHat

Solution for: Cannot find libc.musl-x86_64.so.1

The library 'musl' was used to compile the necessary node modules and node plugins that are included in each installation package by default. Musl is included as dependency within the CentOS/Oracle/RedHat installation packages and needed to be obtain from the official repository like described in the linked guide above (see CentOS installation tab). After Analytics installation, the following symbolic link to this shared library should already have been created. Please check that this link exists and can be used by the compiled node plugins:

Centos/Oracle/RedHat
ln -sf /usr/lib64/libc.so.6 /usr/lib64/libc.musl-x86_64.so.1

Solution for: Undefined symbol __cxa_throw_bad_array_new_length OR npm install of argon2 fails

This error message may be thrown by the node plugin named argon2 which contains improved password hash functionality. The plugin argon2 depends on newer development tools that are missing on older operating systems (like e.g. CentOS 7). Basically the dependency resolution is done by the Node.js building tool (npm). The following guide helps you to resolve this problem step by step for CentOS 7 that contains the incompatible gcc version 4.8 by default:

Centos/Oracle/RedHat
# stop Analytics
sudo countly stop
 
 
# download and install the required development tools via Centos Software Collections Feature (SCL)
sudo yum -y install centos-release-scl
sudo yum -y install openssl-devel devtoolset-7-gcc-c++ make git wget unzip make binutils autoconf automake makedepend libtool pkgconfig zlib-devel libxml2-devel python-setuptools which
 
# switch to gcc > 4.8 (can be checked via gcc --version)
source /opt/rh/devtoolset-7/enable
 
# remove old compilation of your Analytics installation
cd /opt/aomanalytics
sudo rm package-lock.json
sudo rm -r node_modules/
 
 
# compile the node modules with the correct settings
sudo npm install --unsafe-perm=true --allow-root
sudo npm install argon2 --build-from-source --unsafe-perm=true --allow-root
 
 
# restart Analytics and check your api log again
sudo countly start
Suse

Solution A for: Cannot find libc.musl-x86_64.so.1

The library 'musl' was used to compile the necessary node modules and node plugins that are included in each installation package by default. In Suse packages the library musl is not included, because it is not officially obtainable via Suse repositories. The easier solution approach to solve this issue is to re-run npm install:

Suse
# install git since it is used by the npm command to access public sources
sudo zypper install -y git
 
 
# re-install the needed node modules and plugins that are defined in /opt/aomanalytics/package.json
cd /opt/aomanalytics
rm package-lock.json
rm -r node_modules/
sudo npm install

Solution B for: Cannot find libc.musl-x86_64.so.1

Another solution approach would be to compile the musl library for Suse by yourself and to set the symbolic link:

Suse
# download and unpack musl from offical website
curl https://musl.libc.org/releases/musl-1.2.2.tar.gz -o musl-1.2.2.tar.gz
tar -xvf musl-1.2.2.tar.gz
 
 
# compile musl
cd musl-1.2.2
./configure
make
sudo make install
 
# provide a link to your musl libc.so.6 compilation (read additional comment below)
ln -s /usr/lib64/libc.so.6 /usr/lib64/libc.musl-x86_64.so.1

If there is no file named libc.so.6 within /usr/lib64 try to find it ( e.g. via find / -name 'libc.so*' ). The linked destination file should be at the same directory like the found with libc.so.6 .
(E.g. if the file has a path of /usr/lib/libc.so.6 then your command would be: ln -s /usr/lib/libc.so.6 /usr/lib/libc.musl-x86_64.so.1 )