Nexfs Logging, Monitoring and Alerting
Jump to Section
Service Health Check Monitoring
Nexfs Syslog Logging
Nexfs Logs via syslog on the local host, the Log level and Syslog facility to be used is configurable.
Note: Performance will be degraded while running Nexfs at the DEBUG or INFO level log level
Nexfs also retains a copy of the last 100 Errors and Critical messages in memory, this can be viewed using the Nexfs Admin Portal or retrieved using the Nexfs Admin API
The last 100 critical and error messages are displayed on the System Status page of the Admin Portal
Retrieving and changing the Nexfs Log Level
Web Admin Portal
The Log Level can be view and set under the Syslog Settings heading in the Log & Alert tab of the Web Admin Portal, changes made through the portal go live instantly and will also be retained when Nexfs restarts
Nexfscli
Retrieve the current (live) log level of a running nexfs server, run
nexfscli liveconfig get loglevel
Change the log level, run
nexfscli liveconfig set loglevel {NEWLOGLEVEL}
Replace {NEWLOGLEVEL} with one of:
CRIT, ERR, WARNING, NOTICE, INFO or DEBUG
For example to set the nexfs server to log at debug level, run
nexfscli liveconfig set loglevel DEBUG
Note: this will only change the loglevel for the running nexfs server, when the server is restarted the loglevel will revert back to that set in the configuration file. To permanently change the log level you also need to change the configfile value:
nexfscli configfile set loglevel DEBUG
Retrieving and changing a Nexfs Syslog Facility
Web Admin Portal
The Syslog Facility can be viewed and set under the Syslog Settings heading in the Log & Alert tab of the Web Admin Portal, changes made through the portal go live instantly and will also be retained when Nexfs restarts
Nexfscli
Retrieve the current (live) syslog facility, run
nexfscli liveconfig get value SYSLOGFACILITY
Change the syslog facility, run
nexfscli liveconfig set SYSLOGFACILITY {NEWSYSLOGFACILITY}
Replace {NEWSYSLOGFACILITY} with one of:
LOG_USER, LOG_LOCAL0, LOG_LOCAL1, LOG_LOCAL2, LOG_LOCAL3, LOG_LOCAL4, LOG_LOCAL5, LOG_LOCAL6 or LOG_LOCAL7
For example to set the syslog facility to LOG_LOCAL6 level, run
nexfscli liveconfig set SYSLOGFACILITY LOG_LOCAL6
Note: this will only change the syslog facility for the running nexfs server, when the server is restarted the facility will revert back to that set in the configuration file. To permanently change the log level you also need to change the configfile value:
nexfscli configfile set SYSLOGFACILITY LOG_LOCAL6
Nexfs Service Health Check Monitoring
Nexfs can expose endpoints for probing overall and sub system service monitoring, currently these endpoints are unauthenticated.
The Nexfs Integrated Admin Web Portal and Health-Checks must be enabled for the endpoints to be exposed.
By default Nexfs will only report a service problem if that service is in Error state, optionally Nexfs can report services in a Warning state. Warning includes the states: inShutdown, Shutdown, Warning, Degraded and HighWaterMark.
-
When a service state is OK, a probe returns HTTP status 200 with the html body text 'OK'
-
If Warning reporting is disabled and a service is in Warning state, the probe returns HTTP status 200 with the html body text 'OK'
-
If Warning reporting is enabled and a service is in a Warning state, a probe returns HTTP status 409 with the html body text 'Warning'
-
When a service is in a Error state, a probe returns HTTP status 500 with the html body text 'Error'
The health check endpoint is http://nexfs.example.net:9200/nexfshealth?Service={SERVICE}
Where:
nexfs.example.net is the DNS resolvable hostname or the ipaddress of the Nexfs server, and 9200 is the port that the integrated admin web portal is configured to listen on.
{SERVICE} is one off:
Nexfs Report the overall status of Nexfs
StructureReplication Report the Status of structure replication
Tier1Structure Report the Status of Tier-1 Structure Storage
Tier2Structure Report the Status of Tier-2 Structure Storage
Tier1 Report the Status of Tier1 Data Storage
Tier2 Report the Status of Tier2 Data Storage
Tier3 Report the Status of Tier3 Data Storage
Enabling Service Health Check Monitoring from the Web Admin Portal
Server Health Check can be configured under the Advanced Settings tab of Nexfs Configuration in the Web Admin Portal, changes made through the portal go live instantly and will also be retained when Nexfs restarts.
1. Enable API / MgmtWeb Server
2. Enable Mgmt Server Health Check Reporting
3. (Optional) Enable Mgmt Health Check Report Warning
Enabling Service Health Check Reporting using Nexfscli
To enable Server Health Check Reporting in the running instance as well as on restart, using Nexfscli, run:
nexfscli liveconfig set MGMTWEBSERVERENABLED 1
nexfscli configfile set MGMTWEBSERVERENABLED 1
nexfscli liveconfig set MGMTWEBENABLEHEALTHCHECK 1
nexfscli configfile set MGMTWEBENABLEHEALTHCHECK 1
Optionally enable Health Check Warning Reporting
nexfscli liveconfig set MGMTWEBENABLEHEALTHCHECKWARN 1
nexfscli configfile set MGMTWEBENABLEHEALTHCHECKWARN 1
If Nexfs is not running then only run the configfile commands above
To disable the services, run the same commands but set the values to 0 instead of 1
SNMPTrap Alerts
Nexfs can be configured to send SNMP version 2c or 3 traps when Nexfs services change state, SNMPTrap functionality is integrated into Nexfs, no addition system software or packages need to be installed.
You can download a copy of nexustorage.mib here
SNMPTraps can be configured using the Admin Portal using nexfscli or the management api
Configuring SNMPTraps Using the Admin Portal
SNMP Traps are configurable under the Advanced Settings tab in Nexfs Configuration , changes made through the portal go live instantly and will be retained after Nexfs restarts.
More than one SNMP Target Agent can be specified by separating each target with a comma, targets must be DNS resolvable hostname or valid ipaddresses.
Using Nexfscli and the Management API
See the Logging, Alerting, Monitoring and Stats section of Nexfs System settings for a list and description of SNMP Trap settings
See Managing Nexfs with Nexfscli for information on using Nexfscli to configure Nexfs
See Managing Nexfs Using the Management API for details on configuring Nexfs using the management api
