Steps for moving an Opsview database to a new server

Recently I’ve had to move a few opsview servers databases from the Opsview master onto an alternate server, mostly because of disk space issues, but I’ve found this to be a pretty good performance increase on busy servers too, so its well worth doing anyway.

Firstly you want to build yourself a new server, I’m using Ubuntu and so have rolled out the 12.04 LTS beta. Installing mysql is pretty simple – apt-get install mysql-server.

There’s a few things you’ll want to change in the my.cnf config file (/etc/mysql/my.cnf on Ubuntu).

Change the bind address so that we can connect to our server across the LAN:

bind-address = 0.0.0.0

And the following performance based settings as recommend by Opsview: (check out http://docs.opsview.com/doku.php?id=opsview-community:mysql)

key_buffer = 256MB
query_cache_size = 16M
table_cache = 768
innodb_buffer_pool_size = 1024M
innodb_file_per_table=1
innodb_flush_log_at_trx_commit=2

Now give your mysql a restart, check the log for errors: /etc/init.d/mysql restart if you’re oldschool or service mysql restart.

Now we need to get the data off the master server and onto the new dedicated mysql server. We can do this over SSH to save storing multiple copies of the database (mine was 50GB or so).

From the master server first stop opsview:

/etc/init.d/opsview stop
/etc/init.d/opsview-web stop

Now run the following command to export the data and directly transfer into mysql on the new server over SSH. You should be a tad cautious here as the passwords are entered on the CLI in clear text.

mysqldump -u root -p<rootpassword> --databases odw opsview reports runtime |  ssh user@<new-server-hostname> 'mysql -u root -p<rootpassword>'

The data may take some time to copy, depending on its size. You should consider running this command inside screen or tmux in case your SSH session gets disconnected.

Once all your data has been transferred, you need to change the /usr/local/nagios/etc/opsview.conf file to point to the new database server. It’s simply a mater of assigning a few variables like so:

$dbpasswd = "yourpassword";
$dbhost = 'mysql-hostname.domain.local';
$runtime_dbpasswd = "yourpassword";
$runtime_dbhost = 'mysql-hostname.domain.local';
$odw_dbpasswd = "yourpassword";
$odw_dbhost = 'mysql-hostname.domain.local';
$reports_dbpasswd = "yourpassword";
$reports_dbhost = 'mysql-hostname.domain.local';

There’s a handy wee script provided for configuring credentials in MySQL – you just need to run it, again we’ll pipe it directly to the new server ready for execution:

/usr/local/nagios/bin/db_mysql -t | ssh user@mysql-hostname ~/opsview_access.sql

I prefer the database access to be slighly more secure that what this script generates so I’d recommend replacing the % (any host) entries in the file with the hostname of your opsview server. This can by done easily using sed:

sed -i 's/%/opsview-master.domain.local/g' opsview_access.sql

Now you can go ahead and import this into mysql:

mysql -u root -p < opsview_access.sql

All thats left to do is to regenerate the Opsview configuration and then start up the services. This can be done as follows:

/usr/local/nagios/bin/rc.opsview gen_config
/etc/init.d/opsview-web start

Once you’re happy that everything is running as expected, you can get rid of the old databases on the master server. Drop them like this:

drop database opsview;
drop database odw;
drop database runtime;
drop database reports;

Problem with certificate checking on check_esx.pl plugin

Either a change in newer Perl libraries or version 5 of the vSphere Perl SDK has meant that certificates are now being checked by default, and the check_esx.pl plugin is returning something like the following as output:

CHECK_ESX.PL CRITICAL - Server version unavailable at 'https://vcenter-server.domain.com:443/sdk/vimService.wsdl' at /usr/share/perl/5.10/VMware/VICommon.pm line 545, <AUTH_FILE> line 2.

Here’s a quick patch to add a -i option which will disable the certificate check.

Opsview – Notification via Mikrotik SMS gateway

So in my last post I showed how to build yourself a SMS gateway on the cheap using a Mikrotik router and the included API coupled with a scratched together Perl script.

This post is all about doing something useful with your new gateway and how to get Opsview to send alerts with it.

In order to make our job much simpler, we can use the built in submit_sms_script that is shipped with Opsview. On my Ubuntu box it’s located at /usr/local/nagios/libexec/notifications/submit_sms_script.

I’ve placed the script from my previous article (http://snipt.org/xmngp) under /usr/local/bin/mikrotik_sms, we just need to make a few small modifications to the Opsview script to use our SMS gateway script instead of theirs. Make a copy of submit_sms_script to say submit_sms_mt and open it for editing.

All we want to change the $command variable to the location of our script, but there are a few other lines which are no longer needed so here’s a quick patch to make your life easier:

Now we can give this a quick test to prove functionality before defining the notification method. Opsview provide a test_notifications script for this purpose. We need to add in your mobile number as CONTACTPAGER so the script can emulate this. Place it in the list of existing environment variables that are defined as %a:

$a{CONTACTPAGER}        = "0123456789";

Now you can execute the test_notifications script as follows:

/usr/local/nagios/utils/test_notifications hostproblem
/usr/local/nagios/libexec/notifications/submit_sms_mt

All going well you should get a response something like this:

Running: /usr/local/bin/mikrotik_sms -n 0123456789 -t PROBLEM: host1 is DOWN: Test host failure ()
message sent to 0123456789

Next we are going to create the notification method in Opsview. Choose Advanced > Notification Methods from the menu. Fill in the fields with a name to identify the method, the script name and PAGER in the Contact Variables field.

Now we need to assign this to a contact, first either open up an existing contact or create a new one under the Configuration > Contacts menu. Fill in the mobile filed with the mobile phone number to be notified.

Now hit Submit and Edit Notification Profiles – and create a new Notification profile. Give this a meaningful name and select the Notification Method you created above.

Choose the relevant host groups and services you wish to be alerted on. Something that works really well for us is to set the receive from alert value to 2 – this ensures if an email alert is not acknowledged then it is escalated to an SMS alert, yet does not ‘spam’ us with trigger happy alerts. It’s also a good idea to create additional Notification profiles for mission critical equipment that you need to know about in a timely fashion.

To ensure your alert is working as expected, setup a dummy host or break something to get Opsview to trigger an alert. You can watch the nagios log by executing `tail -f /usr/local/nagios/var/nagios.log` to see alerts that are being triggered.