How to Configure Dell OMSA Monitoring on ESX

This document will describe how to configure Perl to send email. It will also describe how to setup Dell Hardware monitoring using check_dell shell scriprt. 

We will enable cron on ESX and setup cronjobs to monitor Dell hardware and send email if there is an issue.

Document Creator: Todd Walters

Requirements

·         Have Perl Installed.

·         Have MIME-Lite-3.01 installed

·         Open firewall ports for SMTP

·         Scripts Directory for Check Scripts.

·         Dell OMSA must be installed. Check using  omreport 

·         note: need to configure scripts for checking vmfs filesystems using vdf or another script

Step I: SCP Files to ESX Host

1.       Login to adminsmx00

2.       Run the following command to scp MIME-Lite-3.01

a.       [twalters@srv0smx00 ~]$ scp /home/admins/software/other/MIME-Lite-3.01.tar.gz admin@esxserver01:/tmp
Password:
 MIME-Lite-3.01.tar.gz                                               100%   49KB  48.6KB/s   00:00

3.       Run the following command to scp the Linux Bash Shell scripts

a.       [twalters@srv0smx00 ~]$ scp /home/admins/software/other/esxcheck.tar admin@esxserver01:/tmp
 Password:
 esxcheck.tar                                                   100%   10KB  10.0KB/s   00:00

4.        Run the following command to scp the Perl Send SMTP Perl scripts:

a.       [twalters@srv0smx00 ~]$ scp /home/admins/software/other/smtp_send.pl admin@esxserver01:/tmp
 Password:
 smtp_send.pl                                                 100% 3879     3.8KB/s   00:00

5.       Run the following command to scp OMSA Files if OMSA is not installed:

a.       [twalters@srv0smx00 Dell]$ scp OM_6.1.0_ManNode_A00.tar.gz admin@esxserver01:/tmp
 Password:
 OM_6.1.0_ManNode_A00.tar.gz                    67%   70MB  22.3MB/s   00:01 ETA

6.       Install OMSA following Linux Dell OMSA 

Step II: Install Mime Lite 3.01

1.       Login to srv0esx002 as admin then su to root. ( in this example - use correct ESX Node)

a.       [twalters@srv0smx00 ~]$ ssh admin@csan0esx002
Password:
 Last login: Wed Dec 15 10:48:53 2010 from csan0smx00.corp.cox.com

b.       [admin@srv0esx002 ~]$ su -

2.       Change directory to /usr/lib/perl5/ and create MIME Dir if not there.

a.       [root@srv00esx002 5.8.8]# cd /usr/lib/perl5/5.8.8/

b.      [root@srv0esx002 5.8.8]# ls MI*  ls: MI*:

c.        Since no MIME directory, create one:

d.      [root@srv0esx002 5.8.8]# mkdir MIME

e.       [root@csrv0esx002 5.8.8]# cd MIME

3.       Untar MIME to MIME directory

a.       [root@srv0esx002 MIME]# tar xvfz /tmp/MIME-Lite-3.01.tar.gz

4.       Change directory to MIME-Lite-3.01 and Install MIME.

5.       Create Makefile

a.       [root@srv0esx002 MIME-Lite-3.01]# perl Makefile.PL

b.      Checking if your kit is complete...
 Looks good
 Writing Makefile for MIME::Lite

6.        Test Make Install

a.       [root@srv0esx002 MIME-Lite-3.01]# make test

7.       cp changes.pod blib/lib/MIME/changes.pod

8.        cp lib/MIME/Lite.pm blib/lib/MIME/Lite.pm

a.        PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t

b.       t/addrs.....ok
 t/data......ok
 t/head......ok
 t/parts.....ok
 t/types.....#
 #Interaction with MIME::Types has not been tested
 #as it doesn't seem to be present.
 t/types.....ok
 t/verify....ok
 All tests successful.
 Files=6, Tests=43,  0 wallclock secs ( 0.14 cusr +  0.08 csys =  0.22 CPU)

9.        Install using Make

a.       [root@srv0esx002 MIME-Lite-3.01]# make install
Manifying blib/man3/MIME::changes.3pm

b.       changes.pod:49: Unknown escape E<#228>
 changes.pod:52: Unknown escape E<#228>
 Manifying blib/man3/MIME::Lite.3pm
 Installing /usr/lib/perl5/site_perl/5.8.8/MIME/Lite.pm
 Installing /usr/lib/perl5/site_perl/5.8.8/MIME/changes.pod
 Installing /usr/share/man/man3/MIME::changes.3pm
 Installing /usr/share/man/man3/MIME::Lite.3pm
 Writing /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/auto/MIME/Lite/.packlist
 Appending installation info to /usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/perllocal.pod

10.   Move send_smtp.pl script

a.       [root@srv0esx002 MIME-Lite-3.01]# mv /tmp/smtp_send.pl /usr/local/bin

Step III: Setup ESX check Scripts 

1.       Create three directories for Check shell Scripts in /opt

2.       [root@srv0esx002 opt]# mkdir check check/bin check/etc

3.       Untar the esxcheck.tar tarbal to /opt/check/bin

4.       [root@srv0esx002 opt]# tar xvf esxcheck.tar -C /opt/check/bin

check_filesystem.conf

check_filesystem.sh

check_dell

5.       Change directory to /opt/check/bin/ and move files

6.       [root@srv0esx002 opt]# cd check/bin

7.       [root@srv0esx002 bin]# mv check_filesystem.conf ../etc

8.       Use vi to edit the check_dell script to use perl smtp script

·          note: editing files not needed now because updated scripts are in esxcheck.tar file. This just documents changes below:

9.       [root@srv0esx002 bin]# vi check_dell

·         type :107 to move to line 107 in file

·         Edit this line cat $TEMPFILE | mail -s "$Subject" $EMAIL to read like this :

·         cat $TEMPFILE | /usr/local/bin/smtp_send.pl -s "$Subject" -t $EMAIL -r 192.168.mail.mail?

10.   Edit the check_filesystem.conf file to configure filesystem checks.

[root@srv0esx002 etc]# cd /opt/check/etc

vi file and set to these values:

 [root@srv0esx002 etc]# vi check_filesystem.conf

 /:90

 /boot:70

 /var/log:80
      admins:alertemail@mydomain.com

11.   Edit the check_filesystem.sh script to use send_smtp.pl script

·         note: editing files not needed now because updated scripts are in esxcheck.tar file

12.   [root@srv0esx002 bin]# vi check_filesystem.sh

·         type :72 to go to line 72 or this line: cat ${TEMPFILE}.mail | mail -s "$Subject" $EMAIL

13.   Edit just like #4 in this section, line should look like below:

·         cat $TEMPFILE | /usr/local/bin/smtp_send.pl -s "$Subject" -t $EMAIL -r 10.62.232.42

Step IV: Configure Firewall and Test

14.   Open up port 25 on ESX server with this command:

·         [root@srv0esx002 ~]# esxcfg-firewall -o 25,tcp,out,SMTP

15.   Check that firewall rule shows up:

·         [root@srv0esx002 ~]# esxcfg-firewall -q | grep SMTP

·         SMTP                : port 25 tcp.out

16.   Send Test Email to yourself

·         [root@srv0esx002 bin]# ./smtp_send.pl -t todd.walters@mydomain.com -s "Test Email from ESX" -r 10.18.232.42

smartHost: 10.62.232.42

to: todd.walters@mydomain.com

subject: Test Email from ESX

attach: none

17.   Verify you receive the email.

Step V: Configure cronjob to Run Monitoring Scripts

18.   Check if there is existing crontab for root

·         [root@srv0esx002 bin]#       crontab -l
       no crontab for root

19.   Edit new crontab for root

·         [root@srv0esx002 bin]# crontab -e

·         no crontab for root - using an empty one

20.   Copy the following into       the file

##################################################

        #*  *  *  *  *  command to be executed

        #-  -  -  -  -

        #|  |  |  |  |

        #|  |  |  |  +-- day of week (1 - 7) (monday = 1)

        #|  |  |  +---- month (1 - 12)

        #|  |  +------ day of month (1 - 31)

        #|  +-------- hour (0 - 23)

        #+---------- min (0 - 59)

        ##################################################

        # This script is an 'early-warning system for disk trouble:

        */15 * * * * /opt/check/bin/check_filesystem.sh

        # This script run OMSA every 10 minutes

        */10 * * * * /opt/check/bin/check_dell

21.   Can add this line to scripts called from crontab (ie check_dell)  to open/close smtp port if not needed open all the       time. Place at beginning and end of script.

·         # temporarily enable firewall rule to allow access to smtp

 $cmdFirewallOn = “/usr/sbin/esxcfg-firewall -o 25,tcp,out,SMTP”;

 system($cmdFirewallOn);

 and

 # re-apply firewall to block SMTP

 $cmdFirewallOn = “/usr/sbin/esxcfg-firewall -c 25,tcp,out,SMTP”;

 system($cmdFirewallOn);

22.   Check cron daemon to verify it's running

·         [root@srv0esx002 ~]#       service crond status

                                  crond (pid 3634) is running...

23.   Check cron log files to verify command run.      (notice last two      lines)

·         [root@srv0esx002 /]# cd /var/log

·         [root@srv0esx002 log]# tail       -n 4 cron

Dec 28 13:25:01 srv0esx002 crond[8939]: (root) CMD       (/usr/lib/vmware/bin/memwatchdog.py > /dev/null 2>&1)

       Dec 28 13:30:01 srv0esx002 crond[9896]: (root) CMD       (/usr/lib/vmware/bin/memwatchdog.py > /dev/null 2>&1)

       Dec 28 13:30:01 srv0esx002 crond[9899]: (root) CMD (/opt/check/bin/check_dell)

       Dec 28 13:30:01srv0esx002 crond[9904]: (root) CMD (/opt/check/bin/check_filesystem.sh)