Monit stats capturing through Nagios

There is a Nagios plugin available through which services monitored by Monit can be monitored - http://code.google.com/p/nagios-monit-plugin/

But there are a few limitations to this script like -

(1) Added granular information about services like Load, Memory and HTTPD information monitored by monit.
(2) Added compatibility with Monit version 4.9 and version 5.2.5 to show the details mentioned in point (1)
(3) Added performance data for the results, so that trend graphs can be plotted

These issues are catered by this updated version of my check_monit.py script submitted as a patch at http://nagios-monit-plugin.googlecode.com/issues/attachment?aid=20000000&name=check_monit.py.patch&token=32qXj5orfcsWNcrbIZ4gvT5WoPA%3A1328588504677

Managed and SysAdmin Services comparison

Following is some of the Analysis that I had done for Network Managed services a while earlier. Just sharing it across for a wider audience:

There are few service providers available, who are providing Managed and SysAdmin Services for IaaS Cloud. Following table provides details on service offerings by these known providers compared with Nagios Monitoring (last column), which is an Open-Source network monitoring tool. Though Nagios might not have all the features provided by these providers, but it will provide comparison of where Nagios stands w.r.t. other tools in the market.

A brief description of the offerings being compared is as follows:

Freedom OSS Managed Services:

Freedom OSS’s post production care services cover scaling, maintaining, monitoring and security applications as well as the underlying Cloud Infrastructure. All instances created using Freedom’s managed service come automatically pre-installed with monitoring agents capable of monitoring all low level OS attributes and all common server applications such as web servers, databases and other executable processes. This solution natively subscribes to the AWS CloudWatch even stream related to that instance which allows managed service to have full visibility of the health of the infrastructure.



GroundWork OS Monitor Enterprise:

GroundWork Monitor Enterprise Edition delivers a highly scalable monitoring platform designed to meet the needs of complex, large and distributed environments. Enterprise Edition provides availability and performance visibility in large, heterogeneous and distributed environments by utilizing industry-standards (SNMP, syslog, WMI) and de-facto standards (NRPE, Nagios plug-ins) for data collection.

GroundWork Monitor Enterprise can serve as a customer’s only network management system or serve as part of an integrated solution for IT Service Management that includes products from a selection of vendors. An Enterprise deployment may integrate the outputs of multiple third party monitoring tools or be easily configured to report its results to the consoles of other major monitoring tools.



Nagios Monitoring Tool:

Nagios® is a system and network monitoring application. It watches hosts and services that you specify, alerting you when things go bad and when they get better.

Nagios was originally designed to run under Linux, although it should work under most other unices as well.

Some of the many features of Nagios include:

• Monitoring of network services (SMTP, POP3, HTTP, NNTP, PING, etc.)
• Monitoring of host resources (processor load, disk usage, etc.)
• Simple plugin design that allows users to easily develop their own service checks
• Parallelized service checks
• Ability to define network host hierarchy using "parent" hosts, allowing detection of and distinction between hosts that are down and those that are unreachable
• Contact notifications when service or host problems occur and get resolved (via email, pager, or user-defined method)
• Ability to define event handlers to be run during service or host events for proactive problem resolution
• Automatic log file rotation
• Support for implementing redundant monitoring hosts
• Optional web interface for viewing current network status, notification and problem history, log file, etc.

OpenNMS:

OpenNMS, the application, is the first enterprise-grade network management platform to be developed under the open-source model.

The goal is for OpenNMS to be a truly distributed, scalable platform for all aspects of the FCAPS (Fault, Configuration, Accounting, Performance, Security) network management model, and to make this platform available to both open source and commercial applications.

Currently, OpenNMS focuses on three main areas:

• Service Polling - determining service availability and reporting on same.
• Data Collection - collecting, storing and reporting on network information as well as generating thresholds.
• Event and Notification Management - receiving events, both internal and external, and using those events to feed a robust notification system, including escalation.

OpenNMS features:

· http://technocrat.watson-wilson.ca/blosxom/computer/onmsreview.html?seemore=y

· http://www.opennms.org/wiki/Docu-overview



Comparison Table:

Features	Freedom OSS	GroundWork OS	Nagios Monitoring Tool	OpenNMS
Instance availability alert (up/down)	Yes	Yes	Yes	Yes
Critical processes availability	Yes	Yes	Yes	Yes
System resource over / under utilization	Yes	Yes	Yes	Yes
Application feature availability	Yes			Yes
Email notification	Yes	Yes	Yes	Yes
SMS / MMS	Yes		SMS	SMS, Pager, XMPP, Growl
Integration with enterprise datacenter ecosystem	No	Yes	No	Yes
Self-control over monitoring	No	Yes	Yes	Yes
User-level control with access rights	No	Yes	No	Yes + LDAP support
Enterprise Management systems (e.g. Tivoli, HP Openview)	Yes	Yes	No	Yes It is itself an enterprise management
Cross metrics correlation	Yes		No
Trend analysis	Yes	Yes	No	Yes
Automatic instance scaling	Yes	Yes	No	No
Backup and data synchronization	Yes	No	No	No
Security	Yes	No	No	Spring security
Industry security compliance & Data privacy	yes	No	no
Available support level	Yes	Yes	No	no
Amazon EC2 support	Yes	Yes	No	no
Eucalyptus Private cloud support	No	Yes	No	no
SNMP support	No	No	Snmp plugin available	Full support
Auto-discovery of nodes	Yes		No	Yes
Layer-2 & 3 device support	No		No	Yes
Scalability	No		Yes	Yes
Platform supported	Linux	Linux,	Linux, windows	Linux, MAC OS, solaris, Windows, FreeBSD

Good article on Nagios Vs OpenNMS

http://www.rootdev.com/tech/opennms-vs-nagios



Enterprise-level Monitoring System Comparison:

An extensive comparison of various network monitoring systems was made and results are available at the URL - http://en.wikipedia.org/wiki/Comparison_of_network_monitoring_systems

HP OpenView

http://www.openview.hp.com/uploads/emanate_white_paper.pdf

HDFS explained in a nice comic way

I found following article very easy, explaining the functionality of HDFS. A must read for people working on Hadoop:

https://docs.google.com/open?id=0B-zw6KHOtbT4MmRkZWJjYzEtYjI3Ni00NTFjLWE0OGItYTU5OGMxYjc0N2M1

Steps for configuring OpenAM as IdP (Identity Provider) and Shibboleth as SP (Service Provider)

Following are detailed steps for configuring OpenAM as IdP (Identity Provider) and Shibboleth as SP (Service Provider)

Versions used for configuration:
OpenAM (Identity Provider) – 9.0
Shibboleth (Service Provider) – 2.3.1
OS Version – Ubuntu 9.04
Apache version – 2.2.6

(1) Configure OpenAM with necessary Directory Service configuration on host1.
(2) Configure Shibboleth SP version with some basic configuration on host2.
(3) Generate and save Shibboleth SP metadata using URL – http://host2/Shibboleth.sso/Metadata
(4) Edit the Shibboleth SP metadata and remove all XML digital signature and the nodes.
(5) Copy the generated SP metadata on the IdP server.
(6) Go to “Common Tasks” section and configure “Create Hosted Identity Provider”. If you want to use it in production, make sure to have your credentials in the keystore, for proof-of-concept scenarios the keystore contains one test key.
(7) Add a new “Circle of Trust” name within “Hosted Identity Provider” and save necessary settings.
(8) Grab the newly created OpenSSO IdP metadata XML (you can use either ssoadm.jsp export entity command or access directly /opensso/saml2/jsp/exportmetadata.jsp?entityid=)
(9) Put the metadata in a location which is accessible through a web URL.
(10) Login to the OpenAM UI and go to the Common Tasks section.
(11) Click on “Add Remote Service Provider” link.
(12) Select the file option and upload the Shibboleth Service Provider metadata file.
(13) Select common attributes and finish the setup.
(14) Now edit shibboleth2.xml file on the Shibboleth SP server and do following configurations:
1. In the section, add the site information.
2. Under section, add a block with proper site configuration.
3. In the section, select type=“Chaining” and add entityID of the OpenAM, as that configured in OpenAM metadata.
4. In the section, add line like:

This configuration would be read when shib daemon is restarted.
(15) In the apache config file, include Shibboleth's apache configuration file available in directory /etc/shibboleth/
(16) Restart apache configuration.
(17) Restart shibboleth daemon.
(18) Check the shibboleth log-files to ensure that shibboleth daemon was able to load IdP metadata without any issues.

Testing:
(a) Test accessing a secure URL from Shibboleth SP server.
(b) Verify that client is redirected to SSO login URL of the IdP.
(c) Enter valid user authentication credentials and verify that client is redirected back to the Service Provider URL.

Special day (21st Dec, 2009)- MS Azure training program closing ceremony

21st Dec, 2009 was a very special day for me. I got acknowledged for voluntary contribution that I did by helping train 2009 pass-outs that did not had jobs / offers with them on Windows Azure. Microsoft Windows Azure was in Beta stage when this training program was planned by joint venture of Microsoft, Persistent, ICertis and PuneUserGroup in the month of July 2009. Microsoft Azure in the latest Cloud offering by Microsoft as PaaS (Platform as a Service), which allow developers to stay away from Infrastructure worries and only focus on development of application. Though this technology was new to me as well, but event coordinators arranged for Training the trainer program under which some initial jumpstart walk-through was given to trainers. Almost 150+ students attended the program regularly. After the training program, Microsoft had arranged for a contest in which students were suppose to develop an application using Azure and deploy it on to the test-accounts provided by Microsoft. I took this initiative to build onto my cloud knowledge of Amazon EC2 and adding another dimension of Microsoft Cloud offering. I was champion for 1st week and also took an introductory session of introducing Cloud + Azure to the students, also I acted as supporting volunteer during other weeks where sessions were held after office hours on weekdays and on weekends. After 3 and half months of contribution, my day became special because I got recognition for my efforts at the hands of Dr. Anand Deshpande who is the owner of the firm I am working at i.e. Persistent Systems. It is an achievement of a lifetime for me.

My 1st blog post

This is my 1st blog post putting it on my website ebadgujar.com . Let me take this opportunity to introduce myself -

I started my career as Electronics Engineer from Vishwakarma Institute of Technology (VIT), Pune in year 2000. After graduation, I got offer from L&T, but on joining got rejected because of not having first class in each of the semesters. I again returned to Pune and did crash course in Java Development. Soon I joined a startup company Interactive Web Solutions. There I got real IT experience of a life-time. My mentor Proshanto Mitra trained me with useful technologies and supported me quite a lot. To my bad luck it was a recession period and our company got merged into Balasai Net Pvt Ltd. There I worked as Systems Programmer and gradually also got exposed to Web server Administration. This was a useful experience for me and learning curve of a life-time. All the knowledge that I learnt gave a firm concrete base to my career.

After working for 2 years in 2003, I decided to move on and then soon joined Ensim India Pvt Ltd as Support Engineer. It was a nice work environment and got good experience of work-flow in product oriented company. It helped me grow up in hierarchy of career with good technical experience. But working in a product oriented company and doing repeatative tasks was not something I was looking for in my career.

In Sept 2004, I joined Persistent Systems Ltd (a Services oriented Company) as Senior Software Engineer taking care of Support Activity for one of its customers in Email domain. Last 5 years at Persistent have helped me bring stability to my career and also I grew up in the hierarchical ladder to be Technical Lead.