Metrics on your way : What do you when you want to see statistics of you project (graphite+statsd)

Okay so you have project working spik and spank .. but now you want to monitor the health of your project . you want to know how much data is it handling? how well is it handled? how long does this call take? how many success and how many failures? e.t.c e.t.c

Basically its like getting the health stats of your project .. so how do you do it ?

These are the phases my project saw :-

1. we all start with good old pint statements :)
2.  at some point they have to be removed .. so introduced logging.
3.  big log files were generated .. we needed better understanding of all this data.
4.  Splunk(www.splunk.com) cam to rescue .. feed it any log file and put in proper   queries and it will give you nice data. graphs., trends e.t.c
5. Now if  splunk and big log data file is the concern .. I introduce a database (sqllite) ..did some smart queries and displayed is neatly on a django dashboard. worked as a charm ( i still love this option) and looked very professional but then scalability issues alas!
6.  The came "statsd+graphite" ..Generate counters/stats using stasd. Run statsd, configure and feed it to graphite ..run graphite and watch the magic.

Here are very nice instructions to " installing and configuring statsd and graphite" thanks digital ocean "https://www.digitalocean.com/community/tutorials/installing-and-configuring-graphite-and-statsd-on-an-ubuntu-12-04-vps"

I used the python "statsd" library and to use it was as simple as that :-

>>> pip install statsd
>>> import statsd
>>> c = statsd.StatsClient('localhost', 8125)
>>> c.incr('foo')  # Increment the 'foo' counter.
>>> c.timing('stats.timed', 320)  # Record a 320ms 'stats.timed'.

one issue which i hit during the installation and configuration part was a twisted error of "unknown command : carbon-cache"

more about it here http://stackoverflow.com/questions/27951317/install-graphite-statsd-getting-error-unknown-carbon-cache?noredirect=1#comment44298261_27951317

I was able to bypass is by manually deleting twisted ( i was not using twisted anywhere)

when i ran graphite locally i was abel to see my stats where and graph them.

Pretty cool!