22

What will I need to use Etsy's Statsd in a Windows Environment? My intentions are to create a .net client to use Statsd.

Michael Dillon
  • 31,973
  • 6
  • 70
  • 106
Christopher
  • 371
  • 1
  • 4
  • 11
  • 1
    The article says it's a node.js daemon, so I'm assuming the first step would be to build node.js in Windows. See https://github.com/joyent/node/wiki/Building-node.js-on-Cygwin-(Windows)?view Have you done that much, at least? – Dan Esparza Mar 25 '11 at 21:36
  • Yes my problem more stems from installing graphite. – Christopher Apr 07 '11 at 20:52

7 Answers7

15

I have statsd+graphite running in my Windows environment using the C# client NStatsD.

Here are my notes for getting the Linux VM setup:

Note: I know enough Linux to be dangerous but am otherwise a noob and could be doing something unwittingly horrible.

  1. Install Ubuntu Server 12.04. I used VirtualBox for dev and then later EC2 for prod.

  2. Download graphite-fabric to your home folder. This is a script that will download, compile and install graphite and statsd. It expects a clean box and uses nginx for the web server.

    sudo apt-get install git

    git clone git://github.com/gingerlime/graphite-fabric.git

    cd graphite-fabric/

  3. Install prereq's for fabric

    sudo apt-get install python-setuptools

  4. The next steps are a download, compile and install which can take some time. It is worthwhile setting a keep alive on any putty ssh session before continuing.

  5. Now install as per gingerlime's instructions in the README.md - including the requirements section.

  6. Install statsd as per gingerlime's instructions.
  7. Reboot
  8. Execute netstat -nulp and observe 8125 is in use to confirm statsd is listening.
  9. Check carbon is running tail /opt/graphite/storage/log/carbon-cache/carbon-cache-a/listener.log. If it isn't, try sudo /etc/init.d/carbon start

Now you have your server running, try throwing some counters at it with the NStatsD client.

Timezone fix:

This will fix graphite to graph times in your local zone

  1. cd /opt/graphite/webapp/graphite
  2. sudo cp local_settings.py.example local_settings.py
  3. sudo chown www-data:www-data local_settings.py (check with ls -l that permissions look right)
  4. sudo pico local_settings.py Set TIME_ZONE to something like Australia/Sydney. Discover what timezones you can use in /usr/share/zoneinfo/
  5. Save and restart the box (not sure how to make it pick up the change without restart)

EC2 Notes

root is disabled on EC2. Fabric prompts for a root password which you don't have. Use the -i keyfile argument with fab to give it your ssh keyfile instead.

VirtualBox Notes

VBoxVMService was handy to automatically run the VM as a service in my Windows dev environment.

fiat
  • 15,501
  • 9
  • 81
  • 103
7

TL;DR:

  • statsd.net is a scalable statsd clone with lots of cool extras, and designed for windows-centric environments. First production release is on the 29th of May, 2013
  • statsd-csharp-client is a lightweight statsd and statsd.net client, available for .net 3.5, 4.0 and 4.5 via nuget.

Full Disclosure: I'm the author of these two projects and I wrote them because I couldn't take etsy's statsd service to production - my organisation isn't ready to deploy and support nodejs-based services.

I had the same problem in my organisation - we're a windows-centric environment that wants to collect stats from all over the world into a single graphite repository. I had initially thought of using etsy's statsd but my company is not yet ready to roll with nodejs services in production. Along the way I found that having my own aggregation service meant I can do lots of interesting things like add memcached support for scalability, add new aggregators and so on.

The service is usable in console mode right now, and will be ready for production use from the 29th of May 2013 onwards.

user141682
  • 432
  • 6
  • 9
5

We've re-built Graphite/StatsD in pure .NET. It is currently being used in production environment, processing around 600M datapoints daily.

Statsify:

enter image description here

Anton Gogolev
  • 113,561
  • 39
  • 200
  • 288
  • This looks nice. I looked over the wiki but couldn't find any mention of support for output plugins. I may have overlooked, but is that supported? For example, the use-case I am looking at is: sending collected metrics to various different backends. – ayaz Nov 28 '15 at 06:37
5

Your best bet is to have a Linux server with statsd and Graphite installed. You would then just need to write some C# code to make the UDP call to get the metric into the system.

[UPDATE (6/23/2014): I came across a statsd/Graphite clone called statsd.net that looks promising but largely lacking on the Graphite side of the equation. I still think the best bet is to use the actual statsd/Graphite projects since any clone is necessary playing catch-up.]

bbrown
  • 6,370
  • 5
  • 37
  • 43
  • 1
    github @salerth has had a pull-request for a C# client example into the statsd master a couple of months ago - see https://github.com/etsy/statsd/pull/21. – Peter Mounce Apr 10 '12 at 12:53
  • 3
    Hi @bbrown, I'm the author of statsd.net and can tell you that we are feature complete, save for histograms. Statsd.net is currently in operation at a number of sites that run graphite. One such site has statsd.net running at 15 different hosting locations globally, capturing in excess of 220 million metrics a day. I'm not sure how you came the conclusion that it's lacking on the Graphite side - can you help me understand why you thought that? For the sake of completion statsd.net also supports server chaining, MS SQL staging areas, librato.com output and lots more. – user141682 Sep 03 '14 at 14:12
  • 1
    I did not see it as a replacement for Graphite. It looks to match features with StatsD and can act as a sort-of replacement for Carbon (by writing out to SQL Server). But it doesn't have anything else of Graphite. Graphite, from what I can gather, does not work on Windows. So for the original questioner, statsd.net would not be a sufficient replacement. No need to get defensive and downvote. – bbrown Sep 03 '14 at 21:07
  • 1
    And if you think statsd.net is a feature-complete replacement for Graphite, then you're wrong. It's missing functions, graphing, and self-contained storage. (I don't think you'd claim that it's a replacement; it's nowhere on the GitHub page.) – bbrown Sep 03 '14 at 21:11
  • 2
    The OP's question was "What will I need to use Etsy's Statsd in a Windows Environment?" - and statsd.net is indeed a suitable replacement for this. I agree, Graphite is not able to run on Windows, but that not what he was asking. When you said lacking on the Graphite side, I thought you were referring to the output capabilities of statsd.net. And, since we're gentlemen, let's shake and make up. – user141682 Sep 18 '14 at 12:22
  • 1
    Actually, his [subsequent comment](http://stackoverflow.com/questions/5436606/using-etsys-statsd-in-a-windows-environment/6285981?noredirect=1#comment6358834_5436606) (left prior to my answer) suggests that Graphite was the issue he was facing. – bbrown Sep 18 '14 at 19:56
2

There is quite a few open-source StatsD client implementations available in different Language including C#.NET. Etsy provide one on their github examples, also check out AppFirst version of statsd_clients.

Leonmax
  • 121
  • 1
  • 7
2

Current statsd and nodejs versions allow you to run statsd on Windows without amendments. statsd package.json file already contains scripts to install and uninstall it as Windows service.

The installation procedure would be:

  • Install nodejs for Windows
  • Create a directory like C:\StatsD
  • Open a console window and do cd \StatsD
  • Then do npm install https://github.com/etsy/statsd.git
  • In node_modules\statsd directory create your own config.json
  • In the console window do cd node_modules\statsd and npm run-script install-windows-service

However, Graphite only runs on Linux.

Alexey Zimarev
  • 17,944
  • 2
  • 55
  • 83
1

You need a statsd server to connect to.
You also need a client library to connect to it, e.g. this one and the nuget package of it.

Anthony
  • 5,176
  • 6
  • 65
  • 87