3

We want to set up an easy mechanism for engineers to throw experimental code and features into our app, in order to try out new things for a sub-group of users, do A/B testing on different app behaviors, and generally offer a sandbox environment where developers can go crazy without impacting the main production code. Some of our requirements for Experimental Features (EF) are:

  • Adding EF's should be dead simple for anyone with basic familiarity of Rails, JS/Coffee and our app(s)
  • EF code should live outside of the production code ("on an island") as much as possible.
  • EF code should not grow cancerous tentacles into the production code, i.e. stay loosely coupled as much as possible.
  • It should be clear at first glance what is EF code and what is not.
  • EF code does not have to uphold full organizational policies of TDD, UX, etc. In fact quick and dirty experimentation is a goal of this, we don't want creativity and enthusiasm hampered by process and policy. Only is an experiment is deemed successful (through user testing), do we want to spend the effort to bring it up to all the policies.
  • EF features can be turned on/off on a dashboard site, and can be rolled out to specific users.
  • Developers working on production code should not be burdened in any way to deal with experimental code--ideally full separation. Experiments may break if production code is updated, and that's preferable, than forcing production code devs to keep all the experiments up to date. It's up to the experimenter to keep the experiment working.
  • Our system is built on Rails for the server and has a rich client-side app written in CoffeeScript with Knockout.js and Backbone.js. EF's may touch Rails code (controller actions, routes, models), view templates, CoffeeScript code (data bindings, jQuery templates, models and view models, etc), CSS/SCSS, and the mechanism should permit all of these to be sandboxed.

We're not so concerned at the moment about the measuring process for split testing or multivariate testing, this is a solved problem with tools like the Vanity gem, or commercial solutions like KissMetrics and MixPanel. We're more concerned with how to create a setup that solves the separation, merging, and maintenance problems. Do such tools for Rails exist?

Wolfram Arnold
  • 7,159
  • 5
  • 44
  • 64

1 Answers1

1

There are Rollout and Degrade gems for that if I understand you right.

VitalyP
  • 1,867
  • 6
  • 22
  • 31
  • Thanks, I didn't know about these. But at cursory glance they only address a small slice of the problem. The big problem is how to not clutter your code with if/else branches all over the place. Degrade addresses this for outside services, but that's not our main problem at present. I'm looking for something more general that can deal with HTML, JS, Ruby, etc. and specifically addresses the maintenance issue. – Wolfram Arnold Mar 29 '12 at 20:06