1

Can anyone provide any pointers on the following problem, basically we’re using a CMS (currently Drupal) and all changes take place in production by the editors of the site, this means that our Dev/QE/Stage environments are always out of sync with production ie behind it instead of in front. Obviously in order to get to continuous integration/delivery this cannot continue.

There are a few ways to fix this, one of which is to have a “Gold “environment that’s not in production where all editorial changes etc take place and then push this forward to QE/Stage, then out to Prod i.e…

Dev -> Gold -> QE -> Stage -> Prod

This way we can always tell what’s in production and we can lock it down etc. and we have an up to date QE environment that has exactly what’s going to get pushed.

The problem is though that doing this in Drupal looks like it’s going to involve a ton of complexity because configuration is stored in the Database and editors can make changes that we have no real control over and we would need to build or install a bunch of modules to effectively “diff” the database and push the SQL scripts forward, something screams error prone to me here and I’m loathe to start writing tools to do this.

Can anyone point me at any resources that cover this for any CMS, we’re not tied to Drupal, in fact a simpler CMS would appeal to us quite a bit if it solves this problem or at least makes it very easy.

Mikko Ohtamaa
  • 82,057
  • 50
  • 264
  • 435
Harry
  • 11,298
  • 1
  • 29
  • 43
  • This might help you out: http://stackoverflow.com/questions/2893774/working-with-version-control-on-a-drupal-cms-project – Trendy Jul 26 '13 at 23:30
  • The point of a CMS is to *separate* content from design and code, so that you *can* do continuous integration without having the exact same content. In fact, a continuous integration test suite typically creates it's own content from scratch on each run. – Lennart Regebro Jul 27 '13 at 02:48
  • @Lennart: I understand this, but some features require content ie links to new pages, checkboxes etc. These cannot be tested by QE unless the content is there in their environment, nor can you write the tests unless the content is there to test with. – Harry Jul 28 '13 at 22:22

2 Answers2

2

It is a feature of many CMS:es that configuration is partly stored in the database, and that this blurs the line between what is development, what is configuration and what is content editing. This is largely an effect of the flexibility of the CMS, and it is desirable, as it leaves it up to the site to decide.

But if you do not create such a distinction in your site, it means all your content editors need to be proficient in all details of the CMS, as they can change configuration, and even in some CMS's like Plone, do development through-the-web.

Therefore you need to make sure that each person only have the rights to change what they need to change. This in turn requires a highly flexible and detailed permission system, like Plone's.

An example of a typical site with problems in this area is where you start out with just one or two content editors who are also managers, and they use accounts with full permissions, typically they use the admin account to do content editing. When more content editors arrive they are just given the admin password to do a small change, and then grow into having more editing responsibilities, but continue to use the admin account. As a result, everyone who edits the site can do everything, and you can not even see who made the change afterwards.

In a well managed site, the content editors can not edit or change anything that is required for the site to function. Say that you have a site with product categories, where each category also needs an overview document, then the editors should not have the right to delete the overview document, as doing so would break all little category links. You will need to identify the editing/managing roles (there may be many) and make sure each role has exactly the permissions that it needs, and then assign roles to the accounts.

If you end up in a situation where it is hard to give out permissions and editors tend to have too few or too many rights, that is an indication that you have properly failed to separate content from site development, and might need to look into restructuring some of the code and/or content. This will of course happen faster on simple CMS's without detailed permission systems, which may be an indication that you have outgrown the system (which brings us to the main sales point of Plone IMO: You are highly unlikely to ever outgrow it as a CMS, while it is still easy to get started with).


You seem to want continuous integration tests for the finished web site, including it's content. This is both a good and a bad idea. Bad first:

I think having continuous integration tests for content is not feasible and slightly misguided. It is also plain impossible, if you allow editors to edit tested content on the live site, as they can break it at will at any time, see above.

But having testing of the finished site is not necessarily a bad idea at all. I wouldn't have it as a part of the continuous integration tests, though. This is because those tests are typically triggered by changes in the code.

My suggestion would be to have a setup split up into development and editing. The development environment is the normal environment of local development servers and continuous integration setup, plus a test server where the editors can test new features.

The test server would get the contents copies over every time you deploy to the test server, which typically will be done a couple of times per development iteration. The continuous integration tests should not have tests that depend on content editable by normal content editors, but those tests should instead create test content from scratch on each run as a part of the test suite.

You then also need a staging server, which is the server on which content editing is done. It receives software updates once per development iteration, or when the editors approve the functionality on the test server. On this server, content editing is done, and you have a separate test suite for this site, testing the integrity of the content, including running reports for broken links etc. Once the editor in chief is happy with the content changes, and the content tests pass, then the content is deployed to the production server and made public. Testing can be done both on demand and periodically, say once a day.

This type of setup of course only makes sense for big sites where you have many content editors, perhaps ten or more. But in those cases it definitely makes sense. In smaller cases you can get rid of the staging server and deploy functionality directly to the production server, and run the content testing once a night to trap errors reasonably quickly.

Lennart Regebro
  • 167,292
  • 41
  • 224
  • 251
1

In Plone, database change patches can be deployed through GenericSetup mechanishm (custom format XML files).

When you install a new version of your addon, you have migratable settings bundled with your code in the version control system as GenericSetup files. After code has been installed on the server, Plone has been restarted, from Plone control panel you reinstall/upgrade your addon. This reads all database changes from XML files, performs custom Python scripted changes and modifies for site database for new addon version.

However, this means that you must make all your changes available in GenericSetup XML files and not do them directly on the site. Plone offers some export tools to export through-the-web changes to XML, but the support is abysmal.

Mikko Ohtamaa
  • 82,057
  • 50
  • 264
  • 435