8

We are maintaining a large website based on Lotus Notes, running on Domino Server 8.5.3. Recently we have been fed up with the lack of source control in our project, so we thought we would try to Git things up a little. But how to do this right?

For various reasons that I will not get into here, we cannot run our applications locally on the development PCs. We can only run them on the development, staging and production servers. (Local Domino server running on the development PCs is not an option, unfortunately.)

This figure illustrates the current situation:

Current situation. Oh, the horror...

In production (and staging) there are multiple web sites that all have the same functionality. They are basically the same site with the same design, but with different content. Each site consists of a number of Notes databases (News, Archive, Discussion etc). The design of these databases is the same for all sites, so they all inherit their design from the same set of master templates (NewsTemplate, ArchiveTemplate, DiscussionTemplate etc).

A set of replicas of the master templates resides on the Development server. When we deploy new functionality, the required code changes are added to the templates on the Dev. server (more details on this below). Then these templates are replicated to the Staging server for testing, and finally to the production server. This part works quite well.

The problems may be found in the lower part of the figure. We are currently two developers, and we both work in the same set of dev. databases (News, Archive, Discussion etc) on the dev. server. Naturally we keep tripping over each others' work, which is quite frustrating. When it is time to deploy, the code that is to be deployed is selectively copied/pasted from the set of dev. databases to the dev. templates. This is done manually, and thus the risk of mistakes is high. We cannot simply replace the design of the templates with that of the dev. databases, because the dev. databases at all times contain code that is under development. So we must take great care to only deploy code that is "finished". Also, we have no history of changes. This is a horrible, horrible, horrible (add as many horribles as you see fit) way to do software development, and we know it.

Now, using Git, local replicas, database copies or whatnot, what would be the best way to achieve a more streamlined development and deployment scheme?

Øystein Grande Jaren
  • 500
  • 1
  • 5
  • 15

1 Answers1

5

This figure illustrates a possible solution to the problem:

Envisioned deployment architecture incorporating Git for version control

As explained here, each developer should have his own COPY (not replica) of the database/template (or in our case set of templates) to develop in. So on each dev. PC there is a set of templates, and each template is synchronized with an on-disk-project. The set of on-disk projects will be under source control with Git.

The developer will not be able to run the application locally, so he needs a set of test databases on the dev. server. There will be one such set for each developer. The test databases will inherit their design from the templates on the dev. PC. Thus, in order to test his code changes, the developer must refresh the design of his designated set of test databases with that of his templates (which reside on his PC).

The workflow for each developer will be as follows:

  1. Make code changes in the set of templates on developer's own PC. Test continuously by refreshing the design of the test databases on the dev. server. Sync with on-disk-projects, commit and branch "at will".

  2. When developer is ready to push to remote Git repo:

    a) Pull changes from remote Git repo.

    c) Push changes to remote Git repo. Sync databases with on-disk-projects.

    d) If currently on develop branch: Replace design of the set of templates on the dev. server with the design from the set of templates on the dev. PC. This will ensure that the develop branch of the remote Git repo is always consistent with the templates on the dev. server, even though there is no direct connection between them.

  3. From here changes can be deployed to staging and production by replicating the templates from the dev. server. Before deploying to production, the develop branch should be merged into the master branch, so that master always reflects the current production version.

If anyone has any comments, additions or objections, I would love to hear them.

Edit: To shorten the feedback loop in point 1 above, development can be done directly in the testing databases on the dev. server. This removes the need to refresh the design of the testing databases every time you need to test your changes, though you must be careful to copy all changes back to the dev. templates at regular intervals to keep your code safe and keep the Git repo updated. This can be done pretty quickly by doing an element-by-element compare of the database and the template. This also serves as an opportunity to review your changes before making a commit, which is a very good habit.

Another option for copying the changes back to the template is to make the testing database on the dev. server a master template and temporarily set the template on the dev. PC to inherit the design from that master template. Then the changes can be copied with "Refresh design" on the dev. PC's template, after which you can remove the inheritance again.

Community
  • 1
  • 1
Øystein Grande Jaren
  • 500
  • 1
  • 5
  • 15
  • That is a good concept. We follow the same pattern on Git - however, we have installed local development servers on each developers computer (in a VM) to run and test the application ;-) – John Dalsgaard Sep 24 '13 at 09:30
  • Yeah, I wish we could use local development servers. Unfortunately our customer will not allow it. – Øystein Grande Jaren Sep 24 '13 at 09:49
  • I hope that you are working with branches. So developers can work on theire changes and test it. And after that the developers branch can be merged into the development branch, to bring all together. – Christian Güdemann Sep 24 '13 at 09:49
  • Yes, that is the idea, and something we have sorely missed up until now in this project. Both developers working in the same code base with no possibilities of branching (nor any change history for that matter) is a nightmare. – Øystein Grande Jaren Sep 24 '13 at 10:53
  • This looks very interesting. I'd love to see an eval of this when the project is over! – Mikael Andersson Wigander Sep 24 '13 at 18:27
  • 2
    You definitely want to look at GitFlow (e.g. http://blog.sourcetreeapp.com/2012/08/01/smart-branching-with-sourcetree-and-git-flow/, http://nvie.com/posts/a-successful-git-branching-model/ and https://www.atlassian.com/git). I totally agree with Christian that you want to use branching (features in GitFlow) for each of the developers. – John Dalsgaard Sep 25 '13 at 08:02
  • Good point, @JohnDalsgaard. Tools like SourceTree are unfortunately unavailable to us, because the PCs we are working on are locked down tighter than Fort Knox (no .NET framework for us), but we will definitely use a branching model along those lines. – Øystein Grande Jaren Sep 26 '13 at 11:58
  • This is excellent - but it does mean I'll need a local development server. Does anyone know how to set one up correctly? I haven't found any good sources yet. – Andrew Magerman Sep 09 '14 at 14:08
  • @AndrewMagerman You do not need a local Domino Server. As I mentioned, we do not have local Domino server on the development PCs (ie. we cannot run the web sites on the development PCs). If we had that, it would be much easier to continuously test while developing. Instead, we have to frequently copy the design from the templates on dev PC over to the test applications on the dev server in order to test the changes, or alternatively do development directly in the dev server applications and routinely copy the changes back to the dev PC templates to keep the Git repo updated. – Øystein Grande Jaren Oct 17 '14 at 11:38
  • A little late to the party, but still: How do you get to manage concurrent edits on the same files - e.g. an Agent or worse, a Form? – Rodrigo Gama Jun 08 '15 at 15:56
  • @RodrigoGama: We were just two developers mostly sitting in the same room, so we managed to avoid concurrent edits most of the time. Probably not the answer you were hoping for, I know. – Øystein Grande Jaren Jun 11 '15 at 08:32