CVS Best Practices

CVS Best Practices Vivek Venugopalan

&MYEMAIL;

0.6 2002-09-10 vv Added content related to tagging and daily builds. Changed Linuxdoc URLs to tldp. Fixed stale links and added other corrections suggested by readers. 0.5 2002-08-25 vv Fixed some more errors in the document and added references to other CVS sources and some server side scripting 0.4 2002-03-10 vv Added new email address, Added an example flow to show how the practices help 0.3 2001-12-06 vv Grammatical errors cleanup 0.2 2001-11-27 vv Incorporated first round of feedback and some minor fixes 0.1 2001-11-20 vv Created CVS Best Practices This article explores some of the best practices that can be adopted while using CVS as the configuration management tool in your software projects. Introduction

Henry David Thoreau (1817-1862) Men have become the tools of their tools.

This article outlines some of the best practices that can be adopted when &CVS; is used as the configuration management tool in your software project. &CVS; (&CVSAB;) is an &OPENSOURCE; configuration management tool that is now being looked at seriously by many commercial organizations as a viable alternative to other commercial &SCM; tools. This spotlight on &CVSAB; has led to the inevitable question of best practices for deploying &CVSAB; as the backbone &SCMAB; tool for large software development projects. Having answered this question many times verbally as a bunch of gotchas on &CVSAB;, it was time to put down on paper some of the best practices that will work well for &CVSAB; based projects. This paper assumes that the reader is familiar with the fundamentals of software version control. Including features like branching, merging, tagging (labelling) etc., offered by modern version control tools such as &CVSAB; Further, This paper is not an introduction to &CVSAB; and its usage. There are excellent articles available on the net for the same. This paper assumes that the reader is familiar with &CVSAB; commands and is looking at deploying &CVSAB; in his or her organization. Some of the popular &CVSAB; related links that can provide &CVSAB; education are. The &CVS; site where current informaton about CVS is available. Including the &CVSAB; manual. Karl Fogel's book Open Source Development with CVS has GPL'd sections available online. Copyright Information This document is Copyright © 2001 Vivek Venugopalan. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license can be found in . This document may be reproduced and distributed in whole or in part, in any medium physical or electronic, as long as this copyright notice is retained on all copies. Commercial redistribution is allowed and encouraged; however, the author would like to be notified of any such distributions. All translations, derivative works, or aggregate works incorporating this document must be covered under this copyright notice. That is, you may not produce a derivative work from this document and impose additional restrictions on its distribution. Exceptions to these rules may be granted under certain conditions; please contact the author at the address given below. In short, we wish to promote dissemination of this information through as many channels as possible. However, we do wish to retain copyright on the document, and would like to be notified of any plans to redistribute the same. Disclaimer No liability for the contents of this document can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, the author(s) do not take any responsibility for that. All copyrights are held by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark. Naming of particular products or brands should not be seen as endorsements. You are strongly recommended to take a backup of your system before major installation and backups at regular intervals. New Versions The version number of this document is &DOCVERSION;. The latest version of this document can be obtained from My website The linux documentation project Credits The list of people who have provided information and correction for this paper in no particular order are. Jens-Uwe Mager Jorgen Grahn Thomas S. Urban Cam Mayor Sally Miller Niels Jakob Darger Feedback Feedback is most certainly welcome for this document. Without your submissions and input, this document wouldn't exist. Please send your additions, comments and criticisms to the following email address : &MYEMAIL;. Focus Areas The focus areas for best practice are Using <acronym>GUI</acronym> Tools The traditional interface available for CVS is the command-line client. There has also been a slew of GUI client applications that can talk to a &CVSAB; server. These GUI clients provide a point and click interface to the &CVSAB; repository. §ion2-useguiclient; This paper recommends using such GUI clients during the initial deployment of &CVSAB; in an organization. Developers typically use integrated development environments that have the CM tools integrated into them. These tools minimize the learning for the developers about the intricacies of &CVSAB; usage and instead allow them to be productive from day one. Developers who are accustomed to other CM tools will find the &CVSAB; command-line interface daunting. The adoption and usage of &CVSAB; can be improved by using GUI tools for &CVSAB; clients. GUI tools for &CVSAB; are available at http://cvsgui.sourceforge.net/. GUI interfaces are available for most of the popular platforms (Windows, Mac and Linux). In addition, on the Windows platform there is an SCC extension that allows integration of &CVSAB; as the configuration control tool with popular IDE. §ion1-devsandbox; The developer sandbox is where each developer keeps his or her working copy of the code base. In &CVSAB; this is referred to as the working directory. This is where they build, test and debug the modules that they are working on. A sandbox can also be the area where the staging build or the production build is done. Changes made in the work area are checked into the &CVSAB; repository. In addition, changes made in the repository by others have to be updated in the sandbox on a regular basis. The best practices related to developers sandbox are: §ion2-clockinsync; &CVSAB; tracks change to source files by using the timestamp on the file. If each client system date and time is not in sync, there is a definite possibility of &CVSAB; getting confused. Thus system clocks must be kept in sync by use of a central time server or similar mechanism. &CVSAB; is designed from ground up to handle multiple timezones. As long as the host operating system has been setup and configured correctly, &CVSAB; will be able to track changes correctly. §ion2-dontshare; Sandboxes have to be unique for each developer or purpose. They should not be used for multiple things at the same time. A sandbox can be a working area for a developer or the build area for the final release. If such sandboxes are shared, then the owner of the sandbox will not be aware of the changes made to the files resulting in confusion. In &CVSAB;, the sandbox is created automatically when a working copy is checked out for a &CVSAB; project using the cvs checkout {project-name} command. In very large projects, it does not make sense for the developers to check-out the entire source into the local sandbox. In such cases, they can take the binaries generated by the build team on a regular basis for all those components of the application that is not changed by them and only check-out the parts that are built by the developer. For example, in a Java project, the build team can keep the results of their last successful build in a standard location in the form of JAR files on the network file servers. Individual developers will use a standard classpath setup that has the network drives mounted on standard paths. Thus, the developers will automatically get the latest version of the files as required by them. §ion2-syncup; To gain the benefits of working within a sandbox as mentioned above, the developer must keep his or her sandbox in sync with the main repository. A regular cvs update with the appropriate tag or branch name will ensure that the sandboxes are kept up to date. §ion2-workinside; The sandbox can be thought of as a controlled area within which &CVSAB; can track for changes made to the various source files. Files belonging to other developers will be automatically updated by &CVSAB;. Thus the developer who lives within the sandbox will stand to gain a lot of benefits of concurrent development. This benefit cannot be achieved for work done outside a sandbox. §ion2-cleanupatcompletion; Make sure that the sandbox is cleaned up after completion of work on the files. Cleanup can be done in &CVSAB; by using the cvs release command. This ensures that no old version of the files exists in the development sandbox. As explained previously, pre-built binaries from the build team can be used to ensure that all the parts of the application are available to the developer without the need for a complete compilation in the sandbox. §ion2-checkin; To help other developers keep their code in sync with your code, you must check-in your code often into the &CVSAB; repository. The best practice would be to check-in soon as a piece of code is completed, reviewed and tested, check-in the changes with cvs commit to ensure that your changes are committed to the &CVSAB; repository. &CVSAB; promotes concurrent development. Concurrent development is possible only if all the other developers are aware of the ongoing changes on a regular basis. This awareness can be termed as situation awareness One of the bad practices that commonly occur is the sharing of files between developers by email. This works against most of the best practices mentioned above. To share updates between two developers, &CVSAB; must be used as the communication medium. This will ensure that &CVSAB; is aware of the changes and can track them. Thus, audit trail can be established if necessary. §ion1-serverconfig; This section deals with best practices for &CVSAB; server side setup and configuration. §ion2-scripting; Server side scripting refers to the ability to make &CVSAB; server execute certain scripts when an event occurs. A common script that helps is to verify that all cvs commits contain acomment entered by the developer. The process involves setting up the CVSROOT/verifymsg file to run a script when a file is checked-in. ------CVSROOT/verifymsg--------- #Set the verifymsg file to fire a script DEFAULT /usr/local/bin/validate-cvs-log.sh ------/usr/local/bin/validate-cvs-log.sh --------- #!/bin/sh # # validate-cvs-log.sh logfile # test that log message has some characters in it if [ `cat $1 | wc -c ` -lt 10 ] ; then echo "log message too short; please enter a description for the changes" exit 1 else exit 0 fi §ion2-notification; The &CVSAB; server can be configured to notify through e-mails in case of a commit happening. This can be used to verify whether commits are occurring during the course of a daily/release build. If such commits occur, based on the project policy, the commits can be ignored or the entire build automatically restarted. §ion1-branchmerge; Branching in &CVSAB; splits a project's development into separate, parallel histories. Changes made on one branch do not affect the other branches. Branching can be used extensively to maintain multiple versions of a product for providing support and new features. Merging converges the branches back to the main trunk. In a merge, CVS calculates the changes made on the branch between the point where it diverged from the trunk and the branch's tip (its most recent state), then applies those differences to the project at the tip of the trunk. §ion2-branchowner; The main trunk of the source tree and the various branches should have a owner assigned who will be responsible for. Keep the list of configurable items for the branch or trunk. The owner will be the maintainer of the contents list for the branch or trunk. This list should contain the item name and a brief description about the item. This list is essential since new artifacts are always added to or removed from the repository on an ongoing basis. This list will be able to track the new additions/deletions to the repository for the respective branch. Establish a working policy for the branch or trunk. The owner will establish policies for check-in and check-out. The policy will define when the code can be checked in (after coding or after review etc.,). Who is responsible to merge changes on the same file and resolve conflicts (the author or the person who recently changed the file). Identify and document policy deviations Policies once established tend to have exceptions. The owner will be responsible for identifying the workaround and tracking/documenting the same for future use. Responsible for merge with the trunk The branch owner will be responsible for ensuring that the changes in the branch can be successfully merged with the main trunk at a reasonable point in time. §ion2-tagrelease; As part of the release process, the entire code base must be tagged with an identifier that can help in uniquely identifying the release. A tag gives a label to the collection of revisions represented by one developer's working copy (usually, that working copy is completely up to date so the tag name is attached to the latest and greatest revisions in the repository). The identifier for the tag should provide enough information to identify the release at any point in time in the future. One suggested tag identifier is of the form. release_{major version #}_{minor version #} As one reader pointed out to me, a good practice here is to tag the release first. Checkout the entire codebase using the tag, and then proceed to go through a build / deploy / test process before making the actual release. This will absolutely ensure that what leaves the door is a verified and tested codebase. §ion2-branchatrelease; After each software release, once the &CVSAB; repository is tagged, a branch has to be immediately created. This branch will serve as the bug fix baseline for that release. This branch is created only if the release is not a bug fix or patch release in the first place. Patches that have to be made for this release at any point in time in the future will be developed on this branch. The main trunk will be used for ongoing product development. With this arrangement, the changes in the code for the ongoing development will be on the main trunk and the branch will provide a separate partition for hot fixes and bug fix releases. The identifier for the branch name can be of the form. release_{major version #}_{minor version #}_patches §ion2-bugfixbranches; This practice extends from the previous practice of creating a separate branch after a major release. The branch will serve as the code base for all bug fixes and patch release that have to be made. Thus, there is a separate repository sandbox where the hot fixes and patches can be developed apart from the mainstream development. This practice also ensures that bug fixes done to previous releases do not mysteriously affect the mainstream version. In addition, new features added to the mainstream version do not creep into the patch release accidentally. §ion2-patchesfrombranches; Since all the bug fixes for a given release are done on its corresponding branch, the patch releases are made from the branch. This ensures that there is no confusion on the feature set that is released as part of the patch release. After the patch release is made, the branch has to be tagged using the release tagging practice (see ). §ion1-chgpropagation; Change propagation practices explore how changes made to one version of the application are migrated to other living versions of the application. §ion2-mergebugfix; After each release from a branch, the changes made to the branch should be merged with the trunk. This ensures that all the bug fixes made to the patch release are properly incorporated into future releases of the application. This merge could potentially be time consuming depending on the amount of changes made to the trunk and the branch being merged. In fact, it will probably result in a lot of conflicts in &CVSAB; resulting in manual merges. After the merge, the trunk code base must be tested to verify that the application is in proper working order. This must be kept in mind while preparing the project schedule. In the case of changes occurring on branches for a long period, these changes can be merged to the main branch on a regular basis even before the release is made. The frequency of merge is done based on certain logical points in the branch's evolution. To ensure that duplicate merging does not occur, the following practice can be adopted. In addition to the branch tag, a tag called {branch_name}_MERGED should be created. This is initially at the same level as the last release tag for the branch. This tag is then moved after each intermediate merge by using the -F option. This eliminates duplicate merging issues during intermediate merges. §ion1-softwarebuild; This section deals with the best practices for software builds. Build is the process of creating the application binaries for a software release. They are done in a periodic manner by the build teams to provide baseline binaries for daily work. §ion2-bebo; (<acronym>BEBO</acronym>) A variation of this adage has been around in the &OPENSOURCE; community called "Release Early and Release Often" for quite some time albeit for a different reason. BEBO helps a development team identify issues that can arise from checking in the wrong files. BEBO will address integration issues at the application level that might have slipped passed individual developer builds. It will also improve the team morale when they see a working version of the application. Builds must be done on a regular basis. There should be a dedicated resource(s) assigned to do the same. The entire project team must be trained to view the daily build as an important activity and not as a chore. Builds must be completed without any failures on a regular basis. Build failures must be a rare event and should be treated with utmost seriousness. The project team should ensure that successful builds are top priority on their agenda. The seriousness can be emphasised by setting up a penalty for breaking the build. Each build can be tagged in CVS using a standard naming convention. This can help developers checkout a working version of the entire system from daily builds for local development. §ion2-automate; Another key practice for software builds is to automate the build process completely. The automation process must also include automatic retrieval of the right source files from the &CVSAB; repository. This ensures that the build process is completely repeatable and consistent. In addition, the chances of a build with the wrong version of the application source files are reduced to a large degree. By automating the build process, the task of building often becomes less burdensome. §ion2-ensurecheckin; This adage sounds trivial at first but this problem is very common even with experienced development teams due to oversight. The problem of oversight cannot be easily addressed since the onus is on the individual developer to ensure that his or her file has been checked in. This practice should be drummed into the team in the form of training and pre-build announcements to ensure that the right version of source code is available in the repository. Automated build process as explained above will help in catching this problem to a certain degree since they will automatically take the source code from the &CVSAB; repository and perform the software build. Any missed items will surface during the build process itself (makefiles etc.,) or during the regression testing of the product (older version of the file checked in). A penalty based system can be setup to handle wrong check-in. Having a kitty for a post project party to which each person who makes a wrong check-in will contribute a fixed amount will act a good penalty system. §ion1-instprocess; Here we will look at the best practices for institutionalizing &CVSAB; usage in the organization. §ion2-chngmgmt; All organizations must implement a good Change management process (CMP). A good CMP will define how changes are received, recorded, tracked, executed and delivered. &CVSAB; provides version control for your project. Change management addresses the bigger picture of how enhancements and bugs are received, tracked and closed. &CVSAB; will play a smaller but a very important part in this entire picture. With a formal change management process in place in the organization, tools such as &CVSAB; will be looked at as aiding this process instead of acting as a general development overhead. Change management is quite a vast topic that cannot be done justice here. Please look up other sources of information on change management. §ion2-objectives; To institutionalize &CVSAB;, it can be made as part of the performance objectives for the developer to use &CVSAB; in the project. In addition, it can also be made as part of the objective for the project manager to deploy &CVSAB; in his or her project. Compliance of this can then be reviewed as part of the appraisal cycle for the employee. §ion2-metrics; &CVSAB; usage metrics can be collected in terms of percentage of deployment in the organization, project size handled etc., This information will spur other line managers and program managers to look at &CVSAB; as a tool that will aid them in their daily operations. Best Practices in Action The best way to explain the need for these best practices is by putting together an example of a real world project scenario and show how exactly will these best practices fit into the bigger picture. Also, a lot of readers have told me that the sections on and will require examples for better explanation. Listening to the readers is a Good Thing so I have put together a particular project scenario and then create a series of events to show how the best practices, if followed, would help is making operations smoother. Inception Consider a software project where version 1.0 has just been put into production and everyone is done celebrating. The next step is to start working on the new features of the subsequent release. Also, the users of the system have started to use it full-time and bug reports of various levels have started to come in. Before jumping into new enhancements or bug fixes, the best practices for should be followed. Few of the important practices are and . These practices will effectively established two development environments, one for regular enhancements and the other for bug fixes and minor enhancements on the last release. Let us assume that the release was tagged as release_1_0 Then the branch was created with the branch name release_1_0_patches Development and Delivery Now, we are ready for business. Let us examine the bug fixes and enhancements track. Assume that there are three bugs of which two are of a high priority that should be fixed right away (possibly within a week) and the third can be delivered after some time (say after 4 weeks). In the middle of this schedule there is a regular release scheduled in three weeks. Considering that we have a busy month ahead, let us see how exactly we can use the Best practices to ease the days ahead. The timeline for the various release in the next month looks like this. Fix Enhancement Fix Today Release 1 Release Release 2 |_______|______________|_________| Time --> We have two teams, one working on the bug fix branch and another team working on the features for the next release on the main trunk. These teams must make sure that they start out with the right version in their sandbox. The bug fix team will check out using the command line cvs checkout -R -r release_1_0_patches {project name} The team that is working on the next release will use the command line cvs checkout -R {project name} As soon as the bug fix team completes the two top priority bugs, they will update, verify a successful build and commit their changes to the bug fix branch using the command line cvs update -R -r release_1_0_patches {module name} The team should perform a build at this point to verify that the update did not break any code on the branch. Once the build is successful, the branch should be committed back into the repository. cvs commit -R -r release_1_0_patches {module name} : On a daily basis, each developer will check in code to &CVSAB; and to ensure sanity of code, daily builds on the bug fixed branch will be undertaken by checking out from &CVSAB; on a clean environment and completely rebuilt. These daily builds can be tagged in &CVSAB; using the following naming convention build_1_1_yyyymmdd : for the branch build_2_0_yyyymmdd : for the trunk The regular process of build-test-fix is followed to make a version ready for delivery. The tag will help developers checkout a working copy of the latest build as and when necessary. When the source code is released to the outside world, two practices have to be followed. : This ensures that the bug fix release is tagged correctly and so can be traced out at a later point in time if necessary. : This ensures that the bug fix is merged back into the main trunk ensuring that all future releases is a truly cumulative delivery. Conclusion These best practices are meant to help software teams get a head start on using &CVSAB; for their development. The ideas presented here have to be constantly reviewed and evolved. I would like this to be a growing and evolving document. Please send your comments and ideas to &MYEMAIL; &GFDL-FILE;