< Back to Webtools

Gotchas

• Always update your other code to the latest for that branch before you start your production routine
• Don't ever submit a log without a description or corresponding bug number (if applicable) -- blank changelogs should be avoided
• Talk to your team
  • Don't duplicate effort
  • Be aware of other patches in progress
  • Talk to each other to ensure that your patches won't cause regressions
  • Run the usual smoke tests and (ideally) some sort of automated script that can spot SNAFUs immedeately
• Don't file IT requests for production updates every hour, and don't email them every 5 minutes (or text-message them either) -- your best bet is to group your updates and push them weekly.

General Tagging Rules (quick version)

• Whatever is checked out in staging always has the _STAGING tag on it at all times
• Whatever is checked out in production always has the _PRODUCTION tag on it at all times
• Any previous _PRODUCTION tag should be tagged w/ an RTM_DATE tag before moving on to the next _PRODUCTION tag

Tagging for Staging

• Check in all the updates/patches you're going to stage.
• Add the STAGING tag to the code you want to stage:

[morgamic@khan aus]$ cvs up -dP
[morgamic@khan aus]$ cvs ci -m 'some message'
[morgamic@khan aus]$ cvs tag -F AUS2_STAGING

Tagging for Production

• Check in all the updates/patches you're going to check in.
• When you're ready to do a release-to-production rollup, make sure you have a current copy of the source, and tag is with the current date, including the hour/minute:

[morgamic@khan aus]$ cvs up -dP
[morgamic@khan aus]$ cvs tag -f AUS2_RTM_200608081426

If on a Unix box, you can use the following trick (preferred):

[morgamic@khan aus]$ cvs tag -f AUS2_RTM_`date +%Y%m%d%H%M`

• Find out the diffs between the RTMs; to do this, run:

[morgamic@khan aus]$ cvs diff -rAUS2_PRODUCTION -rAUS2_RTM_200608081426

• When you're SURE that this is the patches are correct/make sense, you need to tag it as PRODUCTION:

[morgamic@khan aus]$ cvs tag -F AUS2_PRODUCTION
cvs tag: Tagging .
T README
T htaccess.dist
T index.php
cvs tag: Tagging inc
T inc/aus.class.php
T inc/config-dist.php
T inc/init.php
T inc/patch.class.php
T inc/update.class.php
T inc/xml.class.php
[morgamic@khan aus]$ cvs ci

Note: that if you are starting this standard on a project that hasn't had an RTM tag previously, you will have to do it twice -- once for what's already out there, with a date approximation, and once for the one you're pushing out.

• Confirm that the PRODUCTION tag matches the RTM tag you created

[morgamic@khan aus]$ cvs diff -rAUS2_PRODUCTION -rAUS2_RTM_200608081426
[morgamic@khan aus]$ # Should be no output.

When to push

• Only initiate a request if you are sure you are not causing a seriuos regression
• For major updates, you should contact major parties and discuss the update -- at least briefly -- so they are aware. When in doubt, just email everyone.  :)

Requesting an update in production

• File an IT Request when you are sure you are ready to go
• Include in the request the previous RTM tag, so an easy rollback can occur! Also include any other rollback instructions.
• Assign the appropriate severity -- note that critical or blocker pages the oncall.

Actually pushing updates

• Updating production should be a simple step, and should always be consistent if we follow this procedure:

[root@do-stage01]$ cvs up -dP -r AUS2_PRODUCTION

• Most apps are updated on osadm01
• AUS2 is a special case, and the command for pushing is done on do-stage01:/opt/aus2/app

After the push

• Always be available after updates are pushed
• Tell the community you have made an update -- blog, planet, etc.
• Keep an eye open for what's going on, file an IT request or page sysadmin for a rollback if necessary

Rolling back

• Sysadmins, if the developer did things right, you should be able to rollback by using:

[root@khan aus]$ cvs up -r AUS2_RTM_[date of original release]

For example:

[root@khan aus]$ cvs up -r AUS2_RTM_200608081426