Fork me on GitHub

n. Slang a rough lawless young Kuali developer.
[perhaps variant of Houlihan, Irish surname]
kualiganism n

Blog of an rSmart Java Developer. Full of code examples, solutions, best practices, et al.

Saturday, June 5, 2010

Liquibase Testing with SVN Keywords

Abstract

In KIS of the Dragon, I documented using the liquibase rollback functionality for testing. Since then, I have found that liquibase has a small bug with rollbacks. Some of you may have encountered it. It uses regex substitution when rolling back (not sure what exactly). Part of defining the group in regex substitution is the '$', so that makes '$' kind of a reserved character that isn't very well documented in liquibase. It doesn't effect SQL at all, but it does mean using '$' in changeSet attributes is forbidden. Just to give an example of what will happen, when you rollback in this scenario, you get an exception:

Jun 5, 2010 8:07:24 AM liquibase.commandline.Main main
SEVERE: Illegal group reference
java.lang.IllegalArgumentException: Illegal group reference
at java.util.regex.Matcher.appendReplacement(Matcher.java:725)
at java.util.regex.Matcher.replaceFirst(Matcher.java:872)
at java.lang.String.replaceFirst(String.java:2158)
at liquibase.database.AbstractDatabase.removeRanStatus(AbstractDatabase.java:1328)
at liquibase.parser.visitor.RollbackVisitor.visit(RollbackVisitor.java:23)
at liquibase.parser.ChangeLogIterator.run(ChangeLogIterator.java:41)
at liquibase.Liquibase.rollback(Liquibase.java:273)
at liquibase.Liquibase.rollback(Liquibase.java:248)
at liquibase.commandline.Main.doMigration(Main.java:682)
at liquibase.commandline.Main.main(Main.java:97)


Below, we have the offending XML responsible for this.
<databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog/1.9"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog/1.9
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-1.9.xsd"&gh;

<changeSet id="$Revision$" author="$Author$">

Problem

You probably thought the above was the problem. No no no no no. That was just background on it. Such a thing is actually easily fixed. The real problem is testing in your IDE. The reason we have '$' is for the Subversion keywords. We want these in our changelogs so we can rollback in our other environments. Subversion doesn't actually add this information until after the svn:keywords property and the file are committed to the VCS. When it does, you get something that looks like this:
<databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog/1.9"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog/1.9
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-1.9.xsd"&gh;

<changeSet id="$Revision: 1$" author="$Author: przybyls$">

There are still '$' in there.

What does this have to do with the IDE?

At UA, we use a liquibase template, so all developers are consistent with their usage. It looks something like:

<?xml version="1.0" encoding="UTF-8"?>

<databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog/1.9"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog/1.9
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-1.9.xsd">

<changeSet id="$Revision$" author="$Author$">
<sql splitStatements="false" endDelimiter=""><![CDATA[
DECLARE
BEGIN
END;
]]>
</sql>
<rollback>
<sql><![CDATA[
DECLARE
BEGIN
END;
]]>
</sql>
</rollback>
</changeSet>
</databaseChangeLog>

The first thing you should notice is the '$' in the id and author attributes. This actually makes the problem part of our IDE. Now developers have to remember to change this. That makes this template pretty much...useless.

Solution (it's not as cool as you think)

At UA, we decided to take liquibase testing out of our IDE and instead move it into continuous integration with Ad-Hoc builds. In the end, developers still use the template. Only now, they don't need to install liquibase at all! It's one of those, "Why didn't I think of this before!?" moments.

Instructions for Hudson

I hope you're using Hudson. If you're not, you have my sympathies. I am truly sorry that you will have to find your own way of doing this.

Step 1: Create an ad-hoc build

To do this, you simply setup a build parameter in your project/build configuration. When you do this, Hudson will want to know about the parameter you are adding. I used a file. This way developers simply upload their file to Hudson. I called mine test.xml and you will see that come up in a second.

Step 2: Setup the Execution Script

I just used shell with the following contents:


/home/tomcat/liquibase/liquibase --defaultsFile=/home/tomcat/renwoluk.properties tag before
sed -e 's/\$Revision\$/test/g' test.xml > /tmp/test.xml;mv /tmp/test.xml test.xml
sed -e 's/\$Author\$/test/g' test.xml > /tmp/test.xml;mv /tmp/test.xml test.xml
/home/tomcat/liquibase/liquibase --defaultsFile=/home/tomcat/renwoluk.properties --changeLogFile=test.xml updateSQL
/home/tomcat/liquibase/liquibase --defaultsFile=/home/tomcat/renwoluk.properties --changeLogFile=test.xml update
/home/tomcat/liquibase/liquibase --defaultsFile=/home/tomcat/renwoluk.properties --changeLogFile=test.xml rollbackSQL before
/home/tomcat/liquibase/liquibase --defaultsFile=/home/tomcat/renwoluk.properties --changeLogFile=test.xml rollback before


Notice the sed commands that do the replace on test.xml. It is then simply doing an update and rollback just like in KIS of the Dragon.

Conclusion

That's it. Now developers can all test their liquibase changelogs in a consistent manner without any modification to their IDE. Hudson is also really effective here because changelog testing is really really fast. It takes a few seconds and won't hang up Hudson. Hudson also can facilitate multiple executors. I will include in another post on liquibase automation how to integrate this into configuration management and automated build processes.

Apologies

I am very sorry for the content in KIS of the Dragon because it was somewhat incorrect. I'm glad to remedy it here.

I also apologize for having not put out a screencast in awhile. My goal was one a week. I've had some family crisis to deal with lately and I made the mistake of working on a few at the same time. In the coming week, expect a torrent of posts.

No comments:

Post a Comment