Fork me on GitHub

n. Slang a rough lawless young Kuali developer.
[perhaps variant of Houlihan, Irish surname]
kualiganism n

Blog of an rSmart Java Developer. Full of code examples, solutions, best practices, et al.

Saturday, April 2, 2011

KIS Me Kate - RPM Packaging KFS Part 4

KC Packaging

Packaging KC is much the same as KFS. The only difference is that when you build, you are using maven instead of ant.

Setting up Hudson

Therefore, to get around adding my own hooks to the build, all I did was use Hudson. First, I setup a maven build.
It is the equivalent to
% mvn -Dmaven.test.skip=true package

Then, I created an invoke shell command

TARFILE=kuali-coeus-2.0-$(cut -d= -f 2 kc-2.0/version.properties).tar.gz
rm -rf kuali-coeus-2.0
mkdir kuali-coeus-2.0
cp -rf kc-2.0/target/kc_custom-2.0 kuali-coeus-2.0

for x in $HOME/kuali/main/[a-z]*; do 
mkdir -p kuali-coeus-2.0/kuali/main/$(basename $x)/
cp -rf $HOME/kuali/main/$(basename $x)/kc-config.xml kuali-coeus-2.0/kuali/main/$(basename $x)
done
cp $HOME/lib/ojdbc* kuali-coeus-2.0/kc_custom-2.0/WEB-INF/lib
cp $HOME/*.properties kuali-coeus-2.0
tar -czf $TARFILE kuali-coeus-2.0 kc-cfg-dbs/
mv $TARFILE /mosaic/data/KITT/SOURCES

You can see that I am using a version.properties. It looks like
leo@behemoth~/.workspace/kc
(11:02:12) [21] cat version.properties 
release=11

Finally, I create the package by calling the build-rpm.xml using ant.
The build-rpm.xml looks like
<?xml version="1.0" encoding="UTF-8"?>
<project             name="kc" 
default="build" 
xmlns:kitt-tools="urn:com.rsmart.ant">
<target name="build" depends="filter-spec">
<exec executable="rpmbuild">
<arg value="-bb" />
<arg value="kc.spec" />
</exec>
</target>

<target name="filter-spec">
<property file="version.properties" />
<kitt-tools:filter srcfile="kc.spec.template"
filename="kc.spec" />

</target>

<macrodef uri="urn:com.rsmart.ant" name="filter">
<attribute name="srcfile" />
<attribute name="filename" />
<sequential>
<loadfile property="buildroot.filter.template"
srcfile="@{srcfile}">
<filterchain>
<expandproperties/>
</filterchain>
</loadfile>

<echo file="@{filename}">${buildroot.filter.template}</echo>
</sequential>
</macrodef>
</project>

A lot like our original build.xml for KFS. The differences here are
  • No vendor path, so we use build-rpm.xml instead of build.xml
  • no workflow
  • no changelogs
  • Building doesn't happen here since hudson did that for us
  • No importing the other build.xml
You can see we use a kc.spec.template and filter that just as we did with kfs.

KC spec file


Here is the spec file I used
%define __os_install_post %{nil}
%define debug_package %{nil} 

Summary: Kuali Coeus
Name: kuali-coeus
Version: 2.0
Release: %release
Provides: kuali-coeus
License: EPL
BuildArch: noarch
Source0: kuali-coeus-2.0-%release.tar.gz
BuildRoot: /tmp/kc
Requires: ant
Group: Development/Tools
Packager: przybyls@arizona.edu

%package settings-${build.environment}
Summary: External configuration settings for Kuali Coeus
Group: System/Base
Requires: kuali-coeus

%package changelogs
Summary: Kuali Coeus KITT Customization Schema
Group: System/Base
Requires: kc,liquibase,wget

%description
The Kuali Foundation research administration software

%description changelogs
Mosaic Kuali Coeus Environment Database Schema for KITT customizations based on KITT
modification set %release

%description settings-${build.environment}
Mosaic Kuali Coeus external configuration and settings. These files are located
in /home/tomcat/app

%prep
%setup -q

%install
TOOLSDIR=%{_builddir}/kuali-coeus-2.0/kitt-tools-1.0
mkdir -p %{buildroot}/home/tomcat/kitt-tools
mkdir -p %{buildroot}/home/tomcat/kitt-tools/bin
mkdir -p %{buildroot}/home/tomcat/kitt-tools/config
mkdir -p %{buildroot}/home/tomcat/kitt-tools/lib
mkdir -p %{buildroot}/home/tomcat/kuali/main/2.0-%release/changesets/
mkdir -p %{buildroot}/usr/share/tomcat5/webapps/
mkdir -p %{buildroot}/usr/share/tomcat5/webapps/kra-${build.environment}/WEB-INF/
mkdir -p %{buildroot}/usr/share/tomcat5/webapps/kra-${build.environment}/WEB-INF/classes/META-INF

cp ${build.environment}.properties %{buildroot}/home/tomcat/kitt-tools
cp $HOME/apache-ant/lib/ant.jar %{buildroot}/home/tomcat/kitt-tools/lib
cp $HOME/apache-ant/lib/ant-launcher.jar %{buildroot}/home/tomcat/kitt-tools/lib

cd kc_custom-2.0/WEB-INF/classes/META-INF/
sed -e 's/\(.*build.environment.*"false">\).*/\1${build.environment}<\/param>/' kc-config-defaults.xml | sed -e 's/\(.*build.version.*"false">\).*/\12.0-%release<\/param>/' > /tmp/kc-config-defaults.xml
mv /tmp/kc-config-defaults.xml %{buildroot}/usr/share/tomcat5/webapps/kra-${build.environment}/WEB-INF/classes/META-INF
cd -

cp kc_custom-2.0/WEB-INF/web-${build.environment}.xml %{buildroot}/usr/share/tomcat5/webapps/kra-${build.environment}/WEB-INF/web.xml
cp -rf kuali/main/${build.environment} %{buildroot}/home/tomcat/kuali/main/


cat <<EOF > %{buildroot}/home/tomcat/.rpmmacros
%%_topdir /home/tomcat/.workspace/redhat
%%_dbpath /home/tomcat/rpm
EOF

cat <<EOF > %{buildroot}/home/tomcat/kitt-tools/.envrc
${build.environment}
EOF

mv %{_builddir}/kuali-coeus-2.0/kc_custom-2.0 %{buildroot}/usr/share/tomcat5/webapps/kra/
mv %{_builddir}/kc-cfg-dbs/update* %{buildroot}/home/tomcat/kuali/main/2.0-%release/changesets/

set -x 

ant -f /dev/stdin -Dbuild.environment=${build.environment}<<EOF
<?xml version="1.0"?>
<project name="kitt-tools" default="run" basedir="." xmlns:kitt-tools="urn:com.rsmart.ant">
<target name="run">
<property file="$TOOLSDIR/\${build.environment}.properties" />
<echo file="%{buildroot}/home/tomcat/kitt-tools/credentials.properties">
source.driver=\${dbcopy.default.driver}
source.url=jdbc:oracle:thin:@uaz-kf-d02.mosaic.arizona.edu:1521:UAZKRDEV
source.username=sandbox
source.password=kulowner
source.schema=SANDBOX
target.driver=\${dbcopy.default.driver}
target.url=\${oracle.datasource.url}
target.username=\${datasource.username}
target.schema=KULOWNER
encrypted.password=\${encrypted.password}
</echo>
</target>
</project>
EOF

%clean
rm -rf %{buildroot}

%files settings-${build.environment}
%defattr(2770,tomcat,kuali)
/home/tomcat/.rpmmacros
/home/tomcat/kitt-tools
/home/tomcat/kuali/main/${build.environment}/kc-config.xml
/usr/share/tomcat5/webapps/kra-${build.environment}/WEB-INF/web.xml
/usr/share/tomcat5/webapps/kra-${build.environment}/WEB-INF/classes/META-INF/kc-config-defaults.xml


%files changelogs
%defattr(-,tomcat,kuali)
%config /home/tomcat/kuali/main/2.0-%release/changesets/update.xml
%config /home/tomcat/kuali/main/2.0-%release/changesets/update/

%files
%defattr(2770,tomcat,kuali)
/usr/share/tomcat5/webapps/kra/

%pre
rm -rf /usr/share/tomcat5/webapps/kra-*

%post
set -x

MYPWD=$PWD
cd /usr/share/tomcat5/webapps
mv kra kra-$(cat ~tomcat/kitt-tools/.envrc)
cd $MYPWD

%post changelogs 
set -x

VERSION=%release
CURRENT=$(ls -t ~tomcat/kuali/main/|grep -v %release|head -1|cut -d- -f 2)
REPO_URL=
KC_VERSION=2.0

for x in $(seq $(expr $CURRENT + 1) $VERSION);
do
if [ ! -e ~tomcat/kuali/main/$KC_VERSION-$x ];
then
cd ~tomcat/kuali/main
wget -r -l 2 --no-parent -nH --cut-dirs=5 $REPO_URL/$KC_VERSION-$x/
fi

cd ~tomcat/kuali/main/$KC_VERSION-$x/changesets
liquibase --changeLogFile=update.xml --logLevel=finest update
liquibase --logLevel=finest tag $KC_VERSION.$x
cd 
done

cd -

if [ -e ~tomcat/kuali/main/$KC_VERSION-$(expr %release + 1) ];
then
echo '%release' > /tmp/lquninst
fi

%postun changelogs
if [ -e /tmp/lquninst ];
then

START=%release
END=$(cat /tmp/lquninst)
for x in $(seq $START -1 $END); 
do 
cd /home/tomcat/kuali/main/2.0-$x/changesets
liquibase --changeLogFile=update.xml rollback 2.0.$(expr $x - 1)

cd -;
done
rm /tmp/lquninst
fi 
exit
The script is pretty crazy looking, so I will take a jab at explaining it. It will help to explain the major differences in project organization and structure between KFS and KC.
  • New XML Configuration for KC/Rice in kuali/main/dev
  • no more security.properties (normally where database passwords are stored)
  • No build.properties since we're not using ant. Configuration is now in kc-config.xml in kuali/main/<environment>

Handle PreProcessing

Keeping that in mind, let's have a look at the %install section which is where most of the work that is going on is. Since we can't bootstrap or hook into the regular maven packaging for the WAR, what I've done instead is to handle that preprocessing here. Keep in mind that since this is the %install directive, this is not packaging yet. This is just creating the build sandbox we're going to package later. So first we need to prepare the structure
TOOLSDIR=%{_builddir}/kuali-coeus-2.0/kitt-tools-1.0
mkdir -p %{buildroot}/home/tomcat/kitt-tools
mkdir -p %{buildroot}/home/tomcat/kitt-tools/bin
mkdir -p %{buildroot}/home/tomcat/kitt-tools/config
mkdir -p %{buildroot}/home/tomcat/kitt-tools/lib
mkdir -p %{buildroot}/home/tomcat/kuali/main/2.0-%release/changesets/
mkdir -p %{buildroot}/usr/share/tomcat5/webapps/
mkdir -p %{buildroot}/usr/share/tomcat5/webapps/kra-${build.environment}/WEB-INF/
mkdir -p %{buildroot}/usr/share/tomcat5/webapps/kra-${build.environment}/WEB-INF/classes/META-INF
You can see this is building out the path for the webapp to live. There is also this new concept of a kitt-tools path. These are actually environment utilities for administrating KC. They are outside the scope of this explanation, but we do have to add to them a few things during installation. One of these is the credential.properties
ant -f /dev/stdin -Dbuild.environment=${build.environment}<<EOF
<?xml version="1.0"?>
<project name="kitt-tools" default="run" basedir="." xmlns:kitt-tools="urn:com.rsmart.ant">
<target name="run">
<property file="$TOOLSDIR/\${build.environment}.properties" />
<echo file="%{buildroot}/home/tomcat/kitt-tools/credentials.properties">
source.driver=\${dbcopy.default.driver}
source.url=jdbc:oracle:thin:@uaz-kf-d02.mosaic.arizona.edu:1521:UAZKRDEV
source.username=sandbox
source.password=kulowner
source.schema=SANDBOX
target.driver=\${dbcopy.default.driver}
target.url=\${oracle.datasource.url}
target.username=\${datasource.username}
target.schema=KULOWNER
encrypted.password=\${encrypted.password}
</echo>
</target>
</project>
EOF
Ant is used to populate this based on values set in the configuration. Each environment is going to get its own kc-config.xml. For example, TST would be kuali/main/tst/kc-config.xml. Usually, the there are very slight differences between each environment configuration. Usually, the only difference is the database password. For most of the common settings, we use a kc-config-defaults.xml. These files are maintained on the build system (where hudson is living). As a result our build version is usually clobbered by the default. I do a little hacking to get around that. The following takes care of that for us.
cd kc_custom-2.0/WEB-INF/classes/META-INF/
sed -e 's/\(.*build.environment.*"false">\).*/\1${build.environment}<\/param>/' kc-config-defaults.xml | sed -e 's/\(.*build.version.*"false">\).*/\12.0-%release<\/param>/' > /tmp/kc-config-defaults.xml
mv /tmp/kc-config-defaults.xml %{buildroot}/usr/share/tomcat5/webapps/kra-${build.environment}/WEB-INF/classes/META-INF
We have environment specific web.xml files for any weird changes that are by environment
cp kc_custom-2.0/WEB-INF/web-${build.environment}.xml %{buildroot}/usr/share/tomcat5/webapps/kra-${build.environment}/WEB-INF/web.xml
cp -rf kuali/main/${build.environment} %{buildroot}/home/tomcat/kuali/main/
Once a package is installed, tools that we install take advantage of knowing what environment they're on. We handle this by creating an .envrc file on the environment.
cat <<EOF > %{buildroot}/home/tomcat/kitt-tools/.envrc
${build.environment}
EOF
The last thing done is to move the changelogs and the webapp to it's environment agnostic locations. Remember from Part 2 we move these files into their appropriate locations during post processing
mv %{_builddir}/kuali-coeus-2.0/kc_custom-2.0 %{buildroot}/usr/share/tomcat5/webapps/kra/
mv %{_builddir}/kc-cfg-dbs/update* %{buildroot}/home/tomcat/kuali/main/2.0-%release/changesets/

New packages

Aside from the %install, the spec file and packages are very much the same or at least familiar to the KFS packages. For KC, it was decided that workflow would be handled manually instead of automatically, so the packages are:
  • kuali-coeus-2.0-1.noarch.rpm
  • kuali-coeus-settings-2.0-1.noarch.rpm
  • kuali-coeus-changelogs-2.0-1.noarch.rpm

No comments:

Post a Comment