Fork me on GitHub

n. Slang a rough lawless young Kuali developer.
[perhaps variant of Houlihan, Irish surname]
kualiganism n

Blog of an rSmart Java Developer. Full of code examples, solutions, best practices, et al.

Monday, June 27, 2011

UPDATE (ATTN: Eclipse Users!): Hot Code Replacement for KFS Development With Tomcat

Ok. A lot of Eclipse users out there are having a lot of trouble. They are encountering a lot of errors like this
"Access restriction: Class is not accessible due to restriction on required library"
. The best way to get around this is to go to Window->Preferences->Java->Compiler. A dialog will appear. Make sure yours looks like this Forbidden reference (access rules) needs to be set to Warning. That will solve your problem and you will be able to use Hot Code Replacement for KFS

Sunday, June 26, 2011

Hot Code Replacement for KFS Development With Tomcat

Overview

This is not hot deployment as in reloadable="true" like here:
<Context path="/kfs-dev" reloadable="true"  docBase="/Users/leo/.workspace/kfs/release-4-0-overlay/target/kfs-dev">
No no. That will force the webapp context to reload which is the trouble with KFS in tomcat. When the context reloads there are often memory problems. Worst of all, Spring has to reload everything. That's just a joy and a pleasure to experience for all developers too. Not only do you wait forever and a day for Spring to reload your DD, but then it finishes to remind you there's a memory leak with context redeployment. Pleasant, isn't it?

Solution 1: Use Maven

Honestly, the best thing you can do is this Convert Your KFS Distribution to Maven. You can then use the much coveted Tomcat Maven Plugin oooor the lesser Jetty Maven Plugin. These will support not only hot redeploys, but Hot Code Replacement. You may even avoid Spring reloading.

Solution 2: JDI (Java Debugging Interface)

There may be some of you that would rather not be bothered with converting to maven. You may just not be ready yet, but you love the idea of Hot Code Replacement because you're a developer and developers hate 2 things
  • Waiting around
  • Waiting around when they don't need to.
What's a developer to do? Brilliantly, before Sun was overtaken by the evil empire (Oracle), they left us with a few magical treasures to save our dying world from the evil that is upon us poised to deliver its reign of destruction and bind and afflict us. One is JDI (Java Debugging Interface) or JPDA (Java Platform Debugger Architecture) as it is sometimes referred. A veritable sword of omens as you will soon find out.

This is actually how Eclipse handles its Hot Code Replacement in debug mode. It's actually using JDI/JPDA. Given is that this will run slower than normal applications. It is also Debug mode. That is, you would never ever ever ever use this in production...ever. This is a development tool. The great part is that since Eclipse is also using JDI, that means that attaching a debugger or starting Eclipse in debug mode works seamlessly with this technique.

In a Nutshell

The simple explanation for my solution is:
  1. Create a Quartz Job (WebappWatchJob) with a step (WebappWatchStep)
  2. Set the job to run every 30 seconds
  3. Add necessary configuration to directory.properties and configuration.properties so this only shows up when dev.mode is true
  4. Implement WebappWatchJob to traverse your webapp installation path for .class files and check for last modified updates to reload the class on change

...in a nutshell.

The Details and Steps

Deep breath...now go!

1. Setup Properties

In configuration.properties I added
##############################################################################################################
## Properties from institutional.configuration.file (${institutional.configuration.file}) are appended after this point.
##############################################################################################################
webapp.classes.directory=${webapp.classes.directory}

I then set in directory.properties
# Application server directories - these assume the Tomcat 5.5 structure
tomcat.version=5
#appserver.lib.dir=${appserver.home}/common/lib
#appserver.classes.dir=${appserver.home}/common/classes
appserver.deploy.dir=${appserver.home}/webapps
appserver.config.dir=${appserver.home}/conf
appserver.localhost.dir=${appserver.config.dir}/Catalina/localhost
appserver.work.dir=${appserver.home}/work/Catalina/localhost
webapp.classes.directory=${appserver.deploy.dir}/${project.name}-${build.environment}/WEB-INF/classes

The above defines a property we will eventually use to set which path to watch for changes in.

2. Create WebappWatchJob

I add the following to my spring configuration. In KFS, my module is tem, so I add it to my spring-tem.xml:
<?xml version="1.0" encoding="UTF-8"?>
<!-- Copyright 2006-2008 The Kuali Foundation Licensed under the Educational 
Community License, Version 2.0 (the "License"); you may not use this file 
except in compliance with the License. You may obtain a copy of the License 
at http://www.opensource.org/licenses/ecl2.php Unless required by applicable 
law or agreed to in writing, software distributed under the License is distributed 
on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either 
express or implied. See the License for the specific language governing permissions 
and limitations under the License. -->
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"
xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://www.springframework.org/schema/tx
http://www.springframework.org/schema/tx/spring-tx-2.0.xsd
http://www.springframework.org/schema/aop
http://www.springframework.org/schema/aop/spring-aop-2.0.xsd">
...
...
<property name="jobNames">
<list>
<value>webappWatchJob</value>
</list>
</property>
<property name="triggerNames">
<list>
<value>webappWatchTrigger</value>
</list>
</property>
...
...
</beans>

Clearly, I have declared a webappWatchJob and a webappWatchTrigger. I later define those in the same file this way:
<?xml version="1.0" encoding="UTF-8"?>
<!-- Copyright 2006-2008 The Kuali Foundation Licensed under the Educational 
Community License, Version 2.0 (the "License"); you may not use this file 
except in compliance with the License. You may obtain a copy of the License 
at http://www.opensource.org/licenses/ecl2.php Unless required by applicable 
law or agreed to in writing, software distributed under the License is distributed 
on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either 
express or implied. See the License for the specific language governing permissions 
and limitations under the License. -->
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"
xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://www.springframework.org/schema/tx
http://www.springframework.org/schema/tx/spring-tx-2.0.xsd
http://www.springframework.org/schema/aop
http://www.springframework.org/schema/aop/spring-aop-2.0.xsd">
...
...
<bean id="webappWatchStep" class="org.kuali.kfs.module.tem.batch.WebappWatchStep" parent="step">
<property name="webappPath" value="${webapp.classes.directory}" />
</bean>

<bean id="webappWatchJob" parent="scheduledJobDescriptor">
<property name="steps">
<list>
<ref bean="webappWatchStep" />
</list>
</property>
</bean>

<bean id="webappWatchTrigger" parent="cronTrigger">
<property name="jobName" value="webappWatchJob" />
<property name="cronExpression" value="0/30 * * * * ?" />
</bean>
</beans>

Notice that our property we created before is used here. It is injected into the step so that it may be watched. Also, the cronExpression is set for every 0 and 30th second or every 30 seconds.

3. Implement WebappWatchStep

Now that we have the spring definitions setup, we need to implement the WebappWatchStep. I will first explain its pieces and then paste the full class implementation.

Properties

There are just 2 properties.
...
... 
/**
* Gets the value of WebappPath
*
* @return the value of WebappPath
*/
public String getWebappPath() {
return this.webappPath;
}

/**
* Sets the value of WebappPath
*
* @param argWebappPath Value to assign to this.WebappPath
*/
public void setWebappPath(final String argWebappPath) {
this.webappPath = argWebappPath;
}

/**
* Gets the value of LastUpdated
*
* @return the value of LastUpdated
*/
public Date getLastUpdated() {
return this.lastUpdated;
}

/**
* Sets the value of LastUpdated
*
* @param argLastUpdated Value to assign to this.LastUpdated
*/
public void setLastUpdated(final Date argLastUpdated) {
this.lastUpdated = argLastUpdated;
}
...
...
  • webappPath - is the path injected to define what directory to recurse and watch
  • lastUpdated - is a cached date of the last run. This is compared to the file lastModified property to determine whether the file has changed or not


Traversing webappPath

...
...
/**
* @see org.kuali.kfs.sys.batch.Step#execute(java.lang.String, java.util.Date)
*/
public boolean execute(String jobName, Date jobRunDate) throws InterruptedException {
info("WebappWatch triggered. I'm going in");
final Collection<File> classFiles = new ArrayList<File>();
traverseForClasses(new File(getWebappPath()), classFiles);
...
...
}
...
...
/**
* Recursively searches for classes. If <code>file</code> is a directory, it traverses recursively. If it is a
* file, then it compares the <code>lastModified</code> date with the cached {@link #getLastModified()} value.
* 
* @param file {@link File} instance for traversing
* @param classFiles is a {@link Collection} of {@link File} instances where the {@link File#lastUpdated()} value is
* before {@link #getLastUpdated()}
*/
protected void traverseForClasses(final File file, final Collection<File> classFiles) {
if (file.isDirectory()) {
final String[] fileNames = file.list();

for (final String fileName : fileNames) {
traverseForClasses(new File(file, fileName), classFiles);
}
}
else if (new Date(file.lastModified()).after(lastUpdated) && file.getName().endsWith(".class")) {
classFiles.add(file);
}
}
...    
...

A classFiles Collection is passed into a method that recursively adds class names to that collection. It is later iterated over in the execute method like this:
...
...
try {
for (final File classFile : classFiles) {
final byte[] classData = new byte[new Long(classFile.length()).intValue()];
FileInputStream classStream = null;
try {
classStream = new FileInputStream(classFile);

classStream.read(classData, 0, classData.length);
}
catch (Exception e) {
}
finally {
if (classStream != null) {
try {
classStream.close();
}
catch (Exception e) { } // failure to close. Do nothing.
}
}

final String className = classFromFileName(classFile.getAbsolutePath()); 
boolean classExists = false;
try {
Class.forName(className);
classExists = true;
}
catch (ClassNotFoundException cnfe) {
}

if (classExists) {
addRedefineClass(toRedefineMap, className, classData);
}
else {
defineClass(className, classData, 0, classData.length);
}
}

debug("Delivering the redefinition map");
vm.redefineClasses(toRedefineMap);
}
catch (Exception e) {
warn(e.getMessage());
e.printStackTrace();
Throwable cause = e.getCause();
while (cause != null) {
cause.printStackTrace();
warn("Caused by: ");
cause = cause.getCause();
}
}
finally {
try {
vm.resume();
if (conn != null && conn.isOpen()) {
debug("Closing the JPDA connection...");
conn.close();
debug("Connection closed");
}
}
catch (Exception e) {
e.printStackTrace();
}
}
...
...

The Juicy Part

So what is this stuff we saw in the execute method?
...
...
/**
* @see org.kuali.kfs.sys.batch.Step#execute(java.lang.String, java.util.Date)
*/
public boolean execute(String jobName, Date jobRunDate) throws InterruptedException {
...
...
try {
conn = openJdiConnection();
vm = Bootstrap.virtualMachineManager().createVirtualMachine(conn);
}
catch (Exception e) {
return false;
}
...
...
try {
vm.resume();
if (conn != null && conn.isOpen()) {
debug("Closing the JPDA connection...");
conn.close();
debug("Connection closed");
}
}
catch (Exception e) {
e.printStackTrace();
}
...
...
}
...
...

Near the beginning of the execute method, we establish a connection with JPDA using the openJdiConnection described here
...
...
/**
* Creates a {@link SocketTransportService} instance and uses that to attach to localhost:8080 and return a {@link Connection} to the 
* current running JVM instance. This is used by bootstrap to create a {@link VirtualMachine} instance.
* 
* @throws Exception when there's a problem attaching to localhost:8080 or if it cannot create an instance of {@link SocketTransportService}
* @return an open {@link Connection} to the JPDA instance of the current running JVM
* @see com.sun.jdi.VirtualMachineManager#createVirtualMachine(Connection)
* @see com.sun.tools.jdi.SocketTransportService
* @see com.sun.jdi.Bootstrap#virtualMachineManager()
*/
protected Connection openJdiConnection() throws Exception {
TransportService ts = null;
try {
Class c = Class.forName("com.sun.tools.jdi.SocketTransportService");
ts = (TransportService)c.newInstance();
} catch (Exception x) {
throw new Error(x);
}

return ts.attach("localhost:8000", 5000, 5000);
}
...
...

The builtin com.sun.tools.jdi.SocketTransportService is used to connect to the JPDA. I've set it to default to port 8000. I believe that's a standard, but you may want to customize this further. Tomcat uses the following environment variables to define this. It may be leveraged here.
#   JPDA_TRANSPORT  (Optional) JPDA transport used when the "jpda start"
#                   command is executed. The default is "dt_socket".
#
#   JPDA_ADDRESS    (Optional) Java runtime options used when the "jpda start"
#                   command is executed. The default is 8000.
#
#   JPDA_SUSPEND    (Optional) Java runtime options used when the "jpda start"
#                   command is executed. Specifies whether JVM should suspend
#                   execution immediately after startup. Default is "n".
#
#   JPDA_OPTS       (Optional) Java runtime options used when the "jpda start"
#                   command is executed. If used, JPDA_TRANSPORT, JPDA_ADDRESS,
#                   and JPDA_SUSPEND are ignored. Thus, all required jpda
#                   options MUST be specified. The default is:
#
#                   -Xdebug -Xrunjdwp:transport=$JPDA_TRANSPORT,
#                       address=$JPDA_ADDRESS,server=y,suspend=$JPDA_SUSPEND
#


Once the connection is made, it eventually also needs to be closed
...
...
try {
vm.resume();
if (conn != null && conn.isOpen()) {
debug("Closing the JPDA connection...");
conn.close();
debug("Connection closed");
}
}
catch (Exception e) {
e.printStackTrace();
}
...
...

You can see that right before the connection is closed, the virtual machine is resumed with the resume method. This is in case some process caused the vm to become suspended. It resumes all threads before disconnecting.

While looping over the modified class names, we ran into this
...
...
final String className = classFromFileName(classFile.getAbsolutePath()); 
boolean classExists = false;
try {
Class.forName(className);
classExists = true;
}
catch (ClassNotFoundException cnfe) {
}

if (classExists) {
addRedefineClass(toRedefineMap, className, classData);
}
else {
defineClass(className, classData, 0, classData.length);
}
...
...

Simply put, we use a method defineClass for classes that are new and addRedefineClass for methods that are already loaded with the current ClassLoader. Loading a class is easy reflection with the ClassLoader, so I am going to skip that. The tough part is handling classes that are already loaded. This is where JPDA comes in. Normally, you can only define a class once per ClassLoader, but with JDPA, you can use the redefineClasses method in the VirtualMachine. This redefineClasses method takes a Map though. This means we do not typically redefine a class at a time. We have to round them all up into a Map, and then call redefineClasses on the map. To do this, we created a addRedefineClass method which takes our already created Map, and puts classes into it. Let's have a look
...
...
/**
* Populates a map of {@link ReferenceType} and {@link byte[]} data. This {@link Map} is used by JDI for 
* redefining classes.
*
* @param toRedefineMap {@link Map} of {@link byte[]} mapped by {@link ReferenceType}. This map is populated by the method, so it shouldn't be null.
* @see com.sun.jdi.VirtualMachine#redefineClasses(Map, byte[])
*/
protected void addRedefineClass(final Map<ReferenceType,byte[]> toRedefineMap, final String className, final byte[] b) throws Exception { 
debug("Found change in ", className);
for (ReferenceType ref : vm.classesByName(className)) {
if (ref.name().equals(className) && className.indexOf("WebappWatch") < 0) {
debug("redefining ", className);
toRedefineMap.put(ref, b);
return;
}
}
}
...
...

Since I made vm a local variable in the WebappWatchStep, I can call it from addRedefineClass. I use it for its nifty classesByName method. I pass it my fully qualified class name, and it returns a list of ReferenceType instances. Typically, this will return only one class. Actually, I cannot think of any case where it wouldn't, but just to be sure, I double check that. I also omit the WebappWatchStep class. We wouldn't want to reload the class we're currently accessing VirtualMachine from. That would be bad. We simply add the ReferenceType as a key to the byte array representing the actual class. This will get used later by the VirtualMachine#redefineClasses method. For more information on ReferenceType, check out http://download.oracle.com/javase/6/docs/jdk/api/jpda/jdi/com/sun/jdi/ReferenceType.html


As promised

Here is the full source code
/*
* Copyright 2007 The Kuali Foundation
* 
* Licensed under the Educational Community License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
* 
* http://www.opensource.org/licenses/ecl2.php
* 
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.kuali.kfs.module.tem.batch;

import java.lang.reflect.Method;

import java.io.File;
import java.io.FileInputStream;

import java.util.ArrayList;
import java.util.Collection;
import java.util.Date;
import java.util.HashMap;
import java.util.Map;

import com.sun.jdi.Bootstrap;
import com.sun.jdi.ArrayType;
import com.sun.jdi.ClassType;
import com.sun.jdi.ReferenceType;
import com.sun.jdi.InterfaceType;
import com.sun.jdi.VirtualMachine;
import com.sun.jdi.connect.spi.Connection;
import com.sun.jdi.connect.spi.TransportService;

import org.kuali.kfs.sys.batch.AbstractStep;

import static org.kuali.kfs.module.tem.util.BufferedLogger.*;
import static org.apache.commons.lang.StringUtils.replace;

/**
* Batch step that traverses the webapp for changed classes. It then reloads the classes into the current {@link ClassLoader}
* this gives to appearance of hot redeploys.
*
* @author leo [at] rsmart.com
*/
public class WebappWatchStep extends AbstractStep {
private Date lastUpdated;
private String webappPath;
private Connection conn;
private VirtualMachine vm;

public WebappWatchStep() throws Throwable {
super();
lastUpdated = new Date();
}

/**
* @see org.kuali.kfs.sys.batch.Step#execute(java.lang.String, java.util.Date)
*/
public boolean execute(String jobName, Date jobRunDate) throws InterruptedException {
info("WebappWatch triggered. I'm going in");
final Collection<File> classFiles = new ArrayList<File>();
traverseForClasses(new File(getWebappPath()), classFiles);
final Map<ReferenceType,byte[]> toRedefineMap = new HashMap<ReferenceType,byte[]>();

try {
conn = openJdiConnection();
vm = Bootstrap.virtualMachineManager().createVirtualMachine(conn);
}
catch (Exception e) {
return false;
}

try {
for (final File classFile : classFiles) {
final byte[] classData = new byte[new Long(classFile.length()).intValue()];
FileInputStream classStream = null;
try {
classStream = new FileInputStream(classFile);

classStream.read(classData, 0, classData.length);
}
catch (Exception e) {
}
finally {
if (classStream != null) {
try {
classStream.close();
}
catch (Exception e) { } // failure to close. Do nothing.
}
}

final String className = classFromFileName(classFile.getAbsolutePath()); 
boolean classExists = false;
try {
Class.forName(className);
classExists = true;
}
catch (ClassNotFoundException cnfe) {
}

if (classExists) {
addRedefineClass(toRedefineMap, className, classData);
}
else {
defineClass(className, classData, 0, classData.length);
}
}

debug("Delivering the redefinition map");
vm.redefineClasses(toRedefineMap);
}
catch (Exception e) {
warn(e.getMessage());
e.printStackTrace();
Throwable cause = e.getCause();
while (cause != null) {
cause.printStackTrace();
warn("Caused by: ");
cause = cause.getCause();
}
}
finally {
try {
vm.resume();
if (conn != null && conn.isOpen()) {
debug("Closing the JPDA connection...");
conn.close();
debug("Connection closed");
}
}
catch (Exception e) {
e.printStackTrace();
}
}

setLastUpdated(new Date());
return false;
}

/**
* Defines a class. Only used for new classes that don't already exist in the {@link ClassLoader}
* @param className name of the {@link Class} to define
* @param b byte array of data
* @param offset where in the byte array to start
* @param len size of the byte array to read
* @throws Exception <code>defineClass()</code> on the {@link ClassLoader} is protected. It cannot normally be called.
* an exception can be thrown if something goes wrong.
*/
protected void defineClass(final String className, final byte[] b, final int offset, final int len) throws Exception {
info("Defining class ", className);
final ClassLoader classLoader = getClass().getClassLoader();

final Method method = ClassLoader.class.getDeclaredMethod("defineClass", 
new Class[] { String.class, byte[].class, int.class, int.class });
method.setAccessible(true);
final Class definedClass = (Class) method.invoke(classLoader, className, b, offset, len);
info("Defined class ", definedClass);
// info("Loaded class ", getClass().getClassLoader().loadClass(className));
}

/**
* Populates a map of {@link ReferenceType} and {@link byte[]} data. This {@link Map} is used by JDI for 
* redefining classes.
*
* @param toRedefineMap {@link Map} of {@link byte[]} mapped by {@link ReferenceType}. This map is populated by the method, so it shouldn't be null.
* @see com.sun.jdi.VirtualMachine#redefineClasses(Map, byte[])
*/
protected void addRedefineClass(final Map<ReferenceType,byte[]> toRedefineMap, final String className, final byte[] b) throws Exception { 
debug("Found change in ", className);
for (ReferenceType ref : vm.classesByName(className)) {
if (ref.name().equals(className) && className.indexOf("WebappWatch") < 0) {
debug("redefining ", className);
toRedefineMap.put(ref, b);
return;
}
}
}

/**
* Creates a {@link SocketTransportService} instance and uses that to attach to localhost:8080 and return a {@link Connection} to the 
* current running JVM instance. This is used by bootstrap to create a {@link VirtualMachine} instance.
* 
* @throws Exception when there's a problem attaching to localhost:8080 or if it cannot create an instance of {@link SocketTransportService}
* @return an open {@link Connection} to the JPDA instance of the current running JVM
* @see com.sun.jdi.VirtualMachineManager#createVirtualMachine(Connection)
* @see com.sun.tools.jdi.SocketTransportService
* @see com.sun.jdi.Bootstrap#virtualMachineManager()
*/
protected Connection openJdiConnection() throws Exception {
TransportService ts = null;
try {
Class c = Class.forName("com.sun.tools.jdi.SocketTransportService");
ts = (TransportService)c.newInstance();
} catch (Exception x) {
throw new Error(x);
}

return ts.attach("localhost:8000", 5000, 5000);
}

/**
* Tries to create a canonical class name from a .class file.
*
* @param fileName is the fully qualified path of a .class file
* @return a canonical class name (java.lang.String)
*/
protected String classFromFileName(final String fileName) {
String retval = fileName.substring(0, fileName.indexOf(".class"));
retval = retval.substring(getWebappPath().length() + 1);
retval = replace(retval, File.separator, ".");
retval = replace(retval, "/", ".");
return retval;
}

/**
* Recursively searches for classes. If <code>file</code> is a directory, it traverses recursively. If it is a
* file, then it compares the <code>lastModified</code> date with the cached {@link #getLastModified()} value.
* 
* @param file {@link File} instance for traversing
* @param classFiles is a {@link Collection} of {@link File} instances where the {@link File#lastUpdated()} value is
* before {@link #getLastUpdated()}
*/
protected void traverseForClasses(final File file, final Collection<File> classFiles) {
if (file.isDirectory()) {
final String[] fileNames = file.list();

for (final String fileName : fileNames) {
traverseForClasses(new File(file, fileName), classFiles);
}
}
else if (new Date(file.lastModified()).after(lastUpdated) && file.getName().endsWith(".class")) {
classFiles.add(file);
}
}

/**
* Gets the value of WebappPath
*
* @return the value of WebappPath
*/
public String getWebappPath() {
return this.webappPath;
}

/**
* Sets the value of WebappPath
*
* @param argWebappPath Value to assign to this.WebappPath
*/
public void setWebappPath(final String argWebappPath) {
this.webappPath = argWebappPath;
}

/**
* Gets the value of LastUpdated
*
* @return the value of LastUpdated
*/
public Date getLastUpdated() {
return this.lastUpdated;
}

/**
* Sets the value of LastUpdated
*
* @param argLastUpdated Value to assign to this.LastUpdated
*/
public void setLastUpdated(final Date argLastUpdated) {
this.lastUpdated = argLastUpdated;
}
}


That's it. Use this and you should have no problem modifying classes as you are developing without needing to restart Spring or Tomcat. Enjoy.

Tuesday, June 14, 2011

Convert Your KFS Distribution to Maven

Overview

I am going to cover the process/steps for converting your KFS distribution to use maven. The two greatest reasons for doing this is for development and building/releasing the software. Using maven will make your distribution more consistent with other Kuali projects, get your transitive dependency management, simplify your build process, and simplify your development process with convention over configuration.

At first, this may seem to be a daunting task to move from ant to maven, but it is actually pretty simple. The properties file conventions in KFS do not make this any more difficult as one would think. We can still use maven.

Steps

Before I begin, I want to point out that this assumes you are working with a fresh checkout of the KFS from http://test.kuali.org/svn/kfs/branches/release-4-0-br. You can use 4.1.1 as well. The important thing to note is that it's KFS 4.x.

1 Create path structure

In the directory where KFS is checked out, I suggest first creating all the necessary directories for maven. On Mac or Linux, this is as simple as the following command(s):
% mkdir -p src/main/java src/main/resources src/main/webapp src/main/config src/test/resources src/test/config

2 Copy build components to necessary project configuration locations

build components are typically located in your build/ directory in you KFS project. To do this, I like to use rsync because then I can exclude .svn and .git metadata.
% rsync -avC distribution/ external/ helper-scripts/ project/ properties/ tomcat/ upgrades/ src/main/config/
I have left out the library directories because maven handles our dependencies for us. There is no longer any reason to include the jars in the project. I am not completely sure if tomcat is necessary either, but I left it until later when I can decide if I really need it or not.

3 Move any other project configuration

I consider workflow doctypes, ddl, and dml to be project configuration metadata. These are typically in work/db and work/workflow respectively. I want them to be in src/main/config with the rest of my project configuration.
% mv work/workflow work/db src/main/config

4 Relocate java sources and project resources

Project resources are really anything that isn't a java source. These are going into src/main/resources. Java sources will go into src/main/java. Currently, the KFS project lumps these up into work/src. I am going to split this up.

First, I want to copy the source.

% rsync -avC work/src src2
Now I can do with it what I want which is to remove all the java source from it.
% find src2/ -name \*.java -exec {} \;

Copy project resources

Now that the java code is gone I copy the resources
% rsync -avC src2/* src/main/resources
% rm -rf src2

Copy java sources

I got all the resources, now I will move all my java source.
% find work/src \! -name \*.java -type f -exec rm {} \
% rsync -avC work/src/* src/main/java

5 Relocate tests

I like the way KFS has separated tests. They're interdependent still, but maven also differentiates between test types. I'll keep this separation and move the tests over to src/test/java
% rm -rf test/lib
% mv test/* src/test/java

6 Clean up a little

Now I get rid of all the unnecessary stuff
% rm -rf test work build *.xml
You should now have something that looks like this
leo@behemoth~/.workspace/kfs/release-4-0-mvn
(15:59:41) [6] ls
cnv-build.properties ptd-build.properties src
dev-build.properties reg-build.properties
leo@behemoth~/.workspace/kfs/release-4-0-mvn
(15:59:42) [7]

Crazy, right?

7 Get the pom.xml

To make things easy. I have a pom already. You can get it like this:
% svn export https://svn.rsmart.com/svn/kuali/contribution/community/travel_module/branches/release-4-0-mvn/pom.xml

I'm not going to paste it in here. It's too long.
For the most part, you will not need to change the pom.xml. The groupId, artifactid, or version may be properties about the project you will want to change. Other than that, everything should be peachy and ready to go (with the pom.xml at least).

8 Fix some properties in KFS

There are a couple things that do not jive with maven's conventions in the old KFS. Some are directory structure related. Mostly is the use of ant.project.name allover the place. This property is specific to ant only and not very generic. Therefore, it cannot be used with maven. Maven had the right idea by going with generic property name like project.name

Fix ant.project.name

First, thing to do is clean this up. I used grep to determine where al it's used
leo@behemoth~/.workspace/kfs/travel_module/build/properties
(16:07:09) [5] grep -l ant.project.name *
build-foundation.properties
directory.properties
email.properties
url.properties
I just went through those files and replaced ant. with nothing.

Fix directory properties

Next, is to fix all the properties that point to non-conventional places.
webroot.directory=src/main/webapp
build.directory=src/main/config

Maven Overlay!

The following are more about setting up an overlay project.

9 Install the war

We install the war in our local repository
% mvn -Dmaven.test.skip=true install

10 Create a jar and install it

We need both jar and war to do the overlay, so let's create one and install it.
% mvn jar:jar
% mvn install:install-file -Dpackaging=jar -DgroupId=org.kuali.kfs -DartifactId=kfs -Dversion=4.0 -DgeneratePom=true -Dfile=target/kfs-dev.jar

11 Setup your overlay

Now you pretty much just repeat steps 1-3. This is mostly to get the build configuration going again. Everything will be overlaid on top of the existing war, so we don't need to add resources our java sources.

12 Copy overlay pom.xml

Just like with the KFS project pom.xml, I have a good overlay pom with the dependencies setup just right!
%  svn export https://svn.rsmart.com/svn/kuali/contribution/community/travel_module/branches/release-4-0-overlay/pom.xml


That's it!!! You now have mavenized KFS and you're using an overlay to boot!

Want to see how I did it? Checkout the Mavenize KFS Screencasts

Monday, June 6, 2011

Another Game Changer: Mavenize KFS Screencasts

Overview

I have put together what I believe to be as a reproducable methodology to converting a KFS project to Maven. There will be a formal blog post detailing the adventure, but for now wet your appetite with these screencasts.

Screencasts


Mavenize KFS Part 1


Mavenize KFS Part 2


Mavenize KFS Part 3: Create an Overlay Project

Constants Done with Spring Part 2: Inheriting Constants

Motivation

In another chapter, I wrote about constants implemented through Spring. In one of my reasons for doing so, I illustrated that in Kuali, constraints are difficult to manipulate because of their static final nature. I called them an antipattern. Here's an example again:

Notice again that these constants cannot be manipulated. In the other chapter, I approached the use case that one would want to modify an existing constant and change it for their institution. In this chapter, I am going to approach the use case of adding a new constant. Why would you do this? Why not just make a new constants class, right? Well, you could do that. I find it to be more elegant to reuse your constants interface instead of having more than one.

The Pattern

Since we are using Spring, let's assume that we are using the aforementioned Spring constants implementation instead of the antipattern. Now that we know we are using Spring, there are a couple assumptions we can make.
  • Inversion of Control - we can assume that we can overwrite any known beans with the name we want to occupy. That is, if there is an existing constants implementation, we can not only reuse it, but replace it as well.
  • Dependency Injection - we can assume that we can inject any known constants we may already have into our new constants implementation.
  • We can reuse the existing KFS parentBean pattern.


With that, our methodology here is to first inject our existing constants into a new bean, then overwrite our old bean with it.

The Screencast




The Steps

1 Create a new Constants bean to be the delegate.

I created the following as my delegate.
<!--
Copyright 2007 The Kuali Foundation. Licensed under the Educational
Community License, Version 1.0 (the "License"); you may not use this
file except in compliance with the License. You may obtain a copy of
the License at http://www.opensource.org/licenses/ecl1.php Unless
required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the specific language governing
permissions and limitations under the License.
-->
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">

<bean id="kimDelegateConstants" class="org.kuali.rice.kim.util.ConstantsImpl">
<property name="kimLdapIdProperty"         value="uaid" />
<property name="kimLdapNameProperty"       value="uid" />
<property name="snLdapProperty"            value="sn" />
<property name="givenNameLdapProperty"     value="givenName" />
<property name="entityIdKimProperty"       value="entityId" />
<property name="employeeMailLdapProperty"  value="mail" />
<property name="employeePhoneLdapProperty" value="employeePhone" />
<property name="defaultCountryCode"        value="1" />
<property name="mappedParameterName"       value="KIM_TO_LDAP_FIELD_MAPPINGS" />
<property name="mappedValuesName"          value="KIM_TO_LDAP_VALUE_MAPPINGS" />
<property name="parameterNamespaceCode"    value="KR-SYS" />
<property name="parameterDetailTypeCode"   value="Config" />
<property name="personEntityTypeCode"      value="PERSON" />
<property name="employeeIdProperty"        value="emplId" />
<property name="departmentLdapProperty"    value="employeePrimaryDept" />  
<property name="employeeTypeProperty"      value="employeeType" />
<property name="employeeStatusProperty"    value="employeeStatus" />
<property name="affiliationLdapProperty"   value="affiliationProperty" />
<property name="primaryAffiliationLdapProperty"   value="eduPersonPrimaryAffiliation" />
<property name="defaultCampusCode"         value="MC" />
<property name="defaultChartCode"          value="UA" />
<property name="taxExternalIdTypeCode"     value="TAX" />
<property name="externalIdProperty"        value="externalIdentifiers.externalId" />
<property name="externalIdTypeProperty"    value="externalIdentifiers.externalIdentifierTypeCode" />
<property name="affiliationMappings"       value="staff=STAFF,faculty=FCLTY,employee=STAFF,student=STDNT,affilate=AFLT"/>
<property name="employeeAffiliationCodes"  value="STAFF,FCLTY" />
</bean>
...
...
</beans>
The important thing to notice is that it's in a new bean and not kimConstants any longer.

2 Override the original kimConstants bean

Now you need to create a new kimConstants bean that uses your implementation and delegates to the delegate created in Step 1
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright 2007 The Kuali Foundation. Licensed under the Educational
Community License, Version 1.0 (the "License"); you may not use this
file except in compliance with the License. You may obtain a copy of
the License at http://www.opensource.org/licenses/ecl1.php Unless
required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the specific language governing
permissions and limitations under the License.
-->
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">
...
...
<bean id="kimConstants" class="edu.arizona.kim.util.ConstantsImpl">
<property name="phoneNumberMask"           value="###-###-####" />
<property name="delegate"                  value-ref="kimDelegateConstants" />
</bean>
</beans>
Here a new kimConstants is created. It only has 2 properties. One is for the delegate and the other is the field I wanted to add.

3. Create a new interface for your constants that includes your new field

/*
* Copyright 2007-2009 The Kuali Foundation
*
* Licensed under the Educational Community License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.opensource.org/licenses/ecl2.php
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package edu.arizona.kim.util;


/**
* Extension interface for constants
*/
public interface Constants extends org.kuali.rice.kim.util.Constants {

/**
* Retrieve the phoneNumberMask constant
*
* @return String instance of the phoneNumberMask
*/
String getPhoneNumberMask();
}
You can see it extends the original constants interface and thereby gaining all its abstract methods for all the delegate method's properties.

4 Create the implementation class for the new bean

/*
* Copyright 2007-2009 The Kuali Foundation
*
* Licensed under the Educational Community License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.opensource.org/licenses/ecl2.php
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package edu.arizona.kim.util;

import java.util.Collection;

import org.kuali.rice.kim.bo.entity.dto.KimEntityInfo;

/**
* Extension implementation for constants
*
*/
class ConstantsImpl implements Constants {
private String phoneNumberMask;
private org.kuali.rice.kim.util.Constants delegate;

public void setDelegate(final org.kuali.rice.kim.util.Constants delegate) {
this.delegate = delegate;
}

public org.kuali.rice.kim.util.Constants getDelegate() {
return delegate;
}

public Collection getTestPrincipalNames() {
return getDelegate().getTestPrincipalNames();
}


/**
* Gets the value of entityPrototype
*
* @return the value of entityPrototype
*/
public KimEntityInfo getEntityPrototype() {
return getDelegate().getEntityPrototype();
}

/**
* Gets the value of externalIdTypeProperty
*
* @return the value of externalIdTypeProperty
*/
public String getExternalIdTypeProperty() {
return getDelegate().getExternalIdTypeProperty();
}

/**
* Gets the value of taxExternalIdTypeCode
*
* @return the value of taxExternalIdTypeCode
*/
public String getTaxExternalIdTypeCode() {
return getDelegate().getTaxExternalIdTypeCode();
}


/**
* Gets the value of externalIdProperty
*
* @return the value of externalIdProperty
*/
public String getExternalIdProperty() {
return getDelegate().getExternalIdProperty();
}

/**
* Gets the value of employeePhoneLdapProperty
*
* @return the value of employeePhoneLdapProperty
*/
public String getEmployeePhoneLdapProperty() {
return getDelegate().getEmployeePhoneLdapProperty();
}

/**
* Gets the value of employeeMailLdapProperty
*
* @return the value of employeeMailLdapProperty
*/
public String getEmployeeMailLdapProperty() {
return getDelegate().getEmployeeMailLdapProperty();
}

/**
* Gets the value of defaultCountryCode
*
* @return the value of defaultCountryCode
*/
public String getDefaultCountryCode() {
return getDelegate().getDefaultCountryCode();
}

/**
* Gets the value of personEntityTypeCode
*
* @return the value of personEntityTypeCode
*/
public String getPersonEntityTypeCode() {
return getDelegate().getPersonEntityTypeCode();
}

/**
* Gets the value of kimLdapIdProperty
*
* @return the value of kimLdapIdProperty
*/
public String getKimLdapIdProperty() {
return getDelegate().getKimLdapIdProperty();
}

/**
* Gets the value of kimLdapNameProperty
*
* @return the value of kimLdapNameProperty
*/
public String getKimLdapNameProperty() {
return getDelegate().getKimLdapNameProperty();
}

/**
* Gets the value of snLdapProperty
*
* @return the value of snLdapProperty
*/
public String getSnLdapProperty() {
return getDelegate().getSnLdapProperty();
}

/**
* Gets the value of givenNameLdapProperty
*
* @return the value of givenNameLdapProperty
*/
public String getGivenNameLdapProperty() {
return getDelegate().getGivenNameLdapProperty();
}

/**
* Gets the value of entityIdKimProperty
*
* @return the value of entityIdKimProperty
*/
public String getEntityIdKimProperty() {
return getDelegate().getEntityIdKimProperty();
}

/**
* Gets the value of parameterNamespaceCode
*
* @return the value of parameterNamespaceCode
*/
public String getParameterNamespaceCode() {
return getDelegate().getParameterNamespaceCode();
}

/**
* Gets the value of parameterDetailTypeCode
*
* @return the value of parameterDetailTypeCode
*/
public String getParameterDetailTypeCode() {
return getDelegate().getParameterDetailTypeCode();
}

/**
* Gets the value of mappedParameterName
*
* @return the value of mappedParameterName
*/
public String getMappedParameterName() {
return getDelegate().getMappedParameterName();
}

/**
* Gets the value of unmappedParameterName
*
* @return the value of unmappedParameterName
*/
public String getUnmappedParameterName() {
return getDelegate().getUnmappedParameterName();
}

/**
* Gets the value of mappedValuesName
*
* @return the value of mappedValuesName
*/
public String getMappedValuesName() {
return getDelegate().getMappedValuesName();
}

/**
* Gets the value of employeeIdProperty
*
* @return the value of employeeIdProperty
*/
public String getEmployeeIdProperty() {
return getDelegate().getEmployeeIdProperty();
}

/**
* Gets the value of departmentLdapProperty
*
* @return the value of departmentLdapProperty
*/
public String getDepartmentLdapProperty() {
return getDelegate().getDepartmentLdapProperty();
}

/**
* Gets the value of employeeTypeProperty
*
* @return the value of employeeTypeProperty
*/
public String getEmployeeTypeProperty() {
return getDelegate().getEmployeeTypeProperty();
}

/**
* Gets the value of employeeStatusProperty
*
* @return the value of employeeStatusProperty
*/
public String getEmployeeStatusProperty() {
return getDelegate().getEmployeeStatusProperty();
}

/**
* Gets the value of defaultCampusCode
*
* @return the value of defaultCampusCode
*/
public String getDefaultCampusCode() {
return getDelegate().getDefaultCampusCode();
}

public String getDefaultChartCode() {
return getDelegate().getDefaultChartCode();
}

public String getEmployeeAffiliationCodes() {
return getDelegate().getEmployeeAffiliationCodes();
}
public String getAffiliationMappings() {
return getDelegate().getAffiliationMappings();
}

/**
* Gets the mappings for the affiliation ldap property
* @return mapping for KIM affiliation and LDAP
*/
public String getAffiliationLdapProperty() {
return getDelegate().getAffiliationLdapProperty();
}

/**
* Gets the mappings for the primary affiliation ldap property
* @return mapping for KIM primary affiliation and LDAP
*/
public String getPrimaryAffiliationLdapProperty() {
return getDelegate().getPrimaryAffiliationLdapProperty();
}

/**
* @see edu.arizona.kim.util.Constants#getPhoneNumberMask()
*/
public String getPhoneNumberMask() {
return this.phoneNumberMask;
}

/**
* Sets the phone number mask constant
*
* @param phoneNumberMask value to set for the phoneNumberMask constant
*/
public void setPhoneNumberMask(final String phoneNumberMask) {
this.phoneNumberMask = phoneNumberMask;
}
}
Notice the following allover the place
/**
* Gets the mappings for the primary affiliation ldap property
* @return mapping for KIM primary affiliation and LDAP
*/
public String getPrimaryAffiliationLdapProperty() {
return getDelegate().getPrimaryAffiliationLdapProperty();
}
The getDelegate() call is accessing the delegate we setup in spring. The dependency injection takes care of everything for us, so we don't need to worry about it being a package level class either. We also only override the getters because this is a constant. We're not supposed to change it.

Conclusion

There you have it. Another reason to use constants in spring. Not only are they easy to define, but now you can use object composition to inherit constants from other beans.