Fork me on GitHub

n. Slang a rough lawless young Kuali developer.
[perhaps variant of Houlihan, Irish surname]
kualiganism n

Blog of an rSmart Java Developer. Full of code examples, solutions, best practices, et al.

Sunday, June 26, 2011

Hot Code Replacement for KFS Development With Tomcat

Overview

This is not hot deployment as in reloadable="true" like here:
<Context path="/kfs-dev" reloadable="true"  docBase="/Users/leo/.workspace/kfs/release-4-0-overlay/target/kfs-dev">
No no. That will force the webapp context to reload which is the trouble with KFS in tomcat. When the context reloads there are often memory problems. Worst of all, Spring has to reload everything. That's just a joy and a pleasure to experience for all developers too. Not only do you wait forever and a day for Spring to reload your DD, but then it finishes to remind you there's a memory leak with context redeployment. Pleasant, isn't it?

Solution 1: Use Maven

Honestly, the best thing you can do is this Convert Your KFS Distribution to Maven. You can then use the much coveted Tomcat Maven Plugin oooor the lesser Jetty Maven Plugin. These will support not only hot redeploys, but Hot Code Replacement. You may even avoid Spring reloading.

Solution 2: JDI (Java Debugging Interface)

There may be some of you that would rather not be bothered with converting to maven. You may just not be ready yet, but you love the idea of Hot Code Replacement because you're a developer and developers hate 2 things
  • Waiting around
  • Waiting around when they don't need to.
What's a developer to do? Brilliantly, before Sun was overtaken by the evil empire (Oracle), they left us with a few magical treasures to save our dying world from the evil that is upon us poised to deliver its reign of destruction and bind and afflict us. One is JDI (Java Debugging Interface) or JPDA (Java Platform Debugger Architecture) as it is sometimes referred. A veritable sword of omens as you will soon find out.

This is actually how Eclipse handles its Hot Code Replacement in debug mode. It's actually using JDI/JPDA. Given is that this will run slower than normal applications. It is also Debug mode. That is, you would never ever ever ever use this in production...ever. This is a development tool. The great part is that since Eclipse is also using JDI, that means that attaching a debugger or starting Eclipse in debug mode works seamlessly with this technique.

In a Nutshell

The simple explanation for my solution is:
  1. Create a Quartz Job (WebappWatchJob) with a step (WebappWatchStep)
  2. Set the job to run every 30 seconds
  3. Add necessary configuration to directory.properties and configuration.properties so this only shows up when dev.mode is true
  4. Implement WebappWatchJob to traverse your webapp installation path for .class files and check for last modified updates to reload the class on change

...in a nutshell.

The Details and Steps

Deep breath...now go!

1. Setup Properties

In configuration.properties I added
##############################################################################################################
## Properties from institutional.configuration.file (${institutional.configuration.file}) are appended after this point.
##############################################################################################################
webapp.classes.directory=${webapp.classes.directory}

I then set in directory.properties
# Application server directories - these assume the Tomcat 5.5 structure
tomcat.version=5
#appserver.lib.dir=${appserver.home}/common/lib
#appserver.classes.dir=${appserver.home}/common/classes
appserver.deploy.dir=${appserver.home}/webapps
appserver.config.dir=${appserver.home}/conf
appserver.localhost.dir=${appserver.config.dir}/Catalina/localhost
appserver.work.dir=${appserver.home}/work/Catalina/localhost
webapp.classes.directory=${appserver.deploy.dir}/${project.name}-${build.environment}/WEB-INF/classes

The above defines a property we will eventually use to set which path to watch for changes in.

2. Create WebappWatchJob

I add the following to my spring configuration. In KFS, my module is tem, so I add it to my spring-tem.xml:
<?xml version="1.0" encoding="UTF-8"?>
<!-- Copyright 2006-2008 The Kuali Foundation Licensed under the Educational 
Community License, Version 2.0 (the "License"); you may not use this file 
except in compliance with the License. You may obtain a copy of the License 
at http://www.opensource.org/licenses/ecl2.php Unless required by applicable 
law or agreed to in writing, software distributed under the License is distributed 
on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either 
express or implied. See the License for the specific language governing permissions 
and limitations under the License. -->
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"
xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://www.springframework.org/schema/tx
http://www.springframework.org/schema/tx/spring-tx-2.0.xsd
http://www.springframework.org/schema/aop
http://www.springframework.org/schema/aop/spring-aop-2.0.xsd">
...
...
<property name="jobNames">
<list>
<value>webappWatchJob</value>
</list>
</property>
<property name="triggerNames">
<list>
<value>webappWatchTrigger</value>
</list>
</property>
...
...
</beans>

Clearly, I have declared a webappWatchJob and a webappWatchTrigger. I later define those in the same file this way:
<?xml version="1.0" encoding="UTF-8"?>
<!-- Copyright 2006-2008 The Kuali Foundation Licensed under the Educational 
Community License, Version 2.0 (the "License"); you may not use this file 
except in compliance with the License. You may obtain a copy of the License 
at http://www.opensource.org/licenses/ecl2.php Unless required by applicable 
law or agreed to in writing, software distributed under the License is distributed 
on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either 
express or implied. See the License for the specific language governing permissions 
and limitations under the License. -->
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p"
xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://www.springframework.org/schema/tx
http://www.springframework.org/schema/tx/spring-tx-2.0.xsd
http://www.springframework.org/schema/aop
http://www.springframework.org/schema/aop/spring-aop-2.0.xsd">
...
...
<bean id="webappWatchStep" class="org.kuali.kfs.module.tem.batch.WebappWatchStep" parent="step">
<property name="webappPath" value="${webapp.classes.directory}" />
</bean>

<bean id="webappWatchJob" parent="scheduledJobDescriptor">
<property name="steps">
<list>
<ref bean="webappWatchStep" />
</list>
</property>
</bean>

<bean id="webappWatchTrigger" parent="cronTrigger">
<property name="jobName" value="webappWatchJob" />
<property name="cronExpression" value="0/30 * * * * ?" />
</bean>
</beans>

Notice that our property we created before is used here. It is injected into the step so that it may be watched. Also, the cronExpression is set for every 0 and 30th second or every 30 seconds.

3. Implement WebappWatchStep

Now that we have the spring definitions setup, we need to implement the WebappWatchStep. I will first explain its pieces and then paste the full class implementation.

Properties

There are just 2 properties.
...
... 
/**
* Gets the value of WebappPath
*
* @return the value of WebappPath
*/
public String getWebappPath() {
return this.webappPath;
}

/**
* Sets the value of WebappPath
*
* @param argWebappPath Value to assign to this.WebappPath
*/
public void setWebappPath(final String argWebappPath) {
this.webappPath = argWebappPath;
}

/**
* Gets the value of LastUpdated
*
* @return the value of LastUpdated
*/
public Date getLastUpdated() {
return this.lastUpdated;
}

/**
* Sets the value of LastUpdated
*
* @param argLastUpdated Value to assign to this.LastUpdated
*/
public void setLastUpdated(final Date argLastUpdated) {
this.lastUpdated = argLastUpdated;
}
...
...
  • webappPath - is the path injected to define what directory to recurse and watch
  • lastUpdated - is a cached date of the last run. This is compared to the file lastModified property to determine whether the file has changed or not


Traversing webappPath

...
...
/**
* @see org.kuali.kfs.sys.batch.Step#execute(java.lang.String, java.util.Date)
*/
public boolean execute(String jobName, Date jobRunDate) throws InterruptedException {
info("WebappWatch triggered. I'm going in");
final Collection<File> classFiles = new ArrayList<File>();
traverseForClasses(new File(getWebappPath()), classFiles);
...
...
}
...
...
/**
* Recursively searches for classes. If <code>file</code> is a directory, it traverses recursively. If it is a
* file, then it compares the <code>lastModified</code> date with the cached {@link #getLastModified()} value.
* 
* @param file {@link File} instance for traversing
* @param classFiles is a {@link Collection} of {@link File} instances where the {@link File#lastUpdated()} value is
* before {@link #getLastUpdated()}
*/
protected void traverseForClasses(final File file, final Collection<File> classFiles) {
if (file.isDirectory()) {
final String[] fileNames = file.list();

for (final String fileName : fileNames) {
traverseForClasses(new File(file, fileName), classFiles);
}
}
else if (new Date(file.lastModified()).after(lastUpdated) && file.getName().endsWith(".class")) {
classFiles.add(file);
}
}
...    
...

A classFiles Collection is passed into a method that recursively adds class names to that collection. It is later iterated over in the execute method like this:
...
...
try {
for (final File classFile : classFiles) {
final byte[] classData = new byte[new Long(classFile.length()).intValue()];
FileInputStream classStream = null;
try {
classStream = new FileInputStream(classFile);

classStream.read(classData, 0, classData.length);
}
catch (Exception e) {
}
finally {
if (classStream != null) {
try {
classStream.close();
}
catch (Exception e) { } // failure to close. Do nothing.
}
}

final String className = classFromFileName(classFile.getAbsolutePath()); 
boolean classExists = false;
try {
Class.forName(className);
classExists = true;
}
catch (ClassNotFoundException cnfe) {
}

if (classExists) {
addRedefineClass(toRedefineMap, className, classData);
}
else {
defineClass(className, classData, 0, classData.length);
}
}

debug("Delivering the redefinition map");
vm.redefineClasses(toRedefineMap);
}
catch (Exception e) {
warn(e.getMessage());
e.printStackTrace();
Throwable cause = e.getCause();
while (cause != null) {
cause.printStackTrace();
warn("Caused by: ");
cause = cause.getCause();
}
}
finally {
try {
vm.resume();
if (conn != null && conn.isOpen()) {
debug("Closing the JPDA connection...");
conn.close();
debug("Connection closed");
}
}
catch (Exception e) {
e.printStackTrace();
}
}
...
...

The Juicy Part

So what is this stuff we saw in the execute method?
...
...
/**
* @see org.kuali.kfs.sys.batch.Step#execute(java.lang.String, java.util.Date)
*/
public boolean execute(String jobName, Date jobRunDate) throws InterruptedException {
...
...
try {
conn = openJdiConnection();
vm = Bootstrap.virtualMachineManager().createVirtualMachine(conn);
}
catch (Exception e) {
return false;
}
...
...
try {
vm.resume();
if (conn != null && conn.isOpen()) {
debug("Closing the JPDA connection...");
conn.close();
debug("Connection closed");
}
}
catch (Exception e) {
e.printStackTrace();
}
...
...
}
...
...

Near the beginning of the execute method, we establish a connection with JPDA using the openJdiConnection described here
...
...
/**
* Creates a {@link SocketTransportService} instance and uses that to attach to localhost:8080 and return a {@link Connection} to the 
* current running JVM instance. This is used by bootstrap to create a {@link VirtualMachine} instance.
* 
* @throws Exception when there's a problem attaching to localhost:8080 or if it cannot create an instance of {@link SocketTransportService}
* @return an open {@link Connection} to the JPDA instance of the current running JVM
* @see com.sun.jdi.VirtualMachineManager#createVirtualMachine(Connection)
* @see com.sun.tools.jdi.SocketTransportService
* @see com.sun.jdi.Bootstrap#virtualMachineManager()
*/
protected Connection openJdiConnection() throws Exception {
TransportService ts = null;
try {
Class c = Class.forName("com.sun.tools.jdi.SocketTransportService");
ts = (TransportService)c.newInstance();
} catch (Exception x) {
throw new Error(x);
}

return ts.attach("localhost:8000", 5000, 5000);
}
...
...

The builtin com.sun.tools.jdi.SocketTransportService is used to connect to the JPDA. I've set it to default to port 8000. I believe that's a standard, but you may want to customize this further. Tomcat uses the following environment variables to define this. It may be leveraged here.
#   JPDA_TRANSPORT  (Optional) JPDA transport used when the "jpda start"
#                   command is executed. The default is "dt_socket".
#
#   JPDA_ADDRESS    (Optional) Java runtime options used when the "jpda start"
#                   command is executed. The default is 8000.
#
#   JPDA_SUSPEND    (Optional) Java runtime options used when the "jpda start"
#                   command is executed. Specifies whether JVM should suspend
#                   execution immediately after startup. Default is "n".
#
#   JPDA_OPTS       (Optional) Java runtime options used when the "jpda start"
#                   command is executed. If used, JPDA_TRANSPORT, JPDA_ADDRESS,
#                   and JPDA_SUSPEND are ignored. Thus, all required jpda
#                   options MUST be specified. The default is:
#
#                   -Xdebug -Xrunjdwp:transport=$JPDA_TRANSPORT,
#                       address=$JPDA_ADDRESS,server=y,suspend=$JPDA_SUSPEND
#


Once the connection is made, it eventually also needs to be closed
...
...
try {
vm.resume();
if (conn != null && conn.isOpen()) {
debug("Closing the JPDA connection...");
conn.close();
debug("Connection closed");
}
}
catch (Exception e) {
e.printStackTrace();
}
...
...

You can see that right before the connection is closed, the virtual machine is resumed with the resume method. This is in case some process caused the vm to become suspended. It resumes all threads before disconnecting.

While looping over the modified class names, we ran into this
...
...
final String className = classFromFileName(classFile.getAbsolutePath()); 
boolean classExists = false;
try {
Class.forName(className);
classExists = true;
}
catch (ClassNotFoundException cnfe) {
}

if (classExists) {
addRedefineClass(toRedefineMap, className, classData);
}
else {
defineClass(className, classData, 0, classData.length);
}
...
...

Simply put, we use a method defineClass for classes that are new and addRedefineClass for methods that are already loaded with the current ClassLoader. Loading a class is easy reflection with the ClassLoader, so I am going to skip that. The tough part is handling classes that are already loaded. This is where JPDA comes in. Normally, you can only define a class once per ClassLoader, but with JDPA, you can use the redefineClasses method in the VirtualMachine. This redefineClasses method takes a Map though. This means we do not typically redefine a class at a time. We have to round them all up into a Map, and then call redefineClasses on the map. To do this, we created a addRedefineClass method which takes our already created Map, and puts classes into it. Let's have a look
...
...
/**
* Populates a map of {@link ReferenceType} and {@link byte[]} data. This {@link Map} is used by JDI for 
* redefining classes.
*
* @param toRedefineMap {@link Map} of {@link byte[]} mapped by {@link ReferenceType}. This map is populated by the method, so it shouldn't be null.
* @see com.sun.jdi.VirtualMachine#redefineClasses(Map, byte[])
*/
protected void addRedefineClass(final Map<ReferenceType,byte[]> toRedefineMap, final String className, final byte[] b) throws Exception { 
debug("Found change in ", className);
for (ReferenceType ref : vm.classesByName(className)) {
if (ref.name().equals(className) && className.indexOf("WebappWatch") < 0) {
debug("redefining ", className);
toRedefineMap.put(ref, b);
return;
}
}
}
...
...

Since I made vm a local variable in the WebappWatchStep, I can call it from addRedefineClass. I use it for its nifty classesByName method. I pass it my fully qualified class name, and it returns a list of ReferenceType instances. Typically, this will return only one class. Actually, I cannot think of any case where it wouldn't, but just to be sure, I double check that. I also omit the WebappWatchStep class. We wouldn't want to reload the class we're currently accessing VirtualMachine from. That would be bad. We simply add the ReferenceType as a key to the byte array representing the actual class. This will get used later by the VirtualMachine#redefineClasses method. For more information on ReferenceType, check out http://download.oracle.com/javase/6/docs/jdk/api/jpda/jdi/com/sun/jdi/ReferenceType.html


As promised

Here is the full source code
/*
* Copyright 2007 The Kuali Foundation
* 
* Licensed under the Educational Community License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
* 
* http://www.opensource.org/licenses/ecl2.php
* 
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.kuali.kfs.module.tem.batch;

import java.lang.reflect.Method;

import java.io.File;
import java.io.FileInputStream;

import java.util.ArrayList;
import java.util.Collection;
import java.util.Date;
import java.util.HashMap;
import java.util.Map;

import com.sun.jdi.Bootstrap;
import com.sun.jdi.ArrayType;
import com.sun.jdi.ClassType;
import com.sun.jdi.ReferenceType;
import com.sun.jdi.InterfaceType;
import com.sun.jdi.VirtualMachine;
import com.sun.jdi.connect.spi.Connection;
import com.sun.jdi.connect.spi.TransportService;

import org.kuali.kfs.sys.batch.AbstractStep;

import static org.kuali.kfs.module.tem.util.BufferedLogger.*;
import static org.apache.commons.lang.StringUtils.replace;

/**
* Batch step that traverses the webapp for changed classes. It then reloads the classes into the current {@link ClassLoader}
* this gives to appearance of hot redeploys.
*
* @author leo [at] rsmart.com
*/
public class WebappWatchStep extends AbstractStep {
private Date lastUpdated;
private String webappPath;
private Connection conn;
private VirtualMachine vm;

public WebappWatchStep() throws Throwable {
super();
lastUpdated = new Date();
}

/**
* @see org.kuali.kfs.sys.batch.Step#execute(java.lang.String, java.util.Date)
*/
public boolean execute(String jobName, Date jobRunDate) throws InterruptedException {
info("WebappWatch triggered. I'm going in");
final Collection<File> classFiles = new ArrayList<File>();
traverseForClasses(new File(getWebappPath()), classFiles);
final Map<ReferenceType,byte[]> toRedefineMap = new HashMap<ReferenceType,byte[]>();

try {
conn = openJdiConnection();
vm = Bootstrap.virtualMachineManager().createVirtualMachine(conn);
}
catch (Exception e) {
return false;
}

try {
for (final File classFile : classFiles) {
final byte[] classData = new byte[new Long(classFile.length()).intValue()];
FileInputStream classStream = null;
try {
classStream = new FileInputStream(classFile);

classStream.read(classData, 0, classData.length);
}
catch (Exception e) {
}
finally {
if (classStream != null) {
try {
classStream.close();
}
catch (Exception e) { } // failure to close. Do nothing.
}
}

final String className = classFromFileName(classFile.getAbsolutePath()); 
boolean classExists = false;
try {
Class.forName(className);
classExists = true;
}
catch (ClassNotFoundException cnfe) {
}

if (classExists) {
addRedefineClass(toRedefineMap, className, classData);
}
else {
defineClass(className, classData, 0, classData.length);
}
}

debug("Delivering the redefinition map");
vm.redefineClasses(toRedefineMap);
}
catch (Exception e) {
warn(e.getMessage());
e.printStackTrace();
Throwable cause = e.getCause();
while (cause != null) {
cause.printStackTrace();
warn("Caused by: ");
cause = cause.getCause();
}
}
finally {
try {
vm.resume();
if (conn != null && conn.isOpen()) {
debug("Closing the JPDA connection...");
conn.close();
debug("Connection closed");
}
}
catch (Exception e) {
e.printStackTrace();
}
}

setLastUpdated(new Date());
return false;
}

/**
* Defines a class. Only used for new classes that don't already exist in the {@link ClassLoader}
* @param className name of the {@link Class} to define
* @param b byte array of data
* @param offset where in the byte array to start
* @param len size of the byte array to read
* @throws Exception <code>defineClass()</code> on the {@link ClassLoader} is protected. It cannot normally be called.
* an exception can be thrown if something goes wrong.
*/
protected void defineClass(final String className, final byte[] b, final int offset, final int len) throws Exception {
info("Defining class ", className);
final ClassLoader classLoader = getClass().getClassLoader();

final Method method = ClassLoader.class.getDeclaredMethod("defineClass", 
new Class[] { String.class, byte[].class, int.class, int.class });
method.setAccessible(true);
final Class definedClass = (Class) method.invoke(classLoader, className, b, offset, len);
info("Defined class ", definedClass);
// info("Loaded class ", getClass().getClassLoader().loadClass(className));
}

/**
* Populates a map of {@link ReferenceType} and {@link byte[]} data. This {@link Map} is used by JDI for 
* redefining classes.
*
* @param toRedefineMap {@link Map} of {@link byte[]} mapped by {@link ReferenceType}. This map is populated by the method, so it shouldn't be null.
* @see com.sun.jdi.VirtualMachine#redefineClasses(Map, byte[])
*/
protected void addRedefineClass(final Map<ReferenceType,byte[]> toRedefineMap, final String className, final byte[] b) throws Exception { 
debug("Found change in ", className);
for (ReferenceType ref : vm.classesByName(className)) {
if (ref.name().equals(className) && className.indexOf("WebappWatch") < 0) {
debug("redefining ", className);
toRedefineMap.put(ref, b);
return;
}
}
}

/**
* Creates a {@link SocketTransportService} instance and uses that to attach to localhost:8080 and return a {@link Connection} to the 
* current running JVM instance. This is used by bootstrap to create a {@link VirtualMachine} instance.
* 
* @throws Exception when there's a problem attaching to localhost:8080 or if it cannot create an instance of {@link SocketTransportService}
* @return an open {@link Connection} to the JPDA instance of the current running JVM
* @see com.sun.jdi.VirtualMachineManager#createVirtualMachine(Connection)
* @see com.sun.tools.jdi.SocketTransportService
* @see com.sun.jdi.Bootstrap#virtualMachineManager()
*/
protected Connection openJdiConnection() throws Exception {
TransportService ts = null;
try {
Class c = Class.forName("com.sun.tools.jdi.SocketTransportService");
ts = (TransportService)c.newInstance();
} catch (Exception x) {
throw new Error(x);
}

return ts.attach("localhost:8000", 5000, 5000);
}

/**
* Tries to create a canonical class name from a .class file.
*
* @param fileName is the fully qualified path of a .class file
* @return a canonical class name (java.lang.String)
*/
protected String classFromFileName(final String fileName) {
String retval = fileName.substring(0, fileName.indexOf(".class"));
retval = retval.substring(getWebappPath().length() + 1);
retval = replace(retval, File.separator, ".");
retval = replace(retval, "/", ".");
return retval;
}

/**
* Recursively searches for classes. If <code>file</code> is a directory, it traverses recursively. If it is a
* file, then it compares the <code>lastModified</code> date with the cached {@link #getLastModified()} value.
* 
* @param file {@link File} instance for traversing
* @param classFiles is a {@link Collection} of {@link File} instances where the {@link File#lastUpdated()} value is
* before {@link #getLastUpdated()}
*/
protected void traverseForClasses(final File file, final Collection<File> classFiles) {
if (file.isDirectory()) {
final String[] fileNames = file.list();

for (final String fileName : fileNames) {
traverseForClasses(new File(file, fileName), classFiles);
}
}
else if (new Date(file.lastModified()).after(lastUpdated) && file.getName().endsWith(".class")) {
classFiles.add(file);
}
}

/**
* Gets the value of WebappPath
*
* @return the value of WebappPath
*/
public String getWebappPath() {
return this.webappPath;
}

/**
* Sets the value of WebappPath
*
* @param argWebappPath Value to assign to this.WebappPath
*/
public void setWebappPath(final String argWebappPath) {
this.webappPath = argWebappPath;
}

/**
* Gets the value of LastUpdated
*
* @return the value of LastUpdated
*/
public Date getLastUpdated() {
return this.lastUpdated;
}

/**
* Sets the value of LastUpdated
*
* @param argLastUpdated Value to assign to this.LastUpdated
*/
public void setLastUpdated(final Date argLastUpdated) {
this.lastUpdated = argLastUpdated;
}
}


That's it. Use this and you should have no problem modifying classes as you are developing without needing to restart Spring or Tomcat. Enjoy.

No comments:

Post a Comment