Thursday, April 17, 2008

Quartz - Job Processing Done Right

On our site, we have some background jobs that run periodically - we used JBoss to manage them, and it usually works. Sometimes they don't. The problem there is that once the job scheduler stops doing its thing, you can't really do anything but rerun the jobs periodically until you have a chance to bounce the server, which you can't do until night time (and obviously that isn't even particularly desirable). Well, today was one of those days. JBoss stopped running its scheduler, with nary a log message or stack trace. Just stopped. Yesterday it worked. Today, not so much. That sucks. We decided that it was a good time to untether ourself from JBoss and move to Quartz. In about a half of an afternoon, I had downloaded the code, perused the documentation, rewritten our scheduled processes to use Quartz, and tested and committed it. You can't often say that when you undertake something like this (believe me - we are currently in the testing stage of a long-overdue Hibernate 2-3 upgrade that hasn't been fun or easy), so I thought in honor of Quartz's simplicity and ease of use, I'd write a post, including how we got it working.

What is Quartz?

From their site:

"Quartz is a full-featured, open source job scheduling system that can be integrated with, or used along side virtually any J2EE or J2SE application"
How Do I Use Quartz?

Quartz is simple to use. To get started, download the distribution from their site, stick it in your classpath, and you are ready to go. First things first - you need to get your properties file set up. A simple configuration need only include this (or less if you want to fall back to defaults):

#===============================================================
# Configure Main Scheduler Properties
#===============================================================

org.quartz.scheduler.instanceName = QuartzScheduler
org.quartz.scheduler.instanceId = AUTO

#===============================================================
# Configure ThreadPool
#===============================================================

org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount = 5
org.quartz.threadPool.threadPriority = 5

#===============================================================
# Configure JobStore
#===============================================================

org.quartz.jobStore.misfireThreshold = 60000
org.quartz.jobStore.class = org.quartz.simpl.RAMJobStore


Once you have this going, you need to setup your jobs file. Most of the examples that came with Quartz show how to declare jobs in Java code. This is fine and all, but what if you don't want to have to recompile to change a quick setting? No good. I want to use XML. If you don't agree, feel free to follow one of the many examples they provide. To use XML, there are a couple things to keep in mind. First off - Quartz doesn't want to read these jobs from XML by default - you have to tell it to. In order to do so, you just need to add the following to quartz.properties:

org.quartz.plugin.jobInitializer.class = org.quartz.plugins.xml.JobInitializationPlugin
org.quartz.plugin.jobInitializer.fileName = my-jobs.xml
org.quartz.plugin.jobInitializer.overWriteExistingJobs = false
org.quartz.plugin.jobInitializer.failOnFileNotFound = true


Then just create a file my-jobs.xml:

<?xml version="1.0" encoding="UTF-8"?>
<quartz xmlns="http://www.opensymphony.com/quartz/JobSchedulingData"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
overwrite-existing-jobs="true">

<job>
<job-detail>
<name>AwesomeProcessor</name>
<group>awesome-group</group>
<description>Run the awesome job</description>
<job-class>
com.sportsvite.AwesomeProcessor
</job-class>
<job-data-map allows-transient-data="false">
<entry>
<key>superbad</key>
<value>true</value>
</entry>
</job-data-map>
</job-detail>

<trigger>
<cron>
<name>awesomeTrigger</name>
<group>awesome-trigger-group</group>
<job-name>awesomeProcessor</job-name>
<job-group>awesome-group</job-group>
<!-- trigger every 30 min -->
<cron-expression>0 0/30 * * * ?</cron-expression>
</cron>
</trigger>
</job>
</quartz>

Pretty simple, right? That's all you have to do on the configuration side.

NOTE: When you deploy the code, make sure that these two files end up in WEB-INF/classes so they are picked up in the classpath.

Now you need to kick off the Scheduler somehow. Most of the exercises show you how to do it in the code, but we don't want to rely on that, right? If you are running in a servlet container, they give you a simple servlet, the QuartzInitializerServlet. It is included in the jar file, and all you have to do is add it to the web.xml:


<servlet>
<servlet-name>QuartzInitializer</servlet-name>
<display-name>Quartz Initializer Servlet</display-name>
<servlet-class>org.quartz.ee.servlet.QuartzInitializerServlet</servlet-class>
<load-on-startup>1</load-on-startup>
</servlet>

So we have told the Quartz scheduler to kick off a job called AwesomeProcessor, so we need to code the class. This is simple - you just need a class that implements the Quartz interface Job, and has the following method:

public void execute( JobExecutionContext ctxt )
{
String s = (String) ctxt.getString( "superbad" );
doFoo( s );
}

Note the use of the JobExecutionContext - you can use that to pass a bunch of arguments in a Map-like structure. It corresponds to the job-data element in the my-job.xml file.

Now when you fire up your server, the Quartz scheduler will activate and you'll see a message like this in standard out:

18:27:45,722 INFO [JobSchedulingDataProcessor] Scheduling 1 parsed job.
18:27:45,723 INFO [JobSchedulingDataProcessor] Adding job: awesome-group.AwesomeProcessor
18:27:45,735 INFO [JobSchedulingDataProcessor] 1 scheduled job.
18:27:45,735 INFO [QuartzScheduler] Scheduler QuartzScheduler_$_NON_CLUSTERED started.
18:27:45,736 INFO [Engine] StandardContext[]QuartzInitializer: Scheduler has been started...
18:27:45,736 INFO [Engine] StandardContext[]QuartzInitializer: Storing the Quartz Scheduler Factory in the servlet context at key: org.quartz.impl.StdSchedulerFactory.KEY

That's it! It's going to work now. The best thing about Quartz is that if you have bad XML or some other issue, it gives you a meaningful error message that you can easily remedy. Not all open source products can say the same.

Kudos to Quartz. It made my programming day pretty pleasant. Hope this helps others get up to speed. For a bunch of examples and other helpful information, check out the Quartz Wiki.

6 comments:

Brendan said...

Thanks for this post.

Can you change the XML file without restarting the server or redeploying the application? For example, if I have a cron expression set for a 5 minute interval, can I change it to 10 without restarting the application?

Kirk Gray said...

Brendan,
I honestly am not sure - we deploy to JBoss, and the configuration file is in the Ear, so we can't make that work dynamically. I don't specifically see any documentation about it on the site or anywhere else on the web.
If you find anything, please do post a comment here.
Thanks,
Kirk

Brendan said...

Thanks, Kirk. Unfortunately, I have not been able to get the example to work yet... When I deploy, it appears that the server is using the default quartz.properties file and not mine (I too am deploying to JBoss; version 4.2.2 in my case).

I have the quartz.properties and the my-jobs.xml file in the WEB-INF/classes folder.

Brendan said...

Great news! I was able to get the example working, and externalize the quartz jobs xml file. Your quartz.properties file setting: org.quartz.plugin.jobInitializer.fileName = my-jobs.xml was important. It seems that when I tried to externalize the xml file when I had it named quartz_jobs.xml, it caused errors on the server. I know that this is the default name that Quartz looks for, so maybe this was causing a conflict with some Quartz jobs that the server was managing. Once I pointed to a file of a different name (my-jobs.xml), it was fine...

So I basically kept the quartz.properties file in my project as you've shown, and then I put the my-jobs.xml file in the JBoss server's /conf directory. You do not have to fully qualify the my-jobs.xml file in quartz.properties. It will find it as long as you have it in the /conf directory...

Now, one more thing - I tried changing the xml file in the /conf directory to see if I could change the interval while the job was running. This did not work. Quartz kept using the setting that the app started with. However, my buddy here told me that you can force a redeployment of a web app by doing a cd into the exploded project directory (under the server's deploy directory), and then you position underneath the /WEB-INF subdirectory and then do: touch web.xml (for Unix, of course). This basically restarts the app and the new interval will kick in. Note that this "touch trick" only forces a redeploy if you touch the web.xml file.

I would say that if you have a long enough interval, (and I guess it also depends on what the job does ;-) it shouldn't be a problem to redeploy in this manner. Might be less than ideal, but I think this will work in my case.

Thanks a lot, Kirk.

Ramya said...

I tried this example and it worked out. Thanks

I do have a doubt. How to trigger more than one job in jobs.xml. In our application we have to schedule as per the date and have to send out an email to the customer. I tried but only one job works and the other didnt work out. Got email for one job and didnt receive email for other job.

Can you please send some sample for more than one job?

Thanks in advance

Rajesh Kumar said...

Thank U Very much Krik...
i try to implement by practising quartz 2.1.5\examples\example10 but that was giving java.lang.NoClassDefFoundError: javax/transaction/UserTransaction.
But after i follow your blog, i achieved what i want....