Generating .ldif Files using Groovy

I recently needed to do a bulk update of an openLDAP directory to add a password for a really big batch of test users.  I wrote a little Groovy script to do it.  Thought it might be helpful.  First things first, generate a password for the test users using the slappasswd command.  Using the defaults will give a you password with SSHA encryption.  Now we need an .ldif file that describes the change to make.  For more information on ldapmodify commands, head here.  The script here will loop through a group of sequentially id'd users in a couple of ous and assign them our nicely hashed password:

        File newFile = new File('modify-script.ldif')
        String username = ''
        String toAdd = ""

        def stuCount = 80000
        def admCount = 2000
        def teachCount = 18000

        StringBuilder sb = new StringBuilder()

        def ous = [ 'ou1', 'ou2' ]

        for ( ouname in ous ) {

            for ( i in 1..stuCount ) {
                username = "user_${epname}_${i}"

                toAdd = """
dn: uid=${username},ou=people,ou=${ouname},dc=yourdc,dc=com
changetype: modify
add: userPassword
userPassword: {SSHA}zW7Q/yQQ8IKZiX8ANJIGugi0deNebN1o
                    
"""
                sb.append( toAdd )
            }

            sb.append( "\n\n" )

            newFile << sb.toString()
        }

This will produce a file (modify-script.ldif) with a bunch of entries like these:

dn: uid=user_ou1_1,ou=people,ou=ou1,dc=yourdc,dc=com
changetype: modify
add: userPassword
userPassword: {SSHA}zW7Q/yQQ8IKZiX8ANJIGugi0deNebN1o
                    

dn: uid=user_ou1_2,ou=people,ou=ou1,dc=yourdc,dc=com
changetype: modify
add: userPassword
userPassword: {SSHA}zW7Q/yQQ8IKZiX8ANJIGugi0deNebN1o

...

Now you can simply run the ldapmodify command to update the users:

ldapmodify -x -D "cn=admin,dc=yourdc,dc=com" -w yourpass -f modify-script.ldif

I've been able to use derivatives of this script for a few different tasks.  Hope it helps someone else.

Grails Image Processing with the ImageTools Plugin

I recently had a requirement on my side project to add some simple image processing.  I had previously used the ImageTools plugin, but that seems to have falled into a bit of disrepair.  I had a multistep process to get this going, and I figured I'd share it.

According to the git repository, the last checkin on version 1.0.4 was way back in January 2010.  Grabbing the plugin using the standard grails install-plugin as documented on the plugin documentation page just doesn't cut it.  When attempting to use the documentation, you get a message that tells you

unable to resolve class ImageTool @ line xxx, column xxx. def imageTool = new ImageTool();

This is troubling, as that is the only class included in the plugin.  Fortunately I found this comment on a closed (???) issue on github:

So, to solve this I had to run a git clone on this source tree and then follow the instruction at the link below to create my own version of this plugin with correct package.

http://grails.org/doc/latest/guide/12.%20Plug-ins.html

So it would seem that you need to grab the source, then create a plugin.  This is not as easy as you might think.  First of all, you might not have git installed.  Follow the instructions here to 'git git'.  Next, you need to grab the code.  From the command prompt in the directory you want the code, run the following command:

git clone http://github.com/ricardojmendez/grails-imagetools.git

Now you have the code.  Even though it's a plugin, you have to tell grails it's a plugin, by running the grails create-plugin command.  Now you have a zip file that you can install in your project, using the standard grails install-plugin command.

From there you can follow the instructions.  The import statement is recognized, and we can get finally get down to actually writing the controller code:

import org.grails.plugins.imagetools.*

...

//the controller method we submit to
def myProfileSave = {

...

uploadPhoto( file, user )

...

}

def uploadPhoto( upfile, user ) {

 if(upfile.empty) {
  flash.message = 'File cannot be empty'
  redirect( action: 'myProfile' )
 }

 def imageTool = new ImageTool()

 imageTool.load( upfile.getBytes() )

 // Crops it to a square
 imageTool.square()
 
        //make the new squared image the image to operate on 
 imageTool.swapSource()
  
 // Reduces the image size
 imageTool.thumbnail( 125 )

 def fileBase = grailsApplication.config.imageLocation
 def fullPath = "${fileBase}/user/photo"
        File fullPFile = new File( fullPath )

 if ( !fullPFile.exists() ) {
  fullPFile.mkdirs()
 }
  
 def filePath = "${fullPath}/user_${user.id}_125.jpg"

 File toMake = new File( filePath )

 if ( !toMake.exists() ) {
  toMake.createNewFile()
 }
  
 // Saves the result
 imageTool.writeResult( filePath, "JPEG" )
}

Once you get the plugin working, it's a breeze, as you can see!  It's a great, simple wrapper around the nasty and complicated JAI library.

Some Tips:

1) You'll probably want to increase your heap size if you are running with a default or small heap.  This requires a bit of memory.
2) Remember to specify a multi-part form.  A good explanation of this can be found in the grails documentation.
3) Profit!

Grails Wizardry: Mocking all your web services with Grails

Background

At work, we are using EC2 to run capacity tests on our environment, which would normally be pretty trivial, but we are a big SAAS shop with tons and tons of endpoints internally.  Without setting up EVERYTHING in the cloud, we can't really run our tests...or can we?  We would love to use Amazon VPC for EC2, but it requires a fair amount of intervention from your network team.  That will be a great alternative down the road - being able to point back into the internal network to access machines rather than deploying them in the cloud.  So, plan b - Using grails, I was able to mock every single .NET service that our application talks to in about three hours.  Here are the details:

We use SOAP services that return some XML, I simply used Fiddler to see what responses are sent back over the wire for each service, and I was ready to rock.  There are two main pieces to the puzzle:

MarkupBuilders


Generating XML is almost too easy in Groovy.  Using the instructions here, I was able to write some code that spits out some xml based on info passed in.  For instance this sample Grails controller will take a login and company and return an XML document with a user id and company id:

import groovy.xml.MarkupBuilder

class UserController {

  /**
  * /MyService/users/getuser?login=myuser&company=mycompany
  */
  def returnUser = {

        def user = params.login
        def comp = params.company

        def writer = new StringWriter()
        def xml = getXMLBuilder( writer )

        xml.'user'('xmlns:xsi': getNsXsi(), 'xmlns:xsd': getXsd(), 'xmlns': getXmlNs() ) {
          userId( "${comp }@${user}" )
          compId( "${comp}@compId" )
        }

        render( text: writer.toString(), contentType:"text/xml", encoding:"UTF-8" )
  }

  def getXMLBuilder( StringWriter writer ) {

        writer.append(  "" )

        return new MarkupBuilder( writer )
  }

  def getNsXsi() {
        return "http://www.w3.org/2001/XMLSchema-instance"
  }

  def getXsd() {
        return "http://www.w3.org/2001/XMLSchema"
  }

  def getXmlNs() {
        return "http://yourcompany.com/UserStuff/2009/11/01"
  }
}

Now that you have defined your controller, you need to make it so your application can make its normal request to the url it has configured but still get to your controller.  That's where the URLMappings.groovy class comes in:

UrlMappings

The UrlMappings file already exists by default when creating a project.  This file is built to map external urls to your controllers - you can use it to create nice shiny RESTful URLs, or just to do some utilitarian mappings for backwards compatibility, etc.  Obviously not every project's requirements map to the grails conventions on how urls map to controllers.  To add access to your controller from a different url, you just need to add the following snippet - this will allow Grails to map your request from something like

/MyService/users/getuser?login=me&company=acme

to

/user?login=me&company=acme 

    "/MyService/users/getuser" {
          controller = "user"
          action = "returnUser"
    }

More information can be found at this link.

At this point, you have a service running that will return the valid XML for your service, and will respond to a call at a URL that doesn't actually match.  Now you can rinse and repeat for each service that you need to mock out.  Happy Grailing!

Grails Wizardry: FCKEditor

So I am working on this side project, and obviously I chose Groovy and Grails to do the work.  I continuously marvel at how awesome, easy-to-use, and powerful the framework is, but lately I've been even more amazed at the plugins and how many there are now, how well documented (most) are, and how the quality has increased greatly.

We are using rich editors, and the FCKeditor was a no-brainer.  I was delighted to find out that it was available as a grails plugin.  I pulled it into my project, and defining an editor in one of my views was as hard as this:

<fckeditor:editor
id="materials"
name="materials"
width="70%"
height="200"
toolbar="Standard"
fileBrowser="default"
value="${fieldValue(bean:mybean, field:'materials').decodeHTML()}">               ${fieldValue(bean:contentInstance, field:'materials').decodeHTML()}
</fckeditor:editor>

The important parts here are:

1. toolbar:  This gives you the ability to choose the set of tools for your editor.  More on this one later
2. value: You can identify the preset value.  If this is html that was saved, then you must include the decodeHTML() call or else the value you will see is raw html.

The rest is pretty much boilerplate - obviously the height/width will depend on your site.

To customize the contents of your toolbar, you will need to specify a custom configuration file, which is in the form of javascript.  Your gsp page must contain:

<fckeditor:config CustomConfigurationsPath="${resource(dir:'js',file:'myconfig.js')}"/>

This file looks like this:

FCKConfig.ToolbarSets["ed-limited"] = [
   ['Cut','Copy','Paste'],
   ['Undo','Redo','-','Bold','Italic','Underline','StrikeThrough'],
   '/',
   ['OrderedList','UnorderedList','-','Outdent','Indent'],
   ['Link'],
   //'/',
   ['Style'],
   ['Table','Image','SpecialChar'],
   ['About']
   ];

You can define as many of these as you want.  See this link for configuration details.

Once you defined this, you can simply update the toolbar attribute on the FCKeditor to point to your new toolbar, in this case 'ed-limited'.

One big gotcha is that this uses 2.6 currently, while the newest stable version seems to be 3.1.

Consuming RSS Feeds with Groovy/Grails

Building off of my recent post about RSS Feed parsing using the ROME Library, I had an idea for a fun application to build using Grails. The first part of this application, which I'll share when I am done, is the part where we read the feed. Now, the last time I did this in Java, it was pretty easy to do, but man, this is just a little bit better:

    def readFeed( url )
  {
      def xmlFeed = new XmlParser().parse(url);

      def feedList = []
    
      (0..< xmlFeed.channel.item.size()).each {

         def item = xmlFeed.channel.item.get(it);
         RSSFeed feed = new RSSFeed( item.title.text(), item.link.text(),
             item.description.text(), item.pubDate.text() )
         feedList << feed
      }

      feedList
  }

Yep, that's it. One line to pull back the feeds. The iterator, and one line to create my RSSFeed object. Then add the feed to a list, and return the list to your controller. In 25 minutes, I have a feed reader application that's basically functional, taking a feed as an input, and returning a page that displays the post title linked to the post, and the posts contents and date. All of life should be this easy.

Why Grails is Sweet

Grails is sweet because you do things that are relatively complex, like upload a photo, resize it, save it, and persist the metadata using this code in a controller class, and nothing more:

        def photo = new Photo(params)
Member m = authenticatedMember()
def myFile = request.getFile( "file" )
def imageTool = new ImageTool()
photo.path = "";
photo.member = m

if(myFile && photo.save())
{
String imagepath = grailsApplication.config.imagePath +
File.separatorChar + "${photo.id}.jpg"
myFile.transferTo(new File(imagepath))

imageTool.load(imagepath)
imageTool.thumbnail(640)

String fixedImagePath = grailsApplication.config.imagePath + File.separatorChar + "${photo.id}-fixed.jpg"
imageTool.writeResult(fixedImagePath, "JPEG")
imageTool.square()
imageTool.swapSource()
photo.path = fixedImagePath
}

Oh by the way, when you define a domain class with a byte array to store a file, then generate the edit/create view, it automatically does all the multipart form submission stuff in the .gsp file. It NEVER gets this easy with Java/JSP. Hmm. Awesome. This is made possible by the innate goodness of grails and a plugin called ImageTools. It makes me dread going back to work and using Java sometimes. Sigh. It's amazing how much you can get done in a short time with this framework, and the more I have a-ha moments with it, the more I think I won't be going back to just Java when I make the switch full time, especially in the knowledge that whether it's Groovy/Grails or JRuby/Rails, I can still fall back on familiar Java libraries. Yessir, this is a good time to be a nerd...

Groovy is the Only Way To Code

Well - any sort of awesome scripting language will do. I had a particularly nasty query that I was going to need to write today to see if there were people who had a value in a column in one table and no backing data in another table. This is an odd bit of legacy data model stuff that doesn't utilize any sort of foreign key constraints, and apparently there was a blip at some point more than 6 mos back that caused a bit of data corruption. Instead of figuring out this bizarre query, I thought, "Gee Kirk, you are always looking for a reason to use Groovy - here it is!"

Here's what I came up with:

import groovy.sql.Sql

sql = Sql.newInstance("jdbc:mysql://localhost:3306/mydb", "mydbuser",
                     "mydbpass", "com.mysql.jdbc.Driver" )

def badIdList = []
sql.eachRow( "select id, weird_col, active from table1 where active = 1 and weird_col is not null" ) {

    myId = it.id

    checkCount =
      sql.firstRow( "select count(*) as count from table2 where table1_id = ${myId} and is_right = 1").count

    if ( checkCount == 0 )
    {
      println( "bad id: ${myId}")
      badIdList.add( myId )
    }

}

println( badIdList )

Now I guess I am just a really big nerd, but I felt oh so satisfied after writing that. For sh*!ts and grins, I wrote the same thing in java, and it was a lot of code, and it was annoying, and you had to do the while rs.hasNext thing, and get the current resultset, and then do the rs.getString() thing, and do the Statement = this that. It's a lot of code to do very little, and it kind of sucks. Now granted most people don't really do a ton of direct to db stuff anymore, with all the ORM solutions like Hibernate for Java and ActiveRecord for Ruby/Rails, but when you need something surgical like this, it just can't be beat. Man oh man it's awesome.

My name is Kirk, and I am a freakin nerd.