Grails Wizardry: FCKEditor

So I am working on this side project, and obviously I chose Groovy and Grails to do the work.  I continuously marvel at how awesome, easy-to-use, and powerful the framework is, but lately I've been even more amazed at the plugins and how many there are now, how well documented (most) are, and how the quality has increased greatly.

We are using rich editors, and the FCKeditor was a no-brainer.  I was delighted to find out that it was available as a grails plugin.  I pulled it into my project, and defining an editor in one of my views was as hard as this:

<fckeditor:editor
id="materials"
name="materials"
width="70%"
height="200"
toolbar="Standard"
fileBrowser="default"
value="${fieldValue(bean:mybean, field:'materials').decodeHTML()}">               ${fieldValue(bean:contentInstance, field:'materials').decodeHTML()}
</fckeditor:editor>

The important parts here are:

1. toolbar:  This gives you the ability to choose the set of tools for your editor.  More on this one later
2. value: You can identify the preset value.  If this is html that was saved, then you must include the decodeHTML() call or else the value you will see is raw html.

The rest is pretty much boilerplate - obviously the height/width will depend on your site.

To customize the contents of your toolbar, you will need to specify a custom configuration file, which is in the form of javascript.  Your gsp page must contain:

<fckeditor:config CustomConfigurationsPath="${resource(dir:'js',file:'myconfig.js')}"/>

This file looks like this:

FCKConfig.ToolbarSets["ed-limited"] = [
   ['Cut','Copy','Paste'],
   ['Undo','Redo','-','Bold','Italic','Underline','StrikeThrough'],
   '/',
   ['OrderedList','UnorderedList','-','Outdent','Indent'],
   ['Link'],
   //'/',
   ['Style'],
   ['Table','Image','SpecialChar'],
   ['About']
   ];

You can define as many of these as you want.  See this link for configuration details.

Once you defined this, you can simply update the toolbar attribute on the FCKeditor to point to your new toolbar, in this case 'ed-limited'.

One big gotcha is that this uses 2.6 currently, while the newest stable version seems to be 3.1.

Facebook Connect Users Beware!!!

We just ran across a bug that must have been introduced very recently, where all users with new Facebook accounts were unable to access our application when 'connect'-ing their user name to Facebook. It was a weird issue, because I was able to connect my account with no issue. I finally noticed that the Facebook profile ID we were storing was the same across all users who were running up against this issue.

Truncation issue!

We had defined it as a 12-length INT, which really doesn't matter - the biggest value an INT can store in MySQL is 2147483647. Anyhow, updating the table to store the value as a BIGINT fixed the problem, but I can't image we're the only ones to run across this mess, and the FB Connect documentation is weak enough that I have never seen anything about suggested storage. This must have happened in just the last few days, so hopefully others will come across this and make the change proactively if needed.

Why Grails is Sweet

Grails is sweet because you do things that are relatively complex, like upload a photo, resize it, save it, and persist the metadata using this code in a controller class, and nothing more:

        def photo = new Photo(params)
Member m = authenticatedMember()
def myFile = request.getFile( "file" )
def imageTool = new ImageTool()
photo.path = "";
photo.member = m

if(myFile && photo.save())
{
String imagepath = grailsApplication.config.imagePath +
File.separatorChar + "${photo.id}.jpg"
myFile.transferTo(new File(imagepath))

imageTool.load(imagepath)
imageTool.thumbnail(640)

String fixedImagePath = grailsApplication.config.imagePath + File.separatorChar + "${photo.id}-fixed.jpg"
imageTool.writeResult(fixedImagePath, "JPEG")
imageTool.square()
imageTool.swapSource()
photo.path = fixedImagePath
}

Oh by the way, when you define a domain class with a byte array to store a file, then generate the edit/create view, it automatically does all the multipart form submission stuff in the .gsp file. It NEVER gets this easy with Java/JSP. Hmm. Awesome. This is made possible by the innate goodness of grails and a plugin called ImageTools. It makes me dread going back to work and using Java sometimes. Sigh. It's amazing how much you can get done in a short time with this framework, and the more I have a-ha moments with it, the more I think I won't be going back to just Java when I make the switch full time, especially in the knowledge that whether it's Groovy/Grails or JRuby/Rails, I can still fall back on familiar Java libraries. Yessir, this is a good time to be a nerd...

jQuery Goodness

This week I did a lot of client-side development.  Usually, this would be reason to cry and/or drink heavily, but this week, it was one of the more pleasant parts of my week.  I thought that I'd go ahead and share a one of the goodies that I experienced.

Thickbox

Thickbox is an awesome library that pops up a page that you defined as a dialog that can be modal if you specify that option.  It's pretty peppy, and looks sharp.  Thickbox uses decoration of links on a page (set up by specifying a class of 'thickbox').  Anchor tags are set up to register an onclick event that pops up a page that you specified, be it a static page, or the result of a server-side call.  Anything can be displayed in a thickbox.  It can be information, or a form that you submit back to the server.  It's been pretty simple for us, but I did run across an issue today.  I am loading some pre-rendered html into another div that displays as a floating popup on the page.  One of the links there needs to call an overlay that is loaded via thickbox.  The button was defined as an input, which should work, but all the documentation refers to anchor tags, so I figured that we could get around it by doing this:

<script type="text/javascript">

function showOverlay {

jQuery( '#floatingDiv' ).hide();
jQuery( '#hiddenTbLink' ).click();

</script>
<input type="button" id="id" onclick="javascript:showOverlay();" value="showIt">
<a id="hiddenTbLink" style="display: none;" class="thickbox" href="http://www.blogger.com/path/to/action"></a>

This hides the floating div and invokes the click action on the anchor tag that contains the thickbox element.  This should work, right?  Wrong.  The html is dynamically inserted into the DOM after the document.ready code is called.  That is where you perform the thickbox initialize call.  So, what we had to do is add call to tb_init() in the code that loads the floating div, thus doing the jQuery DOM manipulation and allowing the thickbox-enabled link.  Much better now.  I sat back with glee, realizing that in a few hours, I had enabled this floating div and figured out how, from that floating div, I could make a call to load a beautiful thickbox element.  It was all too easy.

The Part Where The Java Developer Uses PHP

Any illusions I may have had about my PHP prowess were all but shattered over the past couple of days. What initially seemed like a fantastically successful migration disintegrated into a day of vigorous LAMP-stration, as I struggled through what seemed like a virtually impossible issue. Most everything was working - all read operations were just fine. Many inserts and updates were working as well. After about half an hour of poking around, I realized that the problem was that we weren't able to submit any form that included text with a single quote in it. Shouldn't be a problem right? I know the issue! Let's fix it. Not so fast my friend...

Here is the tale of my woe:

1) My first thought was "Google, single quotes php mysql". That's exactly what I did, and immediately found a reference to 'magic_quotes'. After reading the php help, I was pretty sure this was it. A thing that is built in to php to save the unsafe programmers from the evils of sql injections, by escaping special characters that are submitted as part of a form. That certainly includes single quotes. A-ha! This'll be quick. I verified that on the old server, and locally, this option was being used. I did this by running php -i, to get the php info. Here's what I saw on both of those servers:

magic_quotes_gpc => On => On

So I assumed that when I ran php -i on the new server, I would see

magic_quotes_gpc => Off => Off

Imagine my surprise when I ran php -i | grep magic, and saw

magic_quotes_gpc => On => On

How could that be? This was a definite crossroads in my debugging process, and I chose the wrong road. I could have verified that this was actually the case, but instead what I did was this:

2) Figure out how to escape strings in php prior to inserting into the database. Of course this application isn't using prepared statements - that would be too simple. Despite the availability of this option, no dice. That would solve everything, because the strings would be escaped as part of the persistence logic. Wrong. I played around with mysql_real_escape_string, which would work, but since this codebase has no persistence layer, I would to make a change to every single php file that posts to the database, and that just isn't practical for a product that we hope not to touch too much. So...the next option is to override the query() function in mysqli, to do some logic to escape the characters that need escaping. I started trying to do this, and wasn't getting anywhere, because it was pretty difficult to debug through this issue on the production to make sure that the new query logic was even being called. This was made especially difficult because of the sheer amount of cruft in the php_error_log.

Aside: Broken Windows

The Broken Windows theory is something that I heard a lot about during my time at Blackboard. The idea is that if you clean up the trash on a sidewalk, people are much less likely to litter. If you paint over all the graffiti, people won't tag on that wall. It was discussion in Freakonomics, and it a central theme in New York City's recovery from their crime epidemic of the 70s and 80s. At Blackboard, we had gotten pretty lax about what ended up in the log files. This sucks for a number of reasons. Under heavy use, this fills up the log files pretty quickly, using up disk space and cpu cycles to write the files. Under any usage level, it certainly makes it harder to see what the problems are in your software. The log is a place for things that you want to be there, like errors, warnings, and informational statements that you want there. It's not a good place for "Made it here at line 262", and "id coming in is: 12". That doesn't help anyone, and hasn't helped anyone since the person who put it in the log in the first place. My old manager was very astute in pointing this out, and making sure we did somethin about it. We made it a point at Blackboard to edit any code that did unnecessary print statements, and it really made a difference. Furthermore, nobody wanted to be the one who left gratuitous logging in the code.

Back to the story.

Once I decided that it couldn't be a php code thing, and definitely wasn't a mysql thing, I moved on to sheer despair and anger. This is probably not the best way to go about things, but it's where I was at. A walk to the coffee store with my boss led me to re-examine the php.ini file. See, I had compiled and tried to install php to include some packages that weren't installed initially, and it was a bad bad idea. Now, on this machine there are two versions of php. One is running on apache for the site - it's php 5.1.6. The php I installed was 5.2.6. Now I check and see what the options are for php again. Same thing. No way, I think. I finally did what I should have done a while ago and read the php.ini file. Guess what:

magic_quotes_gpc => Off => Off

How about that. I fixed that, and the problem is now solved. Voila.

The Moral of the Story

When I was doing interviews at Blackboard, I usually asked the candidates a few standard java questions to make sure they had a basic level of competency. Then I dove into problem solving techniques. So many people simply failed to even start any sort of critical thinking. I think that next time I am interviewing, I am going to use a version of this issue as my problem solving question.

Lessons learned - don't install php when it's already there as part of the OS. If you do, make sure that you are using the right version of php. Once you've done that, verify that what is in php -i is actually correct, by looking at php.ini. Biggest lesson learned - if your gut tells you something and you just know it's right, stick with it and follow up on it. It's usually the right path.

Groovy is the Only Way To Code

Well - any sort of awesome scripting language will do. I had a particularly nasty query that I was going to need to write today to see if there were people who had a value in a column in one table and no backing data in another table. This is an odd bit of legacy data model stuff that doesn't utilize any sort of foreign key constraints, and apparently there was a blip at some point more than 6 mos back that caused a bit of data corruption. Instead of figuring out this bizarre query, I thought, "Gee Kirk, you are always looking for a reason to use Groovy - here it is!"

Here's what I came up with:

import groovy.sql.Sql

sql = Sql.newInstance("jdbc:mysql://localhost:3306/mydb", "mydbuser",
                     "mydbpass", "com.mysql.jdbc.Driver" )

def badIdList = []
sql.eachRow( "select id, weird_col, active from table1 where active = 1 and weird_col is not null" ) {

    myId = it.id

    checkCount =
      sql.firstRow( "select count(*) as count from table2 where table1_id = ${myId} and is_right = 1").count

    if ( checkCount == 0 )
    {
      println( "bad id: ${myId}")
      badIdList.add( myId )
    }

}

println( badIdList )

Now I guess I am just a really big nerd, but I felt oh so satisfied after writing that. For sh*!ts and grins, I wrote the same thing in java, and it was a lot of code, and it was annoying, and you had to do the while rs.hasNext thing, and get the current resultset, and then do the rs.getString() thing, and do the Statement = this that. It's a lot of code to do very little, and it kind of sucks. Now granted most people don't really do a ton of direct to db stuff anymore, with all the ORM solutions like Hibernate for Java and ActiveRecord for Ruby/Rails, but when you need something surgical like this, it just can't be beat. Man oh man it's awesome.

My name is Kirk, and I am a freakin nerd.

Holy Ruby Deliciousness!

So I am mucking about with Ruby, Rails, Groovy, Grails, and whatnot. I find them all pretty incredible and the more I use scripting languages, the more I decide that unless you really NEED a precompiled piece of code, this is the only way to fly. I just took results from a database and did some string replaces and wrote a new file in Ruby. It went a little like this:

file = File.new("/awesomedirectory/mylist.TXT", "r")
out_file = File.new("/awesomedirectory/mylist-formatted.TXT","w")
while (line = file.gets)
newline = line.gsub(/ /, '')
out_file.puts( newline )
end
file.close
out_file.close

Now that is silly easy. Anyone who has used Java IO knows it's cumbersome and kinda annoying - not hard - but definitely kinda annoying. This is sweet. I am going to do more with this. When you spend 7-10 hrs a day programming, even a 10% increase in productivity that can be gained from the faster feedback loop and the terseness of the scripting syntax can really add up, and let's face it, much of what we do doesn't REALLY need to be precompiled code - it's okay if it compiles on the fly!

Software is Weird

So we recently upgraded from Hibernate 2.1 to Hibernate 3.2. It was a good exercise in understanding how the code in this incredible ORM library really works. In fact, I had to actually make a chance to the Hibernate source code to support an odd case of a left outer join with multiple join criteria, so I had to step through an entire query's lifecycle in Eclipse (what did people do before debuggers?!!), and figure it out. I saw that there is a lot of automagical stuff happening throughout the code. Most of this is okay. We as users of these libraries, open source and proprietary, just take for granted the behavior, because it's right, about 99.5% of the time. Good work Hibernate developers. You are true champions of software. Here's a weird one though. Pretend for a moment that you ran a program like this:

Session s = openSession();

User u = UserManager.loadUser( 1 );

u.setName ( "fakeName" );

s.flush();
s.close();

This is the equivalent of opening a hibernate session, loading an object, updating it but not saving it, and then closing it. I think the intuitive interpretation of this is that when you flush and close, nothing CRUD-related would happen, since I never called save or update on that User object. To the contrary, that object did get updated - flush() actually took that change and automagically formed an update statement like

update user set name = 'fakeName' where id = 1

for us, and executed and committed the query. That's weird to me, but it is what it is.

As some background, we use the "Session in View" pattern, which has been great. Most of the recommended settings on the interwebs recommend FlushMode.AUTO to be used in the ServletFilter that sets up the session. We figured that the wisdom of crowds was good here, so we rolled with that. We deployed to production and started seeing bizarre data corruption, that was traced the problem to an instance like the one above.

So when you want to avoid this sort of unexpected data modification, you have a couple choices:

1) Use FlushMode.AUTO, and

a) Write the app from scratch and make sure you never update a hibernate-managed object with data you don't want to be persisted to the database.

b) Scour the app to make sure you don't do anything silly like change an object and not persist it yourself

2) Use FlushMode.COMMIT, and wrap all your CRUD calls with

Transaction t = session.beginTransaction();

DAO.saveStuff( object );

t.commit();

We went with option two, and since we have a pretty clean DAO layer, there was minimal work involved, and so far it has worked like a charm. This may be pretty self-evident to many of you, but it was weird to me, so I thought I'd write about it.

What It Is!

Been a while since I did an update on what's what.

Precana was actually a good time. It's a long weekend - I think total we put in almost 30 hours, but it was well set up. Not a lot of endless lectures. Good discussion. It was definitely one of those "Get out what you put in" type things, but I am glad we did it. After that we went to watch the fourth installment in the Indiana Jones series. I have been looking forward to seeing this since I heard about it.

It sucked.

The dialog was bad, the plot weak, the premise sillier than most action flick premises. Harrison Ford was actually still believable, and the action scenes, especially the part with the ants, were pretty sweet. Disappointing though. Ah well.

Reading: I am reading a few books as usual. The two most active are 'The World Without Us', by Alan Weisman and 'The First Crusade: A New History', by Thomas Asbridge. The World Without Us is a pretty popular one, and it kept seeming interesting, so I finally checked it out. It's pretty crazy to think about all the infrastructure that would just crumble if we humankind ceased to exist. While the book has the potential to be really depressing, every time it veers in that direction, it gets interesting enough that you don't feel like you are being preached to or warned. I like it half way through.

I just started the book about the First Crusade, as part of my reading regimen to understand Italian and papal history. I want to be somewhat educated on what I am seeing when I go to Italia. So far the book is written in a pretty accessible way, though as with much early history, I have a problem with the inherent required guessing that takes place due to lacking solid written evidence.

Have I mentioned that I can't freakin' wait to go Italy!?!?

As far as work, I have all manner of code out on the interwebs. The php stuff seems to have solidified enough so that I won't be doing any emergency deployments any time in the next week or so, and I will take that. We are in the midst of ironing out some issues with the latest deployment of sportsvite, and I got to dust off my performance engineering skills. It's good to know that I haven't already forgotten everything. I have to admit that working on those problems is fun, and I am lucky to have worked somewhere that gave me the ability to learn some of those relatively rare skills.

Good And Bad

Well, in the midst of doing an emergency php deployment, Jena and I found out that her parents got us a timeshare in Sardinia for our honeymoon. Makes the dire php ecommerce issues seem slightly less painful. Can't wait to get to Italy. Can't wait!

I leave you with these nuggets of wisdom:

Php deployments on a live ecommerce site from a Barnes and Noble Cafe in Bethesda - no matter what anyone tells you, it's not awesome.

Things I find to be awesome:

• Juno Soundtrack (good movie too)
• Old Alice Cooper music
• Sardinia timeshares
• Tigers sweeping the evil White Sox (about time Cabrera!)
  

Things that aren't great:

• $4 for 2 hrs of wireless access
• Cafes with one available wall outlet where the sun is shining thus making it 200 degrees while doing crazy php debugging
• php debugging
• The nagging feeling that after sweeping the evil White Sox, the Tigers will find a way to lose most of their next 20 games

This weekend is pre-cana for Jena and I. Hopefully we will pass. Peaceout!

Ya down with PHP? Yeah you know me!

So my company also operates another related site that required a little love. It's written in php. We had to make some changes to make a client happy. I volunteered, as I thought it might be nice to learn php. After some initial trepidation, I dove in and got the application running. Getting the LAMP stack running on Mac is kind of annoying - minor weirdness with different packages and odd compatibility issues - that kind of crap doesn't exist on linux - in Ubuntu the work is accomplished by doing this:

sudo apt-get install apache2 php5-mysql libapache-2-mod-php5 mysql-server

And BAM! It works. How easy is that! Man I love Ubuntu. I should get a t-shirt that says that or something.

ANYWAY, back to the point of the post - I had a hard time getting the LAMP stack running, but I finally got it going. I got the application I needed to work with up and running in no time once the servers were setup, but we hit a snag. Some of the work I am doing requires SSL. SSL is a pain in the ass. I could not, for the life of me get it setup. Missing .so files in mac, weirdness once I found them, compilation errors, how many times must I run make and make install? This sucked, until I found this awesome MAMP program, that bundles php5, apache2, and mysql into one Mac application. This had some promise. It installed just like a regular app, and provided a GUI interface for the most common apache setup stuff, like document root. Did I mention IT'S FREE. Who are these awesome nerds that do these incredible things for us? Once I discovered this awesome tool, I was able to follow simple directions on how to setup SSL, create a bogus self-signed certificate, and holy crap, it just worked!

Since then, I have been diving right into the craziness that is php. Little intricacies like php pages that have html forms that submit back to the same php! Odd. Symbols like -> and => where there should be dots. Some things are similar. Braces. Semicolons at line endings. All in all, I found php to be a pretty simple language. I can't say I understand it fully or anything, but I am certainly able to poke around this application and make things work. I LOVE arrays that are really just like maps, or can be just like arrays. I love the convenience methods like print_r to print out all the contents of an array. Now as someone who has done almost all my programming in java, and especially in java web applications, here's the kicker. I would make a change, then refresh the page, and the change was already there. Yep. Pretty awesome. No build.xml that then deploys to the server that you then restart, then re-login to your application to see what the change did. No sir. So my lack of php skill didn't put too much of a dent in productivity. That's awesome.

I am now truly excited to rock out with Groovy or Scala or one of these other JVM-based scripting languages, so that I can get the best of both worlds. Things are going to get interesting! Man I am a horrendous nerd.

What's New, 2008!

Well, since I last wrote, it became 2008! I went to Denver, where I did some sightseeing, and a whole lot of snow shoveling. It was great to finally get out to Colorado. My mom, myself, my Aunt Mary, Uncle Al, cousin Stacey, and her husband Tony spent a lovely White (10 in) Christmas together. Sadly, it was too cold and windy to get any skiing in, but we still had a good time and it was good for my Mom to be around family. Before we left, Jena and I had Christmas with my mom in Arlington. We got a lot of great gifts (not least of which was a bunch of stuff to get us ready for our Italian honeymoon), and had a great dinner. Once I returned, it was finally time to have our Christmas. We exchanged gifts, figured out we both went WAY over our predetermined spending limits for each other, and had a great night.

I have always loved Christmas. It's a great time to think about how you can really make the people around you smile. My dad always loved Christmas, and I think it really rubbed off on me. Watching him every year as he started the shopping ritual, and kept on buying little stuff here and there, right up until the stores closed on Dec. 24th. Then coming down in the morning, seeing all those presents, some of them from Santa, some from the Elves, and some from Rudolph himself (my dad took great license signing the to/from cards). The stockings filled to the brim with gimmick gifts, bizarre stuff, odd foods, and the like. The smell of breakfast and coffee. I hope to be the kind of dad and husband who makes Christmas great for his family like it was for me every year.

We got an awesome Christmas tree this year. It rivaled the awesomeness of the tree from last year. We are lucky because the guy who sells trees at Eastern Market has reasonably priced, fresh trees, and he's a really nice, friendly and helpful fellow. I will continue to buy trees from him as long as we're here and he keeps coming to the Market. The tree was still completely alive when I finally took it out last Saturday.

What else is going on. Geez, I have read a lot of books...I finished Manhunt - great book, I really enjoyed it, and it's no often I read a history book where I don't know the ending - definitely an added bonus. I also knocked out a couple of Pelecanos books, and DC Noir, a collection of Noir short stories that take place throughout DC, edited by Mr. Pelecanos himself. I also finished Michael Crichton's Andromeda Strain on the airplane on the way back. I am now reading a book by an old fave of mine, Dick Francis, who writes mysteries that center around racehorses, and a book by David Baldacci, The Winner, which I must say hasn't been the page turner that the others I have read were. Can't win em all. I am also most of the way through the programmer management book Herding Cats, which is reasonably educational.

Jena and I settled on a band for the wedding, which is exciting. Radiant is their name, and they survived the grueling selection process that included four booking agencies, a few trips to some odd places to watch bands play weddings and holiday parties, and quite a few hilarious sample DVDs. I am excited, as I think they will keep things fun, and they seem to be good singers and musicians. In a perfect world, I would have just packed my Nano full of old favorites from my music collection, but I have to admit that having a band just adds some cool factor to a wedding.
Geez, what else - the primary season is in full swing, and Hillary has already been beaten out of the race and won New Hampshire in the same 24 hours. The press is really something to behold during the election year. I for one can't wait til the contenders emerge and we can get the election done. I am not sure who I am for, but I do know who I am against, and that's a good start (please no Hucklebee theocracy for this guy). My resolutions this year include learning enough Italian not to be a complete buffoon when we travel there, and to keep learning guitar, and to get married without screwing it up!

In my nerd life, I have been doing a bunch of scripting language exploration, diving into Scala, Groovy, and Ruby. I must say that Scala doesn't seem to quite be 'there', but it is really nice. Ruby and RoR are fantastic, but I have to admit a predisposition to Groovy, as it runs in a JVM and can be mixed and matched with Java code.

So that pretty well sums up what's been happening from this guy for the past month. I turn (gulp) 30 in one week, so next weekend promises a bunch of fun and festivities (and annoying jokes). Enough rambling for now...