Tomcat & IntelliJ – Deploy war files outside webapps folder

At present I am working on developing an Android application that needs to be supported by a slew of REST services hosted in the cloud. I chose Google App Engine based on its support for Java, Groovy and most importantly Spring. I developed a Spring MVC based REST application and used ContentNegotiatingViewResolver to negotiate content based on request URL extensions. For example, an XML response will be returned if the request URL ends with .xml, a JSON response for .json and an HTML response if he URL doesn’t have any extension. Don’t get me started on Accept Header versus URL extension based content negotiation. That is a rant for another day.

I was attempting to Serialize a Map<Enum, List<Model>>. All was well and I was able to retrieve both HTML and JSON representations, but when I tested retrieving the XML representation, JAXB complained that it cannot handle a Map instance in the root although Jackson was totally cool about it. As usual, Googling revealed that JAXB expected a Container class at its root which I didn’t want to create. I didn’t want to give up either. So, I tried my luck using XStreamMarshaller. This time GAE complained that XStream used a restricted API. WTH?

Just out of curiosity, I wanted to check if XStreamMarshaller would work as expected when used outside of GAE. So, I created a Tomcat context file “myapp.xml” with the following definition and carefully placed it inside TOMCAT_HOME/conf/Catalina/localhost. I could have just started Tomcat from TOMCAT_HOME/bin/startup.bat to check if it works, but being an IDEA addict, I created a Run Configuration for the IDEA Tomcat plugin and started the server from inside IDEA. But the app refused to even be discovered, let alone be deployed. After a few frustrated attempts, I tried starting Tomcat directly outside IDEA. Thankfully the app got deployed successfully and to my surprise, the XStreamMarshaller skillfully streamed out the serialized XML. Problem Solved!

<?xml version='1.0' encoding='utf-8'?>
<Context docBase="PATH_TO_MY_APP"
         reloadable="true"
         path="/myapp">
</Context>

But, why didn’t the app get deployed when I started Tomcat from inside IDEA? After all, I have linked IDEA to my local Tomcat installation and the script it executes is clearly in my TOMCAT_HOME/bin folder. Then, why why why in the world does the app refuse to be discovered? The solution came in the form of CATALINA_BASE. It seems that IDEA copies the contents of TOMCAT_HOME/conf folder into its HOME folder with some name like Unnamed_MyApp and sets this folder to be the CATALINA_BASE. That explains why “myapp.xml” is so totally ignored by Tomcat. Then, I navigated to “Tomcat Run Configuration -> Startup/Connection -> Environment Variables” and added CATALINA_BASE as an environment variable and pointed it to your local TOMCAT_HOME folder. After this configuration change IDEA started Tomcat as expected and my app was both discovered and deployed. Another Problem Solved!

But the real problem – JAXB complaining about Map and GAE rejecting XStreamMarshaller as restricted – is yet to be solved. Maybe I should try one of the CastorMarshaller, XmlBeansMarshaller or JibxMarshaller.

Any ideas?

Blackberry: Code signing for Tablet and Smartphone

As all experienced Blackberry developers will know, to use the restricted BB APIs, you will have to code sign your application. The registration process used to be 20 bucks. Now it is free though. Actually what is the point of code signing, that too when it is free? Maybe there is a real valid reason… Anyways, that is another story, for another day…

Coming back to topic; I went to the now familiar BB developer site to get the code signing keys one more time. This time I was greeted with a new combo box that asked me if i wanted the keys for the Tablet OS or the Smartphone OS or both. I gladly selected both although I didn’t know if the keys were going to be the same or different. I don’t do Tab development as of now. So, I am not familiar with the Dev environment for that.

After a few hours (although they generally say 2 or 3 days), I received 3 emails with the code signing keys. All the 3 emails talked about Tablet OS and not about the Smartphone at all. I didn’t know if I should install the received keys assuming that they are the same for both the OS or wait and see if I get the Smartphone keys separately. I crossed my fingers and decided to wait for a few more hours. Fortunately, I received another set of 3 emails with keys for the Smartphone as well. I installed it and all is well again.

It would have been nice if the BB developer site clearly mentioned that I will be receiving two sets of keys. It may look obvious now and I may even sound silly, but it would have definitely saved me a few hours of anxiety at that time.

Install Blackberry SDK 6.0 to your existing Eclipse Helios

Blackberry provides a “Java plugin for Eclipse” that can be downloaded and used for eclipse-based blackberry development. But beware, what you download is not an eclipse plugin. Instead, it is a full eclipse download with the plugins pre-installed. I didn’t want a new copy of eclipse just for Blackberry development. I wanted to use my already well configured Helios installation to be used for Blackberry development as well. So, i chose to use the provided update site – http://www.blackberry.com/go/eclipseUpdate/3.6/java – to install the plugin to my existing Helios installation.

The issue is that, I was unable to install the plugin using the aforementioned update site and noticed that others faced the same problem too. After a lot of failed attempts, I finally discovered a different way to solve the problem. I downloaded the full “Java Plugin for Eclipse” from the Blackberry download site and just copied all the net.rim.* files from the plugins and features directory to my Helios installation.

Worked for me and I am a happy camper!

Heroku – Trouble with Windows and SSH Keys

Heroku, one of my favorite cloud platforms, is a hosted platform built specifically for deploying Rails and other Ruby based web applications. Heroku makes deploying Rails applications to cloud ridiculously easy – as long as your source code is under version control with Git.

The general process of installing and using Heroku on Windows is fairly simple. You should already have Ruby, Rails and Git for Windows (msysgit) installed. Once you have them, go to Heroku.com and sign up for an account. After signing up, install the Heroku gem with the following command.

$ gem install heroku

Typically, as in GitHub you will need to create SSH keys if you haven’t already, and then tell Heroku about your public key so that you can use Git to push your application repository up to their servers. Detailed information for setting up Git and creating ssh keys on Windows can be found here.

# Create new keys
$ ssh-keygen -t rsa -C "your_email@yourdomain.com"

# Tell heroku about your public key
$ heroku keys:add

Remember? Your source code should be under version control with git. So, from within the root of your application folder, use typical git gimmicks to create a local git repository and commit your application source to the local repository.

$ cd my_app
$ git init
$ git add .
$ git commit -m "initial version"

Finally, from within the root of your application folder, use the heroku command to create a place on the Heroku servers for the application to live. Then, you can deploy your application to the cloud by pushing your local git repo to heroku…

# Create an application space on the server
$ heroku create

# Deploy your application to the cloud
$ git push heroku master

If until now everything went well except for the last step and instead of getting your application deployed if you got the following error, then it was worth your time to have read this post until this point…

Permission denied (publickey).
fatal: The remote end hung up unexpectedly

It was a frustrating experience trying to figure out the problem. The keys were created fine and they got added to heroku just fine as well. In fact i could verify it with the following command and my keys were right there.

heroku keys:list

Whatever I do, I kept getting the same error. Finally after a lot of googling, experimenting and hair plucking the problem got resolved. Here goes the solution…

Solution:
Typically once you create the keys as mentioned above, two files – “id_rsa” and “id_rsa.pub” – are stored in the “.ssh” folder within the user’s home folder. If you are working with linux that seems to be good enough. But for the windows version of git that doesn’t seem to cut it. It wants the keys to be stored inside the “.ssh” folder within the “msysgit” installation folder as well. If you don’t find a “.ssh” folder inside the “msysgit” installation folder, feel free to create one. Once you drop these two key files there and repeat the entire process, everything went as smooth and my application got deployed in the heroku cloud and the world is again a better place to live in.

I am sure many people will hit the same road block. At least I am sure I will come across it again and wouldn’t remember how I solved it in the first place. Happens, doesn’t it? So, I thought i will document it here for future reference.

If there is any other solution, or if this solution is incorrect, feel free to leave a comment.

GWT – Pros and Cons

I love JavaScript. With the advent of jQuery and Mootools, my love for JavaScript has only increased plenty-fold. Given a choice I would use either of the aforementioned frameworks for any web application I develop. But being in the service industry, time and again I have to succumb to the client’s pressure and work in their choice of technology – whether or not it is the right one (The one who pays the piper calls the tune. Isn’t it?). One such client exposed me to the world of GWT.

I have given GWT a shot couple of years back on the day it was released. I didn’t like it that much then, so I dismissed it and never returned back. But, over the past six months working on this project I have a slightly different impression on this framework. I still cannot say that GWT is the next big thing since sliced bread, but at least it is not as bad as I thought it was. I have just documented my observations, both good and bad during the course of this project and thought some fellow developer might find it useful while evaluating GWT.

Pros:

  1. If you are a Java veteran with experience in Swing or AWT, then choosing GWT should be a no-brainer. The learning curve is the least with this background.
  2. Even if you are not experienced in Java GUI development, the experience in working on server-side Java for years will come in handy while developing GWT apps
  3. You can create highly responsive web applications with heavy lifting on the client-side and reduced chattiness with the server-side
  4. Although there are numerous JavaScript libraries out in the wild and most of them are worth their salt, many conventional developers don’t understand its true power. Remember, a powerful language like JavaScript is a double-edged sword. If you don’t know how to use it, even you won’t be able to clean the mess you create
  5. You can migrate from a typical web application to a GWT application iteratively. It is not an all or nothing proposition. You can use a clever trick called JSNI to interact with loads of JavaScript functions you already possess. But it is always better to move them to GWT sooner rather than later
  6. The IDE support for GWT cannot be better. Java IDEs have matured over the past decade to be one of the best in the world and GWT can take direct advantage of it
  7. The integrated debugging beauty is something you can kill for. The excellent debugging support offered by the mature Java IDEs is one feature that could sway anybody’s decision in favor of GWT
  8. The built-in IDE support to refactor Java code can directly be put to good use to maintain a simple design at all times. Doing this in JavaScript is not for the faint at heart
  9. The IDE syntax highlighting, error checking, code completion shortcuts etc are overwhelming – to say the least
  10. GWT is being actively developed by Google. We know that the project is not going to die off anytime soon. Until now their commitment towards the project says a lot about its future in the industry.
  11. The community behind the project is also a big PLUS. Discussions take place daily in Stack overflow, discussion forums, wikis and personal blogs. A simple search with the right keyword could point you in the right direction
  12. GWT is a well thought-out API; not something that was put together in a hurry. This helps you as a developer to quickly comprehend the abstractions and makes it really intuitive to use
  13. You can use GWT’s built-in protocol to transfer data between the client and the server without any additional knowledge of how the data is packaged and sent. If you prefer more control, you can always use XML, JSON or another proprietary format of your choice. Even in that case, while using JSON, you don’t have to use an non-intuitive java JSON library. You can use JSNI to ‘eval’ the JSON using straight javascript. Cool huh!
  14. You have the advantage of being able to use standard Java static code analyzers like FindBugs, CheckStyle, Detangler, PMD etc to monitor code and design quality. This is very important when you are working in a big team with varying experience.
  15. You can use JUnit or Test NG for unit testing and JMock or another mock library for mocking dependencies. Following TDD is straight-forward if you already practice it. Although there are JavaScript based unit testing frameworks like jsunit and qunit, come on tell me how many people already know that or are itching to use that.
  16. The GWT compiler generates cross-browser JavaScript code. Today, any marketing person who says this will probably be beaten. It has now become a basic necessity, not a luxury
  17. The GWT compiler optimizes the generated code, removes dead code and even obfuscates the JavaScript for you all in one shot
  18. Although the compilation process takes hell a lot of time, you don’t have to go through that during development. There is a special hosted mode that uses a browser plug-in and direct java byte-code to produce output. That is one of the main reasons you are able to use a Java debugger to debug client side code.
  19. Rich third-party controls are available through quite a few projects like Smart GWT, Ext GWT etc. They are well designed, easy to use and theme-able. So, if you have a requirement where existing controls don’t just cut it, you should be looking into one of these projects. There is a really fat chance that one of those components will work out. Even if that doesn’t work out, you can always roll out your own.
  20. GWT emphasizes the concept of a stateful client and a stateless server. This results in extremely less load on the server where many users have to co-exist and high load on the client where only one user is working
  21. I18N and L10N are pretty straight-forward with GWT. In fact locale based compilation is taken care by the GWT compiler itself. The same cannot be said about regular client-only frameworks
  22. GWT comes built-in with browser back button support even while using AJAX. If you are an AJAX developer, I can almost feel your relief. This is priceless.

Cons:

  1. GWT is a fast developing project. So, there are a lot of versions floating around. Many functions, interfaces and events get deprecated and keeping up with their pace is not too much fun when you have other work to do
  2. There were quite a few GWT books during the beginning. Not so much these days. For example, I haven’t found many books on the 2.0 version of GWT. This leaves us only with Google’s documentation. I agree that the documentation is good and all, but nothing can beat a well written book
  3. GWT is not fun to work with. After all it is Java and Java is not a fun language to work with. If you add the fact that entire layouts and custom controls should be created in java, you can easily make a grown programmer cry. With the introduction of UI binder starting version 2.0, that problem is kind of solved, but now you have a new syntax to learn.
  4. The Java to JavaScript compilation is fairly slow, which is a significant con if you choose GWT.
  5. I personally prefer defining structure in HTML and styling it using CSS. The concepts used in HTML are clean and straight-forward and I have years of experience doing just that. But in GWT, I am kind of forced to use proprietary methods to do the same. That combined with the fact that GWT doesn’t solve the styling and alignment incompatibilies for me compounds the problem. So, writing layout code in GWT is something I despice. But with UI Binder and HTMLLayout from version 2.0 onwards, I feel I am back in my own territory
  6. It requires some serious commitment levels to get into GWT, coz, after that, a change in client side technology could require a complete rewrite of your app, as it is a radically different approach than other client side frameworks
  7. There is not a defined way to approach an application development using GWT. Should we use only module per app or one module per page or somewhere in between. These design patterns are slowly evolving only now. So, typically people tend to develop all in one module until the module size goes beyond being acceptable and then they refactor it into multiple modules. But if it is too late, then refactoring could not be that easy either
  8. Mixing presentation and code doesn’t sound right although typical desktop GUI applications does just that. But these days, even desktop application frameworks like Flex and Silverlight have taken an XML based declarative approach to separate presentation from logic. I think GWT 1.x version had this disadvantage. With the introduction of UI Binder starting from version 2.0, I think this disadvantage can be written off although it is yet another painful XML language to learn
  9. You would often be punching in 3X to 5X more code than you would with other client libraries – like jQuery – to get simple things done
  10. You should also remember that with GWT, the abstraction away from HTML isn’t complete. You’ll still need to understand the DOM structure your app is generating in order to style it, and GWT can make it harder to see this structure in the code.
  11. GWT is an advantage only for Java developers. Developers with .NET or PHP background won’t gain anything here
  12. If you have tasted the power of JavaScript and know how to properly utilize it to your advantage, then you will feel crippled with an unexpressive language like Java

I am sure many of you will have differences of opinion. Difference is good. So, if you think otherwise, feel free to leave a comment. We’ll discuss…

GWT – Not enough methods, expecting 3 saw 2

In my current project we use GWT extensively. I also develop prototypes and POCs for clients using GWT when I am in presales mode. During one such prototype, when I was using GWT RPC to perform an AJAX call from the client to the server, I hit a road block. GWT was throwing me an error I couldn’t comprehend. 

java.lang.AssertionError: Not enough methods, expecting 3 saw 2

After a bit of fiddling and googling around, I realized that this was happening because I recently upgraded from version 2.0.3 to version 2.2. I guess the protocol GWT uses to serialize the data to be transported between the client and the server has changed between these two versions. Fair enough. But I don’t understand why I received that error although I created this project after upgrading to version 2.2.

Anyways, I found out that for some odd reason the gwt-servlet.jar that was stuck to my lib folder was the older 2.0.3 version. When I replaced it with the 2.2 version of the jar file the problem got fixed. FYI, the 2.0.3 version of the jar is around 1.5 MB whereas the 2.2 version of the jar is around 4.5 MB.

Meanwhile, I am working on a list of pros and cons for using GWT. Will be posting that soon…

Ceylon – Yet Another JVM Language (YAJL)

It is all too familiar, yet surprising; it is all too common, yet shocking; to see yet another JVM language created to scratch an itch that countless other languages are already trying to solve.

Of course Java is not expressive enough, it doesn’t have higher order functions, it doesn’t have modularity as a language feature, it doesn’t have clean way to do meta-programming, it does NOT have so many more features we love and does have so many more features we hate. These are most of what has frustrated Gavin King (the creator of Hibernate) as well and made him think about creating a new language – Ceylon. A few days ago, in his presentation at the QCon Beijing 2011 he gave a first glimpse of the language features and a few code snippets showing its beauty.

A lot of people have raised concerns and expressed strong opinions questioning a need for yet another language. Scala fans, the Groovy(++) camp, Gosu and Fantom hackers all think that Ceylon does not solve anything that is not solved already or could have been solved by just contributing to one of these modern languages. So, I am not going to be yet another anti Ceylon person but I am not a fan either.

I believe Ceylon is more of a strategic approach from RedHat than a language created out of true necessity. They would like to control a language and its followers like most other giant companies do today. Think about it. Oracle is controlling Java, Microsoft is controlling C# (VB, VC++, etc), Google is controlling Go (and Python?), Apple is controlling Objective C and VMWare is controlling Groovy. RedHat has just joined the party leaving only IBM out of the equation. I am sure they are not far behind. I just sincerely hope that they adopt an existing language (Scala?) instead of creating yet another one.

That said, the language itself looks cool, is very expressive and adds a lot of syntactic sweetness to say the least. I just wanted to highlight a few of them here…

String Interpolation:
In Java we use a “+” sign for string concatenation. There is no concept of string interpolation in either Java or Scala. In Groovy ${} construct is used while Ruby uses #{}. Ceylon’s syntax looks better than either of them though. It uses a “space” as the string interpolation operator and the result looks a lot cleaner. Don’t you think?

String name;
writeLine("Hello " name "!");

Getter:
In Java we use getXxx() methods to get a property value. This looks like a method and quacks like a method and does not give the feeling of accessing a property at all. In Ceylon, a very simple innovation has resulted in much more succinct getter methods. They have decided to get rid of paranthesis to both define and call getter methods. Look at the last line of the following snippet. Does it look like a method call? NO. Does it look like accessing a property? Of course it does !!!

class Counter() {
   variable Natural count := 0;
   shared Natural currentCount {
      return count;
   }
}
Counter c = new Counter();
writeLine(c.currentCount);

Constructor:
In Ceylon there is no separate constructor. The class definition itself acts as a constructor as shown below. Since, there is no concept of method or operator overloading in Ceylon there is no necessity to have a separate method and this syntax induces a “Why didn’t I think of this before ?” moment… The necessity for an overloaded method is handled by optional /defaulted parameters concept. So, you don’t have to worry too much about it…

class Customer (String cName, Natural cAge, Date cDob) {
   variable String name :=  cName;
   …
}

Builder:
With the support of named parameters and higher-order functions and quite a bit of thought, a syntactic structure as expressive as given below has been achieved in a general purpose and statically typed language like Ceylon.

Html hello {
   Head head { title = "Squares"; }
   Body body {
      Div {
         cssClass = "greeting";
         "Hello" name "!";
      }
   }

And then there is the rest of the now so common features like array like access to Sequences (equivalent of List in Java), higher-order functions, Closures, Currying, etc

It is not like I loved every single feature of Ceylon. I hated a few, but I am reserving that rant for another post…

Twitter moves from Rails to Java

A colorful feather up Rails’ cap is on the ground now. Twitter has decided to go away from RoR in favor of Java, this time for their entire search stack. Earlier in 2008-09, they decided to move their message queue back-end from ruby to Scala (a Java Platform) and now it is the time for their front-end to move to Java as well.

They have built a scalable platform called Blender that uses Java NIO based server (Netty) to be efficient in the face of heavy incoming traffic, replaced MySQL with a Java based Lucene search engine, created an engine that parallelizes execution of multiple backend services with dependency management and more. With this setup there is a 3X drop in search latencies and can scale to 10X more requests per machine.

Wow, that is quite an achievement. Could this mean that Java is a better platform than Rails for high scalability needs? Even if that is the case, for simpler scenarios, the beauty of RoR out-weighs Java’s performance.

They say that this change will enable them to rapidly iterate on search features in the coming months. That along with the news that Twitter has hired 25 more employees kinda tells that Java’s code base is practically more maintainable than equivalent Ruby code – at least when the code base is huge and the team size is large. Or that could mean that this time they really put a lot of thought into designing a maintainable system than when they started out. But for smaller team size and code base, RoR is still an unbeaten champion.

iPhone SDK – Initial Thoughts

I have been following iPhone since its inception. Initially, when Steve Jobs revealed that there was not going to be any developer SDK, I was just one of the millions to be really upset. Then, one fine day the iPhone SDK was released, the now infamous App Store was released, all are happy and the rest is history. I did try out some examples immediately after the SDK was released but then got busy with other interesting stuff (like trying out a startup) and didn’t focus much on it – until recently.

A few months ago, my company picked me to get trained on the iOS SDK. Five days of Objective-C and XCode and Interface Builder later, I was coding for the fifth mobile platform in my career. I initially started with BREW, moved on to J2ME (now Java ME), then Blackberry (includes J2ME), then into the Bold and Beautiful Android and now into iOS.

I cannot help but compare my Android experience with iOS. Coming from a Java background, it should not be a surprise that I will be comfortable with Android SDK than with the iOS SDK. Even when that is the case, I feel that the iOS SDK and its tools is not all that developer friendly compared to Android. Apple has created the best layman products ever but they are quite bad at creating developer products, I guess.

Objective-C as a language is a lot more difficult to master than Java. The complex memory management gymnastics, multiple files to define one class and weird method definition syntax are just a few of my gripes. XCode 3.x is a hell to work with at least from an Eclipse/Netbeans/Idea user perspective. Why should I have to move between two applications called XCode and Interface Builder to develop one iPhone app? Why can’t I develop in Windows or Linux if I want to? Why doesn’t the debugger work well most of the times? Why isn’t Garbage Collection part of the language? Why isn’t incremental compilation available? These are just a few questions that come to my mind when I develop for iOS…

That said there are quite a bit to like about the iOS platform as well. Objective-C has some cool features like id type, protocols, named parameters and categories that I would have loved Java to have. I keep hearing from people that once you deploy an iPhone app in the App Store, the benefit in terms of payback is good enough to justify the pain. Interface Builder is very good at what it does compared to the equivalent Android designer. XCode 4.x vastly improves the development experience by integrating Interface Builder tightly.

Although both iOS and Android are vastly different platforms, they are together ruling the world today. As developers we don’t have a choice other than learning and working with these heavenly beasts until the cross-platform world for mobile (like PhoneGap, Titanium, Rho Mobile etc) matures

Intellij IDEA key map for Eclipse

Eclipse and IDEA logo

It is not the least easy to switch to Eclipse after years of addiction to Intellij IDEA. IDEA is the best IDE I have worked with until now and will continue to remain that way until there is another visionary who attempts to improve the developer pleasure instead of just technical modularity. Eclipse may (or may not) have a better architecture, better design, better modularity, incomparable number of plug-ins and better written code than IDEA, but that doesn’t seem to attract the end developers like me.

I just love IDEA for the way it understands me, knows what I am trying to do at any moment and most importantly for the complete integrated development experience it gives. That doesn’t mean that Eclipse is not famous. Eclipse is very famous and many people love it too. But I think it is because it is free and more so because every other software vendor creates plug-ins for eclipse and that level of community support is not there for Netbeans or IDEA.

Anyways, recently when I had to switch to Eclipse for a few months for one particular development project, I was not a happy camper. I wanted to bring at least some level of IDEA like familiarity into Eclipse and with a happy accident I was able to get to the IDEA key scheme for Eclipse. I sincerely thank Santhosh for having created an Eclipse plug-in for that. It works as advertisied except for a very few gotchas.

To create an IDEA key scheme for your Eclipse Helios installation, download the plug-in from here. Drop the downloaded jar file into $ECLIPSE_HOME\plugins folder and restart Eclipse. Then goto Window -> Preferences -> General -> Keys and modify the scheme drop-down to change it from “Default” to “Intellij IDEA”. Click Apply and OK and you are done. Every key combination like Ctrl+N, Ctrl+Shift+N, Alt+F7, Shift+F6, Ctrl+F12, Ctrl+Shift+-, etc should work as expected.

It did not give me THE COMPLETE IDEA experience but I guess this is the best I could get for now. If anyone has any suggestions for a better IDEA experience please do share them in the comments section. I most definitely will be grateful.