Monthly Archives: June 2011

Why we don’t use Doubles for Financial Calculations

My current client is in the middle of hiring some Java developers and as I mentioned earlier in March (Interviewing – the importance of PASSION!) I’ve been doing some of the interviewing. One of the things we’ve done is to create a technical task to see how the candidates actually code. It’s a simple exercise that requires them to think through some of the basics of financial operations and one thing that has surprised me has been the common use of doubles to represent financial values. It’s been highlighted for some time that this is not a great thing to do but someone actually challenged me to show that it wasn’t.

So here we go…

package com.bloodredsun;

public class DoubleOperation {

    public static void main(String[] args) {

        double t1 = 10.266d;
        double t2 = 10.0d;

        //Outputs 0.266
        System.out.println(t1-t2);

        double h1 = 100.266d;
        double h2 = 100.0d;

        //Outputs 0.26600000000000534
        System.out.println(h1-h2);
    }
}

Ouch! That is not what we want but it is the classic behaviour of doubles. The inability to represent some decimals in the IEEE-754 format (as binary fractions) causes this. If we want correct precision the answer is to use BigDecimals but we have to remember to use Strings in the constructors or you end up with the same issues that you were trying to avoid.

package com.bloodredsun;

import java.math.BigDecimal;

public class BigDecimalOperation {

        public static void main(String[] args) {

        BigDecimal t1 = new BigDecimal("10.266");
        BigDecimal t2 = new BigDecimal("10.0");

        //Outputs 0.266
        System.out.println(t1.subtract(t2));

        BigDecimal h1 = new BigDecimal("100.266");
        BigDecimal h2 = new BigDecimal("100.0");

        //Outputs 0.266
        System.out.println(h1.subtract(h2));
    }
}

That’s great but wouldn’t it be nice to use the normal operators rather than the overly-verbose method calls for the mathematical operations.

Now there is no way that we can do this in Java but if we let ourselves use another language on the JVM…

package com.bloodredsun

object ScalaBigDecimalOperation {

  def main (args: Array[String]) {
    var t1 = BigDecimal("10.266")
    var t2 = BigDecimal("10.0")
    //Outputs 0.266
    println(t1 - t2)

    var h1 = BigDecimal("100.266")
    var h2 = BigDecimal("100.0")
    //Outputs 0.266
    println(h1 - h2)
  }
}

Scala FTW!

PS if you want to know more about floating point operations have a read of What Every Computer Scientist Should Know About Floating-Point Arithmetic

S2G Conference, London May 31st – Notes on Tuning Grails Apps, Peter Ladbroke

How to get the most out of the grails app

Many people make decision based on delusions that you will scale up to Facebook sizes. Making decisions based on those requirements is ridiculous.

Premature Optimization
Profile then optimise. Where can things be slow – DB/BusinessLogic/Network/UI

Profilers
Spring Insight – free with STS. gives a broad range of metrics
Google Speed Tracer
Reduce the number of DB queries
use appropriate fetch mode
don’t fetch the data you don’t need
Add appropriate indexes
Don’t be afraid to change the model

Caching
Aternative Data store – e.g. NoSQL for example but how do you know the sweetspot
The key us referential integrity and simple models
Spring caching abstraction
Go Asynchnoous – Spring Events, Executor, Messaing JMS/AMQP

Fall back to Java for Performance – especially for numerically intensive operations

Groovy++ @Typed(TypePolicy.MIXED) for speed increase

View rendering in GSPs
Reduce number of requests
Bundle CSS and JavaScript + minify and compress
Image Spriting
Expires Headers on static resources
Spring Cache with Views
Zipped Static resources

Grails in Action

S2G Conference, London May 31st – Notes on Spring Data, Oliver Gierke

This was the lecture I was most interested in. Given the interest in NoSQL, as well as the FUD and general handwaving from the industry on this topic I really wanted to see what SpringSource were making of it.

2 part lecture : motivation to why and then examples

Motivation
Until recently, we have stored everything in RDBMS. This is sub-optimal for objects but especially or trees or graphs or networks. The last couple of years has seen th rise of NoSQL is the umbrella term for the desire

So which do I use? And if I do have a shortlist, how do I interact with them. Well there’s some Spring for that! The Spring Data Project –  aims to provides a familiar and constant Spring based programming model while not over-abstracting custom traits of the specific store. The last point is emphasised because an overly flat abstraction means that you would lose the things that make each implementation special.

The First point is just the usage of familiar spring idioms like Templates and Repositores. Riak and Redis are quite well supported. MongoDB and CouchDB is quite well supported.Haddop and Neo4J are also supported. JDBC and JPA is aso supported since this project is about Data, not just NoSQL.

The examples for this talk are based on MongoDB. JSON based with nested KV pairs  with consistency within the document.

The building blocks of Spring Data: Spring Core (DI, AOP, Namespaces, JMX, JDBC), Templates for each implementation for resource management (connections) and exception handling – also some convenience methods for simple queries, Mapping support  via annotations to map the domain model to the store. Repositories layer – finder methods and query methods like in Spring Roo.

Examples are all on github. – org.springframework.data.mongodb.examples.music

In the mongo example, the domain model is decorated with @Document annotation and the index member decorated with @Id

A kv pair is a field and it’s value. By default the mapping are with the member name but these can be overridden. @DBRef allows you to connect to a separate collection in another DB. Multiple constructors can be filtered with @PersistenceConstructor

To use it you just need to declare the implementation namespace and then create a template. Like RestTemplate, the templates are underpinned by the equivalent *Operations Interface.

Repositories for CredRespoitory  for finder methods. NBoth names and locations (co-ordinates) which auto-creates the repositories which give you a lot of power for simple queries like findOne(Id) find<Member><Property>[<Keywords>].

THe auto-gen stuff is done at context startup so you don’t have to wait until runtime for it to fail.

What about complex queries where you don’t want ugly names. Annotations of course! You can have queries in the annotations

SPring Data supports pagination out of the box. All you need is  Pageable argument where the PageRquest is the implementation. Declare the result type as Page andd you get Context like isLast and getTotalPages.

QueryDSL open source project?

Cross storage persistence is also possible with a nice annotation model that makes it pretty seemless.

Great lecture.

S2G Conference, London May 31st – Notes on Spring Integration, Josh Long

Tough lecture after lunch, Josh covered the Spring Integration project. Based on Gregor Hophe’s book, it has a lot of similarities with the Apache Camel Project. I wanted to take more notes but my laptop started shedding battery life and I really wanted it for the next lecture.

I did want to ask why someone should use this project rather than Apache Camel but it seemed a negative question and I didn’t want to ask one of those.

S2G Conference, London May 31st – Notes on Spring MVC 3.1, Rossen Stoyanchev

Java based Web MVC Configuration
Contimizable @MVC
URI templates variable
Serverside testing

Java based Web Annotation Namespace
Java based config allows you to create  a class, annotate it with a @configuration to indicates that this is going to created bean instances. This allows you to keep your config with your code
Most Spring MVC is java based already so why not enhance it. Servlet 3.0 spec further reduces the xml load and xml is not transparent anyway.

@EnableWebMvc
Enables declarations of HandlerMapping, Converters, Resolvers, MessageSources, Multiplart

@Configuration
@EnableWebMvc
public class WebCnfig {

}

Also have the ability to exclude configuration classes that have already been discovered

//WebMvcConfiguration example class already exists but also extending WebMvcConfigurationAdaptor allows selective overrides for easier start up.

This does not exclude xml based config. It just means that Java is loaded first and then xml can be loaded secondarily

An example of this os the  GreenHouse application – the WebConfig.java class.

Nice use of the Environment object which plays nicely with the profiles. The properties are are enabled with the code

Cusomtizable MVC
The chance to annotation based MVC was a big change. More flexible mapping, better information on interception

New classes. HandlerMethod not just for @RequestMapping but also @InitBinder, @ModelAttribute and @ExceptionHandler. These underlying infrastructure classes have numerous advantages. THe ability to know the handler method that is going to be invoked, different controller has handler different HTTP methods on the same URL

Fully customisable processing pf all method arguments – HandlerMethodArgumentResolver
Same of all return types
HandlerMethodRetutnValueHandler

Transition from 3.0 should be seeless since all the current classes will still exist but the new classes are the future and are recommended

Handler Method Interception
Pr/post bind and pre/post validation hooks
COntrollerethod name conventions for default views, e.g. Account.show = account/show

URI Template Variable
Path variables can be automatically added to the model
Redirect supports path variables, e.g. RequestMapping /events/{year}/{month} has a redirect of “redirect:/return/events/{year}”{month}
It also offers databinding on none model parameters in the request mapping decorated methods

Matching MediaTypes
In 3.0 you used the headers attribute in the @RequestMapping annotation. In 3.1 the consumers and produces similar to JAX-RS. The consumers/produces offers the benefits of a proper return value on mismatches like 415 on a bad request type or 406 NOT_ACCEPTABLE

Serverside Test Support.
Testing a controller properly. Unit test only does the class and not the framework features. This normally requires an integration test that requires an in-memory  servlet  (JWebUnit)
We want to verifier framework features like binding errors or redirects and forwards or even view names
This is work in progress and will be in the spring-test project when it is ready

S2G Conference, London May 31st – Notes on Spring 3.1, Jurgen Holler (Spring Framework Project Lead)

Spring 3.1 Jurgen Holler (Spring Framework Project Lead)

Themes and Trends

Background and Current Trends
Again a mention of the cloud within the principle of war files becoming dominant
Given that most cloud stacks are simple arrangements of best of breed  – what happens with the application servers? Only Glassfish supports all, JBoss 6 and Websphere 8 only support the Java EE Profile.
Tomcat 7 is the first server that SS test against. He expects all Clouds to offer Tomcat 7 as a default fairly soon.

From a spring perspective, Spring is DI, AOP and Portable Service Abstractions (Persistence, Scheduling, Transaction Strategies) around simple objects.

Spring 3.0 Review
Annotated component model, including JSR-330 support
Spring EL
REST support
Portlet 2.0 support
Declarative Model Validation (JSR-303 Bean Validation)
JPA 2.0

Spring 3.1 Key Themes
Environment Profile for Bean definitions
Java-based application configuration
Cache abstraction and declarative caching (fully fledged not just some EHCache beans)
Support for Servlet 3.0
Customizable @MVC processing (this was very popular in Spring 3.0)
Conversation Management

Environment Profile for Bean definitions
A core api then provides an environment type
It is an SPI
PropertyPlaceholderConfigurer will now delegate
Profiles activate beans depending on attribute in the beans element e.g.

<beans>
<beans profile=”production”>

</beans>
<beans profile=”embedded”>

</beans>
</beans>

or @Profile on configuration classes or even on individual classes

Activated by -Dspring.profiles.active=production or something else like JNDI or environment variable or even programmatic. IDeally this should be done outside the deployment unit. This works well with Continuous Delivery

The Application Configuration model is very elegant. Able to offer almost as much power as the xml, including the tx name space. Builder classes are used to the session Factory for Hibernate for example. Builders are starting to crop up in Spring 3.1 in JPA for example to set up complex object in Java

Namespace
SS are still investing in the xml namespace. Shortcut for constructor args, c:namespace. Using the constructor names gets around the issues for reflection  (constructors have an index and a type) and other runtime issues. Allows final fields so always nice. But this uses ASM to read the debug symbols so how fragile is it even if it is expressive?

Cache Abstraction
Very important with the rise of distributed caching – (cloud again!)
Before now, only had EhCache support
CA Allows you to integrate the likes of EhCache, Gemfire and Coherence.
The framework may not have specific examples since the caches are often closed source and proprietary
Setup is often on per environment bases so again plays well with the Environment Profiles

Declarative caching
Not code, just annotations on methods, concise and readable
@Cacheable,
Caching bypass @Cacheable(condition=”name.length < 10″),
@CacheEvict
Strategies are customisable. Simple but powerful for the 80% that is this simple. More complex interactions can be done via the AbstractCacheManager which you can just inject
Also a cache namespace in xml.
Need to create a cacheManager, just like Tx namespace. If you don’t declare one then annotations are just bypassed so nice in dev.
CacheManager SPI

Servlet 3.0
such as Tomcat 7.0 and Glassfish 3
XML free spec (if you want)
Async request processing (Spring 3.1 does not support this yet)
File upload support as standard but how much of  a pain point is this?
WebApplicationInitializer allows programmatic startup of the web app. I think that some best practise needs to spring up around this. How many do you want? What is the security implications of web.xml fragments, especially in 3rd party libs.
JPA requires an xml config so that may be the only one your application needs
Jurgen makes a good point that xml config is not necessarily bad. It’s readable and has great support in modern iDEs but now the choice is yours and not made for your by the framework.

Configurable MVC processing
REST support is improved
Arbitary mappings better supported

Conversation Sessions
HttpSessions++
more flexible lifecycle and storage options
Seems to be more around the integration/management rather than a new scope: isolation between tabs and windows where they share HttpSessions but this is a flag rather than anything more sophisticated.

Spring 3.1 M2 is going to be released this week. No M3 , RC is next

Spring 3.2 planed for Spring 3.2. Direct follow up for 3.1 but based around Java 7 (GA release Q3 2011). JDBC 4.1 support and support for fork-join framework. Also JSF 2.2 and JPA 2.1 updates and maybe Hibernate and Quartz updates

Java 7 offers some new concurrent programming APIs (like java.util.concurrent.ForkJoinPools) that offer really good functionality

Writing a Technical Test..with a little help from the LJC

As someone who interviews developers, I had been looking at ways to improve our current interviewing process. I’m not a fan of written tests or of coding on white boards since they don’t accurately represent the day to day requirements of a developer and I don’t even think they are good surrogate markers.

Knowing Barry Cranford through the London Java Community, I thought I would ask him. Given that he probably sees more interview processes than anyone else I know it seemed a fair idea. After a quick chat over ice-creams (it was a hot day!) I suggested to Barry that I thought we should get potential candidates to do a small technical task before they came in so that we could go through it. That way I could get to see exactly how they coded and how they thought. Barry suggested we could improve it by building on it in a paired-programming exercise during the interview. That way we would be able to see how they worked in a team and in a pressurised environment.

I thought this was great and spend some time coming up with a task that could be completed within 1.5 hours but had some natural extension points for the paired programming exercise. The trouble was, I had no one to test it on other than myself and guys in my team. Again, the guys at the LJC came to rescue by volunteering to take it. I was pretty proud of the test and thought it was quite good as it allowed candidates to showcase their skills without being too hard but it was completely shredded by the reviewers! As one of the guys had pointed out, the task made too many assumptions about the domain knowledge of the candidate and the requirements were too vague for someone outside of the industry.

So I went back to work and rewrote it. The acceptance criteria became much tighter, I removed some of the ‘help’ than was actually misleading and reworded it so that the target goal was as clear as possible. The result is a solid task that is now being used for teams other than my own. So extra kudos for me and better candidates for the company.

Without the community input there is no chance that it would have been half as good. Worse still I would have been stuck with a bad test that I thought was good. I am very grateful for the various guys at the LJC for the time and effort they put in to help a fellow LJC member, so special thanks to Mustapha Hanafi, Martijn Verburg and most especially to Ged Byrne for all the help. Thanks guys.