TIM Group Scala Dojo

TIM Group hosts a monthly Scala Dojo as part of the London Scala Users’ Group on Meetup (http://www.meetup.com/london-scala). The dojo is free to attend with drinks and pizza provided to the participants. The dojo always takes place on the third Thursday of the month in the TIM Group London offices.

The dojo is driven by participant in suggesting and choosing shorter programming challenges and collaboratively finding a solution in smaller groups. The atmosphere is relaxed with a focus on fun and learning. The range of scala skills range from total beginners that heard about scala to seasoned enthusiasts that work with scala. Towards the end of the meetup, the different groups present their work and discuss what they have learned.

Example of programming challenges are:

  • Secret Santa problem where the challenge is to find an algorithm for matching people giving each other gifts. No one should get left out and additionally no should be matched with their spouse.
  • finding an approximation of PI using concurrency with akka.
  • using the coursera Scala survey data to find alternatve stats such as ranking countries based on average perceived difficulty.
  • solving the Data Munging Kata from http://codekata.pragprog.com

I have taken my first steps in Scala in the dojo and found it a great learning experience. The time absolutely flies and you often end up with a great feeling of achievement by the end of the night!

The next Scala Dojo is on Thursday Jan 17th.
I hope to see you there!

FOSDEM 2012

Last weekend I attended FOSDEM, “the biggest free and non-commercial event organized by and for the community”. The Free and Open-Source Software Developers Meeting, held in Brussels, pulls in roughly 4000 FOSS hackers from around the world. I was one of the speakers on the schedule in the Free Java Dev Room, introducing my open source project, Mutability Detector. The lineup was pretty illustrious, including Charles Nutter, co-lead of JRuby, and Mark Reinhold, the Chief Architect of Java Development at Oracle. Here I’ll describe some of my experiences at FOSDEM, and relay some of the interesting stuff people presented. (Apologies if I misremember anything – there was a lot going on at the conference!)

Lost in Translation

First off, let this be a warning to any potential FOSDEM-goers. ULB, the University that hosts the event, has two campuses. Campus Solbosch, where the entirety of FOSDEM happens, and about 11km from that, way over on the other side of the city, Campus Erasmus, where I went. Top tip: when planning international trips to foreign countries that you’ve never visited before, don’t rely on Google Maps.

After a 1hr 30min commute, with the help of a few fellow location-impaired, and thankfully French-and-English-speaking FOSDEM attendees, I finally made it to Campus Solbosch. Unfortunately that meant missing the welcome keynote, the introduction to the Free Java Dev Room, and talks about the OpenJDK from Mark Reinhold and Dalibor Topic. Massive facepalm. Hugely embarrassing. Mothers, don’t let your children grow up to be… someone who goes to the wrong ULB campus for FOSDEM.

Lambdas and Traits and Bears, Oh My

The first talk I saw was by Jeroen Frijters, demonstrating the work he’s been doing to bring JSR292 (Invoke Dynamic) to IKVM.NET, the virtual machine that runs Java on the CLR. Next up came Remi Forax to discuss lambdas in Java. Following a clear disclaimer that design choices and syntax he discussed was still in the air, and subject to change, he showed off an expected feature for the Java language. Named “Virtual Extension Methods”, they allow adding methods to interfaces… with implementations. The motivation being that, along with lambdas, you could put a method like Iterable<T> filter(Predicate<T> predicate) onto, say, java.lang.Iterable, with an implementation, without causing compiler errors in everyone’s code. There was a bit of a collective gasp when some people in the audience assumed this meant multiple inheritance was coming to Java. However, the subtle but crucial point is that with a virtual extension method, it’s not possible to reference state, such as fields. The body of a virtual extension method must rely on the concrete implementations to supply state, with the developer resolving conflicts from inherited methods in the concrete implementation at compile time. That leaves the feature looking a lot more like traits (awesome) than C++ style multiple inheritance with the diamond problem (not awesome).

Following Remi were Raoul Gabriel Urma and Janina Voigt, two PhD students from Cambridge, who used the OpenJDK to explore the use of covariant arrays in the wild. Taking a look at lots of open-source projects, and using the novel approach of modifying the javac compiler to find the code they were interested in, they found that covariance is rarely required, and normal Java generics is usually enough to add type safety to methods employing covariance. I really liked the empirical approach taken to a language theory question, and love the idea they had of setting up some kind of open-source corpus, where those doing similar surveys could quickly get their hands on a huge amount of real-world code.

Community Is Not Just A TV Show

Then came Ben Evans and Martijn Verburg, giving a talk entitled “Politics, Positives, Paperwork and Pain: Our year to date in the JCP”. Ben and Martijn are the leaders of the London Java user group, the London Java Community (LJC), which, in the past year, has won a voting seat on the JCP (Java Community Process). They discussed some of their experiences, from the bad:

– the acronym soup that appear in most every conversation (JCP, JSR, JSPA, TCK, RI… TLA ETC – okay those last two aren’t real)
– phone conference calls with lawyers representing some of the companies involved in the JCP, discussing in minute detail whether to change “notwithstanding” to “nevertheless” in clause 27(g)
– no-show members of the JCP who only participate in discussions after decisions they don’t like have already been made

But they did have a lot of positives to recount:

– completion of the first of several reforms to the JSR (Java Specification Request) process which will make it more open and inclusive, and hopefully “prevent another EJB 2”
– the (surprising) willingness and motivation of several of the bigger JCP members to move towards being more open
– significant takeup of the LJC-led “Adopt a JSR” appeal, which gets everyday developers involved in particular JSRs, from development to evangelism (I myself am signed up to take part in JSR 353 – API for JSON Processing, as we have been building a JSON-based API for one of our products recently)
– drumming up lots of interest in contributing to the OpenJDK from LJC developers, which included a couple of ‘Warnings Cleanup’ hack sessions, where a bunch of hackers dived into the OpenJDK code to pick some low hanging fruit, fix the code, and submit patches. This is slowly pushing the OpenJDK project to make building, modifying and contributing much easier. One such event was hosted here at TIM Group, with all available spaces being taken about an hour after announcing it on meetup.com.

Overall Ben and Martijn are optimistic about the future of the OpenJDK as a community.

Open Sesame

For the rest of the afternoon, we were introduced to several open-source tools: Thermostat for monitoring performance of your Java application; GCSpy for inspecting and visualising garbage collection (here was the first time I heard the humorous quote “If Java had true garbage collection, most programs would delete themselves upon execution.” — Robert Sewell); Byteman which allows you to inject bytecode into classes at runtime, and a DSL to allow you to talk about bytecode at a higher level of abstraction; and CacioWeb, which can let you run a Swing app inside your browser, which struck me as weird thing to want to do, but it had some pretty interesting use cases (e.g. rather than people using VNC to access their own Swing app running in their own JVM, the app can be hosted by a single JVM for multiple users). Steve O’ Grady argued, with some metrics and empirical counts, that Java the language is still not dead, and it’s not only about the JVM either.

The audience were also given the fine treat of listening to me harp on about they should design their classes to be immutable, and how they should totally use Mutability Detector to keep them that way, the lucky devils. It’s not every day you get to hear me crack some lame jokes about how XML sucks (my colleagues, on the other hand…). Some good ideas and suggestions came out of the Q&A after my talk, and I also got a lot of positive feedback (like Mark Reinhold tweeting “@grundlefleck’s Java immutability detector is really cool” — after which I almost audibly squee‘d). Overall Mutability Detector was received well, and I’m really glad I applied to present.

Absinthe Makes The Heart Grow Fonder

Once Saturday’s talks were finished, I was invited to dinner with other speakers and sponsors of the Free Java Dev Room in a restaurant just off Brussel’s Grand Place. Lots of interesting code talk, nice food, and Belgian beer. Having spent a lot of Saturday suffering from Friday night’s excesses at FOSDEMs Free Beer event at a local bar named Delerium, I didn’t fancy a repeat on Sunday. So I ducked out early-ish, before Raoul, the PhD student and Belgian native, could convince me to visit a nearby absinthe bar. Judging by the ‘tender’ disposition of some of the folks on Sunday morning, it appeared to be one of the more sensible choices I made over the weekend.

Fast Ruby and ARMed Java

The first Sunday talk I attended was Charles Nutter discussing JRuby and some of the achievements they’ve made with the project. I was surprised to hear that JRuby on the JVM was (for most things) the fastest implementation of Ruby around, even including native implementations. Then came a talk on how the Eclipse project works. It was interesting to see just how big an ecosystem and community it is, and they managed to ship, on time, on the same date for the past six years. The speaker, Mike Milinkovich, emphasized the importance of predictable releases, and how the Eclipse project has and will drop features to make the scheduled release date. We then heard from Andrew Haley of RedHat, the work he’d been doing to port OpenJDK to ARM processors, and someone from the audience showed an Eclipse installation which was built, and currently running, on an ARM processor machine, which was impressive.

The next talk which (I am told) was excellent, was on a music-based Java project named Gervill. It included a musical representation of a Java program, using the Byteman project demonstrated the previous day, where every call to list.add(), was translated into a note, where the index argument became a position on the scale. I say I was told it was excellent, because I’d actually ducked out of this talk, and into one of FOSDEM’s Hack Rooms, where I introduced Martijn Verburg to the internal workings of Mutability Detector, which lead to the first bit of activity (“frsit psot”, and a patch!) on my project’s mailing list.

The day concluded with talks about IcedRobot (an Android implementation for the desktop), IcedTea and IcedTea-Web (RedHat’s repackaging of Oracle’s OpenJDK sans proprietary parts), and then a free-form discussion on Java, with the likes of Mark Reinhold, Dalibor Topic, Georges Raab (the VP of Software Development, Java Platform Group at Oracle) answering questions and discussing community issues in and around the OpenJDK project. It’s great to see this kind of discussion take place, and it’s somehow always astonishing to me to hear just how many people and projects depend on Java, and how much effort goes into releases, distribution and upgrades. I’m so glad that in my day job here at TIM Group, we generally only work in two week release cycles, developing a hosted web application — some of our problems can seem slightly trivial in comparison!

And thus ended the scheduled talks at FOSDEM. I capped off the weekend enjoying a few beers and dinner with a few of the speakers from the Free Java Dev Room.

Belgium, man! Belgium!

Overall, despite the calamitous beginning of the weekend, attending FOSDEM was a hugely rewarding experience. The speaker line-up for the Java room was impressive, and it looked to be the same for the other tracks and dev rooms, with well known names in the open-source world. The conference also gave me the chance to meet and personally thank Koshuke Kawaguchi for his and the rest of the project’s efforts on the Jenkins CI system that is used heavily at TIM Group. There was a very friendly vibe around the place, and what the organizers accomplish, without ever charging a penny for tickets, is astonishing.

So in summary, if you can make it to FOSDEM (and it definitely helps if you work for a company like TIM Group, that lets you take the time to go, and puts up the cash for travel and accommodation), you definitely should… but you should go to the correct campus.

More Thought-Provoking Lightning Talks

Five minutes per person. Every Thursday. From the practical to the theoretical, we manage to cover everything at TIM Group lightning talks. A sampling below.

Dominic on Categories and Time

Categories and Time from youdevisetech on Vimeo.

Tony on Levels of Team Maturity

Levels of Team Maturity from youdevisetech on Vimeo. Based on ideas from a course by Roy Osherove.

Joe on Continuous Integration at LMAX

Continuous Integration at Lmax from youdevisetech on Vimeo. Learn more from the LMAX blog.

Exploiting Lightning Talks For Fun (But Not Profit)

One of the cool things about working at youDevise is the weekly lightning talks, which give you the opportunity to peddle your own open-source projects to a group of attentive, intelligent and (healthily) cynical developers.

In my case I was able to introduce my static analysis tool for Java, called Mutability Detector. The video of the lightning talk is now publicly available (thanks Tom W), which you can watch here or in the embedded player below.

[video:vimeo:17218830]

The tool is free, open-source, available from the project home page, while my echoing discussions and descriptions of the tool can be found at the project blog. It’s in a very alpha state, though a colleague here has used it and found it informative. Suggestions, criticisms and bug reports are very welcome!

Okay, so the lightning talks may not offer an opportunity for profit when all you’ve got to give, you give away for nothing, but it is definitely fun!

Developing custom FindBugs detectors – a test-driven approach.

There are precious few examples ([1] and [2], below, being good ones) on the interwebs showing how to develop custom detectors for FindBugs, a handy feature of the static analysis tool I’ve blogged about previously. There are even fewer (zero?) examples showing how to create automated tests for your new detector. The implication being that the way to test a detector is to run the FindBugs program and manually inspect the results. As you can probably guess this quickly becomes tedious, and the feedback loop for your code changes is frustratingly long. Fortunately,  there is now a better way. Using a new open-source utility from youDevise, test-driven-detectors4findbugs, this blog post will show how to test-drive development of your custom FindBugs detector. Thus shortening the feedback loop, guiding development, and leaving behind a helpful set of automated unit tests to catch later regressions.

For the sake of this discussion, lets say we want to raise a bug against classes whose names are too long – not that FindBugs is the best tool for this job, but it’s a trivial enough ‘bug’ to demonstrate test-driving a FindBugs detector.

It’s often best to write some examples (or ‘benchmarks’) to test your detector with, for example, a class which you would wish FindBugs to report a bug for, and one which would you wouldn’t. If you had a more complicated code pattern you wished to detect, you would add a benchmark for each interesting case. For now, the ‘class name too long’ bug only requires a couple of cases. These could be:


class ExampleClassWithAnAllowedName { }
class ExampleClassWithANameThatIsTooLongForThisSillyDetector { }

Next, we can write a failing unit test. This will use functionality from the test-driven-detectors4findbugs, which is a JAR file which needs to be added to your testing classpath. Although there are a couple of tests that you could write at this point, I’ll show just one. This assumes a JUnit test, but the utility should work with other unit test frameworks which use the AssertionError mechanism for failing tests, e.g. TestNG:


import com.youdevise.fbplugins.tdd4fb.DetectorAssert;

@Test public void
raisesBugAgainstClassWithLongName() throws Exception {
    // Obtain a BugReporter instance from this method
    BugReporter bugReporter = DetectorAssert.bugReporterForTesting();

    // And pass the same BugReporter to your detector
    Detector detector = new CustomClassNameLengthDetector(bugReporter);

    // Next assert that your detector has raised a bug against your benchmark class
    DetectorAssert.assertBugReported(ExampleClassWithANameThatIsTooLongForThisSillyDetector.class,
                                     detector,
                                     bugReporter);
}

This will fail to compile, since theres currently no Detector class which conforms to the specification required by FindBugs. So we can, write out the template for your detector class. For FindBugs to execute your custom detector, it must implement the Detector interface, as well as having a convention-based constructor signature,  since the detector is instantiated with reflection. Thus your template should look roughly like this:


import edu.umd.cs.findbugs.BugInstance;
import edu.umd.cs.findbugs.BugReporter;
import edu.umd.cs.findbugs.Detector;
import edu.umd.cs.findbugs.ba.ClassContext;

public class CustomClassNameLengthDetector implements Detector {
    private final BugReporter bugReporter;

    public CustomClassNameLengthDetector(BugReporter bugReporter) {
        this.bugReporter = bugReporter;
    }

    @Override public void report() { }
    @Override public void visitClassContext(ClassContext classContext) { }
}

The test should now compile, but fail, since the CustomClassNameLengthDetector does not raise a bug. Now you’re ready to add in the real functionality, while listening to the tests telling you when your code runs as you expect, without you having to manually execute the whole FindBugs process through the Eclipse, Swing or command-line interfaces.

Filling in the implementation of the detector is not particularly interesting for this example, so I’ll wrap it up there. To see the whole code listing, you can check out the source on GitHub. There are also some examples of  more sophisticated assertions, using Hamcrest matchers.

Additional info on test-driven-detectors4findbugs:

  • a pre-built JAR is available for download from the download section of the GitHub homepage
  • released under the OSI-approved MIT license
  • has runtime dependencies on FindBugs 1.3.9 and Hamcrest 1.1+
  • although requiring version 1.3.9 of FindBugs to test against, detectors will work with FindBugs versions since (at least) 1.3.7

Writing your own FindBugs detectors is a great way to catch bugs you and your team haven’t written yet. If you’re developing one, hopefully test-driven-detectors4findbugs will make the process quite a bit less painful.

[1] http://www.ibm.com/developerworks/java/library/j-findbug1/

[2] http://www.danielschneller.com/2007/04/findbugs-writing-custom-detectors-part.html

If you’re walking, start crawling, if you’re crawling, start inching

Last Friday, Keith Braithwaite came to youDevise and ran his “TDD as if you mean it” workshop here in our offices. For those not familiar with how the workshop runs, Gojko has a good writeup of the session as it was run at the Software Craftsmanship 2009 Conference.

The puzzle as presented is one of the basic rules of the game of Go – Atari: A stone (or chain of stones) is in Atari when it has only one liberty.

Our job was to express that in code, but we had some constraints:

  • Only have one failing test at a time
  • Write implementation code in the test itself
  • Only move passing code
  • All new methods are written in the test class

But more than the contstraints was the ongoing extreme exercising of YAGNI, and that’s where a lot of the magic of the exercise resides.

As I was the organizer of the event I wasn’t actively participating in the workshop myself. At first I was a bit dissapointed by this, that quickly faded as I realized that it gave me the chance to see all the different ways my co-workers came at the problem. Initally I expected we would all be doing the exercise in Java, but it turned out that three languages (Java, Ruby, and Scala) were out to play. At first a number of the participants jumped ahead into setting up domain objects in an attempt to create the example scenarios, however this was discouraged, and everyone was pushed (both by the constraints, and by Keith himself) to go small and stay small until it was absolutely necessary to do more. It was the repeated exercise of pushing against that pain, and making ourselves stay there that was the real eye-opener. By not putting things into the code until we absolutely needed to, we left our code open to surprising possibilities, like discovering that even the concept of a stone isn’t really needed, by modeling a chain of stones not only do you easily cover both the single and multi-stone scenarios, but you can also remove the need to determine the color of neighbours. My first instinct when I heard the scenario was to model a stone, but even the act of doing that simple modeling pushed me away from the above discovery of the elegance of using the chain.

I felt a lot of push-back internally with the exercise (and I wasn’t even doing most of it!) Going small feels so *slow* sometimes, but all of that questioning can give you incredibly small, elegant, and most importantly, nimble code, and that can save you incredible amounts of time going forward.

Many thanks to Keith for running a very interesting and thought-provoking workshop!

Lightning Strikes Again

We gather at lunchtime every Thursday for lightning talks – 5 minutes, no questions till we’re done, talk fast! Thanks to excellent video work (thanks Wendy, Waz, and Tom!) we now have a nice permanent record of each talk. Here’s a sampling of recent talks.

Tom D on Hamcrest Matcher Matchers

A Matcher Made In Heaven from youdevisetech on Vimeo.

Joe on Gradle

Understanding Gradle from youdevisetech on Vimeo.

Ryan on DevFort and the Final Frontier

DevFort 5 – To the Moon and Back from youdevisetech on Vimeo.

Mocking the proud

A mocking framework such as JMock provides a way of specifying and checking the interactions between an object under test and its collaborators, without having a full implementation of those collaborators to hand. However some prideful classes don’t like being mocked!

Initially, when all we have is an interface specifying the contract that some collaborator must fulfil, the mock objects created by the framework act as stubs for implementation code that has yet to be written. Later on, however, we use mock objects as replacements for actually existing implementations, so that we can unit test an object that drives other parts of a larger system without having to come up with a complex test configuration for the system itself.

In both cases, mocking allows us to pretend that the object under test has a fully functioning environment to interact with, when in fact it is in the position of a “brain in a vat” being fed artificial stimuli by the testing code.

User:Кирилл Мартынов [GFDL (http://www.gnu.org/copyleft/fdl.html) or CC-BY-SA-3.0-2.5-2.0-1.0 (www.creativecommons.org/licenses/by-sa/3.0)], via Wikimedia Commons

In the usual case, a mock object fills out some entirely abstract interface: by default, JMock will complain if you try to mock a concrete class. Although with a bit of persuasion JMock is able (thanks to some cunning bytecode manipulation) to create mock implementations even of final classes, I’d argue that whenever you find yourself having to do this you have a prideful class and should consider it as a potential code smell.

Prideful classes are a particular problem when they show up at the public boundary of a system (e.g. as part of its API). Objects under unit test should ideally interact with the wider system through collaborators on that boundary ,rather than “reaching in” to work with internal system objects, in violation of the Law of Demeter.

So when you find you have to mock out an interface that a) belongs on the external boundary of some part of the system, and b) can’t be mocked without bytecode manipulation, because it’s tied to a specific implementation, alarm bells should go off in your head – separate modules within the system are in danger of becoming tightly and inflexibly coupled.

In this situation, consider humbling the proud by refactoring prideful classes to extract an abstract interface, which can be mocked freely.

Devops comes to youDevise!

The devops movement is one we’re quite sympathetic to – our Ops team were the first at youDevise to implement continuous deployment, for example. We recently hosted a London Devops meetup in our nice new Theatre room and discussed deployment in depth – very important to us since we are currently working to rationalise our various deployment tools.

We videoed the session, though there were a few gaps and the video wound up coming out in three sections as the operator (me!) was pretty inexperienced. Comments and thoughts are very welcome, as would attendance at the next meetup, which we hope to host again in January.

Dev Ops Part 1 from youdevisetech on Vimeo.

Dev Ops Part 2 from youdevisetech on Vimeo.

Dev Ops Part 3 from youdevisetech on Vimeo.

Code Dojo VIII – It’s full of stars!

Rather than having a Code Dojo focused on a good practice like many of our past dojos, I decided to take us down the path of exploring Google Web Toolkit (GWT).

Many of us at YouDevise have been flexing our web-development skills and been writing very interactive, AJAX-filled web pages. Lately, we have been fed up with the standard path of server-side Java and servlets spitting out HTML with a whole lot more code written in Javascript and CSS/HTML hackery. These web app setups are a pain to test and certainly hard to get right in all browsers, all the time. So, I decided to plunge us into the monolith and see what was like when you wrote (nearly) no JS/HTML code and left that to GWT.

It's full of stars

After a very brief intro into GWT and the tools, I introduced the problem: I asked everyone to enhance a sample web app by pairing up. Every pair of developers was given a different enhancement to the system to add. (There were 6 groups enhancing the system concurrently, for the curious.) In coding up the example app, I had done my best to give the initial code base some decent unit tests (both mocking and GWT-style,) so that people could explore the testability of GWT. I had also tried to carve out small enough classes with enough coherent and singular purpose to allow people to explore maintainability of the code as well.

After the dust had settled, the last check-ins had been merged, and the pizza arrived, the verdict was in: people really liked working with GWT. Given maybe 2 hours to code up a non-trivial feature, nearly all of the groups had succeeded. Some had enough time to go on to contemplate better designing of the system and best practices for testing this code base.

Now, every rose has its thorns. GWT was slow to compile and the error messaging is cryptic, but given the speed of development, ease of use of the platform, and all that you get for free, everyone was bullish on further use of GWT. All were looking forward to a chance with working with GWT again.