SoCraTes Germany

SoCraTes Germany 2015 has again been a conference full of superlatives — or should I say “unconference”, given that it consists mainly of a two-day Open Space? Whatever you call it, we had more participants (over 180), more proposed sessions (90 – 100 sessions per day), and more sponsors than ever.

Speaking of sponsors: Of course TIM Group was among them, and all contributions were spent on reducing hotel bills for participants. Not that it was an expensive event anyway: the participants cover their hotel room, breakfast and dinner, and that’s it. But let me start at the beginning…

Six or maybe even seven years ago, my boyfriend Andreas Leidig decided to create a conference for developers — after all, developers deliver the software. If you remove all managers from a software project, chances are you might get something usable out of it, but if you remove all developers, well, I’d say your chances are zero. There were lots of agile events for managers, coaches and other “tree-huggers”, but no general-purpose developer conferences along the lines of software craftsmanship and improvement.

So Andreas created a developer conference, and named it SoCraTes: Software Craftsmanship and Testing Conference. It was designed as a two-day event falling on a Friday and Saturday so that both the developer and the developer’s employer would contribute time to the event. Arrival on the previous evening was mandatory in order to start the first day early and in a relaxed fashion. Another important factor was that it was held off-site — everybody stayed in the conference hotel, which we had to ourselves, allowing us to be together from the first yoga and jogging workout before breakfast to the last beer, song, or boardgame long after midnight.

We started our first SoCraTes in 2011, with the help of a small handful of friends. The first year we had one day of talks and one day of Open Space, because we were afraid companies would not let their employees attend a conference without a programme. The feedback of the 50+ participants reassured us we didn’t need the programmed talks, and from the second year we ran SoCraTes as a two-day Open Space. We sold out in the second year with about 75 participants, confirming our decision. On Saturday evening of the second year, a group of participants decided that they had not yet had enough, so they extended their stay and spontaneously ran a Coderetreat on Sunday. This Coderetreat is now an integral (though optional) part of SoCraTes, accompanied by workshops and other half- or full-day sessions.

In the third year, we sold out so quickly (literally within a minute) that we felt the need to do something. Of course it was nice for everybody who could register, but thinking of those who could not get a spot diminished the joy quite a bit. Also, the SoCraTes “brand” had expanded to the UK, where a sibling event was being run by the London Software Craftsmanship Community. Meeting friends from abroad was an important part of attending SoCraTes; how could this reliably happen with such high demand and such a small number of participant slots?

After some discussion among the organizers, we decided to experiment with increasing the size of the event. We were curious — and also a bit tense — because we wondered whether it would still be the same event, with the same familiar feel we had come to love so much.

The fourth SoCraTes took place last year with twice as many participants, approximately 150. And it was amazing! Of course the event felt different. I was not able to speak to everybody any more. Everything took longer, from the signup to the marketplace to walking around the new hotel, which was of course a bit bigger than the previous one. But at the same time, the SoCraTes spirit was there just like the years before, as the seasoned participants carried it over while the newbies blended in naturally. We heard many positive voices and read enthusiastic blog posts. Our experiment had been a success, and so we did not think twice about continuing in this direction. In 2015, we increased the number of participants to 180, and next year we will try to cross the 200 mark. Although I will slowly pass the management of SoCraTes on to the next “generation” of organizers, I’m looking forward to seeing SoCraTes thrive together with its sibling conferences that sprout all over Europe.

Again, my thanks go to TIM Group and all our other sponsors for supporting SoCraTes and encouraging their employees to participate!

Oh, by the way, are you curious now? Please feel free to have a look at http://www.socrates-conference.de.

Report from DevOpsDays London 2013 Fall

This Monday and Tuesday a few of us went to DevOpsDays London 2013 Fall.

We asked for highlights from every attendant and this is what they had to say about the conference:

Francesco Gigli: Security, DevOps & OWASP

There was an interesting talk about security and DevOps and a follow up during one of the open sessions.
We discussed capturing security related work in user stories, or rather “Evil User Stories” and the use of anti-personas as a way to keep malicious users in mind.
OWASP, which I did not know before DevOpsDays, was also nominated: it is an organization “focused on improving the security of software”. One of the resources that they make available is the OWASP Top 10 of the most critical web application security flaws. Very good for awareness.

Tom Denley: Failure Friday

I was fascinated to hear about “Failure Fridays” from Doug Barth at PagerDuty.  They take an hour out each week to deliberately failover components that they believe to be resilient.  The aim is not to take down production, but to expose unexpected failure modes in a system that is designed to be highly available, and to verify the operation of the monitoring/alerting tools.  If production does go down, better that it happens during office hours, when staff are available to make fixes, and in the knowledge of exactly what event triggered the downtime.

Jeffrey Fredrick: Failure Friday

I am very interested in the Failure Fridays. We already do a Failure Analysis for our application where we identify what we believe would happen with different components failing. My plan is that we will use one of these sessions to record our expectations and then try manually failing those components in production to see if our expectations are correct!

Mehul Shah: Failure Fridays & The Network – The Next Frontier for Devops

I very much enjoyed the DevOpsDays. Apart from the fact that I won a HP Slate 7 in the HP free raffle, I drew comfort from the fact that ‘everyone’ is experiencing the same/similar problems to us and it was good to talk and share that stuff. It felt good to understand that we are not far from what most people are doing – emphasizing on strong DevOps communication and collaboration. I really enjoyed most of the morning talks in particular the Failure Fridays and the The Network – The Next Frontier for Devops – which was all about creating a logically centralized program to control the behaviour of an entire network. This will make networks easier to configure, manage and debug. We are doing some cool stuff here at TIM Group (at least from my stand point), but I am keen to see if we can toward this as a goal.

Waseem Taj: Alerting & What science tells us about information infrastructure

At the open space session on alerting, there was a good discussion on adding context to the alert. One of the attendee mentioned that each of the alert they get has a link to a page that describes the likely business impact of the alert (why we think it is worth getting someone out of the bed at 3am), a run book with typical steps to take and the escalation path. We have already started on the path of documenting how to respond to nagios alerts, I believe expanding it to include the perceived ‘business impact of the alert’ and integration with nagios will be most helpful in moments of crisis in the middle of night when the brain just does not want to cooperate.

The talk by Mark Burgress on ‘What science tells us about information infrastructure’ indeed had the intended impact on me, i.e. I will certainly be reading his new book on the subject.

You can find all the Talks and Ignites video on the DevOpsDays site.

Puppet Camp Barcelona

I recently had the pleasure of being asked to speak at Puppet Camp Barcelona. I’d submitted a talk a few months ago about some of the problems my team was having with our uses of puppet, and how we’re adapting to change how we use puppet.

I was extremely pleased to be asked to present, and also extremely pleased that TIM Group was willing to fund my flights and give me the time to attend the conference.

I was pleased by how the presentation went, and I gained a whole bunch of ideas we hadn’t thought of from chatting to people afterwards. From the official writeup, at https://puppetlabs.com/blog/puppet-camp-barcelona-wrap-up/ I think the talk was generally well received.

I’m looking forward to finding the time to write up further details of how we use puppet at TIM Group, and what problems this solves for us.

Monitorama

This is a blog post that was written in 2013, but somehow was forgotten about. So here is a bit of history!

— Andrew Parker


Last month I got the chance to attend the Monitorama conference.

This was out and out the best conference I’ve visited so far this year for learning. The conference was organised as a day of lectures by notable people in the field, followed by a day of workshops (practical ‘follow the talk on your laptop’ style sessions) on various monitoring and visualisation tools in parallel with a day of ‘Hackathon’ (working on projects).

The talks I attended covered some topics I’m already very familiar with (E.g. logstash), and some topics which I’m much less familiar with (E.g. the R workshop). The level of technical detail was generally great – not overfacing if you were a beginner, but also with something to learn if you’re a seasoned user.

In the second day, in the morning, I teamed up with Jason from TIM Group, and we were a little selfish – working on scratching our own itch. In the afternoon we broke off and attended some of the workshops, which were interesting. I was extremely surprised (and pleased!) that our project won the 3rd prize for a group project at the conference – that was unexpected given that we’d gone off and solved our own reporting issue, rather than tackling a more generic monitoring problem which would have helped a larger subset of people! We also managed to build something functional for our needs, and get it deployed into production within the 5 hours we had to work on it.

I hope that readers will forgive me if I spend a few paragraphs telling you about what we built (and why!):

We used to deploy Foreman, which we used to view / search through puppet run reports. Unfortunately, with our recent upgrade to Puppet 3.0, our Foreman installation needed upgrading too.

The version of Foreman we were running was ancient, and it’s since gained a massive number of features – however the only feature we were using from it was the report browser. This was causing us to have mysql on our monitoring machines, just to support this application, and re-packaging the latest version to our internal standards proved to be a non-trivial exercise.

We’d basically just disabled it to go ahead with the puppet 3.0 upgrade, with a plan to experiment with (try writing a proof of concept) using our logstash/Elasticsearch solution for the data transport and storage. I was able to very quickly hack up a reporting plugin for puppet, based off of some earlier work I’d found on github, and I’d been playing with the angularjs framework on the plane on the way over.

So, after about 5 hours hacking, we had bolted together Norman (excuse the bad pun).

This is, of course, still a simple and barely functional prototype; however it’s useable enough that after a couple more hours work we have unit tests (and green builds in Travis) and so that it was possible to deploy as an Elasticsearch plugin. There is still missing functionality from what we replaced in Foreman, however none of it is truely essential and we should be able to add that gradually as we have time.

Devopsdays London

This is a blog post that was written in 2013, but somehow was forgotten about. So here is a bit of history!

— Andrew Parker


Most of our Infrastructure team and a couple of developers we had seconded to the team all attended the Devopsdays London conference a couple of weeks ago.

There are a load of reviews/notes about the conference online already, however we also made a set.

I think everyone attending found the conference valuable, although for varying reasons (depending upon which sessions they had attended). Personally I found that the 2nd day was more valuable, with better talks and more interesting openspace sessions (that I attended). As I had expected (from my previous attendance at Devopsdays New York), I found the most value in networking and comparing the state of the art with what others are doing in automation / monitoring / etc.

I was very pleased that TIM Group is actually among the leading companies to have implemented devops practices. I’m well aware that what we’re doing is a long way away from perfect (as I deal with it 5 days a week), however it’s refreshing to find out that our practices are among the leaders, and that the issues we’re currently struggling with are relevant to many other people and teams.

I particularly enjoyed the discussion in the openspaces part of the conference about estimating and planning Infrastructure and Operations projects – at the time we were at the end of a large project, in which we’d tried a new planning process for the first time (and we had a number of reservations). The thoughts and ideas from the group helped us to shape our thinking about the problems we were trying to solve (both within the team, and by broadcasting progress information to the wider company).

Afterwards (in the last week) we have taken the time to step back and re-engineer our planning and estimation process. We’ve subsequently set off work on another couple of projects, with the modified planning and estimation process, and the initial feeling from the team is much more positive. Once we’ve completed the current projects and we’ve had a retrospective (and made more changes) I’ll be writing up the challenges that we’ve faced in estimating and how we’ve overcome them – as being able to deliver accurate and consistent estimates in the face of un-planned work (e.g. outages, hardware failures etc) is even more challenging than for operations projects than in an agile development organisation.

Software Craftsmanship ‘Round Table’ Meetings

Have you done something cool at work recently?  Or are you faced with a difficult technical problem or a team that just won’t gel?  Have you heard of a new technology buzzword and want to understand the implications?

Wouldn’t it be great to discuss it with a group of other software developers who care about what they do, from other industries and companies, who could bring different experience and fresh perspectives to bear?  Well, the London Software Craftsmanship Community is just such a group of developers, and the monthly round-table discussions (hosted at TIM Group) are that opportunity.

http://www.meetup.com/london-software-craftsmanship/events/98383322/

This Tuesday’s topics included ‘Is continuous deployment worth it?’, ‘How to test code with many collaborators’, and ‘Dealing with anti-ORM sentiment’.

When considering Continuous Deployment to production, it was surprising how quickly the group came up with so many different organisational impacts.  We need to understand who our customers are and what the true impact of any bugs would be on them.  The traditional QA role barely makes sense if the code goes through to production without any manual checks.  Partially-built features may need to be behind feature switches so the customers don’t see them until they are ready.  If the worst comes to the worst, we need to be able to roll back to a known good version very promptly, which means database schemas must be compatible between versions.

As the discussion progressed, it became very apparent that continuous deployment isn’t just a new deployment technique, but a whole new way of working, with implications for the entire company.  This may go some way to explaining why it is both so popular, yet so rarely implemented.

The diversity and strength of experience (and opinions!) that you will hear at the round table meetups are what makes them so interesting, frequently surprising and always worthwhile.

If you would like to join in, just join the London Software Craftsmanship group on meetup.com http://www.meetup.com/london-software-craftsmanship/ , and you will be emailed when each new event is scheduled (normally a couple of weeks in advance).

TIM Group Scala Dojo

TIM Group hosts a monthly Scala Dojo as part of the London Scala Users’ Group on Meetup (http://www.meetup.com/london-scala). The dojo is free to attend with drinks and pizza provided to the participants. The dojo always takes place on the third Thursday of the month in the TIM Group London offices.

The dojo is driven by participant in suggesting and choosing shorter programming challenges and collaboratively finding a solution in smaller groups. The atmosphere is relaxed with a focus on fun and learning. The range of scala skills range from total beginners that heard about scala to seasoned enthusiasts that work with scala. Towards the end of the meetup, the different groups present their work and discuss what they have learned.

Example of programming challenges are:

  • Secret Santa problem where the challenge is to find an algorithm for matching people giving each other gifts. No one should get left out and additionally no should be matched with their spouse.
  • finding an approximation of PI using concurrency with akka.
  • using the coursera Scala survey data to find alternatve stats such as ranking countries based on average perceived difficulty.
  • solving the Data Munging Kata from http://codekata.pragprog.com

I have taken my first steps in Scala in the dojo and found it a great learning experience. The time absolutely flies and you often end up with a great feeling of achievement by the end of the night!

The next Scala Dojo is on Thursday Jan 17th.
I hope to see you there!

TIMGroup is coming to Manchester!

Following on our success presenting Comic Collaboration and Communication at this year’s SPA conference, the nice people at the SPA invited to present again, this time at MiniSPA up in Manchester. A single-day conference on Monday October 3rd, it’s a taster of the best that the SPA Conference has to offer, and we’re honoured to have been chosen to present again.

Hopefully we’ll see some of you there!

Bring The Outside In at TIMGroup

Communication is a core principle of agile methodologies. It’s important for teams to disseminate information well to keep every individual in the team working effectively, and what’s true for the teams is also true for companies. As part of our dedication to learning and growing the best developers at TIMGroup, we actively seek to communicate outside of our own company with other individuals and organizations with whom we hope to exchange ideas. We’ve recently had speakers from Mozilla and Reevoo. We also do developer exchanges as another way to get some of the best ideas from outside of our company, into our company – in a developer exchange, one of us visits you for a day and works with your team, and then one of you returns the favour on another day.

So if you think you could do with a fresh infusion of ideas, and you don’t mind sharing yours, why not get in touch with us here at TIMGroup? We probably have something we can learn from each other.

Annual Stack Overflow Meetup Day @ youDevise London

Last week was the Annual Stack Overflow Meetup Day. youDevise was proud to host the London meetup for this worldwide event. Our dev teams work almost exclusively with open source tools and frameworks to deliver high-quality on-demand financial applications. Many of our coders are active contributors to Stack Overflow and all of us have benefited from the advice we find (and provide!) there.

Over 40 people filled up the 4th floor of our London City offices last Wednesday night. Mounds of pizza were enjoyed by all, and many brave souls tried their hands (actually their whole bodies!) at a few rounds on the Kinect we set up in the board room. We had a second big screen set up to watch the #SOMeetup-tagged tweets coming in from all over the globe as we joined in on the truly world-wide event.

It was a great way to give something back to a community that’s given a lot of help to us, and we had loads of fun doing it. Big thanks to everyone who made it out, it was great to get to meet you!