2015 London tmTechEvent – Ticketmaster’s in-house tech conference

hotel

“Welcome to the tmTechEvent 2015,” John McIntyre, head of PMO at Ticketmaster International announced as he scanned my badge. The scanner beeped appreciatively as it recognised my credentials and I was granted entry – just like going to the O2! Apart from the fact that this was our annual Ticketmaster technology summit meeting in a conference room just up the road from our offices at the edge of London’s Silicon Roundabout. But this wasn’t any old conference. Being in the live entertainment space, we had quizzes (using Kahoot.it), a live Twitter feed (@TMTechEvent) and a party in our London HQ’s basement bar complete with gourmet burger van!

As I entered the room there was a palpable sense of tension and nervous excitement in the air as I greeted my colleagues from the Sports division: with the Rugby World Cup starting on Friday, their systems would be under the spotlight – or was it just a matter of doubt over England’s ability to deal with a tricky opening tie with Fiji? Ultimately both fears proved unfounded – all the events ran smoothly and England prevailed.

Along with leaders from all of Ticketmaster’s other technology teams, we had joined together to take stock of our progress in revamping our technology real estate. The 4 day event was packed full of seminars, workshops and group sessions, with the overall aim of evaluating our strategies and determining where to correct our course. We followed the guiding principle of “focus where you want to go, not what you fear.”

wheretogoquote

We pulled in leaders from all over the business to help shape our vision of where we’re going. We combined this with focused feedback sessions on the various aspects of our strategy to determine the best way to pivot to adapt to the changing landscape.  Using workshops to facilitate a rich exchange of ideas we covered subjects from staff satisfaction to talent management to deeper technical subject matter such as engineering KPIs and creating reference architectures.

Other highlights included live demos of in-house tools, including one that had been created to show the visibility of progress in our DevOps program. It was really cool to see each team’s progress in one place across our home grown four- part maturity model. Even better was sharing the whole event with colleagues across Engineering and Operations and feeling a real sense of unity, proving that good DevOps is about culture change and not just a bunch of new processes!

conference 2015

The sheer scope and depth of material covered was brain bending. We rolled out our new career mapping program, providing a structured career map and promotion process across all of our engineering teams. We had a thorough review of the initial results of our innovative and much talked about technical debt management program that we rolled out at the start of the year. We reviewed the progress being made on our employee feedback survey, to ensure that the concerns of our engineers are being taken seriously (it’s not just about having more opportunity to play ping pong!)

Overall this was an inspiring event. There was a tangible confidence and will to achieve our ambitions, based on a very real sense of achievement from how much we had already changed things for the better in Ticketmaster Engineering. The desire to increase collaboration and tackle the bigger challenges ahead together was strong.

Having reset our sights on the vision of the organisation, our technology vision and our engineering vision, the result was a noticeably energised and motivated group of rock-star tech leaders ready to take that vision back out to their teams and the company. The future of better live entertainment starts here!

In Search of a Reactive Framework (or: How we select new technologies)

About seven or eight months ago we started looking at new and improved ways of creating services instead of our tried and tested Spring framework based approaches. The Spring framework certainly has its merits and we have used it with much success, however, the service we were about to write, called Gateway, was to route millions of requests to other services further down in our architecture layers:

GatewayDiagram_v1

We knew that the service had to run efficiently (thereby saving on hardware) and scale effectively. The event driven, reactive approach was one that we were looking to embrace. After much research we had some concerns. Not only is that space full of different frameworks at different levels of maturity, but we also had to consider our current skill set, which is predominantly Java with a small amount of Scala, Ruby, Clojure (which our data science guys use) and a handful of other languages we’ve picked up through company acquisitions. How could we adopt this new paradigm in the easiest possible way?

What this blog post will detail is the approach we used to select the framework we chose. It’s a tried and tested approach we’ve used before and will continue to use and improve upon in the future.

How we did it

Before we describe the stages of what we did, suffice to say that there is no point in doing a technology selection without a business context and an idea of the service(s) a framework will be used to build.

The technology selection was broken up into the following steps:

  • Identify principles – these are the rules that the framework must adhere to. These are properties that a framework can meet in different ways and contain a degree of flexibility
  • Identify constraints – these are rules that cannot be broken or deviated from in any way. If any are broken, the framework is no longer a candidate.
  • Create a short list of 5 – 10 candidate frameworks
  • Determine high-level requirements and rank using MoSCoW prioritisation
    • Read the documentation, Google groups and other relevant articles and trend data to determine conformance to the requirements, including all options and workarounds if applicable.
    • If any of the Musts are broken by a candidate then it is dropped
    • Create a short list of, ideally, three candidates
  • Create a second set of more detailed requirements, rank using MoSCoW and weight in terms of importance
  • Determine the architecturally significant business stories and error scenarios that the service to be built from the framework needs to implement:
    • Write the end-to-end acceptance tests for these stories. The primary scenario with one or two error scenarios is sufficient.
    • Implement these end to end acceptance tests in all three frameworks – this will give an idea of how well the framework meets the service’s paradigm(s), how easy it is to work with in the develop/test cycle and also make sure to post on the message boards or mailing lists to see how quickly a response arrives from the framework maintainers.
    • Update the second set of more detailed requirements with the results of this experience

Our results are pretty detailed so have been added into a separate PDF that you can download here. We have left this in raw format and hope that they will be a good reference for others.

Outcome and experiences to date

As you can see from the PDF of results, we chose Vertx. It won out not only because of its raw power, but because of it’s fantastic architecture, implementation, ease of use, Google Groups support and the fact that Red Hat employs a small team to develop and maintain it. Indeed, a few weeks after we selected it, it was announced that Red Hat hired two more engineers to work on Vertx.

So overall we have been very happy with our selection of Vertx. We had version 2.1.5 running in production for several months and recently upgrade to Vertx 3. The maintainers’ swift response on the Vertx Google Group definitely helped during our initial development phase and during the upgrade to version 3. Performance wise, the framework is extremely fast and we know that any slow down is most likely due to what we have implemented. Adoption has been a success. From a team of two developers, we scaled to four and now eight. Choosing a Java based framework has been a boon as the only additional complexity that needed to be learned by the developers joining the team was the event driven nature of Vertx (i.e. the framework itself). Had we chosen Scala/Play it would have been much harder. Indeed, with the success of Vertx, our decision to standardise on the JVM as a platform and our embracing of the reactive approach, we have a couple of services being built using Scala and one using Scala/Play. It would be great to hear of your experiences using reactive frameworks. Which ones did you choose? How easy were they to adopt? Please leave a comment, below.