Interview with our Test Manager of the Year

isabelle

Isabelle Magnusson leads the Quality Assurance team in Ticketmaster’s Gothenburg office in Sweden, and recently won Test Manager of the Year at the 2016 European Software Testing Awards. We caught up with her to find out a bit more about her success:isabelle

  1.       How does winning this award make you feel?

I feel very proud and a bit surprised to be honest.

  1.       What is it that you love about being a test manager at Ticketmaster?

I’ve been given a lot of freedom to do what I believe in from the start. My team is always striving for improvement, but will still keep challenging new processes. I have a team of testers with very different backgrounds that makes my role as manager very attractive.

  1.       How long have you been part of the Ticketmaster family?

5 years

  1.       How has Quality Assurance and software testing in general changed from when you first started?

I started my career as a trainee at a consultant company where I practiced all kind of test techniques and processes in a variety of organisations. Back then the most important part of my CV would be the ISTQB certificate. Today certificates like that are not very attractive.

  1.       What does the future of QA and software testing look like from your point of view?

I don’t believe in QA “departments”. The tester is a part of the development team and the quality assurance is not not just an activity for the tester, but rather a whole team approach. The tester in the team is still the test expert and can coach and drive the test strategy though. The opinions regarding manual and automated testing differs a bit. I believe we will not be able to automate everything but the role of the manual tester will not just be about following step by step test cases. Instead it will focus on validation from the start of a project.  

  1.       What does a day in the life of Isabelle Magnusson look like?

I scroll through Slack and mails from bed. The benefits of working with several time-zones, someone is always working, right?  After leaving my daughter, Juni at preschool, I take the bus to the office where I have my breakfast while catching up with the team. I spend quite some time coordinating bugs reported from the markets and aligning with product managers and project managers. I’m coaching the testers in their job in the development process, representing QA in different contexts. I still like to be a part of the development and step in where we are low in resources when it’s needed. We are currently recruiting a whole new team so I also spend a lot of time in interviews and reviewing CVs. After office hours I try to spend as much  time as possible with my family.

  1.       Please describe what you feel is your biggest career achievement to date:

Being a part of moving the agile process at Ticketmaster forward have been a great journey. I’m really proud of how far we as a team have come.

  1.       Please describe the vibe in the Ticketmaster Gothenburg office:

It’s a very friendly atmosphere; I consider my colleagues my extended family. We have a lot of fun and we work hard to achieve what we have committed to.

  1.       What’s it like to work at Ticketmaster?

It’s a great place to work. Being a global company the opportunities are endless.

  1.   What do you love about working at Ticketmaster?

It is a large organisation but I still feel that each individual counts. Ticketmaster invests a lot in people. I love having the freedom to do what I think is best and the responsibility that comes with it.

  1.   You recently participated in the Ticketmaster International Hackathon. How was the experience?

It was a lot of fun. I was collaborated with colleagues from Sweden and Canada. It was a great opportunity to innovate with new features and methods.

  1.   What was the first live event you went to?

Tracy Chapman

  1.   If you could go to any live event, which one would it be?

I love Christmas. I’ve been listening to Christmas songs since early November and my favourite artist is currently “She and him”. I would love to see a small, intimate live event where they perform songs from their Christmas albums.

  1.   What’s your passion?

Except testing? I’m a foodie. I love to find new restaurants and have their tasting menu. I plan my vacations based on what restaurant I want to try out. Unfortunately I’m not a very frequent customer since I had kids, so currently I have to settle with experimenting in my own kitchen. I’m planning for my next stop to be Fävikin Magasinet in Åre though.

  1.   What’s one thing that no one at work knows about you?

Hmm, I think they know most of me, I’m like an open book.

What Ticketmaster is doing about professionalising software careers

“My wife is doing her accountancy exams so is busy tonight, so I’m staying in to look after the kids” is something you expect to hear in every day conversation. But what about “My partner is doing his software engineering training tonight so I’m unable to come out for a beer?” Why is this still unusual? Are software developers still a product of their bedroom and dorm room past, hacking things together and so often saying ‘yes’ when no other professional would in a similar situation?

Software development, let’s face it, is an immature industry. We build software, but not many of our parents generation did. What we often fail to realise is the number of different roles we play. When we started out in software, we most likely will have felt the excitement of building something, and seeing it work. But to get it to work properly, we likely had to do some testing. As our initial projects mushroom, we start to learn various constraints – so without realising it, we start thinking about the design. Then when we have something that other people start using, they might have found a problem with it. So we need to do some investigative work. Then we find that there are additional requirements which can’t be met by the current design – so we start to think about the architecture.

What we haven’t realised is that we’re doing all the parts of what is known as the Software Development Lifecycle (SDLC): Requirements, Design, Development, Testing & Support.

Let’s think about other roles we play in our life. We may be parents, sons & daughters, friends and neighbours. But what about surfers, fans, climbers, shoppers and garage organisers? What we learn from looking at our lives that way, is that if we don’t plan for these roles, or put any time into them, then there are most likely negative consequences. If as a parent, we put no time or effort into parenting, or trying to be better as a parent, then that’s likely going to have a detrimental effect on us and our children. If as a garage organiser we put no time or effort planning or being a garage organiser, then our garage will remain a mess, and it’ll be impossible to find things when we need them.

sdlc

So the same goes for us as software engineers. We need to plan, prepare and learn how to master the different roles we play, or there will be negative consequences. Software support is often seen as less glamorous than developing, but as engineers it requires sharpening our detective skills to be able to find a metaphorical needle in a haystack. This requires a completely different set of skills to development. So in order to take ourselves more professionally, we need to gain an awareness of these different roles, and gain maturity in developing our skills in these different areas.

Here at Ticketmaster we’ve been Agile for over 6 years now, and we’ve come to the realisation that with that level of maturity, multi-disciplined skills are required in order for development teams to continue to grow in a healthy way. When we reviewed our career progression processes across multiple engineering locations, we also found a lack of consistency across job roles and criteria for career progression. We came to the conclusion that without a transparent process that was consistent across all our teams, it was hard to set expectations and provide our engineers with the optimum experience for growing how they wanted to in their chosen profession.

So what we’ve done at Ticketmaster to address these issues is create a career map – a flexible set of pathways built from what we do as engineers day to day, but that provides both a range of options for career progression along with outline steps of how to get there. This provides mastery of any chosen career progression route, with a clear vision of the outcome that you’re looking for as an engineer.

How did we do this? We used a number of industry standard resources, building on the SWECOM model created by the IEEE, and using influences from other industry models such as SFIA. This has enabled us to build a three part progression guide:

progression guide

  1. SDLC skills – these are built up from the SWECOM model and outline 5 maturity steps in each professional skill, built from the SDLC.
  2. Technical skills – these have been built up using a platform and technology neutral approach, again providing a 5 step program of technical skill for all areas pertinent to Ticketmaster’s engineering strategy.
  3. Behavioural skills – we have outlined four areas for awareness and growing maturity that we believe make the biggest difference to our success as a people first organisation: Team work, Innovation, Results orientation, Professionalism. We can’t and don’t want to mandate behaviour, so we provide lots of examples of what maturity looks like at each level.

So armed with a clear map of what is required to progress, engineers are empowered to make this journey with the support of their colleagues. How do we do that? We used the guiding principles from Drive by Daniel Pink.

Autonomy

It was really important to us that we avoid this scenario, which you may be familiar with:dt_c131214

Again, we considered the engineering profession from the viewpoint of different roles, and saw the role of the manager as a coach, like Boris Becker working with Novak Djokovic. We want to create an environment in which engineers can excel, rather than be micro-managed or stressed by unnecessary distractions. So we trained all our managers to operate with a coaching model in mind at all times, supporting and only intervening from the sidelines when required.

Mastery

We’ve empowered our engineering teams by giving them the tools and resources required to progress their careers – and them giving them responsibility for progression. The progression guide makes it clear what is required to progress to the next role. The guide was put together through numerous rounds of consultation, to ensure everyone had a say in how it was constructed. In order to satisfy the requirements of the next role, we have introduced a peer review and assessment process, whereby individuals submit their evidence to show they have mastered the professional, technical and behavioural skills to the appropriate level. What we learnt from engineering led organisations such as Google as well as other industries is not to have a single peer reviewing, but to have a minimum of three to ensure bias is minimised and the process is as objective, fair and transparent as possible.

We’ve rolled out training across all our engineering teams on how the career map process works, and run workshops on how to create suitable objectives in order to reach the requirements for each role. At each stage we have looked to put responsibility for progression with the engineer, but provided suitable support structures to enable and facilitate progression. This includes an online tool specifically created to help manage progress, providing a simple view for instantly seeing what steps are required and a convenient way to store evidence.

Purpose

Software engineering is constantly exploring unchartered territory. That’s why agile has won out over waterfall as a methodology because iterative exploration and learning is almost always better than educated guesswork. Like software projects, software careers are also subject to a high level of flux because of the nature of the industry. Established skills can be rendered redundant overnight by the introduction of a new technology, product or service (e.g. the iPhone, or Uber). By providing engineers with multiple options in terms of charting their career path, we believe that we can avoid exposure to the threat of redundancy of specific technologies by building opportunities for career breadth and depth.

This enables engineers to not only change as the industry does, but also to facilitate proactivity rather than being constantly reactionary, which is far more stressful. Undue stress does not contribute to a healthy working environment, which is what we’re constantly striving for.

But we’re also looking to professionalise our industry, and make software engineering an even more exciting destination than it was a generation ago, when it was really in its infancy. By building our approach on existing standards and the best disciplines from other industries, we’re aiming to show that we’re part of an extremely vibrant but also professional industry which will endure for millennia to come.

Ticketmaster win big at the Techworld Techie awards

techies

A huge night for Ticketmaster last night taking four prizes at the Techworld Techie awards! The Ticketmaster tech team were a very happy bunch taking the award for Most Innovative Team, Rockstar Developer of the Year, Best Place for Developers to Work, and the big one…the Techies 2016 Grand Prix award.

“This is great recognition for great people working for a great company” said Gerry McDonnell, SVP of Technology for Ticketmaster International who was especially proud of his team and their achievements.

On taking the prize for most innovative team, Adam Gustavsson said “Our team story is about a journey, a journey that started over four years ago in an attempt to answer a simple question: how can we move faster? That journey has taken us from a team that released once or twice a year to one that releases every other week, with even shorter cycles in sight. But more importantly, it is a story demonstrating that seismic change can be driven from the bottom up, even within large organisations.”

Nicolo Taddei was overjoyed on receiving the award for Rockstar Developer of the Year, “I personally believe that the biggest impact didn’t come directly from my software, but more from my personality on the team of engineers within Ticketmaster. With a positive mind set I enabled myself and my colleagues to take more action and calculated risks to improve our developer experience, code quality and innovation which have a real and positive impact on our clients and customers.”

We’re super proud of winning the best place for developers to work too. We love live entertainment, we live and breathe it. Everyone at Ticketmaster is a fan, hence our mission to bring out the fan in all! The move to our new offices in Angel three years ago came as part of a pivot in our company strategy, reinventing ourselves as a tech leader and the owner of the largest ticketing ecosystem in the world. “We look at ourselves as a technology company. A few years ago we didn’t do that. We had to redefine ourselves culturally”, Mark Yovich, President of Ticketmaster International said. With this award, it’s fair to say that this transformation is complete, as we now look to build on the solid base we have to provide fans and clients with the best possible experiences.

A big thanks to everyone at Techworld for the recognition and organising such a fun and inspiring evening, and hats off to Ticketmaster for sweeping the Techies 2016!

2015 London tmTechEvent – Ticketmaster’s in-house tech conference

hotel

“Welcome to the tmTechEvent 2015,” John McIntyre, head of PMO at Ticketmaster International announced as he scanned my badge. The scanner beeped appreciatively as it recognised my credentials and I was granted entry – just like going to the O2! Apart from the fact that this was our annual Ticketmaster technology summit meeting in a conference room just up the road from our offices at the edge of London’s Silicon Roundabout. But this wasn’t any old conference. Being in the live entertainment space, we had quizzes (using Kahoot.it), a live Twitter feed (@TMTechEvent) and a party in our London HQ’s basement bar complete with gourmet burger van!

As I entered the room there was a palpable sense of tension and nervous excitement in the air as I greeted my colleagues from the Sports division: with the Rugby World Cup starting on Friday, their systems would be under the spotlight – or was it just a matter of doubt over England’s ability to deal with a tricky opening tie with Fiji? Ultimately both fears proved unfounded – all the events ran smoothly and England prevailed.

Along with leaders from all of Ticketmaster’s other technology teams, we had joined together to take stock of our progress in revamping our technology real estate. The 4 day event was packed full of seminars, workshops and group sessions, with the overall aim of evaluating our strategies and determining where to correct our course. We followed the guiding principle of “focus where you want to go, not what you fear.”

wheretogoquote

We pulled in leaders from all over the business to help shape our vision of where we’re going. We combined this with focused feedback sessions on the various aspects of our strategy to determine the best way to pivot to adapt to the changing landscape.  Using workshops to facilitate a rich exchange of ideas we covered subjects from staff satisfaction to talent management to deeper technical subject matter such as engineering KPIs and creating reference architectures.

Other highlights included live demos of in-house tools, including one that had been created to show the visibility of progress in our DevOps program. It was really cool to see each team’s progress in one place across our home grown four- part maturity model. Even better was sharing the whole event with colleagues across Engineering and Operations and feeling a real sense of unity, proving that good DevOps is about culture change and not just a bunch of new processes!

conference 2015

The sheer scope and depth of material covered was brain bending. We rolled out our new career mapping program, providing a structured career map and promotion process across all of our engineering teams. We had a thorough review of the initial results of our innovative and much talked about technical debt management program that we rolled out at the start of the year. We reviewed the progress being made on our employee feedback survey, to ensure that the concerns of our engineers are being taken seriously (it’s not just about having more opportunity to play ping pong!)

Overall this was an inspiring event. There was a tangible confidence and will to achieve our ambitions, based on a very real sense of achievement from how much we had already changed things for the better in Ticketmaster Engineering. The desire to increase collaboration and tackle the bigger challenges ahead together was strong.

Having reset our sights on the vision of the organisation, our technology vision and our engineering vision, the result was a noticeably energised and motivated group of rock-star tech leaders ready to take that vision back out to their teams and the company. The future of better live entertainment starts here!

What Ticketmaster is doing about technical debt

This post describes the journey Ticketmaster has been on over the last year to define and measure technical debt in its software platforms. The issue kept surfacing from multiple sources, and yet as an engineering organisation we had no consensus on how to define technical debt, yet alone measure it or manage it. So we embarked on a journey to answer these questions, and gain agreement across the engineering organisations in order to effectively provide a common approach to solving the problem.

We started with research and found that technical debt is all around us:

fowler-on-debt

A chilling example is of Knight Capital who ran updates on their systems that reactivated some dead code, causing the system to spit out incorrect trades – losing $460 million in under an hour (Source).

Ultimately debt management is a business decision – so how do we as IT professionals source and present the right data to influence the decision makers? Part of articulating the size of the problem was to compare the size of the Ticketmaster codebase to other codebases of a similar size:

tm-tech-debt

Over 4.5 million lines of code are spread across 13 different platforms, from legacy to greenfield, across a whole mix of technology stacks including different flavours of .Net and LAMP. We formed a working group with members from different platforms and locations in order to build a model that would work across all these boundaries, and would have the buy in from all areas. Our research can be summarised as follows:

key-findings

We used these 3 different areas as the top level of categorisation for the following reasons:

Application Debt – Debt that resides in the software package; unchecked build up can lead to:

  • Inflexibility and much harder to modify existing features or add new ones
  • Poor user experience
  • Costly maintenance and servicing

Infrastructure Debt – Debt that resides in the operating environments; unchecked build up can lead to:

  • Exposure to security threats and compliance violations (e.g. PCI compliance)
  • Inability to scale and long queuing times for customers
  • Poor response and recovery times to outages
  • Costly maintenance and servicing

Architecture Debt – Debt that resides in the design of the entire system; unchecked build up can lead to:

  • Software platforms that are highly inflexible and costly to maintain and change
  • Flawed design that can’t meet the requirements of the business
  • Single points of failure which cause outages or exacerbate them
  • Unnecessarily complex systems that can’t be adequately managed
  • This gives opportunities for more nimble rivals to gain competitive advantage through technology

Model

Having selected the areas to measure, we needed a model that could be applied across the huge range of technologies used throughout TM’s platforms. Testing both automated and manual processes with various teams and tools helped to refine the model so it could be applied consistently:

tech-debt-model1

We aimed to automate the collection of as many of the metrics as possible via automated tooling to make the process repeatable:

  • Application Debt:
  • Infrastructure Debt:

Manual Mapping

Where automated tooling hasn’t existed to measure the metrics we identified, a manual process has been introduced to help measure the intangible. We used the following guiding principle:

measure-anything

It was clear that a lot of what was known about the limitations of a platform or component wasn’t being reported by tooling, but was readily available in the minds of engineers. We just needed to figure out a consistent, repeatable and transparent process for extracting that data and making it useful. Out of this need was born the technical debt mapping process:

mapping

Reporting: Telling the story

At this point, we’re left with lots of data. One of the core driving philosophies behind the whole process was to present pertinent information to enable executive level decisions to be made as to how to manage debt from a strategic point of view.

need-to-know

For example, if debt remains stubbornly high for a critical application or area and is reducing developer throughput to a crawl, then a strategy of downing tools to address the debt may be the best option – but this is likely only to be able to be made at executive level. Executives only really require the most pertinent information, but if required the data behind the summary would also need to be readily available. The report is divided into three parts:

parta

Part A (above) contains a rolled up score for each of the three main debt categories for each of the systems being reported on.  It also contains a summary of the debt in the platform which have been contributed by the Engineering, Tech-ops and Architecture groups.

partb

Part B (above) contains a more detailed breakdown of each main category’s definitions for each of the systems being reported on.  This gives the reader a better insight into which definitions have higher or lower debt levels and an indication where work needs prioritising.  It also shows the next items to be worked on.

partc

Part C (above) contains the longer term technical debt backlog of for each of the systems being reported on, broken down by category. There is no indication of time for each item but some could span months or even years.  This section is aimed more towards the Engineering and Architecture teams.

interpret-report

What do we do with the output?

Update Technical backlogs

  • Updated with new debt items as they are identified during the mapping processes
  • Technical debt items prioritised according to criticality, not level of debt – some components may contain a lot of debt but are stable, or no longer strategic

Update Product Roadmaps

  • Selected technical debt items prioritised against product roadmap items
  • Product teams need to be bought into the importance of maintenance work
  • Value needs to be clearly defined and communicated, in order to make the right strategic decisions for scheduling the maintenance work and managing the debt.

What next?

Management of technical debt is more than just about identifying and scheduling maintenance work. With the plan to issue the report quarterly, the intention is also that the visibility the report provides, plus the tooling provided to engineering teams will help to stem and reduce the introduction of technical debt. By adhering to industry best practices, and being conscious of the implications of introducing further debt, engineering teams can take steps to build quality into the products and platforms as they go. The debt that is introduced as projects progress can therefore be minimised, and product and engineering teams can make more informed decisions about how and when to pay debt down.

Ultimately the goal of each of us is to delight our fans. Running well maintained systems will benefit them with better stability, better response times and ultimately faster delivery of features as debt is brought under control, managed and interest rates reduced to a sensible level. Debt management will benefit the whole business in a similar way – less firefighting, fewer outages and platforms that are easier to develop. Ultimately, we all stand to gain.