Shipping High Quality Mobile Apps

I want to share a story about a recent career transition I made – moving from program management to quality engineering . I was very nervous about the change since I wasn’t 100% sure whether the decision I made was the right one. There was a huge opportunity on the table to deliver more value to our fans more often. The risk of course was that quality would slip.

Read on to hear how we became featured in the Google Play Store and in Android Central as “AC editors’ Apps of the Week”!

Joining the Android Team

As I joined the passionate team of Android Engineers I knew it was a new beginning of something great and challenging for all of us. As a team we quickly built a strong app foundation and developed a culture of teamwork and support for one another. Our fundamental rule was: check your ego at the door! Everyone on the team supported each other like family. We had a few goals in mind:

  • Reduce Regression time from 2-3 weeks to 8 hrs
  • Build once and ship everywhere
  • Ship high quality software together

In this post, I want to share my personal experience as well as how my team was able to set up our automation framework.

Why Automation?

It all started with a few automation training sessions with our Quality Architect – Eric Lee – in a small conference room on the 2nd floor of our building. As I remember the room was packed with eager QA engineers wanting to learn about making automation work for our mobile apps. Eric demoed sample tests that navigated through a few pages of our app utilizing Cucumber, Appium and Java. When we came out of the training session we realized how we could utilize type of tests to help us meet our main goal “Reduce regression time from 2-3 weeks to 8 hrs”.

Regression test
One of our regression tests

We used our 10% time to setup a small device lab. We started with one device since we came across some challenges running the Ticketmaster Android app across multiple devices due to our Appium configuration. We made an update to our configuration assigning each instance of Appium a unique port as well as assigning a unique port to each of the devices.

Config Changes
Screen Shot 2015-10-23 at 12.06.12 PM

After making these changes, we’re able to execute our test suite across four different devices simultaneously. Our regression tests run overnight and detect any ongoing problems while development is in progress. The team gets a detailed report of what failed in our automation tests bringing everyone aware of any issues that may have been introduced during development. This helps our team bring focus on critical release regressions and allows us to collaborate with one another to come up with the highest quality app.

Android Build pipeline
AndroidBuildpipeline

Automation report
Automationreport

In 2014 having no automation coverage we released our apps only 4 times. On the other hand, this year we have published more than 19 versions of our TM app to the market place. What a difference a year makes! To date, our automation coverage is up to 63% focusing on our critical path of our app. We are still not done and continuing to add more features into our automation suite to get us closer to our goal in reducing our regression time to 8 hrs. With automation in place, we are equipped to have more frequent releases to the marketplace bringing key features to our fans quicker.

Detailed automation report

Next up for the android team is to publish our newly designed app across all international markets. This will give our team a more maintainable code base supporting all regions. As you can imagine, our automation is critical to supporting more markets. The ability to run the same tests across the different binaries for each region and get detailed test reports for very little additional effort is invaluable. Our monthly releases going forward will support a total of 5 markets – build once and ship everywhere!

Culture Change

Shipping High Quality Mobile Apps – Our Android team has the “do it and let’s do it together” attitude that makes me want to come to work every day. During the past few months, our android team have made significant improvements to how we managed our releases and our sprint schedules. With our project and product manager’s help organizing our stories, our grooming sessions have become more engaging and interactive. That helps our team bring focus to the features that we will be developing. Our demos and retrospectives have improved as well – our engineers showcase their work to the stakeholders and other teams, giving each engineer ownership of the work they completed. One of our stakeholders stated this after one of our demos:

“This is a TEAM!

Well done guys. Your pride shows in your work, and it also showed in the demos you gave at the last showcase. I am inspired and impressed. You have improved over the past couple of months, and that shows in this latest release of your app.”

Our quality and software engineers are very passionate about delivering the highest quality features to our fans. For each and every submission and release, the team’s ownership and responsibility is second to none. Whether it’s looking at the reviews on the Google Play store, our crash-free rate, daily active users – the team has great pride and commitment to deliver a high quality app.

Being featured on Google Play Store and Android Central as “AC editors’ Apps of the Week” was a great achievement thanks to the hard work and dedication of the team. We’re continuing to make changes every day to help our team be more efficient. A great example of this is our recent decision to include an app release within each sprint, inclusive of the sprint’s work (as opposed to releasing in the subsequent sprint). Our Android team has become like a family. We enjoy working with each other and we build software in a cohesive way. We are a dedicated team and committed to a mission to be better each day when we come to work at Ticketmaster.

References:

Jenkins: https://jenkins-ci.org/

Appium: http://appium.io/

Selenium: http://www.seleniumhq.org/projects/grid/

Cucumber: https://cucumber.io/docs