How To Reduce Your Time Spent On Data Quality Down From Three Weeks To Three Hours

How many times have you come up against the block of “that’s just how long things take here”?

How To Reduce Your Time Spent On Data Quality

As soon as a firm grows to a size where it needs enterprise IT architecture, changes – understandably – need to have greater scrutiny, and that takes time.

Time, however, is a double-edged sword. If a change takes too long to implement, the opportunity is missed, and more agile competitors can out-flank you. Secondly, failing to make that change can lead to a resigned acceptance in the company’s culture that actively stifles innovation.

Many of these problems arise when firms think about change where all the budgets are located. It stands to reason to think that those big changes mean big budgets, but try turning this on its head: look at where the changes are most needed. This is the approach that British Cycling refers to as “marginal gains” – looking at any change that can give any margin of improvement and getting it implemented.


Starting small doesn’t mean lacking ambition.

I can see my data from here!

Thinking big – winning Olympic gold – was the motivator for pursuing change that matters. Yet British Cycling didn’t turn that big ambition into a series of big infrastructure projects, like fleets of support vehicles or several new velodromes.

“By analyzing the mechanics’ area in the team truck, we discovered that dust was accumulating on the floor, undermining bike maintenance. So we painted the floor white, in order to spot any impurities. We hired a surgeon to teach our athletes about proper hand-washing so as to avoid illnesses during the competition (we also decided not to shake any hands during the Olympics). We were precise about food preparation. We brought our own mattresses and pillows so our athletes could sleep in the same posture every night.

Sir Dave Brailsford, in Harvard Business Review

hourglass
Don’t fear time, use it!

None of these changes represents a massive investment in infrastructure, but they’ve led from one medal in 76 years to multiple Olympic golds. The same approach is being taken right now by leading financial services firms. In a notable case, one London-based financial services firm did exactly this in their COVID response, reducing the usual time to configure rules and query the data from three weeks to three hours.

Like them, successful data leaders are focusing on the pain points of data quality right where they’re most keenly felt.

  • In the teams who are trying to submit reports to regulators;
  • By those who are responding to immediate Operational Risk issues, like the impact of COVID;
  • By analytics people who are struggling to gain an edge on poor data.

In each case, they’re identifying a specific problem in one area, for example, client data used in customer communications, choosing off-the-shelf tooling that can be implemented rapidly and then simply getting on with it.

Brailsford’s team invested in paint and the time to actually paint the workshops. Data management heads are investing small sums in business user-friendly tools on one subset of data so that they can quickly achieve successes without having to overhaul the entire infrastructure of the organisation.

This kind of work means that practically speaking it is no longer a pipe-dream to reduce data quality processes that can take up many weeks down to a matter of hours. It stands to reason that by iteratively implementing these processes, in short, clearly defined sprints, the same processes that drove British Cycling to multiple golds, can yield similar rapid benefits the bigger picture of data quality at financial services firms worldwide.

What about you? Have you adopted a “marginal gains” approach to work that’s paying dividends?

Where can Datactics help you on your data management journey? We know your time is precious so if you’d like to find out how we can help deliver self-service data quality to your business, simply book 15 minutes of Kieran’s time by choosing a slot below.

To learn more about how how Data Quality fits into modern Data Ops strategies, catch our webinar as part of 2020’s Data Management Insight event (or read a blog post version here).

Get ADQ 1.4 today!

With Snowflake connectivity, SQL rule wizard and the ability to bulk assign data quality breaks, ADQ 1.4 makes end-to-end data quality management even easier.