4 min read

Data Normalization at OMG

Jul 7, 2021

OMG Engineering Greenfield Blog Images - Data Normalization

Big Data is one of the most popular buzzwords and truth is, we generate a lot of data. That said, data does us no good unless it contains meaning. Data is power.

Problem is, the data can be as varied and diverse as the people using it, leading to messy data and ambiguity in insights. A majority self-serve platform like OrderMyGear can easily lead to messy data when one person types “Spirit Wear”, another types “spirit gear”, yet another types “Spiritwear”, and unfortunately another types “spriit wear”.
We place great importance on making sense and deriving meaning from this varied data, and as a growing technology platform looking to improve customer experience and expand to new markets, it was essential to find a solution. That’s where data normalization comes in.

OMG Teams utilize Looker as our data engine. Looker actions gave our Team the power to build a data normalization tool to enhance our process and make sense of the data our platform collected.

Why Care About Data Normalization?

Data normalization is the art and science of reorganizing data in a way that allows users to utilize it for effective analysis and insight. It puts rules in place to help streamline the data and keep it clean.

OMG relies on accurate data to improve not only our own business, but the business of our clients as well. Without normalizing the data we obtain, we cannot accurately understand the way our customers actually use our software. Extending to our clients, we want accurate insights to share with them so they can make informed decisions that will enable them to sell more on the OMG platform.

data normalization

Figure 1: A simple normalization strategy is to convert disparate yet similar data into a single agreed upon value. That allows us to make clearer generalizations about how our customers use our platform.

 

Learn From Our Mistake

As a self-serve online store building platform, every OMG store is labeled with an activity that expresses the purpose of the store. Serving contrasting industries with vastly different client types and servicing nuances can lead to many different store activities - everything from “healthcare” to “spirit wear” to “freshman volleyball” to “employee apparel” and many more.

When the OMG platform was first being built over a decade ago, we opted to make activity names a free-form field. It’s a mistake we continue to pay for. Typical activity field naming issues include a simple misspelling or spelling variation. For example, Baseball could be input as anything from “Base ball” to “Basebal” to “BASEBALL”. The more unfavorable naming issues seem to be a complete misuse of the activity field by inputting something incredibly specific like “2019 Football, West High Football, JV Football Team Gear”.

This variety (in the activity field alone!) proved to make our data-driven endeavors near impossible, as consistent and actionable information was difficult to come by - or even trust.


Approaching Data Normalization

Understanding the necessity of getting our data under control, the OMG Team strategized and implemented a plan to normalize our data. This took distinct action from Teams across the company - Engineering, Sales, JumpStart, Support, Account Management, Revenue Operations, and more. The OMG Team started by creating a Gold Master Activity List informed by the activities that our customers used most frequently. We then moved to defining and implementing normalization techniques.

The Gold Master Activity List defines the most valuable and common activities to track, based on total Gross Merchandise Volume (GMV) through the platform. If the activity doesn’t do significant GMV through the platform, it is rolled-up into a higher activity category. For example, the “School Club” activity may contain smaller or lesser-used activities like Debate Team, Chess Club, or Spirit Group. The Gold Master Activity List is continually reviewed, and once a particular activity meets our GMV expectations, they can warrant a breakout group of their own on the list.

 

Three different normalization techniques were outlined and implemented across our Teams.

  • Automated Normalization: Establishing simple text-cleanup operations, our Team normalized the majority of listed activities into one activity category.

  • Manual Global Activity Normalization: Defining some global normalizations based solely on the activity name to catch many common cases.

  • Manual Store-level Activity Normalization: Selecting appropriate activities by manually editing specific stores, where the activity name is too generic to normalize globally.

data normalization chart OMG

Figure 2: Automated techniques can normalize most of our store data, shown here as icons that go from yellow to green. Stores that are skipped due to poor data quality are manually updated in increasingly granular manners until a large majority of stores are normalized.

Looker was critical in this endeavor, particularly because it gave us those GMV numbers that determined what activities belonged in our Master List. Once we were satisfied with the coverage our normalization techniques provided, it became a matter of handing off the list and normalized values to our Engineering Team. They were able to update the old, poorly formatted data to use our new Gold Master Activity List. Now stores are using consistent data!


Conclusion

Data normalization is essential to making impactful business decisions with trustworthy, understandable data. Looker proved to be an essential and effective tool for OMG, and there are millions of ways Looker auctions can be used to take action on your own vast dataset. After all, data is really only valuable if you’re doing something with it, so create more actions!!

 

About OrderMyGear

OrderMyGear is an industry-leading sales tool, empowering dealers, distributors, decorators, and brands to create custom online pop-up stores to sell branded products and apparel. Since 2008, OMG has been on a mission to simplify the process of selling customized merchandise to groups and improve the ordering experience. With easy-to-use tools, comprehensive reporting, and unmatched support, the OMG platform powers online stores for over 3,000 clients generating more than $1 billion in online sales. Learn more at www.ordermygear.com.

Media Contact: Lauren Seip | lauren.seip@ordermygear.com | 281-756-7915

Daniel Hodges

Written by Daniel Hodges

Featured