Data-Driven Decisions – The Other DDD
When empowered with the data, you can make the right decisions. You can formulate the best strategies. There are no distortions.
A multitenant SaaS application that helps hundreds of warehouses manage their inventory and production will sooner or later encounter data analytics scalability issues. This was the case here at Nulogy with PackManager, our flagship product. The report that we encountered performance issues with is one of our most commonly used reports and it is called the Inventory Transaction History Report.
Inventory is like the currency of our WMS functionality in PackManager. Creating, modifying, transforming, and deleting inventory is what our application performs on a fundamental level. In a bank, you have accounts with a balance that gets either debited or credited. In PackManager, a combination of parameters like SKU and location act to define the equivalent of a bank account with inventory levels of those parameter combinations as the account balance. So when 100 cases of ACME Sweets are moved from location A to location B, they get “debited” from location A and “credited” to location B. We call these “debits” and “credits” inventory deltas. Together, a remove inventory delta and an add inventory delta forms an inventory transaction.
Another common example of an inventory transaction from our domain is when packaged goods consisting of subcomponent inventory are produced. The subcomponents are removed as “consumption” and the finished goods are produced as “production”. So far there have been millions of such transactions across all our customers. Since all this is stored in a single underlying database table, it makes reporting on the entire dataset practically impossible.
There are certain scenarios where our customers want to know exactly what inventory activity occurred. They want to know what inventory transactions took place given a set of parameters. The parameters can be a SKU code, an inventory status, a lot code or expiry date, or the unique identifier of a pallet called a “pallet number”. The solution that PackManager implements is an Inventory Transaction History Report. For example, by using this report, a customer can find out what caused all the ACME Sweets SKU to go missing from the warehouse. The system will scan the inventory transactions dataset given the SKU code as a parameter and generate a report.
The problem is that the dataset is too large for the Inventory Transaction History Report to return in a practical amount of time. So a restriction was added to prevent the user from querying a date range greater than 31 days. This was sufficient for a practical response time early on until the dataset got even larger and even this became too slow. As our customers used PackManager over the years, they needed to investigate between date ranges that spanned more than 31 days. We learned that the report was being used primarily for investigating issues on a specific pallet. So if we created a report specifically for inventory activity on a pallet, we could remove the 31 day restriction on the date range, since the dataset would be orders of magnitude smaller.
Using this information, the reporting team decided to create a Pallet Transaction History Report, a close sibling to the Inventory Transaction History Report. The main difference is that this report only accepts the pallet number as a parameter. We also added an index to the pallet number field on the database table mentioned earlier so that the report is generated quickly. The vision was that this report would be used to find out what happened on a specific pallet only. And you’d use the Inventory Transaction History Report for a more complex scenario of investigation consisting of many parameters.
The Pallet Transaction History Report was released on January 3rd of this year. After a few months, we deployed a tool for analyzing our application usage logs called Kibana. The reporting team was curious to see how our customers had actually adopted the Pallet Transaction History Report. We were disappointed to find out that this rollout hadn’t made any impact (please see graphs below). Our users were still using the Inventory Transaction History Report for their pallet specific queries. But what was cool now was that we had data. We could analyze how our system was being used. We could run experiments to see if we can increase adoption of the new report since it addresses an important use case.
So on March 2nd, 2015, a few of months after the initial rollout, we used UserVoice to communicate with our users that this new feature was available and that it could be used to diagnose problems on any pallet in their warehouse. We started to notice a downward slope of the usage of the Inventory Transaction History Report with only a pallet specified. But this still wasn’t enough. The pallet specific case was still the majority of the usage. So on March 29, 2015, we rolled out a helpful message on the Inventory Transaction History Report that prompted you to use the new report whenever it detected that you only specified the pallet number. We saw a significant reduction in the following week. This pattern continued and the usage of the Pallet Transaction History Report kept increasing. Finally we were able to get this report adopted by our userbase. Success!
An overview of the adoption experiments that we ran and how it impacted the usage of the Inventory Transaction History Report.
Number of times the pallet number was specified (along with other parameters) when running the Inventory Transaction History Report.
Number of times only the pallet number was specified when running the Inventory Transaction History Report. Its clear that this scenario was the majority of the usage.
Number of times that the Pallet Transaction History Report was accessed per week. The success of the adoption is clear based on the increasing usage over time.
Domain-driven design, or more commonly known as DDD, is a cornerstone of how we develop our products at Nulogy. The supply chain and logistics industry presents us with a complex domain and without the wisdom of DDD, our software would not be as agile and scalable as it is today.
Through this story of our Pallet Transaction History Report rollout, I present to you another kind of DDD, one that I feel is more and more important. Data-driven decisions. Without it, we may have never known whether we should try and increase adoption of the Pallet Transaction History Report. Without it, we may have never known whether we were actually able to improve our users’ experience.
When empowered with the data, you can make the right decisions. You can formulate the best strategies. There are no distortions. No he said she said. Just the facts. Steve Jobs famously said, “Stay Hungry. Stay Foolish.” I interpret that as stay hungry for the data because you can never have enough. And stay foolish by being humble in your views because you can never be too certain. There will always come newer, and better data to describe this beautiful place we live in.