Three or four-ish years ago I was an intern working at a website design and development company. We worked exclusively with WordPress. For those that are not familiar, WordPress is a Content Management System (CMS) that runs roughly 40% of the world's websites. WordPress has plugins that run on top of the core WordPress functionality to provide additional features. WooCommerce is a popular third-party plugin that allows you to add an online store right into your website to sell anything you could imagine. WooCommerce is highly flexible, feature rich, and customizable for developers to tweak as needed. On top of that, WooCommerce has a high selection of its own plugins to further extend its functionality. These two tools were at the center of my problem.
At the time, one of our clients shipped critical electrical components to customers domestically and internationally. Their website was built on WordPress and their store was built with WooCommerce with several plugins to further extend Woo's functionality.
Over a period of several weeks, shipping estimates for particular orders grew to be 3-5x higher than the actual shipping cost. Only certain orders had this issue to begin with, so our client was going to continue to monitor the issue. Then it began to affect their bottom-line, sales. Their customers were calling to ask why shipping estimates were so high. They got notes in orders instructing them to use their customer's own UPS accounts to ship their orders. Customers even openly admitted they were placing orders from our client's competitors because of the shipping estimates being displayed with their orders.
Shipping estimates can be a tricky thing. You want the shipping estimate to be just slightly higher than the actual cost (or equivalent) to ship the order. You do not want shipping estimates to be lower than the actual cost to ship an order as the business either eats the loss or has to ask the customer for more money. At the same time, you don't want the shipping estimate to be so high that it turns customers away.
To be clear, our client was refunding their customers the difference between the shipping estimate they were charged and the actual cost to ship their order to them. Anything else would have been scummy on their part. They were having the issue of shipping estimates being so high that it was scaring customers away. That was the problem we now needed to solve.
Debugging is like a puzzle. Damn challenging, but incredibly rewarding. As the intern on the team at the time I had developed a knack for debugging a variety of issues the company had run across. My boss had passed a series of emails on to me and told me that not fixing this issue was not an option. It was one of those issues where you need to stop everything else you are doing and jump in feet first. So, I got started.
First I had to determine where to start. I picked Google, naturally. The great thing about WordPress and WooCommerce is that since they are so widely used if you are having a problem...usually...someone else has had the same problem.
45 minutes of Google-ing later I had found nothing about WooCommerce shipping estimate issues. So much for someone else having the same issue. I was hoping for some initial settings and details to start looking at in our client's WooCommerce administration area. I didn't even find that. Oh boy.
Okay, where to next? Excel. I started digging into specifics about the orders that were affected by this issue where shipping estimates were 3-5x higher than the actual cost to ship the order. WooCommerce tracks everything. About an hour later I noticed a common thread between the affected orders. All of the affected orders had a total package weight right around 1 lb. and the orders were typically comprised of a large number of power supply cables. An interesting hint.
Then I hit a wall. Out of new leads and ideas I decided to enable WooCommerce's debug mode. Kudos to the WooCommerce team because their debug mode is incredibly thorough. With debug mode on, essentially every WooCommerce feature spits out all of the background work and data it has to share. Admin views, front-end store views, the shopping cart, everything.
As great as the feature is, it is like trying to sip from a fire hydrant. So, I refocused on the biggest hint I had: the affected orders. I opened an order and went to the cart page. With debug mode on the cart page spits out raw data in the form of multidimensional arrays about product details for each item in the cart, information on how the shipping estimate is calculated, and how the total is calculated.
I started parsing the information and came across an interesting detail towards the end of the cart. The cart held data for two different total weights. The first was a total "physical" weight and the other was a "dimensional" weight.
Huh? Let me give you some background.
When you sell and ship physical products with WooCommerce you have to do two things:
At the time, one of our clients shipped critical electrical components to customers domestically and internationally. Their website was built on WordPress and their store was built with WooCommerce with several plugins to further extend Woo's functionality.
Over a period of several weeks, shipping estimates for particular orders grew to be 3-5x higher than the actual shipping cost. Only certain orders had this issue to begin with, so our client was going to continue to monitor the issue. Then it began to affect their bottom-line, sales. Their customers were calling to ask why shipping estimates were so high. They got notes in orders instructing them to use their customer's own UPS accounts to ship their orders. Customers even openly admitted they were placing orders from our client's competitors because of the shipping estimates being displayed with their orders.
Shipping estimates can be a tricky thing. You want the shipping estimate to be just slightly higher than the actual cost (or equivalent) to ship the order. You do not want shipping estimates to be lower than the actual cost to ship an order as the business either eats the loss or has to ask the customer for more money. At the same time, you don't want the shipping estimate to be so high that it turns customers away.
To be clear, our client was refunding their customers the difference between the shipping estimate they were charged and the actual cost to ship their order to them. Anything else would have been scummy on their part. They were having the issue of shipping estimates being so high that it was scaring customers away. That was the problem we now needed to solve.
Debugging is like a puzzle. Damn challenging, but incredibly rewarding. As the intern on the team at the time I had developed a knack for debugging a variety of issues the company had run across. My boss had passed a series of emails on to me and told me that not fixing this issue was not an option. It was one of those issues where you need to stop everything else you are doing and jump in feet first. So, I got started.
First I had to determine where to start. I picked Google, naturally. The great thing about WordPress and WooCommerce is that since they are so widely used if you are having a problem...usually...someone else has had the same problem.
45 minutes of Google-ing later I had found nothing about WooCommerce shipping estimate issues. So much for someone else having the same issue. I was hoping for some initial settings and details to start looking at in our client's WooCommerce administration area. I didn't even find that. Oh boy.
Okay, where to next? Excel. I started digging into specifics about the orders that were affected by this issue where shipping estimates were 3-5x higher than the actual cost to ship the order. WooCommerce tracks everything. About an hour later I noticed a common thread between the affected orders. All of the affected orders had a total package weight right around 1 lb. and the orders were typically comprised of a large number of power supply cables. An interesting hint.
Then I hit a wall. Out of new leads and ideas I decided to enable WooCommerce's debug mode. Kudos to the WooCommerce team because their debug mode is incredibly thorough. With debug mode on, essentially every WooCommerce feature spits out all of the background work and data it has to share. Admin views, front-end store views, the shopping cart, everything.
As great as the feature is, it is like trying to sip from a fire hydrant. So, I refocused on the biggest hint I had: the affected orders. I opened an order and went to the cart page. With debug mode on the cart page spits out raw data in the form of multidimensional arrays about product details for each item in the cart, information on how the shipping estimate is calculated, and how the total is calculated.
I started parsing the information and came across an interesting detail towards the end of the cart. The cart held data for two different total weights. The first was a total "physical" weight and the other was a "dimensional" weight.
Huh? Let me give you some background.
When you sell and ship physical products with WooCommerce you have to do two things:
- For each product you sell you need to enter its weight and its physical dimensions
- For each box your company uses to ship its products you need to enter its inner dimensions
These two critical details are needed to allow WooCommerce to use its packing algorithm to determine:
- How many boxes are needed for an order
- What items to pack in what box
But what the hell is this "dimensional" weight and why do we need it?
FedEx defines "dimensional" weight as:
Dim weight is the amount of space a package occupies in relation to its actual weight. For each shipment, you are charged based on the dimensional weight or actual weight of the package—whichever is greater.
Dimensional weight is calculated by taking the volume of the box you are using to ship your goods and divides it by a constant that is set by each major shipping provider (USPS, UPS, FedEx, DHL, etc.).
After another several hours of reading documentation from the team at WooCommerce (another kudos to the team for their superb documentation) it turns out that to provide a shipping estimate to the end user Woo will take the greater between the physical and dimensional weight and send that out to the shipping provider to get an estimate.
Great, but what does this have to do with our issue?
Remember the detail shared between our affected orders? They were predominately comprised of many, light power supply cables. They were relatively small, short cables (under 1 lb. total) that were being packed into a medium to large-ish sized box.
Bingo.
The total physical weight of the cables in the shipping box was, on average, about 1 lb. The box they were being packed into was overkill for what was being shipped. So, when the volume of these larger boxes was calculated and then divided by the shipping provider's constant, the dimensional weight came out to just over 36 lbs. That would explain why:
- The shipping estimate issue was only present on certain orders and not others
- Why the shipping estimate was 3-5x greater than the actual cost to ship the order
Great. We found the issue. How do we fix it?
I dug into the backend area of WooCommerce and noticed a small, important detail. When you set up the shipping box dimensions in the administration area of WooCommerce, you have to provide the same set of dimensions for the box twice. Once for the WooCommerce packing algorithm and once for WooCommerce to send out to the shipping provider to get a shipping estimate with (to calculate the dimensional weight).
Well, what if we were to change the dimensions that are sent to shipping providers for every box to be 1" x 1" x 1"? We need to leave the dimensions that are used by the WooCommerce packing algorithm intact, so orders are packaged appropriately.
If we change the dimensions sent to shipping providers to get a dimensional weight to 1" x 1" x 1”, we will effectively ensure that the physical weight for every order will be greater than the dimensional weight. Thus, the physical weight will always be used by WooCommerce to generate a shipping estimate for the end user.
After some testing and analysis in Excel the change created a 50-75% improvement in accuracy of the shipping estimates for the orders that were previously incorrect. For all other orders, things remained just as accurate (within margin of error that was deemed acceptable).
After 10-12 hours of reading, testing, failing, and trying again - I had a solution. Later that evening we pushed the change through, and while I was still at the company I never heard of the issue reoccurring.
Although this debugging never involved touching code it was the most satisfying bug that I have ever fixed to date. More importantly, I learned several valuable lessons that day that will serve me for the rest of my life:
- Thorough documentation is invaluable.
- Value people that can solve problems you can't.
- When attacking a problem head on doesn't seem to be working, come at it backwards.
- Document everything you do. It makes the life of those that come around after you are gone easier.
- When running a business, the customer is not always right. However, when your customer's business is losing money because of an issue with a service you were paid to build and manage for them, they are always right.
Trying to use technology for things it has never been used for is challenging and problems are bound to arise. Problems no one has ever encountered before. There is no step-by-step guide to solve problems in computer science. That is what I love about the field.
There is nothing more rewarding than to solve a problem like I did that summer day. A problem that no one on planet Earth had ever faced before, now had a solution. I hope by writing this piece up it could help someone else in the future solve a similar problem.