eBay
pic1.jpg
Problem
It is not uncommon to see disputes between buyers and sellers in eBay transactions. Under the current system, it may even be argued that sellers are actually at a disadvantage relative to the buyers in a dispute mainly due to one specific rule.

Under the current system, if a buyer is not satisfied with a purchase, he can file a non-receipt claim and will likely get refunded unless the seller has proof of delivery, which is usually a tracking number that indicates delivery on the shipping company's website. Note that a tracking number that indicates shipment but not delivery is not sufficient. This can be troublesome for sellers shipping internationally. It is common that the tracking number shows the package accepted in the U.S. and shipped overseas, but the destination country's carrier does not scan the package. In this case, the seller may be forced to refund the buyer should there be a claim, even if he does not believe that the package got lost. It is unfortunate for the seller if it is false claim. However, ensuring that the package gets scanned in an international destination will require much higher shipping costs neither the seller nor buyer wants to bear. As a result, the seller may block the buyer or all buyers in that country altogether, and that means reduced revenues for the sellers and eBay who charges commissions based on the value of the items sold.

pic3.jpg
A package scanned in the U.S. but not in the destination country will stay "in transit" forever. If the buyer files a claim, there is no protection for the seller due to lack of proof of delivery.


Hypothesis
The hypothesis is that removing the rule of requiring proof of delivery will not affect the honest buyers. If only proof of shipment, but not delivery, is necessary to protect the seller in case of a non-receipt claim, it would not prevent the honest buyers from buying. Moreover, it may help identify the buyers who tend to file false claims.

Action
The hypothesis can be tested using a randomized experiment. We can first identify a category with high frequency of non-receipt claims filed by buyers. Ideally, all sellers in this category should be very similar in all aspects. Then we choose 100 sellers from this category at random to be in the treatment group. This group will be provided with protection from non-receipt claims if they can show proof of shipment, not delivery. This relaxed restriction should be added as a note to the sellers' items so buyers know before they buy. Another group of 100 sellers should be chosen at random from the same category, but will be required to show proof of delivery in case of disputes. These two groups will be observed over a period of time.

pic2.png

Metrics
To measure the impact of the rule change, we can compare the change in number of non-receipt claims filed for sellers in the treatment and control groups before and after the experiment. If the treatment group has a significantly greater decrease in the number of non-receipt claims, then it is likely due to the extra protection of the sellers. Moreover, it could mean that some buyers are deterred from buying when they cannot abuse the system any more. The hypothesis is also supported if buyers with a history of filing non-receipt claims stop buying from the treatment group but keeps buying from the control group despite that they are similar sellers in the same category.

Evaluation
This experiment relies on the assumption that shipping companies are reliable and that buyers trust them, because changing the requirement of proof of delivery to only proof of shipment shifts the responsibility from sellers to shipping companies. Also, the results are more reliable if the sellers in the treatment and control groups are similar in almost every aspect except for the requirement of proof of delivery versus shipment. For example, if sellers in one group sell high-end items while sellers in another group sells low-end items despite in the same category, it is unclear whether the effect is due to the extra protection offered to the sellers.