Over the past week we’ve received emails from real estate professionals expressing skepticism about Zillow Offers using the Zestimate as a live offer to homeowners. The concern is that the Zestimate, fundamentally an algorithm, is inferior to a human at assessing the value of a home. We largely agree. Today, the Zestimate has shortcomings and understandably should be viewed with a degree of skepticism. Over the next five years, we expect it will meaningfully improve, thereby earning more credibility, and unlocking an opportunity to transform an industry.
Selling or buying a home is often the largest and most emotional transaction people make in their lifetime. For homeowners to forgo the traditional human-centric sales process of realtors and brokers and sell directly to Zillow, they will need to feel confident they’re not getting ripped off. The bottom line is Zillow knows it has to align its interests with those of sellers for Zillow 2.0 to be a success. While honing the Zestimate will take time, we believe eventually getting it right will win the skeptics and bring a new level of transparency and liquidity to residential real estate.
Skepticism is reasonable
There is concern the Zestimate can’t keep pace with a fast-moving housing market, given it relies heavily on past sales data to inform its home valuations. On average, home sales are pending for around 45 days, and Zillow gets the data when final sale prices are reported to Multiple Listing Services (MLSs) 1-3 days after closing. A month plus lag can have a material impact on valuations. Humans, on the other hand, have an advantage over the machine because they talk, opening the window for pending sales data to reach potential buyers and sellers before the transaction closes. Information is power.
Additionally, critics say the Zestimate can’t capture the intangible aspects of a home that materially impact its value, things such as curb appeal, the aesthetics of a kitchen remodel, the “nice home in a bad neighborhood” dynamic and vice versa. On top of this, the Zestimate may fail to capture the emotional side of the housing market, which would indicate whether a home is likely to have a bidding war that would be advantageous to the seller.
In the end, the Zestimate needs work, and Zillow needs to get it right to make iBuying a reality. Inaccurate valuations would pose a brand risk for the company, as well as the risk of unprofitably exchanging homes.
Zestimate 101
To better understand the future of the Zestimate, we first need to understand where it is today. At its most basic level, the Zestimate is a data-driven estimate of the value of a home. Thus, the accuracy of the Zestimate is a function of the quality and quantity of input data. The Zestimate primarily utilizes publicly available data from multiple listing services (MLSs) and county tax records. The Zestimate also incorporates user-generated data from homeowners that manually update their home specifications and renovations. In 2019, Zillow said more than 70 million homes, or about 65% of all homes on its site, had been updated by users. Separately, when a home is listed for sale, Zillow captures data from seller disclosure statements that cover about 20 aspects of the home. Sellers can also elect to have the Zestimate removed.
Zestimates are not given for homes lacking sufficient data to make a reliable estimate. Currently, they’re available for about 103m homes in the US, or 82% of all occupied homes.
While there’s only one Zestimate, it should be differentiated into two buckets, on- and off-market homes:
- Off-market homes: This of course represents the vast majority of homes on Zillow, likely 95% plus of all homes at any given time. We view most off-market Zestimates as directional indicators, given they can be inaccurate anywhere from +/- 2% to 20% plus. We believe off-market Zestimates are the most important for Zillow 2.0 (iBuying), given they’ll be the basis for most live Zestimate offers. We discuss more below.
- On-market homes: This represents homes for sale, usually less than 5% of all homes at any given time. On-market Zestimates are often spot on because they have the benefit of additional data, including the listing price, which influences the Zestimate. The criticism of on-market Zestimates is that they merely follow a home’s listing price. More below.
Off-market Zestimates are unpredictable
The data inputs for off-market Zestimates are primarily public historical tax assessments and comparable home sales in the area. Today, as a whole, they are not accurate in our opinion:
The takeaway from the chart above is 38% of off-market Zestimates are accurate within +/- 5% of the home’s final sale price. In the middle, 23% of Zestimates are accurate within 5-10%, and 39% of Zestimates are off by more than 10%.
We believe off-market Zestimates are the most important Zestimate for the success of Zillow 2.0. That’s because the majority of live Zestimate cash offers will go to homeowners whose homes are not on the market. While sellers who have already listed their home could pivot and sell to Zillow Offers, they would likely have to navigate a listing agent contract, which we view as unlikely.
Starting out, live Zestimates will be used for a select subset of homes that Zillow feels it can accurately price. As pricing algorithms improve, we believe Zillow will judiciously scale the pool of eligible homes. There will still be a human inspection before a final offer to ensure Zillow is making fair offers to sellers.
At first glance, it might appear that live off-market Zestimates are too inaccurate for Zillow to profitably buy and sell homes. However, while off-market Zestimates can generally be inaccurate, we believe there is a more than ample pool of accurate ones for Zillow to build inventory using live Zestimates offers.
This is how we get there. There are roughly 100m off-market homes for which Zestimates are available. As the above chart illustrates, 38% of them are within +/- 5% of the eventual sale price. If we assume an even distribution of this 38% across a spectrum from -5 to +5, this implies about 8% of off-market Zestimates are accurate within +/- 1% of a home’s sale price. Putting it together, that gives Zillow Offers a live Zestimate addressable market of 8m homes. For perspective, Zillow purchased 4,162 homes in 2020.
On-market Zestimates are robust
On-market Zestimates are more accurate. Half of the on-market Zestimates are within +/- 1.9% of the home’s final sale price. Just 1% of on-market Zestimates are outside +/- 20%:
The Zestimate’s accuracy is found by comparing the final sale price to the Zestimate that was published on or just prior to the sale date.
“My Zestimate jumped”
Often times, there is a sharp change in the Zestimate shortly after a home comes on the market, in what appears to be the Zestimate adjusting to the listing price. This is why some real estate professionals view the on-market Zestimate as laughable, saying it merely mirrors the listing price. In fact, listing price is one of what we estimate to be 30 key factors that influence the algorithm. Putting it together, on-market Zestimates are more accurate because they incorporate more data.
The rush of data comes when a home is listed for sale, as homeowners submit detailed information to Multiple Listing Services (MLSs) of a home’s description, square footage, number of bathrooms, along with the official appraisal-based list price. Zillow receives this information through its direct feeds from hundreds of MLSs. In addition, on-market Zestimates factor in comparable home sales in the area, number of days on market, number of views on Zillow, along with location characteristics such as school districts, walkability, and crime rates.
In terms of capturing the qualitative aspects of a home from listing photos, since 2019 Zillow has used “neural networks and computer vision to distinguish between high and low-end finishes and to incorporate the value of features like updated bathroom fixtures, fireplaces, and remodeled kitchens.” While we don’t believe this capability is at the level of human expertise today, past experience (ImageNet) tells us that with time and repetition it will likely surpass it.
Humans vs. the Zestimate
The above data show the Zestimate has a long way to go. That said, humans aren’t perfect at estimating real estate value either. To ballpark human accuracy, we looked at 30 recent home sales in the Duluth, Minneapolis, and Saint Paul areas to compare the initial list price against each home’s final sale price. We found 70% of the time humans were accurate within 5%, which compares to 38% and 85% for off- and on-market Zestimates, respectively.
While our sample is small, at a minimum it shows that, regardless of whether it’s a human or a machine, estimating real estate value is complex, as it must account for many factors, including unpredictable ones like market psychology.
Perfecting the Zestimate will take time
The Zestimate’s accuracy is key to the success of Zillow 2.0:
- If the live Zestimate undershoots, Zillow won’t build inventory. Regardless of whether it’s a machine or a human, if you bid too low, you’re not going to get the house.
- If the initial live Zestimate overshoots and the company must materially lower its final cash offer after an in-person inspection, consumer trust will erode.
At first take, the above data appear to show there is risk in Zillow standing behind the Zestimate to accurately price and profitably exchange homes. In reality, that’s a manageable risk because Zillow will start with a small subset that’s in its pricing sweet spot, and there will be a human inspection before a final offer. The reason Zillow needs to get the Zestimate right is for brand and trust building.
The fundamental question is whether a machine will eventually be capable of consistently appraising the value of a home better than humans. We are long-term believers in the power of AI, and see appraising home values as a wheelhouse machine learning use case. The question is how long will it take before the US housing market trusts the Zestimate. Our prediction is five years.