The “Cares act” allows $100,000 of 401k withdrawal without penalty. What are the reasons not to do this?

From this website , we can pull out $100,000 from our 401k without penalty.

Distributions taken from qualified retirement plans received during 2020 of up to $100,000 for COVID-19 related purposes are allowed without a 10% penalty, taxable evenly over 3 years beginning with year of distribution

What are the reasons not to cash out our 401k? In my opinion, my would greatly benefit to have this money in real estate or other investments. We have already saved up about a $150k for a house down payment, and having $100k more would allow us to buy a house in our area outright or with a very small mortgage. Alternatively, we could put it into the S&P500 or Tbills, etc.

Edit: By "other investments" I mean something like a Roth IRA, which can be invested in similar stuff but with better long term tax implications.

Also, please note that prices of many investments will drop; so selling when its low is acceptable, assuming it is reinvested in something else that can also be purchased for cheap.