It's very easy to build for 4.0 and target the devices that cannot upgrade... so I reckon 3.x will be around for some time.
Apple says no, case closed. http://www.engadget.com/2010/06/30/iphone-os-2-apps-no-longer-welcome-on-the-app-store/
I suppose that at some point you can't support all the device gens and releases of the iphone system. Are there still lots of people who only have a 1st or 2nd gen device?
2nd gen users can upgrade to 4.0; only 1st gen users have to stay with 3.x. So solely supporting 4.0 would only leave out 1st gen users.
Then it's just a matter of supporting 3.x and 4.0. Some developer friends of mine told me they were going to be making 3.x and 4.0 compatible releases of their games from now on.
That people "can" upgrade doesn't mean they have or will in the near future. Especially when iOS 4 doesn't work very smoothly on the iPhone 3G. So, for the time being, it'd be a safe bet to still support 3.x. At least Apple has made our life easier by dropping support for 2.x.