America or Europe could easily be demographically swamped if even a fraction of the world's five billion other citizens headed for the West. The physical infrastructure of buildings and roads is not up to a sudden massive increase in population, much less the government institutions.
Worries about not having enough buildings are rather silly; if demand goes up, so do real estate prices. Eventually people either downsize their living space or decide that the cost of living in the U.S. is so high it's not worth coming. (That's how New York City handles the problem, after all). Worries about roads are more serious, but only because we eschew user fees and most of the advanced technology that makes them extremely easy to collect.
In writing this, I can just imagine the anti-foreign reading: Immigrants are going to make real estate unaffordable! Now let's see. What's the nationality of the owners of almost all U.S. real estate?