Over at the our sister site, the Library of Law and Liberty, my friend Patrick Lynch wrote a post titled "Libertarians Can Believe in Borders." While I can conceive of good arguments against completely unrestricted immigration, and while some of those arguments have caused me to hesitate to recommend no such restrictions, I don't think Patrick's arguments hold up.
Support for open borders implies the elimination of national boundaries for the purposes of political organization and is much more consistent with anarchism than with classical liberalism-- both of which are now commonly referred to as libertarian. It's a position that rejects the entire experiment in constitutional governance and different political systems that has been a foundational belief in liberalism for hundreds of years.
I'm not sure why support for open borders implies support for the elimination of meaningful national boundaries. Think of it this way: there's an open border between Virginia and West Virginia but that doesn't mean that we've eliminated state boundaries for the purposes for political organization. It just means that the state of Virginia can't forcible exclude West Virginians from entering.
So what would be a good argument against unrestricted immigration? Commenter Sean II, responding to Freiman, gives it, writing, "[the] dose makes the poison." I don't endorse everything he said and I don't like his tone, but the point he makes is valid and important. Here's how I would put it: How can we say, based on relatively tiny increases in the number of immigrants to this country, what would happen if the annual number of immigrants ten-tupled?