People fear that immigrants will compromise American culture and vote enthusiastically for policies that restrict our freedoms. Laying aside the fact that immigration restrictions are already a massive encroachment upon my freedom--why should I be prevented from hiring anyone I please at terms we find mutually satisfactory?--I'm unconvinced that immigrants represent a threat to the Republic.
For an extreme case, consider Cuba. The migration flow is almost entirely in one direction: people are looking to escape Cuban tyranny, poverty, and stagnation. I suspect that most immigrants from Mexico, Africa, and Asia have more in common with Cuban immigrants than they have with nefarious invaders looking to impose their political institutions on us. Cuba might be a special case, but my sense is that this is a difference in degree rather than kind. Even if they are coming here and burdening our welfare system, preventing them from getting here isn't a free lunch. "Border security" requires real resources, and preventing me from trucking and bartering with anyone who will deal with me is an encroachment on my freedom that rivals (or equals, or exceeds) what I would have to pay in extra taxes.