So I have no problem with the American people collectively owning the country to the extent that they have a veto over who can be invited in. I know of no realistic theory that would make that wrong or unethical. And I know of no theory that would make "open border" a sustainable idea. As a libertarian, I'm all for reducing the power of the state. But I'm also for the state exercising its few legitimate powers, among which is to allow immigration only when it's in the interest of the people as a whole.
That may satisfy you, but it you want closer, more painstaking argument, all based firmly in libertarian theory, with plenty of Rothbard and Hoppe, I give you this, reprinted from the Libertarian Alliance [link];