Because the vast majority of available visas are for skilled workers.
There are some seasonal nonimmigrant visas. There are 3rd pref immigrants visas.
And before we say "oh, just claim asylum" that's not a category for asylum. So you may get some unskilled labor tangentially through that path.