It ultimately comes down to the First Amendment. Pornography is protected under the First Amendment as artistic free expression. Prostitution, on the other hand, is defined by U.S. law as a sexual act given or received for monetary payment. Not only that, but the U.S. government maintains that prostitution "directly contributes to the modern-day slave trade and is inherently demeaning." Proponents of decriminalizing sex work say this is one of the major obstacles facing sex workers. Ultimately, in the eyes of the U.S. government, prostitution carries a stigma as being detrimental to society, while pornography offers some form of artistic value.
That's not to say there is no regulation for pornography. There are strict rules in place. Producers have to maintain multiple business filming licenses as well as detailed records on performers. Companies have to be able to prove they intend to distribute the filmed work.
Source