If there's a coffee shop on-site, just say you're going to the cafe.
If not, maybe smile sheepishly and say "I'm just coming in to use the bathroom. "
Even add "I know it's not really for non-guests," or "Is that against the hotel's policy? I could buy a newspaper or coffee to make me a customer"
In any case, I would smile, not rush, pause and chat like a NORMAL INNOCENT passerby would
all curious "What's this about?" "What's going on?" "are y'all having some kind of issue inside here?" and be MORE that willing to turn about-face and make other plans. (i.e. find another bathroom/coffee shop).