It seems like you are heading in the right direction as far as being open-minded and experimenting.
As far as him having a lady and you just watching, one thing you might want to try is to have him tie you to the chair, snug but comfortably, and put a gag in your mouth so you can't talk but you can see. The addition of toys inside of you would be an option as well. Then, when the lady comes over, you can watch the two of them enjoy each other while you squirm there. If you feel like it (or actually if he does...) at some point he can blindfold you as well so that you hear and know what's going on, but you can't see. Which, depending on how that makes you feel, can make you even more excited. Don't forget that swinging can go both ways, and your b/f can also find himself tied to the chair while you enjoy a gentleman.
Another thing to try is to have each of you write down 10 fantasies you want to do, with the knowledge of what the other partner likes and won't do. And then put those fantasies in a jar and every weekend, or whenever, draw out a fantasy and do that with your partner. Since you've already explored so many other things, its easy to come up with fun, sexy fantasies that the both of you can do to one another. And the best part is the surprise of whose fantasy you are going to fulfill that night.