Perhaps, like many rich old men, he has a beautiful, young, vivacious "kept" woman; and merely fears losing her attention to younger, virile, handsome men, who may (in his mind) may be able to provide an attribute that an 81 year old no longer can consistently or easily provide. Being cuckolded by a black man may be the biggest fear some racists possess (at least that seems to be behind many of the old laws and actions of the South). Indeed, such beliefs may be the root of many of the racist words and actions by the backward and ignorant among us.
That knife may cut both ways, how many young black men (or fathers) fear losing girls to rich white men?
It's kind of pathetic the way the old follow truly believes he is the "benefactor" for his players. He probably does not understand all the conflagration that has developed from trying to keep his "girl' solely with him (especially if he feels he has paid for that privilege).
It is difficult to say from our limited view, few social interactions can be simply explained. (In general, generalizations don't apply)