Interracial marriage is not the same thing as same-sex marriage. Duh.
Consider. Would it be racist for whites to only date whites, or only marry whites? Probably. Certainly President Kimball still gets flack for having encouraged LDS youth to date within their “culture.”
Would it be homophobist for me to only date women, and to have only considered marrying a woman?