While I don't think that preferences in and of themselves are racist, I believe that how we come to many of our preferences are a product of the isms under which we live. As a young brown man growing up in Minneapolis, I was told, oh so many times, to my face...almost always by white men (though it was sadder when it came from brown men) that I was "cute," but he (the man in question) didn't do black, brown, dark men (and I am by no means dark, so I can't imagine what would have been said to a dark skinned black person).
Our standards of beauty are absolutely based in race and racism just as is our fetishism and what each individual finds attractive is crafted and shaped by those forces just as much as the messages they got at home and from family. Sometimes family and personal messages/community messages can counteract societal and media messages but not often and never completely.
And, frankly, though I am with a white man now, for five years I refused to date any white men because I understood that it was radical, in our society, to love a brown man...particularly when I came to understand that as a product of where I grew up, when I grew up, and the messages I received that I had been taught that brown/black/yellow wasn't beautiful unless it was exotified.