Generative AI is rapidly evolving from a tool of innovation to a mechanism that amplifies existing inequalities, especially for marginalized communities. As tech giants like Meta and xAI utilize vast amounts of user data from social media to train their models, the implications for privacy and representation are profound. This trend not only risks distorting the realities of underrepresented voices but also exposes them to new forms of harm without adequate oversight.