If you look at the modern demographics, actually all countries around the world are seeing a reduction in birth rates. This includes Africa. As women are better educated and have access to healthcare they have fewer children. It has nothing to do with race.

Where this becomes 'racist' is when the Democrats and Republicans try to 'game' the system to promote their party. Democrats (neoliberals who have taken over the party since Clinton) have the idea that if you eliminate enough white middle class voters and replace them with poor racially diverse voters, they will naturally vote Democrat. This is the fear of the Republicans, and based on the perceived agenda of the Democrats. Of course this is racist, but also is setting up a neoliberal class power structure where elites rule over an underclass of poor voters by eliminating the middle class.

Both the neoliberals in the Democratic party and the Republicans have bought into this racist concept. It is impossible to say which party is more racist at this point.