Fantasy does NOT have to follow real world rules. Fantasy does NOT have to relate to some real world event, country, concept, law, or history. Fantasy does NOT have to mirror any particular time period or country, even if you’re basing your world on a real world one. There is NO SUCH THING as “historical accuracy” in fantasy as it relates to the real world.
THE ONLY THING Fantasy has to do to be believable is follow the established rules OF ITS OWN WORLD. Fantasy can literally be anything you imagine it to be.
If your fantasy world excludes people of color or those belonging to the LGBT+ community, if it’s grossly misogynistic and white cis-male centric, that’s because YOU made it that way. Stop blaming “historical accuracy” or “believability”. It’s not the genre; it’s YOU.
If I may add something here (which I’m sure others have already pointed out):
People of color, LGBTQ+ people, disabled people, and neurodivergent people have existed everywhere throughout all periods of history. The way mainstream history is told has always been shaped by ideas about national and ethnic identity. We have images in our heads about what a historical place/period “should” look like – but those images are simply not accurate. Example: “everyone in Elizabethan England was white.” Nope. People of African descent were there. “No one before the 20th century was autistic, it’s a new thing.” Wrong again. Because of the way mainstream history is told, we don’t hear about these people, but that does not mean they didn’t exist.
When you bring up “historical inaccuracy” to argue against the presence of PoC and other minority groups in historical/historically-inspired fiction, it’s because the picture you’ve been fed of that historical time has been laundered to suit a certain racist/homophobic/ableist ideology. Or it means you yourself are a racist/homophobe/ableist who wants to hide behind the guise of “historical inaccuracy” to attack diverse fiction. Either way, you’re wrong.
So, bringing this back to fantasy: Even if your fantasy story does relate to a real-world event, country, concept, law, or history, you still don’t have the excuse of “historical inaccuracy” to avoid making your story diverse.
(This is not to take away from the point OP was making – they’re absolutely right. I just wanted to take it one step further.)