It's already been explained it's likely because of the inbreeding, same thing for the Targaryians.