Topological triple-vortex lattice stabilized by mixed frustration in expanded honeycomb Kitaev-Heisenberg model

The expanded classical Kitaev-Heisenberg model on a honeycomb lattice is investigated with the next-nearest-neighboring Heisenberg interaction considered. The simulation shows a rich phase diagram with periodic behavior in a wide parameter range. Beside the double 120° ordered phase, an inhomogeneous phase is uncovered to exhibit a topological triple-vortex lattice, corresponding to the hexagonal domain structure of vector chirality, which is stabilized by the mixed frustration of two sources: the geometrical frustration arising from the lattice structure as well as the frustration from the Kitaev couplings.