Abstract
Since its discovery, iron-based superconductivity has been known to develop near an antiferromagnetic order, but this paradigm fails in the iron chalcogenide FeSe, whose single-layer version holds the record for the highest superconducting transition temperature in the iron-based superconductors. The striking puzzle that FeSe displays nematic order (spontaneously broken lattice rotational symmetry) while being nonmagnetic, has led to several competing proposals for its origin in terms of either the 3d-electron's orbital degrees of freedom or spin physics in the form of frustrated magnetism. Here we argue that the phase diagram of FeSe under pressure could be qualitatively described by a quantum spin model with highly frustrated interactions. We implement both the site-factorized wave-function analysis and the large-scale density matrix renormalization group (DMRG) in cylinders to study the spin-1 bilinear-biquadratic model on the square lattice, and identify quantum transitions from the well-known (π,0) antiferromagnetic state to an exotic (π,0) antiferroquadrupolar order, either directly or through a (π/2,π) antiferromagnetic state. These many phases, while distinct, are all nematic. We also discuss our theoretical ground-state phase diagram for the understanding of the experimental low-temperature phase diagram obtained by the NMR [Wang et al., Phys. Rev. Lett. 117, 237001 (2016)] and x-ray scattering [Kothapalli et al., Nat. Commun. 7, 12728 (2016)] measurements in pressurized FeSe. Our results suggest that superconductivity in a wide range of iron-based materials has a common origin in the antiferromagnetic correlations of strongly correlated electrons.
