Inorganic solid-state electrolytes (SSEs), especially Li7La3Zr2O12 (LLZO), are promising candidates for all solid-state batteries. Generally, SSEs are synthesized by solid-state reactions at high temperatures. The high temperature required to synthesize useful polymorphs (e.g., cubic phase, in the case of LLZO) increases their production cost. To lower the synthesis temperature, several approaches, including doping and crystallite size control via sol-gel processes, have been explored. Recently, electrospinning was used to synthesize the inorganic SSE fibers to control the crystallite size and lower the calcination temperature. Several publications followed that investigated the properties of fibers and demonstrated the use of these fibers in inorganic-polymer composite SSEs. This paper presents a concise discussion of electrospinning, the parameters that control fiber morphology, the synthesis of inorganic SSE fibers, the effect of sintering conditions on the fiber morphology and composition, and the recent use of these fibers in making composite SSEs and other battery applications. Most of the discussion focuses on LLZO, but this paper also discusses applications of other types of electrospun inorganic electrolyte fibers, the challenges of using these fibers, and possible future directions in this research field.