Biofuels produced from lignocellulosic biomass hold great promise as a renewable alternative energy and fuel source. To realize a cost and energy efficient approach, a fundamental understanding of the deconstruction process is critically necessary to reduce biomass recalcitrance. Herein, the structural and morphological changes over multiple scales (5–6000 Å) in herbaceous (switchgrass) and woody (hybrid poplar) biomass during dilute sulfuric acid pretreatment were explored using neutron scattering and X-ray diffraction. Switchgrass undergoes a larger increase (20–84 Å) in the average diameter of the crystalline core of the elementary cellulose fibril than hybrid poplar (19–50 Å). Switchgrass initially forms lignin aggregates with an average size of 90 Å that coalesce to 200 Å, which is double that observed for hybrid poplar, 55–130 Å. Switchgrass shows a smooth-to-rough transition in the cell wall surface morphology unlike the diffuse-to-smooth transition of hybrid poplar. Yet, switchgrass and hybrid poplar pretreated under the same experimental conditions result in pretreated switchgrass producing higher glucose yields (∼76 wt %) than pretreated hybrid poplar (∼60 wt %). This observation shows that other aspects like cellulose allomorph transitions, cellulose accessibility, cellular biopolymer spatial distribution, and enzyme–substrate interactions may be more critical in governing the enzymatic hydrolysis efficiency.