We explore the magnetically ordered ground state of the isovalently substituted Mott insulator Y1−xLaxTiO3 for x≤ 0.3 via single-crystal growth, magnetometry, neutron diffraction, x-ray magnetic circular dichroism, muon spin rotation, and small-angle neutron scattering (SANS). We find that the decrease in the magnetic transition temperature on approaching the ferromagnetic-antiferromagnetic phase boundary at the La concentration xc≈0.3 is accompanied by a strong suppression of both bulk and local ordered magnetic moments, along with a volumewise separation into magnetically ordered and paramagnetic regions. The thermal phase transition does not show conventional second-order behavior since neither a clear signature of dynamic critical behavior nor a power-law divergence of the magnetic correlation length is found for the studied substitution range; this finding becomes increasingly obvious with increasing La substitution. We find no evidence for a spin-glass phase. Finally, from SANS and magnetometry measurements, we discern a crossover from easy-axis to easy-plane magnetocrystalline anisotropy with increasing La substitution. These results indicate complex changes in magnetic structure upon approaching the phase boundary.