The ability to controllably position single atoms inside materials is key for the ultimate fabrication of devices with functionalities governed by atomic-scale properties. Single bismuth dopant atoms in silicon provide an ideal case study in view of proposals for single-dopant quantum bits. However, bismuth is the least soluble pnictogen in silicon, meaning that the dopant atoms tend to migrate out of position during sample growth. Here, we demonstrate epitaxial growth of thin silicon films doped with bismuth. We use atomic-resolution aberration-corrected imaging to view the as-grown dopant distribution and then to controllably position single dopants inside the film. Atomic-scale quantum-mechanical calculations corroborate the experimental findings. These results indicate that the scanning transmission electron microscope is of particular interest for assembling functional materials atom-by-atom because it offers both real-time monitoring and atom manipulation. We envision electron-beam manipulation of atoms inside materials as an achievable route to controllable assembly of structures of individual dopants.