Color centers in nanodiamonds offer light-matter coupling rates in the near-THz range, and are compatible with any photonic platform. Due to these unusually fast dynamics, they constitute promising building blocks for high-bandwidth quantum photonics, potentially at non-cryogenic temperatures. However, the properties of the color centers and the nanodiamonds themselves are heterogenous. To interface these emitters with the on-chip photonic circuitry one must pre-select and deterministically manipulate them with nanoscale precision. I will present a study of fundamental plasmon-enhanced emission rate limits in quantum emitters and a suite of recently developed techniques for realizing deterministically assembled plasmon-enhanced single-photon sources. These techniques include rapid automatic focusing, optical nanoparticle sizing, neural network-driven quantum optical measurements, probe-assisted nanoantenna assembly and the optical control of plasmonic cavity mode volume. I will also briefly discuss a new research direction aiming at achieving electro-optic microwave-to-optical quantum transduction via terahertz states.