The recent gravitational wave observations of the collision of black holes and neutron stars have allowed us to pierce into the extreme gravity regime, where gravitational fields are unfathomably strong and wildly dynamical. Some of these observations can be combined with electromagnetic observations of the light emitted by isolated and by merging neutron stars. Such multi-messenger observations encode a trove of information that is prime for the taking, including potential revelations about the validity of Einstein's theory in strong gravity and the properties of matter at supranuclear densities. In this talk, I will describe how we can use multi-messenger observations to put Einstein's theory to the test and to learn about nuclear physics in the extreme gravity regime.