Molecular strong coupling presents exciting opportunities in physics, chemistry, and materials science. While significant attention has been given to modeling the molecular systems involved, the crucial role of the photonic mode structure of optical cavities has been relatively underexplored. Our research demonstrates that the effectiveness of molecular strong coupling is critically dependent on the cavity's finesse. Specifically, emission associated with a dispersive lower polariton is observed only in cavities with sufficiently high finesse. By developing an analytical model of cavity photoluminescence in a multimode structure, we elucidate the impact of finite finesse on polariton formation, showing that a lower finesse diminishes the mixing of light and matter in polariton states. We propose that understanding the detailed characteristics of the photonic modes supported by a cavity is as important for developing a coherent framework for molecular strong coupling as incorporating realistic molecular models.