Holding at room temperature is the first step in most boar semen cryopreservation protocols. It is well accepted that a holding time (HT) of 24 h increases sperm cryotolerance. However, the effect of HT on ejaculates with different freezability is not entirely clear. The aim of this study was to understand how HT influences spermatic and seminal plasma metabolite profiles of boar ejaculates and how these possible changes affect freezability. Twenty-seven ejaculates were collected and extended to 1:1 (v: v) with BTS and split into two aliquots. The first aliquot was cryopreserved without holding time (0 h), and the second was held at 17°C for 24 h before cryopreservation. Spermatozoa and seminal plasma were collected by centrifugation at two times, before HT (0 h) and after HT (24 h), and subsequently frozen until metabolite extraction and UPLC–MS analysis. After thawing, the semen samples were evaluated for kinetics, membrane integrity, mitochondrial potential, membrane lipid peroxidation, and fluidity. The ejaculates were then allocated into two phenotypes (good ejaculate freezers [GEF] and poor ejaculate freezers [PEF]) based on the percent reduction in sperm quality (%RSQ) as determined by the difference in total motility and membrane integrity between raw and post-thaw samples cryopreserved after 24 h of HT. The metabolic profile of the seminal plasma did not seem to influence ejaculate freezability, but that of the spermatozoa were markedly different between GEF and PEF. We identified a number of metabolic markers in the sperm cells (including inosine, hypoxanthine, creatine, ADP, niacinamide, spermine, and 2-methylbutyrylcarnitine) that were directly related to the improvement of ejaculate freezability during HT; these were components of metabolic pathways associated with energy production. Furthermore, PEF showed an up-regulation in the arginine and proline as well as the glutathione metabolism pathways. These findings help to better understand the effect of holding time on boar sperm freezability and propose prospective metabolic markers that may predict freezability; this has implications in both basic and applied sciences.