The capacitance that primarily affects the high-frequency response of a Common Emitter (CE) amplifier is the collector-gate capacitance (\( C_{gd} \)). This capacitance causes the Miller effect which greatly reduces the high-frequency response of the amplifier by feeding back amplified output to input resulting in lower bandwidth of the amplifier.