Molecular orientation at the donor-acceptor interface of organic solar cells has been suggested to play an important role in determining photovoltaic performance, primarily due to varied charge transfer state interactions between the donor and acceptor. Here we present a comprehensive study on the effects and consequences of molecular orientation in organic solar cells. As a model system, we prepare bilayer solar cells with the molecular donor p-SIDT(FBTTh2)2, which has achieved high photoconversion efficiencies of up to 6.5% in bulkheterojunctions. Through the addition of a small amount of diiodooctane to a solution of pristine p-SIDT(FBTTh2)2 we are able to control the solid-state molecular orientation of the donor molecule, giving us the unique opportunity to study the impact of molecular orientation on the photovoltaic characteristics while keeping other variable equivalent between the two cases. The molecular orientation and bilayer quality are extensively characterized by means of transmission electron microscopy (TEM), cross-section TEM, and x-ray diffraction. The photovoltaic response of the bilayers is then understood by studying the energetics, charge transport, charge generation and recombination as a function of molecular orientation. Most strikingly, the open circuit voltage (VOC) of the bilayers increases by up to 200 mV from the edge-on case to the face-on case. It has been shown that the energy of the charge transfer state (CTS) strongly correlates with the VOC under illumination. However, in this case, the energy differences of the CTS, as determined both by luminescence and by external quantum efficiency, are not enough to explain the shifts observed in VOC. Instead, we observe that the varied molecular orientation allows us to access different energy levels that correlate, though overshoot, the shifts in VOC, implying there are more properties that must be taken into account to understand the VOC than what has been common practice to date. Interestingly, we also observe a strong electroluminescence from the singlet state of the donor molecule when very low applied voltages are applied to the face-on bilayer that is not present in the edge-on bilayer, suggesting of more intrinsic differences in the donor-acceptor charge-transfer interaction between the two orientations.