Perylene diimides (PDIs) are attractive chromophores that exhibit singlet exciton fission (SF) and have several advantages over traditional SF molecules such as tetracene and pentacene; however, their photophysical properties relating to SF have received only limited attention. In this study, we explore how introduction of bulky bromine atoms in the so-called bay-area PDIs, resulting in a nonplanar structure, affects the solid-state packing and efficiency of singlet fission. We found that changes in the molecular packing have a strong effect on the temperature dependent photoluminescence, expressed as an activation energy. These effects are explained in terms of excimer formation for PDIs without bay-area substitution, which competes with singlet fission. Introduction of bromine atoms in the bay-positions strongly disrupts the solid-state packing leading to strongly reduced excitonic interactions. Surprisingly, these relatively amorphous materials with weak electronic coupling exhibit stronger formation of triplet excited states by SF because the competing excimer formation is suppressed here.