The choice between residing in tall buildings and in few-story or detached dwellings has been the subject of considerable debate, with each approach attracting its own proponents and critics. This study investigates and compares the heating and cooling energy consumption of multi-story and low-elevation buildings, incorporating the Fanger comfort index as a measure of thermal comfort. Energy performance is evaluated on eight building configurations with varying numbers of floors, under three distinct climatic conditions, using the Predicted Mean Vote (PMV) index as the primary comfort criterion. The building scenarios range from single-story detached houses to 50-story high-rise structures. The climatic cases—representing hot (Yazd), moderate (Arak), and cold (Shahr-e Kord) environments—are simulated and analyzed using DesignBuilder software. The results section presents detailed analyses of heating and cooling loads, comfort index values, and electricity and gas demand for each building–climate combination, with monthly and annual performance trends. The findings reveal that the number of shared walls exerts a greater influence on energy consumption than the number of floors. Specifically, detached single-story buildings, which lack shared walls, exhibit up to 58.9% higher heating demand and 67.1% higher cooling demand compared to their counterparts with shared walls.