I chose a modular design for my site, and as a performance test, I took one of my page layout templates and combined all of the includes into one template.
The result was that the non-modular template was approximately .1 seconds faster in processing times.
This was before I had enabled cache blocks, and it rendered in 3 seconds with modular template (about 12 different templates called) and 2.9 seconds when I combined everything into one template.
After enabling cache blocks, load times are down to .8 seconds on average.
I left it modular, as the slight performance improvement was not worth the development headache of having everything combined.