I've now rebuilt php with --enable-memory-limit so that I can track memory usage. The results are rather alarming!
A really basic page (already view cached & compiled) uses 9460KB of memory and the time bottleneck I'm seeing coincides with an increased usage from 2247KB to 5570KB. More complex pages/templates use even more!
Does eZPublish really need to use this much memory?