If you're running version 3.2, there's a bug in the cache clearing via the admin interface. Read this thread for more information: http://ez.no/developer/ez_publish_3/forum/install_configuration/cache_wont_clear_in_32
If you're working alot on the templates, it's probably better to turn off all template caching (NodeTreeCaching, TemplateCompile and TemplateCache under TemplateSettings in site.ini) so you don't have to worry about clearing the cache.
If you enable the template debug, and nothing changes in the layout, you can be sure it's a cache problem.