I had the same problem as yours - I needed different blocks for the different siteaccesses in my project. The first thing that I did was to put my custom block.ini files in each of the siteaccesses. I think in your case you already did that as you put them respectively in ez403/settings/siteaccess/my_site/ and ez403/settings/siteaccess/my_site_admin/. Now, try to comment the defined blocks in the block.ini file of the ezflow extension. This way that file won't override your custom block.ini files as the system will take only your blocks definitions, but not those in the extension. That helped in my case.
I hope my post could be useful, but in case you already found another more efficient solution, I'll appreciate if you share it in the forum!