Finally I've found what was wrong with my configuration. I looked in my site.ini.append.php in my mysite\settings\override folder and found a RegionalSettings block which was declared with french (fre-FR) as Locale. So I removed totally from that file the [RegionalSettings] block (because everything is declared for that block in my mysite\extension\ezwebin\settings\siteaccess\fre\site.ini.append.php and mysite\extension\ezwebin\settings\siteaccess\eng\site.ini.append.php.
I went then in the site.ini.append.php which is in mysite\extension\ezwebin\settings and reduced the content of the [RegionalSettings] block to the following :
