No way to permit empty on file upload

"permit_empty" is a rule for classic field form.
For file uploads, you need to use only Rules for File Uploads 
The rule to say if the file is required or not is : "uploaded".
If the file is required, add the rule : uploaded[file_name].
If it is optional, just do not add this rule.