If a user -- whether internal or external -- has only View rights to a folder, that user should only be able to view/download content from that folder and not add content to it. (If a user has Edit rights to a specific document in that folder, then they can modify the document or add a new version to it.)
Can you double check to make sure that you have given these users view-only access to the folders? If you have double checked this and view-only users can still upload documents, please open a ticket with Support providing more information about this situation and we can investigate.
Note that the Cabinet Inbox is an exception and any user with access to a cabinet can add documents to the Inbox.