File Upload ValidatorZach MosheRotem Naar
File upload vulnerabilities overview FUV – detailed overview Live demonstration In the future…
Agenda
Many applications take advantage of the band width available today and allow users to upload file, either for storage or usage within the flow of the software. This allows the software to be more appealing and interactive with the user
The uploaded file is a “jack in the box”. It may convey all sorts of trouble within, from viruses to extremely large sizes
Background
Avoid vulnerabilitiesSafe file upload principals
Check file type◦ Avoid dangerous extensions◦ Validate MIME-type
Safe file upload principals
Module
Use random filename◦ Avoid XSS attacks◦ Avoid file inclusion attacks
Safe file upload principals
Module Utility
Keep upload directory security
Safe file upload principals
Module
Scan file with AntiVirus
Safe file upload principals
Module
Limit file size◦ Avoid DoS attack
Safe file upload principals
ModuleUtility
FUV packageDesign and Details
Java package, which exposes an API that allows file validation through a single validate(file) method
The application is configured by an XML file that the caller supplies. Only relevant modules will be enabled
Utilities for application developer Using Java 1.6
Spec
Design
FUV package
Validation modules
Utils
After the file is uploaded
Before/While uploading the file
Validation modules
Design – validation modules
FileValidator<< interface>>
boolean Validate(File)
FileValidatorImpl
Module<< interface>>
boolean Validate(File)
File TypeModule
File NameModule
UNIX File Permissions
Module
Anti virusModule
*
The primary interface of the system ◦ public boolean validate(File file)
Holds set of modules Returns true if all configured modules
approved the file according to their configuration
If at least one of the modules rejected the file, the method returns false
Design - FileValidator
Open archive/compressed files and check the inner files using the modules
In case one of the inner files is archive/compressed file too, the same operation is done recursively
The maximum file depth allowed is configured in the XML configuration file
Opens archive/compressed files using Apache-Commons-Compress package
Supported formats: ZIP, TAR, GZIP, BZIP2
Design - FileValidator
The main operation: public boolean validate(File file)
All modules have: “scanInnerFiles” attribute (“true” by default) unique configuration
In case “scanInnerFiles” is “true” and the validated file is archive/compressed file, the module will scan the inner files too
Design - Module
Validates file types according to a predefined set of accepted MIME types (white-list validation)
Uses Apache-Tika package for content analysis of the file
Configuration:◦ Allowed types◦ Force extension check
Design – File Type Module
Can be enabled only in UNIX environment Validates that the file on the server has the
appropriate permissions The module is configured by 3 “maximal”
allowed permissions for the user, group and all (similar to UNIX file permissions)
Using ls UNIX command
Design - UNIX Permissions Module
Validates filename strings Configuration:
◦ Filename length◦ Allowed character strips – from the strips
configured in the system (white-list validation)
Design – Filename Module
Uses an external program as an AntiVirus Approves/Rejects the file according to its
return code Configuration:
◦ AntiVirus path◦ Success return code
We’re using Clam-AV
Design - AntiVirus Module
Design - Demonstration
FileValidatorModule
ModuleModule
File File
TrueFalse False
Design - Demonstration
FileValidatorModule
ModuleModule
File File
TrueFalse True
If archive/comressed: Foreach inner file: send to validation
False
Design
FUV package
Validation modules
Utils
After the file is uploaded
Before/While uploading the file
Utils
Design - utils
SizeBoundedInputStream extends InputStream
• Read()
• hasReachedLimit()
FileNameGenerator
• String generateNewRandomFilename()• String censorFilename(String filename)
Allow the user generate safe filenames Contains 2 methods:
1. censorFilename(String fileName) Censors given filename: limits the filename length and removes not-allowed charactersConfiguration:
filename length Allowed characters strips
2. generateNewRandomFilename() Generates random filename according to the configured patternConfiguration: filename pattern
File Name Generator
Creates safe way to upload a file without a problem with its size
Extends InputStream and warps the original InputStream
In case the number reached the maximum allowed, it returns -1 (EOF) and set the limitReached flag to “true”
Configuration: maximum size allowed
Size Bounded Input Stream
Configure engine, modules and utilities parameters
XML Configuration file
<file-validator-config>
<application-name>Application Name</application-name>
<archive-recursion-depth>7</archive-recursion-depth>
<modules>…</modules>
<file-name-generator> …
</file-name-generator>
<max-file-size>1024</max-file-size>
<char-strips> …
</char-strips>
<types-collections> …
</types-collections></file-validator-config>
XML Configuration file
<modules><!-- File name module --><file-name-module>
<max-file-name-length>50</max-file-name-length><allowedCharStrips>D C O</allowedCharStrips>
</file-name-module>
<!-- Anti Virus module --><anti-virus-module scanInnerFiles="false">
<anti-virus-path>bin/av_wrapper.sh</anti-virus-path><success-rc>0</success-rc>
</anti-virus-module>
<!-- File type module --><file-type-module>
<allowed-types>word text application/x-gzip </allowed-types><force-ext-check/>
</file-type-module>
<!-- File permissions module--><unix-file-permissions-module scanInnerFiles="false">
<user-max-permissions>rwx</user-max-permissions><group-max-permissions>r-x</group-max-permissions><all-max-permissions>r-x</all-max-permissions>
</unix-file-permissions-module></modules>
XML Configuration file
<types-collections>
<types-collection name="word"><type allowed-exts="doc">application/x-tika-msoffice</type><type allowed-exts="doc">application/msword</type><type allowed-exts="dotx,docx">application/x-tika-ooxml</type><type allowed-exts="docx">application/vnd.openxmlformats-
officedocument.wordprocessingml.document</type><type allowed-exts="dotx">application/vnd.openxmlformats-
officedocument.wordprocessingml.template</type></types-collection>
<types-collection name="text"><type allowed-exts="rtf">application/rtf</type><type allowed-exts="txt">text/plain</type>
</types-collection>
</types-collections>
Logging
2011-03-04 18:51:01,859 INFO [main] c.a.f.c.FileValidatorImpl [FileValidatorImpl.java:63] Validating file : C:\tmp_rotem\tmp\out.zip2011-03-04 18:51:01,859 DEBUG [main] c.a.f.c.FileValidatorImpl [FileValidatorImpl.java:68] Validating module com.amdocs.filevalidator.modules.FileNameModule2011-03-04 18:51:01,875 DEBUG [main] c.a.f.m.FileNameModule [FileNameModule.java:61] File name length (excluding extension) is 3. Maximum length allowed: 502011-03-04 18:51:01,875 DEBUG [main] c.a.f.m.FileNameModule [FileNameModule.java:81] Allowed chars: 0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz_-)(2011-03-04 18:51:01,875 DEBUG [main] c.a.f.c.FileValidatorImpl [FileValidatorImpl.java:68] Validating module com.amdocs.filevalidator.modules.FileTypeModule2011-03-04 18:51:01,875 DEBUG [main] c.a.f.m.FileTypeModule [FileTypeModule.java:61] FileTypeModule was called for out.zip2011-03-04 18:51:01,875 DEBUG [main] c.a.f.m.FileTypeModule [FileTypeModule.java:65] AllowedTypes are {application/x-tika-msoffice=[doc], image/jpeg=[jpg, jpeg], text/plain=null, application/x-bzip2=null, application/x-gtar=null, application/vnd.openxmlformats-officedocument.wordprocessingml.document=[docx], application/msword=[doc], application/x-gzip=null, application/x-tika-ooxml=[docx], application/zip=null}2011-03-04 18:51:02,296 DEBUG [main] c.a.f.m.FileTypeModule [FileTypeModule.java:90] content type is application/zip2011-03-04 18:51:02,296 DEBUG [main] c.a.f.m.FileTypeModule [FileTypeModule.java:93] forcing ext check2011-03-04 18:51:02,343 INFO [main] c.a.f.c.FileValidatorImpl [FileValidatorImpl.java:110] Found ZIP file2011-03-04 18:51:02,343 INFO [main] c.a.f.c.FileValidatorImpl [FileValidatorImpl.java:323] Entry: cfvxcbcf.txt…
XML Configuration – using JAXB Logging - using SLF4J and LogBack Unit Testing Code Examples Building the project – using Maven Version Control – using SVN JAR, sources and documents can be found
on: http://code.google.com/p/fuv/
Quality
DemonstrationValidate files using FUV package
In the FutureHow to improve the project
Add support in client side (JavaScript/PHP packages)
Add module for special treatment to images (malicious code inside image)
Create secure upload server using the FUV package
DoS Attack – limit the size and number of files one user can upload in a given period (track the user using cookies or IP)
In the Future
Thank You!