File Upload ValidatorZach MosheRotem Naar

File upload vulnerabilities overview FUV – detailed overview Live demonstration In the future…

Agenda

Many applications take advantage of the band width available today and allow users to upload file, either for storage or usage within the flow of the software. This allows the software to be more appealing and interactive with the user

The uploaded file is a “jack in the box”. It may convey all sorts of trouble within, from viruses to extremely large sizes

Background

Avoid vulnerabilitiesSafe file upload principals

Check file type◦ Avoid dangerous extensions◦ Validate MIME-type

Safe file upload principals

Module

Use random filename◦ Avoid XSS attacks◦ Avoid file inclusion attacks

Safe file upload principals

Module Utility

Keep upload directory security

Safe file upload principals

Module

Scan file with AntiVirus

Safe file upload principals

Module

Limit file size◦ Avoid DoS attack

Safe file upload principals

ModuleUtility

FUV packageDesign and Details

Java package, which exposes an API that allows file validation through a single validate(file) method

The application is configured by an XML file that the caller supplies. Only relevant modules will be enabled

Utilities for application developer Using Java 1.6

Spec

Design

FUV package

Validation modules

Utils

After the file is uploaded

Before/While uploading the file

Validation modules

Design – validation modules

FileValidator<< interface>>

boolean Validate(File)

FileValidatorImpl

Module<< interface>>

boolean Validate(File)

File TypeModule

File NameModule

UNIX File Permissions

Module

Anti virusModule

*

The primary interface of the system ◦ public boolean validate(File file)

Holds set of modules Returns true if all configured modules

approved the file according to their configuration

If at least one of the modules rejected the file, the method returns false

Design - FileValidator

Open archive/compressed files and check the inner files using the modules

In case one of the inner files is archive/compressed file too, the same operation is done recursively

The maximum file depth allowed is configured in the XML configuration file

Opens archive/compressed files using Apache-Commons-Compress package

Supported formats: ZIP, TAR, GZIP, BZIP2

Design - FileValidator

The main operation: public boolean validate(File file)

All modules have: “scanInnerFiles” attribute (“true” by default) unique configuration

In case “scanInnerFiles” is “true” and the validated file is archive/compressed file, the module will scan the inner files too

Design - Module

Validates file types according to a predefined set of accepted MIME types (white-list validation)

Uses Apache-Tika package for content analysis of the file

Configuration:◦ Allowed types◦ Force extension check

Design – File Type Module

Can be enabled only in UNIX environment Validates that the file on the server has the

appropriate permissions The module is configured by 3 “maximal”

allowed permissions for the user, group and all (similar to UNIX file permissions)

Using ls UNIX command

Design - UNIX Permissions Module

Validates filename strings Configuration:

◦ Filename length◦ Allowed character strips – from the strips

configured in the system (white-list validation)

Design – Filename Module

Uses an external program as an AntiVirus Approves/Rejects the file according to its

return code Configuration:

◦ AntiVirus path◦ Success return code

We’re using Clam-AV

Design - AntiVirus Module

Design - Demonstration

FileValidatorModule

ModuleModule

File File

TrueFalse False

Design - Demonstration

FileValidatorModule

ModuleModule

File File

TrueFalse True

If archive/comressed: Foreach inner file: send to validation

False

Design

FUV package

Validation modules

Utils

After the file is uploaded

Before/While uploading the file

Utils

Design - utils

SizeBoundedInputStream extends InputStream

• Read()

• hasReachedLimit()

FileNameGenerator

• String generateNewRandomFilename()• String censorFilename(String filename)

Allow the user generate safe filenames Contains 2 methods:

1. censorFilename(String fileName) Censors given filename: limits the filename length and removes not-allowed charactersConfiguration:

filename length Allowed characters strips

2. generateNewRandomFilename() Generates random filename according to the configured patternConfiguration: filename pattern

File Name Generator

Creates safe way to upload a file without a problem with its size

Extends InputStream and warps the original InputStream

In case the number reached the maximum allowed, it returns -1 (EOF) and set the limitReached flag to “true”

Configuration: maximum size allowed

Size Bounded Input Stream

Configure engine, modules and utilities parameters

XML Configuration file

<file-validator-config> 

<application-name>Application Name</application-name>

<archive-recursion-depth>7</archive-recursion-depth>

<modules>…</modules>

<file-name-generator> …

</file-name-generator>

<max-file-size>1024</max-file-size>

<char-strips> …

</char-strips> 

<types-collections> …

</types-collections></file-validator-config>

XML Configuration file

<modules><!-- File name module --><file-name-module>

<max-file-name-length>50</max-file-name-length><allowedCharStrips>D C O</allowedCharStrips>

</file-name-module>

<!-- Anti Virus module --><anti-virus-module scanInnerFiles="false">

<anti-virus-path>bin/av_wrapper.sh</anti-virus-path><success-rc>0</success-rc>

</anti-virus-module>

<!-- File type module --><file-type-module>

<allowed-types>word text application/x-gzip </allowed-types><force-ext-check/>

</file-type-module> 

<!-- File permissions module--><unix-file-permissions-module scanInnerFiles="false">

<user-max-permissions>rwx</user-max-permissions><group-max-permissions>r-x</group-max-permissions><all-max-permissions>r-x</all-max-permissions>

</unix-file-permissions-module></modules>

XML Configuration file

<types-collections>

<types-collection name="word"><type allowed-exts="doc">application/x-tika-msoffice</type><type allowed-exts="doc">application/msword</type><type allowed-exts="dotx,docx">application/x-tika-ooxml</type><type allowed-exts="docx">application/vnd.openxmlformats-

officedocument.wordprocessingml.document</type><type allowed-exts="dotx">application/vnd.openxmlformats-

officedocument.wordprocessingml.template</type></types-collection>

<types-collection name="text"><type allowed-exts="rtf">application/rtf</type><type allowed-exts="txt">text/plain</type>

</types-collection>

</types-collections>

Logging

2011-03-04 18:51:01,859 INFO [main] c.a.f.c.FileValidatorImpl [FileValidatorImpl.java:63] Validating file : C:\tmp_rotem\tmp\out.zip2011-03-04 18:51:01,859 DEBUG [main] c.a.f.c.FileValidatorImpl [FileValidatorImpl.java:68] Validating module com.amdocs.filevalidator.modules.FileNameModule2011-03-04 18:51:01,875 DEBUG [main] c.a.f.m.FileNameModule [FileNameModule.java:61] File name length (excluding extension) is 3. Maximum length allowed: 502011-03-04 18:51:01,875 DEBUG [main] c.a.f.m.FileNameModule [FileNameModule.java:81] Allowed chars: 0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz_-)(2011-03-04 18:51:01,875 DEBUG [main] c.a.f.c.FileValidatorImpl [FileValidatorImpl.java:68] Validating module com.amdocs.filevalidator.modules.FileTypeModule2011-03-04 18:51:01,875 DEBUG [main] c.a.f.m.FileTypeModule [FileTypeModule.java:61] FileTypeModule was called for out.zip2011-03-04 18:51:01,875 DEBUG [main] c.a.f.m.FileTypeModule [FileTypeModule.java:65] AllowedTypes are {application/x-tika-msoffice=[doc], image/jpeg=[jpg, jpeg], text/plain=null, application/x-bzip2=null, application/x-gtar=null, application/vnd.openxmlformats-officedocument.wordprocessingml.document=[docx], application/msword=[doc], application/x-gzip=null, application/x-tika-ooxml=[docx], application/zip=null}2011-03-04 18:51:02,296 DEBUG [main] c.a.f.m.FileTypeModule [FileTypeModule.java:90] content type is application/zip2011-03-04 18:51:02,296 DEBUG [main] c.a.f.m.FileTypeModule [FileTypeModule.java:93] forcing ext check2011-03-04 18:51:02,343 INFO [main] c.a.f.c.FileValidatorImpl [FileValidatorImpl.java:110] Found ZIP file2011-03-04 18:51:02,343 INFO [main] c.a.f.c.FileValidatorImpl [FileValidatorImpl.java:323] Entry: cfvxcbcf.txt…

XML Configuration – using JAXB Logging - using SLF4J and LogBack Unit Testing Code Examples Building the project – using Maven Version Control – using SVN JAR, sources and documents can be found

on: http://code.google.com/p/fuv/

Quality

DemonstrationValidate files using FUV package

In the FutureHow to improve the project

Add support in client side (JavaScript/PHP packages)

Add module for special treatment to images (malicious code inside image)

Create secure upload server using the FUV package

DoS Attack – limit the size and number of files one user can upload in a given period (track the user using cookies or IP)

In the Future

Thank You!