Censorship-Resistant Publishing Systems
Marc WaldmanMarc Waldman
Computer Science DepartmentComputer Science Department
New York UniversityNew York University
What is a Censorship-Resistant Publishing System?
A system that maintains document availability in the
presence of adversaries who wish to suppress the
document.
Why Censorship-Resistant Publishing?
Political DissentPolitical Dissent
““Whistleblowing”Whistleblowing”
Human Rights ReportsHuman Rights Reports
Possible Solutions
Collection of WWW serversCollection of WWW servers
- CGI scripts to accept files- CGI scripts to accept files
- each file replicated on other participating - each file replicated on other participating serversservers
UsenetUsenet
- Send file to Usenet server- Send file to Usenet server
- Automatically replicated via NNTP- Automatically replicated via NNTP
Small group of WWW servers Censorship-resistant propertiesCensorship-resistant properties
- replication of content- replication of content
- multiple administrators- multiple administrators
ProblemsProblems
- Small static set of servers- Small static set of servers
- Flooding- Flooding
- Overwriting or deleting- Overwriting or deleting
- Name Squatting- Name Squatting
Usenet Censorship-resistant propertiesCensorship-resistant properties
- globally distributed - globally distributed (resists admin threats)(resists admin threats)
- huge capacity - huge capacity (resists storage flooding)(resists storage flooding)
ProblemsProblems
- published document (article) short lived- published document (article) short lived
- propagation time unpredictable- propagation time unpredictable
- no tamper check mechanism- no tamper check mechanism
- cancel/supercede requests- cancel/supercede requests
- easily filled with meaningless articles - easily filled with meaningless articles
Document Availability Threats Legal and illegal threats against server adminLegal and illegal threats against server admin
Adversarial content modificationAdversarial content modification
Document FloodingDocument Flooding
Legal and illegal threats against publisherLegal and illegal threats against publisher
Name SquattingName Squatting
Malicious hosting serversMalicious hosting servers
“Eternity Service” Proposal Worldwide collection of servers that store Worldwide collection of servers that store
documents documents (prevents legal threats)(prevents legal threats)
Publisher pays (anonymous e-cash) for document to Publisher pays (anonymous e-cash) for document to be published on random subset of servers be published on random subset of servers (prevents document flooding)(prevents document flooding)
Once published, document can’t be deletedOnce published, document can’t be deleted(prevents illegal threats against publisher)(prevents illegal threats against publisher)
Request and receive documents via anonymous Request and receive documents via anonymous communication channelcommunication channel (protects readers)(protects readers)
“Eternity Service” Design Challenges ServersServers
- Adding, removing, adversarial servers- Adding, removing, adversarial servers
Document NamingDocument Naming
- name squatting, updating, searching- name squatting, updating, searching
Replica PlacementReplica Placement
- efficient retrieval- efficient retrieval
“Eternity Service” Design Challenges Content StorageContent Storage
- File or block based storge, encryption- File or block based storge, encryption
Tamper ProtectionTamper Protection
- Detect malicious & accidental tampering- Detect malicious & accidental tampering
Untraceable Communication ChannelUntraceable Communication Channel
- “Real-time” or e-mail based- “Real-time” or e-mail based
Eternity Service Inspired Censorship-Resistant Systems
Design goals similar to Eternity ServiceDesign goals similar to Eternity Service Scaled down design, some implementations Scaled down design, some implementations
availableavailable- Janus - Janus - Rewebber- Rewebber- Usenet Eternity - Usenet Eternity - Freenet- Freenet- FreeHaven- FreeHaven- Publius- Publius- Tangler- Tangler
Janus Provides URL rewriting service to hide true Provides URL rewriting service to hide true
location of WWW pagelocation of WWW page Based on public key cryptographyBased on public key cryptography
EEkk (U)=Encrypt URL U with public key k (U)=Encrypt URL U with public key k
U=http://www.cs.nyu.edu/U=http://www.cs.nyu.edu/
Janus URL hides true location of UJanus URL hides true location of U
http://www.rewebber.de/surf-encrypted/Ehttp://www.rewebber.de/surf-encrypted/Ek(U)(U)
Janus acts as HTTP proxy, retrieving and Janus acts as HTTP proxy, retrieving and rewriting pages.rewriting pages.
Janus In Action
Internet
http://www.cs.nyu.edu
http://www.rewebber.de/surf-encrypted/Ehttp://www.rewebber.de/surf-encrypted/Ek(U)(U)
User
Janus
index.html
index.htmlwith URLs encrypted
1
2
3
4
Janus For Censorship-Resistant Publishing
Must trust Janus not to divulge true URLMust trust Janus not to divulge true URL
Not fault-tolerantNot fault-tolerant
- Janus URL encodes single server- Janus URL encodes single server
- Access available only through Janus- Access available only through Janus
Janus controls all returned contentJanus controls all returned content
- Content could be modified or censored- Content could be modified or censored
Taz and Rewebber Collection of volunteer serversCollection of volunteer servers
- Each has public/private key pair- Each has public/private key pair- Public keys well known to all users- Public keys well known to all users- Each runs a special HTTP proxy server- Each runs a special HTTP proxy server
URL to hide is encrypted using layered URL to hide is encrypted using layered techniquetechnique- Similar to onion-routing- Similar to onion-routing- Results in long URLs - Results in long URLs
TAZ servers translate names to URLsTAZ servers translate names to URLs
Server 1
Server 2
Server 3
Server 4
nyu.edu
Rewebber Layered Encryption
Server 5
http://VeryLongURLLongURL MediumURL SmallURL
Publisher uses public keys of servers to encrypt URL “nyu.edu”Want URL to be hidden behind 5 other servers.Encrypt in reverse path order (use public key of server 5 first)
Taz and Rewebber In Action
User 1. Apple_Pie_Recipe.taz TAZServer
2. http://VeryLongURL
LongURL
4MediumURL
5
SmallURL
3. http://VeryLongURL
ApplePie.com
6
7. get recipe.html
Rewebber For Censorship-Resistant Publishing
Do not need to trust single entityDo not need to trust single entity- Single coopering server hides true URL- Single coopering server hides true URL
Allows anonymous retrievalAllows anonymous retrieval- No limit on URL size- No limit on URL size- Padding can be applied after each decryption- Padding can be applied after each decryption
Not fault tolerant Not fault tolerant - Single faulty or malicious server can prevent - Single faulty or malicious server can prevent document from being retrieveddocument from being retrieved
No tamper protection mechanismNo tamper protection mechanism- A server can modify content on return trip- A server can modify content on return trip
Publius Collection of volunteer serversCollection of volunteer servers
- Each server donates disk space- Each server donates disk space- Runs script to interpret Publius commands- Runs script to interpret Publius commands
Publication process encrypts documentPublication process encrypts document- encrypted document stored on subset of servers- encrypted document stored on subset of servers- part of encryption key stored with document- part of encryption key stored with document
Publication process results in a Publius URLPublication process results in a Publius URL- Tells location of encrypted documents- Tells location of encrypted documents- Provides tamper check mechanism- Provides tamper check mechanism
Provides secure update and support for mutually Provides secure update and support for mutually hyperlinked contenthyperlinked content
Cryptographic HashA function that takes an arbitrary sized input and A function that takes an arbitrary sized input and
maps it to a fixed sized output value such thatmaps it to a fixed sized output value such that 1)1) It is computationally infeasible to find a specific It is computationally infeasible to find a specific
input that matches a pre-specified outputinput that matches a pre-specified output
2)2) It is computationally infeasible to find any two It is computationally infeasible to find any two distinct inputs that map to the same outputdistinct inputs that map to the same output
MD5 cryptographic hash output = 128 bitsMD5 cryptographic hash output = 128 bitsSHA-1 cryptographic hash output = 160 bitsSHA-1 cryptographic hash output = 160 bits
Publius Servers
whitehouse.gov
library.fr
publius.uk
www.redcross.org
www.nyu.edu
Publius Server Table
publius.ukpublius.uk
www.nyu.eduwww.nyu.edu
library.frlibrary.fr
whitehouse.govwhitehouse.gov
www.redcross.orgwww.redcross.org
Publish OperationD = Document To Publish K=Encryption Key
Shamir Secret Sharing
ShareShare11 ShareShare22 ShareShare33
K
ShareShare44
MD5 ( D . Sharei ) Mod 5 = Index Into Server Table
Index 3 = www.nyu.edu
Store D encrypted under K, and Sharei on www.nyu.edu
Publius URLCryptographic hash value determines location of document.
MD5 ( D . Sharei ) Mod 5 = Index Into Server Table
To Form Publius URL –Perform hash on each Share and concatenate resulting MD5 values.
http://!publius!/1e6adsg673h0=hgj7889340=yareyoureadingthis=12asbnm8945
The URL is cryptographically tied to document. Provides a tamper check mechanism.
Publius Retrieve Operation
Break apart URL to discover document Break apart URL to discover document locationslocations
Retrieve encrypted document and share Retrieve encrypted document and share from k locationsfrom k locations
Reassemble Key Reassemble Key KK from shares from shares Decrypt retrieved documentDecrypt retrieved document Check for tamperingCheck for tampering View in WWW browserView in WWW browser All work done by a client-side HTTP proxyAll work done by a client-side HTTP proxy
Publius For Censorship-Resistant Publishing
Fault tolerant – don’t need all shares or documents to Fault tolerant – don’t need all shares or documents to retrieve documentretrieve document
Tamper resistant – All documents retrieved from servers Tamper resistant – All documents retrieved from servers are checked for tamperingare checked for tampering
Encryption protects hides content from someone who Encryption protects hides content from someone who doesn’t know URL (including server admin)doesn’t know URL (including server admin)
Scalability problems – Everyone needs list of serversScalability problems – Everyone needs list of servers
Flooding can be a problem. Publius file size limit is 100K.Flooding can be a problem. Publius file size limit is 100K.
The Tangler Censorship-Resistant Publishing System
Designed to be a practical and implementable Designed to be a practical and implementable censorship-resistant publishing system.censorship-resistant publishing system.
Addresses some deficiencies of previous workAddresses some deficiencies of previous work
Contributions include –Contributions include –
- A unique publication mechanism called - A unique publication mechanism called entanglemententanglement
- The design of a self-policing storage network - The design of a self-policing storage network that ejects faulty nodes that ejects faulty nodes
Tangler Design Small group (<100) of volunteer serversSmall group (<100) of volunteer servers
Each server has public/private key pairEach server has public/private key pair
Each server donates disk space to system (publishing limit)Each server donates disk space to system (publishing limit)
Agreement on volunteer servers, public keys and donated Agreement on volunteer servers, public keys and donated disk spacedisk space
Published documents are divided into equal sized blocks, Published documents are divided into equal sized blocks, and combined with blocks of previously published and combined with blocks of previously published documents (documents (entanglemententanglement))
Entangled blocks are stored on servers Entangled blocks are stored on servers
Each server verifies other servers compliance with Tangler Each server verifies other servers compliance with Tangler
protocolsprotocols
Tangler Goals Anonymity – Users can publish and read documents Anonymity – Users can publish and read documents
anonymouslyanonymously
Document availability through replication Document availability through replication
Integrity guarantees on data (tamper & update)Integrity guarantees on data (tamper & update)
No server is storing objectionable documentsNo server is storing objectionable documents
- Decoupling between document and blocks- Decoupling between document and blocks
- Blocks not permanently tied to specific servers- Blocks not permanently tied to specific servers
- Server cannot chose which blocks to store or serve- Server cannot chose which blocks to store or serve
Misbehaving servers should be ejected from systemMisbehaving servers should be ejected from system
Publish Operation Document broken into Document broken into data blocksdata blocks
Data blocks transformed into Data blocks transformed into server blocksserver blocks
Server blocks combined with those of previously Server blocks combined with those of previously published server blocks (published server blocks (entanglemententanglement))
Entangled server blocks are stored on serversEntangled server blocks are stored on servers
++
Data Data BlocksBlocks
Previously Published Previously Published Server BlocksServer Blocks
New Server New Server BlocksBlocks
ServerServer BlocksBlocks
Document Retrieval Operation
Retrieve entangled server blocks from serversRetrieve entangled server blocks from servers
Entanglement is fault tolerant – don’t needEntanglement is fault tolerant – don’t needall entangled blocks to re-form data blocksall entangled blocks to re-form data blocks
DisEntangle Operation re-forms original data blocksDisEntangle Operation re-forms original data blocks
Data BlocksData Blocks
Entangled Entangled Server BlocksServer Blocks
Block Entanglement Algorithm Utilizes Shamir’s Secret Sharing AlgorithmUtilizes Shamir’s Secret Sharing Algorithm
- Given a secret - Given a secret SS can form can form nn shares shares
- Any - Any kk of them can re-form of them can re-form SS
- Less than - Less than kk shares provide no information about shares provide no information about SS
Entanglement is a secret sharing scheme with n=4 Entanglement is a secret sharing scheme with n=4 and k=3and k=3
Two shares are previously published server blocksTwo shares are previously published server blocks
Two additional shares are createdTwo additional shares are created
Benefits Of Entanglement Dissociates blocks served from documents Dissociates blocks served from documents
publishedpublished
- Single block belongs to multiple documents- Single block belongs to multiple documents
- Servers just hosting blocks- Servers just hosting blocks
IncentiveIncentive
- Cache server blocks of entangled documents- Cache server blocks of entangled documents
- Monitor availability of other server blocks- Monitor availability of other server blocks
- Re-inject blocks that have been deleted- Re-inject blocks that have been deleted
Tangler Servers (Tangle-Net) All servers fall into one of two categories –All servers fall into one of two categories –
non-faultynon-faulty = follow Tangler protocols = follow Tangler protocols
faultyfaulty = servers that exhibit Byzantine failures = servers that exhibit Byzantine failures
All All non-faultynon-faulty servers are synchronized to within servers are synchronized to within 10 minutes of correct time.10 minutes of correct time.
Time is divided into Time is divided into roundsrounds (24 hour period) (24 hour period)
- Round 0 = Jan 1, 2002 (12:00AM)- Round 0 = Jan 1, 2002 (12:00AM)
Fourteen consecutive rounds form an Fourteen consecutive rounds form an epochepoch
Tangler Round Round Activity (concurrent actions)Round Activity (concurrent actions)
- - Request storage tokens from other serversRequest storage tokens from other servers
- Grant storage tokens to other servers- Grant storage tokens to other servers
- Send and receive blocks - Send and receive blocks
- Monitor protocol compliance of other servers- Monitor protocol compliance of other servers
- Process join requests- Process join requests
- Entangle new collections and retrieve old collections- Entangle new collections and retrieve old collections
End of round End of round
- Commit to blocks received from servers (Merkle Tree) - Commit to blocks received from servers (Merkle Tree)
- Generate public/private key pair for the round- Generate public/private key pair for the round
- Broadcast next round commitment and public key- Broadcast next round commitment and public key
Storage Tokens Two step protocol to store blocksTwo step protocol to store blocks
First Step - Acquire storage tokensFirst Step - Acquire storage tokens
- Every server entitled to number of storage tokens - Every server entitled to number of storage tokens from every other serverfrom every other server
- Tokens acquired - Tokens acquired non-anonymouslynon-anonymously, requests are , requests are signed by requestorsigned by requestor
Second Step – Redeem TokenSecond Step – Redeem Token
- Send block & token anonymously to storing server- Send block & token anonymously to storing server
- Anonymous communication supported by Mix-Net- Anonymous communication supported by Mix-Net
Storage Token Request
Server BServer BServer AServer A
92180
XXXXXXXXXX
Server A
Server_A_Tokens--Server_A_Tokens--
XXXXXXXXXX
Server B
Unblind TokenUnblind Token
9218092180
Server A wants to store block 92180 on Server BServer A wants to store block 92180 on Server B Server A creates a blinded request for a tokenServer A creates a blinded request for a token The blinded request is sent to server BThe blinded request is sent to server B Server B signs the request and returns it to AServer B signs the request and returns it to A Server A unblinds request obtaining the tokenServer A unblinds request obtaining the token
Redeeming A Token Server A sends token & block through Server A sends token & block through
Mix-Net to BMix-Net to B Server B checks token signature, stores block, and Server B checks token signature, stores block, and
returns signed receipt over Mix-Netreturns signed receipt over Mix-Net Server B commits to hash tree of all blocksServer B commits to hash tree of all blocks
Mix-NetMix-Net
storage receipt storage receipt
block 92180block 92180
Server AServer A Server BServer B92180
Server B
Membership Changes At end of epoch all non-faulty servers perform At end of epoch all non-faulty servers perform
Byzantine Consensus algorithmByzantine Consensus algorithm
Each server can vote out any other members Each server can vote out any other members
New servers can join at any time but must serve as New servers can join at any time but must serve as a storage-only server for a probationary period of a storage-only server for a probationary period of two complete epochstwo complete epochs
A probationary server is admissible if it was not A probationary server is admissible if it was not ejectable for at least two consecutive epochs.ejectable for at least two consecutive epochs.
Majority vote winsMajority vote wins
Threats Majority of servers are adversarialMajority of servers are adversarial
- Adversarial servers join- Adversarial servers join
- Force non-faulty servers off- Force non-faulty servers off
Publishing server discoveryPublishing server discovery
- Force suspected server off network- Force suspected server off network
- Should be able to republish on another - Should be able to republish on another server but may not have same credit limitserver but may not have same credit limit
Probabilistic failure (difficult to remove)Probabilistic failure (difficult to remove)
Summary There is a need for censorship-resistant There is a need for censorship-resistant
publishing tools.publishing tools.
Several systems have been proposed and Several systems have been proposed and
some have been implemented.some have been implemented.
Each system has strength and weaknesses. Each system has strength and weaknesses. System design is greatly influenced by System design is greatly influenced by your adversary model. your adversary model.
Publius and Tangler URLs
PubliusPublius
www.cs.nyu.edu/~waldman/publius.htmlwww.cs.nyu.edu/~waldman/publius.html
TanglerTangler
www.scs.cs.nyu.edu/tanglerwww.scs.cs.nyu.edu/tangler