entropy accoun,ng Measuring change
in the data-‐to-‐informa,on value chain
that affects risk mi,ga,on and new value crea,on in networked informa,on infrastructures
serving systems in mul,ple sectors and jurisdic,ons
Sco= L. David
Execu,ve Director – Law, Technology and Arts Group University of Washington (Sea=le) School of Law
To the
MIT Media Lab Kerberos Conference October 7, 2013
Concept – Measuring risk and value in networked informa,on systems
• Social and economic rela,onships form systems using shared data resources over shared informa,on infrastructure
• Law and contracts document systems • Evaluate data-‐informa,on value chain using thermodynamics
to reveal new solu,ons and value • Map virtual landscape of rela,onships
From energy-‐ to informa,on-‐based economies
• 2 ways to transfer energy – Heat – Work
• Heat is transfer of energy by random mo+on of molecules
• Work is transfer of energy by non-‐random mo,on of molecules
Harnessing energy started with heat
• People stumbled on the value of heat energy early in our history
We learned to control heat flows • Heat is energy transfer by random molecular mo,on
• Heat value was first captured by regula,ng its flow, but it was s,ll energy in the form of heat
Energy as work
• It took thousands of years to figure out how to efficiently capture and transfer energy as work
The value of the transfer of energy as work was captured by the human mind and technology
• Work is energy tamed by observa,on, mathema,cs and technology of heat engine
• Heat engine requires – Hot energy source – “Engine” – Cold sink
Can thermodynamics guide informa,on ?
• Thermodynamics is a set of measurements and laws
• Relevant in the design of systems to harness the value of energy transfer (by heat and work) with maximum efficiency
• What if look at thermodynamic system parameters, but in the informa,on context ?
The measurement of info-‐dynamics?
• What might thermodynamics laws tell us about the genera,on of informa,on from data and the control of its flow ?
• Can the measurements of thermodynamics help us to design data-‐to-‐informa,on conversion engines with maximum efficiency ?
“Fill-‐in-‐the-‐gap” inference • Use thermodynamics laws as guidance for info-‐dynamics rules • Like Mendeleev blanks in periodic table led to new elements • Like Dirac’s predic,on of the existence of the positron
Thermodynamics of Informa,on. . .Really?
• “Informa,on thermodynamics” is not new
• John von Neumann told Claude Shannon that his mathema,cal theory of computa,on (MTC) should consider informa,on as entropy
• He said: “You should call it entropy for two reasons: first, the func8on is already in use in thermodynamics under the same name; second, and more importantly, most people don’t know what entropy really is, and if you use the word entropy in an argument you will win every 8me.“
Thermodynamics and Informa,on? • Claude Shannon correlated thermodynamics and informa,on
theory in MTC
• Shannon demonstrated entropy to be equivalent to a shortage in the informa,on content (a degree of uncertainty) in a message (how much “needle” and how much “haystack”)
• “Shannon entropy” is the basis for many informa,on de-‐
iden,fica,on protocols such as under U.S. HIPAA legisla,on (like “chaff” from a plane to defeat radar)
• What value and risk measurements could be applied to quan,fy
de-‐iden,fica,on (and re-‐iden,fica,on) of data and its conversion into inform-‐a,on using Shannon’s formula,on?
• This is the basis of U.S. FICAM (NIST 800-‐63 and OMB 04-‐04)
which call for ordinal ranking and matching of risk and iden,ty assurance based on a quanta of data as inference fodder
Possible insights of the energy/informa,on rela,onship
• System efficiency – Energy: Maximum work that can be performed by heat engine is propor,onal
to temperature difference between hot energy source and cold sink – Informa,on: Maximum arbitrage value in a market is limited by the
informa,on disequilibrium differen,als (regulated markets have smaller delta)
• System design – Energy: Iden,fy heat engine parts, piston, cylinder, valves, cold sink, etc. – Informa,on: Create both “Shannon entropy” engine from contract parts and
create “context entropy” engine from market parts that contract enables
• System measurement – Energy: Consider implica,ons of first law (temperature) and second law
(entropy) measurements of system a=ributes – Informa,on: Compare to system interoperability (temperature equilibrium)
and mul,ple entropies of informa,on (Shannon entropy and context entropy) and consider candidates for informa,on market measurements
System Efficiency
• What might thermodynamics laws rela,ng to system efficiency tell us about emerging data, iden,ty and informa,on markets?
System efficiency -‐ The differen,al of the hot source and cold sink
The cold sink is where the extra energy goes that is not in the form of work No cold sink=No work
System efficiency – Hot and cold differen,al
• Carnot: hot and cold difference is sole efficiency variable • Heat was considered a fluid called “caloric” that “flowed
downhill” from hot to cold • The metaphor of “flow” informed Carnot’s observa,ons even
though inaccurate • Our metaphor for informa,on flow
is energy transfer
System efficiency – differen,als* and arbitrage
• Differen,als of temperature are key in physical engine efficiency in Carnot’s equa,on
Efficiency = 1 -‐ Temp (L) Temp (H)
• Differen,als of value are key in informa,on (and market) arbitrage 1 -‐ Price (L) Price (H)
*Note that in both cases, absence of a differen,al between “low” and “high” values indicates system equilibrium
System efficiency – Hot and cold
• Kelvin also recognized the need for a cold sink • Kelvin’s statement of the second law: – No cyclic process is possible in which heat is taken from the hot source and converted completely* into work
* Note: Nature requires a tax paid (in the form of heat causing entropy in the surrounding “cold sink”) when heat is converted to work
System efficiency – Hot and cold
• The cold sink was central to work of Clausius • Clausius’ statement of the second law: – Heat does not pass from a body at low temperature to one at high temperature without an accompanying change elsewhere
System efficiency – What is differen,al for informa,on? • These minds perceived and characterized the rela,onships of energy, heat and work in the physical world in ways that enabled the exploita,on of those a=ributes to create value and reduce risk
• We have that same opportunity today for informa,on systems
• How might concepts of thermodynamics be applied to non-‐physical world of informa,on?
System efficiency – contracts and markets are the engines
powered by informa,on (entropy) differen,als • Carnot, Kelvin and Clausius revealed that accoun,ng for
entropy (not just energy) is key to thermodynamics • Heat engines are systems where both energy and entropy
are accounted for, and temperature differen,als are put to work
• Entropy as informa,on disorder (“Shannon entropy”) correlates with system risk
• Greater informa,on differen,als among par,es yield greater risks to some (and greater arbitrage to others)
• Like heat engines for informa,on, markets (and contracts) are systems where value crea,on and system risks are accounted for as informa,on differen,als and put to work
System efficiency – entropy and entrepreneurial risk
• Posi,ve -‐ Regulated markets (and contracts) reduce informa,on differen,als to reduce risk of market par,cipants
• Nega,ve 1 -‐ Regula,on reduces arbitrage opportuni,es by crea,ng informa,on equilibrium among market par,cipants (less work can be performed without differen,al)
• Thus, regulated market par,cipants (Telco's, banks, etc.) onen seek higher returns outside of regulated markets
• Nega,ve 2 -‐ Every decrease in entropy (risk) from successful regula,on breeds entropy that must manifest elsewhere (Clausius)
• Thus, new (onen larger) differen,als are established between par,cipants within and en,,es outside of new markets – Popula,ons in markets, a=ending only to market measures, can become “prey” for arbitrageurs opera,ng
outside the market with other measures – Regulated par,es experience a compe,,ve disadvantage as technology enables un-‐regulated par,es to
deliver similar products and services, but outside of the regula,on
System efficiency – Entropy is preserved (Clausius)
• Every new market creates a risk-‐reduced interac,on zone by elimina,ng a defined set of informa,on differen,als
• This “zone” is defined by the laws and regula,ons that establish par,cipant du,es to support par,cipant rights in the market
• Every successful market also creates new risky and valuable differen,als of those within and outside the market, since unregulated par,cipants are not bound by the market du,es
• Gödel's “incompleteness theory” provides that a system (such as a market) cannot be known without knowing its externality
• Market risk is Gödelian risk.
System efficiency – examples of entropy flow in markets
• 2008 was a result of Gödelian incompleteness in the deriva,ves market (outside the regulated financial markets)
• Dodd Frank financial reform legisla,on seeks to correct the structural differen,al by dragging deriva,ves into a market
• Since 1800, waves of disorder have swept through financial markets in 30 year cycle in U.S. and U.K.
• Are periodic financial breaks waves (solitons) of entropy passing through financial system?
• Does every new market have an entropy shadow always lurking at the doorstep; one that is at its most dangerous as a market is most successful at reducing risk?
System Design
• What might thermodynamics laws rela,ng to system design tell us about the future design parameters of sustainable data and informa,on architecture, and in par,cular the value and risk reduc,on engines emerging data, iden,ty and informa,on markets?
System design – Moving from random to ordered flows of informa,on
• How can we move from being data and informa,on entropy cavemen (enjoying the benefits of only the more random “informa,on energy”) to harnessing the equivalent of “informa,on “work” in an “industrial age” of informa,on?
• What virtual mechanisms might we use to harness the equivalent work of informa,on flows?
System design – two engines for two entropy values in the market
• What is the “engine” for informa,on • Heat engine has “reac,on chamber” for capture and use of work in physical energy transfers.
• What virtual mechanisms might we use to harness the equivalent work of the two types of “informa,on energy” reac,ons that: – Reduce intrinsic entropy: The “Shannon informa,on entropy” of the data is changed (i.e., it is de-‐iden,fied or re-‐iden,fied to and from PI)
– Reduce extrinsic entropy: The data is made accessible to a third party user who is “informed” by the data resul,ng in “inform-‐a,on” that has value as inference fodder to the observer (the difference that makes a difference)
System design – a data-‐to-‐informa,on engine
• What does the engine “reac,on chamber” look like for PI, not for fuel – Need reac,on vessel that can “account” for variables of interests of people.
• Risk and disorder are cousins related by indirect measurement of entropy
• Look at exis,ng mechanisms that address informa,on disorder/risk as candidates for engine components
System design – exis,ng risk engines
• These engines are all around us – right now • They are “markets,” “supply chains,” and contracts each of which creates
“risk adjustment” spaces that represent the a form of “state collapse” when mul,ple par,es co-‐exploit an opportunity to enjoy a lower risk system energy state that cannot be achieved unilaterally.
System design – How do these virtual “reac,on chambers” form
• Mutually-‐beneficial rela,onships act as feedback mechanisms encouraging self-‐binding of par,cipants to market du,es – Markets also rely on compulsion of public law, but self-‐regulatory structures are more scalable and sustainable
• “Nectar-‐based reac,on chambers”
System design – Nectar-‐based informa,on markets
• Where are examples in the online world of nectar-‐based informa,on markets?
• Most of you are probably on the trading floor of one of those markets right now as you sit in this room
System design – Data leverage in social network services as nectar
• Online service providers use nectar-‐based systems (e.g., Facebook, Google, etc.) – Same 1960’s business model as network TV (CBS, NBC, etc.) -‐ adver,sing pays
for programming, period – If you are not paying, you are not the customer – Adver,sers pay and they are the customer; the data you generate is the
product, period • TOUs are the “reac,on chambers” that control the “flows” of data to be
leveraged into informa,on when made available to enable inference (primarily adver,sers’ inference regarding future purchasing behavior)
• Current TOUs are among the first forms of “reac,on chamber,” but design parameters were more rudimentary – 1960’s business model was accommodated, but that is 50 years old !
• How should we design today’s reac,on chamber for today’s business model – mul,-‐stakeholderism, etc.
System design – Leverage happens in reac,on vessel
• Communica,on is stored as data (sniffles, e.g.) • At present, Shannon entropy is only addressed in regulated industries where de-‐iden,fica,on is required to share data (HIPAA, e.g.)
• Non-‐regulated industries can ignore Shannon entropy • Data is offered to third par,es to reduce their risk and informa,on needs
• Data is valueless without velocity/use • Data collector leverages data use by increasing velocity and by making data available to mul,ple par,es – Leverage in mul,ple use of single data
System design – It is a virtual vessel
• Engine is not within a single en,ty • Engine is “within” set of mutually desired rela,onships • Reac,on vessel is a virtual set of reliable, coherent,
repeatable, predictable rela,onships described/created by the TOU
• Not “bounded” by a vesicle, but coherent in its mutual a=rac,on of the parts – RNA metacycle ar,cle in Nature
System design – Vessel design parameters
• To design the informa,on engine, it is important to understand the nature of the flows and conversions involved.
• The math is the same for energy and informa,on, but the object of the calcula,ons is not physical, but virtual, so how measure what “flows”
• Flows of disorder in the form of heat can be broadly conceived of as flows of disorder in the form of risk.
• Therefore, current mechanisms that channel risk are among the candidates as pipes for this system.
System design – Vessel design parameters
• Contracts are the mechanism through which par,es allocate share and assign risk.
• Contracts are the engine for informa,on flows
• We already see this in TOS and TOU of online companies
• Exis,ng TOUs only siphoning off modest value (from old network TV paradigm), but s,ll shows how contract can characterize and create “data inventory” that can be co-‐managed
• Need to do for widely distributed data (on 3000 servers average)
what eBay did with contract and UI – turned crap in people’s garage into co-‐managed inventory.).
System design – Vessel design parameters -‐ users
• Like eBay for data – need to match internet scale • Internet and data poten,ally used in commerce, academia, civil society, government, etc. – all with different needs
• Since share informa,on infrastructure, but have different needs, start with most generic shared needs for system design – Iden,ty assurance – Reliability – Etc.
System design – Vessel design parameters – contextual entropy • Different types of disorder/risk
– Quan,ta,ve (Shannon risk) – Qualita,ve (Shannon doesn’t address contextual or seman,c “entropy”
measurement) • UW/MIT agreement form is a primary reac,on chamber for intrinsic,
Shannon entropy associated with de-‐iden,fica,on and re-‐iden,fica,on of data
• UW/MIT agreement form also enables the markets that are the secondary reac,on chamber for the harnessing of changes in seman,c entropy.
• Shannon entropy is amount of “surprise,” the rarity of one bit in many • HIPAA deiden,fica,on is that exactly. The market value of data as it is
degraded in LOA is a measure of Shannon entropy of informa,on vis iden,fy.
• Changing Shannon entropy can be done to data itself, because know the goal is to “de-‐iden,fy” from a par,cular PI.
System design – accoun,ng for seman,c/contextual entropy
• Seman,c content and entropy are not covered by Shannon’s MTC
• Cannot alter the data in a way to make it less useful to all possible third par,es in advance of knowing what they are interested in.
• Deiden,fica,on works because know that the object of it is to not make it about a person. Much harder to obscure if don’t know what parameter it is that trying to obscure.
• How can changes in seman,c/contextual entropy be best addressed?
System design – accoun,ng for seman,c/contextual entropy
• Seman,c or contextual entropy can best be addressed through a sta,s,cal analysis of myriad par,es in a market that can generate data about informa,on value in context
• The market data on informa,on value is generated based on the actual importance of the data to par,es in myriad unknown and unexpected contexts.
• There is NO WAY to tell the value of data un,l it is observed/discovered by an interested party. • The reac,on vessel for measuring seman,c and contextual entropy changes is not the contract and
the du,es that it invokes, but the larger market that is enabled by the broad adop,on of iden,cal du,es across a popula,on.
• This is because when Shannon entropy is being addressed in reliable LOA 1-‐4, people can interact with each other and exchange data in normalized data markets (in all sectors, financial, healthcare, etc.), lowering their contextual and seman,c risk also.
• In other words, if I know who I am dealing with , I can deal appropriately based on the risk of dealing with that party. I can then engage in markets for informa,on used in context (inform-‐a,on).
• The MIT UW agreement is the instrumentality to normalize the quan,fica,on of iden,ty integrity, and the markets emerge from its use.
• The contract enables accoun,ng generally, which permits repor,ng and visibility of where value was earned.
System design – The “thermodynamics” of informa,on use in efficient markets
• What is equivalent in informa,on terms • Hot/cold differen,al equals informa,on differen,als. • Markets are the “reac,on vessels” where informa,on arbitrage is worked out – Commodi,es exchanges – Flea markets – The Agora
• Work equals arbitrage -‐ differen,als • The bigger the temperature differen,al the more work, the bigger the informa,on differen,al the greater the arbitrage value, period.
System design – regula,on harnesses the valuable energy of informa,on
• Pre 1934 (unregulated) securi,es markets • Trading unconstrained, informa,on not normalized, risky
• Unsophis,cated commerce – Energy random – like fire
• Smaller scale
System design – regula,on harnesses the valuable energy of informa,on
• Post 1934 – Securi,es and Exchanges act • Orderly processing • Informa,on energy directed – like engine
System design – regula,on harnesses the valuable energy of informa,on
• Deriva,ves -‐ Pre-‐Dodd Frank reform legisla,on • Energy random – nothing to measure because grew outside
of regulated markets
System design –
• Post – Dodd frank • Deriva,ves traded in markets • Like new vessel for reac,on • Effort to reduce informa,on differen,als among stakeholders by using normalized market
• Heavy lobbying means that we aren’t going to have to wait 30 years for next cycle.
System design – • All markets have standard contracts at core • Normalize behaviors, du,es, rights across popula,ons
• MIT/UW contract is standard contract • Intended to be reactor for Shannon entropy change of de-‐iden,fica,on (entropy up and down)
• Capture the “work” of the state change (which is now just dissipated (as heat is dissipated from an engine).
System Measurement
• What might thermodynamics laws rela,ng to system measurement tell us about the future measurement parameters of sustainable data and informa,on architecture, and in par,cular the value and risk reduc,on engines emerging data, iden,ty and informa,on markets?
System measurement – Interoperability and the first law of Thermodynamics
• Under the first law of thermodynamics, temperature has two roles. – From the perspec,ve of a party outside of a system, temperature tells whether two or more systems are in thermal equilibrium – same temperature equals equilibrium.
– From perspec,ve of a molecule in the system, temperature tells about the distribu,on of various molecules in the system over the possible energy states. Higher temperature equals a broader distribu,on of molecules in different states, and a higher Boltzmann distribu,on.
• Either way in the contract context, “interoperability” is a measure equivalent to temperature – it tells you about equilibrium.
System measurement – New markets and the first law
• Markets enable equilibrium (and lower risk) internally • But market creates large popula,on in equilibrium • Like herd of sheep to an arbitrageur • If can have superior (rare), non-‐market informa,on, can exploit
informa,on arbitrage • Enables dis-‐equilibrium in other senses • Dis-‐equilibrium is a differen,al that can to leverage more work ,
generate more value • That is why regulated industries always envy and try to emulate
unregulated interlopers • Arbitrage exists at edges of regulated markets in equilibrium –
throwing them out of equilibrium and crea,ng new arbitrage opportuni,es
System measurement – enablement of design
• Future governance structures will be SROs that can con,nuously “mine” the unknown externality to bring it inside. Consume the surroundings to lower risk (entropy) inside markets.
• Use measurement to map entropy gradients like Lagrangian coherent structures in data sets
• Iden,fy objects of future rulemaking for SRO structures
