Multiscale Entropy in the Spatial Context of CitiesMartin Barner*,1,2, Clementine Cottineau1,3, Carlos Molinero1, Hadrien Salat1, Kiril Stanilov4, and Elsa Arcaute1

1Centre for Advanced Spatial Analysis, University College London, 90 Tottenham Court Road, London W1N 6TR, UK2Impact Initiatives, International Environment House 2, 7 Chemin de Balexert, 1219 Geneva, Switzerland

3CNRS, UMR 8097 Centre Maurice Halbwachs, Paris, France4Department of Architecture, University of Cambridge, 1-5 Scroope Terrace, Cambridge, CB 1PX, UK

*m@martinbarner.de

Entropy relates the fast, microscopic behaviour of the elements in a system to its slow, macroscopic state. Wepropose to use it to explain how, as complexity theory suggests, small scale decisions of individuals form cities. Forthis, we offer the first interpretation of entropy for cities that reflects interactions between different places throughinterdependently linked states in a multiscale approach. With simulated patterns we show that structural complexity inspatial systems can be the most probable configuration if the elements of a system interact across multiple scales. Inthe case study that observes the distribution of functions in West London from 1875 to 2005, we can partly explainthe observed polycentric sprawl as a result of higher entropy compared to spatially random spread, compact mixed usegrowth or fully segregated patterns. This work contributes to understanding why cities are morphologically complex,and describes a consistent relationship between entropy and complexity that accounts for contradictions in the literature.Finally, because we evaluate the constraints urban morphology imposes on possible ways to use the city, the generalframework of thinking could be applied to adjust urban plans to the uncertainty of underlying assumptions in planningpractice.

INTRODUCTION

Entropy in thermodynamics is a concept relating themicroscopic behaviour to the macroscopic dynamicsof a system.1 We therefore see it as a suitable toolto be used in studying the relationship between thefast dynamics of individual behaviour and the slow,larger scale dynamics of change in urban structures.Batty recognised that there are no substantiveinterpretations of entropy for cities yet and calls fora whole new research agenda.2

In its essence, entropy in statistical mechanicsis concerned with the number of (microscopic)configurations of the individual elements of a systemthat lead to the same macroscopic state.3. If morecombinations of possible microstates of the elementsin a system create the same macroscopic systemstate, that macroscopic state has higher entropyand is more likely to occur. The space of possiblemicrostates of elements is called the phase space. Themore evenly elements are distributed in the phasespace, the greater the entropy. There are for examplemore ways to distribute molecules evenly in a roomthan to place them all in one corner. Molecules floataround randomly in space and randomly exchangeenergy, and so in a phase space with dimensionsdescribing their position and momentum, they aremost likely to be distributed as evenly as possible.4

Buildings do not float around in space randomly,and they do not collide and randomly exchangeenergy. To define a relevant phase space for urban

morphology, we must carefully consider the processesthat produce the spatial patterns in cities. Mostof the existing measures of entropy in an urbancontext are either non-spatial, or literally adoptpart of the phase space in thermodynamics anduse the geographical space directly as the phasespace. Entropy then becomes a proxy for spatialevenness. While on the surface this phase space isvery similar to thermodynamics, it is not necessarilyrepresentative of how the macroscopic state of a cityevolves.

In this paper, we attempt to formulate a measureof entropy that is conceptually consistent with bothentropy in statistical mechanics and an understandingof cities as complex systems. Two fundamentalaspects thus differentiate our approach from existingspatial measures of entropy:

First, the phase space dimensions reflect charac-teristics of places instead of absolute locations,because cities are complex systems: Viewing citiesas complex systems brought a fundamental shiftin our understanding of how cities function, grow,and change over time5–8. Essential to this view isthat the global order of a city emerges from thesmall scale decisions and interactions of individualsin a process of self organisation. [9, p.38]. Whatpeople do in the city has an impact on its spatialstructure over long periods of time10,11. From thiswe conclude that if there are more combinations ofpossible use for a city’s morphological macrostate,there are also more ways that macrostate could have

arX

iv:1

711.

0981

7v1

[ph

ysic

s.so

c-ph

] 2

7 N

ov 2

017

formed. It follows that this macroscopic state shouldthen also have greater entropy and be more likelyto occur. We are not interested in the randomnessof geographic coordinates, but in the randomnessof how people could use the city depending on itsphysical structure. Instead of measuring spatialevenness - the uncertainty about where things are -we take a first step to measure the uncertainty abouturban life that is built into the spatial structure ofthe city. This directly translates into the practicalphase space definition. Based on the assumptionthat what people can do in two places differs if theplaces have different characteristics, our phase spacedimensions describe the characteristics of differentplaces.

Second, the phase space dimensions must reflectthat places in a city are inherently dependent oneach other. Boltzmann’s entropy assumes thatinteractions between particles are negligible. Existingmeasures of entropy inherit this for cities, andassume that there were no interactions betweenplaces. We recognise that different places are highlydependent on each other due to flows and interactionsbetween the people in them. This interdependencebetween places is widely recognised in geographyin general12, urban theory5,13 and quantitativelydemonstrated for example in the success of spatialinteraction models14–17, and further in the studyof agglomeration economics18 and neighbourhoodeffects19. It follows that the states of individualplaces must be connected spatially, because in termsof how they are used, places in a city are connectedspatially as well. In that sense the state of anobserved place should in some way also incorporatethe characteristics of surrounding places. Our newmeasure observes the characteristics of places, andtakes relationships between places into account witha multiscale approach: we observe the characteristicsof a place at multiple scales by aggregating thecharacteristics of neighbourhoods with differentsizes around it. The resulting values then define itsposition in the phase space. The state of a place isthen not only given by its own characteristic, butalso the characteristics of the directly adjacent places,its local neighbourhood and larger scale surroundings.

Our analysis of simulated patterns shows that urbanstructures that are completely randomised and evenlydistributed in geographical space do not simultane-ously have the highest randomness in a phase spacethat describes the variety of available types of interde-pendent places. Instead, we can show that spatiallycomplex patterns are most evenly distributed in the

multiscale phase space and have the highest entropy.That the patterns with the highest entropy are notevenly distributed in geographical space is in perfectharmony with the fact that by definition, greater ran-domness and an even distribution in the phase spacemeans greater entropy: the geographical space is notthe phase space.In our case study we measure the change in multiscaleentropy in land use patterns in West London in seventime steps from 1875 to 2005. The results show thatthe observed polycentric sprawl corresponds to thegrowth pattern with the highest entropy (comparedto randomised and ordered patterns) and thus suggestthat urban development tends towards a maximisa-tion of multiscale entropy.

METHODS: A MULTISCALE APPROACH TOENTROPY IN CITIES

In this section we first give an overview over urbanand scale dependent entropy measures. We thenrevisit the formal definition of entropy and thephase space in statistical mechanics and identifycommon spatial phase space interpretations. Finally,we formally define the theoretical multiscale phasespace in which each state is given by a matrixcontaining multiple characteristics aggregated inneighbourhoods of different sizes, and introduce apractical method for multiscale entropy estimation.

Existing approaches to spatial, scale de-pendent and urban entropy

Entropy is applied extensively in spatial analysis.It appears in Wilson’s spatial interaction models20

that use entropy maximisation to predict trafficand financial flows. Further, there are attempts todiscuss the energy and resources entering and exitingan urban systems in relation to entropy.21. Vrankenet al.22 summarise 50 different measures of entropyin landscape ecology, some of which discuss scaledependence, usually in the context of the modifiableareal unit problem. Further, entropy across multiplescales appears in measures of complexity in timeseries, for example by Zhang23 and Costa et al.24–27

for medical time series. It has been applied asa measure of complexity in numerous fields ofresearch28. Finally, there are methodological andconceptual parallels of our approach to methodsfor estimating fractal dimensions29, specifically boxcounting30 that could be worth exploring further.

2/12

Entropy in statistical mechanics

In statistical mechanics, entropy is defined as31

H = −∫

f (x) log( f (x))dx (1)

where f (x) is the probability density of a continuousphase space. The equivalent to equation 1 in thediscrete phase space is31:

H = −∑ p log p (2)

which reduces to the Boltzmann entropy S if all mi-crostate probabilities p are the same:

S = kB log(Ω) (3)

Where kB is the Boltzmann constant and Ω thenumber of accessible microstates [32, p.44]. Allmicrostates can be allocated a location in the phasespace. If the phase space is discrete, we can countthe number of possible permutations that producethe same macrostate. The highest entropy is alwaysgiven by a uniform probability distribution in thephase space, leading to sometimes misleading butcommon metaphors for entropy33: if the entropy ofa system is high, the state of a randomly selectedelement is unpredictable and “uncertain”, and if weare uncertain about where things are in a system,one might describe it as “disordered”.

It is commonly understood in thermodynamics thatif one refers to the Boltzmann phase space, it usuallyrelates to the six dimensional phase space that de-fines a particle’s state by its location and momentum.Similarly, the Gibbs phase space relates to the 6Ndimensional phase space describing the location andmomentum of all N particles in the system.34

Common phase space definitions forspatial entropy

In contrast to thermodynamics where there is generalclarity about what the parameters of a microstateare, there are fundamental differences between inter-pretations of entropy in cities. They come down todifferent definitions of the phase space dimensionsthat can be summarised in the following four groups:

• The first essential approach in the literature takesthe word “space” literally and defines the phasespace as the geographical space.2,35,36 The high-est entropy is then given by a pattern with auniform distribution in geographical space. Thisinterpretation answers the question: how uncer-tain is the absolute location of a place with agiven characteristic?

• The second basic phase space uses a character-istic of places or objects in space as the phasespace.37 All patterns with the same global pro-portions of occurrences of different types have,according to this phase space definition, the sameentropy. This interpretation answers the ques-tion: how uncertain is the characteristic of agiven place or observed element in space in gen-eral, independent from the spatial configuration?

• Most reviewed approaches to spatial entropy usea combination of the two phase spaces above.They are measures of spatial evenness widelydiscussed in the literature on measures of seg-regation38–49. They have the highest entropy ifentropy is maximised in both phase spaces aboveat the same time while trying to overcome themodifiable areal unit problem50. Nonetheless,they all try to answer the question: how evenlyare observations of different types or characteris-tics distributed geographically?

• Finally, there are approaches that define spatialco-occurrences of different elements or charac-teristics as different states in the phase space.51.They generally have higher entropy if observa-tions are distributed more evenly in space, andif there is no spatial correlation between obser-vations of different types. They partly look atvarying scales, mainly in terms of index sensitiv-ity52. Most closely related to our approach areJohnson et al.’s conditional entropy profiles53.In terms of information theory, one could saythey evaluate how much of the information at agiven resolution is contained in observations ata coarser resolution. This group of phase spacesbroadly answers the question: how well does thespatial distribution of one type of observationspredict the distribution of another?

The multiscale entropy phase space

In contrast to the existing measures of entropy dis-cussed above, we want to answer the question: howuncertain are the characteristics of places a personcould be in, considering that the characteristics of aplace are defined not only by its own value, but alsoby the characteristics of the places around it? Wesee this as relevant because it reflects the uncertaintyabout what a randomly selected resident does, basedon the structure of the city. In that sense, none of theapproaches described above presents a conceptuallyconsistent interpretation of entropy that reflects theidea of cities as emergent phenomena. Instead we needa measure that fulfills the following requirements:

3/12

• It should observe how places are distributedacross characteristics, to reflect the uncertaintyabout how the city is used.

• It should reflect that the characteristics of placesspatially depend on each other, because the sur-roundings of a place fundamentally alter how itcan be used.

We can illustrate why the above requirements areimportant and how a multiscale approach can fulfillthem with a simple example: imagine the patterns infigure 1 were real cities, and black and white pixelswould refer to residential and commercial buildings.Of course pattern a) is more evenly distributed inspace, but this is not what we are interested in.Taking into account the surroundings of each pixel,pattern a) only has two different types of places:residential or commercial buildings, but always inmixed blocks in mixed neighbourhoods in mixeddistricts of a homogeneously mixed city. If we picka single place at random from both pattern a) andpattern f), we have the same probability to pick a“residential” or a “commercial building” in both cases,but in pattern f) there is much less certainty aboutthe type of neighbourhood we pick. We want toextend the description used to compare individualbuildings from “a residential building” to somethinglike “a residential building in a mixed use blockwhich itself lies in a mainly commercial district thatis surrounded by residential areas”. All of thesesurroundings at different scales should be part ofthe state of that place: if the direct surroundingsand larger scale neighbourhoods of two places areidentical, their function is more similar. In reverse, iftwo places are identical but their surroundings arefundamentally different, they can be used in differentways and their states should differ. This is where“Multiple scales” becomes important: the state of aplace includes values describing not only the place’sown characteristics, but some aggregate descriptionof its environments within increasing distance: itsimmediate surroundings, its local neighbourhood andits larger scale environment.

Therefore we define for our quantitative measure thephase space like this: the first dimension of the phasespace is the value of a place’s own characteristic. Wethen add further phase space dimensions describingthe place’s surroundings at N different scales. Whenwe consider only one characteristic, the state of eachplace xi in the city observed at N neighbourhood scalesis given by the vector

~xi = (xd0i ,xd1

i , . . . ,xdNi ) (4)

where xd0i is the local value of a characteristic of place

xi itself. The value xdni at scale n is given by the local

characteristics’ values of all places within distance dnfrom xi, aggregated by a function:

xdni = f (xd0

k1,xd0

k2, . . . ,xd0

km) (5)

for all xd0k of the m places within distance dn from xi.

What this achieves is that we can distinguish betweenlocally identical places based on what kind of areathey are in, because the state of a place is literally afunction of its surroundings.Extending this to C scalar characteristics, for examplethe amount of area covered by different categories ofland use, the whole state of a place in the system isgiven by the matrix

Ψi =

xd0,1

i xd1,1i . . . xdN ,1

ixd0,2

i xd1,2i . . . xdN ,2

i...

......

...

xd0,Ci xd1,C

i . . . xdN ,Ci

(6)

We discretise the phase space by defining a discreteset of values for all the elements in matrix Ψ thatwill be given by binning the values after aggregation.Places are assumed to have the same state only iftheir state matrices are exactly identical. Becausethis simplified phase space is discrete, we canestimate the probability of discrete states directlyfrom their frequency, and the system’s entropy withequation 2.

A discrete phase space has multiple advantages. First,we avoid properties of the unit dependent54 contin-uous entropy such as negative entropy55,56 that aredifficult to interpret in terms of statistical mechanics.Furthermore, it removes the difficulty of evaluatingeuclidean distances between different place character-istics. Finally, it avoids discussing complicated esti-mators for multivariate continuous data57,58. Theyare unreliable for high dimensional data because theywork with the spaces between observations, and thenumber of data points on the edges of the phase spaceincreases exponentially with increasing dimensions.The Supplementary material online contains furtherdetails. In the following section we test this methodon a range of simulated patterns.The practical entropy estimation in our simulatedpatterns and case study is as simple as possiblewithout compromising the general concept. We usesimplified square neighbourhoods with varying sidelength because it makes the results easy to trace,is computationally convenient and is sufficient todemonstrate the concept.

4/12

We then calculate the place state matrix Ψ using themean as the aggregation function in equation 5 forthe same reasons.

SIMULATED PATTERNS

Here we measure the entropy of synthetic spatialpatterns according to the two essential phasespace definitions in the literature, and accordingto our multiscale entropy measure. We first showhow the results of our new method are inherentlydifferent from the non spatial phase space andthe geographical phase space. We then comparethe multiscale entropies of patterns with differentstructures and show that if interactions betweenplaces are accounted for, complex patterns have ahigher entropy than simple ones.

We use the 6 artificial patterns from figure 1. Thepatterns are 512 pixels wide and high. Each pixelcorresponds to a ”place”. Each pixel is assigned avalue from 0 to 1 (black to white), defining the onlycharacteristic of that place. The patterns are selectedto represent varying degrees of complexity. When wespeak of “complexity” here, we mean patterns whichcould be described intuitively as having “meaning-ful structural richness”.59 Pattern a) has a spatiallyuniform probability for all pixels to be either whiteor black. It is fully random and arguably has nostructure at all. The randomised checker board b) isessentially a fully random pattern like pattern a), butpixels of each colour appear in rectangular patches.It is locally segregated and could be seen to be in-creasingly ordered with increasing patch size, but theorder is very simple. Pattern c) is fully segregated be-tween the left and right half, a most simple but strictspatial order. Pattern d) is a sample from a uniformdistribution between 0 and 1 (corresponding to blackand white), sorted linearly from left to right and fromtop to bottom. There is structure, but no structuralrichness. Potentially viewed as slightly more complexmight be pattern e), in which pixels are assigned a1 or a 0 with the probability to find a black pixeldecreasing linearly from left to right. It serves asthe binary spatial counterpart to what Zhang23 con-siders a complex time series, but arguably does notdiffer greatly from pattern d) in terms of structuralrichness. pattern f) results from a binarised additivecascade process, which produces patterns with multi-fractal self-similar properties that occur in complexprocesses60 and are regularly associated with highcomplexity.61–63. Further details on the constructionand the following analysis of simulated patterns are

available in the supplementary material online.

Existing measures

Before applying the new multiscale entropy measure,we show the behaviour of the existing non spatialand the geographical phase space.

In the non spatial phase space observing onlyglobal characteristic proportions, we can directlytell that patterns display the same entropy as longas they differ only in the spatial configuration.If we consider only two states, values greater orsmaller than 0.5, all patterns’ entropy accordingto equation 2 is Hnonspatial = log(2) because in allpatterns approximately half the pixels have a valuegreater than 0.5. We could reduce the multiscale

entropy phase space to this by using only the xd0,ci

column on the left of Ψ in equation 6.

Measures of entropy using the geographical spacedirectly as the phase space are essentially measures ofhow evenly elements are distributed across differentzones. We split the patterns into square zones witha side length of 128 pixels, and count the number ofblack pixels (see fig. 2). This approach is inherentlydifferent from our measure in its goals and results.As expected from a measure of spatial evenness, thegeographical phase space entropy (results in fig. 4) ishighest for the uniform distribution (figure 1 a)), andlowest for patterns segregated spatially at a largerscale than the used zones (fig. 1 c)).The frequencies in the discrete geographical phasespace in figure 2 show the conceptual difference to ourmeasure. When the geographical space is used directlyas the phase space, the spatially even distribution ofpattern a) also gives an even distribution in the phasespace. In contrast, we see an even distribution offrequencies for the sorted patterns d) and e) and forthe additive cascade f), which receive higher entropyin a phase space that is focused on how much placesdiffer from each other.

Multiscale entropy and complexity

Now we apply our new multiscale entropy measureto the simulated patterns in fig. 1 and show that ifinteractions between locations are taken into account,spatially complex patterns have higher entropy. Inthe analysis, for “neighbourhoods” that go over theedge of the patterns, the invisible part is assumedto have the same proportion of values as the visiblepart. We bin the mean values in three categories:low (mean 0-0.33), medium (mean 0.33-0.66) and

5/12

Fig. 1: patterns a) - f)

Fig. 2: Frequency distribution in the geographical phase space. Each state corresponds to a spatial zone. The zonesare sorted by frequency - patterns fig. 1

Fig. 3: Multiscale phase space distribution - patterns fig. 1. the x and y axis are the fraction of black pixels withinneighbourhoods of 13 and 65 pixels side length respectively - patterns fig. 1

Fig. 4: Geographical phase space entropy - patterns fig. 1

high (mean 0.66-1.0). We use 3 different scales withneighbourhood side lengths with 3, 13 and 65 pixels.

Figure 5 shows the multiscale entropies for the pat-terns in figure 1. we discuss them considering thedistributions in the multiscale phase space (fig. 3which shows two dimensions, specifically at the scalesof 13 and 65 pixels neighbourhood side length.

The relatively complex additive cascade is most

Fig. 5: Multiscale entropy - patterns fig. 1

evenly distributed in the phase space and thereforehas the highest entropy. The uniform probability inthe geographical space of pattern a) is very similareverywhere except on a very local scale. All pixelslie in very similar mixed neighbourhoods, and sothe distribution has less variation in the largerneighbourhoods of the y axis. The pattern b) hasincreased variance on scales of observation close tothe scale of segregation, but fails to maintain variance

6/12

across multiple scales. In the fully segregated patternc), places only differ in their large scale environment,but locally almost all places are concentrated in thetwo extremes. The sorted uniform distribution ofpattern d) is very evenly distributed on each scaleviewed individually. However, there is no variation inwhich type of small scale neighbourhood is combinedwith which type of larger scale neighbourhood.This effect also applies to pattern e): while thereis some variation on all scales, small white pixelneighbourhoods are systematically more likely to liein larger white pixel neighbourhoods and vice versa.The only pattern that has places spread considerablyevenly across characteristics across scales is the morecomplex additive cascade.

Imagine we would try to change any of these patternsto spread the observations more evenly in the phasespace as in figure 3 and increase the entropy. Wewould need to add more and more layers of variationon different scales, while simultaneously trying toavoid creating simple random noise. The result wouldinevitably be a non trivial spatial configuration.

This may seem rather abstract. However, it shouldapply to any system in which elements interact withand influence each other to a degree at which theyfundamentally change each other’s meaning over mul-tiple scales of some type of “nearness”. As discussedin the introduction this is certainly the case for placesin cities. Under these circumstances, complex pat-terns have a higher entropy. Therefore, we can andshould expect the whole system to eventually arrangein a complex pattern, simply because that is the mostprobable configuration.

RESULTS: LONDON 1875 - 2005

Data

In the case study, we analyse the spatial patterns ofland use in west London from 1875 - 2005. The datasetused in the analysis was originally built and providedby Stanilov et al.64. It covers 200 square kilometers,spanning 20km from east to west, from London’sgreen belt in the west to Hyde Park’s west corner,and 10km from north to south. The data providesthe land use of individual buildings in 32 categoriesfor seven moments in time, namely 1875, 1895, 1915,1935, 1960, 1985 and 2005. Details on the originaldata and maps can be found in supplementary Fig. S1online, and further in Stanilov et al.’s publication64.

Entropy estimation

To keep the number of dimensions reasonably low,the 32 land uses are grouped into three categories of”business”, ”residential” and ”leisure” and we use 5scales of observation at 50m, 150m, 450m, 1350m,4050m. We discretise the values in the place state Ψ

equidistantly in three bins. The data is rasterised ata resolution of 50m. Neighbourhood parts outsidethe bounding box are assumed to have the sameproportion as the parts within. The asymmetricnature of the data and the null models makes thatpreferable compared to edge wraparound.

We compare the observed patterns with three nullmodels that are constructed to preserve the globalamount of different land uses and differ only in thespatial configuration. The data and null models areshown in figure 6. We compare three null models intotal:

• spatially random uniform spread: the pixels ofthe original data are reallocated in a randomorder. This would be the maximum entropydistribution if the phase space was directly takenfrom the geographical space.

• compact mixed use growth: the pixels of theoriginal data are redistributed in a fully randomfashion, but separated between developed andundeveloped land and fit compactly to the eastedge, corresponding to the general direction ofgrowth in the original data.

• compact segregated growth: the pixels of theoriginal are sorted by function and fit compactlyto the east edge.

We also compute the non spatial entropy of the globalproportion of functions for each year.Further details on the data, preprocessing and theconstruction of the null models can be found in thesupplementary material online, and a sensitivity anal-ysis in supplementary Fig. S2 - S5.

Results

Figure 7 shows the development of multiscale entropyover time in comparison to three null models andnon spatial entropy. For all cases, entropy generallyincreases until 1935, stagnates around 1965 andthen slightly decreases until 2005. This is based onthe non spatial entropy of the global distributionof functions: almost the entire area is undevelopedin the beginning, giving very little potential for

7/12

Fig. 6: The probability of each pixel’s state, and the corresponding spatial distribution of functions. From left to right:random pixel allocation, compact mixed use growth, compact segregated growth and observed data. Global proportionof functions and observed data correspond to 1875, 1895, 1915, 1935, 1960, 1985, 2005 from top to bottom. Grey:undeveloped or no data. Red: residential. Blue: commercial. Black: leisure. Grayscale images decreasing probabilitywith increasing brightness (logarithmic)

Fig. 7: Multiscale entropy in West London over timecompared to 3 null models

variation in general. It is filled almost entirely lateron, and entropy stagnates with a general stagnationof change. In the end, entropy slightly declinesbecause there are almost no “empty” areas left; thereare no more places in states with low values on allland use categories in the observed area.

The multiscale entropies in figure 7 show thatthe observed multiscale entropy of West Londonis substantially higher than all three null models.Especially between 1915 and 1960, entropy increasesconsiderably in the observed data, while the nullmodels stagnate.

The gray scale images in figure 6 show the probabilityof each pixel’s state which lets us investigate whichplaces contribute to the total entropy. It shows whatchanges to the patterns would increase their entropy,which in return explains what the features are thatgive higher entropy to the observed growth patternin West London.

In the spatially uniform randomised case, uniqueplaces appear only beyond a certain global density,where only very small segregated clusters appear bychance. In the early stages entropy would be higherif growth was more concentrated, and later if therewere also larger segregated and non segregated localconcentrations.

In the compact mixed use growth case, the onlyunique places are on the city edge, while most placesare either completely undeveloped or evenly mixed.Entropy could be increased by a less stringent cityedge and partial concentration of the less frequentfunctions. In the compactly segregated case, the mostunique places are along the edges between functions,as well as along the city edge. Entropy could be in-

8/12

creased by a less stringent city edge, as well as moresmaller clusters of segregated or mixed functions.All of these alterations would change the null modelpatterns closer to what we actually observe:First, clusters of different sizes with varying degreeof functional segregation. Second, no strict cityedge. In the language of urbanists, we could call thispolycentricity65 and sprawl66. From this perspective,we can give an explanation of the polycentric sprawlthat dominates the growth patterns of the observedarea in terms of entropy: in this situation, there aresimply substantially more combinations of individualchoices that lead to polycentric sprawl, making it themost likely pattern to occur.

There are great limitations in terms of data andmethodology that make any conclusions or generalisa-tions speculative. First of all, we are only observinga small window of the city, and as the city growsthe city edge passes through our field of view. Fur-thermore, the results may be biased towards higherentropy because in the original data collection, thearea was selected specifically for it’s high functionaldiversity.64

The results are consistent with varying parameters(see supplementary material online). However, thefunctional categories, the aggregation function, thescale of rasterisation, the equal treatment of differentcategories that in fact may be more or less similar,the selection of neighbourhood scales and their rectan-gular shape are all rather arbitrary. While sufficientto demonstrate the basic ideas, neighbourhood sizesand shapes as well as the aggregation function coulduse a network based measure of distance, take intoaccount subjective travel cost and relate to insightsinto the actual connectivity between places.

DISCUSSION

The ambition of this work is to make a contributionto explaining the emergence of spatial patternsfrom microscopic behaviour and establish a morecoherent relationship between entropy and complexity.Further, the general framework of thinking may beused as a strategy to deal with uncertainty andunpredictability in planning practice.

The understanding of the relationship betweenentropy and complexity is highly incoherent inthe literature.67 Attempts have been made toassociate complexity with decreasing thermodynamicentropy68,69, regarding the occurring order as highercomplexity than the original randomness. Othersregard fully unpredictable signals such as white

noise as fully complex70 in contrast to fully orderedsignals such as strictly periodical signals. Battyet al. adopted this view for cities as well.36 Incontradiction, Costa et al. conclude that relatinggreater entropy to greater complexity would befundamentally misleading.24.

The point we make is that almost arbitrary resultscan be obtained depending on how the phase spaceis defined. The key to a meaningful measure ofentropy is to define a phase space that is conceptuallygrounded in how the macroscopic state of the systemis produced. We argue that elements are spatiallydependent, and that this must be considered inthe microstates. The analysis of synthetic patternswith multiscale spatial entropy shows that in thatcase, complex patterns have the highest entropy.We can thus explain to some degree the spatialcomplexity that is frequently observed in cities71–74 -and more generally the complexity of patterns withinterdependent observations - as simply the kindof pattern we are most likely to observe becausethey can occur in more ways than others. In thecase study we find that West London did in factevolve towards higher multiscale entropy than thenull models.

Beyond these theoretical considerations, the generalframework of thinking in terms of urban entropy maygive a useful perspective on practical planning anddesign decisions. What is ignored entirely so far ex-cept for a vague notion of some interaction betweendifferent places is essentially everything else we al-ready know about cities: how people use them, or howsocial and economic processes shape their structure.Paradoxically, that is precisely why this might be apowerful concept. It allows us to make the statisti-cally best guess about what we do not know. Froma planner’s perspective, we would try to optimiseour planning effort based on some assumptions aboutpeople and societies, how they should or want to usecities, and beyond that based on some predictionabout the future and an assessment of what shouldbe considered a “good” city. There is a limit to howcertain we can be about these assumptions. With agood interpretation of entropy for cities, this uncer-tainty could be physically expressed in the structureswe build, to increase the probability to build “good”cities even if our assumptions were wrong, if circum-stances change unexpectedly or if what is considereda “good” city changes.

9/12

ACKNOWLEDGEMENTS

The Authors thank Robin Morphet, Remi Louf andMike Batty for insightful discussions and support.E.A., M.B., C.C. and C.M. were partly funded by theMECHANICITY Project (249393 ERC-2009-AdG).K.S. received funding from the European Commu-nity’s Seventh Framework Programme [FP7/2007-2013] under grant agreement number 220151.

AUTHOR CONTRIBUTIONS STATEMENT

M.B. was responsible for all main ideas, conceivedand conducted the experiments and data analysis,analysed the results and wrote the manuscript. E.A.,C.C., C.M. and H.S. critically revised the methodol-ogy and the manuscript. K.S. produced and providedthe data. All authors approved the final manuscript.

ADDITIONAL INFORMATION

The authors declare no competing financial interests.

REFERENCES

1. Gibbs, J. W. Elementary principles in statisticalmechanics (Courier Corporation, 2014).

2. Batty, M. Space, scale, and scaling in entropy-maximising. Geographical Analysis 42, 4,395–421 (2010).

3. Kleidon, A. & Lorenz, R. D. Non-equilibriumthermodynamics and the production of entropy:life, earth, and beyond (Springer Science & Busi-ness Media, 2005).

4. Wehrl, A. General properties of entropy. Reviewsof Modern Physics 50, 221 (1978).

5. Jacobs, J. The death and life of great Americancities (Vintage, 1961).

6. Batty, M. The new science of cities (Mit Press,2013).

7. Batty, M. Cities and complexity: understandingcities with cellular automata, agent-based models,and fractals (The MIT press, 2007).

8. Allen, P. M. Cities and regions as evolutionary,complex systems. Geographical systems 4, 103–130 (1997).

9. Johnson, S. Emergence: The connected lives ofants, brains, cities, and software (Simon andSchuster, 2002).

10. Portugali, J. What makes cities complex? Com-plexity, Cognition, Urban Planning And Design.Springer (2014).

11. Allen, P. M. Cities: the visible expression ofco-evolving complexity. In Complexity Theoriesof Cities Have Come of Age, 67–89 (Springer,2012).

12. Tobler, W. R. A computer movie simulatingurban growth in the detroit region. Economicgeography 234–240 (1970).

13. Alexander, C. A city is not a tree. Ekistics 23,344–348 (1967).

14. Wilson, A. G. A family of spatial interactionmodels, and associated developments. Environ-ment and Planning 3, 1–32 (1971).

15. Haynes, K. E. & Fotheringham, A. S. Grav-ity and spatial interaction models, vol. 2 (Sagepublications Beverly Hills, 1984).

16. Maguire, D. J., Batty, M. & Goodchild, M. F.GIS, spatial analysis, and modeling (Esri Press,2005).

17. Harris, B. & Wilson, A. G. Equilibrium val-ues and dynamics of attractiveness terms inproduction-constrained spatial-interaction mod-els. Environment and planning A 10, 371–388(1978).

18. Fujita, M. & Thisse, J. Economics of Agglomer-ation: Cities, Industrial Location, and Globaliza-tion (Cambridge University Press, 2013).

19. Galster, G. C. The mechanism(s) of neighbour-hood effects: Theory, evidence, and policy impli-cations. In Neighbourhood effects research: Newperspectives, 23–56 (Springer, 2012).

20. Wilson, A. G. Entropy in urban and regionalmodelling (Pion Ltd, 1970).

21. Zhang, Y., Yang, Z. & Li, W. Analyses of urbanecosystem based on information entropy. Ecolog-ical Modelling 197, 1–12 (2006).

22. Vranken, I., Baudry, J., Aubinet, M., Visser, M.& Bogaert, J. A review on the use of entropy inlandscape ecology: heterogeneity, unpredictabil-ity, scale dependence and their links with thermo-dynamics. Landscape Ecology 30, 51–65 (2015).DOI 10.1007/s10980-014-0105-0.

23. Zhang, Y.-C. Complexity and 1/f noise. a phasespace approach. Journal de Physique I 1, 971–977 (1991).

24. Costa, M., Goldberger, A. L., Peng, C. et al.Multiscale entropy analysis (mse) (2000).

10/12

25. Costa, M., Goldberger, A. L. & Peng, C.-K. Mul-tiscale entropy analysis of complex physiologictime series. Physical review letters 89, 068102(2002).

26. Costa, M., Goldberger, A. L. & Peng, C.-K. Mul-tiscale entropy analysis of biological signals. Phys-ical review E 71, 021906 (2005).

27. Costa, M. D. & Goldberger, A. L. Generalizedmultiscale entropy analysis: Application to quan-tifying the complex volatility of human heartbeattime series. Entropy 17, 1197–1203 (2015).

28. Humeau-Heurtier, A. The multiscale entropyalgorithm and its variants: A review. Entropy17, 3110 (2015). DOI 10.3390/e17053110.

29. Mandelbrot, B. B. The fractal geometry of nature,vol. 173 (Macmillan, 1983).

30. Addison, P. Fractals and Chaos: An illustratedcourse (Taylor & Francis, 1997).

31. Shannon, C. A mathematical theory of com-munication. Bell System Technical Journal,The 27, 379–423 (1948). DOI 10.1002/j.1538-7305.1948.tb01338.x.

32. Capek, V. & Sheehan, D. P. Challenges to thesecond law of thermodynamics (Springer, 2005).

33. Leff, H. S. Entropy, its language, and interpre-tation. Foundations of Physics 37, 1744–1766(2007). DOI 10.1007/s10701-007-9163-3.

34. Jaynes, E. T. Gibbs vs boltzmann entropies.American Journal of Physics 33, 391–398 (1965).

35. Batty, M. Spatial entropy. Geographical analysis6, 1–31 (1974).

36. Batty, M., Morphet, R., Masucci, P. & Stanilov,K. Entropy, complexity, and spatial informa-tion. Journal of geographical systems 16, 363–385(2014).

37. Gudmundsson, A. & Mohajeri, N. Entropy andorder in urban street networks. Nature ScientificReports 3, Article number: 3324 (2013).

38. Henri Theil, A. J. F. A note on the measurementof racial integration of schools by means of infor-mational concepts. The Journal of MathematicalSociology 1:2, 187–193 (1971).

39. Duncan, O. D. & Duncan, B. A methodolog-ical analysis of segregation indexes. Americansociological review 210–217 (1955).

40. Jahn, J., Schmid, C. F. & Schrag, C. The mea-surement of ecological segregation. AmericanSociological Review 293–303 (1947).

41. Williams, J. J. Another commentary on so-calledsegregation indices. American Sociological Re-view 298–303 (1948).

42. Bell, W. A probability model for the measure-ment of ecological segregation. Social Forces357–364 (1954).

43. White, M. J. The measurement of spatial segre-gation. American journal of sociology 1008–1018(1983).

44. Morgan, B. S. An alternate approach to thedevelopment of a distance-based measure of racialsegregation. American Journal of Sociology 1237–1249 (1983).

45. Stearns, L. B. & Logan, J. R. Measuring trendsin segregation three dimensions, three measures.Urban Affairs Review 22, 124–150 (1986).

46. Lieberson, S. & Carter, D. K. Temporal changesand urban differences in residential segregation:a reconsideration. American Journal of Sociology296–310 (1982).

47. Massey, D. S. & Denton, N. A. The dimensions ofresidential segregation. Social forces 67, 281–315(1988).

48. Wong, D. W. Spatial dependency of segregationindices. The Canadian Geographer/Le Geographecanadien 41, 128–136 (1997).

49. Taeuber, K. E. & Taeuber, A. F. Negroes incities: Residential segregation and neighborhoodchange (Atheneum, 1969).

50. Dark, S. J. & Bram, D. The modifiable areal unitproblem (maup) in physical geography. Progressin Physical Geography 31, 471–479 (2007). DOI10.1177/0309133307083294.

51. Leibovici, D. G. Defining spatial entropy frommultivariate distributions of co-occurrences. InInternational Conference on Spatial InformationTheory, 392–404 (Springer, 2009).

52. Leibovici, D. G., Claramunt, C., Le Guyader, D.& Brosset, D. Local and global spatio-temporalentropy indices based on distance-ratios and co-occurrences distributions. International Journalof Geographical Information Science 28, 1061–1084 (2014).

53. Johnson, G. D., Myers, W. L., Patil, G. P. &Taillie, C. Characterizing watershed-delineatedlandscapes in pennsylvania using conditional en-tropy profiles. Landscape Ecology 16, 597–610(2001).

54. Hazewinkel, M. Encyclopaedia of Mathemat-ics: Coproduct — Hausdorff — Young Inequal-

11/12

ities. Encyclopaedia of Mathematics (SpringerUS, 2013).

55. Bartz-Beielstein, T., Chiarandini, M., Paquete, L.& Preuss, M. Experimental methods for the anal-ysis of optimization algorithms (Springer, 2010).

56. Michalowicz, J., Nichols, J. & Bucholtz, F. Hand-book of Differential Entropy (CRC Press, 2013).

57. Miller, E. G. A new class of entropy estima-tors for multi-dimensional densities. In Acous-tics, Speech, and Signal Processing, 2003. Pro-ceedings.(ICASSP’03). 2003 IEEE InternationalConference on, vol. 3, III–297 (IEEE, 2003).

58. Stowell, D. & Plumbley, M. D. Fast multidi-mensional entropy estimation by k-d partition-ing. IEEE Signal Processing Letters 16, 537–540(2009). DOI 10.1109/lsp.2009.2017346.

59. Grassberger, P. Information and complexity mea-sures in dynamical systems. In Information dy-namics, 15–33 (Springer, 1991).

60. Cheng, Q. Multifractality and spatial statistics.Computers & Geosciences 25, 949–961 (1999).

61. Arneodo, A., Muzy, J.-F. & Sornette, D. ” direct”causal cascade in the stock market. The Euro-pean Physical Journal B-Condensed Matter andComplex Systems 2, 277–282 (1998).

62. Schertzer, D. & Lovejoy, S. Physical modelingand analysis of rain and clouds by anisotropicscaling multiplicative processes. Journal of Geo-physical Research: Atmospheres 92, 9693–9714(1987).

63. Mandelbrot, B. B. Intermittent turbulence inself-similar cascades: divergence of high mo-ments and dimension of the carrier. Journalof Fluid Mechanics 62, 331–358 (1974). DOI10.1017/S0022112074000711.

64. Stanilov, K. & Batty, M. Exploring the historicaldeterminants of urban growth patterns throughcellular automata. Transactions in GIS 15, 253–271 (2011).

65. Kloosterman, R. C. & Musterd, S. Thepolycentric urban region: Towards a researchagenda. Urban Studies 38, 623–633 (2001). DOI10.1080/00420980120035259.

66. Brueckner, J. K. & Fansler, D. A. The economicsof urban sprawl: Theory and evidence on thespatial sizes of cities. The Review of Economicsand Statistics 65, 479–482 (1983).

67. Shalizi, C. R., Shalizi, K. L. & Haslinger, R.Quantifying self-organization with optimal pre-dictors. Physical Review Letters 93, 118701(2004).

68. Wolfram, S. Statistical mechanics of cellularautomata. Reviews of modern physics 55, 601(1983).

69. Klimontovich, I. Turbulent motion and the struc-ture of chaos: a new approach to the statisticaltheory of open systems.

70. Garland, J., James, R. & Bradley, E. Model-free quantification of time-series predictability.Physical Review E 90, 052910 (2014).

71. Bettencourt, L. M., Lobo, J., Helbing, D., Kuh-nert, C. & West, G. B. Growth, innovation,scaling, and the pace of life in cities. Proceed-ings of the national academy of sciences 104,7301–7306 (2007).

72. Batty, M. & Longley, P. A. Fractal cities: ageometry of form and function (Academic Press,1994).

73. Chen, Y., Wang, J. et al. Multifractal character-ization of urban form and growth: the case ofbeijing. Environment and Planning B: Planningand Design 40, 884–904 (2013).

74. Chen, Y. & Zhou, Y. Multi-fractal measuresof city-size distributions based on the three-parameter zipf model. Chaos, Solitons & Fractals22, 793–805 (2004).

12/12