Linköpings universitetSE–581 83 Linköping
+46 13 28 10 00 , www.liu.se
Linköping University | Department of Computer and Information ScienceBachelor thesis, 16 ECTS | Innovativ Programmering
2018 | LIU-IDA/LITH-EX-G--18/040--SE
Evaluating performance of aReact Native feature set
Jesper SöderbergErik Johansson
Tutor : Dennis PerssonExaminator : Jody Foo
Upphovsrätt
Detta dokument hålls tillgängligt på Internet – eller dess framtida ersättare – under 25 årfrån publiceringsdatum under förutsättning att inga extraordinära omständigheter uppstår.Tillgång till dokumentet innebär tillstånd för var och en att läsa, ladda ner, skriva ut enstakakopior för enskilt bruk och att använda det oförändrat för ickekommersiell forskning och förundervisning. Överföring av upphovsrätten vid en senare tidpunkt kan inte upphäva dettatillstånd. All annan användning av dokumentet kräver upphovsmannens medgivande. Föratt garantera äktheten, säkerheten och tillgängligheten finns lösningar av teknisk och admin-istrativ art. Upphovsmannens ideella rätt innefattar rätt att bli nämnd som upphovsman iden omfattning som god sed kräver vid användning av dokumentet på ovan beskrivna sättsamt skydd mot att dokumentet ändras eller presenteras i sådan form eller i sådant sam-manhang som är kränkande för upphovsmannens litterära eller konstnärliga anseende elleregenart. För ytterligare information om Linköping University Electronic Press se förlagetshemsida http://www.ep.liu.se/.
Copyright
The publishers will keep this document online on the Internet – or its possible replacement– for a period of 25 years starting from the date of publication barring exceptional circum-stances. The online availability of the document implies permanent permission for anyone toread, to download, or to print out single copies for his/hers own use and to use it unchangedfor non-commercial research and educational purpose. Subsequent transfers of copyrightcannot revoke this permission. All other uses of the document are conditional upon the con-sent of the copyright owner. The publisher has taken technical and administrative measuresto assure authenticity, security and accessibility. According to intellectual property law theauthor has the right to be mentioned when his/her work is accessed as described above andto be protected against infringement. For additional information about the Linköping Uni-versity Electronic Press and its procedures for publication and for assurance of documentintegrity, please refer to its www home page: http://www.ep.liu.se/.
c© Jesper SöderbergErik Johansson
Evaluating performance of a React Native feature set
Jesper SoderbergLinkoping, Sweden
Erik JohanssonLinkoping, Sweden
ABSTRACTReact Native has quickly become one of the most popularcross-platform frameworks for developing Android andiOS applications since it was first released by Facebook in2015. In this study a proof of concept application is de-veloped and a set of features are evaluated with regardto performance efficiency. Results of measurements showthat while React Native does perform worse than it’s An-droid counterpart, it performs sufficiently well for build-ing most applications with but can be more problematicfor applications running heavier client-side logic.
Author KeywordsReact Native; cross-platform; performance; frames persecond; proof of concept; limitations; native module;
INTRODUCTIONThe mobile phone industry has become a vast industry andits user base has become split between different devices thatutilize different operating systems. For a software develop-ment company to reach this divided customer base they wouldhave to develop separate applications for the different operat-ing systems, which could become expensive to develop andmaintain.
A solution to this problem is to use a mobile cross-platformframework which allows you to develop an application thatworks on multiple operating systems. The purpose of thispaper is to construct a proof of concept application from afeature set inspired by another application and evaluate if oneof the most modern of these kinds of frameworks, React Na-tive is a suitable framework for the implementation of suchan application. The chosen evaluation method is to analyzeperformance in comparison to native Android.
React NativeReact Native is being developed by Facebook and was firstreleased in 2015. React Native works for both Android andiOS devices, and allows you to develop an application in Re-act that is then compiled into native applications for Androidor iOS. This allows you to develop and maintain one applica-tion that works with both Android and iOS, instead of havingto develop and maintain two separate applications for the dif-ferent operating systems.
There are however a few potential downsides to consider be-fore switching to React Native. The framework is anotherlayer that could potentially have bugs and technical limita-tions, requiring you to wait for Facebook to fix it or find away to circumvent the bugs. Another problem might stemfrom the fact that Facebook released React Native in 2015
which makes it relatively new, this in turn has given it lesstime to develop and might therefore be more difficult to findgood documentation for certain parts. Some applications willneed features written in native Android or iOS but in mostcases everything should be able to be implemented withouthaving to write native code.
Research QuestionIn our article we will determine important features in the na-tive applications that we will attempt to port over to React Na-tive from an already existing application that is owned by thecompany TaxiCaller. We will then measure the performanceof the implementation of the features and compare them withthe original application, in an attempt to get a better under-standing of how suitable React Native is as a framework whentrying to implement the set of features that we have hand-picked from the original application.
Suitability of using React Native when implementing a featureset with regards to performance efficiency of the implementa-tion.
The suitability is based on appropriate performance efficiencymetrics, which are chosen on a per feature basis. In the dis-cussion we will look at further potential problems that mightoccur when using React Native that might need to be con-sidered before deciding if it is the right framework to use forbuilding a certain application. Performance efficiency was se-lected because it is important to considered the performanceof a framework before porting or developing an applicationfor a framework. Our study aims to give developers a helpinghand in deciding if they should use React Native.
The Feature set that we have been selected based on the appli-cation we were given to make a corresponding React Nativeversion of and are all important features of that application.Some features both in the original version and the React Na-tive version might have been altered to be easier to measureduring our tests. The feature set we chose is the following:
Scrollable ListHaving a list is in a lot of applications a very common featurewhen you want to present things in a scroll-able view. Dueto how common this kind of UI component is in applicationsit is important to see if there are any potential problems withhaving a list in React Native. The current TaxiCaller applica-tion uses lists in several places and it is of importance to havelists that perform efficiently.
1
Screen TransitionTransitioning between different screens is an essential part ofalmost any application. Therefore a poorly performing navi-gation system will affect the entire feel of an application in avery negative way. This feature involves screen changes (trig-gered by buttons or other actions) and a back action (triggeredby a hardware button or a button in the upper left corner).
Background ProcessBeing able to run tasks in the background could be a require-ment for certain types of applications. In TaxiCaller’s ap-plication there is a need to be able to ping their server ona frequent basis to keep the session active with the currentback-end implementation. This feature is therefore consid-ered quite important for the application to work as intendedwhich is one of the reason it was chosen. For us pingingthe server on a frequent basis equates to approximately ev-ery 5 minutes. Another requirement for this feature to befully implemented is the possibility for it to work when theapplication is in the background. It is also likely that morebackground processes would need to be added if the whole ofthe native application would be ported over to React Nativein the future. So there might be more complex requirementsthat could be part of certain applications when it comes tobackground processes that we won’t cover in this article.
Native ModuleLast of the features that were chosen is a native module. Thiswas chosen due to the fact that not all features are supportedby React Native but should be able to be implemented withnative code instead and then connected to JavaScript. In ourproject we decided to focus on a Java module instead of anObjective C module for iOS. The reason we chose to do thiswas due to the time constraints that we had and between An-droid development and iOS development we have more expe-rience working with Android so it should therefore be easierfor us to implement. While the original application includeda module that was meant to listen to user-datagram-packageswe decided to switch this into a module that runs Fibonaccicomputations to get more noticeable values during the perfor-mance test.
BackgroundHistorically developing applications for smart-phones has re-quired knowledge in different programming languages anddevelopment environments depending on the operating sys-tem used, iOS (Objective-C or Swift) and Android(Java). Onecommon way to circumvent this knowledge requirement fordifferent operating systems is to develop mobile web appli-cations. However these applications tend to perform worsethan native applications1 and for more advanced features youwould still have to partially write native code therefore mak-ing a hybrid app.
Another way to deal with this problem of the developer re-quiring knowledge of several systems is to instead make crossplatform applications. However historically there hasn’t beena good track record of frameworks that have managed to do
1https://www.lifewire.com/p-2373173(last accessed 2018-05-13)
this and perform well enough to compete with native applica-tions.
Facebook’s React Native is seen as having much potential asfar as cross platform frameworks are concerned. Being devel-oped by Facebook and released two years after they releasedReact, a JavaScript library for building user interfaces. ReactNative is already being used in a multitude of popular appli-cations for example Facebook, Instagram, Uber, Discord andSkype.2 Though studies show that React Native in terms ofperformance doesn’t perform as well as native applications(see section ”related work”), but it doesn’t fall much behindin performance either and manages to uphold a native feeling.
We will be working with TaxiCaller for this study. TaxiCalleris an IT company that specializes in cloud based traffic con-trol systems for taxi- and transportation systems. They cur-rently have a presence in 60 countries spanning 6 differentcontinents. 3
Part of their current system include two mobile applications,each with a version for Android and another for iOS. Main-taining and developing the four versions of these applicationstakes a lot of time and effort which in turn makes them costly.It should for that reason be beneficial for TaxiCaller to adoptReact Native instead. This would allow them to develop oneapplication for React Native, which with some extra work forthe specific platforms can be compiled to both iOS and An-droid applications. An important part of our work is that wecreate an application that will constitute a good proof of con-cept for porting the current application over to React Native.
The feature set that we will be focusing on implementing andevaluating is derived from the native application that Taxi-caller wants us to make a proof of concept for before convert-ing to React Native. We have chosen a handful of featureswhich are varied in nature. The choice to focus on these fea-tures arose from studying the application we had been askedto port and pick out features that had to be implemented at onepoint or another to give the same functionality as the Taxi-Caller application. Sometimes we have altered the featuresof the original application to better be able to test them. Ourintention was to look at features that could make for a bad ex-perience for either developers or users if they do not performrelatively close to their native counterparts.
THEORYThere have been four different approaches to the developmentof cross-platform applications according to CP Rahul and S.Tolety[11]. The four approaches are web, hybrid, interpretedand cross compiled. While they all aim to overcome the sameproblem of having multiple code bases for different platformsthey have different advantages and challenges.
Web approaches are web applications executed in the browserof the application and is therefore limited to the browser en-vironment not being able to utilize platform specific features.
2https://facebook.github.io/react-native/showcase.html (last accessed 2018-05-21)
3https://www.taxicaller.com/en/about(last accessed 2018-05-21)
2
Hybrid approaches are part web application and part nativecode for platform specific features. But usually suffer in per-formance since the executions happen in the browser engine.
Interpreted approaches execute the code in runtime on theplatform accessing the native features via an abstraction layer.
Cross compiling approach allows you to compile your codefor different platforms but makes you unable to reuse code forplatform specific features.
React Native is a newer approach to cross-platform frame-works from those described in [11] but it resembles the inter-preted approach the most however it uses API calls for render-ing the application which is not explained in the interpretedapproach. React Native runs JavaScript on top of the nativeiOS or Android OS. Since JavaScript is not the native lan-guage of either of these two platforms you need a way to ex-ecute the code. The technique to do this is commonly knownas bridging and is not unlike how normal React interacts withthe virtual DOM. By running the ”JavaScript Core”, which isa runtime environment for JavaScript that when run on top ofthe native platform allows us to be able to run JavaScript codeon the platform. See Figure 1 for a visual explanation of thearchitecture.45
Figure 1. React Native architecture explained4
React Native packagesReduxThe Redux library for React Native is a JavaScript library thatimplements the flux architecture that was created by Face-book. One of the purposes of flux is to allow you to easiermanage the data flow by allowing the data only to be sent oneway. In Redux the state is stored in what is simply known asthe store. You change the state by dispatching actions which
4https://www.logicroom.co/react-native-architecture-explained/(last accessed 2018-05-13)
5http://www.discoversdk.com/blog/how-react-native-works (last accessed 2018-05-13)
are then caught by the Reducers which in turn modify thestate.6
Redux-LogicRedux-Logic is a library that complements Redux by han-dling logic in an asynchronous manner. It allows you to cre-ate Action side-effects, fetch data and intercept actions. Thislets applications dispatch actions from a component to bothchange the state with a reducer as well as perform businesslogic, like API or database calls.7
List componentsListView is the early version for handling lists in React Na-tive, however this version is now deprecated and has beenreplaced by FlatList and SectionList.
FlatList is the most basic of lists of the two. FlatList only ren-ders items that are on the screen at the moment, and doesn’tkeep the items that are outside of the view within the mem-ory. This allows for more efficient utilization of the memory.However the downsides to this is that you instead need tostore the items in the list outside of the list for example withinthe Redux state. Pure components unlike normal componentsare more optimized for re-rendering since they automaticallyhave an implementation of a method called shouldCompo-nentUpdate(). This method does a shallow compare betweenthe new state and the previous state to see what needs to be re-rendered, and allows it to avoid re-rendering identical infor-mation. However be aware that shallow copies do not alwaystrigger a re-render on certain elements in the list.8
React-NavigationReact-Navigation is a React Native community solution forscreen navigation within React Native and is even recom-mended by Facebook in the official documentation. It al-lows you to navigate between different screens on both iOSand Android versions of the application unlike NavigatorIOS.React-Navigation allows for easy to override navigation logicthat integrates well with Redux. It also uses the React Na-tive animated library to allow for animations that can be cus-tomized and allows for easy applicability of custom headersto pages. A few of the different types of navigator compo-nents are for example stacks, tabs and drawers.9
Performance EfficiencyPerformance efficiency is according to the ISO standard de-fined as ”performance relative to the amount of resourcesused under stated conditions”.[1] Different sub category qual-ities are relevant depending on the feature being observed, itis therefore important that we look at each feature and tryto determine which metrics of performance efficiency are the
6https://medium.com/@pavsidhu/using-redux-with-react-native-9d07381507fe(last accessed 2018-05-13)
7https://github.com/jeffbski/redux-logic/blob/29ffc28b46a1e1fe9ddc7f130bff43a20e636ba7/README.md(last accessed 2018-05-13)
8https://facebook.github.io/react-native/docs/flatlist.html (last accessed 2018-05-13)
9https://facebook.github.io/react-native/docs/navigation.html#react-navigation(last accessed 2018-05-13)
3
most relevant to determining the performance of the featurein the application. Performance efficiency plays an importantrole in deciding which framework to use since we do not wantapplications with poor performance.
Related workCross-platformAn article by Ahamed, S. I., Pezewski, A., and Pezewski,A.[2] where they attempt to explore the subject area and se-lect different criteria which might be of interest to look atwhen selecting a framework. Some of the criteria they de-cided might be useful were the domain of the framework,the ability to extend the framework, coupling, language re-quirements and platform requirements. In our article we havetouched upon which the intended domain for React Native is,the language requirements, platform requirements and alsothe ability to extend the framework using native modules.
D. Garlan, R. Allen and J. Ockerbloom [5] look at architec-tural mismatches when a developing a system from differentparts. The article demonstrates the problems of developingthese kinds of systems by constructing a prototype using avariety of different parts all written in c++ or C and with ex-tensive usage by companies. Their results show that thesesystems can cause a lot of mismatches and make it both timeconsuming to develop and create bad performance. Theytherefore recommend others to avoid using existing parts toconstruct a system when they can. In our work we will takea look at a Java module which can constitute as a differentpart then the rest of our system and can therefore cause mis-matches on an architectural level.
React NativeA study by N. Hansson and T. Vidhall[6] evaluate and com-pare a React Native application to native Android and iOSapplications. In their study they look at the performance ef-ficiency and conduct a user experience study on the applica-tion. Evaluating performance efficiency by looking at CPUusage, memory usage, response time, frames per second andapplication size. Response time was measured by conductinguser tasks and measuring the response time of the applica-tion during these tasks, while frames per second, CPU usageand memory usage were tested via performance test scenar-ios. There were three performance test scenarios that were ranusing automated test scenarios to make them easier to repli-cate. Their comparison shows that React Native performsworse than the native counterparts but that the difference inperformance was smaller than expected.
A similar study is done by F. Carlstrom and O. Axelsson[3]in which they attempt to evaluate a React Native applicationin a large extent of different ways including measuring per-formance efficiency, conducting a user experience study, test-ing the reliability and by observations during the developmentprocess. In terms of performance efficiency in their case theylook at CPU usage, memory usage, battery, network and ap-plication size. It is clear that the benchmark performance testwas not the main focus of the work in this study and it istherefore just a simple benchmark test on the final productwhere they measure the usage of different resources in the ap-plication. They conclude that while React Native performed
worse than the native applications the historical gap that hasbeen between cross-platform frameworks and native code hasshrunk a lot and it is therefore a viable option to use React Na-tive. In this article they also go through their implementationprocess in detail, where the time taken to implement a spe-cific part of the application in both React Native, native iOSand native Android has even been given.
Finally in an article by A. Lelli and V. Bostrand[9] we no-tice a slightly different method. While the study itself fo-cuses on performance efficiency the article looks at differentscenarios. These scenarios are extremely similar to what wecall features. These features vary from graphically intenseactivities, application navigation and a ListView. They areevaluated with different metrics depending on the feature andare all compared to an Android native counterpart. Since theListView is now a deprecated technology in React Native andhas been replaced by FlatList which we will evaluate in ourstudy, it will be interesting to see the performance differencebetween the two. The Method that is used when evaluatingthe ListView looks at two different ListView implementa-tions. One containing a list of contacts including image, nameand phone and styled to look like a card and the other imple-mentation represents a photogrid with each element being 3images. They are tested having 500, 1000 and 1500 elementsand look at the frames per second when scrolling. The resultsof this feature shows that React Native performs worse thannative applications in terms of memory usage, CPU usageand frame rate stability it does however take less GPU usage.Navigation, startup time and battery is evaluated creating a“budget divider application” which is based on an applicationcalled “Tricky Tripper”. It is believed that this will be moreof a realistic model for evaluating these metrics since you gotseveral components being used in one application. Overallthe article concludes that the performance of React Native isworse than native counterparts, but is still a viable options forapplications that do not rely on heavy calculations.
EvaluationC. Ryan and P. Rossi[4] assess context-aware mobile appli-cation performance and resource utilization. By specifyingdifferent software attributes and defining a concrete metricand measurement technique to each attribute and testing aseries of proposed hypothesis using linear correlation anal-ysis. Results show the impact of the attributes on the effi-ciency of context-aware mobile applications. [4]proposes aformal model of mathematical equations in order to facilitatedynamic runtime placement of mobile objects and tests themodel on a practical application. Concluding that the localadaptation of the strategy results in higher efficiency relatedto performance and resource utilization.
J.K. Lee and J.Y. Lee [8] propose guidelines on how to pro-gram Android applications more efficiently. By comparingthe performance between two different programming lan-guages used for creating Android applications namely Javaand Native C. Looking at how the languages perform on fivedifferent categories; JNI delay, Integer, Floating-point, Mem-ory access algorithm and String processing. The results show
4
that Native C was faster on all categories except for Stringprocessing.
C. Ming, J. Lin, C. Dow and C. Wen[10] benchmark DalvikJava code and native code for Android. Conducted twelve dif-ferent tests to analyze the performance, concluding that nativecode is faster for about 34.2% while Dalvik Java code wasfaster in three of the tests and in one test according to [10] theDalvik Java code was said to perform very bad.
In a study by C. Hsieh, H. Falaki and N. Ramanathan[7]evaluating the performance of Android Inter-Process Com-munication(ICP) for Continuous sensing applications by con-structing a benchmark application and measuring the perfor-mance of a)Latency, b)Memory footprint and c)CPU usage.
METHOD
ImplementationWe will implement a proof of concept mobile application inReact Native that has all of our features. The following havebeen provided to us by TaxiCaller and have been utilized dur-ing our development:
• Current TaxiCaller Android/iOS code
• Code skeleton and common features of another applicationin development at TaxiCaller
• TaxiCaller back-end API
All development was done at TaxiCaller’s office where wecould get help with their systems. All features except for theJava module were developed to be functional for both iOSand Android.
The provided code skeleton includes router navigation, per-sistent storage, network actions, styled components and morecommon smaller components. This skeleton was provided tous by TaxiCaller and allowed us to spend more time on im-plementing our feature set. TaxiCaller also provides us witha back-end which the application communicates with for allnetwork actions.
React NavigationIn our implementation of the navigation system we chooseto go with React Navigation due to it being a popular optionand was not run using native code. By dispatching navigationactions we are able to navigate between different screens. Wealso store the last screen that we navigated to, to prevent re-navigation to the same screen without having navigated toanother screen in between.
The code Snippet in Figure 2 shows how navigation is usedinside the Redux reducer.
FlatListThe code snippet in Figure 3 shows how the FlatList is addedinto a view. InitialNumToRender defines how many items areinitially rendered. We chose eight items because that is howmany items filled just over the height of the screen. Keepingthis at a low value reduces the render time when the list isfirst shown. getItemLayout is an optimization that specifiesthe height of each element manually which results in smooth
1 var nextState = RootNavigator.router2 .getStateForAction(3 NavigationActions.navigate({4 routeName: action.routeName5 }),6 state7 );8 return nextState;
Figure 2. Navigation Implementation from a redux reducer
scrolling with no loading. Every item in the list will be of theShiftCard type which is a pureComponent. Data and buttonpress handling is passed along to the ShiftCard component.
1 const renderItem = ({item}) => (2 <ShiftCard3 setSelShift={this.onPress.bind(this)}4 shiftData={item}5 />6 );
1 <FlatList2 data={listData}3 initialNumToRender={8}4 onEndReachedThreshold={0.5}5 keyExtractor={(item, index) => index}6 renderItem={renderItem}7 getItemLayout={(data, index) => (8 {length: 106, offset: 106 * index, index}9 )}
10 />
Figure 3. Flatlist implementation inside a component’s render function
React Native Background processOur implementation of the background process was done byutilizing Redux Logic. As can be seen in the code in Figure4 by using Redux Logic we created a process that is activatedwhen it receives a specific action and cancelled when it re-ceives another specific type of action. In this process we thenuse a setInterval method to call the API once every intervalwhile the code has been activated.
1 const INTERVAL = 60000;2 export const bgProcessLogic = createLogic({3 type: NetworkActionTypes.REQUEST_SUCCESS,4 cancelType: localActionsType.SIGN_OUT,5 warnTimeout: 0,6 validate({getState,action}, allow, reject){7 action.requestType === "SIGN_IN" ?8 allow(action) : reject(action);9 },
10 process({ cancelled\$ }, dispatch, done) {11 const tick = setInterval(() => {12 dispatch(13 apiActions.sendProcess({})14 );15 }, INTERVAL);16 cancelled\$.subscribe(() => {17 clearInterval(tick);
Figure 4. Background process implementation inside a redux logic file
5
Java moduleOur current Java module implementation is a process that iscomputing Fibonacci. At first we were going to listen forUser Datagram Protocol packages on a specific port, after re-ceiving a package we would process the package so that wecould retrieve the appropriate content via an API call. But in-stead we decided to use Fibonacci numbers computations dueto the fact that it left a much larger footprint then our currentimplementation on the CPU usage which made it easier tocompare the results and Fibonacci is easy to implement in al-most any language therefore making it easier to compare withother frameworks in the future. The Fibonacci computationis recursive and calculates the first 30 numbers.
To get access to our Java module in React Native we exportedsome of the functions to React Native functions which arethen called to interact with the Java module.
Environments and toolsAll our tests will be performed on an Android system withboth our React Native application and the native Androidcounterpart. We will repeat the tests five times each to seeif there are any deviations in performance and to give amore reliable result. The physical device that will be usedis a OnePlus Two smartphone running Android version 6.0.1/ OxygenOS version 3.6.1. The OnePlus Two features anocta-core snapdragon 810 processor, 4 GB RAM and anAdreno 430 GPU. The most relevant node module versionsare react-native v0.54.2, react-navigation v1.5.8 and reduxlogic v0.12.3.
We will be using GameBench as the tool for measuring andanalyzing the performance. This tool allows us to look at amultitude of different metrics during our tests. GameBenchruns on Windows and installs a service on the Android de-vice which it then communicates with over USB. This servicedoes not affect the application being tested and every metricwe will be using is for the application and not the devices’total resource usage.10 On Android, GameBench uses dataprovided by the standard proc filesystem which is used byother utilities for real-time monitoring. Because the CPU thatwill be used is an octa-core, 12.5% (100% / 8 cores) total us-age means that one core could be running at 100% and causeissues. 11
Evaluation methodIn this section we will talk about what qualities should beevaluated, why we chose those qualities and how they will bemeasured. To evaluate our features’ performance efficiency,a method similar to that of T. Vidhall[6] is the most appropri-ate. This method will be based on running test scenarios werewe measure the different qualities. We decided against usingautomated test scenarios due to technical difficulties and timeconstraints.
FlatListWhen picking the qualities for the FlatList evaluation we triedto include the qualities used in the study by Lelli that were
10https://www.gamebench.net/ (last accessed 2018-05-13)11https://docs.gamebench.net/web-dashboard/
the-cpu-pane (last accessed 2018-05-21)
used to evaluate the ListView. Since FlatList is the replace-ment for the now deprecated ListView it will be valuable tobe able to compare performance. The qualities that we lookat are CPU usage, GPU usage, memory usage and frames persecond. Since the FlatList loads and renders elements whenscrolling the list these are the most important qualities to mea-sure.
Our test scenario is based on the study by A. Lelli[3]. Inthis test scenario we will have 500 list elements containingsome formatted text labels. The test will be started with a twosecond idle period before we start manually scrolling throughthe list for 15 seconds, followed by another two second idleperiod.
Screen TransitionThe most relevant qualities to look at when it comes to navi-gation are those that indicate how well the navigation handlestransitioning between activities. We chose the metrics CPUusage and GPU usage because similarly to FlatList, duringtransition a new activity will be loaded and rendered in theapplication.
This test scenario is based on the transition phase betweentwo screens, since this is the main purpose of navigation. Thetest will be started with a two second idle period followed bya button press to navigate, a three second delay, a back buttonpress and a two second delay.
Background ProcessIn terms of performance efficiency we do not expect any ex-treme performance differences in our current implementationof a background process. Since our background process onlypings the server it is a very light task and should barely affectperformance at all. If however we had a background processthat was a heavy computational process we could potentiallysee a big performance hit on the CPU. We will therefore lookat CPU usage since it is used during background computationunlike the GPU.
The test scenario for this feature is simple, it will run for 30seconds and only consists of a ping call to the server, whichwill be called every 500 milliseconds. We chose a low fre-quency of 500 milliseconds to simulate more network trafficrunning in the background. We will also measure the appli-cation in an idle state for the same amount of time. Then weshould be able to compare the results and see the spikes inperformance between when the background process is beingrun and the application is idle.
Java moduleSince the code is the same code in both React Native and na-tive Android we do not expect to see a noticeable difference init affecting performance. However since this module does norendering and mainly works similar to a background processwe will only look at the CPU usage.
We should expect a very similar result when the code is be-ing run independently to the native Java application. Our testscenario will consist of running Fibonacci calculations fre-quently to simulate the heavy calculations some applications
6
need to run. This math will run approximately every 100msfor the 30 second test.
RESULTS
FlatListWhen we look at the results of comparing the React NativeFlatlist with Android list in Figure 5 we can quite clearly seethat the CPU usage on the FlatList implementation is substan-tially higher then the Android version. However the Androidlist does have a higher utilization of the GPU which is mostlikely not a bad thing as we will discuss later.
The median frame rate of the Android version was 57.8 andfor FlatList it was 51. Both applications ran at a stable framerate and there were no visible stuttering or drops in frameratethat could be felt when scrolling.
Figure 5. CPU & GPU usage when scrolling the list
The memory usage of the applications can be seen in Figure6 and yet again show Android outperforming React Nativeby using less of the memory both during the application be-ing idle and while scrolling the list. The difference in thememory increase between idling and scrolling is far less theAndroid version, being only 20% compared to the differencethat React Native displays which is 35%.
Figure 6. Memory usage when Idle and during list usage
RN Idle #1 #2 #3 #4 #5CPU STDEV 0.156 0.166 0.19 0.158 0.164
CPU AVG 0.345 0.331 0.341 0.341 0.339RN Background #1 #2 #3 #4 #5
CPU STDEV 0.346 0.383 0.373 0.398 0.337CPU AVG 0.867 0.905 0.911 0.915 0.916
Android Idle #1 #2 #3 #4 #5CPU STDEV 0.163 0.151 0.123 0.162 0.136
CPU AVG 0.175 0.151 0.144 0.155 0.142A. Background #1 #2 #3 #4 #5
CPU STDEV 0.151 0.124 0.141 0.165 0.144CPU AVG 0.233 0.208 0.243 0.246 0.242
Table 1. Background and idle CPU average and CPU standard deviationfrom the test runs
Background ProcessThe background process results show us that the CPU usageis higher in React Native both when the application is in anidle state as well as when pinging the server frequently whichcan be seen in Figure 7.
By looking at the standard deviation in Table 1 of the test re-sults we can see that the deviation is much higher for ReactNative when running the background task then Android. Infact the idle deviation and background process deviation isabout the same for Android. Which should imply that An-droid does not get as huge peaks as React Native in CPUdifference. The growth in CPU utilization between idle andpinging is about 2.68 times. Compared to the growth of theAndroid version which is much lower being only 1.56 timesbetween idle and pinging.
Figure 7. CPU usage when idle and when pinging at 500ms delay
Screen TransitionResults from the navigation, just like previous results, showthat React Native has a higher CPU usage than the Androidversion as we see in Figure 8. And just like the FlatList theAndroid version instead has a higher utilization of the GPU.
7
Figure 8. CPU & GPU usage when navigating
Java ModuleThere was a drastic difference in terms of CPU usage whenit comes to the Java module as can be seen in Figure 9. Re-act Native CPU usage averaged 5.7% while Android was at amuch lower 0.49%.
Figure 9. Java module running Fibonacci, CPU usage
DISCUSSION
ResultsFlatListThe results show us that there is a higher CPU usage in Re-act Native while the Android application has a higher GPUusage. This result is in favor of Android since having betterutilization of the GPU for rendering related activities insteadof doing those rendering related calculations on the CPU ispreferable. So instead of properly utilizing the GPU ReactNative instead causes a heavier work load to be performed onthe CPU instead were it should be possible to have the GPUdo that work.
The frame-rate is a bit lower on the React Native applicationbut due to the problems when measuring FPS (see discussionmethod) the results might be less accurate than expected andsince the difference is rather small we did not notice a differ-ence ourselves.
The memory usage is also a bit worse on the React Nativeapplication and the growth rate of the memory usage is also
higher which tells us that the Android view for lists is moreefficient or potentially that React Native as a whole does notdeal as well with memory headers. Since Android Java codeallows you greater responsibility over the memory usage thisprobably allows for greater optimization in terms of memorycompared to React Native which is written in JavaScript.
What is not shown in the measurements is that React Na-tive has difficulties keeping up with list item rendering whenscrolling fast. When the list is used normally and is quiteshort this is not an issue. But with long lists and the userwanting to scroll up or down to skip past most of the contentin the list, the rendering will be too slow. While the user isnot actually reading the content when scrolling this fast it ittakes a while to render when stopping which might cause fora worse user experience.
Screen TransitionJust like previous results we can once again see that ReactNative does not utilize the GPU as much as the Android ap-plication.
When developing using React Navigation we ran into oneissue. When quickly tapping a navigation button the appli-cation would dispatch several actions and create duplicatescreens or go back several steps if the back button was pressedquickly. On Android the hardware back button also had to beproperly connected to the system. In the end we managed tofix this problem in a good and simple way that seems to havesolved the ”bug”. Our Redux state stores the screen last nav-igated to and it can not be navigated to again without anothernavigation action in between. The back button can now onlydispatch an action once for every time a screen mounts whichsolves the double tap back clicking that caused the applicationto skip back to the sign in menu.
Due to large amount of different packages for managing nav-igation it means that there are potentially a lot of optimizationthat can be done and packages that present more performanceefficient solutions. Most likely the React Native Navigationpackage will perform better then our current implementationsince it utilizes more native code which can be seen in ourresult is usually better in terms of performance.
We also believe from our experience and due to the largeamount of different packages that more advanced navigationfeatures should not cause a huge problem for creators due tothe flexibility that React Native has in terms of navigation.
Transition animations used by React Navigation we did notoptimize appropriately instead we opted to disable them tospeed up the navigation and to factor them out of the tests.This gave the navigation of both applications a similar feeleven though the Android version has a slight fade in anima-tion when navigating. It is possible that these animationscould be tweaked to work better but we prioritized perfor-mance and minimal transition delay.
Background processIn this case, unlike the other two features, there was no ren-dering taking place but the Android application still outper-forms the React Native application by quite a lot in terms
8
of CPU usage. In fact the difference in CPU usage growthbetween idle and pinging the server is much larger in the Re-act Native application. While our task is a very lightweighttask resource wise if the CPU usage growth trend would bemaintained for more resource heavy tasks it would make Re-act Native a bad alternative in terms of resource heavy tasks.This is also shown in the article [9] where they conclude thatReact Native is not fit for handling resource heavy tasks.
In terms of solutions for implementing the background pro-cess we had two other options that we decided against. Onewas using a setInterval() inside of a component but this feltvery incorrect due to the fact that we want to separate thelogic side of things and the components. The other optionwas to create a worker that would work on a different threadand call the API every interval. While this solution provideswith more accuracy due to it being on a separate thread itdoes make it harder to dispatch actions since the worker stateis separate from the main application state.
A very important problem arises when you have slightly dif-ferent requirements for this feature. If the feature has a re-quirement that the background process should be run every5 minutes while the application is in the background state ofthe phone it will not be able to do this straight out of the box.All packages that are meant for background process only al-lows background processes in the background state to run ev-ery 15 minutes. This is due to a restriction on iOS deviceswhere normally application are only allowed to run things inthe background every 15 minutes unless they are registeredas needing a specific type of background process like play-ing music or using the GPS. The packages for React Nativehowever are either built for the strictest requirement and onlyallow for background process to be run every 15 minutes orare solutions exclusive to the Android version. It is unclearif there is a viable solution for this on iOS devices especiallyif you still have the desire for the process to be run inside theReact Native code and not a iOS module. More research isrequired to determine whether or not it is possible to do inReact Native but we can state that it is currently not a simplesolution that works by just installing a package but instead amore complex solution would be needed.
Java ModuleThe Java module results are to us quite unexpected, we hadassumed that the Java module would perform almost identi-cally to that of the Android application since they were boththe same Java code. Instead the result showed a staggeringdifference in performance between the two, where the ReactNative Java module performs considerably less efficient thenthe Android version.
We are not at all certain as to why the React Native Java mod-ule performs so much worse. It might potentially be a prob-lem in our implementation or it might be a difference in howReact Native handles the Java modules.
Setting up a Java module that works with React Native is notthat complicated. However in our case it took quite a lotof time and effort because we had to take already existingcode from the Android application and try to decouple it and
bring over the appropriate dependencies. We later decided tochange the code to a Fibonacci calculation as to make a morenoticeable print in the CPU usage.
MethodProperly measuring the mobile phone performance of an ap-plication proved to be a much more difficult feat than ex-pected. We encountered problems both when trying to mea-sure the GPU usage as well as the FPS. The GPU was dif-ficult to test accurately because of the effect the interactionwith the smart-phone screen had on the GPU usage. Everytime we touched the screen it would cause a spike in GPUusage. These spikes would vary in how long they were de-pending on how long you touched the screen. So when wetried to do a test that involved us pressing a button the slightvariation in time it took to press the button would change theresult, causing the test to be less accurate.
Our other difficulty was measuring FPS because when noth-ing was rendering or moving on the screen the FPS woulddrop to 0 instead which causes the result to be worse then itactually should be. This could potentially be caused by per-formance optimization in the Android device but it does makebenchmarking FPS more difficult.
We did not perform automated test scenarios due to problemswith finding a working tool in the scope of time we had. In-stead we did our test manually which most likely decreasedthe accuracy of our results. A good example of this is theGPU usage spikes, these were varied due to our human inter-action with the screen if they were automated we would havehad more consistent spikes and it could therefore be easier toignore them in the results. The tests also varied in executiontime between a few seconds because of the human elementwhich might sometimes affect the results accuracy. This willalso lower the replicability and the reliability of the results.
We are not certain how accurate the performance differenceis since we only constructed one of the applications and theAndroid application that we compared with has a lot morefeatures implemented that might affect the performance andhave had several years in which it most likely has been opti-mized more than our React Native application.
Since large parts of the code that is used belongs to TaxiCallerit also makes it more difficult to replicate the exact tests how-ever replicating the features should still be possible with yourown implementation.
Due to time constraints we chose to implement a Androidmodule only and not an iOS module. Given more time wewould have wanted to implement an iOS module and alsoperform tests on an iOS device to give a more extensive per-formance analyzes.
The performance tests we ran were not very heavy on ei-ther CPU or GPU and it could have been more interestingto benchmark more resource heavy features.
Due to the problems explained above the results become lessaccurate however their validity is still fairly good. We haverealized that it might not be optimal using averages when talk-
9
ing about things like CPU usage and GPU usage since theymight give miss leading results but we have chosen not to in-clude standard deviations for any more then one result. Thereason being that the standard deviation is already expected tobe very high for the other tasks since they are not constant inutilization of resources but include resting and active phases.
The academic sources that we have been using our article ap-pear to be reliable sources of information. We largely chosethe articles based on how they related to React Native apartfrom two of the articles which were chosen more due to themrelating to the theme of framework suitability. We attemptedto find articles that provided a fairly high replicability, relia-bility and validity and we succeeded fairly well in doing this.We have taken inspiration from certain articles both in thechoice of what features to look at and in terms of evaluationmethod.
ETHICSOur results could help people who are considering using Re-act Native to determine whether React Native is a solid choicefor building their application. Potential future work that couldbe of interest is creating possible solutions to solve back-ground tasks when the application is in the background stateon a more frequent basis then every 15 minutes. It wouldalso be useful to see other studies taking on the problem ofperformance differences related to Java modules compared toJava to see if this is actually a big problem or maybe just animplementation related problem.
CONCLUSIONThe aim of this thesis was to evaluate the performance as-pects of React Native by creating a proof of concept appli-cation. The developed application was a port of TaxiCaller’sAndroid application and was compared to that application.Implemented features include scrollable list, screen transitionsystem, background process and native Java module. Eachfeature was evaluated by measuring its performance and theimplementation was discussed in regard to difficulties and op-timization details.
The study concludes that React Native does perform worsein all tests but it is unclear if the difference in performanceis large enough to affect most developers. The most impor-tant observation we can make is that there is a noticeableperformance difference when comparing a Java module forReact Native to a completely native Android app. There isalso a large uncertainty whether it is possible to run frequentbackground tasks when the application is running in the back-ground.
The benefits of using a cross-platform framework mightstill outweigh the weaker performance of React Na-tive. Developing resource demanding applications inReact Native is not something we can recommend sincethe performance difference may be more noticeable forthose kinds of applications that utilize more resources.
REFERENCES
1. ISO/IEC 25010:2011 systems and software engineering- systems and software quality requirements andevaluation (square) - system and software qualitymodels, 2011.
2. Ahamed, S. I., Pezewski, A., and Pezewski, A. Towardsframework selection criteria and suitability for anapplication framework. In International Conference onInformation Technology: Coding and Computing, 2004.Proceedings. ITCC 2004., vol. 1 (April 2004), 424–428Vol.1.
3. Axelsson, O., and Carlstrom, F. Evaluation targetingreact native in comparison to native mobiledevelopment. http://lup.lub.lu.se/student-papers/record/8886469,2016.
4. Caspar, R., and Pablo, R. Software, performance andresource utilisation metrics for context-aware mobileapplications.
5. Garlan, D., Allen, R., and Ockerbloom, J. Architecturalmismatch or why it’s hard to build systems out ofexisting parts. In 1995 17th International Conference onSoftware Engineering (April 1995), 179–179.
6. Hansson, N., and Vidhall, T. Effects on performance andusability for cross-platform application developmentusing react native.http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-130022, 2016.
7. Hsieh, C.-K., Falaki, H., Ramanathan, N.,Tangmunarunkit, H., and Estrin, D. Performanceevaluation of android ipc for continuous sensingapplications. SIGMOBILE Mob. Comput. Commun. Rev.16, 4 (Feb. 2013), 6–7.
8. Lee, J. K., and Lee, J. Y. Android programmingtechniques for improving performance. In 2011 3rdInternational Conference on Awareness Science andTechnology (iCAST) (Sept 2011), 386–389.
9. Lelli, A., and Bostrand, V. Evaluating applicationscenarios with react native : A comparative studybetween native and react native development.http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-314516, 2016.
10. Lin, C. M., Lin, J. H., Dow, C. R., and Wen, C. M.Benchmark dalvik and native code for android system.In 2011 Second International Conference onInnovations in Bio-inspired Computing and Applications(Dec 2011), 320–323.
11. Raj, C. P. R., and Tolety, S. B. A study on approaches tobuild cross-platform mobile applications and criteria toselect appropriate approach. In 2012 Annual IEEE India
Conference (INDICON) (Dec 2012), 625–629.
10
