OpenFed: An Open-Source Security andPrivacy Guaranteed Federated Learning

Framework

Chen Dengshengdensechen@foxmail.com

National University of Defense Technology

China

September 17, 2021

Abstract

The broad application of artificial intelligence techniques ranging fromself-driving vehicles to advanced medical diagnostics afford many benefits.Federated learning is a new breed of artificial intelligence, offering tech-niques to help bridge the gap between personal data protection and utiliza-tion for research and commercial deployment, especially in the use-caseswhere security and privacy are the key concerns. Here, we present OpenFed,an open-source software framework to simultaneously address the demandsfor data protection and utilization. In practice, OpenFed enables state-of-the-art model development in low-trust environments despite limited localdata availability, which lays the groundwork for sustainable collaborativemodel development and commercial deployment by alleviating concerns ofasset protection. In addition, OpenFed also provides an end-to-end toolkit tofacilitate federated learning algorithm development as well as several bench-marks to fair performance comparison under diverse computing paradigmsand configurations.

1

arX

iv:2

109.

0785

2v1

[cs

.CR

] 1

6 Se

p 20

21

1 IntroductionIn traditional machine learning, the efficiency and accuracy of models rely oncomputational power and training data of a centralized server. Shortly speaking,user data is stored on the central server and utilized for training and testing pro-cesses in order to develop comprehensive machine learning models ultimately.Such centralized-based machine learning approaches, in general, are associatedwith different challenges including computational power and time, and most im-portantly security and privacy with respect to users’ data that has been neglectedfor a long time. Federated learning is a newly introduced technology McMahanet al. (2017) that has attracted a lot of attention from researchers to explore itspotential and applicability Wang et al. (2020); Yu et al. (2020) to address suchissues.

Federated learning Aledhari et al. (2020) offers a way to preserve user pri-vacy by decentralizing data from the central server to end-devices and enablesartificial intelligence (AI) benefits to domains with sensitive data and heterogene-ity. Within the federated learning framework, the focus is geared towards collab-oration, which is not always achieved through standard machine learning algo-rithms SUZEN and SIMSEK (2020). This paradigm came to light mainly for tworeasons: (1) The unavailability of sufficient data to reside centrally on the server-side (as opposed to traditional machine learning) due to direct access restrictionson such data; and (2) Data privacy protections using local data from edge de-vices, i.e., clients, instead of sending sensitive data to the server where networkasynchronous communication comes into play. Preserving data privacy providesfeasibility to leverage AI benefits enabled through machine learning models effi-ciently across multiple domains, ranging from medical to Internet of Things (IoT),transportation, defense and mobile apps. It allows the algorithm(s) used to gainexperience, which is also something that cannot always be guaranteed throughtraditional machine learning methods Lin et al. (2020); Pandey et al. (2020). Ad-ditionally, computational power is shared among the interested parties instead ofrelying on a centralized server by iterative local model training process on end-devices. Its applicability makes federated learning highly reliable, with severalexperiments having been conducted already.

Despite its promising potential, federated learning is still not widely under-stood in regard to some of its technical components such as platforms, hardware,software and others regarding data privacy and data access Alexander et al. (2020).In a less positive way and as good as the federated learning sounds, the introduc-tion of federated learning has arguably required more profound research into its

2

confirmation, particularly with respect to security and privacy aspects. Therefore,our focus in this paper is to expand on the technical aspects of federated learningwhile presenting OpenFed with detailed examples of federated leveraging basedarchitectures that offers techniques to help bridge the gap between personal dataprotection and data utilization for research and industry.

2 Library functionality and contributionsOpenFed was developed as an extension to PyTorch ecosystem for open-sourcesecure and privacy-preserving machine learning tools in terms of research, appli-cability and usefulness. Although a lot of progress has been made, existing effortsare confronted with a number of limitations that we argue are critical to federatedlearning research:

Lack of diverse computing paradigms and configurations supports. Feder-ated learning is diverse in network topology, exchanged information and train-ing procedures. Although simulation-oriented federated learning libraries such asTensorFlow-Federated (TFF) Ingerman and Ostrowski (2019), PySyft Ziller et al.(2021) and LEAF Caldas et al. (2018) are developed, they only support centralizedtopology-based federated learning algorithms like FedAvg McMahan et al. (2017)or FedProx Li et al. (2018) with simulation in a single machine, making them un-suitable for federated learning algorithms which require the exchange of complexauxiliary information and customized training procedure. Production-oriented li-braries such as FATE Yang et al. (2019) and Paddle-Federated-Learning Ma et al.(2019) are released by industry. However, they are not designed as flexible frame-works that aim to support algorithmic innovation for open federated learning prob-lems.

Lack of standardized federated learning algorithm implementations and bench-marks. The diversity of libraries and benchmarks used for federated learningin existing work makes it difficult to fairly compare their performance. What’sworse, the not independent and identically distribution (non-I.I.D.) characteristicof federated learning makes such comparison even more challenging Hsieh et al.(2020). Firstly, training the same deep neural network on the same dataset withdifferent non-I.I.D. distributions produces varying model accuracies. Secondly,one algorithm that achieves higher accuracy on a specific non-I.I.D. distributionthan the other algorithms may perform worse on another non-I.I.D. distribution.

3

OpenFed provides an end-to-end toolkit to facilitate federated learning algo-rithm development and a benchmark to fair performance comparison under di-verse computing paradigms and configurations. In doing so, OpenFed can con-tribute an overall blueprint for data scientists and other researchers on designingfederated learning based solutions for alleviating future challenges. We summa-rize the key differences between OpenFed and existing federated learning librariesand benchmarks in Table. 1. The outline of the contributions of this paper providesthe following innovations:

• Compared to other open-source libraries on federated learning, OpenFedprovides a PyTorch-style framework to enable researchers to get up to speedabout experiments or deployments quickly, giving them enough knowledgeto pursue the topic of federated learning without having to endure possiblesteep learning curves.

• We also provide solid examples of applications and use cases of federatedlearning to illustrate how different architectures of federated learning canbe applied for multiple scenarios, allowing the audience to better under-stand how OpenFed can be applicable. Besides, highlighting use cases andapplications of federated learning particular medical settings would allowhealthcare professionals to have more faith in streamlining their data forfederated learning under OpenFed.

• OpenFed is compatible with a wide range of existing machine learningprojects, easily user-configurable and introduces functional improvementsto federated learning training (weighted gradient descent/federated averag-ing, diverse data augmentation, local early stopping and federation-widehyper-parameter optimization), increasing flexibility, usability, security andperformance.

• Finally, OpenFed also provides an end-to-end toolkit to facilitate federatedlearning algorithm development and a benchmark to fair performance com-parison under diverse computing paradigms and configurations. Addition-ally, we offer insight into best design practices for designing federated learn-ing models.

4

Platform Focus Supporting softwareCrypTen 2019 Preserving privacy PyTorchFATE 2019 Deployment FATEBoard and FATEFlowFedML 2020 Research PyTorchFFL-ERL 2018 Real-time system ErlangLEAF 2018 Benchmark Python libraryPySyft 2021 Preserving privacy Python libraryTensor IO 1 Mobile devices deployment TensorFlowTFF 2019 Deployment TensorFlowOpenFed Deployment with benchmark PyTorch

Table 1: A summary of federated platforms, their focus and supporting software.

Figure 1: Three-layer abstraction framework of OpenFed.

5

Figure 2: The training and deployment phases of OpenFed.

3 Architecture designEach time a new technology is introduced and a new ecosystem is created, arange of technical ripple effects typically come into fruition over time. Privacy-preserving promises of federated learning attracts different domains that may con-tain sensitive data. As seen in recent publications, the majority of work proposedin the federated learning space aims to apply this new framework in some shapeand form to different domains, one example being in the healthcare sector Stoianet al. (2008); Truex et al. (2019) for the COVID-19 pandemic analysis Raisaroet al. (2020); Vaid et al. (2021). To an extent, federated learning does solve privacyconcerns of sensitive data in machine learning environments. However, model pa-rameter sharing, an increased number of training iterations and communicationsexposes the federated environment to a new set of risks at the same time and opensnew avenues for hacking Narayanan and Shmatikov (2008) as well as curious at-tackers to trace vulnerabilities to manipulate machine learning model output orget access to sensitive user data. To ensure that we out-turn the benefits of fed-erated learning over risks and utilize the features of federated learning properly,we have an immediate need to be on top of this area of research to investigate allpossible security and privacy attacks on federated learning. Without precise infor-mation and clear vision, federated learning may be pushed back without giving afair chance to explore and leverage its benefits.

6

The federated learning technology adopters and developers should adhere toinformation security fundamentals such as Confidentiality, Integrity and Avail-ability. The decentralized approach of having a huge number for collaborativetraining and exposure of model parameters makes federated learning vulnerableto various attacks and open to risks. Current research on exploring vulnerabilitiesand proposing frameworks to mitigate the risks is very limited.

A vulnerability can be defined as a weakness in a system which gives an oppor-tunity to curious/malicious attacker to gain unauthorized access Mothukuri et al.(2021). Knowledge of knowing (open) vulnerabilities of a system or frameworkhelps to manage and defend against the possible attacks. Identifying vulnerabili-ties will help to build a more secure environment by implementing pre-requisitesfor defending loopholes. Failing to protect usage and exposure of personally iden-tifiable information (PII) or failing to adhere to data protection laws will not justcause bad publicity, it can also cost many more consequences by law. It is amandatory step for federated learning developers to scan for all sources of vulner-abilities and tighten defenses to ensure the security and privacy of the data.

Figure. 1 provides an overview of OpenFed library. OpenFed is built uponPyTorch, which provide an flexible communication inference between differentnodes and users can implement their own algorithms in a PyTorch-style. Open-Fed also supports a variety network topologies that can be used in many existingfederated learning algorithms Lian et al. (2017). In addition, security/privacy-related functions are also supported to defence various attacks, such as data andmodel poisoning Nuding and Mayer (2020); Feng et al. (2020).

With OpenFed, the models get trained at the device level. In other words, themodels are brought over to the data sources or devices for training and predic-tion Li et al. (2019). The models (i.e., models’ updates) are sent back to the mainserver for aggregating. Then, one consolidated model gets transferred back to thedevices using concepts from distributed computing Yedder et al. (2021). This isso that we can track and re-distribute each of the models at various devices. Suchapproach is very advantageous for utilizing low-costing machine learning modelson devices such as cell phones and sensors Doku et al. (2019).

We also provides a distributed computing paradigm that is essential for scenar-ios in which large deep neural network training cannot be handled by standalonesimulation due to GPU memory and training time constraints. This distributedcomputing design is not only used for federated learning, but it can also be usedfor conventional in-cluster large-scale distributed training. OpenFed also suggestsa machine learning system practice that separates the implementations of models,datasets and algorithms. This practice enables code reuse and fair comparison,

7

(a) Automatic driving (b) Drug discovery

(c) Clinical diagnosis (d) Recommendation system

Figure 3: Real-life case

avoiding statistical or system-level gaps among algorithms led by non-trivial im-plementation differences. Another benefits is that federated learning applicationscan develop more models and submit more realistic datasets without the need tounderstand the details of different distributed optimization algorithms.

4 Real-life application scenarios and benchmarkThere have been positive impacts on using federated learning, Fig. 3. Federatedlearning has been used by three major companies to develop computer visionbased safety hazard warning applications. It has also managed to help consumersimprove their operational efficiency, achieve data privacy protection and reducedcost. Such an application of federated learning is a notable example of how feder-ated learning can be fully applicable and garner a positive impact, which inspiresother applications such as the medical and healthcare settings, since these indus-tries greatly need newer, efficient and accurate approaches for handling their data.

8

Due to the number of applications that incorporate federated learning, here, wecan build a few use-cases of how federated learning would be applied towardscertain problems:

Automatic driving Automated driving systems promise a safe, comfortable andefficient driving experience. The architecture of the autonomy system of self-driving cars is typically organized into the perception system and the decision-making system. The perception system is generally divided into many subsys-tems responsible for tasks such as self-driving-car localization, static obstaclesmapping, moving obstacles detection and tracking, road mapping, traffic signal-ization detection and recognition among others. The decision-making system iscommonly partitioned as well into many subsystems responsible for tasks suchas route planning, path planing, behavior selection, motion planning and control.However, fatalities involving vehicles equipped with automated driving systemsare on the rise. Unfortunately, it is often difficult to form enough training data dueto privacy concerns and high costs of transmitting video data. Notably, while fed-erated learning is a promising approach to solve this issue, not everyone is familiarwith it. As a result, there is no easy-to-use tool that can enable automated driv-ing developers who are not experts in federated learning to fully apply it to theirsystems to gain the most out benefit. Thus, OpenFed provide benchmark-CV fordeploying federated learning about computer vision which can be used to improvethe perception system and benchmark-RL for deploying federated learning aboutreinforcement learning which can be used to improve the decision-making system.

Drug discovery Most works cover applicability of federated learning for drugdiscovery, however, these works tackle specific problems related to drug discoveryand apply federated learning to solve them Xiong et al. (2020); Chen et al. (2020).Biased data can be a huge problem for model training because the data itself canbe skewed due to these biases. Others also focus on using what is called the quan-titative structure-activity relationship (QSAR) and applying federated learning toit for drug discovery Chen et al. (2020). QSAR analysis is typically used in, andhaving facilities collaborate usually leads to better results. Unfortunately, there isstill hindrance regarding intellectual property, which can hinder collaboration fordrug discovery via QSAR. It is typically used to investigate and predict variousproperties of compounds, which is crucial for drug discovery. OpenFed providesa flexible API for researchers to deploy various experiments about drug discovery.

9

Clinical diagnosis OpenFed manages to propose benchmark-medical for ac-cessing and analyzing biomedical data without sharing information. Specifically,the focus is on analyzing brain structure relationships across various diseases.There is a lot of data containing brain images, so there is a lot of opportunities tofully comprehend the genetic mechanisms for various brain-related diseases. Un-fortunately, datasets, which are stored in unique places, cannot always be sharedbecause of privacy and legality concerns, so we are limited in fully exploiting datafor studying brain disorders. Patients are also concerned about their medical databeing used for future health insurance decisions and shared with their employers.Health providers also worry that if their health statistics are made publicly avail-able, they will lose patients or suffer huge consequences of they cannot assess theirperformance. OpenFed demonstrated the advantages of using federated learningand has possible use-cases for identifying rarer diseases with fewer patients.

Recommendation system Federated learning was also applied for the recon-struction system, such as ranking browser history suggestion Hartmann et al.(2019), Google keyboard query suggestions Yang et al. (2018) and mobile key-board prediction Leroy et al. (2019). Those scenarios are mostly based on thenatural language processing and appearing in mobile phones. Thus, those appli-cations needs to not only respect the consumers’ privacy but also not have anylatency, which is crucial for a mobile settings. Because this particular applicationof federated learning is mobile-based, developers had to ensure the consumersdata usage and user experience were not negatively impacted. Here, OpenFedprovide a benchmark-NLP for providing a more convenient comparison on theperformance of various recommendation systems.

5 DiscussionFederated learning as well as OpenFed have a set of challenges that need furtherresearch and developed. Based on our observations and results, we have identifiedthe following issues that could form future avenues for avenues of research.

Zero-day adversarial attacks and their supporting techniques Current de-fense efforts in federated learning are designed to protect against known vulner-abilities and specific predefined malicious activities, making them less useful indetecting attacks outside their design parameters when tested. Although, this phe-nomenon applies to virtually any machine learning application’s defense mech-

10

anisms, the probability is more in federated learning as we do not have manyversions in production that would have demonstrated the possibility of various at-tacks. OpenFed can provide a flexible way to help researchers better simulatingand analysing various zero-day adversarial attacks.

Trusted trace-ability A major challenge of federated learning is trace-abilityof the global machine learning model throughout the life-cycle of the underlyingmachine learning process. For instance, if a prediction value is changed in theglobal machine learning model, we will need to have backward tracking abilityto identify which clients aggregation values resulted in that change. If the logicbehind machine model behavior is a black-box, then we are forced to lose grip tological reality and blindly rely on human-made AI. OpenFed provides a few pre-liminary works leveraging block-chain technology Yazdinejad et al. (2020, 2019);Nyaletey et al. (2019) with federated learning to provide and trace transaction up-dates to the global machine learning model Kim et al. (2019); Salah et al. (2019);Majeed and Hong (2019), hoping to achieve a more transparent tracing of thetraining process in deep learning machine learning models.

Well-defined process with APIs Federated learning is a fairly new approachthat requires a detailed requirements of federated learning in different domains.As privacy is a key factor in federated learning, OpenFed helps you to focus onenhancing privacy and standardizing approaches for each requirement and pro-vides a process (with generic APIs) to implement such enhanced approaches moreeasily.

Optimize trade-off between privacy protection enhancement and cost Cur-rent research work shows how to enhance privacy protection in federated learningat the expense of sacrificing efficiency or accuracy. However, there are no researchworks on finding the proper encryption level for secure multiparty computation al-gorithm and the quantity of added noise. If the encryption level or the quantity ofnoise is not enough, the participants still suffer from the risk of privacy leakage.On the contrary, if the encryption level is too high or too much noise added tothe parameters, the federated learning model severally suffers from low accuracy.You can do the validation experiments with OpenFed without hesitation to find areasonable balance between them.

11

Build federated learning privacy protection enhanced frameworks in prac-tice There are currently some federated learning frameworks that can be uti-lized to implement federated learning-based systems such as TensorFlow Feder-atd, PySyft and FATE. OpenFed can be easily integrated and execute federatedlearning with most popular projects, such as such as MMCV 2 for computer vi-sion, MONAI 3 for medical analysis and transformers 4 for natural language pro-cessing.

Client selection and training plan in federated learning Training plan andstrategy for client selection for training rounds are crucial in federated learning.Research work in Nishio and Yonetani (2019) suggests optimal approaches, butstill there is a need to have a standardized approach for each machine learningalgorithm use-case in federated learning. OpenFed also provides a flexible wayfor users to choose clients.

Optimization techniques for different machine learning algorithms Basedon different machine learning algorithms, there is a need to have predefined andstandardized optimizing algorithms to build the federated learning model. Thereare many proposed aggregation/optimizing algorithms suggesting to optimize orenhance federated learning but still there is a need to have dedicated researchto provide federated learning specific optimizing algorithms for all the currentmachine learning applications/use-case. OpenFed provides most of them, andwill keep updating. This helps future implementers/adaptors of federated learningto develop federated learning specific solutions with ease. We provide most latestoptimizer methods in a uniform way.

Vision on training strategies and parameters Research work in Jiang andYing (2020) proposes an optimal strategy that helps the central server to set anoptimal trigger point to stop/restart the training rounds. Similar research workneeds to be done with respect to different models and domains of machine learningapplications which can help in understanding federated learning specific hyper-parameters and possible trigger conditions to configure in federated learning train-ing rounds. As federated learning training rounds are time, cost and computationalconsuming, having vision on setting optimal values will help in establishing robust

2https://github.com/open-mmlab/mmcv3https://github.com/Project-MONAI/MONAI4https://github.com/huggingface/transformers

12

and cost-efficient federated learning solutions. OpenFed will add these features infuture iterated updating.

Ease in migrating and productionising It is noticeable that there is no simpleand straight forward approach to productionise federated learning environment.Research work in Bonawitz et al. (2019) proposes many factors to be consideredwhile moving to production, but still, there is a need for well-established guide-lines for implementing a new use-case in federated learning or migrating an exist-ing machine learning environment to decentralized federated learning approach.You can deploy various algorithms with OpenFed to productionise environments.

Limitations. The success of federated learning models is largely dependent onhigh data quality on the nodes. The auditing and curation of the data and its qual-ity, methods to quantify the contribution of individual datasets to the model or todetect local over-fitting are still under investigation Kairouz et al. (2019). Fur-thermore, we point out that discussions of the theoretical threat model are a levelof abstraction that cannot fully represent the complexity of real-life situations.For instance, threats modelling is typically undertaken on the level of federatedlearning participants representing entire hospitals, however this cannot take everyindividual person working fore these hospitals and there specific motivations intoaccount. Further studies in this developing field are required to fully illuminatesuch details.

6 ConclusionWe present a free, open-source software framework for privacy-preserving feder-ated learning and end-to-end encrypted inference, which we showcase in severalrelevant real-life case study. Further research and development will enable thelarger-scale deployment of our framework, the validation of our findings on di-verse cross-institutional data and further the wide-spread utilization of OpenFed.

7 code availabilityThe current version of the OpenFed source code is publicly available at here5.

5https://github.com/FederalLab/OpenFed

13

ReferencesMohammed Aledhari, Rehma Razzak, Reza M Parizi, and Fahad Saeed. 2020.

Federated learning: A survey on enabling technologies, protocols, and applica-tions. IEEE Access 8 (2020), 140699–140725.

Alan Alexander, Adam Jiang, Cara Ferreira, and Delphine Zurkiya. 2020. Anintelligent future for medical imaging: a market outlook on artificial intelli-gence for medical imaging. Journal of the American College of Radiology 17,1 (2020), 165–170.

Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex In-german, Vladimir Ivanov, Chloe Kiddon, Jakub Konecny, Stefano Mazzocchi,H Brendan McMahan, et al. 2019. Towards federated learning at scale: Systemdesign. arXiv preprint arXiv:1902.01046 (2019).

Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Konecny,H Brendan McMahan, Virginia Smith, and Ameet Talwalkar. 2018. Leaf: Abenchmark for federated settings. arXiv preprint arXiv:1812.01097 (2018).

Shaoqi Chen, Dongyu Xue, Guohui Chuai, Qiang Yang, and Qi Liu. 2020. FL-QSAR: a federated learning-based QSAR prototype for collaborative drug dis-covery. Bioinformatics 36, 22-23 (2020), 5492–5498.

Ronald Doku, Danda B Rawat, and Chunmei Liu. 2019. Towards federated learn-ing approach to determine data relevance in big data. In 2019 IEEE 20th In-ternational Conference on Information Reuse and Integration for Data Science(IRI). IEEE, 184–192.

Yan Feng, Xue Yang, Weijun Fang, Shu-Tao Xia, and Xiaohu Tang. 2020. Practi-cal and bilateral privacy-preserving federated learning. (2020).

David Gunning, Awni Hannun, Mark Ibrahim, Brian Knott, Laurens van derMaaten, Vinicius Reis, Shubho Sengupta, Shobha Venkataraman, and XingZhou. 2019. CrypTen: A new research tool for secure machine learning withPyTorch. Blog post available at https://ai. facebook. com/blog/crypten-a-new-research-tool-for-secure-machine-learning-with-pytorch (2019).

Florian Hartmann, Sunah Suh, Arkadiusz Komarzewski, Tim D Smith, and IlanaSegall. 2019. Federated learning for ranking browser history suggestions. arXivpreprint arXiv:1911.11807 (2019).

14

Chaoyang He, Songze Li, Jinhyun So, Xiao Zeng, Mi Zhang, Hongyi Wang, Xi-aoyang Wang, Praneeth Vepakomma, Abhishek Singh, Hang Qiu, et al. 2020.Fedml: A research library and benchmark for federated machine learning. arXivpreprint arXiv:2007.13518 (2020).

Kevin Hsieh, Amar Phanishayee, Onur Mutlu, and Phillip Gibbons. 2020. Thenon-iid data quagmire of decentralized machine learning. In International Con-ference on Machine Learning. PMLR, 4387–4398.

A Ingerman and K Ostrowski. 2019. Introducing tensorflow federated. ExternalLinks: Link Cited by 4 (2019).

Pengfei Jiang and Lei Ying. 2020. An optimal stopping approach for iterativetraining in federated learning. In 2020 54th Annual Conference on InformationSciences and Systems (CISS). IEEE, 1–6.

Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurelien Bellet, MehdiBennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cor-mode, Rachel Cummings, et al. 2019. Advances and open problems in federatedlearning. arXiv preprint arXiv:1912.04977 (2019).

Hyesung Kim, Jihong Park, Mehdi Bennis, and Seong-Lyun Kim. 2019.Blockchained on-device federated learning. IEEE Communications Letters 24,6 (2019), 1279–1283.

David Leroy, Alice Coucke, Thibaut Lavril, Thibault Gisselbrecht, and JosephDureau. 2019. Federated learning for keyword spotting. In ICASSP 2019-2019IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP). IEEE, 6341–6345.

Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, andVirginia Smith. 2018. Federated optimization in heterogeneous networks. arXivpreprint arXiv:1812.06127 (2018).

Tian Li, Maziar Sanjabi, Ahmad Beirami, and Virginia Smith. 2019. FairResource Allocation in Federated Learning. In International Conference onLearning Representations.

Xiangru Lian, Ce Zhang, Huan Zhang, Cho-Jui Hsieh, Wei Zhang, and Ji Liu.2017. Can decentralized algorithms outperform centralized algorithms? a

15

case study for decentralized parallel stochastic gradient descent. arXiv preprintarXiv:1705.09056 (2017).

Sen Lin, Guang Yang, and Junshan Zhang. 2020. Real-time edge intelligence inthe making: A collaborative learning framework via federated meta-learning.arXiv preprint arXiv:2001.03229 (2020).

Yanjun Ma, Dianhai Yu, Tian Wu, and Haifeng Wang. 2019. PaddlePaddle: Anopen-source deep learning platform from industrial practice. Frontiers of Dataand Domputing 1, 1 (2019), 105–115.

Umer Majeed and Choong Seon Hong. 2019. FLchain: Federated learning viaMEC-enabled blockchain network. In 2019 20th Asia-Pacific Network Opera-tions and Management Symposium (APNOMS). IEEE, 1–4.

Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, andBlaise Aguera y Arcas. 2017. Communication-efficient learning of deep net-works from decentralized data. In Artificial intelligence and statistics. PMLR,1273–1282.

Viraaji Mothukuri, Reza M Parizi, Seyedamin Pouriyeh, Yan Huang, Ali De-hghantanha, and Gautam Srivastava. 2021. A survey on security and privacyof federated learning. Future Generation Computer Systems 115 (2021), 619–640.

Arvind Narayanan and Vitaly Shmatikov. 2008. Robust de-anonymization of largesparse datasets. In 2008 IEEE Symposium on Security and Privacy (sp 2008).IEEE, 111–125.

Takayuki Nishio and Ryo Yonetani. 2019. Client selection for federated learningwith heterogeneous resources in mobile edge. In ICC 2019-2019 IEEE Interna-tional Conference on Communications (ICC). IEEE, 1–7.

Florian Nuding and Rudolf Mayer. 2020. Poisoning attacks in federated learning:An evaluation on traffic sign classification. In Proceedings of the Tenth ACMConference on Data and Application Security and Privacy. 168–170.

Emmanuel Nyaletey, Reza M Parizi, Qi Zhang, and Kim-Kwang Raymond Choo.2019. BlockIPFS-blockchain-enabled interplanetary file system for foren-sic and trusted data traceability. In 2019 IEEE International Conference onBlockchain (Blockchain). IEEE, 18–25.

16

Shashi Raj Pandey, Nguyen H Tran, Mehdi Bennis, Yan Kyaw Tun, Aunas Man-zoor, and Choong Seon Hong. 2020. A crowdsourcing framework for on-device federated learning. IEEE Transactions on Wireless Communications 19,5 (2020), 3241–3256.

Jean Louis Raisaro, Francesco Marino, Juan Troncoso-Pastoriza, Raphaelle Beau-Lejdstrom, Riccardo Bellazzi, Robert Murphy, Elmer V Bernstam, HenryWang, Mauro Bucalo, Yong Chen, et al. 2020. SCOR: A secure internationalinformatics infrastructure to investigate COVID-19. Journal of the AmericanMedical Informatics Association 27, 11 (2020), 1721–1726.

Khaled Salah, M Habib Ur Rehman, Nishara Nizamuddin, and Ala Al-Fuqaha.2019. Blockchain for AI: Review and open research challenges. IEEE Access7 (2019), 10127–10149.

A Stoian, R Ivan, I Stoian, and A Marichescu. 2008. Current trends in medicalimaging acquisition and communication. In 2008 IEEE International Confer-ence on Automation, Quality and Testing, Robotics, Vol. 3. IEEE, 94–99.

Ahmet Ali SUZEN and Mehmet Ali SIMSEK. 2020. A novel approach to ma-chine learning application to protection privacy data in healthcare: Federatedlearning. Namık Kemal Tıp Dergisi 8, 1 (2020), 22–30.

Stacey Truex, Nathalie Baracaldo, Ali Anwar, Thomas Steinke, Heiko Ludwig,Rui Zhang, and Yi Zhou. 2019. A hybrid approach to privacy-preserving fed-erated learning. In Proceedings of the 12th ACM Workshop on Artificial Intelli-gence and Security. 1–11.

Gregor Ulm, Emil Gustavsson, and Mats Jirstrand. 2018. Functional federatedlearning in erlang (ffl-erl). In International Workshop on Functional and Con-straint Logic Programming. Springer, 162–178.

Akhil Vaid, Suraj K Jaladanki, Jie Xu, Shelly Teng, Arvind Kumar, Samuel Lee,Sulaiman Somani, Ishan Paranjpe, Jessica K De Freitas, Tingyi Wanyan, et al.2021. Federated learning of electronic health records to improve mortality pre-diction in hospitalized patients with COVID-19: Machine learning approach.JMIR medical informatics 9, 1 (2021), e24207.

Xiaofei Wang, Chenyang Wang, Xiuhua Li, Victor CM Leung, and Tarik Taleb.2020. Federated deep reinforcement learning for Internet of Things with de-

17

centralized cooperative edge caching. IEEE Internet of Things Journal 7, 10(2020), 9441–9455.

Zhaoping Xiong, Ziqiang Cheng, Xiaohong Liu, Dingyan Wang, Xiaomin Luo,Kaixian Chen, Hualiang Jiang, and Mingyue Zheng. 2020. Facing small andbiased data dilemma in drug discovery with federated learning. BioRxiv (2020).

Qiang Yang, Yang Liu, Yong Cheng, Yan Kang, Tianjian Chen, and Han Yu. 2019.Federated learning. Synthesis Lectures on Artificial Intelligence and MachineLearning 13, 3 (2019), 1–207.

Timothy Yang, Galen Andrew, Hubert Eichner, Haicheng Sun, Wei Li, NicholasKong, Daniel Ramage, and Francoise Beaufays. 2018. Applied federatedlearning: Improving google keyboard query suggestions. arXiv preprintarXiv:1812.02903 (2018).

Abbas Yazdinejad, Reza M Parizi, Ali Dehghantanha, and Kim-Kwang RaymondChoo. 2019. Blockchain-enabled authentication handover with efficient privacyprotection in SDN-based 5G networks. IEEE Transactions on Network Scienceand Engineering (2019).

Abbas Yazdinejad, Reza M Parizi, Ali Dehghantanha, and Kim-Kwang RaymondChoo. 2020. P4-to-blockchain: A secure blockchain-enabled packet parser forsoftware defined networking. Computers & Security 88 (2020), 101629.

Hanene Ben Yedder, Ben Cardoen, and Ghassan Hamarneh. 2021. Deep learningfor biomedical image reconstruction: A survey. Artificial Intelligence Review54, 1 (2021), 215–251.

Han Yu, Zelei Liu, Yang Liu, Tianjian Chen, Mingshu Cong, Xi Weng, DusitNiyato, and Qiang Yang. 2020. A fairness-aware incentive scheme for feder-ated learning. In Proceedings of the AAAI/ACM Conference on AI, Ethics, andSociety. 393–399.

Alexander Ziller, Andrew Trask, Antonio Lopardo, Benjamin Szymkow,Bobby Wagner, Emma Bluemke, Jean-Mickael Nounahon, Jonathan Passerat-Palmbach, Kritika Prakash, Nick Rose, et al. 2021. PySyft: A Library for EasyFederated Learning. In Federated Learning Systems. Springer, 111–139.

18