High Performance Computing for Artificial

Intelligence Applied to Finance

Muhammad Adil RajaResearch ProposalNovember 23, 2012

1

Abstract

Financial sector poses some challenging problems that need to besolved by computer scientists. This is primarily because of the in-fluence of some intricate factors. These include political events, eco-nomic conditions and traders’ expectations etc. This document aimsat proposing some thoughts and ideas that can be used to solve areal world problem in finance. The ideas that follow are written inthe light of existing literature and mainly ensue from the domain ofapplied machine learning.

1 Introduction

A notable application domain of machine learning has been to apply var-ious algorithms to solve problems in the area of finance. For instance,numerous attempts have been made in the past to develop classifier sys-tems for stock market forecasting [Grosan et al., 2005]. Artificial neural net-works (ANNs) have by far been the most widely used. Various types ofneural networks that have been used include Time Delay neural networks[W. Kreesuradej and Lane, 1994], probabilistic neural networks [Tan et al., 1995],recurrent neural networks [E. Saad and Wunsch, 1996]. Apart from neuralnetworks, Bayesian Belief networks [Wolfe, 1998], evolutionary algorithms[Kanoudan, 2000, Kim, 2000, Allen and Karjalainen, 1999, Kaboudan, 2000,Wang, 2000], classifier systems [Schulenburg and Ross, 2001] and fuzzy sets[Castillo and Melin, 2001] have also been employed.

There has also been an interest in hybridizing various machine learningalgorithms to find solutions to various financial problems. These include hy-brids of ANNs and genetic algorithms (GAs) [Kim and Shin, 2007], a hybridof a rule-based technique and an ANN [Tsaih et al., 1998], a genetic algo-rithm and support vector machines (SVM) [Choudhry and Garg, 2008]. Thework reported in [Choudhry and Garg, 2008] is of particular interest wherethey have used a GA to select optimal set of features from the input spaceto be supplied to a SVM as inputs. The GA has been used to serve as adimensionality reduction algorithm and SVM has been used for optimiza-tion. The goal was to develop a two-class classifier to be used for predictionof stock market direction. Similarly, [Charles W. Richter et al., 1999] usedgenetic programming (GP) and finite state automata to compute biddingstrategies for competitive auction markets.

2

Given the high computational requirements posed by algorithms like GP,or possibly hyper-heuristics, one may also leverage from parallel processingsystems proposed by various research studies. In [Dracopoulos and Kent, 1996]Kent and Dracopoulos have proposed a parallel implementation of GP us-ing bulk synchronous parallel programming (BSP). They propose to use amaster-slave architecture in which the master node distributes subpopula-tions of individuals to multiple slave nodes for fitness evaluation, which is nor-mally the most compute intensive process in GP. Similarly, in [Fernandez et al., 2000]Fernandez et al. have proposed a distributed computing environment for GPusing MPI in which each of a number of slave nodes is assigned a GP popu-lation and to perform genetic evolution over them, whereas a master node isutilized to perform migrations within the populations held by the slave nodes.Their system also allows to easily model various communication topologiessuch as the mesh and ring topologies. Their results show that partitioningsubpopulations on to multiple slave nodes results in faster convergence tothe optimal solution. However, they also argue that further partitioning thesubpopulations on to multiple nodes is not beneficial beyond a certain limit.[Oussaidene et al., 1997] have also proposed another parallel implementationof GP particularly for a trading model induction problem. In their schemethey present a scalable parallel implementation of GP on distributed memorymachines. The system runs multiple master-slave instances each mapped onall the allocated nodes. Asynchronous migration of individuals is performedamong processes to avoid premature convergence to a locally optimal solu-tion. Considerable speedups have been claimed for problems of large enoughsizes.

One may also benefit from work proposed by Keijzer to speed up GP on asingle CPU. For instance, in [Keijzer, 2004a] has proposed a number of sub-tree caching mechanisms that are capable of adapting during the course of aGP run while maintaining a fixed size cache of already evaluated subtrees.This is known to speed up a GP system as the whole trees do not need to beevaluated. Where ever a subtree is found in the cache its evaluation is usedinstead of re-evaluating it. Keijzer has also proposed to use vectorized eval-uation of GP trees as opposed to standard case-by-case evaluation methodfor interpreted GP.

Apart from other methods one may also benefit from hybridizing a GPsystem with linear scaling as proposed by Keijzer [Keijzer, 2004b]. In thisscheme Keijzer proposes to further optimize a GP tree to the target datain a least squared sense. He proposes to find a slope and an intercept value

3

that may be attached to a GP tree. The benefits of such a scheme arenumerous and are cited elsewhere in the literature. One of the major benefitsis that it forms a good hybrid algorithm with only a small computationaloverhead. It was found beneficial over hybrids of other techniques by Rajaet al. [Raja and Flanagan, 2008]. Another benefit that can be foreseen is toperform online optimization of a given GP tree in real-time. To this end, itis thought that once a befitting GP tree has been found for any hypotheticalclassification or regression problem along with its slope and intercept usinglinear scaling. One may iteratively re-compute the slope and the interceptterms as the new data arrives. This may be done as a function of past Ninput/output tuples, where N may be an arbitrarily small or large number.

Apart from the above stated mainly meta-heuristic algorithms, numer-ous numerical algorithms like Gauss-Newton or Levenberg-Marquardt canbe used for nonlinear optimization. Nonetheless, meta-heuristic algorithmshave well known advantages over the numerical algorithms. These include,but are not limited to, being able to find globally optimal solutions as op-posed to locally optimal ones, their ease of understanding and use, and theirapplicability to a wider range of problems. One may also wish to develophybrid algorithms using one or more of such numerical algorithms with GP,as in [Topchy and Punch, 2001]. In such a scheme, GP tries to find the opti-mum program, from the program space, for the underlying problem, whereas,the numerical optimization algorithm attempts at fine-tuning the coefficientsof any given program. One may also benefit from more sophisticated ap-proaches lent by coevolutionary algorithms [Ficici, 2004], evolutionary gametheory [Vincent and Brown, 2005] and from ideas in agent based computa-tional economics [Tesfatsion and Judd, 2006].

2 Evolutionary Algorithms for Finding Tech-

nical Trading Rules

This research proposal is primarily aimed at writing technical trading ruleswhich forms a considerable problem domain in computational finance. Aconsiderable point of concern in this domain is to choose a suitable set of al-gorithms for solving the problem. In the light of literature review, as statedabove,it is anticipated that GP (or one of its variants) would be a suit-able choice for solving the problem. Another problem concerns the choice

4

of programming language to be used in solving the problem. GP has beenimplemented in a number of programming and scripting languages includingC, C++, Java, Matlab, Perl and Python etc. The choice of programminglanguage may be driven by many factors such as computational efficiencyor ease of use. For instance, if higher computational efficiency is sought thepractitioner may seek a C or a Java based implementation. Similarly, thepractitioner may opt for a Matlab based implementation if ease of use issought. The choice of programming language may also be driven by otherfactors such as the need for implementing a system for parallel and dis-tributed computing, nature of the underlying problem and of the problemdomain. As a personal experience it has been found that using a Matlabbased implementation is normally as good as a Java based implementationfor smaller problems as far as the quality of solution is concerned. However,a nicely implemented Java based implementation is much faster and handleslarge populations nicely.

It is felt that the solicitors of this proposal have a particular bent towardsthe TradeML programming language due to the nature of the project. It isdocumented that the trade rules that the R&D facility (at EXQIM) receivesare written in TradeML. Exact details of the nature of the programminglanguage are not known as a literature about it is not available on the worldwide web. However, TradeML is a variant of the OCaml 1 programming lan-guage. Unlike the latter, the former, however, does not allow type inference.OCaml has numerous advantages which render it feasible for evolutionarycomputing. Firstly there are a couple of already existing implementation ofGP in OCaml. Secondly it is amenable for parallel and distributed program-ming 2. It is also supposedly easy to learn and has lucid documentation fromthat point of view. It is not known if such TradeML would allow such ex-tensions. However, it is anticipated that it should be possible to implementGP in TradeML by taking inspiration from an already existing model imple-mentation such as Beagle Puppy 3. It might be required to see if TradeMLsupports parallel and distributed computing.

Considerable concern has to be given to what the building blocks of theunderlying evolutionary algorithm for trade rule induction would be. If GPis to be applied (which possibly would be the case), then these would include

1http://caml.inria.fr/cgi-bin/hump.en.cgi?sort=0&browse=772http://caml.inria.fr/cgi-bin/hump.en.cgi?sort=0&browse=773http://beagle.gel.ulaval.ca/puppy/

5

sets of functions and terminals that could be used in developing computerprograms corresponding to technical trading rules. Inspiration for this hasbeen taken from [Allen and Karjalainen, 1999]. These may include real val-ued functions such as average, to compute a moving average of the pastprices in a time window, max and min to compute local maxima and minimaof some metric (possibly of prices) during a time window of a certain length.Other functions may include simple arithmatic functions such as {+,−, /, ∗}.A certain number of boolean functions such as {IF, ELSE,≥, <,>,≤} mayalso be required as it is speculated that the outcome of a feasible tradingrule would be binary. The terminal set may include randomly generated realand boolean constants. The terminal set may also include a variable namedprice as the closing price of the current day and a variable named lag thatdepicts a length of a price series to be lagged by a certain number of days.The system may require other such variables to be figured out at the timeof problem formulation. Genetic operators such as crossover and mutationshall also have to be designed so that they do not violate the syntactical con-straints of the generated trading programs. Specially, since TradeML doesnot allow type inference, appropriate rules for crossover and mutation oper-ations may have to be implemented. Crossover and mutation are normallyexpected to respect the closure property which requires that all componentsand representational constructs be handled uniformly, thereby allowing ar-bitrary subtree expressions to be recombined. Unrestricted operations canresult in solutions that are invalid or difficult to interpret.

Trading models are learnt with the general objective of maximizing oroptimizing certain user specified metrics, such as profits or returns. The pro-cess of specifying the fitness function has to be dealt with carefully becausedifferent fitness functions can lead to different search trajectories and con-sequently different models eventually. The fitness function should thus takeinto account performance objectives of the desired trading rules. Normallythe desired performance criterion is to maximize the profits from the use ofa trading rule. The performance of a trading model is determined by eval-uating it on time-series data, depicting the price history of a certain assetunder consideration, over a certain period. A gist of possible fitness functionsfor trade model induction are stated here, possibly with brief descriptions ofsome of them.

A normally used fitness criterion is to measure compounded returns. Inthis the merit of undertaken decisions is measured by the total realized returnfor a series ofN trades over a time period T . Another fitness measure is based

6

on realized vs non-realized returns. Its maximization entails the maximizingthe realized returns and minimizing the number of trades. Another fitnessfunction may be based on transaction costs and risk-free rate. A transactioncost is incurred only when a trade is conducted. It serves as a check againstrules that change positions very frequently leading to over-fitness to smallfluctuations in market data. Other commonly used fitness functions in the lit-erature are based on excess returns and prediction error. A detailed descrip-tion of various fitness functions is given in [Bhattacharyya and Mehta, 2002]and the references there in.

Trading models for financial time series are normally of the form; IFcondition THEN action, where condition is normally from a pattern of fi-nancial time series data and action is normally a binary choice prescribingwhether to buy (out-of-market) or sell (in-market) some asset. Patterns inthe condition normally depict some aspects of the financial time series datain a meaningful sense. Moving average and volatility of the price series overdifferent time ranges are commonly used indicators. Pseudo code of a hypo-thetical technical trading rule is given in Algorithm 1 possibly according tothe TradeML syntax. Inspiration for this has been taken from Figure 8 in[Allen and Karjalainen, 1999].

Algorithm 1 Pseudo Code for a Typical Technical Trading Rule

let Rule : float ⇒ bool = fun (price) ⇒ if average(price * price / 2 +1.1232) ≥ then 1 else 0.

3 Methodology

The specific methodology that would need to be adopted in this research isnot anticipated as it would depend on the problem statement. However, itmay be anticipated that it would have the components of a typical researchproject. Few thoughts are as follows:

Initially, depending on the problem statement, a literature review wouldbe required. This would primarily be of the problem domain (i.e. the realfinancial problem at hand), and also of the computer scientific methods thathave been or may be used to solve the problem. A critical analysis of thebenefits of one method over the other would be important. This may followwith a data collection phase in which problem critical data may have to

7

be created. It may also need to be ascertained as to whether online oroffline data needs to be dealt with. This would need to be addressed by theresearch supervisor. This would follow with an experimentation phase thatmay require and design and execution of simulations. Specially if computeintensive algorithms like GP are used, it may also need to be decided whetherresources for parallel processing are required. If so, issues concerning designand deployment of software on multiple machines may also be required.

The experimentation phase may end up with writing up of the resultsand findings. The expectation here would be that the work would end up indecent scholarly publications.

4 Timeline

Given that two to three years are normally allotted for a research project,following set of activities can be foreseen:

1. Literature review – 6 months.

2. Design and evaluation of any software that may be required – 3 months.

3. Experimentation and writing (of any papers/journal articles/patents)– 6 to 12 months.

References

[Allen and Karjalainen, 1999] Allen, F. and Karjalainen, R. (1999). Usinggenetic algorithms to find technical trading rules. Journal of FinancialEconomics, 51(2):245–271.

[Bhattacharyya and Mehta, 2002] Bhattacharyya, S. and Mehta, K. (2002).Evolutionary induction of trading models. In Chen, S.-H., editor, Evolu-tionary Computation in Economics and Finance, volume 100 of Studies inFuzziness and Soft Computing, chapter 17, pages 311–332. Physica Verlag.

[Castillo and Melin, 2001] Castillo, O. and Melin, P. (2001). Simulation andforecasting complex financial time series using neural networks and fuzzylogic. In Proceedings of IEEE Conference on Systems, Man, and Cyber-netics, pages 2664–2669.

8

[Charles W. Richter et al., 1999] Charles W. Richter, J., Shebli, G. B., andAshlock, D. (1999). Comprehensive bidding strategies with genetic pro-gramming/finite state automata. IEEE Transactions on Power Systems.

[Choudhry and Garg, 2008] Choudhry, R. and Garg, K. (2008). A hybridmachine learning system for stock market forecasting. In World Academyof Science, Engineering and Technology 39 2008, number 39, pages 315–318.

[Dracopoulos and Kent, 1996] Dracopoulos, D. C. and Kent, S. (1996).Speeding up genetic programming: A parallel BSP implementation. InKoza, J. R., Goldberg, D. E., Fogel, D. B., and Riolo, R. L., editors,Genetic Programming 1996: Proceedings of the First Annual Conference,page 421, Stanford University, CA, USA. MIT Press.

[E. Saad and Wunsch, 1996] E. Saad, D. P. and Wunsch, D. (1996). Ad-vanced neural-network training methods for low false alarm stock trendprediction. In Proceedings of IEEE Int. Conf. on Neural Networks, Wash-ington, DC.

[Fernandez et al., 2000] Fernandez, F., Tomassini, M., Vanneschi, L., andBucher, L. (2000). A distributed computing environment for genetic pro-gramming using MPI. In Dongarra, J. J., Kacsuk, P., and Podhorszki, N.,editors, Recent advances in parallel virtual machine and message passinginterface: 7th European PVM/MPI Users’ Group Meeting, volume 1908of Lecture Notes in Computer Science, pages 322–329, Balatonfured, Hun-gary. Springer-Verlag.

[Ficici, 2004] Ficici, S. G. (2004). Solution Concepts in Coevolutionary Algo-rithms. PhD thesis, Computer Science Department, Brandeis University,USA.

[Grosan et al., 2005] Grosan, C., Abraham, A., Ramos, V., and Han, S. Y.(2005). Stock market prediction using multi expression programming. InBento, C., Cardoso, A., and Dias, G., editors, ALEA-05, Workshop onArtificial Life and Evolutionary Algorithms at EPIA’05 - Proc. of the 12thPortuguese Conference on Artificial Intelligence, pages 73–78, Covilha,Portugal.

9

[Kaboudan, 2000] Kaboudan, M. A. (2000). Genetic programmingprediction of stock prices. Computational Economics, 16:207–236.10.1023/A:1008768404046.

[Kanoudan, 2000] Kanoudan, M. A. (2000). Genetic programming predictionof stock prices. In Computational Economics, number 16, pages 207–236.

[Keijzer, 2004a] Keijzer, M. (2004a). Alternatives in subtree caching for ge-netic programming. In Keijzer, M., O’Reilly, U.-M., Lucas, S. M., Costa,E., and Soule, T., editors, Genetic Programming 7th European Confer-ence, EuroGP 2004, Proceedings, volume 3003 of LNCS, pages 328–337,Coimbra, Portugal. Springer-Verlag.

[Keijzer, 2004b] Keijzer, M. (2004b). Scaled symbolic regression. GeneticProgramming and Evolvable Machines, 5(3):259–269.

[Kim and Shin, 2007] Kim, H.-j. and Shin, K.-s. (2007). A hybrid approachbased on neural networks and genetic algorithms for detecting temporalpatterns in stock markets. Appl. Soft Comput., 7:569–576.

[Kim, 2000] Kim, K. J. (2000). Genetic algorithms approach to feature dis-cretization in artificial neural networks for the prediction of stock priceindex. In Expert Systems with Applications, number 19(2), pages 125–132.

[Oussaidene et al., 1997] Oussaidene, M., Chopard, B., Pictet, O. V., andTomassini, M. (1997). Parallel genetic programming and its applicationto trading model induction. Parallel Computing, 23(8):1183–1198.

[Raja and Flanagan, 2008] Raja, A. and Flanagan, C. (2008). Real-time,non-intrusive speech quality estimation: A signal-based model. In O’Neill,M., Vanneschi, L., Gustafson, S., Esparcia Alcazar, A. I., De Falco, I.,Della Cioppa, A., and Tarantino, E., editors, Proceedings of the 11th Eu-ropean Conference on Genetic Programming, EuroGP 2008, volume 4971of Lecture Notes in Computer Science, pages 37–48, Naples. Springer.

[Schulenburg and Ross, 2001] Schulenburg, S. and Ross, P. (2001). Explo-rations in lcs models of stock trading. In Advances in Learning ClassifierSystems, pages 151–180.

10

[Tan et al., 1995] Tan, H., Prokhorov, D., and Wunsch, D. (1995). Proba-bilistic and time-delay neural-network techniques for conservative short-term stock trend prediction. In In World Congress on Neural Networks,Washington, DC.

[Tesfatsion and Judd, 2006] Tesfatsion, L. and Judd, K. L. (2006). Hand-book of Computational Economics, Volume 2: Agent-Based ComputationalEconomics (Handbook of Computational Economics). North-Holland Pub-lishing Co., Amsterdam, The Netherlands, The Netherlands.

[Topchy and Punch, 2001] Topchy, A. and Punch, W. F. (2001). Faster ge-netic programming based on local gradient search of numeric leaf values.In Spector, L., Goodman, E. D., Wu, A., Langdon, W. B., Voigt, H.-M.,Gen, M., Sen, S., Dorigo, M., Pezeshk, S., Garzon, M. H., and Burke, E.,editors, Proceedings of the Genetic and Evolutionary Computation Con-ference (GECCO-2001), pages 155–162, San Francisco, California, USA.Morgan Kaufmann.

[Tsaih et al., 1998] Tsaih, R., Hsu, Y., and Lai, C. C. (1998). Forecastings&p 500 stock index futures with a hybrid ai system. Decis. Support Syst.,23:161–174.

[Vincent and Brown, 2005] Vincent, T. L. and Brown, J. S. (2005). Evolu-tionary Game Theory, Natural Selection, and Darwinian Dynamics. Cam-bridge University Press.

[W. Kreesuradej and Lane, 1994] W. Kreesuradej, D. W. and Lane, M.(1994). Time-delay neural network for small time series data sets. InIn World Congress on Neural Networks, San Diego, California.

[Wang, 2000] Wang, J. (2000). Trading and hedging in s&p 500 spot andfutures markets using genetic programming. Journal of Futures Markets,20(10):911–942.

[Wolfe, 1998] Wolfe, R. K. (1998). Turning point identification and bayesianforecasting of a volatile time series. In Computers and Industrial Engi-neering, pages 378–386.

11