AI State of Play Dec15 NYC

AI- StateofPlay

1IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016

Outline

• Concepts• AIMarket• Data• Software• Hardware• Conferences• Applications

IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 2©IvyDataScience


TypesofIntelligence


© Ivy Data Science

TheIntelligenceRevolution

5

DeepLearning/AIFrameworks- TheBigPicture

IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 6

AIFrameworksCognitive

Architectures

MLFrameworksSupervised,

Unsupervised&Reinforcement

DeepLearningFrameworks

NeuralNetworks

©IvyDataScience

WhatisIntelligence?• IntelligenceisanAgent’s abilitytoadapttoandtoachievegoalswithinitsEnvironment• Humanvsmachineintelligence– ultimatelythesame• “Thetermartificialintelligenceissomewhatnonsensical.Somethingiseitherintelligentoritisn’t.Justassomethingeitherfliesoritdoesn’t.Wedon’ttalkaboutartificialflying”- Zoubin Ghahramani,CambridgeUniversity• Informationprocessing, computation,physics,hardware• Explorationvsexploitation• Biological(anyspecies)versusmachine(anytype)•Doessubstratematter- carbonvssilicon?

7© Ivy Data Science

WhatisLearning?

• Learningalgorithms– systemgetsbetterwithmoredatauntilnofurtherimprovement• Trainthesystem– justlikeanimalslearn• Supervised,unsupervisedandreinforcementlearning• Physically,itisthestrengtheningofconnections(synapses)betweennodes(neurons)• Memory(shortandlongterm)isinvolved• Deeplearningisasteptowardsthegoalofartificialgeneralintelligence(AGI)• Ensembleoftechniques


© Ivy Data Science

WhatisDeepLearning?

9

DeepLearning=NeuralNetworks

• Referstosystemsthatlearnfromdata• Thesesystemsarebasedonartificialneuralnetworks(ANNs),whichinturnarebasedonbiologicalneuralnetworks(BNN),suchasthehumanbrain

• Inpracticesuchlearningsystemsconsistofdata,multiplelayers,nodes,weightsandoptimisationalgorithms


BiologicalNeuron


DeepLearningIsEatingtheWorld• Whataboutthe“AIwinters”?1974–80and1987–93,whereAIcompaniesover-promisedandunder-deliveredhttps://en.wikipedia.org/wiki/AI_winter

• Duetomorelabeled data,morecomputepower,betteroptimizationalgorithms,andbetterneuralnetmodelsandarchitectures,deeplearninghasstartedtosupersedehumanswhenitcomestoimagerecognitionandclassification

• Workisbeingdonetoobtainsimilarlevelsofperformanceinnaturallanguageprocessing andunderstanding

• AccordingtoJeffDeaninarecentinterview,GooglehaveimplementedDLinoveronehundredoftheirproductsandservicesincludingsearchandphotos

• AIisenjoyingarenaissancenow,notsimplybecauseofthepromiseitholdsforthefuturebutbecauseoftheimpactitishavingonbusinessestoday

• Timingiseverything,andthetimeisnow!


AIMarket




• Hardware(compute)– Nvidia GPU,Intel(Nervana),AMDRadeon• Dataavailable - structuredandunstructured• Researchactivity• Conferenceattendance(e.g.,NIPS)• Meetupgroups• PhDenrolmentsinCSandmachinelearning• Performancemeasures- chess,Jeopardy,Go,computervision,languageprocessing,...• NumberofpapersbeingpublishedinAI/ML

AITrends1

©IvyDataScience


• Availabilityofopensourcedeeplearningframeworks,includingTensorFlow,Mxnet,etc.- Numberofpackages- GitHubcommits- Contributorsetc.

• Forexample,TF ismostdownloadedrepofromGitHubinunderayear• FactthatmajorcorporatesopensourcedtheirAIframeworks,startingwithGoogle(TF,etc.)• NumberofAIrelatedjobsonjobboards• SalariesforAIexperts

AITrends2

©IvyDataScience


• BackingandrealignmentbycorporatestorebrandasAIcompanies-Microsoft,IBM,Amazon(GoogleandFacebookwerealreadythere)• Forexample:- IBMWatsonHQinNYC- Microsoftannouncing5000strongAIdivision- AppleannouncingatNIPSthatitwouldbeopensourcingitsAIresearch- Siri, Cortana,AlexaareallNLPappsusingneuralnets

• NumberofAIproductsandapps• NumberofAI/deeplearningstartups

AITrends3

©IvyDataScience


• VenturecapitalinvestmentinAIstartups• Numberofpress/newsarticles• AnnouncementsfromAIexpertswith30yearsexperienceconfirmingthatthistimeisforreal- therewillbenomoreAIwinters• NumberofprofessorsbeinghiredwayfromacademiatojoinAIcompanies,e.g.UberandCMU,GoogleandOxford,Facebook,Apple,etc.• GovernmentlevelpanelsonthedevelopmentandimpactofAIonjobs,societyandpolicy,e.g.,WhitehouseandU.K.parliament• AISafetyconsortiumannouncedlastmonthbetweenGoogle,Microsoft,IBMandAmazontotrackdevelopmentsinAI• RecentbookspublishedbyprofessorsandengineersonAIdevelopment

AITrends4

©IvyDataScience

AIRelatedBooks• Barrat,James,OurFinalInvention,St.Martin'sGriffin,2014• Bengio,Yoshua etal,DeepLearning,MITPress,2016• Brynjolfsson,ErikandAndrewMcAfee,TheSecondMachineAge,W.W.Norton&Co.,2014• Domingos,Pedro,TheMasterAlgorithm,BasicBooks,2015• Ford,Martin,RiseoftheRobots:TechnologyandtheThreatofaJoblessFuture,BasicBooks,2015• Kaku,Michio,TheFutureoftheMind, Doubleday,2014• Kurzweil,Ray,TheSingularityisNear, PenguinBooks,2006• Kurzweil,Ray,HowtoCreateaMind, PenguinBooks,2013• RussellandNorvig,ArtificialIntelligence,AModernApproach,Pearson,2009• Yampolskiy,Roman- ArtificialSuperintelligence,AFuturisticApproach,CRC,2015

©IvyDataScience 19


Imageclassification

Progressinmachineclassificationofimages- errorratebyyear.Redlineistheerrorrateofatrainedhuman.Under3%asofNov2016.

21IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016©IvyDataScience

DLOutperformsML


ComputerVisionAccuracy

IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 23© Ivy Data Science 23

GPUFasterthanMoore’sLaw




AsofMay2016,Dec>38,000



AIaaS - Industry Partnerships GPU Deep Learning in the cloud is accelerating enterprise AI

Data


Wheredoesthedatacomefrom?• Science – particle,astrophysics• Industry – oil,finance,telecom(allverticals)• Social – Facebook,LinkedIn,Twitter• Medicine – genome,neuroscience• Government – census,education,police• Sports– statistics• Environment – weather,sensors

31©IvyDataScience

DataSets• Rawdatainputintotheneuralnetworkcanoriginatefromanyenvironmentalsource• Itcanberecordedandstoredinadatabase(e.g.,text,images,audio,video),orlive(incidentdirectlyfromtheenvironment)streamingdata

• ExamplesofrecordeddatasetsincludeMNIST,Labeled FacesintheWild(LFW),ImageNet,CIFARandYouTube-8M


MNIST LFW

©IvyDataScience

Algorithms


DeepLearningEvolution


ConvolutionalNeuralNetworks• Firstdevelopedin1970’s• Widelyusedforimagerecognitionandclassification• Inspiredbybiologicalprocesses,CNN’sareatypeoffeed-forwardANN• Theindividualneuronsaretiledinsuchawaythattheyrespondtooverlappingregionsinthevisualfield


RecurrentNeuralNetworks• Firstdevelopedin1970’s• RNN’sareneuralnetworksthatareusedtopredictthenextelementinasequenceortimeseries

• Thiscouldbe,forexample,wordsinasentenceorlettersinaword• Applicationsincludepredictingorgeneratingmusic,stories,news,code,financialinstrumentpricing,text,speech,infactthenextelementinanyeventstream


LSTMandNTM

• LongShortTermMemory(LSTM)• LSTM(Schmidhuber,1997)isanRNNarchitecturethatcontainsblocksthatcanrememberavalueforanarbitrarylengthoftime• Itsolvesthevanishingorexplodinggradientproblemwhencalculatingbackpropagation• AnLSTMnetworkisuniversal inthesensethatgivenenoughnetworkunitsitcancomputeanythingaconventionalcomputercancompute,providedithastheproperweightmatrix• LSTMoutperforms alternativeRNNsandHiddenMarkovModelsandothersequencelearningmethodsinnumerousapplications,e.g.,inhandwritingrecognition,speechrecognitionandmusiccomposition• NeuralTuringMachines(NTM)• NTMsareamethodofextendingthecapabilitiesofrecurrentneuralnetworksbycouplingthemtoexternalmemoryresources


DeepLearningFrameworks

38

Framework = Toolkit = Package = Library

© Ivy Data Science

TensorFlow• TensorFlowisthenewly(Nov2015)opensourceddeeplearninglibraryfromGoogle

• Itistheirsecondgenerationsystemfortheimplementationanddeploymentoflarge-scalemachinelearningmodels

• WritteninC++withapythoninterface,itisbornefromresearchanddeployingmachinelearningprojectsthroughoutawiderangeofGoogleproductsandservices

• InitiallyTFranonlyonasinglenode(yourlaptop,say),butGooglehavenowreleasedaversionthatrunsonadistributedcluster

• AvailableinthecloudonGCP• https://www.tensorflow.org/


CognitiveToolkit• Microsoft opensourcedeeplearningframework(Jan25,2016)• Version2.0releasedOct25,majorupgrade• RenamedCNTKtoMicrosoftCognitiveToolkit• AnnouncedpartnershipwithNvidia andOpenAI(ElonMuskbackedAIstartup),Nov16• LanguagesarePython,C++orBrainScript• CanrunonAzureGPU’s• https://www.microsoft.com/en-us/research/product/cognitive-toolkit/


Torch• Firstreleasedin2000,withover50,000downloads,companyusersincludeGoogle,Facebook,Twitter• ThegoalofTorchistohavemaximumflexibilityandspeedinbuildingscientificalgorithmswhilemakingtheprocessextremelysimple• TorchisaneuralnetworklibrarywritteninLuawithaC/CUDAinterfaceoriginallydevelopedbyateamfromtheSwissinstituteEPFL• AttheheartofTorcharepopularneuralnetworkandoptimizationlibrarieswhicharesimpletouse,whilebeingflexibleinimplementingdifferentcomplexneuralnetworktopologies• http://torch.ch/


AIHardware

IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 42 42© Ivy Data Science

TypesofHardware

• Sensors,processors,storage,memory,network• Processors- CPU,GPU,FPGA,ASIC• GPU - GraphicsProcessingUnitswerefirstbroughttomarketbyNvidia in2007tomeetthedemandsofthegamingmarket• Massivelyparallelprocessing(MPP)• 100xspeedupcomparedwithCPU’s• Widespreadapplication– science,industry,government• Nvidia www.nvidia.com• Intel XeonPhihttp://www.intel.com/content/www/us/en/processors/xeon/xeon-phi-detail.html• AMD Radeonwww.amd.com


CPUvGPUArchitecture

IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 44 44© Ivy Data Science

Nvidia GPUExponentials


DeepLearningFullstack(Example)


DLaaS - CloudServices

• DL/MLasaServiceisofferedbythemajorcloudproviders• AWS(Amazon) http://aws.amazon.com/• Azure(Microsoft) http://azure.microsoft.com/• GCP(Google) https://cloud.google.com/• Bluemix (IBM) https://www.ibm.com/cloud-computing/bluemix/

© Ivy Data Science 47

AIConferences– BusinessFocussed

• O’ReillyAIhttp://conferences.oreilly.com/artificial-intelligence/ai-ny• AIFrontiershttp://www.aifrontiers.com/• AIWorldhttp://aiworldexpo.com/program/• AIEuropehttp://ai-europe.com/• AISummithttps://theaisummit.com/london/• Re:Work https://www.re-work.co/events/• WorldofWatsonhttps://www-01.ibm.com/software/events/wow/


AIConferences– ResearchFocussed• NIPS=NeuralInformationProcessingSystemshttps://nips.cc/• IJCNN=InternationalJointConferenceonNeuralNetworkshttp://www.ijcnn.org/• IJCAI=InternationalJointConferenceonArtificialIntelligencehttp://ijcai.org/• ICANN=InternationalConferenceonArtificialNeuralNetworkshttp://www.icann2017.org/• IWANN=InternationalWork-ConferenceonArtificialNeuralNetworkshttp://iwann.uma.es/• ICONIP=InternationalConferenceonNeuralInformationProcessinghttp://www.iconip2017.org/papers.html• ICAART=InternationalConferenceonAgentsandArtificialIntelligencehttp://www.icaart.org/• ISIS=InternationalSymposiumonAdvancedIntelligentSystemshttp://isis2017.org/• AAAI=AssociationofAdvancementofArtificialIntelligencehttp://www.aaai.org/Conferences/conferences.php• ACM=AssociationofComputingMachineryhttps://www.acm.org/conferences• AGI=ArtificialGeneralIntelligenceConferencehttp://agi-conf.org/• TensorCon =TensorFlowConferencehttps://ti.to/TensorCon/


AIApplications


©IvyDataScience 51

AIinHealthcare- Market

• $600min2014,expectedtoreach$6bnby2021- Frost&Sullivan• Movedfrompilotsandproofofconcepts,tocommercialisation,adoption,andutilisation

• Drivenbyrisingmedicalcostsandincreasingvolumesofdata• Helpingphysiciansmakemoreinformedandaccuratedecisionsaboutpatientcare

• Evaluatesandinterpretsmedicaldata,offersrecommendations,andmakespredictions

• Unlockhiddeninsightswithinthedatawithdeepsemanticunderstandingofthecontent

• Fightdiseaseslikecancer,heartstroke,autismandParkinsons


WatsonTacklesCancer

• Watsonwastestedon1,000cancerdiagnosesmadebyhumanexperts.In99%ofthem,Watsonrecommendedthesametreatmentasthe oncologists• In30%ofthecases,Watsonalsofoundatreatmentoptionthehumandoctorsmissed• Sometreatmentswerebasedonresearchpapersthatthedoctorshadnotread— morethan160,000cancerresearchpapersarepublishedayear• Othertreatmentoptionssurfacedinnewclinicaltrialstheoncologistshadnotyetseenannouncedontheweb


Scan&X-rayDiagnostics

• X-raystodetectfractures• Greaterspeedandaccuracy• Takesworkloadoffdoctors

• Tumor detection• fMRIscans&EEG’s• Anomalydetection

• Eyes,kidneys,lungs,etc.

• Imageclassification• Usingconvolutionalneuralnetworks(CNNs)


The team was able to:• Process 10,900 payments per second• Cut false alarm rates by 50%• Free up resources to combat true fraud

PayPal is improving cybersecurity with deep learning

©IvyDataScience

Fintech

Chatbots (NLP)

• Chatbots,intheformofassistantsandautomatedcustomerservicereps,arebecomingincreasinglycommonacrosstheindustry

• AIisalsoprimedtomakethemassivelycomplicated(anddata-rich)worldoflogisticsmucheasierforretailers

• AIthatcanintuitwhatashopper’sstyleisandadaptitsrecommendationsassheorheshops

• AIthatcanevolveawebsitetospecificconsumerneeds• AIthatcanunderstanduserconcernsandanswercomplicatedquestions

• Makeshoppingbotheasierandmorepersonal


Whereareweheaded?


World Economic Forum (WEF) Report, 2016:Today, we are at the beginning of a Fourth Industrial Revolution. Developments in genetics, artificial intelligence, robotics, nanotechnology, 3D printing and biotechnology, to name just a few, are all building on and amplifying one another. This will lay the foundation for a revolution more comprehensive and all-encompassing than anything we have ever seen

Deepmind Mission: Solve intelligence then solve everything else

©IvyDataScience


WhereIvyDataScienceFitsInTheFirstArtificialIntelligenceBootcamp

CertificationfromExperfyHarvardi-lab|PaidConsultingProjects|JobPlacement

© Ivy Data Science

IvyDataScienceAI- StateofPlayv0.10PeterMorganDec2016 59© Ivy Data Science

TrainingwithJobPlacement

• LedbyExpertsfromHarvard,Columbia,UMass,UCL• Realtimepaidprojectsinthelastfourweeksofbootcamp• Practicalhands-ontrainingthatwillprepareyouforacareer asaDataScientist,MachineLearningorDeepLearningResearcher/Practitioner


TuitionFees

• Online Instructorledcoursesstartingat$1,999- 50hrs• EveningBootcamp startingat$4,999- 50hrs• WeekendBootcamp startingat$6,999- 100hrs• FullImmersive(12week)Bootcamp startingat$11,999*- 320hrs*earlybird• Financingavailable

IvyDataScienceAIwithApplicationsv0.10PeterMorganDec2016 61© Ivy Data Science

NetworkofClients


AnyQuestions?

© Ivy Data Science 63© Ivy Data Science 63

[email protected]

@ivydatascience.com

•www.ivydatascience.com | @ivydatascience


Date post:	13-Feb-2017
Category:	Technology
Upload:	peter-morgan
View:	172 times
Download:	0 times