Metaprogramming
CS251ProgrammingLanguagesSpring2017,LynTurbakDepartmentofComputerScienceWellesleyCollege
TheseslidesborrowheavilyfromBenWood’sFall‘15slides.
Howtoimplementaprogramminglanguage
InterpretaAonAninterpreterwriDenintheimplementaAonlanguagereadsaprogramwriDeninthesourcelanguageandevaluatesit.
TranslaAon(a.k.a.compilaAon)Antranslator(a.k.a.compiler)wriDenintheimplementaAonlanguagereadsaprogramwriDeninthesourcelanguageandtranslatesittoanequivalentprograminthetargetlanguage.
ButnowweneedimplementaAonsof:implementaAonlanguagetargetlanguage
Metaprogramming 2
Metaprogramming:InterpretaAon
InterpreterforlanguageLonmachineM
MachineMPrograminlanguageL
Metaprogramming 3
Interpreters
Data
Output
Source Program
Interpreter =
virtual machine
Metaprogramming 4
Metaprogramming:TranslaAon
InterpreterforlanguageBonmachineM
MachineM
PrograminlanguageA AtoBtranslator
PrograminlanguageB
Metaprogramming 5
if (x == 0) { x = x + 1; } ...
cmp (1000), $0 bne L add (1000), $1 L: ...
C Source Program C Compiler
x86 Target Program
Compiler
x86 Target Program
x86 computer
Data
Output
Metaprogramming 6
ThankstoBenWoodfortheseandfollowingpictures
JavaCompiler
if (x == 0) { x = x + 1; } ...
load 0 ifne L load 0 inc store 0 L: ...
Source Program Java Compiler
Target Program
(compare compiled C to compiled Java) Metaprogramming 7
InterpretersvsCompilers
InterpretersNoworkaheadofLmeIncrementalmaybeinefficient
CompilersAllworkaheadofLmeSeewholeprogram(ormoreofprogram)TimeandresourcesforanalysisandopLmizaLon
Metaprogramming 8
Compilers...whoseoutputisinterpreted
Target Program Java
Virtual Machine
Data
Output
Source Program Java Compiler
Target Program
Doesn’tthislookfamiliar? Metaprogramming 9
Interpreters...thatusecompilers.
Target Program
Virtual Machine
Data
Output
Source Program Compiler
Metaprogramming 10
TypicalCompiler
Source Program
Lexical Analyzer
Syntax Analyzer
Semantic Analyzer
Intermediate Code Generator
Code Optimizer
Code Generator
Target Program
Analysis
Synthesis
Metaprogramming 11
JITCompilersandOpAmizaAon
Target Program
Virtual
Machine Data
Output
Just In Time Compiler
PerformanceMonitor
Source Program
Compiler
• HotSpotJVM• JikesRVM• SpiderMonkey• v8• Transmeta• ...
Metaprogramming 12
VirtualMachineModel
High-LevelLanguageProgram
VirtualMachineLanguage
Bytecodecompiler
Virtualmachine(interpreter)
JITcompiler
runAme
compileAme
Ahead-of-Lmecompiler
NaLveMachineLanguage
Metaprogramming 13
Howtoimplementaprogramminglanguage
InterpreterRuleP-in-Lprogram Linterpretermachine
PmachineTranslatorRule
P-in-Sprogram S-to-TtranslatormachineP-in-Tprogram
Candescribebyderivinga“proof”oftheimplementaLonusingtheseinferencerules:
Metaprogramming 14
ImplementaAonDerivaAonExample
Provehowtoimplementa"251webpagemachine"using:
• 251-web-page-in-HTMLprogram(awebpagewriDeninHTML)• HTML-interpreter-in-Cprogram(awebbrowserwriDeninC)• C-to-x86-translator-in-x86program(aCcompilerwriDeninx86)• x86interpretermachine(anx86computer)
Nopeakingahead!
Metaprogramming 15
ImplementaAonDerivaAonExampleSoluAon
Wecanomitsomeoccurrencesof“program”and“machine”:
Metaprogramming 16
ImplementaAonDerivaAonAreTreesAndsowecanrepresentthemasnestedstructures,likenestedbulletedlists:
251webpagemachine(I)q 251-web-page-in-HTMLprogramq HTMLinterpretermachine(I)
² HTML-interpreter-in-x86program(T)o HTML-interpreter-in-Cprogramo C-to-x86compilermachine(I)
• C-to-x86compiler-in-x86program• X86computer
² x86computer
q 251-web-page-in-HTMLprogramo HTML-interpreter-in-Cprogram
• C-to-x86compiler-in-x86program• X86computer
o C-to-x86compilermachine(I)² HTML-interpreter-in-x86program(T)² x86computer
q HTMLinterpretermachine(I)251webpagemachine(I)
Versionthatshowsconclusionsbelowbullets.MoresimilartoderivaLonswithhorizontallines,buthardertocreate,andread
Preferred“top-down”versionthatshowsconclusionsabovebullets.
Metaprogramming 17
Metaprogramming:BootstrappingPuzzles
HowcanwewriteaJava-to-x86compilerinJava?
HowcanwewriteSchemeinterpreterinScheme?
Metaprogramming 18
MetacircularityandBootstrappingManyexamples:• LispinLisp/SchemeinScheme/RacketinRacket• PythoninPython:PyPy• JavainJava:JikesRVM,MaxineVM• …• C-to-x86compilerinC• evalconstructinlanguageslikeLisp,JavaScript
Howcanthisbepossible?
Keyinsightstobootstrapping:• ThefirstimplementaLonofalanguagecannotbein
itself,butmustbeinsomeotherlanguage.• OnceyouhaveoneimplementaLonofalanguage,you
canimplementitinitself.Metaprogramming 19
Supposeyouaregiven:• Scheme-interpreter-in-Pythonprogram• Pythonmachine• Scheme-interpreter-in-SchemeprogramHowdoyoucreateaSchemeinterpretermachineusingtheScheme-interpreter-in-Schemeprogram?
MetacircularityExample1:Problem
Metaprogramming 20
Supposeyouaregiven:• Scheme-interpreter-in-Pythonprogram• Pythonmachine• Scheme-interpreter-in-SchemeprogramHowdoyoucreateaSchemeinterpretermachineusingtheScheme-interpreter-in-Schemeprogram?
Schemeinterpretermachine#2(I)q Scheme-interpreter-in-Schemeprogramq Scheme-interpretermachine#1(I)
² Scheme-interpreter-in-Pythonprogram² Pythonmachine
ButwhycreateSchemeinterpretermachine#2whenyoualreadyhaveScheme-interpretermachine#1?
MetacircularityExample1:SoluAon
Metaprogramming 21
Supposeyouaregiven:• Scheme-subset-interpreter-in-Pythonprogram(implements
onlycoreSchemefeatures;nodesugaringorotherfrills)• Pythonmachine• Full-Scheme-interpreter-in-SchemeprogramHowdoyoucreateaFull-SchemeinterpretermachineusingtheFull-Scheme-interpreter-in-Schemeprogram?
Full-Schemeinterpretermachine(I)q Scheme-interpreter-in-Schemeprogramq Scheme-subsetinterpretermachine#1(I)
² Scheme-subset-interpreter-in-Pythonprogram² Pythonmachine
MetacircularityExample1:MoreRealisAc
Metaprogramming 22
Supposeyouaregiven:• C-to-x86-translator-in-x86program(aCcompilerwriDeninx86)• x86interpretermachine(anx86computer)• C-to-x86-translator-in-C-subsetprogram
HowdoyoucompiletheC-to-x86-translator-in-C?
MetacircularityExample2:Problem
Metaprogramming 23
Supposeyouaregiven:• C-to-x86-translator-in-x86program(aCcompilerwriDeninx86)• x86interpretermachine(anx86computer)• C-to-x86-translator-in-Cprogram
HowdoyoucompiletheC-to-x86-translator-in-C?
MetacircularityExample2:SoluAon
C-to-x86-translatormachine#2(I)q C-to-x86-translator-in-x86program#2(T)
² C-to-x86-translator-in-C² C-to-x86-translatormachine#1(I)
o C-to-x86-translator-in-x86program#1o x86computer
q x86computer
ButwhycreateC-to-x86-translator-in-x86program#2(T)whenyoualreadyhaveC-to-x86-translator-in-x86program#1?
Metaprogramming 24
Supposeyouaregiven:• C-subset-to-x86-translator-in-x86program
(acompilerforasubsetofCwriDeninx86)• x86interpretermachine(anx86computer)• Full-C-to-x86-translator-in-C-subsetprogram
(acompilerforthefullClanguagewriDeninasubsetofC)
HowdoyoucreateaFull-C-to-x86-translatormachine?
MetacircularityExample2:MoreRealisAc
Full-C-to-x86-translatormachine(I)q Full-C-to-x86-translator-in-x86program(T)
² Full-C-to-x86-translator-in-C-subset² C-subset-to-x86-translatormachine(I)
o C-subset-to-x86-translator-in-x86programo x86computer
q x86computerMetaprogramming 25
AlonglineofCcompilersC-version_n-to-target_n-translatormachine(I)q C-version_n-to-target_n-translatorprogramintarget_n-1(T)
² C-version_n-to-target_n-translatorprograminC-version_n-1² C-version_n-1-to-target_n-1translatormachine(I)
o C-version_n-1-to-target_n_1-translatorprogramintarget_n-2(T)
Ø C-version_2-to-target_2-translator-programintarget_1(T)
§ C-version_2-to-target_2-translatorprograminC-version_1§ C-version_1-to-target_1translatormachine(I)
• C-version_1-to-target_1-translatorprograminassembly_0• assembly_0computer
Ø target_1computer
o target_n-2computerq target_n-1computer
…
…
o TheversionsofCandtargetlanguagescanchangeateachstage.o Trojanhorsesfromearliersourcefilescanremainintranslatormachinesevenif
they’renotinlatersourcefile!SeeKenThompson’sReflec4ononTrus4ngTrustMetaprogramming 26
MoreMetaprogramminginSML
• We’vealreadyseenPostFixandIntexSML• AsequencesofexpressionlanguagesimplementedinSMLthatlookcloserandclosertoRacket:• Bindex:addnaming• Valex:addmorevaluetypes,dynamictypechecking,desugaring
• HOFL:firstclassfuncLonvalues,closurediagrams
Metaprogramming 27
Remember:language!=implementaAon• Easytoconfuse"thewaythislanguageisusually
implemented"or"theimplementaLonIuse"with"thelanguageitself.”
• JavaandRacketcanbecompiledtox86
• CcanbeinterpretedinRacket
• x86canbecompiledtoJavaScript
• CanwecompileC/C++toJavascript?hDp://kripken.github.io/emscripten-site/
Metaprogramming 28