Date post: | 25-Dec-2014 |
Category: |
Technology |
Upload: | puppet-labs |
View: | 279 times |
Download: | 1 times |
2014
Presented by
The Grand Puppet Sub-Systems TourNick Fagerlund Technical Writer / Weird Bugs | Puppet Labs @nfagerlund
Presented by
PIPES
Presented by
What does Puppet DO?• Gather system information • Compile a catalog • Apply catalog • Report
Presented by
What does Puppet DO?AGENT
• Gather system information
• Apply catalog • Report
MASTER • Compile a catalog
Presented by
STOP 1: Command Line and the
Presented by
Someone ran puppet agent
Presented by
???• /opt/puppet/bin/puppet • Puppet::Util::CommandLine.new.execute • Puppet::Application.find • Puppet::Application::Agent.new • Puppet::Application::Agent#setup • Puppet::Application::Agent#run_command
Presented by
#run_command# lib/puppet/application/agent.rb def run_command if options[:fingerprint] fingerprint else # It'd be nice to daemonize later, but we have to daemonize before # waiting for certificates so that we don't block daemon = daemonize_process_when(Puppet[:daemonize]) ! wait_for_certificates ! if Puppet[:onetime] onetime(daemon) else main(daemon) end
Presented by
onetime() and main()# lib/puppet/application/agent.rb begin exitstatus = daemon.agent.run rescue => detail Puppet.log_exception(detail) end ! daemon.stop(:exit => false) ! if not exitstatus exit(1) elsif options[:detailed_exitcodes] then exit(exitstatus) else exit(0) end
# lib/puppet/application/agent.rb def main(daemon) if Puppet[:listen] setup_listen(daemon) end Puppet.notice "Starting Puppet client version #{Puppet.version}" ! daemon.start end
Presented by
Obtaining the daemon# lib/puppet/application/agent.rb def daemonize_process_when(should_daemonize) daemon = Puppet::Daemon.new(Puppet::Util::Pidlock.new(Puppet[:pidfile])) daemon.argv = @argv daemon.agent = @agent ! daemon.daemonize if should_daemonize ! daemon end
Presented by
STOP 2: The Daemon and the
Presented by
DAEMON LAND?!
Nope Bye
Presented by
Puppet::Agent• Run a thing
Presented by
Puppet::Daemon• Run a thing that runs the other thing
Presented by
Lots of layers of abstraction??• Historical reasons!
Presented by
STOP 3: The Configurer
Presented by
Puppet::Configurer
Presented by
Puppet::Configurer#run_internal# lib/puppet/configurer.rb def run_internal(options) # We create the report pre-‐populated with default settings for # environment and transaction_uuid very early, this is to ensure # they are sent regardless of any catalog compilation failures or # exceptions. options[:report] ||= Puppet::Transaction::Report.new("apply", nil, @environment, @transaction_uuid) report = options[:report] init_storage ! Puppet::Util::Log.newdestination(report) … … …
Presented by
Puppet::Configurer#run_internal# lib/puppet/configurer.rb … query_options = get_facts(options) unless query_options …
Presented by
Puppet::Configurer#run_internal# lib/puppet/configurer/fact_handler.rb … facts = Puppet::Node::Facts.indirection.find(Puppet[:node_name_value], :environment => @environment) …
Presented by
Puppet::Configurer#run_internal# lib/puppet/configurer.rb … unless catalog = prepare_and_retrieve_catalog(options, query_options) return nil end …
Presented by
Puppet::Configurer#run_internal# lib/puppet/configurer.rb … apply_catalog(catalog, options) …
Presented by
Where next?
Retrieve Apply
Presented by
STOP 4: The Transaction
Presented by
(Puppet::Resource::Catalog)
Presented by
Puppet::Resource::Catalog
Catalog
Resource Other Stuff
Presented by
Puppet::Configurer (again)# lib/puppet/configurer.rb def convert_catalog(result, duration) catalog = result.to_ral catalog.finalize catalog.retrieval_duration = duration catalog.write_class_file catalog.write_resource_file catalog end
Presented by
RAL Catalog
Presented by
RAL Catalog # lib/puppet/resource/catalog.rb def to_catalog(convert) result = self.class.new(self.name, self.environment_instance) result.version = self.version map = {} resources.each do |resource| next if virtual_not_exported?(resource) next if block_given? and yield resource newres = resource.copy_as_resource newres.catalog = result if convert != :to_resource newres = newres.to_ral end # We can't guarantee that resources don't munge their names # (like files do with trailing slashes), so we have to keep track # of what a resource got converted to. map[resource.ref] = newres result.add_resource newres end
Presented by
RAL CatalogPuppet::Resource
!
↓ !
(something) << Puppet::Type
Presented by
RAL Catalog
???
Presented by
RAL Catalog# lib/puppet/configurer.rb def apply_catalog(catalog, options) report = options[:report] report.configuration_version = catalog.version ! benchmark(:notice, "Finished catalog run") do catalog.apply(options) end ! report.finalize_report report end
Presented by
Puppet::Resource::Catalog#apply# lib/puppet/resource/catalog.rb def apply(options = {}) Puppet::Util::Storage.load if host_config? ! transaction = create_transaction(options) ! begin transaction.report.as_logging_destination do transaction.evaluate end rescue Puppet::Error => detail !
Presented by
Transaction
???
Presented by
Transaction• Ral catalog • Prioritizer • Report
Presented by
Transaction• Ral catalog • Prioritizer • Report • Event manager • Resource harness • Relationship graph
Presented by
Transaction• Ral catalog • Prioritizer • Report • Event manager • Resource harness • Relationship graph • relationship_graph.traverse
Presented by
Transaction• Ral catalog • Prioritizer • Report • Event manager • Resource harness • Relationship graph • relationship_graph.traverse • *ollies out*
Presented by
STOP 5: The Indirector
Presented by
Back to Puppet::Configurer• #prepare_and_retrieve_catalog
Presented by
Back to Puppet::Configurer# lib/puppet/configurer.rb result = Puppet::Resource::Catalog.indirection.find( Puppet[:node_name_value], query_options.merge(:ignore_cache => true, :environment => @environment, :fail_on_404 => true) )
Presented by
The Indirector!
Puppet::Resource::Catalog.indirection.find()
Presented by
The Indirector
← ↑ ↓ ?
Presented by
The Indirector• Globally configured • Abstract • Gets stuff for you
Presented by
The Indirector # lib/puppet/resource/catalog.rb extend Puppet::Indirector indirects :catalog, :terminus_setting => :catalog_terminus
Presented by
The Indirector• find • search • save • destroy
Presented by
The Indirector• find • search • save • destroy • terminus_class • cache_class • lib/puppet/indirector/…
Presented by
Puppet Agent• Catalog => REST • Report => REST • Facts => Facter
Presented by
Puppet Apply• Catalog => Compiler • Report => Processor • Facts => Facter
Presented by
Avoiding the cache# lib/puppet/configurer.rb result = Puppet::Resource::Catalog.indirection.find( Puppet[:node_name_value], query_options.merge(:ignore_cache => true, :environment => @environment, :fail_on_404 => true) )
Presented by
The Indirector• Why?
Presented by
The Indirector• Why? • Global? Rigid?
Presented by
The Indirector• Why? • Global? Rigid? • IT MAKES SENSE IN CONTEXT
Presented by
The Indirector• …Catalog.find( <FACTS> ) • REST terminus • HTTP POST request to master
Presented by
STOP 6: The Puppet Master’s
Presented by
RackWeb server
↓ Rack server
↓ “App” object that responds to #call()
↓ Actual application logic
Presented by
config.ru $0 = "master" !
ARGV << "-‐-‐rack" !
ARGV << "-‐-‐confdir" << "/etc/puppet" ARGV << "-‐-‐vardir" << "/var/lib/puppet" !
require 'puppet/util/command_line' run Puppet::Util::CommandLine.new.execute !
Presented by
Starting master w/ Rack# lib/puppet/application/master.rb def start_rack_master require 'puppet/network/http/rack' !
announce_start_of_master !
return Puppet::Network::HTTP::Rack.new() end
Presented by
The app object# lib/puppet/network/http/rack.rb class Puppet::Network::HTTP::Rack def call(env) request = Rack::Request.new(env) response = Rack::Response.new Puppet.debug 'Handling request: %s %s' % [request.request_method, request.fullpath] ! begin Puppet::Network::HTTP::RackREST.new.process(request, response) rescue => detail # Send a Status 500 Error on unhandled exceptions. response.status = 500 response['Content-‐Type'] = 'text/plain' response.write 'Internal Server Error: "%s"' % detail.message # log what happened Puppet.log_exception(detail, "Puppet Server (Rack): Internal Server Error: Unhandled Exception: \"%s\"" % detail.message) end response.finish
Presented by
#process# lib/puppet/network/http/handler.rb def process(request, response) new_response = Puppet::Network::HTTP::Response.new(self, response) !
request_headers = headers(request) request_params = params(request) request_method = http_method(request) request_path = path(request) … !
Presented by
#process# lib/puppet/network/http/handler.rb if route = @routes.find { |route| route.matches?(new_request) } route.process(new_request, new_response)
Presented by
lib/puppet/network/http/api/v1.rb
Presented by
lib/puppet/network/http/api/v1.rb• GET => indirection.find() • POST => indirection.find() (for catalogs)
• (There was an issue with some servers rejecting GET parameters that were too long. Funny story about that.)
• DELETE => indirection.destroy() • PUT => indirection.save() • GET to magical plural name => indirection.search()
Presented by
BIG PICTUREConfigurer calls catalog.indirection.find()
↓ HTTP POST request to master
↓ Request handler calls catalog.indirection.find()
↓ Compiler terminus handles it
↓
Presented by
STOP 7: The Compiler
Presented by
MOSTLY SKIPPING THISLexer ↓
Parser ↓
AST ↓
Evaluator ↓
Presented by
THIS IS THE TEST“Compiler terminus to the catalog
indirection” !
???
Presented by
THIS IS THE TEST
GOOD WORK EVERYBODY!
Presented by
Questions?
Presented by
THANKS!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!•Hope we’ll see you next year in
Portland!