ManageIQ
Sprint 48 Review - Sprint End October 24, 2016
October 26, 2016
Overview● Sprint Statistics (Oleg Barenboim)● Classic UI (Harpreet Kataria)● Service UI (Chris Kacerguis)● Providers (Greg Blomquist)● Automate (Greg McCullough)● Platform (Gregg Tanzillo)● API (Alberto Bellotti)● Quality Engineering (Pete Savage)● Discussion
Sprint Statistics (O. Barenboim)
404 Pull Requests Merged for ManageIQ/manageiq
Sprint Statistics - Euwe Backports(O. Barenboim)
266 Pull Requests Backported to Euwe
PR Breakdown by Feature Category (O. Barenboim)
* Note that some PRs have more than one category.
Providers
Sprint Comparison (O. Barenboim)
Dar
ga-2
Dar
ga B
eta
Dar
ga -3
Dar
ga G
A
Dar
ga R
C
Dar
ga -
4
Euw
e B
eta
All Repo Stats (O. Barenboim)
Data source:
https://github.com/ManageIQthese are listed by most recent activity. go into each one that has been updated in the past 20 days and add up the PRs that have been merged
Order descending by #PRs merged
Repository PRsmanageiq 404integration_tests 99manageiq.org 53manageiq-ui-self_service 49manageiq-pods 14guides 12manageiq_docs 12manageiq-providers-amazon 11manageiq-providers-lenovo 9manageiq-appliance-build 7azure-armest 6miq_bot 6wrapanapi 6manageiq-appliance 4manageiq-api-client 1Total 693
Classic UI(Harpreet Kataria)
● PRs merged (171)○ Bugs (77)○ Enhancements (35)○ Tech Debt / Refactoring (20/27)○ Enhancements Backported to EUWE (10)
● Tech Debt / Refactoring○ Removed some old redundant and commented code○ RJS: cleanup of flash message div rendering○ Restored all previous TreeNodeBuilder#build functionality○ File chooser form control was moved into a common single partial○ Font icon cleanup(ongoing)○ Network provider converted to use RESTful routes like other providers
Classic UI(Harpreet Kataria)
● Enhancements○ Host Aggregates UI (No screenshots available)○ Snapshots actions support for Volumes○ Snapshots actions support for Openstack VMs○ Support to create/update/delete OpenStack Cloud Tenants○ Support to create/update/delete Cloud Networks.○ Added Container Templates subtab under Containers○ Support for Cinder backup actions○ Added Ironic Controls support for OpenStack Infrastructure Hosts○ Support to show Cloud Volume Backup - Link to list of Cloud Volume
Backups was added on Cloud Volume summary screen, from there you can drill into its details view screen.(No screenshots available)
○ Support to allow self signed certificates on Automate Import screen (covered by GregM’s team)
Classic UI(Harpreet Kataria)
● Enhancements○ Middleware Datasource validation enhancement○ Added Topology views for Cloud & Infra providers○ Support to generate authorization keys for remote regions○ Added Dashboard view for Infrastructure Providers○ Basic support for displaying chargeback report data in Service UI○ Option to hide deprecated template images by default during provisioning○ Schedule Automate Tasks (Demo: GregM’s team)○ Support for proxying the VNC or SPICE connection(Demo: Martin
Povolny)○ Hide VMs & Instances in explorer trees (Demo: Zita Nemeckova)
Classic UI(Harpreet Kataria)
Snapshots actions support for Volumes Snapshots actions support for Openstack VMs
Classic UI(Harpreet Kataria)
Support to create/update/delete OpenStack cloud Tenants
Classic UI(Harpreet Kataria)
Support to create/update/delete Cloud Networks
Classic UI(Harpreet Kataria)
New sub-tab Container Templates under Containers
Classic UI(Harpreet Kataria)
Added Ironic Controls support for OpenStack Infrastructure Hosts
Classic UI(Harpreet Kataria)
Support for Cinder backup actions
Classic UI(Harpreet Kataria)
Middleware Datasource validation enhancement
Classic UI(Harpreet Kataria)
Added Topology views for Cloud & Infra providers
Classic UI(Harpreet Kataria)
Support to generate authorization keys for remote regions
Classic UI(Harpreet Kataria)
Added Dashboard view for Infrastructure Providers
Classic UI(Harpreet Kataria)
Option to hide deprecated template images by default during provisioning
Classic UI(Harpreet Kataria)
Basic support for displaying chargeback report data in Service UI
Classic UI(Harpreet Kataria)
● Schedule Automate Tasks (Demo: GregM’s team)
● Support for proxying the VNC or SPICE connection (Demo: Martin Povolny)● Hide VMs & Instances in explorer trees (Demo: Zita Nemeckova)
Service UI(C. Kacerguis)
● Back of the House Changes○ Completed the renaming of the Repo (ManageIQ-UI-Service)○ Migrated to yarn for dependency management○ Increased test coverage to 58%
● Added confirmation when removing items from the shopping cart
● Added basic support for displaying chargeback data● Added the Service Power operations for Start, Stop and
Suspend● Lots of bug fixes and technical debt cleaned up
Providers(G. Blomquist)
Containers(G. Blomquist)
● Container Templates● Chargeback for container images● Support Custom Attributes
Containers(G. Blomquist)
Container Templates
Containers(G. Blomquist)
Chargeback for Container Images
Containers(G. Blomquist)
Support Custom Attributes
Google(G. Blomquist)
● Add load Balancer health checks● Hide deprecated images during provisioning
Hawkular(G. Blomquist)
● Validate when there is no daily data for live metrics● Conversion of Provider form to Angular● Fixed 'Summary' Link in the accordion menu● Better Datasource validation● Hide operation toolbar buttons
OpenStack(G. Blomquist)
● Lots of UI updates (Harpreet will cover this)● Enable node start/stop
RHEV(G. Blomquist)
Refresh performance improvements when creating a new VM
Lenovo(G. Blomquist)
Getting started … first commits rolling in
vSphere(G. Blomquist)
Provisioning validation when selecting hosts/clusters
vCloud(G. Blomquist)
Checkboxes in Provisioning
Pluggable Providers(G. Blomquist)
● supports :timeline● supports :terminate● supports :shutdown_guest● supports :reboot_guest
Automate(Greg McCullough)
Schedule Automate Tasks● Specify options similar to Automate -> Simulate
○ System/Process Instance○ Message○ Request○ Optional
■ Target Object■ Additional Key/Value Pairs
● Creates Automation Request object● Run Once or on a recurring schedule
Automate(Greg McCullough)
Schedule Automate Tasks
Automate(Greg McCullough)
Schedule Automate Tasks
Automate(Greg McCullough)
New notification methods enable users to create custom notifications from Automate:● $evm.create_notification
● $evm.create_notification! - raises errors.
Currently supports User level notifications.
Notification and NotificationType Service Models added.
Automate(Greg McCullough)
New $evm.create_notification Method
Basic notification:
Specify only a message:
$evm.create_notification(:message => “my custom provisioning job has started”)
Generates a “user” “info” notification.
Automate(Greg McCullough)
New $evm.create_notification Method
Custom notification:
Specify level and subject:
$evm.create_notification(:level => “error”, :subject => vm, :message => “my custom provisioning job has failed”)
Generates a “user” “error” notification.
Automate(Greg McCullough)
New $evm.create_notification Method
Predefined notification:
Specify type with subject:
$evm.create_notification(:type => :vm_provisioned, :subject => vm)
Generates a “user” “info” notification.
Automate(Greg McCullough)
New $evm.create_notification Method
Next Steps:
● Global/Tenant level support.
● The addition of $evm.create_notification to life cycle
state machine methods. (start/finish/error)
Automate(Greg McCullough)
Retirement Enhancements:
An unsuccessful retirement can leave a VM in an archived/orphaned state with an “unknown” power state.
Although the majority of these VM’s have been removed from the provider, our current retirement process prevents these VMs from fully retiring.
Automate(Greg McCullough)
Retirement Enhancements:● Allow VM’s with unknown power state to retire.
○ Changed check_pre_retirement method to allow a
VM with an “unknown” power state to proceed to
retirement.
● Allow Archived/Orphaned VMs to retire.
○ Changed remove_from_provider method to allow an
archived/orphaned VM to complete retirement.
Automate(Greg McCullough)
● Import Automate model from Git Repository○ Support for Self Signed Certificates○ Support for Rake command○ Route the import request to the appliance with the
‘Git Repository Owner’ role
Automate(Greg McCullough)
● Self Signed Certificate support○ Uncheck the “Verify Peer Certificate”
Automate(Greg McCullough)
Rake command● Required Parameters
○ GIT_URL● Optional Parameters
○ USERID○ PASSWORD○ REF = Branch name (default is master)○ VERIFY_SSL = 0 | 1 (default 1)○ REF_TYPE = BRANCH|TAG (default is BRANCH)
Automate(Greg McCullough)
Catalog Items
● Default provisioning state-machine entry-point updated○ Catalog Item - CatalogItemInitialization○ Catalog Bundle - CatalogBundleInitialization
Automate(Greg McCullough)
Generic Objects● Pass “param_#_type” value to automate for method calls:
○ Example:■ param_1 = “1.0”■ param_1_type = “Float”
● Expose service model methods:○ property_attributes○ property_associations○ property_methods○ Find_objects
■ Search by active_record and generic object attributes■ Search by associations (using association IDs)
Automate(Greg McCullough)
Added cinder backup/restore actions to Automate
MiqAeServiceManageIQ_Providers_Openstack_CloudManager_CloudVolume
● Method Definition: ○ backup_create(backup_name = "", incremental = false, options = {})
● Method Definition:○ backup_restore(backup_id, options = {})
● Option○ :sync = true/false (Default: false)
Provisioning● No longer appending “_” to the VM Name when generating unique names
for service provisioning.○ Underscore is an invalid character on some Providers○ Example: VM Name = test_vmware
■ Before: test_vmware_001■ After: test_vmware001
○ If required, append VM Name/Instance Name with “_” in catalog item.
Automate(Greg McCullough)
Automate(Greg McCullough)
That’s it, the party is over.
Platform(Gregg Tanzillo)
Enhancements / Bug Fixes - 95 PRs Merged!● Chargeback
● Tenancy
● Centralized Administration
● Additional updates
● API
Platform: Chargeback(Gregg Tanzillo)
● Major rework of calculations
○ No longer based on the count of metrics
○ Instead, based on interval duration (hour/week/month)
○ Fixed rate costs now correct for weekly and monthly
intervals
○ Metrics are either avg or max of interval instead of the
sum
● Reinstated weekly and monthly rates
Platform: Chargeback(Gregg Tanzillo)
● JSON generation of chargeback report for service
○ Supports showing service cost in Service UI
○ Includes Rest API
GET /api/services/:id?attributes=chargeback_report
● VM GUID a reportable field
● Many PRs for refactoring and performance (thank you
Simon)
Platform: Tenancy(Gregg Tanzillo)
● Ad hoc sharing of resources across tenants
○ Backend complete
○ VMs, templates and Services
○ Work on API and user interface underway
Platform: Centralized Administration(Gregg Tanzillo)
● VM reconfigure
● Provision VM from a template
● User and group ownership working in global region
○ Without triggers :)
Platform: Centralized Administration(Gregg Tanzillo)
● Service provisioning - close
○ Addressing some last minute bugs
○ Classic UI only
● Regional method relay mixin for generic central
administration of models
○ New DSL defined by the mixin
○ Enables dynamic of methods routing to a remote
region
Platform: Additional updates(Gregg Tanzillo)
● Provider authentication retry
○ Default schedule changed from 1 day to 1 hour
○ Retry on recoverable errors
○ Incremental delay between retries - 2 min, 4, 8 and 16
● System memory/swap information saved with status
○ Will now be able to detect when swap is invaded
Platform: Additional updates(Gregg Tanzillo)
● Configure a different disk for log files
○ Through appliance console
○ Longer retention of log history
● Increased log volume from 2gb to 11gb
API(Alberto Bellotti)
● Added ability to Create and Edit /api/requests
Interface to MiqRequest:
create_request(values, requester, auto_approve) update_request(request, values, requester)
POST /api/requests{
“options” : {
“request_type” : “service_reconfigure”,
“src_id” : 4,
“config_param” : “some_value_1”
},
“requester” : {
“user_name” : “admin”
},
“auto_approve” : true
}
POST /api/requests/:id{
“options” : {
“config_param” : “some_value_2”
},
“requester” : {
“user_name” : “admin”
}
}
API(Alberto Bellotti)
● Introduced new /api/automate_domains primary collection○ Query and Actions on MiqAeDomain’s
○ Moved refresh_from_source action from /api/automate to /api/automate_domains
- Simplifies implementation
- refresh_from_source can now be triggered from Api Client gem
miq.automate_domains.find_by(:name => “ManageIQ”).refresh_from_source
● Updated refresh_from_source to work asynchronously
● /api/automate is strictly for Automate Browser queries
miq.get(“automate/manageiq”, :depth => -1, :search_options => “state_machines”)
API(Alberto Bellotti)
● Added support for Provider Custom Attributes○ Same signatures as the custom attributes support for Vms
Queries:
GET /api/providers/:id?expand=custom_attributes
GET /api/providers/:id/custom_attributes
Adding, Editing and Deleting custom attributes via actions on the subcollection:
POST /api/providers/:id/custom_attributes
{
“action” : “add” | “edit” | “delete”,
“resources” : [
{ “name” : “ca_name1”, “value” : “ca_value1” },
{ “name” : “ca_name2”, “value” : “ca_value2” },
...
]
}
API(Alberto Bellotti)
● Added Service Power Operations to /api/services
All service power actions are asynchronous and include the task_id/href in the results.
○ Service resource operations on /api/services/:id
POST /api/services/:id{
“action” : “start” | “stop” | “suspend”}
○ Bulk Service operations on /api/services
POST /api/services{
“action” : “start” | “stop” | “suspend”,“resources” : [
{ “href” : “http://localhost:3000/api/services/101” },{ “href” : “http://localhost:3000/api/services/102” },...
]}
API(Alberto Bellotti)
● Added ability to Create and Edit Service Dialogs○ Create Service Dialog:
POST /api/service_dialogs{
“description” : “...”,“label” : “...”,“dialog_tabs” : [ ]
}
○ Edit Service Dialog (individual and bulk edits)
POST /api/service_dialogs/:id{
“action” : “edit”,“resource” : {
“label” : “...”,}
}
Quality Engineering(Pete Savage)
Manual
● Continuing with upstream testing○ Full regression testing○ Sprint New Feature testing
● 38 new deliverables in Sprint 47○ 53% looking good○ 3% blocked, buggy or not fully implemented○ 44% unknown, testing in progress
Quality Engineering(Pete Savage)
Automation● Still seeing some UI quirks where page is not ready when we interact with it
○ Under investigation
● Navmazing (new navigation framework) conversion○ 677 -> 398 removals of old force_navigate○ More in PRs yet to be merged
● Widgetastic (new page model framework) taking shape○ Have several PRs now using this much more explicit design
Quality Engineering(Pete Savage)
Integration Tests
99 PRs Merged
Discussion
Sprint 49 Review - November 16