DEV-03: Progress Dynamics® Performance Improvements - Application Data Sharing and Generated Objects
Peter JudgePrincipal Software Engineer
2 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Agenda Slide
Designing for runtime performance
Deploying repository data as code
Application data caching
3 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Progress Dynamics Components Overview
Manager Data
Repository
Abstract UI Definitions
Repository Manager
Security Manager
Session / Context Manager
Connection / Configuration Manager
Customization Manager
Localization Manager
General / Utility Manager
Personalization Manager
Audit ManagerDevelopment
Tools
DeploymentTools
ConfigurationTools
ManagersTools Repository
GUI &WEBRendering
Engines
4 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
n -Tier Design Challenges
n-tier design is complex– Define system boundaries– Design for worst case
Design is an ongoing, iterative process– Continually track performance
Network is usually the biggest bottleneck There’s no single solution
– Speed vs. application complexity– # of requests vs. size of request– Speed vs. deployment ease
5 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
How Dynamics Assists with n –Tier
Managers hide n-tier complexity– Common API for client and server
Managers seamlessly cache framework data
Data object architecture– RowsToBatch, RebuildOnReposition
In-built session & context management Caching support for application data
6 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
There Is No Magic Bullet
Guiding principles– All round trips are expensive
– All data is expensive Follow Dynamics n-tier design principles
– Use manager templates
– Common client and server API
– Leverage context manager for application data
– Leverage caching
7 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Designing for runtime performance
Deploying repository data as code
Application data caching
Agenda Slide
8 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Dynamics at Runtime
Manager Data
DynamicsRepository
Abstract UI Definitions
App Data
App DB
4GL GUIRuntime Client
4GL GUIRuntime Client
Rendering Engine
Data Object
Overview
9 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Dynamics at Runtime
Manager Data
DynamicsRepository
Abstract UI Definitions
4GL GUIRuntime Client
4GL GUIRuntime Client
Rendering EngineObject R-Code
Overview
Static Code Generation
10 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Static Code Generator
&scoped-define xp-Assign{set LogicalObjectName '##[ObjectName]##'} ##Loop:ObjectProperties-Assign##{set ##[PropertyName]## ##[PropertyValue]##}##Every:50##
&undefine xp-Assign&scoped-define xp-Assign##Every:End## ##Loop:End##.&undefine xp-Assign /* Keep forced 'Set' properties separate. */&scoped-define xp-Assign{set LogicalVersion '##[ObjectVersion]##'}##Loop:ObjectProperties-Set##{set ##[PropertyName]## ##[PropertyValue]##} ##Loop:End##.&undefine xp-Assign
Window_template.p&scoped-define xp-Assign{set LogicalObjectName ‘CustWin'}{set ObjectTranslated yes}{set ObjectSecured yes}{set HasObjectMenu ?}{set SuperProcedure ''}{set SuperProcedureMode ''} {set Page0LayoutManager '06'}{set RequiredPages '?'}{set PageLayoutInfo ''}{set InitialPageList '0'}{set StartPage 0}.&undefine xp-Assign
/* Keep forced 'Set' properties separate.*/&scoped-define xp-Assign{set LogicalVersion '19.84725'}{set PhysicalVersion '19.84725'}{set WindowName 'customer'}.&undefine xp-Assign
CustWin.pgen
Repository
Overview
GeneratorEngine
11 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Object Templates
Active templates– Standard 4GL code mixed with generator
instruction tags
– Not intended to compile or run Defined instruction tags
– Drive generator functionality Can be fragmentary
– Templates can include other templates
“A pattern used to create documents”
12 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Instruction Tags
Enhance templates– Add functionality & flexibility– Allow customization
Predefined, fixed set of instructions Call API in generator engine to
– Return value for token Store in context, access via API
– Perform some processing Logical, looping etc
Indicated by leading and trailing ##
Adding flexibility
13 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
DEF VAR c AS CHARASSIGN c = {fn getSomeValue}.{set WindowTitle ##[WindowTitle]##}.
...
...
Template Instruction Tags Simple Example
FIND Ryc_Attribute WHERE Object = ‘CustWin’ AND Label = ‘WindowTitle’ NO-LOCK.
Repository
DEF VAR c AS CHARASSIGN c = {fn getSomeValue}.{set WindowTitle ‘Customer Maintenance’}.
...
...
14 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
{set LogicalObjectName ##[ObjectName]##} ##Loop:AssignProps##{set ##[PropertyName]## ##[PropertyValue]##}##Loop:End##
...
...
Template Instruction Tags Looping Example
Function processLoop-AssignProps FOR EACH ryc_attribute WHERE Object = ‘CustWin’: ProcessLoopIteration(). END.End Function.
Repository
{set LogicalObjectName ‘CustWin'}{set ObjectTranslated yes}{set ObjectSecured yes}{set HasObjectMenu ?}{set SuperProcedure ''}{set SuperProcedureMode ''} {set Page0LayoutManager '06'}{set RequiredPages '?'}{set PageLayoutInfo ''}{set InitialPageList '0'}{set StartPage 0}
15 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Generator Engine Architecture
Template Output
Plug-in procedure
TT
Object Definition
TT
Generator Engine
TT
16 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
What is The Plug-in Procedure?
Defines and controls generator behavior Knows where the data lives Transforms data Contains overrides and hooks
The brains of the generator
17 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Using The Plug-in Procedure
Override generator behavior– Initialization, destruction of generator
E.g. Read all relevant data on init., populate temp-table before processing
– Writing of output E.g. Write to DB, not file as per default
Hook points for extension– Loop processing– Customize token getter and setters
e.g. Provide extension for output filename
Overriding the Generator Engine API
18 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Dynamics 4GL Code Generation
Dynamics code generation has own plug-in procedure: rygen4glhp.p
Output files have pgen extension Object types have own templates
– Stored in GenerationTemplate attribute
Transforming abstract definitions
19 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
What’s in Generated Code?
Complete object definition
Security calls– Security data NOT
generated into code
Menu / toolbar data Tree node data
Optional– Translations– Result Code
Customizations
As shipped, includes:
20 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Generated Code Considerations
Output can be stored anywhere– Runtime needs compiled version
Only dynamic objects generated Cannot visually edit generated code Mix-and-match allowed within generated
container Deployment
– Client footprint Dynamics Web not supported
Things to note
21 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Running Generated CodelaunchContainer
( “CustWin”, …)
getMappedFilename
( “CustWin”, …)
Abstract UI Definitions
RUN c:/Work/Clc/Gen/CustWin.rRUN rydyncontw.w
FoundCustWin.r
?
No
Yes
22 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Why Generate?
Faster object startup– Reduced network traffic– Optimized 4GL
Deployment ease & flexibility – Easier to deploy code than data– Impact analysis – Simplifies patch fixes (hotfixes)
The Benefits
23 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Benchmark Window 1
24 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Why Generate?
1.536
1.954
2.116
1.3281.216
0.664
0.50.70.91.11.31.51.71.92.12.3
2.1A03 Dynamic 2.1B Dynamic 2.1B Generated
Ren
der
Tim
e (s
ecs)
Benchmark 1 Benchmark 2
Sample Lab Results over WAN 2.1B performance
improvement
2.1A03 Dynamic to 2.1B Generated:
up to 50% Faster
25 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Agenda Slide
Designing for runtime performance
Deploying repository data as code
Application data caching
26 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Dynamics at Runtime
Manager Data
DynamicsRepository
Abstract UI Definitions
App Data
App DB
4GL GUIRuntime Client
4GL GUIRuntime Client
Rendering Engine
Data Object
Overview
27 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Dynamics at Runtime
4GL GUIRuntime Client
4GL GUIRuntime Client
Application data and caching
Cache Manager
Data Object App Data
App DB
28 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Cache Types
Session– Valid for length of client session
Persistent– Stored on client between sessions
– Valid until explicitly removed Defined duration (timed)
– Valid for specified time
– Data flushed from cache when stale
What kind of caches are available?
29 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Refreshing The Cache
Cache refreshed in various ways– Restart of session
– Explicit user action
– Expiry of defined duration How quickly data becomes stale
determines frequency of cache refresh Refreshing a server cache usually
requires agent bounce
30 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
What Data Do I Cache?
ReadFrequency
NeverContinually
Update Frequency
Very low Very high
Data Volume
InfinityZero
Items
Orders
SalesRep
31 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Determining Cache Type
Update frequency determines type Shorter update frequency = longer lasting
cache– Persistent and session caches used by
abstract UI and manager data
– Defined duration caches useful for application data
Truly static data “cached” in code– E.g. Months in year, days in week
What kind of cache do I use?
32 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Application Data Caching
Implemented in SmartDataObject– Cached data based on rowObject,
NOT database tables
– Data retrieval transparent to client
– Controlled via properties Cache manager manages cache
– Cache populated on demand– getManagerHandle(“CacheManager”)
Existing SmartDataField caching still available– DynCombos can be converted
Defined duration caching in Dynamics
33 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Enabling Caching
Set SDO CacheDuration property to non-zero value
Gentlemen, start your caches …
Value Cache Status
0 Cache disabled
0 < n < ∞ Cache enabled, expires in specified number of seconds
? Cache enabled, no auto-expiration
34 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Enabling Sharing
Set SDO ShareData property to TRUE– Sharing disabled by default
Multiple instances of SDO use same temp-table– Multiple buffers used
Use for dynamic SmartDataObjects– Re-use running instance’s data
Share and share alike
35 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Client SDO 3Client SDO 3
CacheDuration = ?ShareData = Yes
Client SDO 2Client SDO 2
CacheDuration = ?ShareData = No
Client SDO 1Client SDO 1
CacheDuration = ?
Data Caching In ActionHow the Cache Works
Cache ManagerCache Manager
DBTemp-Table
Temp-Table
Temp-TableTemp-Table Server SDOServer SDO
Temp-Table
Temp-Table
TT Buffer
36 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Sharing And Caching
If SDO is shared and not cached– Last instance destroyed deletes data
– Data only around as long as needed
– Get performance benefit by re-using already-running SDO data
– Data only as stale as running SDO
– Don’t have data persistence on client
Playing nicely together
37 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Sharing And Caching
If SDO is cached and not shared– Only really valid for static SDO
– Each instance has own copy of data Deep copy of data from cache to SDO
– Data persists on client for defined duration Respects CacheDuration property
If SDO is shared and cached– Get benefit of no deep copy
– Get benefit of data persistence on client
Playing nicely together
38 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Usage Notes
You decide what to cache and/or share Works in distributed environment only
– No support for database-aware session Batching not supported
– Doesn’t usually qualify for caching Only parent SDO cached
– Cache key depends on SDO query Caching of updates
– Cached data refreshed via server
39 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
In Summary
Continually design for n-tier performance Leverage the static code generator
– Maximize runtime performance
– Simplify hot-fix deployment Fetch data as infrequently as possible
– Faster local access for frequently referenced data
– Relieves network traffic Manage your cache carefully!
40 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Questions?
41 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Thank you for your time!
42 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation
Dynamics Performance Improvements: Application Data Sharing and Generated Objects
The following Progress courses cover related subject matter:
Progress Dynamics Application Development 1 Progress Dynamics Application Development 2
For more information, please visit:
http://www.progress.com/services/education/
na_education/curriculum_maps/index.ssp
43 DEV-03 : Progress Dynamics® Performance Improvements© 2005 Progress Software Corporation