COE Data Replication Using Data Extractor BC DELMIA Apriso 2019 Technical Guide ®2018 Dassault Systèmes. Apriso, 3DEXPERIENCE, the Compass logo and the 3DS logo, CATIA, SOLIDWORKS, ENOVIA, DELMIA, SIMULIA, GEOVIA, EXALEAD, 3D VIA, BIOVIA, NETVIBES, and 3DXCITE are commercial trademarks or registered trademarks of Dassault Systèmes or its subsidiaries in the U.S. and/or other countries. All other trademarks are owned by their respective owners. Use of any Dassault Systèmes or its subsidiaries trademarks is subject to their express written approval.
2 Configuration 52.1 Configuration Overview 52.2 Extract BC Format Configuration 62.3 Client Configration 142.4 Business Integrator Configuration 152.4.1 Removing Mappings 152.4.2 Adding a Synchronization Queue for the Data Extractor 162.4.3 Adding the Destination Routing and Queue 17
2.5 Central Configuration 183 Administration 194 Usage 204.1 Using the Extract Business Component 204.2 Configuring a Job to Run in Order 214.2.1 KPI Master Data Example 22
5 References 24
FiguresFigure 1 Typical COE topography 4Figure 2 Breakdown of how COE works 5Figure 3 Client grid screen 14Figure 4 Client detail screen 14Figure 5 Mapping Configurations to be removed 16Figure 6 Adding the routing 17Figure 7 Adding a queue for the Data Extract 18Figure 8 Job Scheduler – job properties 22Figure 9 Adding a job schedule 22Figure 10 Specifying a queue to perform the jobs 23Figure 11 Adding the KPI grid job action 23Figure 12 KPI element job actions 23
COE Data Replication Using Data Extractor BC | DELMIA Apriso 2019 Technical 2
1 Introduction1.1 PurposeThe purpose of this document is to gather all of the information that users require to understand and use COE Data Replication.
1.2 ScopeThe scope of this document describes how data is to be “replicated” to other systems using DELMIA Apriso Business Integrator, Job Scheduler, and a Business Component called DataExtractor (the “extract” method).
Also included:
Setting up mappingsSetting up extract formats and setting up default valuesWhat clients are and how they are usedHow to migrate any data within the DELMIA Apriso system to any other DELMIA Apriso systemSending data modified since the last migrationHow to do a full migration of all data, regardless of when it was last modifiedSending data to one or more destination DELMIA Apriso systemsXML transformation on a COE or client server (configurable)
1.3 Intended AudienceThis document is to be used by anyone needing to send data from one server to one or more other destination servers.
1.4 VocabularyBusiness Component (BC) – Business Component is a piece of business logic. Logic is
binary code (C# Class with appropriate attributes) contained in DLLs. BCs are invoked by both standard DELMIA Apriso modules and a user’s Processes/Operations. Detailed information on all of the BCs is available in the Business Components Documentation.
1.5 Extract/COE OverviewThe diagram below presents a typical COE topography where data are replicated between plants of an organization. One of the plants is designated as the main plant where master data are maintained and the changes are sent to child plants. Transactional data performed at the child plants are sent to the main plant.
COE Data Replication Using Data Extractor BC | DELMIA Apriso 2019 Technical 3
Figure 1 Typical COE topography
COE Data Replication Using Data Extractor BC | DELMIA Apriso 2019 Technical 4
2 ConfigurationThis chapter describes how to set up the required configuration for COE Data Replication using the DataExtractor Business Component.
2.1 Configuration Overview
Figure 2 Breakdown of how COE works
COE Data Replication Using Data Extractor BC | DELMIA Apriso 2019 Technical 5
As illustrated above (Figure 2 Breakdown of how COE works), the information is triggered by the “extract” method of the DataExtractor Business Component. The method has various prompts that allow the user to send data from any table based on a single query. This formulates the query result information into a correctly formatted XML and creates a job to send and possibly translate the data. This can be to one, many, or all clients.
Once sent, a new job is created on the host system (one for each client) wherein the XML created is then used to insert/update data.
For detailed information on the DataExtractor Business Component, refer to the Business Components Documentation.
The following are prerequisites for sending data to other systems:
1. Business Integrator
The Message Processor Web service (URL) needs to be configured from the Client Configuration Maintenance and Monitoring screen.
XSLT mappings are required to transform the Data Extractor XML format into a DELMIA Apriso XML format (from which Business Integrator can persist the data into the DELMIA Apriso database). You can configure the XSLT transformation to be performed on the source DELMIA Apriso server from which the data is extracted or on the destination Apriso server(s) to which the data is imported. Starting with FlexNet 9.3, the KPI Data Extract mappings are included in the standard installation in DELMIA Apriso Schema Repository. The Mapping Configuration for these is also included.
Please be aware that mappings need to be removed from the Mapping Configuration file in the source DELMIA Apriso system for this KPI Data Extract scenario.
The following chapter describes the configuration process in detail.
2.2 Extract BC Format ConfigurationIn order to extract data from child plants and send it to main plant (COE), configure Extract Business Component method (FlexNet.BusinessFascade.Utility.DataExtractor BC class) in Process Builder at the child plant. There are six predefined extract formats for KPI. It is important to understand that the queries return data from the point where the query was recently run. It is to ensure no duplicate data are sent.
Note that WHERE clause “LASTUPDATEON > @LastRunTime”.
COE Data Replication Using Data Extractor BC | DELMIA Apriso 2019 Technical 6
KPI_GRID
This format configures data extract for KPI_GRID table. The following are Input parameters to the Extract Business Component method:
ExtractionName Input: KPI_GRID
KPI grids will not be exported if they do not have a description.
Query Input:
COE Data Replication Using Data Extractor BC | DELMIA Apriso 2019 Technical 7
SELECTKG.Name, KG.ClientName, KG.FUID, KG.GridActive, KG.RefreshRate AS RefreshRateTicks, KG.SequenceNo, KG.ReferenceID, KG.Active, TEXT_TRANSLATION.Medium AS GridMedium , TEXT_TRANSLATION_K1.Medium AS K1_Medium, TEXT_TRANSLATION_K2.Medium AS K2_Medium, TEXT_TRANSLATION_K3.Medium AS K3_Medium, TEXT_TRANSLATION_K4.Medium AS K4_Medium, TEXT_TRANSLATION_K5.Medium AS K5_Medium, TEXT_TRANSLATION_K6.Medium AS K6_Medium, TEXT_TRANSLATION_K7.Medium AS K7_Medium, TEXT_TRANSLATION_K8.Medium AS K8_Medium, TEXT_TRANSLATION_K9.Medium AS K9_Medium, TEXT_TRANSLATION_K10.Medium AS K10_Medium, TEXT_TRANSLATION.LanguageID AS LanguageID FROM KPI_GRID KG INNER JOIN TEXT_TRANSLATION TEXT_TRANSLATION ON KG.TextID = TEXT_TRANSLATION.TextID LEFT OUTER JOIN TEXT_TRANSLATION TEXT_TRANSLATION_K1 ON KG.Key1TextID = TEXT_TRANSLATION_K1.TextID ANDTEXT_TRANSLATION_K1.LanguageID = TEXT_TRANSLATION.LanguageID LEFT OUTER JOIN TEXT_TRANSLATION TEXT_TRANSLATION_K2 ON KG.Key2TextID = TEXT_TRANSLATION_K2.TextID AND TEXT_TRANSLATION_K2.LanguageID = TEXT_TRANSLATION.LanguageID LEFT OUTER JOIN TEXT_TRANSLATION TEXT_TRANSLATION_K3 ON KG.Key3TextID = TEXT_TRANSLATION_K3.TextID AND TEXT_TRANSLATION_K3.LanguageID = TEXT_TRANSLATION.LanguageID LEFT OUTER JOIN TEXT_TRANSLATION TEXT_TRANSLATION_K4 ON KG.Key4TextID = TEXT_TRANSLATION_K4.TextID AND TEXT_TRANSLATION_K4.LanguageID = TEXT_TRANSLATION.LanguageID LEFT OUTER JOIN TEXT_TRANSLATION TEXT_TRANSLATION_K5 ON KG.Key5TextID = TEXT_TRANSLATION_K5.TextID AND TEXT_TRANSLATION_K5.LanguageID = TEXT_TRANSLATION.LanguageID LEFT OUTER JOIN TEXT_TRANSLATION TEXT_TRANSLATION_K6 ON KG.Key6TextID = TEXT_TRANSLATION_K6.TextID AND TEXT_TRANSLATION_K6.LanguageID = TEXT_TRANSLATION.LanguageID LEFT OUTER JOIN TEXT_TRANSLATION TEXT_TRANSLATION_K7 ON KG.Key7TextID = TEXT_TRANSLATION_K7.TextID AND TEXT_TRANSLATION_K7.LanguageID = TEXT_TRANSLATION.LanguageID LEFT OUTER JOIN TEXT_TRANSLATION TEXT_TRANSLATION_K8 ON KG.Key8TextID = TEXT_TRANSLATION_K8.TextID AND TEXT_TRANSLATION_K8.LanguageID = TEXT_TRANSLATION.LanguageID LEFT OUTER JOIN TEXT_TRANSLATION TEXT_TRANSLATION_K9 ON KG.Key9TextID = TEXT_TRANSLATION_K9.TextID AND
COE Data Replication Using Data Extractor BC | DELMIA Apriso 2019 Technical 8
TEXT_TRANSLATION_K9.LanguageID = TEXT_TRANSLATION.LanguageID LEFT OUTER JOIN TEXT_TRANSLATION TEXT_TRANSLATION_K10 ON KG.Key10TextID = TEXT_TRANSLATION_K10.TextID AND TEXT_TRANSLATION_K10.LanguageID = TEXT_TRANSLATION.LanguageID WHERE KG.LASTUPDATEON > @LastRunTime
KPI
This format configures the data extract for KPI and KPI_TOLERANCE tables.
KPIs will not be exported if they do not have a default tolerance applied.
ExtractionName Input: KPIQuery Input:
COE Data Replication Using Data Extractor BC | DELMIA Apriso 2019 Technical 9
SELECT Name AS Name, Expression AS Expression, Context_ AS Context, Category_ AS Category, BaseUom AS BaseUom, DefaultStrategy AS DefaultStrategy, ContextSQLStatement AS ContextSQLStatement, TimeFromTerm AS TimeFromTerm, TimeToTerm AS TimeToTerm, FUID AS FUID, ClientName AS ClientName, DiagnosisReportID AS DiagnosisReportID, SequenceNo AS SequenceNo, k.ReferenceID AS KpiReferenceID, tt.Short AS KpiShort, tt.Medium AS KpiMedium, tt.Extended AS KpiExtended, tt.LanguageID as LanguageID, kt.LowerSpecLimit, kt.LowerControlLimit, kt.TargetValue, kt.UpperControlLimit, kt.UpperSpecLimit, kt.UOMCOde, kt.NoCustomOverride FROM KPI k INNER JOIN KPI_TOLERANCE kt ON k.Name = kt.KpiName AND EmployeeID is NULL and kt.Key1 is NULL AND kt.Key2 is NULL AND kt.Key3 is NULL AND kt.Key4 is NULL AND kt.Key5 is NULL AND kt.Key6 is NULL AND kt.Key7 is NULL AND kt.Key8 is NULL AND kt.Key9 is NULL AND kt.Key10 is NULL LEFT OUTER JOIN TEXT_TRANSLATION tt ON k.TextID = tt.TextID
KPI_TERM
This format configures data extract for the KPI_TERM table.
ExtractionName Input: KPI_TERMQuery Input:
COE Data Replication Using Data Extractor BC | DELMIA Apriso 2019 Technical 10
SELECT kt.Context AS Context, kt.Name AS Name, kt.Type AS Type, kt.Assembly AS Assembly, kt.Class_ AS Class, kt.Expression AS Expression, kt.MIPoint AS MIPoint, kt.ConnectionString AS ConnectionString, kt.StoredProcedure AS StoredProcedure, kt.GroupNamespace AS GroupNamespace, kt.BCMethodID AS BCMethodID, kt.FUID AS FUID, kt.ClientName AS ClientName, kt.ReferenceID AS ReferenceID, kt.Active AS Active, TEXT_TRANSLATION.Short AS Short, TEXT_TRANSLATION.Medium AS Medium, TEXT_TRANSLATION.Extended AS Extended, TEXT_TRANSLATION.LanguageID as LanguageID, bcm.FUID as BusinessComponentMethodFUID FROM KPI_TERM kt LEFT JOIN TEXT_TRANSLATION TEXT_TRANSLATION ON kt.TextID = TEXT_TRANSLATION.TextID AND LanguageID = 1033 LEFT JOIN BUSINESS_COMPONENT_METHOD bcm ON kt.BCMethodID = bcm.ID WHERE kt.LASTUPDATEON > @LastRunTime
KPI_VALUE
This format configures data extract for the KPI_VALUE table. It inserts or updates the record in the host system.
ExtractionName Input: KPI_VALUEQuery Input:
COE Data Replication Using Data Extractor BC | DELMIA Apriso 2019 Technical 11
SELECT A.ID as KPIValueID, C.ID, C.KPIValueHistoryID, C.Context_, C.Name, C.ParentTermHistoryID, C.Value_, C.UomCode, C.ReferenceID, C.Active, A.ClientName FROM KPI_VALUE A INNER JOIN KPI_VALUE_HISTORY B ON B.KPIValueID = A.ID INNER JOIN KPI_TERM_HISTORY C ON C.KpiValueHistoryID = B.ID WHERE C.LASTUPDATEON > @LastRunTime,
COE Data Replication Using Data Extractor BC | DELMIA Apriso 2019 Technical 13
2.3 Client Configration 1. On the Client Configuration screen (Figure 3 Client grid screen), add a new record by
clicking (Add). You can also use standard filter functionality to single out and edit existing records.
Clients are used to determine where the information is being sent.
Figure 3 Client grid screen
Figure 4 Client detail screen
2. On the client properties screen (Figure 4 Client detail screen), enter the desired data for the new client. The table below describes fields available.
Field DescriptionName The client name. The name should not be generic (e.g., Plant), but it should still
be easily recognizable.Master Flag
This is the client that is the owner of the system. If this flag is not set, then these clients are other systems.
Type The client can be of the Production, Development, or Test type.Description The description of the client.Message This is the location of the message processor. This is used in the Data Extract
COE Data Replication Using Data Extractor BC | DELMIA Apriso 2019 Technical 14
Processor URL
component to send information to other systems.
Export Data Allowed Flag
A master client can export data to other systems.
This must be checked to import data to a site.
CIM Tab ID The ID number of the specific M&M screen which will be displayed from an external DELMIA Apriso server (defined in the Portal URL field). For example, “584” represents the Resources screen.
Portal URL The URL address of the Apriso Classic Portal on a different server. For example: http://<server name>/Apriso/Portal/TRANSACTIONPAGE.ASPX.
2.4 Business Integrator ConfigurationThe below scenario configures KPI mappings to be performed on the destination Apriso server(s).
After performing a standard installation for both the source and destination Apriso servers, do the following:
1. Remove any mappings from the source system that are extracted and being sent out. 2. Add a Synchronization Queue for the Data Extractor component to ensure that the
information is sent out in order. 3. Add a Synchronization Queue for receiving destination(s) to ensure that the data is
processed in the order that it is received (because the KPI Data Extractor scenario consists of multiple KPI tables with dependencies).
4. Set up the extract formats. 5. Set up the clients.
The above steps are described in detail in the sections below.
2.4.1 Removing Mappings
1. On the source DELMIA Apriso server, launch DELMIA Apriso Configuration Manager (available in the DELMIA Apriso group in the Windows Start menu).
2. Switch to the “BI – Mapping Configuration” tab and check that no mappings exist when no transformation is required. This could be done either on the source or destination location.
In this example, you should check that there are no mappings for KPI_GRID, KPI, KPI_TERM, KPI_VALUE, KPI_VALUE_HISTORY, KPI_TERM_HISTORY. The entries are marked in Figure 5 Mapping Configurations to be removed.
Please be careful when doing this! Ensure that these are not used.
COE Data Replication Using Data Extractor BC | DELMIA Apriso 2019 Technical 15
Figure 5 Mapping Configurations to be removed
2.4.2 Adding a Synchronization Queue for the Data Extractor
1. On the source DELMIA Apriso server, run DELMIAApriso Configuration Manager and switch to the “BI – Routing Configurations” tab.
2. Modify the Routing Configuration to route the Data Extractor XML to “29 – FlexNetAdapter” and to use the Synchronization Queue. In this case, the Synchronization Queue name is KPISendQueue.
The outbound Routing Configuration should appear as presented in Figure 6 Adding the routing.
COE Data Replication Using Data Extractor BC | DELMIA Apriso 2019 Technical 16
Figure 6 Adding the routing
2.4.3 Adding the Destination Routing and Queue
1. On the destination DELMIA Apriso server(s), run DELMIAApriso Configuration Manager and switch to the “BI – Synchronization Queues and Job Pools for Inbound Messages” tab.
2. Configure the Synchronization Queue for the Inbound Data Extractor XML. In this case, the Synchronization Queue name is ReceivingKPIQueue (Figure 7 Adding a queue for the Data Extract).
COE Data Replication Using Data Extractor BC | DELMIA Apriso 2019 Technical 17
Figure 7 Adding a queue for the Data Extract
2.5 Central ConfigurationThe job configuration pickup time can be set using “ConfigurationLoaderSleepPeriod” key located in the “Scheduling” section of the Central Configuration file (for details, see Central Configuration Documentation). This key might need to be changed based on the importance of picking up of jobs.
COE Data Replication Using Data Extractor BC | DELMIA Apriso 2019 Technical 18
3 AdministrationOngoing administration comes down to maintaining and adjusting the configuration as described in 2 Configuration.
COE Data Replication Using Data Extractor BC | DELMIA Apriso 2019 Technical 19
4 UsageThis chapter describes how to use the Extract BC method to send the KPI data to another client and how to configure jobs to run in orders.
4.1 Using the Extract Business ComponentThe Extract Business Component has the following inputs and outputs:
Name Type Comment PropertyExtractionName(String)
string The name defining the extracted data. Required
Query(String) String The query used to extract the data. OptionalClients(String[]) String
[]The clients to whom to send the extracted data. Optional
AllClients(Boolean)
Bool Determines whether or not to send the data to all of the clients in the CLIENT table.
Count(Int) Int The number of records the extraction processed.
Sending Only Updated Data (Query @LastRunTime Substitution)
If the query contains @LastRunTime in the query, it is substituted with the EXTRACT_FORMAT.LastRunTime value. This value is updated after every single run of the Extract with that extract format. If no value is found in the EXTRACT_FORMAT table, then the UTC minimum is set, so all of the records are effectively sent.
If the Query contains @LastRunTime, it can be used in conjunction with the LastUpdateOn field in DELMIA Apriso tables. This enables finding what records have been changed and what records need to be sent. Good examples of this can be seen in the extract formats of the KPI information.
The only required input for the component is the Extract name. Like many of our components, this has certain combinations for calling the Business Component.
Sending to One Client with an Existing EXTRACT_FORMAT and Query
Input Input SpecificationQuery This is not required as it is from the EXTRACT_FORMAT table.Clients The client name. This must have the extract data flag turned on.AllClients False.
COE Data Replication Using Data Extractor BC | DELMIA Apriso 2019 Technical 20
Sending to All Clients with an Existing EXTRACT_FORMAT and Query
Input Input SpecificationQuery This is not required as it is from the EXTRACT_FORMAT table.Clients Blank.AllClients True.
Sending to All Clients with an Existing EXTRACT_FORMAT and Query and Clients
Input Input SpecificationQuery This is not required as it is from the EXTRACT_FORMAT table.Clients Blank.AllClients False.
The EXTRACT_FORMAT and the EXTRACT_FORMAT_CLIENT tables are populated this way.
Sending to a Client with an Existing EXTRACT_FORMAT Record (Override Query)
Input Input SpecificationQuery Should be the query used when retracting data.Clients The client name.AllClients False.
Sending to All Clients with an Existing EXTRACT_FORMAT Record (Override Query)
Input Input SpecificationQuery Should be the query used when retracting data.Clients Blank.AllClients True.
4.2 Configuring a Job to Run in OrderJobs are the best way of configuring items to be sent, because they can be triggered automatically and the actions can be sent in order.
COE Data Replication Using Data Extractor BC | DELMIA Apriso 2019 Technical 21
If the order of processing the data is important when sending information, then send the data in that order.
For example, because a KPI references both the KPI_TERM and KPI_GRID tables, these must be sent beforehand. To do this requires setting the jobs in that order.
4.2.1 KPI Master Data Example
Here is an example of a job that sends master KPI data (KPI_GRID, KPI_TERM, KPI):
1. Add a new job in the DELMIA Apriso Job Scheduler. Enter the job details.
Figure 8 Job Scheduler – job properties
2. Add a schedule for it to run on an interval of every minute as presented in Figure 9 Adding a job schedule.
Figure 9 Adding a job schedule
3. Make sure the job is executed in order with a queue.
COE Data Replication Using Data Extractor BC | DELMIA Apriso 2019 Technical 22
Figure 10 Specifying a queue to perform the jobs
4. Call the Extract component either as a BC (although there may be a problem with Job Scheduler) or from a Standard Operation. In this example, APR_TestCOE is the Operation used (see Figure 11 Adding the KPI grid job action).
Figure 11 Adding the KPI grid job action
5. Repeat step 4 for each of the KPI elements while making sure that the QUEUE name is the same in all of them. The result should look similar to Figure 12 KPI element job actions.
Figure 12 KPI element job actions
COE Data Replication Using Data Extractor BC | DELMIA Apriso 2019 Technical 23
All the internal documents referenced in this section are available from the DELMIA AprisoStart page, which can be accessed on your DELMIA Apriso server (<server name>/apriso/start).The newest versions of all documents are available from 3DS Support at:http://www.3ds.com/support/documentation/overview/
5 ReferencesInternal Documentation
1. Business Components Documentation
Enables viewing DELMIA Apriso Business Components (BCs) and their descriptions. The same documentation is available in the Business Components Repository in DELMIA Apriso Process Builder.
2. Central Configuration Documentation
Describes in detail all the keys of the Central Configuration (CC) file for DELMIA Apriso. Various sections group the keys for individual modules or distinct functional areas.
3DS Support Knowledge Base
If you have any additional questions or doubts not addressed in our documentation, feel free to visit the 3DS Support Knowledge Base at https://www.3ds.com/support/knowledge-base/.
COE Data Replication Using Data Extractor BC | DELMIA Apriso 2019 24
