+ All Categories
Home > Documents > FileIO Connector UserGuide

FileIO Connector UserGuide

Date post: 17-Jul-2016
Category:
Upload: francis-t
View: 248 times
Download: 7 times
Share this document with a friend
Description:
User Guide
18
Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com © 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners. File-IO
Transcript
Page 1: FileIO Connector UserGuide

Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com

© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.

File-IO

Page 2: FileIO Connector UserGuide

Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com

© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.

Contents

File-IO ................................................................................................................................... 1

1. Introduction.................................................................................................................... 3

2. Operation Matrix ............................................................................................................ 4

3. Pre-requisites for Installing the File-IO Connector Plug-in ................................................ 5

4. Enabling the File-IO Connector SDK ................................................................................. 5

5. Assumptions & Considerations ........................................................................................ 5

6. Creating a FileIO Connection in Informatica Cloud ........................................................... 5

7. Security .......................................................................................................................... 7

8. Lookup file for Salesforce Attachment parent object ID ................................................... 7

9. File-IO as Source ............................................................................................................. 8

10. File-IO as Target ........................................................................................................ 11

11. Handling Duplicate Files ............................................................................................ 15

12. Handling Large Files .................................................................................................. 16

13. Known future enhancements / Current issues ............................................................ 18

14. Recommendations..................................................................................................... 18

15. Troubleshooting ........................................................................................................ 18

Page 3: FileIO Connector UserGuide

Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com

© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.

1. Introduction

Informatica Cloud Connector SDKs are off-cycle, off-release “add-ins” that provide connectivity to SaaS and on-premise

applications not supported natively in Informatica Cloud. Connector SDKs can be enabled anytime for your cloud org Id, and are

designed to address common use cases – including getting data in to, and out of, the supported application.

Connector SDKs does not require any separate software installation on the Informatica Cloud Secure Agent machine. The

Connector uses an internal Java application to communicate with the Secure Agent, as shown in the architecture diagram below:

The File-IO connector can be used to export Salesforce attachments to a file system and upload files from a local file

system to Salesforce. Once you have enabled the Connector for your org Id, you will create an connection in Informatica Cloud

to access the File-IO Connector.

(Although the primary use case is Salesforce Attachments, the FileIO connector will work with any Source or Target that supports binary or

base64encoded content, you can use it for moving any file)

Page 4: FileIO Connector UserGuide

Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com

© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.

2. Operation Matrix

Sl # Object Name Read Lookup Add Modify Upsert Delete Remarks

1 Folders* (virtual) X NA X NA NA NA Folders- The object names are the child folders present in the „Parent Directory‟, as mentioned in connection property.

* the source and target objects seen in the DS task will be dynamic/vary according to the implementation.

Page 5: FileIO Connector UserGuide

Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com

© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.

3. Pre-requisites for Installing the File-IO Connector Plug-in

You need salesforce application/portal Access (User credentials) for creating integration tasks for File-IO to salesforce

or vice-versa in Informatica Cloud. You need to have .infaccess file in the folder from where you need to upload attachment or

download attachment to.

4. Enabling the File-IO Connector SDK

Note that FileIO Connector SDK needs to be enabled by Support or Informatica Representative. It might take 15

minutes for the Connector SDK to get downloaded to your Secure Agent after being enabled.

5. Assumptions & Considerations

FileIO Connector was developed and tested with few assumptions:

i. The FileIO connector only supports inserts (chose the overwrite files option if you would like to update files).

ii. Deletes are not supported. (If you want to delete you can easily write post processing scripts to do so).

6. Creating a FileIO Connection in Informatica Cloud

Before you can use the File-IO Connector SDK in a data synchronization task, you must follow these steps to create a

connection in Informatica Cloud: 1. Click on the Configuration tab. 2. Click on the Connections page. 3. Click the New button.

Connection Property Description

Connection Name Name of the connection

Description Give appropriate description for the connection. (example : is this a sandbox or production connection)

Type Type of connection. Select File-IO (Cloudlabs)

Secure Agent Secure Agent - Informatica Cloud.

Parent Directory

The parent directory of the directory you would like to read/write

files from/to. This directory must contain an empty file named

„.infaccess‟

Process File Content As

The file content can be read as {binary} or {base64 encoded

strings}. Default is {base64 encoded strings}. If you choose

'base64encodedstring' then you must map the field

'FileContentAsBase64String' if you choose 'binary' then you must

map field 'FileContentAsBinary'

Overwrite Target Files If True, Overwrite files with same name. If false, creates a new file

Page 6: FileIO Connector UserGuide

Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com

© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.

Connection Property Description

after incrementing the name using a counter.

Auto Archive Source Files If True, automatically moves files from source directory to {Success

Directory} after the file is processed.

In Process Directory The Directory to use for files being processed. Defaults to {Parent

Directory}/inprocess. Only used if Auto archive is true

Success Directory The Directory to move the file after it is processed. Defaults to

{Parent Directory}/success. Only used if Auto archive is true.

Error Directory

The Directory to move the file there are errors processing it.

Defaults to {Parent Directory}/error. Only used if Auto archive is

true.

4. Click OK to save the connection.

Page 7: FileIO Connector UserGuide

Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com

© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.

7. Security The FileIO connector can only access directories that have an empty file named „.infaccess‟. If this file

does not exist the connection will fail.

Create an empty file „.infaccess‟ in the directory you want to read or write files from

1. open a console prompt

2. cd to parent directory from which you wish to read or write from

3. type the following command and press enter

windows:

linux:

8. Lookup file for Salesforce Attachment parent object ID

When we upload a file to Salesforce Attachment object we need to know what parent object ID to associate the file with. As

a best practice, keep this filename and parent ID relationship information in a csv file. For example I created a csv file

„SF_ATTACHMENT.csv‟ with following structure:

Page 8: FileIO Connector UserGuide

Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com

© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.

9. File-IO as Source

The following steps gives you use case on File-IO as source,

i. Click on Application > Data Synchronization

ii. Click on the New Button

iii. In Step 1 of the Data Synchronization Wizard click Enter Name and select operation.

iv. In Step 2 of the data Synchronization wizard Choose FileIO Connection you created earlier and the Source folder you

would like to read the files from (The connector will automatically include files from all sub folders, you can filter files in

tab 4 of the wizard)

Page 9: FileIO Connector UserGuide

Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com

© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.

v. In Step 3 of the Wizard choose the target connection and object. In our sfdc attachments use case we are going to

choose a sfdc connection and attachments object.

vi. OPTIONAL: In Step 4 of the wizard we can optionally filter the files based on „FileName‟ and File timestamp. In the

Sample below we are looking for csv files and doing an incremental load based on timestamp. For filename filters do

not use wildcards. The connector expects a string that can be used in a regular expression (regex).

vii. In Step 5 of the wizard map the fields as below. You must map the Fields Body, File Name and

Parent ID.

Page 10: FileIO Connector UserGuide

Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com

© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.

viii. Lookup the value of „Parent ID‟ from the csv file you created earlier (refer to section 8); for testing you can hard code it

to a case object id.

ix. In Step 6 of the wizard. Click save and run.

x. The task will kick off and show in the activity Monitor.

Page 11: FileIO Connector UserGuide

Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com

© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.

xi. Once the task is successful it will show as completed with 0 error in the activity log.

xii. Log into Saleforce to view the attachments

xiii. NA

10. File-IO as Target

i. Click on Application > Data Synchronization.

Page 12: FileIO Connector UserGuide

Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com

© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.

ii. Click on the New Button.

iii. In Step 1 of the wizard Enter task name and select insert as the operation.

iv. In Step 2 of the wizard choose a Salesforce connection and Attachment as the source object.

v. In Step 3 of the wizard choose FileIO connection and the directory you want to write the file to.

Page 13: FileIO Connector UserGuide

Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com

© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.

vi. In Step 4 define any filters on the SFDC source (optional).

vii. In Step 5 Map the fields as below

FileDirectory is optional. By Default the file will be written to the directory that was chosen as the target object in Step

3 of the wizard. If you wish, you can override the target directory by mapping a valid directory name to the field

FileDirectory.

FileChecksum is optional. If you know the MD5 checksum of the File you want to write you can map that to the

FileChecksum field. This will help verify the contents of the file have not been modified.

viii. In Step 6 of the wizard. Click save and run.

ix. The task will kick off and show in the activity Monitor.

Page 14: FileIO Connector UserGuide

Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com

© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.

x. Once the task is successful it will show as completed with 0 error in the activity log.

xi. Open the target folder to check all files have been downloaded successfully.

xii. NA

Page 15: FileIO Connector UserGuide

Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com

© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.

11. Handling Duplicate Files

When the connector encounters a file which already exists in the target directory and 'overwrite files' flag is set to false, then

connector will automatically prefix a number to the file name and save it. For example if a file name is wrapper.txt then the

duplicate file name will be 001_wrapper.txt, 002_wrapper.txt etc.

It is important that you do not name your files with 2 leading zeros ('00') followed by underscore for example your

file name should not be 001_filename.ext, however 1_filename.ext should be fine.

You can also create unique filename using an expression to prefix a unique Id or timestamp to the file name.

Page 16: FileIO Connector UserGuide

Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com

© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.

12. Handling Large Files By default the connector can handle files up to 5MB. To handle larger files complete the following steps:

1. Edit the file „C:\Program Files (x86)\Informatica Cloud Secure Agent\main\tomcat\ saas-infaagentapp.bat‟. Change the

value of INFA_MEMORY JVM settings (-Xms -Xmx) to value suitable for your environment. Add the system property

-DfileIO.maxFileSize=xyz where xyz is the size of the largest file you wish to handle

2. Edit the file „C:\Program Files (x86)\Informatica Cloud Secure Agent\main\bin\rdtm\ pmrdtm.cfg‟. Edit the file

„C:\Program Files (x86)\Informatica Cloud Secure Agent\main\bin\rdtm\ pmrdtm.cfg‟. Change the value of

JVMOPtion1 to value of the JVM max memory setting.

3. Restart the Informatica Agent.

Page 17: FileIO Connector UserGuide

Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com

© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.

4. Edit existing task in Informatica cloud, Step through the wizard untill you reach step 5 (Field Mapping) (do not directly

go to step 5 as the metdata will not be refreshed).

5. Click on „Refresh Fields‟ button.

6. Verify the Precision for the Field „FileContentAsBase64String‟ field reflects the new size set above.

7. Click 'Edit Types..' button.

Page 18: FileIO Connector UserGuide

Informatica Worldwide Headquarters, 100 Cardinal Way, Redwood City, California 94063, USA phone: toll-free in the US: 1.800.653.3871 www.InformaticaCloud.com

© 2011 Informatica Corporation. All rights reserved. Informatica and the Informatica logo are trademarks or registered trademarks of Informatica Corporation in the United States and in jurisdictions throughout the world. All other company and product names may be trade names or trademarks of their respective owners.

8. Change the precision of the Salesforce field „Body‟.

9. Click OK

10. Save the task and test with large files.

13. Known future enhancements / Current issues NA.

14. Recommendations

NA

15. Troubleshooting NA


Recommended