ABBYY FlexiCapture for Invoices: Vendor Optimization at #ABBYYSummit17

ABBYY TechnologySummit2017

© ABBYY Confidential

ABBYY NAHQ, 2017

FlexiCapture Technical Track

FCI – Working with Vendor Records

Presented by:

Todd Pratt


Agenda:1) Why are we having a whole session on Vendor Records?

2) Data Sets

3) Vendor Verification

4) Best Practices and Common Issues

© ABBYY Confidential 3

Why so much fuss?


• Vendor identification is an ‘issue’ with nearly every implementation of AP automation.

• FCI relies on VendorID to match an invoice to a training batch!!

• Vendor data is often outside of our control, and not optimized for OCR.

• Vendors change all the time.

Data Sets - Vendors

A data set is a local copy of table from an external database which stores information about vendors or business units.

This information generally comes from the customer’s ERP system. Retrieving the information from the ERP system to import into the FCI data set is the responsibility of the VAR and Customer working together, and requires different methods depending on the ERP system.


Data Sets – Business Units

If the BusinessUnitID field is specified in the Vendor data set, the Vendor Id field is not treated as unique and is deemed to be dependent on the BusinessUnitId field. The idea is that each business unit (BusinessUnitId) of a company has its own unique list of vendors (Id). The pair BusinessUnitId + Id is the unique key of the vendor's record in the Data Set.

This means that during Verification, when you select a BU, the subsequent vendor list will be filtered for only those vendors who are members of that BU. Conversely, if you select a vendor, the BU for that vendor will be filled out automatically.


Data Sets – Variants

Storing multiple values is often necessary because parameters such as company name, street, etc. may vary from invoice to invoice (e.g. we may have "My Company Name" and "MCN Ltd."), but in order to detect the company reliably, the text in the Data Set must closely match the text captured from an invoice image.

Values for multiple-value columns may be imported from an external database (e.g. an ERP vendor table may contain multiple names for each company – Name/ShortName), or they may be added by the user during data capture (e.g. an operator may add to the Data Set a commonly used variant of a company name that is not known to the external database).


Data Sets – Variants

For one logical field of a record, say, Name, multiple columns are created in the Data Set, which store all the possible values of the logical field. Thus, Name may be termed a "complex column.“

When using column variants (i.e. - Name1, Name2, Name3,…) the software does not give preference to primary columns. Instead it takes everything into account at once. All column variants contribute equally.


Data Sets – Updating

Updating Data Sets when batches are opened

To enable automatic updates, open the properties of the Data Set, select the Update interval and specify the interval between updates. An update will be initiated when a certain amount of time has elapsed since the last update and the batch is opened on any station of the Standalone version, or on the Project Setup Station in the Distributed version of FlexiCapture.



Scheduling Data Set updates

In ABBYY FlexiCapture Distributed, Data Set updates can be initiated from the command line on the Processing Server or on any full-featured station (with the exception of remote stations and web stations). This is only possible when the Processing Server or station has access to the FlexiBr.exe or FlexiBrSvc.exe file. Use the following command to update Data Sets:

update dataset <project path> [<document definition name>] [<Data Set name>]

Only Administrators and the user of the Project Setup Station can use this command in ABBYY FlexiCapture. Example of a command that can be executed on the Processing Server to update the Vendors Data Set:

FlexiBRSvc.exe please update dataset "http://myAppServer/ProjectId" "Invoice" "Vendors"

This command updates the Data Set on the Processing Server. However, full-featured stations will keep using their local copy of the Data Set until they are restarted. Processing stations that are Windows services will update their local copies of Data Sets after five minutes of inactivity starting from the moment all processes that handle tasks from FlexiEx.exe are terminated. You can use the Windows Scheduler to schedule Data Set updates and restart processing stations so that their local copies of Data Sets are updated.



ABBYY FlexiCapture lets operators edit Data Sets during processing. In particular, users can add new vendor and business unit data and modify existing data. These changes are not carried over to the external database.

What happens when you update a data set:


Existing records * All changes made by users will be retained.

* All changes made to the external database will be carried over to the Data Set.

New records * New records created by operators will be deleted!!

* New records from the external database will be carried over to the Data Set.

Deleted records

* Data Set records that have been flagged as deleted will not be restored.

* Records that have been deleted in the external database will be deleted from the Data Set.

Vendor Recognition – The Results


So, what actually happens behind the scenes when we match a vendor?

1) FCI looks first for “unique keys” of a vendor like VATID, IBAN (which is not the case for US, but can work for Canada) by regex. If found, FCI searches for records in the data set with those values.

2) If nothing is found, FCI searches for the most relevant records in database using the whole text of the page. After returning the top 50 records from database, they are ranked by relevance to the query.

3) Then FCI compares the field values from the records to the image, taking in consideration company name and address should be close to each other, if we can find them, and how many character errors are present if we find a corresponding record on the image.

4) Finally, FCI rates the resulting hypothesis (50 records) obtained from the database and returns the best answer (1 record) and its confidence. How we rate the database records to make final decision was manually designed in FC11 and is machine learned in FC12.

Vendor Recognition – The Results


When there are ambiguous or close results from the database lookup, how does the software choose the vendor? (i.e. – 123 Main St. vs. 133 Main St.)

• All fields of record are taken in consideration, the exact weight of each field and of each error on character level is machine learned to produce the best quality.

The following document parameters are produced for vendor and business unit confidence levels:

• fc_Predefined:InvoiceIsVendorSuspicious

• fc_Predefined:InvoiceIsBusinessUnitSuspicious

These values are produced by the machine learning that rates the database hypothesis. In client engagements these values have been used with business rules to notify an operator of a possible discrepancy.

Verification

If a vendor or a business unit was not determined automatically when an invoice was recognized, you can select one from the database. There are two ways to do this:

Begin typing the desired value in one of the fields of the group. A list of suggestions will be displayed. Any records that match the text you typed will be displayed in bold. When you see the desired record, select it from the list by clicking it or choose it using the arrow keys (press the Enter key to confirm your choice). Fields from the record will be inserted into the corresponding fields of the group on the form. If you cannot find the desired record in the list of suggestions, click the Vendor (Business Unit) is not found or press Ctrl+Del to clear all of the fields in the group.


Verification

Click the Find... button in the form, the Lookup for Vendor (Business Unit) dialog box will open. This dialog box can be used to search for entries in the database. Begin typing in the search box. Suggestions will be displayed in the list on the right, and any matches will be displayed in boldface. All of the fields of the selected record will be displayed on the left. Select the desired record using the mouse or arrow keys and click the Select button. The fields from the record will be inserted into the fields of the group on the form. If you cannot find the desired entry in the dialog box, click the Reset Data button to clear all the fields in the group.


Verification – adding Variants

Users can adding a variant to the data set record by clicking the ‘edit’ button and selecting the location of the text on the image.

Variants help disambiguate vendors by introducing additional data for the recognition to use for matching.


Best Practices

Vendor data clean up – make it part of the implementation (even if its just a chore for the vendor).

– Clean-up is an iterative process

– Help them by building a report for all vendors that were not matched automatically.

– How do I disambiguate when the data is ‘dirty’? Clean up the ERP!

– How do I deal with BillTo or ShipTo addresses that are in the vendor master file?


Best Practices


Duplicate Checks and when are they appropriate – business rule?

– Dup checks are great for capture.

– Requires connection to the ERP journal

– (inv number, total, vendor + optional inv date)

– Dup checks happen in several places, including workflow and the ERP.

Best Practices


Adding a new vendors

– Don’t do it in FlexiCapture – make and exception and deal with it in the ERP / credentialing process.

– Merging manually created vendors in FlexiCapture with an ERP update deletes manually created records.

– Send it to workflow!

Questions/Comments


Date post:	21-Jan-2018
Category:	Technology
Upload:	abbyy-usa
View:	313 times
Download:	10 times