USEMARCON

E-EndUser 2003

5.9.2003 in Helsinki

Arne Hedman / Helsinki University Library

Thomas Roos / CSC

Agenda

– What is USEMARCON?– Versions available– What must be defined before use of

USEMARCON?– Where to get help?– Our conversion of 30 million records– Two cases

What is USEMARCON

• MARC-converter for any MARC data in exchange format (ISO 2709)– user control– free of charge

• USEMARCON is developed in late 1990’s by partners of the consortium from Holland, UK, Germany and France, supported by EU’s Telematics Applications Programme

Versions available

• Original Windows-version by Jouve S.I– MS Win 3.1x, MS Win 95 and Solaris 2.4– Requires XVT C++ toolkit to compile– GUI only– Memory leaks– ftp://ftp.bl.uk/pub/nbs/ec/usemarcon

cont…

Versions available

• Usemarcon Plus by Crossnet Systems Ltd– Separate MS Win and Linux/Unix versions

– API calls possible

– Tested by us on Win 2000, Solaris8 and Debian Linux

– Builds with VC++ or GCC 2.8, but not GCC 3.X

– Used by EISI as a platform for on-the-fly conversions

http://www.bl.uk/services/bibliographic/usemarcon.html

cont…

Versions available

• Usemarcon Plus by ATP Library Systems (ere@atp.fi)– Merged Windows and Linux versions– Elimination of all known memory leaks– Command line enhancements– code restructuring for better readability– build tested with

• MS Visual C++ 6 on Windows XP• GCC 3.2 on Red Hat Linux 8.0• GCC 3.3 on Debian unstable

– Will be available from BL shortly

What must be defined before use of

USEMARCON?• All files and tables are text files very easy to

handle• Initialization File (.ini)

– names of tables and input/output files

• Look up tables (.tbl)– codes to be converted

• Format checking tables (.chk)– List of fields and subfields allowed and their occurences

cont…

What must be defined before use of USEMARCON?

• one table for input data and one for output data

• Character Set Translation Table (.trs)– only non-standard characters to be defined– conventional ASCII characters specified in a– default character set table

• Configuration file (.cnf)– a list of fields without indicators (001-009)

What must be defined before use of

USEMARCON?

• Rule file (.rul)– conversion rule for all the tags– a tag without a rule will be dropped out

Where to get help?

• Guide to USEMARCON in web– http://www.bl.uk/services/bibliographic/

guidetousemarcon.pdf

• British library email:usemarcon@bl.uk

Our conversion of 30 million records

• Elements in records can be handled in many ways:– maintain– change– delete– add

Tools for change are numerous:

•Boolean instructions(OR, AND, NOT)•Conditional instructions (if… then)•Loop instructions•Memory instructions•String functions•Number functions

Some experiences

• Editing text files technically easy

• Quick testing possible, because changing rules in text files with the help of error files is fast

• In tricky conversions the manual does not help you very much. There has been bugs in manual, too

cont…

Some experiences

• Repeatable tags with different subfield structure are problematic

• Changing fixed fields can cause problems,if many conditional instructions exist

• Do not convert too big files. It’s better to link the files to be converted.

Weaknesses in USEMARCON

• Not handy in changing the contents (tag/subfield code/constant/data) in one tag

• Character set conversion without other definitions is not possible

• No tool to choose and pick certain material from a file for conversion

cont…

Weaknesses in USEMARCON

• Not possible to merge two files

• If USEMARCON goes down becauseof an unqualified record, the corruptedrecord is not easy to find

To make conversions effectively, some tools

are needed• A program to browse input and output data

for comparison

• A hexeditor to remove bad records

Case 1VTLS to Voyager

• Conversion of cyrillic data from VTLS (FINMARC) to Voyager (MARC21-Fin)– extracted 2 versions of the databases to

ISO2709• cyrillic and translitterated

cont…

Case 1 VTLS to Voyager

Case 1 VTLS to Voyager

•10 conversions later

•7 custom-made programs and 3 usemarcon conversions

Case 1 VTLS to Voyager

Case 2Usemarcon in Voyager

• Use of USEMARCON in Voyager– Convert during import

• with Pbulkimport

– Convert during export• with Pmarcexport

– Convert on-the-fly with the z39.50-server• /m1/voyager/lindadb/ini/z3950svr.ini

– Finmarc=/m1/voyager/usemarcon/mafi2fi.ini

Comments?