Log Parser HelpLog Parser Log parser is a powerful, versatile tool that provides universal query...

transcript

LogParserLogparserisapowerful,versatiletoolthatprovidesuniversalqueryaccesstotext-baseddatasuchaslogfiles,XMLfilesandCSVfiles,aswellaskeydatasourcesontheWindows®operatingsystemsuchastheEventLog,theRegistry,thefilesystem,andActiveDirectory®.YoutellLogParserwhatinformationyouneedandhowyouwantitprocessed.Theresultsofyourquerycanbecustom-formattedintextbasedoutput,ortheycanbepersistedtomorespecialtytargetslikeSQL,SYSLOG,orachart.TheworldisyourdatabasewithLogParser.

Mostsoftwareisdesignedtoaccomplishalimitednumberofspecifictasks.LogParserisdifferent...thenumberofwaysitcanbeusedislimitedonlybytheneedsandimaginationoftheuser.Ifyoufindacreativewaytouseit,letusknowatwww.logparser.com!

Herearesomesamplestowhetyourappetite...

SearchforDataSearchforthelogonsofaspecificuseramongtheeventsintheWindowsEventLog:

C:\>LogParser"SELECTTimeGenerated,SourceName,EventCategoryName,MessageINTOreport.txtFROMSecurityWHEREEventID=528ANDSIDLIKE'%TESTUSER%'"-resolveSIDs:ONAndobtainresultsinatextfileformattedasdesired:

CreateReportsCreatecustom-formattedHTMLreports:

CalculateStatisticsCalculatethedistributionoftheHTTPresponsestatuscodesfromyourIISlogfiles:

C:\>LogParser"SELECTsc-status,COUNT(*)ASTimesINTOChart.gifFROM<1>GROUPBYsc-statusORDERBYTimesDESC"-chartType:PieExploded3D-chartTitle:"StatusCodes"Andproduceachartformattedasdesired:

SystemRequirementsLogParseriscompatiblewiththeWindows®2000,Windows®XPProfessional,andWindowsServerTM2003operatingsystems.

What'sNewinLogParser2.2

NewInputandOutputFormats:

XMLInputFormatReadsXMLfiles(requirestheMicrosoft®XMLParser(MSXML))

TSVInputFormatReadstab-andspace-separatedvaluestextfiles

ADSInputFormatReadsinformationfromActiveDirectoryobjects

COMInputFormatMakesitpossibletopluginuser-implementedcustomInputFormats

REGInputFormatReadsinformationfromtheWindowsRegistry

NETMONInputFormatMakesitpossibletoparseNetMon.capcapturefiles

ETWInputFormatReadsEventTracingforWindowslogfilesandlivesessions

CHARTOutputFormatCreateschartimagefiles(requiresMicrosoftOffice2000orlater)

TSVOutputFormatWritestab-andspace-separatedvaluestextfiles

SYSLOGOutputFormatSendsinformationtoaSYSLOGserverortoaSYSLOG-formattedtextfile

ImprovementstotheSQLEngine:

ExponentialperformanceimprovementinSELECTDISTINCTandGROUPBYqueries

"WITHROLLUP"functionalityintheGROUPBYclause

"DISTINCT"inaggregatefunctions(whennoGROUPBYclauseisspecified)

"PROPSUM(...)[ON<fields>]"and"PROPCOUNT(...)[ON<fields>]"aggregatefunctions

(thesefunctionscalculatetheratiobetweentheSUMorCOUNTfunctionsonafieldandtheSUMorCOUNTfunctionsonthesamefieldinahierarchicallyhighergroup)

Newfunctions:MODBIT_AND,BIT_OR,BIT_NOT,BIT_XOR,BIT_SHL,BIT_SHREXP10,LOG10ROUND,FLOORQNTROUND_TO_DIGIT,QNTFLOOR_TO_DIGITSTRREPEATIN_ROW_NUMBER,OUT_ROW_NUMBERROT13EXTRACT_FILENAME,EXTRACT_EXTENSION,EXTRACT_PATHHEX_TO_ASC,HEX_TO_PRINT,HEX_TO_INTHEX_TO_HEX8,HEX_TO_HEX16,HEX_TO_HEX32IPV4_TO_INT,INT_TO_IPV4HASHSEQ,HASHMD5_FILEEXTRACT_PREFIX,EXTRACT_SUFFIX

STRCNT

Introduceda"USING"clausefordeclaringtemporaryfield-expressions

"BETWEEN"operatorintheWHEREandHAVINGclauses

"CASE"(simple-form)statementintheSELECTclause("SELECTCASEmyFieldWHEN'value1'THEN'0'WHEN'value2'THEN'1'ELSE'-1'END")

Newdateandtimeformats:l(milliseconds-lowercase'L')n(nanoseconds)tt(AM/PM)?(anycharacter)

FieldsandAliasesarenowcase-insensitive

ImprovementstoexistingInputandOutputFormats:

AddedmanynewparameterstomostoftheInputandOutputFormats

TheNCSAinputformatnowparsesalsocombinedandextendedNCSAlogfiles

Added"EventCategoryName"and"Data"fieldstotheEVTinputformat

The"-recurse"optionsofmostinputformatsnowspecifyamaximumsubdirectoryrecursionlevel

TheCSVInputandOutputFormatsnowsupportCSVfileswithdouble-quotedstrings

Added"FileVersion","ProductVersion","CompanyName",etc.fieldstotheFSinputformat

Allowed'*'and'?'wildcardsinthesitenamespecificationsforalltheIISinputformats

("SELECT*FROM<mysite*.com>")

AllowedURL'sastheinputpathofalltext-basedinputformats("SELECT*FROMhttp://www.adatum.com/table.csv")

AlloweduseofenvironmentvariablenamesintheTPLoutputformatsections,andaddedaSYSTEM_TIMESTAMPvariable

PerformanceimprovementintheEVTinputformatwhenreadingfromlocalandremoteeventlogs

AllthepropertynamesoftheinputandoutputformatCOMobjectsnowmatchthecommand-linenames

Generalimprovements:

Addedthepossibilitytospecifyparametersin.sqlfiles("logparser-file:myquery.sql?param1=value1+param2=value2")

InputI/Operformanceimprovementfortextfiles

Addedthepossibilitytopermanentlyoverridethedefaultvaluesofglobaloptions,inputformatoptions,andoutputformatoptions

("logparser-e:10-o:NAT-rtp:-1-savedefaults")

ConceptualOverviewThissectionprovidesinformationontheoperationalmechanismsofLogParser.

LogParserArchitecture:DescribestheinternalarchitectureofLogParser.Records:DescribesthedatathatLogParserprocesseswhenworkingwithInputandOutputFormats.CommandsandQueries:DescribeshowLogParsercommandsarestructured,andhowyouspecifyqueriesinacommand.Errors,ParseErrors,andWarnings:DescribestheruntimeerrorsthatcanbegeneratedbyLogParserwhenexecutingacommand.

LogParserArchitectureLogParserismadeupofthreecomponents:

InputFormatsaregenericrecordproviders;recordsareequivalenttorowsinaSQLtable,andInputFormatscanbethoughtofasSQLtablescontainingthedatayouwanttoprocess.LogParser'sbuilt-inInputFormatscanretrievedatafromthefollowingsources:

IISlogfiles(W3C,IIS,NCSA,CentralizedBinaryLogs,HTTPErrorlogs,URLScanlogs,ODBClogs)WindowsEventLogGenericXML,CSV,TSVandW3C-formattedtextfiles(e.g.ExchangeTrackinglogfiles,PersonalFirewalllogfiles,WindowsMedia®Serviceslogfiles,FTPlogfiles,SMTPlogfiles,etc.)WindowsRegistryActiveDirectoryObjectsFileandDirectoryinformationNetMon.capcapturefilesExtended/CombinedNCSAlogfilesETWtracesCustomplugins(throughapublicCOMinterface)

ASQL-LikeEngineCoreprocessestherecordsgeneratedbyanInputFormat,usingadialectoftheSQLlanguagethatincludescommonSQLclauses(SELECT,WHERE,GROUPBY,HAVING,ORDERBY),aggregatefunctions(SUM,COUNT,AVG,MAX,MIN),andarichsetoffunctions(e.g.SUBSTR,CASE,COALESCE,REVERSEDNS,etc.);theresultingrecordsarethensenttoanOutputFormat.

OutputFormatsaregenericconsumersofrecords;theycanbethoughtofasSQLtablesthatreceivetheresultsofthedataprocessing.LogParser'sbuilt-inOutputFormatscan:

Writedatatotextfilesindifferentformats(CSV,TSV,XML,W3C,

user-defined,etc.)SenddatatoaSQLdatabaseSenddatatoaSYSLOGserverCreatechartsandsavethemineitherGIForJPGimagefilesDisplaydatatotheconsoleortothescreen

Note:Transmittingdatathroughanon-securenetworkmightposeaserioussecurityrisktotheconfidentialityoftheinformationtransmitted.Formoreinformationonthesecurityrisksassociatedwithnon-securenetworks,seeSecurityConsiderations.

TheLogParsertoolisavailableasacommand-lineexecutable(LogParser.exe)andasasetofscriptableCOMobjects(LogParser.dll).Thetwobinariesareindependentfromeachother;ifyouwanttouseonlyone,youdonotneedtoinstalltheotherfileonyourcomputer.

RecordsLogParserqueriesoperateonrecordsfromanInputFormat.RecordsareequivalenttorowsinaSQLtable,andInputFormatsareequivalenttoSQLtablescontainingtherows(data)youwanttoprocess.

FieldsandDataTypesEachrecordgeneratedbyanInputFormatismadeupofafixednumberoffields(thecolumnsinaSQLtable),andeachfieldisassignedaspecificnameandaspecificdatatype;thedatatypessupportedbyLogParserare:IntegerRealStringTimestamp

Fieldsinarecordcanonlycontainvaluesofthedatatypeassignedtothefieldor,whenthedataforthatfieldisnotavailable,theNULLvalue.

Forexample,let'sconsidertheEVTInputFormat,whichproducesarecordforeacheventintheWindowsEventLog.Usingthecommand-lineexecutable,wecandiscoverthestructureoftherecordsprovidedbythisInputFormatbytypingthefollowinghelpcommand:

C:\>LogParser-h-i:ETW

TheoutputofthiscommandgivesadetailedoverviewoftheEVTInputFormat,includinga"Fields"sectiondescribingthestructureoftherecordsproduced:

Fields:EventLog(S)RecordNumber(I)TimeGenerated(T)TimeWritten(T)EventID(I)EventType(I)EventTypeName(S)EventCategory(I)EventCategoryName(S)SourceName(S)Strings(S)ComputerName(S)SID(S)Message(S)Data(S)

Fromtheoutputabove,weunderstandthateachrecordismadeupof15fields,andthat,forinstance,thefourthfieldofeachrecordisnamed"TimeWritten"andalwayscontainsvaluesoftheTIMESTAMPdatatype.

RecordStructureSomeInputFormatshaveafixedstructurefortheirrecords(liketheEVTInputFormatusedintheexampleabove,ortheFSInputFormat),butotherscanhavedifferentstructuresdependingonthevaluesspecifiedfortheirparametersoronthefilesbeingparsed.

Forinstance,theNETMONInputFormat,whichparsesNetMoncapturefiles,hasaparameter("fMode")thatcanbeusedtospecifyhowtherecordsshouldbestructured.WecanseethedifferentstructureswhenweaddthisparametertothehelpcommandfortheNETMONformat.ThefirstexampleshowsthefieldsexportedbytheNETMONInputFormatwhenits"fieldmode"issetto"TCPIP"(eachrecordisasingleTCP/IPpacket),andthesecondexampleshowsthefieldsexportedbytheNETMONInputFormatwhenits"fieldmode"issetto"TCPConn"(eachrecordisafullTCPconnection):

C:\>LogParser-h-i:NETMON-fMode:TCPIP

Fields:CaptureFilename(S)Frame(I)DateTime(T)FrameBytes(I)SrcMAC(S)SrcIP(S)SrcPort(I)DstMAC(S)DstIP(S)DstPort(I)IPVersion(I)TTL(I)TCPFlags(S)Seq(I)Ack(I)WindowSize(I)PayloadBytes(I)Payload(S)Connection(I)

C:\>LogParser-h-i:NETMON-fMode:TCPConn

Fields:CaptureFilename(S)StartFrame(I)EndFrame(I)Frames(I)DateTime(T)TimeTaken(I)SrcMAC(S)SrcIP(S)SrcPort(I)SrcPayloadBytes(I)SrcPayload(S)DstMAC(S)DstIP(S)DstPort(I)DstPayloadBytes(I)DstPayload(S)

Asanotherexample,theCSVInputFormat,whichparsestextfilescontainingcomma-separatedvalues,createsitsownstructurebyinspectingtheinputfileforfieldnamesandtypes.WhenusingthehelpcommandwiththeCSVInputFormat,the"Fields"sectionshowsnoinformationontherecordstructure:

C:\>LogParser-h-i:CSV

Fields:Fieldnamesandtypesareretrievedatruntimefromthespecifiedinputfile(s)However,whenwesupplythenameofaCSVfilethat,forinstance,contains2fields("LogDate"and"Message"),thenwecanseethestructureoftherecordsproducedwhenparsingthatfile:

C:\>LogParser-h-i:CSVlog.csv

Fields:

CommandsandQueriesWhenusingthecommand-lineexecutable,LogParserworksoncommandssuppliedbytheuser.Eachcommandhasfivedistinctcomponents:

TheInputFormattouse;OptionalparametersfortheInputFormat;TheOutputFormattouse;OptionalparametersfortheOutputFormat;TheSQLquerythatprocessestherecordsgeneratedbytheInputFormatandproducesrecordsfortheOutputFormat.

Forexample,let'sconsiderthefollowingsimplecommand:

C:\>LogParser-i:EVT-fullText:OFF-o:CSV-tabs:OFF"SELECT*INTOoutput.csvFROMSYSTEM"Thecommandaboveisstructuredasfollows:TheEVTInputFormatisselectedusingthe-i:<InputFormatname>parameter;Its"fullText"parameterissettothe"OFF"value;TheCSVOutputFormatisselectedusingthe-o:<OutputFormatname>parameter;Its"tabs"parameterissettothe"OFF"value;TheSQLqueryis"SELECT*INTOoutput.csvFROMSYSTEM",whichspecifiesthatallrecordsgeneratedfromtheSystemEventLogshouldbesentdirectlytotheOutputFormatwithnofurtherprocessing.

Insomecases,itmightnotbenecessarytospecifytheInputFormat.Intheexamplecommandabove,thevalueoftheFROMclauseis"SYSTEM",whichisthenameofastandardWindowsEventLog;thisnameisautomaticallyrecognizedbyLogParserasacandidatefortheEVTInputFormat,sowecanavoidspecifyingtheInputFormatnamealtogether:

C:\>LogParser-fullText:OFF-o:CSV-tabs:OFF"SELECT*INTOoutput.csvFROMSYSTEM"AsexamplesofothervaluesofFROMclausesthatcanberecognizedbyLogParser,theIISW3CInputFormatisselectedautomaticallywhenthefilenameintheFROMclausestartswith"ex"andhasthe".log"extension,andtheXMLInputFormatisselectedautomaticallywhenthefilenamehasthe".xml"extension.

ThesameappliestoOutputFormats:intheexamplecommandabove,thefilenameintheINTOclausehasthe"csv"extension,thusselectingautomaticallytheCSVOutputFormat;thesamecommandcanthereforebetypedas:

C:\>LogParser-fullText:OFF-tabs:OFF"SELECT*INTOoutput.csvFROMSYSTEM"WhenanOutputFormatisnotspecified,andtheSQLquerydoesnotcontainanINTOclauseLogParserautomaticallyselectstheNATOutputFormat,whichprintstheresultsofthequerytotheconsolewindow.

TheseexamplesshowtheminimalLogParsercommandismadeupoftheSQLqueryalone.InmostcasestheInputandOutputformatscanbedeductedautomaticallyfromtheINTOandFROMclausesofthequery;however,itisarecommendedgoodpracticetoalwaysexplicitlyspecifytheInputandOutputformatsusingthe-iand-oparameters.

Errors,ParseErrors,andWarningsDuringtheexecutionofacommand,LogParsercanencounterthreedifferenttypesofruntimeerrors:Errors,ParseErrors,andWarnings.

ErrorsErrorsareexceptionaleventsoccurringduringtheexecutionofacommandthatcausethecommandtoabort.

EventhoughErrorscanoccurduetoalargenumberofreasons,themostcommoncausescanbecategorizedasfollows:

Invalidquerysyntax:thequeryspecifiedinthecommandisinvalid.InputFormaterrors:thespecifiedInputFormathasencounteredanerrorthatpreventsitfromgeneratinginputrecords.Thiscouldhappen,forexample,whentheFROMclausespecifiesanentity(e.g.afile)thatdoesnotexist.OutputFormaterrors:thespecifiedOutputFormathasencounteredanerrorthatpreventsitfromconsumingoutputrecords.Thiscouldhappen,forexample,whentheINTOclausespecifiesanentity(e.g.afile)thatcannotbewrittento.ToomanyParseErrors:thespecifiedInputFormathasencounteredtoomanyParseErrors,asspecifiedbythe"-e"command-lineglobalparameter.Catastrophicerrors:forexample,LogParserranoutofmemory.

Whenanerroroccurs,theLogParsercommand-lineexecutableabortsthequeryexecutionandreturnstheerrormessageandtheerrorcode.WhenanerroroccurswhileusingtheLogParserscriptableCOMcomponents,aCOMexceptionisthrowncontainingtheerrormessageandtheerrorcode.Inmostcases,theerrorcodereturnedistheinternalsystemerrorcodethatcausedtheerror.

ParseErrorsParseErrorsareerrorsthatoccurwhiletheselectedInputFormatgeneratesthedataonwhichthequeryoperates.Mostofthetimes,asthenamesuggests,theseerrorsaregeneratedwhenaloghasmalformedentries(forexample,whenusingtheIISW3CInputFormat),orwhenasystemerrorpreventsanInputFormatfromprocessingaspecificentryinthedata(forexample,an"accessdenied"erroronafilewhenusingtheFSInputFormat).Inanyevent,thepresenceofaParseErrorindicatesthattheInputFormathadtoskipthedataentrythatcausedtheerror;forexample,whenaParseErrorisencounteredbytheIISW3CInputFormatwhileparsingamalformedlineinthelog,thatlinewillbeskippedanditwillnotbeprocessedbytheSQLengine.

ParseErrorsdonotgenerallycauseearlyterminationofthecurrentlyexecutingcommand,butrather,theyarecollectedinternallybytheSQLengineandreportedwhenthecommandexecutioniscomplete.Thisbehaviorcanbecontrolledwiththe-ecommand-lineglobalparameter.ThevalueusedwiththisparameterspecifiesamaximumnumberofParseErrorstocollectinternallybeforeabortingtheexecutionofthecommand.Forexample,ifweexecuteaqueryonanIISW3Clogfilespecifying"-e:10",LogParserwillcollectupto10ParseErrorsduringtheexecutionofthecommand.IftheIISW3CInputFormatencounters10orlessParseErrors,thecommandwillcompletesuccesfully,andthecollectedParseErrorswillbereportedindetailattheendoftheexecution.Ontheotherhand,iftheinputlogfilecontainsmorethan10malformedloglines,the11thParseErrorwillcausethecommandtoabortandreturnanError.

Thedefaultvalueforthiscommand-lineparameteris-1,whichisaspecialvaluecausingtheSQLenginetoignoreallParseErrorsandreportonlythetotalnumberofParseErrorsencounteredduringtheexecutionofacommand.

Asanexample,considerthefollowingcommand,whichparsesan

IISW3ClogfileandwritesalltheinputrecordstoaCSVfile:

C:\>LogParser-i:IISW3C-o:CSV"SELECT*INTOOutput.csvFROMex020528.log"Let'sassumethatthe"ex020528.log"logfilecontains3malformedloglines.Afterexecutingthecommandabove,theoutputwillbeasfollows:

Taskcompletedwithparseerrors.Parseerrors:3parseerrorsoccurredduringprocessing

Statistics:-----------Elementsprocessed:997Elementsoutput:997Executiontime:0.03seconds

Thisoutputtellsusthatthecommandexecutedsuccesfully,but3ParseErrorshavebeenencounteredwhileprocessingtheinputdata.Sincethedefaultvalueforthe"-e"command-lineparameteris-1,theSQLenginehasignoredalltheseParseErrors,keepingjusttheirtotalcount.

IfwewantedtheseParseErrorstobereportedindetail,wecouldspecifyavalueforthe"-e"parameterdifferentthan-1:

C:\>LogParser-i:IISW3C-o:CSV"SELECT*INTOOutput.csvFROMex020528.log"-e:10Inthiscase,theoutputwouldbe:

Taskcompletedwithparseerrors.Parseerrors:Errorwhileparsingfieldsc-status:ErrorparsingStatusCode"2b00":Extracharacter(s)foundinintegerLogFile"C:\Logs\ex020528.log",Rownumber23,Value"2b00"Cannotfindend-of-line-extracharactersdetectedattheendoflogentryLogFile"C:\Logs\ex020528.log",Rownumber118LogrowterminatesunexpectedlyLogFile"C:\Logs\ex020528.log",Rownumber188

Thecommandstillexecutedsuccesfully,andthistimethe3ParseErrorshavebeencollectedandreportedattheendoftheexecution.

Ifwehadspecified"2"forthe"-e"parameter,theSQLenginewouldhaveabortedtheexecutionofthecommand,andanErrorwouldbereturned:

Taskaborted.Toomanyparseerrors-abortingParseerrors:Errorwhileparsingfieldsc-status:ErrorparsingStatusCode"2b00":Extracharacter(s)foundinintegerLogFile"C:\Logs\ex020528.log",Rownumber23,Value"2b00"Cannotfindend-of-line-extracharactersdetectedattheendoflogentry

LogFile"C:\Logs\ex020528.log",Rownumber118LogrowterminatesunexpectedlyLogFile"C:\Logs\ex020528.log",Rownumber188

WarningsWarningsareexceptionaleventsoccurringduringtheexecutionofacommandthatrequireattentionfromtheuser.Thereareonlyafewsituationsthatcouldcauseawarning,andthesearehandleddifferentlydependingonwhetherornotthewarningarisesduringtheexecutionofacommand,orwhentheexecutionhascompleted.

Whenawarningisgeneratedduringtheexecutionofacommand,thecommand-lineexecutableshowsaninteractiveprompttotheuseraskingwhetherornottheexecutionshouldcontinue.

Asanexample,consideracommandthatwritesoutputrecordstoaCSVfile.TheCSVOutputFormat"fileMode"parametercanbeusedtospecifywhatactionshouldbetakenincasetheoutputfilealreadyexists.Thevalue"2"specifiesthatalreadyexistingoutputfilesshouldnotbeoverwritten;whenusingthisoption,theCSVOutputFormatwillraiseaWarningwhenanalreadyexistingoutputfilewillnotbeoverwritten:

C:\>LogParser-i:EVT-o:CSV"SELECTTOP5MessageINTOOutput.csvFROMSystem"-fileMode:2WARNING:FileC:\LogSamples\Output.csvexistsanditwillnotbeoverwritten.Doyouwanttocontinue?[Yes/No/Ignoreall]:Whenthispromptappears,theusercanchoosebetweencontinuingtheexecutionofthecommandallowingadditionalwarningstotriggerthepromptagain,abortingtheexecutionofthecommand(inwhichcasethecommandterminateswithanError),orcontinuingtheexecutionofthecommandignoringadditionalwarnings.

Theinteractivepromptcanbecontrolledwiththeglobal-iwcommand-lineparameter.ThisON/OFFparameterspecifieswhetherornot

warningsshouldbeignored;thedefaultvalueis"OFF",meaningthatruntimewarningswillnotbeignoredandwilltriggertheinteractiveprompt.Specifying"ON",ontheotherhand,disablestheinteractiveprompt,andruntimewarningswillbeignoredandtheirtotalcountwillbereportedwhenthecommandexecutionhascompleted:

C:\>LogParser-i:EVT-o:CSV"SELECTTOP5MessageINTOOutput.csvFROMSystem"-fileMode:2-iw:ONTaskcompletedwithwarnings.Warnings:1warningoccurredduringprocessing

Tip:IfyouusetheLogParsercommand-lineexecutableinanon-interactivescript(e.g.inascriptthathasbeenscheduledtorunautomaticallyatspecifictimes),youshouldalwaysuse"ON"forthe"iw"parameter,otherwiseintheeventofaruntimewarningtheLogParsercommandwillstallwaitingforausertopressakeyintheinteractiveprompt.

Warningsthataregeneratedwhenacommandhascompletedaresimplyreportedtotheuser.

Forexample,the"ignoreDspchErrs"parameteroftheSYSLOGOutputFormatcanbeusedtospecifywhetherornoterrorsoccurringwhiledispatchingoutputrecordsshouldbeignoredandreportedaswarningsattheendoftheexecution.ThefollowingexamplecommandusestheSYSLOGOutputFormattosendoutputrecordstoanon-existinguser:

C:\>LogParser-i:EVT-o:SYSLOG"SELECTTOP5MessageINTONonExistingUserFROMSystem"-ignoreDspchErrs:ONSincethespecifieduserdoesnotexist,theSYSLOGOutputFormatwillencounteranerrorforeachoutputrecorditwilltrytosendtotheuser;the"ON"valueforthe"ignoreDspchErrs"tellstheoutputformattoignoretheseerrorsandreportallofthemwhentheexecutionhascompleted:

WritingaQueryWithLogParseryouuseQuerieswritteninadialectoftheSQLlanguagetospecifytheoperationsthattransforminputrecordsgeneratedbyanInputFormatintooutputrecordsthataredeliveredtoanOutputFormat.

InthissectionwewillcovertheeightbasicbuildingblocksoftheSQL-LikequeriesthatyoucanusewithLogParsertoperformdifferentprocessingtasks.

BasicsofaQueryThemostsimplequerythatcanbewrittenwithLogParserspecifiesthatalltheInputRecordsgeneratedbyanInputFormataretobedeliveredtoanOutputFormatwithnointerveningprocessing.

Forexample,let'sassumethatwewanttovisualizeallthefieldsofalltheeventsintheSystemEventLog.Toperformthistask,wefirsthavetospecifytheEVTInputFormatasthesourceofourinputrecords,andwedosobyusingthe"-i:EVT"command-lineparameter.Then,wecanchoosetheNATOutputFormatastheconsumerofouroutputrecords,sincethisOutputFormatisspecificallydesignedtoprintoutputrecordstotheconsolewindow;wedosobyusingthe"-o:NAT"command-lineparameter.Finally,wespecifytheSQLquerythatperformsthedesiredtask;thecompletecommandisasfollows:

C:\>LogParser-i:EVT-o:NAT"SELECT*FROMSystem"

Thequeryabovecontainsthetwobasicbuildingblocksofeachpossiblequery:theSELECTclause,andtheFROMclause.

TheSELECTclauseisusedtospecifywhichinputrecordfieldswewanttoappearintheoutputrecords;inthisexample,thespecial"*"wildcardmeans"allthefields".

TheFROMclauseisusedtospecifywhichspecificdatasourcewewanttheInputFormattoprocess.DifferentInputFormatsinterpretthevalueoftheFROMclauseindifferentways;forinstance,theEVTInputFormatrequiresthevalueoftheFROMclausetobethenameofaWindowsEventLog,whichinourexampleisthe"System"EventLog.

Tobeprecise,theINTOclauseshouldappearineveryqueryaswell.TheINTOclauseisusedtospecifythetargetwewanttheOutputFormattowritedatato.Inourexample,wewanttheNATOutputFormattodisplayresultstotheconsolewindow.Thisisaccomplishedbyspecifying"STDOUT"forthevalueoftheINTOclause,asinthefollowingexample:

C:\>LogParser-i:EVT-o:NAT"SELECT*INTOSTDOUTFROMSystem"

WhenaquerydoesnotspecifyanINTOclause,theNATOutputFormatautomaticallyselects"STDOUT"asitstarget,soinourexamplewecaneliminatetheINTOclausealtogether.

Tip:WhenyouusetheNATOutputFormattodisplayresultstotheconsolewindow,LogParserprints10linesbeforepausingtheprintoutandpromptingtheusertopressakeytodisplaythenext10lines.Tooverridethisbehavior,youcanusethe"-rtp"parameteroftheNATOutputFormattospecifythenumberoflinestobeprintedbeforepausing;ifyouwanttodisablethepausealtogetherandhaveLogParserdisplayalltherecordsinasingleprintout,usethe"-1"value.

SelectingSpecificFieldsWhenyouexecutethebasicqueryabove,LogParserprintsallthefieldsofalltheeventsintheSystemEventLogtotheconsolewindow.Mostofthetimes,aprintoutofallofthe14fieldsoftheEventLogrecordsmightnotbedesired.Forexample,wemightonlywanttoseethetimeatwhicheacheventwasgenerated,thetypeoftheevent,andthenameofthesourceoftheevent.Toaccomplishthis,wehavetosubstitutethe"*"wildcardintheSELECTclausewithacomma-separatedlistofthenamesofthefieldswewishtobedisplayed.WecanseethenamesofthefieldsintheEVTInputFormatrecordsbytypingthefollowinghelpcommand:

C:\>LogParser-h-i:EVT

TheoutputofthiscommandgivesadetailedoverviewoftheEVTInputFormat,includinga"Fields"sectiondescribingthestructureoftherecordsproduced:

Fields:EventLog(S)RecordNumber(I)TimeGenerated(T)TimeWritten(T)EventID(I)EventType(I)EventTypeName(S)EventCategory(I)EventCategoryName(S)SourceName(S)Strings(S)ComputerName(S)SID(S)Message(S)Data(S)

Fromthefieldslisting,weunderstandthatthefieldsweareinterestedinarenamed"TimeGenerated","EventTypeName",and"SourceName";wecannowrewriteourcommandas:

C:\>LogParser-i:EVT-o:NAT"SELECTTimeGenerated,EventTypeName,SourceNameFROMSystem"

Tip:Fieldnamesarecase-insensitive.

Tip:Ifafieldnamecontainsspaces,youneedtoencloseitinsquarebrackets('['and']')forLogParsertobeabletorecognizeit.

Theoutputofthiscommandcontainsthreecolumns,oneforeachofthefieldswehaveselected:

TimeGeneratedEventTypeNameSourceName

-----------------------------------------------------------2004-03-1418:56:55WarningeventW32Time2004-03-1414:02:23InformationeventDisk2004-03-1414:02:23InformationeventDisk2004-03-1412:00:00InformationeventEventLog2004-03-1400:41:47WarningeventW32Time2004-03-1322:17:00InformationeventServiceControlManager2004-03-1322:06:48InformationeventServiceControlManager2004-03-1322:06:48InformationeventServiceControlManager2004-03-1312:00:00InformationeventEventLog2004-03-1222:30:47InformationeventServiceControlManager

ThisexampleillustratesthemostsimpletransformationthatyoucanachievewiththeLogParserSQLlanguage:transforminganinputrecordmadeupofanumberoffieldsintoanoutputrecordmadeupofasubsetofthesefields;inSQLterms,thistransformationiscalledprojection.

UsingFunctionsFunctionsareverypowerfulelementsoftheLogParserSQL-Likelanguagethattakevaluesasarguments,dosomeprocessing,andreturnanewvalue.TheLogParserSQL-Likelanguagesupportsawidevarietyoffunctions,includingarithmeticalfunctions(e.g.ADD,SUB,MUL,DIV,MOD,QUANTIZE,etc.),stringmanipulationfunctions(e.g.SUBSTR,STRCAT,STRLEN,EXTRACT_TOKEN,etc.),andtimestampmanipulationfunctions(e.g.TO_DATE,TO_TIME,TO_UTCTIME,etc.).

Consideringthepreviousexample,assumethatforthe"TimeGenerated"fieldweonlyneedtoretrievethedatewhenaneventhasbeengenerated,ignoringallofthetimeelements.Todothis,weneedtomodifythe"TimeGenerated"fieldwiththeTO_DATEfunction,whichtakesavalueoftypeTIMESTAMPandreturnsanewvalueoftypeTIMESTAMPcontainingonlytheyear,day,andmonthelements:

C:\>LogParser-i:EVT-o:NAT"SELECTTO_DATE(TimeGenerated),EventTypeName,SourceNameFROMSystem"Theoutputofthiscommandis:

TO_DATE(TimeGenerated)EventTypeNameSourceName--------------------------------------------------------------2004-03-14WarningeventW32Time2004-03-14InformationeventDisk2004-03-14InformationeventDisk2004-03-14InformationeventEventLog2004-03-14WarningeventW32Time2004-03-13InformationeventServiceControlManager2004-03-13InformationeventServiceControlManager2004-03-13InformationeventServiceControlManager2004-03-13InformationeventEventLog2004-03-12InformationeventServiceControlManager

Functionscanalsoappearasargumentsofotherfunctions.Forexample,insteadoftheeventtypenameshownintheoutputabove,wemightwantthefirstwordonly("Warning","Information",etc.),allincapitalletters.ThistaskcanbeaccomplishedbyfirstusingtheEXTRACT_TOKENfunction,whichextractsspecificsubstringsfromwithinastring,followedbytheTO_UPPERCASEfunction,whichtransformsastringintoastringwithalluppercasecharacters:

C:\>LogParser-i:EVT-o:NAT"SELECTTO_DATE(TimeGenerated),TO_UP

PERCASE(EXTRACT_TOKEN(EventTypeName,0,'')),SourceNameFROMSystem"TO_DATE(TimeGenerated)TO_UPPERCASE(EXTRACT_TOKEN(EventTypeName,0,''))SourceName-----------------------------------------------------------------------------------------------2004-03-14WARNINGW32Time2004-03-14INFORMATIONDisk2004-03-14INFORMATIONDisk2004-03-14INFORMATIONEventLog2004-03-14WARNINGW32Time2004-03-13INFORMATIONServiceControlManager2004-03-13INFORMATIONServiceControlManager2004-03-13INFORMATIONServiceControlManager2004-03-13INFORMATIONEventLog2004-03-12INFORMATIONServiceControlManager

SpecifyingConstantsSofarwehavewrittenSELECTclausesthatspecifybothfieldsandfunctions.Thereisathirdkindofitemthatwecoulduseinourqueries:constants.ConstantsarespecialelementsintheLogParserlanguagethatrepresentfixedvalues;justlikethefieldvalues,constantvaluescanbeoneoftheLogParsertypes:INTEGER,REAL,STRING,TIMESTAMP,andNULL.Constantscanbespecifiedinqueriesindifferentways,dependingontheirtype.

ConstantvaluesoftheINTEGERtypearespecifiedbysimplytypingtheirvalue;thefollowingquery:

SELECT242,SourceNameFROMSYSTEM

wouldproducethefollowingoutput:

242SourceName-------------242W32Time242Disk242Disk242EventLog242W32Time

ConstantvaluesoftheREALtypearespecifiedexactlyliketheINTEGERvalues,buttheyarerecognizedasbeingoftheREALtypebythepresenceofadecimalpoint:

SELECT242.7,SourceNameFROMSYSTEM

242.700000SourceName--------------------242.700000W32Time242.700000Disk242.700000Disk242.700000EventLog

STRINGconstantsmustbeenclosedwithinsingle-quotecharacters:

SELECT'MyConstant',SourceNameFROMSYSTEM

242.700000W32Time'MyConstant'SourceName----------------------MyConstantW32TimeMyConstantDiskMyConstantDiskMyConstantEventLogMyConstantW32Time

SpecialcharactersinSTRINGconstantscanbespecifiedbyusingcharactersequencesprecededbythe'\'character.Forexample,asingle-quotecharactercanbespecifiedas\',whileabackslashcharactercanbespecifiedby\\:

SELECT'Contains\'aquote','Contains\\abackslash',SourceNameFROMSYSTEM'Contains'aquote''Contains\abackslash'SourceName-----------------------------------------------------Contains'aquoteContains\abackslashW32TimeContains'aquoteContains\abackslashDiskContains'aquoteContains\abackslashDiskContains'aquoteContains\abackslashEventLogContains'aquoteContains\abackslashW32Time

Inaddition,itisalsopossibletospecifyanyUNICODEcharacterusingthe\uxxxxnotation,wherexxxxisthe4-digithexadecimalrepresentationoftheUNICODEcharacter.Forexample,tospecifyatabcharacter(whoseUNICODEvalueis0009),wecouldtype:

SELECT'Contains\u0009atab',SourceNameFROMSYSTEM

ANULLconstantcanbespecifiedwiththe"NULL"keyword:

SELECTNULL,SourceNameFROMSYSTEM

TIMESTAMPconstantsarespecifiedinthefollowingway:

TIMESTAMP('timestampvalue','timestampformat')

Formoreinformationregardingtimestampvalues,constants,andformatspecifications,refertotheTimestampReference.

IntheLogParserSQLlanguage,thethreetermsthatcanbespecifiedinaSQLquery(fields,functions,andconstants)arecollectivelyreferredto

asfield-expressions.

AliasingField-ExpressionsConsideragainoneoftheexamplesseeninthissection:

C:\>LogParser-i:EVT-o:NAT"SELECTTO_DATE(TimeGenerated),TO_UPPERCASE(EXTRACT_TOKEN(EventTypeName,0,'')),SourceNameFROMSystem"TO_DATE(TimeGenerated)TO_UPPERCASE(EXTRACT_TOKEN(EventTypeName,0,''))SourceName-----------------------------------------------------------------------------------------------2004-03-14WARNINGW32Time2004-03-14INFORMATIONDisk2004-03-14INFORMATIONDisk2004-03-14INFORMATIONEventLog2004-03-14WARNINGW32Time2004-03-13INFORMATIONServiceControlManager2004-03-13INFORMATIONServiceControlManager2004-03-13INFORMATIONServiceControlManager2004-03-13INFORMATIONEventLog2004-03-12INFORMATIONServiceControlManager

Wecanseethatforeachfieldintheoutputrecord,theNATOutputFormatprintsacolumnheaderwiththenameofthatfield.Bydefault,outputrecordfieldsarenamedwiththefullfield-expressiontextthatgeneratesthem;inourexample,thenameofthefirstoutputrecordfieldis"TO_DATE(TimeGenerated)",whichmirrorsexactlythefield-expressiontextusedintheSELECTclause.

Wecanchangethenameofafield-expressionintheSELECTclausebyusinganAlias.Inordertoaliasafield-expressionintheSELECTclause,wecanusetheASkeywordfollowedbythenewname:

C:\>LogParser-i:EVT-o:NAT"SELECTTO_DATE(TimeGenerated)ASDateGenerated,TO_UPPERCASE(EXTRACT_TOKEN(EventTypeName,0,''))ASTypeName,SourceNameFROMSystem"DateGeneratedTypeNameSourceName-----------------------------------------------2004-03-14WARNINGW32Time2004-03-14INFORMATIONDisk2004-03-14INFORMATIONDisk2004-03-14INFORMATIONEventLog2004-03-14WARNINGW32Time2004-03-13INFORMATIONServiceControlManager2004-03-13INFORMATIONServiceControlManager2004-03-13INFORMATIONServiceControlManager2004-03-13INFORMATIONEventLog2004-03-12INFORMATIONServiceControlManager

Aliasingafield-expressionmeansassigninganametoit;aswewillseelater,thisnamecanalsobeusedanywhereelseinthequeryasashortcutthatreferstotheoriginalfield-expression.

FilteringInputRecordsWhenretrievingdatafromanInputFormat,itisoftenneededtofilteroutunneededrecordsandonlykeepthosethatmatchspecificcriteria.

Forexample,considerthesimplecommandseenintheprevioussection,whichreturnsselectedfieldsfromalloftheeventsintheSystemeventlog:

C:\>LogParser-i:EVT-o:NAT"SELECTTimeGenerated,EventTypeName,SourceNameFROMSystem"TimeGeneratedEventTypeNameSourceName-----------------------------------------------------------2004-03-1418:56:55WarningeventW32Time2004-03-1414:02:23InformationeventDisk2004-03-1414:02:23InformationeventDisk2004-03-1412:00:00InformationeventEventLog2004-03-1400:41:47WarningeventW32Time2004-03-1322:17:00InformationeventServiceControlManager2004-03-1322:06:48InformationeventServiceControlManager2004-03-1322:06:48InformationeventServiceControlManager2004-03-1312:00:00InformationeventEventLog2004-03-1222:30:47InformationeventServiceControlManager

Let'snowassumethatweareonlyinterestedintheeventsgeneratedbythe"ServiceControlManager"source.Toaccomplishthistask,wecanuseanotherbasicbuildingblockoftheLogParserSQL-Likelanguage:theWHEREclause.

TheWHEREclauseisusedtospecifyabooleanexpressionthatmustbesatisfiedbyaninputrecordforthatrecordtobeoutput.Inputrecordsthatdonotsatisfytheconditionwillbediscarded.InSQLterms,filteringrecordswiththeWHEREclauseisatransformationcalledselection.

UsingtheWHEREclause,wecanrewritethepreviouscommandasfollows:

C:\>LogParser-i:EVT-o:NAT"SELECTTimeGenerated,EventTypeName,SourceNameFROMSystemWHERESourceName='ServiceControlManager'" Tip:TheWHEREclausemustimmediatelyfollowtheFROM

clause.

Theoutputofthiscommandis:

TimeGeneratedEventTypeNameSourceName-----------------------------------------------------------2004-03-1322:17:00InformationeventServiceControlManagerLet'sanalyzeindetailtheWHEREclauseusedinthisexample.Thebooleanconditionthatwehaveusedisaverysimpleone:weonly

2004-03-1322:06:48InformationeventServiceControlManager2004-03-1322:06:48InformationeventServiceControlManager2004-03-1222:30:47InformationeventServiceControlManager2004-03-1222:12:32InformationeventServiceControlManager2004-03-1221:09:14InformationeventServiceControlManager

wantthoseinputrecordswhose"SourceName"fieldhastheexactvalueof"ServiceControlManager".Tospecifythiscondition,wehaveusedthe"="relationaloperator,withtheleftoperandbeingthe"SourceName"field,andtherightoperandbeingaSTRINGconstant.

ComplexConditionsConditionsspecifiedintheWHEREclausecanbemorecomplex,makinguseofcomparisonoperators(suchas">","<=","<>","LIKE","BETWEEN",etc.)andbooleanoperators(suchas"AND","OR","NOT").

Forexample,wemightonlywanttoseetwokindsofevents:

Eventsgeneratedbythe"ServiceControlManager"sourcewhoseEventIDisgreaterthanorequal7024;Eventsgeneratedbythe"W32Time"source.

Toaccomplishthis,thequerycanbewrittenasfollows:

SELECTTimeGenerated,EventTypeName,SourceNameFROMSystemWHERE(SourceName='ServiceControlManager'ANDEventID>=7024)OR(SourceName='W32Time')Asanotherexample,wemightwanttoseealltheeventsthathavebeenloggedinthepast24hours.TranslatedintoWHEREterms,thismeansthatweonlywanttoseerecordswhose"TimeWritten"fieldisgreaterthanorequalthecurrentlocaltimeminus1day:

SELECT*FROMSystemWHERETimeWritten>=SUB(TO_LOCALTIME(SYSTEM_TIMESTAMP()),TIMESTAMP('0000-01-02','yyyy-MM-dd'))Tip:InLogParsertheoriginoftimeisday1ofmonth1ofyear

zero.Thismeansthatatimespanofonedaycanbespecifiedasday2ofmonth1ofyearzero,i.e.24hoursaftertheoriginoftime.

Toseesecurityeventswhose"Message"fieldcontainstheword"logon",wecanusetheLIKEoperator,whichtestsaSTRINGvalueforcase-insensitivepatternmatching:

SELECT*FROMSecurity

WHEREMessageLIKE'%logon%'

IfwewanttoretrieveeventswithanIDbelongingtoaspecificsetofvalues,wecanusetheINoperatorfollowedbyalistofthedesired"EventID"values:

SELECT*FROMSecurityWHEREEventIDIN(547;541;540;528)

Tip:WiththeINoperator,singlevaluesareseparatedbythesemicoloncharacter.

Ontheotherhand,ifwewanttoretrieveeventswithanIDbelongingtoaspecificrangeofvalues,wecanusetheBETWEENoperatorasfollows:

SELECT*FROMSecurityWHEREEventIDBETWEEN528AND547

SortingOutputRecordsAcommonlyusedbuildingblockofSQLqueriesistheORDERBYclause.TheORDERBYclausecanbeusedtospecifythattheoutputrecordsshouldbesortedaccordingtothevaluesofselectedfields.

Inthefollowingexample,weareusingtheFSInputFormattoretrievealistingofthefilesinaspecificdirectory,sortingthelistingbythefilesize:

C:\>LogParser-i:FS-o:NAT"SELECTPath,SizeFROMC:\MyDirectory\*.*ORDERBYSize"PathSize-------------------------------------------C:\MyDirectory\..0C:\MyDirectory\.0C:\MyDirectory\ieexec.exe.config140C:\MyDirectory\csc.exe.config163C:\MyDirectory\vbc.exe.config163C:\MyDirectory\jsc.exe.config163C:\MyDirectory\l_except.nlp168C:\MyDirectory\caspol.exe.config353C:\MyDirectory\ilasm.exe.config353C:\MyDirectory\ConfigWizards.exe.config353

Tip:TheORDERBYclausemustbethelastclauseappearinginaLogParserSQLquery.

Bydefault,outputrecordsaresortedaccordingtoascendingvalues.WecanchangethesortdirectionbyappendingtheDESC(fordescending)orASC(forascending)keywordstotheORDERBYclause,asinthefollowingexample:

C:\>LogParser-i:FS-o:NAT"SELECTPath,SizeFROMC:\MyDirectory\*.*ORDERBYSizeDESC"PathSize----------------------------------------------C:\MyDirectory\mscorsvr.dll2494464C:\MyDirectory\mscorwks.dll2482176C:\MyDirectory\corperfmonsymbols.ini2435148C:\MyDirectory\mscorlib.dll2088960C:\MyDirectory\System.Windows.Forms.dll2039808C:\MyDirectory\System.Design.dll1699840C:\MyDirectory\mscorcfg.dll1564672

Tip:DifferentlythanthestandardSQLlanguage,theLogParserSQL-LikelanguagesupportsonlyoneDESCorASCkeywordforthewholeORDERBYclause.

Ifwewantourlistingtobesortedfirstbyfilesizeandthenbyfilecreationtime,wecandosobyspecifyingbothfield-expressionsintheORDERBYclause:

C:\>LogParser-i:FS-o:NAT"SELECTName,Size,CreationTimeFROMC:\

MyDirectory\*.*ORDERBYSize,CreationTime"NameSizeCreationTime---------------------------------------------------..02004-05-2408:14:07.221.02004-05-2408:14:07.221ieexec.exe.config1402004-05-2408:14:21.441csc.exe.config1632004-05-2408:14:21.191jsc.exe.config1632004-05-2408:14:21.762vbc.exe.config1632004-05-2408:14:26.599l_except.nlp1682004-05-2408:14:21.812caspol.exe.config3532004-05-2408:14:20.920ConfigWizards.exe.config3532004-05-2408:14:21.21cvtres.exe.config3532004-05-2408:14:21.251

Sincethesortoperationisperformedonoutputrecords,theLogParserSQL-Likelanguagerequiresthatfield-expressionsappearingintheORDERBYclausemustalsoappearintheSELECTclause.Inotherwords,thesetoffield-expressionsintheORDERBYclausemustbeasubsetofthefield-expressionsintheSELECTclause.Thus,thefollowingexampleisNOTcorrect:

SELECTSourceName,EventIDFROMSystemORDERBYTimeGeneratedOntheotherhand,thefollowingexampleIScorrect:

SELECTSourceName,EventID,TimeGeneratedFROMSystemORDERBYTimeGenerated

AggregatingDataWithinGroupsAllthequeryexamplesthatwehaveseensofarshareacommoncharacteristic:thevaluesofeachoutputrecordwerebuiltuponthevaluesofasingleinputrecord.Sometimes,however,wemightneedtoaggregatemultipleinputrecordstogetherandperformsomeoperationongroupsofinputrecords.Toaccomplishthistask,theLogParserSQL-Likelanguagehasaspecialsetoffunctionsthatcanbeusedtoperformbasiccalculationsonmultiplerecords.Theseaggregatefunctions(alsoreferredtoas"SQLfunctions")includeSUM,COUNT,MAX,MIN,andAVG.

AggregatingDataToshowaclassicexampleoftheuseofaggregatefunctions,assumethatgivenanIISW3Clogfile,wewanttocalculatethetotalnumberofbytessentbytheIISserverduringthewholeperiodrecordedinthelogfile.ConsideringthatthenumberofbytessentbytheIISserverforeachHTTPrequestisloggedinthe"sc-bytes"field,ourcommandwilllooklikethefollowingexample:

C:\>LogParser-i:IISW3C-o:NAT"SELECTSUM(sc-bytes)FROMex040528.log"SincetheSELECTclauseofthisquerymakesuseoftheSUMaggregatefunction,thequerywillautomaticallyaggregatealltheinputrecords,andcalculatethesumofallthevaluesofthe"sc-bytes"fieldacrossalltheinputrecords;theoutputofthiscommandwillthenlooklikethefollowingoutput:

SUM(sc-bytes)-------------242834732Astheexampleshows,theresultofthequeryisasingleoutputrecord,containingasinglevaluecalculatedacrossalltheinputrecords.

Asanotherexample,wemightwanttocalculatehowmanyrequestshavebeenloggedinthelogfile.ConsideringthateachlogfileentryrepresentsasingleHTTPrequest,thistaskcanbeaccomplishedbysimplycountinghowmanyinputrecordsareloggedinthefile:

C:\>LogParser-i:IISW3C-o:NAT"SELECTCOUNT(*)FROMex040528.log"TheexampleabovemakesuseoftheCOUNTaggregatefunction.Whenusedwiththespecial"*"argument,theCOUNTfunctionreturnsthetotal

numberofinputrecordsprocessedbythequery.

Ifwewanttocalculatehowmanyrequestssatisfyaparticularcondition,forexamplehowmanyrequestswereforanASPpage,wecanaddaWHEREclausetothequery,andtheCOUNTfunctionwillonlycountinputrecordssatisfyingtheWHEREcondition:

SELECTCOUNT(*)FROMex040528.logWHEREEXTRACT_EXTENSION(cs-uri-stem)LIKE'asp'

CreatingGroupsIntheexamplesabove,wehavebeenusingaggregatefunctionstocalculateavalueacrossalltheinputrecords;sometimes,however,wemightwanttocalculatevaluesacrossgroupsofinputrecords.

Asanexample,wemightwanttocalculatethetotalnumberofbytessentbytheIISserverforeachURL.Toperformthistask,weneedtodividealltheinputrecordsintogroupsaccordingtotheURLrequested,andthenusetheSUMaggregatefunctionseparatelyoneachgroup.

ThiscanbeaccomplishedbyusinganotherbuildingblockoftheLogParserSQLlanguage:theGROUPBYclause.TheGROUPBYclauseisusedtospecifywhichfieldswewantthegroupsubdivisiontobebasedon;aftertheinputrecordshavebeendividedintothesegroups,alltheaggregatefunctionsintheSELECTclausewillbecalculatedseparatelyoneachofthesegroups,andthequerywillreturnanoutputrecordforeachgroupcreated.

UsingtheGROUPBYclause,ourexamplequeryanditsoutputwilllooklikethis:

SELECTcs-uri-stem,COUNT(*)FROMex040528.logGROUPBYcs-uri-stemcs-uri-stemCOUNT(*)------------------------------/Home/default.asp5/Home/images/bckgd.gif419/Docs/expl.htm12/Docs/main.htm26/login/frmx.dll1

Tomakeanotherexample,assumethatwewanttocalculatehowmanyrequestshavebeenservedforeachpagetype(ASP,html,CSS,etc.).Firstofall,weneedtocreateseparategroupsaccordingtotheextensionoftheURL;afterthisgroupsubdivisionhasbeendone,wecancalculateaCOUNT(*)oneachgroup:

SELECTEXTRACT_EXTENSION(cs-uri-stem)ASPageType,COUNT(*)FROMex040528.logTheoutputwilllooklike:

GROUPBYPageTypePageTypeCOUNT(ALL*)--------------------htm115css22gif585exe25nsf142swf11jpg77html1dll1asp5js11class5

Ifwesorttheoutputaboveaccordingtothenumberofrequestsforeachgroup,wewillbecreatingalistshowingthemostrequestedpagetypesfirst:

SELECTEXTRACT_EXTENSION(cs-uri-stem)ASPageType,COUNT(*)ASPageTypeHitsFROMex040528.logGROUPBYPageTypeORDERBYPageTypeHitsDESC

Theoutputwilllooklike:

PageTypePageTypeHits--------------------gif585nsf142htm115jpg77exe25css22js11swf11asp5class5dll1html1

Groupscanalsobebuiltonmultiplefields,thuscreatingahierarchyofgroups.

Forexample,considerthefollowingquery:

SELECTEXTRACT_EXTENSION(cs-uri-stem)ASPageType,sc-status,COUNT(*)FROMex040528.logGROUPBYPageType,sc-statusThisquerycreatesgroupsaccordingtotherequestedpagetype,andwithineachofthesegroups,sub-groupsarecreatedaccordingtotheHTTPstatussentbytheIISserverforthegrouppagetype;theaggregatefunction"COUNT"willthenbecalculatedoneachsub-group.Theoutputwilllooklike:

PageTypesc-statusPageTypeHits-----------------------------htm30479css30410gif304450exe20025nsf200129swf2003gif40412css4049

It'simportanttonoteaparticularlanguageconstraintderivedfromtheuseoftheGROUPBYclause.WheneveraquerycontainsaGROUPBYclause,itsSELECTclausecanonlycontainanyofthefollowing:

AggregatefunctionsField-expressionsappearingalsointheGROUPBYclause,orderiving

htm20034css2003jpg20017gif200123jpg30460swf3048nsf4033html4041dll5001asp2005js3047class3044js2004htm4042class2001nsf3049nsf3021

fromthefield-expressionsusedintheGROUPBYclauseConstants

Inotherwords,thefollowingexampleisacorrectquery:

SELECT'hello',TO_UPPERCASE(cs-uri-stem),COUNT(*),SUM(sc-bytes)FROMex040528.logGROUPBYcs-uri-stemInfact,theSELECTclauseintheexampleabovecontains:Aconstant("'hello'");Afield-expression("TO_UPPERCASE(cs-uri-stem)")whoseargumentappearsintheGROUPBYclause;Twoaggregatefunctions.

However,thefollowingexampleisNOTacorrectquery:

SELECTdate,COUNT(*),SUM(sc-bytes)FROMex040528.logGROUPBYcs-uri-stemTheSELECTclauseintheexampleabovecontainsafield-expression("date")thatdoesnotappearintheGROUPBYclause.

ThefollowingexampleisalsoNOTacorrectquery:

SELECTTO_UPPERCASE(cs-uri-stem),COUNT(*),SUM(sc-bytes)FROMex040528.logGROUPBYSUBSTR(TO_UPPERCASE(cs-uri-stem),0,5)TheSELECTclauseintheexampleabovecontainsafield-expression("TO_UPPERCASE(cs-uri-stem)")thatisnotderivedfromanyfield-expressionintheGROUPBYclause;inthiscase,it'sactuallythefield-expressionintheGROUPBYclausethatisderivedfromafield-expressionintheSELECTclause.Thepreviousexamplecanbecorrectedasfollows:

CalculatingPercentagesWhenworkingwithgroupsandaggregatefunctions,itisoftenneededtorepresentanaggregatevalueasapercentage,ratherthanasanabsolutevalue.Wemightwant,forexample,tocalculatethenumberofhitsperpagetypefromaWebserverlogasapercentagerelativetothetotalnumberofhits,ratherthanastheabsolutenumberitself.

Considerthepreviousexamplequery,thatcalculatesthecountofhitsperrequestedpagetype:

SELECTEXTRACT_EXTENSION(cs-uri-stem)ASPageType,COUNT(*)FROMex040528.logGROUPBYPageTypePageTypeCOUNT(ALL*)--------------------htm115css22gif585exe25nsf142swf11jpg77html1dll1asp5js11class5

Ifwewantedtocalculatethepercentageofhitsforeachgroup,wewouldneedtodividethenumberofhitswithineachgroupbythetotalnumberofhitsinthewholelogfile;however,theuseoftheGROUPBYclauserestrictseachaggregatefunctiontooperatewithinthesinglegroups,thusmakingitimpossibletocalculateatthesametimethetotalnumberofhitsacrossallgroups.

Toworkaroundthisproblem,weusetwospecialaggregatefunctionsavailableintheLogParserSQLlanguage:PROPCOUNTandPROPSUM.Whenusedintheirbasicforms,thesefunctionscalculatetheratiooftheCOUNTorADDaggregatefunctionswithinagrouptotheCOUNTorADDaggregatefunctionsonalloftheinputrecords.

UsingthePROPCOUNTfunction,wecanchangethequeryaboveasfollows:

SELECTEXTRACT_EXTENSION(cs-uri-stem)ASPageType,PROPCOUNT(*)Andobtain:

FROMex040528.logGROUPBYPageTypePageTypePROPCOUNT(ALL*)------------------------htm0.115000css0.022000gif0.585000exe0.025000nsf0.142000swf0.011000jpg0.077000html0.001000dll0.001000asp0.005000js0.011000class0.005000

Toshowrealpercentages,wecanmultiplytheaggregatefunctionvaluesby100:

SELECTEXTRACT_EXTENSION(cs-uri-stem)ASPageType,MUL(PROPCOUNT(*),100.0)ASPageTypeHitsFROMex040528.logGROUPBYPageTypePageTypePageTypeHits--------------------htm11.500000css2.200000gif58.500000exe2.500000nsf14.200000swf1.100000jpg7.700000html0.100000dll0.100000asp0.500000js1.100000class0.500000

Fromtheresultsofthisquerywecaninferthat,forexample,requeststo"css"pagesrepresentthe2.2%ofthetotalnumberofrequestsinthislogfile.

CalculatingPercentagesAcrossMultipleGroupHierarchiesTheexamplesaboveshowthebasicformofthePROPCOUNTandPROPSUMfunctions,whichcalculatesthepercentageofanaggregatefunctionwithinagrouprelativetoalloftheinputrecords.However,itisalsopossibletousethePROPCOUNTandPROPSUMfunctionstocalculatepercentagesrelativetohierarchicallyhighergroups.Todoso,wecanusetheONkeywordafterthePROPCOUNTorPROPSUMfunctionnamefollowedbyalistoftheGROUPBYfield-expressionsidentifyingwhichhierarchicallyhighergroupwewantthepercentagetoberelativeto.

Consideroneofthepreviousexamples,inwhichwecalculatedthetotalnumberofhitsperpagetypeperHTTPstatuscode,modifiedtoshowpercentagesratherthanabsolutenumbers:

SELECTEXTRACT_EXTENSION(cs-uri-stem)ASPageType,sc-status,MUL(PROPCOUNT(*),100.0)ASHitsFROMex040528.logGROUPBYPageType,sc-statusORDERBYPageType,sc-status

PageTypesc-statusHits-----------------------------asp2000.500000class2000.100000class3040.400000css2000.300000css3041.000000css4040.900000dll5000.100000exe2002.500000gif20012.300000gif30445.000000gif4041.200000htm2003.400000htm3047.900000

The"Hits"fieldshowsthepercentageofhitsforapagetypeandHTTPstatuscoderelativetothetotalnumberofhits.

IfwewantedtocalculatethepercentageofhitsforapagetypeandHTTPstatuscoderelativetothenumberofhitsforthatpagetype(i.e.thedistributionofHTTPstatuscodeswithineachpagetype),wewouldhavewrittenthequeryasfollows:

SELECTEXTRACT_EXTENSION(cs-uri-stem)ASPageType,sc-status,MUL(PROPCOUNT(*)ON(PageType),100.0)ASHitsFROMex040528.logGROUPBYPageType,sc-statusORDERBYPageType,sc-status

Theoutputwouldbe:

PageTypesc-statusHits---------------------------

htm4040.200000html4040.100000jpg2001.700000jpg3046.000000js2000.400000js3040.700000nsf20012.900000nsf3020.100000nsf3040.900000nsf4030.300000swf2000.300000swf3040.800000

asp200100.000000class20020.000000class30480.000000css20013.636364css30445.454545css40440.909091dll500100.000000exe200100.000000gif20021.025641gif30476.923077gif4042.051282htm20029.565217htm30468.695652htm4041.739130html404100.000000jpg20022.077922jpg30477.922078js20036.363636js30463.636364nsf20090.845070nsf3020.704225nsf3046.338028nsf4032.112676swf20027.272727swf30472.727273

Wecannowinferthat,forexample,about45%ofrequeststo"css"pagesreturnedanHTTPstatuscodeof304.

HerewehaveusedtheONkeywordfollowedbythe"PageType"GROUPBYfield-expression.ThisnotationindicatesthatwewantthePROPCOUNTfunctiontocalculatetheratiooftheCOUNTaggregatefunctionwithinasinglegrouptotheCOUNTaggregatefunctionwithinthehierarchicallyhighergroupidentifiedbythe"PageType"field-expression.

Asanotherexample,wecanmodifythepreviousexamplequerytocreategroupsbasedonthetimetherequestwasmadeat(quantizedat20-secondintervals),thepagetype,andtheHTTPstatuscode:

SELECTQUANTIZE(time,20)ASInterval,EXTRACT_EXTENSION(cs-uri-stem)ASPageType,sc-statusFROMex040528.logGROUPBYInterval,PageType,sc-statusORDERBYInterval,PageType,sc-status

Foreachgroup,wecancalculatethepercentageofhitsrelativetothenumberofhitswithinthetimeintervalandpagetype,thepercentageofhitsrelativetothenumberofhitswithinthetimeintervalalone,andthepercentageofhitsrelativetothetotalnumberofhits:

SELECTQUANTIZE(time,20)ASInterval,EXTRACT_EXTENSION(cs-uri-stem)ASPageType,sc-status,MUL(PROPCOUNT(*)ON(Interval,PageType),100.0)ASHits1,MUL(PROPCOUNT(*)ON(Interval),100.0)ASHits2,MUL(PROPCOUNT(*),100.0)ASHits3FROMex040528.logGROUPBYInterval,PageType,sc-statusORDERBYInterval,PageType,sc-status

IntervalPageTypesc-statusHits1Hits2Hits3-----------------------------------------------------00:28:40css20020.0000001.4705880.10000000:28:40css30460.0000004.4117650.30000000:28:40css40420.0000001.4705880.10000000:28:40exe200100.0000007.3529410.50000000:28:40gif20010.0000001.4705880.10000000:28:40gif30470.00000010.2941180.70000000:28:40gif40420.0000002.9411760.20000000:28:40htm20011.7647062.9411760.20000000:28:40htm30488.23529422.0588241.50000000:28:40jpg20025.0000001.4705880.10000000:28:40jpg30475.0000004.4117650.30000000:28:40nsf200100.00000035.2941182.400000

Fromthequeryresultswecaninfer,forexample,thatduringthe"00:29:20"timeinterval,about78%oftherequeststo"htm"pagesreturnedtheHTTPstatuscode304.Inthesametimeinterval,requeststo"htm"pagesreturningtheHTTPstatuscode304madeupforabout10%oftherequests,andtheserequestsrepresentthe1.5%ofthetotalnumberofrequestsinthelog.

TheexampleaboveshowsthataPROPCOUNTorPROPSUMfunctionwithnoONkeywordislogicallyequivalenttousingtheONkeywordfollowedbyanemptylistofGROUPBYfield-expressions,meaningthatthepercentagetobecalculatedshouldberelativetothehighesthierarchicalgroupidentifiedbynofield-expression,i.e.thewholesetofinputrecords.

00:28:40swf20033.3333331.4705880.10000000:28:40swf30466.6666672.9411760.20000000:29:00ASP200100.0000000.2169200.10000000:29:00GIF200100.0000000.4338390.20000000:29:00asp200100.0000000.2169200.10000000:29:00class20050.0000000.2169200.10000000:29:00class30450.0000000.2169200.10000000:29:00css20014.2857140.2169200.10000000:29:00css30428.5714290.4338390.20000000:29:00css40457.1428570.8676790.40000000:29:00dll500100.0000000.2169200.10000000:29:00exe200100.0000001.9522780.90000000:29:00gif20021.79487214.7505426.80000000:29:00gif30476.92307752.06073824.00000000:29:00gif4041.2820510.8676790.40000000:29:00htm20034.0909093.2537961.50000000:29:00htm30463.6363646.0737532.80000000:29:00htm4042.2727270.2169200.10000000:29:00html404100.0000000.2169200.10000000:29:00jpg20035.0000001.5184380.70000000:29:00jpg30465.0000002.8199571.30000000:29:00js20050.0000000.4338390.20000000:29:00js30450.0000000.4338390.20000000:29:00nsf20094.33962310.8459875.00000000:29:00nsf4035.6603770.6507590.30000000:29:00swf20050.0000000.4338390.20000000:29:00swf30450.0000000.4338390.20000000:29:20NSF200100.0000002.1276600.30000000:29:20asp200100.0000000.7092200.10000000:29:20class304100.0000000.7092200.10000000:29:20css30460.0000002.1276600.30000000:29:20css40440.0000001.4184400.20000000:29:20exe200100.0000002.8368790.40000000:29:20gif30497.14285748.2269506.80000000:29:20gif4042.8571431.4184400.20000000:29:20htm20015.7894742.1276600.30000000:29:20htm30478.94736810.6382981.500000

Inaddition,itisalsoworthmentioningthatthelistoffield-expressionsspecifiedaftertheONkeywordmustbeaproperprefixoftheGROUPBYfield-expressions.If,forexample,theONkeywordisfollowedbythreefield-expressions,thenthesethreefield-expressionsmustmatchthefirstthreefield-expressionsintheGROUPBYclause,andtheymustalsoappearinthesameorderastheydointheGROUPBYclause.Inotherwords,eachPROPCOUNTfunctioninthefollowingqueryiscorrect,sincethelistsoffield-expressionsaftertheONkeywordareallaproperprefixoftheGROUPBYfield-expressions:

SELECTQUANTIZE(time,20)ASInterval,EXTRACT_EXTENSION(cs-uri-stem)ASPageType,sc-status,MUL(PROPCOUNT(*)ON(Interval,PageType),100.0)ASHits1,MUL(PROPCOUNT(*)ON(Interval),100.0)ASHits2FROMex040528.logGROUPBYInterval,PageType,sc-status

However,noneofthePROPCOUNTfunctionsinthefollowingqueryiscorrect,sincethelistsoffield-expressionsaftertheONkeywordarenotaproperprefixoftheGROUPBYfield-expressions:

SELECTQUANTIZE(time,20)ASInterval,EXTRACT_EXTENSION(cs-uri-stem)ASPageType,sc-status,MUL(PROPCOUNT(*)ON(PageType,sc-status),100.0)ASHits1,MUL(PROPCOUNT(*)ON(PageType),100.0)ASHits2,MUL(PROPCOUNT(*)ON(Interval,sc-status),100.0)ASHits2,FROMex040528.logGROUPBYInterval,PageType,sc-status

00:29:20htm4045.2631580.7092200.10000000:29:20jpg20015.3846151.4184400.20000000:29:20jpg30484.6153857.8014181.10000000:29:20js20050.0000001.4184400.20000000:29:20js30450.0000001.4184400.20000000:29:20nsf20061.1111117.8014181.10000000:29:20nsf3025.5555560.7092200.10000000:29:20nsf30433.3333334.2553190.60000000:29:20swf304100.0000002.1276600.300000

FilteringGroupsConsideragainoneofthepreviousexamples,inwhichweusedtheCOUNTaggregatefunctiontocalculatethenumberoftimeseachpagetypehasbeenrequested:

SELECTEXTRACT_EXTENSION(cs-uri-stem)ASPageType,COUNT(*)ASPageTypeHitsFROMex040528.logGROUPBYPageTypeORDERBYPageTypeHitsDESC

PageTypePageTypeHits--------------------gif585nsf142htm115jpg77exe25css22js11swf11asp5class5dll1html1

Let'snowassumethatweareonlyinterestedinseeingpagetypesthathavebeenrequested10timesormore.

Atfirstglance,itmightseemthatwecoulduseaWHEREclausewithaconditiononthevalueoftheCOUNTaggregatefunctiontofilterouttheundesiredgroups.However,wehaveseenthattheWHEREclauseisusedtofilterinputrecords,whichmeansthatthisclauseisevaluatedbeforegroupsarecreated.Forthisreason,useofaggregatefunctionsisnotallowedintheWHEREclause.

ThetaskathandcanbeaccomplishedbyusingtheHAVINGclause.TheHAVINGclauseworksjustliketheWHEREclause,withtheonlydifferencebeingthattheHAVINGclauseisevaluatedaftergroupshavebeencreated,whichmakesitpossiblefortheHAVINGclausetospecifyaggregatefunctions.

Tip:TheHAVINGclausemustimmediatelyfollowtheGROUPBYclause.

UsingtheHAVINGclause,wecanwritetheexampleaboveas:

SELECTEXTRACT_EXTENSION(cs-uri-stem)ASPageType,COUNT(*)ASPageTypeHitsFROMex040528.logGROUPBYPageTypeAndobtain:

HAVINGPageTypeHits>=10ORDERBYPageTypeHitsDESCPageTypePageTypeHits--------------------gif585nsf142htm115jpg77exe25css22js11swf11

EliminatingDuplicateValuesWhenworkingwithinformationfromlogs,itisoftendesiredtoretrievealistofsomevalueswhereeachelementinthelistappearsonlyonce,regardlessofthenumberoftimesthesamevalueappearsintheoriginaldata.

Asanexample,considerthefollowingquery,whichextractsallthedomainaccountsthathaveloggedonacomputerfromthe"Security"eventlog:

SELECTRESOLVE_SID(Sid)ASAccountFROM\\TESTMACHINE1\SecurityWHEREEventIDIN(540;528)Theoutputofthisqueryisalistofallthedomainaccountsappearingineach"Logon"event:

Account------------------------------------------------NTAUTHORITY\LOCALSERVICENTAUTHORITY\NETWORKSERVICENTAUTHORITY\NETWORKSERVICENTAUTHORITY\NETWORKSERVICETESTDOMAIN\TESTUSER1NTAUTHORITY\LOCALSERVICENTAUTHORITY\LOCALSERVICETESTDOMAIN\TESTUSER1TESTDOMAIN\TESTUSER2NTAUTHORITY\LOCALSERVICETESTDOMAIN\TESTUSER1

Ifweareinterestedinretrievingalistinwhicheachaccountnameappearsonlyonce,wecouldusetheDISTINCTkeywordintheSELECTclauseasfollows:

SELECTDISTINCTRESOLVE_SID(Sid)ASAccountFROM\\TESTMACHINE1\SecurityWHEREEventIDIN(540;528)Andobtain:

Account------------------------------------------------NTAUTHORITY\LOCALSERVICENTAUTHORITY\NETWORKSERVICETESTDOMAIN\TESTUSER1TESTDOMAIN\TESTUSER2

TheDISTINCTkeywordisusedtoindicatethattheoutputofaqueryshouldconsistofuniquerecords;duplicateoutputrecordsarediscarded.

Asanotherexample,wemightwanttoretrievealistofallthebrowsersusedtorequestpagesfromourIISserver,witheachbrowserappearingonlyonceinthelist:

SELECTDISTINCTcs(User-Agent)FROM<1>

cs(User-Agent)--------------------------------------------------------------------Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.1)Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+98)Mozilla/4.05+[en]Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.0;+T312461;+Q312461)Mozilla/4.0+(compatible;+MSIE+5.01;+Windows+NT+5.0)Mozilla/4.0+(compatible;+MSIE+6.0;+Windows+NT+5.0)Microsoft+Data+Access+Internet+Publishing+Provider+Cache+ManagerMozilla/2.0+(compatible;+MS+FrontPage+4.0)MSFrontPage/4.0Microsoft+Data+Access+Internet+Publishing+Provider+DAV

ItisalsopossibletousetheDISTINCTkeywordinsidetheCOUNTaggregatefunction,inordertoretrievethetotalnumberofdifferentvaluesappearinginthedata.

Forexample,thefollowingqueryreturnsthetotalnumberofdifferentbrowsersandthetotalnumberofdifferentclientIPaddressesthatrequestedpagesfromourIISserver:

SELECTCOUNT(DISTINCTcs(User-Agent))ASBrowsers, COUNT(DISTINCTc-ip)ASClientsFROM<1>BrowsersClients---------------3563379Tip:IntheLogParserSQL-Likelanguage,theDISTINCTkeyword

canbeusedinsideaggregatefunctionsonlywhentheGROUPBYclauseisnotused.

RetrievingaFixedNumberofRecordsOneofthemostcommonlogreportsisa"TOP10"listshowingthetopentriesappearinginaranking.Thisisusuallyachievedwithaquerythatcalculatessomeaggregatefunctionwithingroups,ordersthegroupsbythevalueoftheaggregatefunction,andthenusestheTOPkeywordintheSELECTclausetoreturnonlyafewrecordsatthetopoftheorderedoutput.

Asanexample,thefollowingqueryreturnstheTOP10URL'srequestedfromanIISlogfile:

SELECTTOP10cs-uri-stemASUrl, COUNT(*)ASHitsFROM<1>GROUPBYUrlORDERBYHitsDESC

UrlHits-----------------------------------/police/laws.nsf25183/cgi-bin/counts.exe5694/police/rulesinfo.nsf5202/police/laws.nsf3980/images/address.gif3609/image/1_m.jpg3540/npanews0.htm3305/images/tibg.gif2955/startopen/startopen920707.htm2502/police/find.nsf2465

ThiskindofreportsisaperfectcandidatefortheCHARTOutputFormat;assumingthatthefollowingqueryissavedinthe"querytop.sql"textfile,thefollowingcommandwillgenerateanimagefilecontainingachartofthequeryoutputabove:

SELECTTOP10cs-uri-stemASUrl, COUNT(*)ASHitsINTOUrls.gifFROM<1>GROUPBYUrlORDERBYHitsDESC

C:\>LogParserfile:querytop.sql-o:chart-chartType:Bar3d-chartTitle:"TOP10URL"

ImprovingQueryReadabilityThefunctionsavailableintheLogParserSQLlanguagemakeitpossibletowritecomplexqueriesoperatingonaverylargenumberofpossibletransformationsoftheinputfields;however,thesecomplexqueriesmightsometimesbecumbersometowrite.

Asanexample,considerthetaskofwritingaquerythatextractsfromtheSecurityeventlogalltheusersbelongingtoaspecificdomainthatloggedonthiscomputer.Forthepurposeoftheexample,let'salsoassumethatwewanttheusernamesaslowercasestrings,andthatwearewritingthequeryasaSQLfilethattakesalowercasedomainnameasaninputparameter.Atfirstthought,thequerywouldlooklikethis:

SELECTEXTRACT_TOKEN(TO_LOWERCASE(RESOLVE_SID(Sid)),1,'\\')ASUsernameFROM SecurityWHERE EventIDIN(540;528)AND EXTRACT_TOKEN(TO_LOWERCASE(RESOLVE_SID(Sid)),0,'\\')='%domainname%'

Toexecutethisquery,wecanusethe"file:"command-lineargument,specifyingavalueforthe"domainname"parameter:

C:\>LogParserfile:myquery.sql?domainname=tstdomain-i:EVT

Whentypingthequeryabove,wehadtorepeattwicethewholeexpressionthattransformstheSidinputrecordfieldintoalowercasefully-qualifiedaccountname:

TO_LOWERCASE(RESOLVE_SID(Sid))

Itwouldbeeasierifwecould,inacertainsense,"assign"thisexpressiontoa"variable",andthenusethevariablewhenneeded.WecoulddefinitelydothatbyaliasingtheexpressionintheSELECTclause:

SELECTTO_LOWERCASE(RESOLVE_SID(Sid))ASFQAccount, EXTRACT_TOKEN(FQAccount,1,'\\')ASUsernameFROM SecurityHowever,theoutputofthisquerynowcontainsanextraneousfield-thefully-qualifiedaccountname:

WHERE EventIDIN(540;528)AND EXTRACT_TOKEN(FQAccount,0,'\\')='%domainname%'FQAccountUsername---------------------------------tstdomain\testusr1testusr1tstdomain\testusr1testusr1tstdomain\testusr2testusr2tstdomain\testusr3testusr3

Toobviatethisproblem,theLogParserSQLlanguagesupportstheUSINGclause.TheUSINGclause,anon-standardSQLlanguageelement,isusedtodeclarealiasesinthesamewayaswewouldintheSELECTclause,withthedifferencethatexpressionsintheUSINGclausewillnotappearintheoutputrecords(unlessexplicitlyreferencedintheSELECTclause).

WiththeUSINGclause,thequeryabovecanbewrittenasfollows:

SELECTEXTRACT_TOKEN(FQAccount,1,'\\')ASUsernameUSING TO_LOWERCASE(RESOLVE_SID(Sid))ASFQAccountFROM SecurityWHERE EventIDIN(540;528)AND EXTRACT_TOKEN(FQAccount,0,'\\')='%domainname%'

Tip:TheUSINGclausemustimmediatelyfollowtheSELECTclause.

Theoutputofthisquerywouldlooklikethefollowingsampleoutput:

Username--------testusr1testusr1testusr2testusr3

AdvancedFeaturesLogParseroffersauniquesetoffeaturesthatenhanceitsflexibilityinthemostcommonlogprocessingscenarios.Thesefeaturesinclude:

ParsingInputIncrementally:someinputformatsallowLogParsertoparseincrementallylogsthatgrowovertime.MultiplexingOutputRecords:someoutputformatsallowtheoutputrecordsofaquerytobewrittentodifferenttargets,dependingonthevaluesofselectedoutputrecordfields.ConvertingFileFormats:duetoitsarchitecture,LogParsercanbeeasilyusedtoconvertlogfilesfromaformattoanother.CustomPlugins:LogParserallowsuserstodeveloptheirowncustominputformats,andusethemwitheithertheLogParsercommand-lineexecutable,orwiththeLogParserscriptableCOMcomponents.

ParsingInputIncrementallyLogParserisoftenusedtoparselogsthatgrowovertime.Forexample,theIISlogsandtheWindowsEventLogarecontinuouslyupdatedwithnewinformation,andinsomecases,wewouldliketoparsetheselogsperiodicallyandonlyretrievethenewrecordsthathavebeenloggedsincethelasttime.Thisisespeciallytrueforscenariosinwhich,forexample,weuseLogParsertoconsolidatelogstoadatabaseinanalmostreal-timefashion,orwhenweuseLogParsertobuildamonitoringsystemthatperiodicallyscanslogsfornewentriesofinterest.

Forthesescenarios,LogParseroffersafeaturethatallowssequentialexecutionsofthesamequerytoonlyprocessnewdatathathasbeenloggedsincethelastexecution.ThisfeaturecanbeenabledwiththeiCheckPointparameterofthefollowinginputformats:

IISW3CNCSAIISHTTPERRURLSCANCSVTSVEVTTEXTLINETEXTWORD

The"iCheckPoint"parameterisusedtospecifythenameofa"checkpoint"filethatLogParserusestostoreandretrieveinformationaboutthe"position"ofthelastentryparsedfromeachofthelogsthatappearinacommand.Whenweexecuteacommandwithacheckpointfileforthefirsttime(i.e.whenthespecifiedcheckpointfiledoesnotexist),LogParserexecutesthequerynormallyandprocessesallthelogsinthecommand,savingfor

eachthe"position"ofthelastparsedentrytothecheckpointfile.Iflateronweexecutethesamecommandspecifyingthesamecheckpointfile,LogParserwillparseagainallthelogsinthecommand,buteachlogwillbeparsedstartingaftertheentrythatwaslastparsedbythepreviouscommand,thusproducingrecordsfornewentriesonly.Whenthenewcommandexecutioniscomplete,theinformationinthecheckpointfileisupdatedwiththenew"position"ofthelastentryineachlog.

Note:Checkpointfilesareupdatedonlywhenaqueryexecutessuccesfully.Ifanerrorcausestheexecutionofaquerytoabort,thecheckpointfileisnotupdated.

Tomakeanexample,let'sassumethatthe"MyLogs"foldercontainsthefollowingtextfiles:

Log1.txt,50linesLog2.txt,100linesLog3.txt,20linesLog4.txt,30lines

Let'salsoassumethatwewanttoparsethesetextfilesincrementallyusingtheTEXTLINEInputFormat,whichreturnsaninputrecordforeachlineintheinputtextfiles.Inordertoparsetheselogsincrementally,wespecifythenameofacheckpointfile,makingsurethatthefiledoesnotexistpriortothecommandexecution.Ourcommandwouldlooklikethis:

logparser"SELECT*FROMMyLogs\*.*"-i:TEXTLINE-iCheckPoint:myCheckPoint.lpcWhenthiscommandisexecutedforthefirsttime,LogParserwillreturnallthe200linesfromallofthefourlogfiles,anditwillcreatethe"myCheckPoint.lpc"checkpointfilecontainingthepositionofthelastlineineachofthefourlogfiles.

Tip:Whenthecheckpointfileisspecifiedwithoutapath,LogParserwillcreatethecheckpointfileinthefoldercurrentlysetforthe%TEMP%environmentvariable,usually"\DocumentsandSettings\<username>\LocalSettings\Temp".;

Let'snowassumethatthe"Log3.txt"fileisupdated,andthattennewlinesareaddedtothelogfile.Atthismoment,thelogfilesandtheinformationstoredinthecheckpointfilewilllooklikethis:

LogFiles CheckpointfileLog1.txt,50lines Log1.txt,line50Log2.txt,100lines Log2.txt,line100Log3.txt,30lines Log3.txt,line20Log4.txt,30lines Log4.txt,line30Ifweexecuteagainthesamecommand,LogParserwillusethe"myCheckPoint.lpc"filetodeterminewheretostartparsingeachofthelogfiles,anditwillonlyparseandreturnthetennewlinesinthe"Log3.txt"file.Whenthecommandexecutioniscomplete,the"myCheckPoint.lpc"checkpointfileisupdatedtoreflectthenewpositionofthelastlineinthe"Log3.txt"file.

Ifnowanew"Log5.txt"fileiscreatedcontainingtenlines,thelogfilesandtheinformationstoredinthecheckpointfilewilllooklikethis:

LogFiles CheckpointfileLog1.txt,50lines Log1.txt,line50Log2.txt,100lines Log2.txt,line100Log3.txt,30lines Log3.txt,line30Log4.txt,30lines Log4.txt,line30Log5.txt,10lines notrecordedIfweexecuteagainthecommand,LogParserwillonlyparsethenew"Log5.txt"file,returningitstenlines.

Asanotherexampleshowinghowthecheckpointfileisupdated,let'sassumenowthatthe"Log2.txt"fileisdeleted.Thelogfilesandtheinformationstoredinthecheckpointfilewillnowlooklikethis:

LogFiles CheckpointfileLog1.txt,50lines Log1.txt,line50

non-existing Log2.txt,line100Log3.txt,30lines Log3.txt,line30Log4.txt,30lines Log4.txt,line30Log5.txt,10lines Log5.txt,line10Whenweexecutethecommand,LogParserwilldetectthattherearenonewentriestoparse,anditwillreturnnorecords.However,uponupdatingthecheckpointfile,itwilldeterminethatthe"Log2.txt"filedoesn'texistanymore,anditwillremovealltheinformationassociatedwiththelogfilefromthecheckpointfile,whichwillnowlooklikethis:

LogFiles CheckpointfileLog1.txt,50lines Log1.txt,line50Log3.txt,30lines Log3.txt,line30Log4.txt,30lines Log4.txt,line30Log5.txt,10lines Log5.txt,line10Atthismomentthecheckpointfiledoesnotcontainanymoreinformationonthe"Log2.txt"file;shouldanew"Log2.txt"fileappearagainforanyreason,asubsequentcommandwouldtreatthefileasanewfile,andallofitsentrieswouldbeparsedfromthebeginningofthefile.

Asalastexample,let'snowassumethatthe"Log1.txt"fileisupdated,butthistimeitssizeshrinksanditendsupcontainingtenlinesonly.Thelogfilesandtheinformationstoredinthecheckpointfilewillnowlooklikethis:

LogFiles CheckpointfileLog1.txt,10lines Log1.txt,line50Log3.txt,30lines Log3.txt,line30Log4.txt,30lines Log4.txt,line30Log5.txt,10lines Log5.txt,line10Whenweexecutethecommand,LogParserwilldetectthatthesizeofthe"Log1.txt"filehaschanged,butinsteadofgrowinglarger,thefileisactuallysmaller.Inthissituation,LogParserassumesthatthefilehasbeenreplacedwithanewone,anditwillparseitasifitwasanewfile,returningallofitstenentries.Afterthecommandexecutioniscomplete,the"myCheckPoint.lpc"

checkpointfileisupdatedtoreflectthenewsituation,andthelogfilesandtheinformationstoredinthecheckpointfilewilllooklikethis:

LogFiles CheckpointfileLog1.txt,10lines Log1.txt,line10Log3.txt,30lines Log3.txt,line30Log4.txt,30lines Log4.txt,line30Log5.txt,10lines Log5.txt,line10

IncrementalParsingandAggregatedDataIt'simportanttonotethatthecheckpointfileonlyrecordsinformationaboutthefilesbeingparsed;itdoesnotrecordinformationaboutthequerybeingexecuted.Inotherwords,whenweexecuteaquerymultipletimesonasetofgrowingfilesusingacheckpointfile,eachtimethequeryresultsarecalculatedonthenewentriesonly.Thismeansthatqueriesusingaggregateddataneedtobehandledcarefullywhenusedwithcheckpointfiles.

Asanexample,consideragainthefourtextfilesinthefirstscenarioabove,andthefollowingcommand:

logparser"SELECTCOUNT(*)ASTotalFROMMyLogs\*.*"-i:TEXTLINE-iCheckPoint:myCheckPoint.lpcWhenthecommandisexecutedforthefirsttime,the"Total"fieldintheoutputrecordreturnedbythequerywillbeequalto200,thatis,thetotalnumberoflinesinthefourlogfiles.Asinthefirstexample,let'snowassumethatthe"Log3.txt"fileisupdated,andthattennewlinesareaddedtothelogfile.Whenweexecutethecommandagain,the"Total"fieldintheoutputrecordreturnedbythequerywillbenowequalto10,thetotalnumberofnewlinesinthefourlogfiles,andnotto210,asonewouldexpectfromthetotalnumberofrows.

Incaseswhereitisdesirabletocalculateaggregateddataacrossmultipleexecutionsofthesamequerywhenusingincrementalparsing,apossiblesolutionistosavethepartialresultsofeachquerytotemporaryfiles,andthenaggregateallthepartialresultswithanadditionalstep.Usingtheexampleabove,wecouldsavetheresultofthefirstquery("200")tothe"FirstResults.csv"file,andtheresultofthesecondquery("10")tothe"LastResults.csv"file.Thetwofilescouldthenbeconsolidatedintoasinglefilewithacommandlikethis:

logparser"SELECTSUM(Total)FROMFirstResults.csv,LastResults.csv"-i:CSV

MultiplexingOutputRecordsManyLogParseroutputformatsallowtheusertospecifymultiplefilesasthetargettowhichoutputrecordsarewrittento.Thisisachievedbyusing'*'wildcardcharactersinthefilenamespecifiedintheINTOclause;duringtheexecutionofthequery,thefirstfieldsineachoutputrecordsubstitutethewildcardcharacterstodeterminetheresultingfilenametowhichtheoutputrecordswiththeremainingfieldsarewritten.Inotherwords,thisfeatureallowsoutputrecordstobemultiplexedtodifferenttargetfilesdependingonthevaluesofthefirstfieldsintheoutputrecord.

Tomakeanexample,let'sassumethatwewanttoquerytheWindowsEventLog,andforeacheventsourcename,wewanttocreateaCSVtextfilecontainingallthedistincteventID'sgeneratedbythatsourcename.Thecommandwouldlooklikethefollowingexample:

LogParser"SELECTDISTINCTSourceName,EventIDINTOEvent_*.csvFROMSystem"-i:EVT-o:CSVForeachoutputrecordgeneratedbythisquery,the"SourceName"fieldwillbeusedtosubstitutethewildcardinthetargetfilename,andthe"EventID"fieldwillbewrittentotheCSVfilewiththeresultingfilename.Afterthecommandexecutioniscomplete,wewillhaveasmanyCSVoutputfilesasthenumberofdifferenteventsourcenames:

C:\>dirVolumeindriveChasnolabel.VolumeSerialNumberis49B5-4736

DirectoryofC:

07/19/200408:56AM<DIR>.07/19/200408:56AM<DIR>..07/19/200408:56AM13Event_ApplicationPopup.csv

EachCSVfilewillcontainthedistincteventID'sgeneratedbytheeventsource:

C:\>typeEvent_Tcpip.csvEventID42014202Thereisnolimitonthenumberofwildcardcharactersthatcanbeusedinthetargetfilenames.Wecanmodifytheexampleabovetogenerateadirectoryforeachevent

07/19/200408:56AM14Event_AtiHotKeyPoller.csv07/19/200408:56AM23Event_DCOM.csv07/19/200408:56AM33Event_Dhcp.csv07/19/200408:56AM23Event_DnsApi.csv07/19/200408:56AM27Event_EventLog.csv07/19/200408:56AM12Event_GEMPCC.csv07/19/200408:56AM13Event_i8042prt.csv07/19/200408:56AM16Event_Kerberos.csv07/19/200408:56AM15Event_NETLOGON.csv07/19/200408:56AM15Event_NtServicePack.csv07/19/200408:56AM13Event_Print.csv07/19/200408:56AM23Event_RemoteAccess.csv07/19/200408:56AM14Event_SCardSvr.csv07/19/200408:56AM39Event_ServiceControlManager.csv07/19/200408:56AM21Event_Tcpip.csv07/19/200408:56AM29Event_W32Time.csv07/19/200408:56AM14Event_Win32k.csv07/19/200408:56AM15Event_Workstation.csv19File(s)372bytes2Dir(s)34,340,712,448bytesfree

sourcename,andforeacheventIDgeneratedbythesource,aCSVfilecontainingthenumberofeventsloggedwiththatID:

LogParser"SELECTSourceName,EventID,COUNT(*)ASTotalINTO*\ID_*.csvFROMSystemGROUPBYSourceName,EventID"-i:EVT-o:CSVAfterthecommandexecutioniscomplete,wewillhaveasmanydirectoriesasthenumberofdifferenteventsourcenames:

C:\>dirVolumeindriveChasnolabel.VolumeSerialNumberis49B5-4736

DirectoryofC:

07/19/200409:08AM<DIR>.07/19/200409:08AM<DIR>..07/19/200409:08AM<DIR>ApplicationPopup07/19/200409:08AM<DIR>AtiHotKeyPoller07/19/200409:08AM<DIR>DCOM07/19/200409:08AM<DIR>Dhcp07/19/200409:08AM<DIR>DnsApi07/19/200409:08AM<DIR>EventLog07/19/200409:08AM<DIR>GEMPCC07/19/200409:08AM<DIR>i8042prt07/19/200409:08AM<DIR>Kerberos07/19/200409:08AM<DIR>NETLOGON07/19/200409:08AM<DIR>NtServicePack07/19/200409:08AM<DIR>Print07/19/200409:08AM<DIR>RemoteAccess07/19/200409:08AM<DIR>SCardSvr07/19/200409:08AM<DIR>ServiceControlManager07/19/200409:08AM<DIR>Tcpip07/19/200409:08AM<DIR>W32Time07/19/200409:08AM<DIR>Win32k07/19/200409:08AM<DIR>Workstation0File(s)0bytes21Dir(s)34,340,712,448bytesfree

EachdirectorywillcontainasmanyCSVoutputfilesasthenumberofdifferenteventID'sloggedbytheeventsource:

C:\>dirDCOMVolumeindriveChasnolabel.VolumeSerialNumberis49B5-4736

DirectoryofC:\DCOM

07/19/200409:08AM<DIR>.07/19/200409:08AM<DIR>..07/19/200409:08AM10ID_10002.csv07/19/200409:08AM10ID_10010.csv

EachCSVoutputfilewillcontainthenumberofeventsloggedwiththeeventID:

C:\>typeDCOM\ID_10010.csvTotal2Followingisalistoftheoutputformatsthatsupportthe"multiplex"feature:

CSVTSVXMLW3CIISTPL

ConvertingFileFormatsConvertingalogfilefromoneformattoanothercanbeeasilyaccomplishedwithLogParserbyexecutingacommandwiththefollowingcharacteristics:

Theinputformatchosenforthecommandshouldmatchtheconversionsourceformat;Theoutputformatchosenforthecommandshouldmatchtheconversiontargetformat;ThequeryshouldcontainaSELECTclausethatperformsthenecessarymodificationsontheinputformatfieldnamesandvaluesinordertomatchtherequirementsofthetargetformat.

WhenusingLogParsertoconvertonelogfileformattoanother,weshouldpaycloseattentiontotheorderandnamesofthefieldsintheinputandoutputformats.Someoutputformats,suchastheIISoutputformat,havefixedfields.WhenconvertingtoIISlogformat,inputformatfieldsshouldbeselectedtomatchtheIISformatexactly.Forexample,whenconvertingaW3CExtendedlogfiletoIISlogformat,weshouldselecttheclientIPaddressfirst,theusernamenext,andsoon.

Inaddition,wemightwanttochangethenameofthefieldsthatweextractfromtheinputformat.Forexample,whenwritingtoaW3CExtendedformatlogfile,LogParserretrievesthenamestobewrittentothe"#Fields"directivefromtheSELECTclause.IfweretrievedatafromanIISlogformatfile,thesenamesarenotthesameasthoseusedbytheW3CExtendedformat,sowemustaliaseveryfieldinordertogetthecorrectfieldname.

Asanexample,considerthefollowingSELECTclausethatconvertsIISlogformatfilestoIISW3CExtendedlogformat:

SELECTTO_DATE(TO_UTCTIME(TO_TIMESTAMP(Date,Time)))ASdate,TO_TIME(TO_UTCTIME(TO_TIMESTAMP(Date,Time)))AStime,ServiceInstanceASs-sitename,WecanseethattheindividualfieldshavebeenrenamedaccordingtotheW3CExtendedconvention,sothattheoutputfileisfullycompliantwith

HostNameASs-computername,ServerIPASs-ip,RequestTypeAScs-method,REPLACE_CHR(Target,'\u0009\u000a\u000d','+')AScs-uri-stem,ParametersAScs-uri-query,UserNameAScs-username,UserIPASc-ip,StatusCodeASsc-status,Win32StatusCodeASsc-win32-status,BytesSentASsc-bytes,BytesReceivedAScs-bytes,TimeTakenAStime-taken

theIISW3CExtendedformat.Inaddition,the"date"and"time"fieldsareconvertedfromlocaltime,whichisusedintheIISlogformat,toUTCtime,whichisusedintheW3CExtendedlogformat.

Thecommand-lineLogParserexecutablecanbeusedtorunbuilt-inqueriesthatperformconversionsbetweenthefollowingformats:

BINtoW3CIIStoW3CBINtoIISIISW3CtoIIS

Formoreinformation,refertotheCommand-LineOperationreference.

CustomPluginsLogParserallowsuserstodevelopcustominputformatsandusethemwithboththecommand-lineLogParserexecutableandwiththeLogParserscriptableCOMcomponents.

Thereisnorequirementonthelanguagethatcanbeusedtoimplementacustominputformat;forexample,custominputformatscanbeimplementedusinganyofthefollowinglanguages:

C++C#VisualBasic®JScript®orVBScript

CustominputformatsaredevelopedasCOMobjectsimplementingthemethodsoftheILogParserInputContextCOMinterface.TherearetwowaystowriteaCOMobjectthatimplementsthemethodsofthisinterface:implementingtheILogParserInputContextinterfacedirectly,orimplementingtheIDispatch(Automation)interfaceexposingthemethodsoftheILogParserInputContextinterface.

ImplementingtheILogParserInputContextInterfaceDirectlyWiththismethod,aLogParsercustominputformatCOMobjectmustimplementtheILogParserInputContextinterfacedirectly.ThismethodusuallyrequireswritingC++orVisualBasiccode.

ImplementingtheIDispatchInterfaceExposingtheILogParserInputContextInterfaceMethodsWiththismethod,aLogParsercustominputformatCOMobjectmustimplementtheIDispatchinterface,andsupportthesamemethodsexposedbytheILogParserInputContextinterface.Thismethodusuallyrequireswritingscriptlets(.wsc)filesinJScriptorVBScript.COMinputformatpluginsthatimplementtheIDispatchinterfacecanalsosupportcustomproperties.

CustominputformatCOMobjectsmustberegisteredwiththeCOMinfrastructureinordertobeaccessiblebyLogParser.Thistaskcanbeusuallyachievedusingtheregsvr32.exetooldistributedwiththeWindowsOS.ThefollowingcommandregistersacustominputformatCOMobjectimplementedasadynamiclinklibrary(dll):

C:\>regsvr32myinputformat.dll

ThefollowingcommandregistersacustominputformatCOMobjectimplementedasascriptletJScriptorVBScriptfile:

C:\>regsvr32myinputformat.wsc

OncedevelopedandregisteredwiththeCOMinfrastructure,custominputformatscanbeusedwitheitherthecommand-lineLogParserexecutable,orwiththeLogParserscriptableCOMcomponents.

UsingCustomInputFormatswiththeCommand-LineLogParserExecutableWiththecommand-lineLogParserexecutable,custominputformatsareusedthroughtheCOMinputformat,whichallowsuserstospecifytheProgIDofthecustomCOMobjectandeventualrun-timeproperties.

Asanexample,let'sassumethatwehavejustdevelopedacustominputformat,andthatitsProgIDis"MySample.MyInputFormat".WiththeCOMinputformat,thecustomCOMobjectcanbeusedasfollows:

C:\>logparser"SELECT*FROMinputfile"-i:COM-iProgID:MySample.MyInputFormatIntheexampleabove,"inputfile"standsforthespecificfrom-entityrecognizedbythecustominputformat.

IfweimplementedourCOMobjectthroughanAutomationinterface,wecouldalsohaveourobjectsupportcustomproperties,andsetthemthroughtheCOMinputformatasshowninthefollowingexample:

C:\>logparser"SELECT*FROMinputfile"-i:COM-iProgID:MySample.MyInputFormat-iCOMParams:ExtendedFields=onFormoreinformationontheCOMinputformat,refertotheCOMInputFormatreference.

UsingCustomInputFormatswiththeLogParserScriptableCOMComponentsWiththeLogParserscriptableCOMcomponents,custominputformatobjectsarepassedastheinputFormatargumenttotheExecuteorExecuteBatchmethodsoftheLogQueryobject.

ThefollowingVBScriptexampleshowshowour"MySample.MyInputFormat"customCOMobjectcanbeusedwiththeLogParserscriptableCOMcomponents:

DimoLogQueryDimoMyInputFormatDimoCSVOutputFormatDimstrQuery

SetoLogQuery=CreateObject("MSUtil.LogQuery")

'CreateourcustomInputFormatobjectSetoMyInputFormat=CreateObject("MySample.MyInputFormat")

'CreateOutputFormatobjectSetoCSVOutputFormat=CreateObject("MSUtil.LogQuery.CSVOutputFormat")oCSVOutputFormat.tabs=TRUE

'CreatequerytextstrQuery="SELECTTimeGenerated,EventIDINTOC:\output.csvFROMSystem"strQuery=strQuery&"WHERESourceName='ApplicationPopup'"

'ExecutequeryoLogQuery.ExecuteBatchstrQuery,oMyInputFormat,oCSVOutputFormat

FormoreinformationontheLogParserscriptableCOMcomponents,seeLogParserCOMAPIOverview,andCOMAPIReference.

CustomInputFormatSamplesLogParsercomeswiththreecustominputformatsamples,locatedinthe"Samples\COM"folder:

Processes:thissampleshowshowtowriteacustominputformatusingtheC++language;BooksXML:thissampleshowshowtowriteacustominputformatthatparsesXMLdocuments,usingtheC#language;QFE:thissampleshowshowtowriteacustominputformatthatreturnsinformationgatheredthroughaWMIquery,usingtheVBScriptlanguage.

FormoreinformationoncustominputformatpluginsandtheILogParserInputContextinterface,refertotheCOMInputFormatPluginsreference.

LogParserCOMAPIOverviewTheLogParserscriptableCOMcomponentsoffernumerousadvantagesandmoreflexibilitythanthecommand-lineexecutablebinary.Forexample,withtheLogParserscriptableCOMcomponentswecanexecuteaquerywithoutprovidinganoutputformat,retrievetheresultoutputrecords,andprocesstheoutputrecordsourselves.

TheLogParserscriptableCOMcomponentsareimplementedasAutomationobjects,whichmeansthattheycanbeusedfromanyprogrammingenvironmentsupportingautomation,includingC++,C#,VisualBasic,JScriptandVBScript.

Tip:BeforeusingtheLogParserscriptableCOMcomponentsonacomputer,the"LogParser.dll"binaryshouldberegisteredwiththecomputer'sCOMinfrastructurebyexecutingthefollowingcommandinthedirectorycontainingthe"LogParser.dll"binary:C:\LogParser>regsvr32LogParser.dll

TheLogParserscriptableCOMcomponentsarchitectureismadeupofthefollowingobjects:

MSUtil.LogQueryobject:thisisthemainCOMobjectintheLogParserscriptableCOMcomponentsarchitecture;itexposesthemainAPImethodsandprovidesaccesstootherobjectsinthearchitecture.InputFormatobjects:theseobjectsprovideprogrammaticaccesstotheinputformatssupportedbyLogParser;eachinputformatobjectexposespropertieshavingthesamenameastheparametersofthecorrespondingLogParserinputformat.OutputFormatobjects:theseobjectsprovideprogrammaticaccesstotheoutputformatssupportedbyLogParser;eachoutputformatobjectexposespropertieshavingthesamenameastheparametersofthecorrespondingLogParseroutputformat.

WhenwritinganapplicationthatusestheLogParserscriptableCOMcomponents,theveryfirststepshouldbetheinstantiationoftheMSUtil.LogQueryCOMobject.ThefollowingJScriptexampleshowshowtheMSUtil.LogQueryobjectis

instantiatedbyaJScriptapplication:

varoLogQuery=newActiveXObject("MSUtil.LogQuery");

ThefollowingVBScriptexampleshowshowtheMSUtil.LogQueryobjectisinstantiatedbyaVBScriptapplication:

DimoLogQuerySetoLogQuery=CreateObject("MSUtil.LogQuery")OncetheMSUtil.LogQueryCOMobjecthasbeeninstantiated,anapplicationwouldusuallyproceedbyexecutingaqueryineitherbatchmodeorinteractivemode,dependingonthetaskthatneedstobeaccomplished.

BatchModeAqueryexecutedinbatchmodewillhaveitsoutputrecordswrittendirectlytoanoutputformat.BatchmodeworksinthesamewayasthecommandsusedwiththeLogParsercommand-lineexecutable,anditisusefulwhenwewanttoexecuteaqueryandhaveitsresultssenttoanoutputformat,withnoapplicationinterventiononthequeryoutputrecords.

AqueryisexecutedinbatchmodebycallingtheExecuteBatchmethodoftheMSUtil.LogQueryobject.Thismethodtakesthreearguments:

ThetextoftheSQL-Likequery;Aninputformatobject;Anoutputformatobject.

ThebasicstepsofanapplicationusingbatchmoderesemblethecommandsusedwiththeLogParsercommand-lineexecutable:

1. InstantiatetheMSUtil.LogQueryobject;2. Instantiatetheinputformatobjectcorrespondingtotheinput

formatchosenforthequery;3. Ifneeded,setinputformatobjectpropertiestochangethe

defaultbehavioroftheinputformat;4. Instantiatetheoutputformatobjectcorrespondingtothe

outputformatchosenforthequery;5. Ifneeded,setoutputformatobjectpropertiestochangethe

defaultbehavioroftheoutputformat;6. CalltheExecuteBatchmethodoftheMSUtil.LogQuery

object,specifyingthequerytext,theinputformatobject,andtheoutputformatobject.

ThefollowingexamplesshowasimpleapplicationthatcreatesaCSVfile

containingselectedrecordsfromtheeventlog.AfterinstantiatingthemainMSUtil.LogQueryobject,theapplicationinstantiatestheMSUtil.EVTInputFormatinputformatobject,whichimplementstheEVTinputformat,andsetsitsdirectionpropertyto"BW",inordertoreadeventsfromthelatesttotheearliest.Then,theapplicationinstantiatestheMSUtil.CSVOutputFormatoutputformatobject,whichimplementstheCSVoutputformat,andsetsitstabspropertyto"ON",inordertoimprovereadabilityoftheCSVfile.Finally,theapplicationcallstheExecuteBatchmethodoftheMSUtil.LogQueryobject,specifyingthequery,theinputformatobject,andtheoutputformatobject;themethodwillexecutethequery,readingfromtheeventlogandwritingtothespecifiedCSVfile,andwillreturnwhenthequeryexecutioniscomplete.

JScriptexample:

//CreateInputFormatobjectvaroEVTInputFormat=newActiveXObject("MSUtil.LogQuery.EventLogInputFormat");oEVTInputFormat.direction="BW";

//CreateOutputFormatobjectvaroCSVOutputFormat=newActiveXObject("MSUtil.LogQuery.CSVOutputFormat");oCSVOutputFormat.tabs=true;

//CreatequerytextvarstrQuery="SELECTTimeGenerated,EventIDINTOC:\\output.csvFROMSystem";strQuery+="WHERESourceName='ApplicationPopup'";

//ExecutequeryoLogQuery.ExecuteBatch(strQuery,oEVTInputFormat,oCSVOutputFormat);

VBScriptexample:

DimoLogQueryDimoEVTInputFormatDimoCSVOutputFormatDimstrQuery

'CreateInputFormatobjectSetoEVTInputFormat=CreateObject("MSUtil.LogQuery.EventLogInputFormat")oEVTInputFormat.direction="BW"

'CreateOutputFormatobjectSetoCSVOutputFormat=CreateObject("MSUtil.LogQuery.CSVOutputFormat")oCSVOutputFormat.tabs=TRUE

'ExecutequeryoLogQuery.ExecuteBatchstrQuery,oEVTInputFormat,oCSVOutputFormat

InteractiveModeQueriesexecutedininteractivemodedonotuseoutputformats,butratherreturntheiroutputrecordsdirectlytotheapplication.Interactivemodeisusefulwhenwewanttoexecuteaqueryandreceivetheoutputrecordsforcustomprocessing.

AqueryisexecutedininteractivemodebycallingtheExecutemethodoftheMSUtil.LogQueryobject.Thismethodtakestwoarguments:

ThetextoftheSQL-Likequery;Aninputformatobject.

TheExecutemethodreturnsaLogRecordSetobject.TheLogRecordSetobjectisanenumeratorofLogRecordobjects;itallowsanapplicationtonavigatethroughthequeryoutputrecords.EachLogRecordobjectrepresentsasinglequeryoutputrecord,anditexposesmethodsthatcanbeusedtoretrieveindividualfieldvaluesfromtheoutputrecord.

Thebasicstepsofanapplicationusinginteractivemodeare:

1. InstantiatetheMSUtil.LogQueryobject;2. Instantiatetheinputformatobjectcorrespondingtotheinput

formatchosenforthequery;3. Ifneeded,setinputformatobjectpropertiestochangethe

defaultbehavioroftheinputformat;4. CalltheExecutemethodoftheMSUtil.LogQueryobject,

specifyingthequerytextandtheinputformatobject,andreceivingaLogRecordSetobject;

5. EnteraloopthatusestheatEnd,getRecord,andmoveNextmethodsoftheLogRecordSetobjecttoenumeratetheLogRecordqueryresultobjects;

6. ForeachLogRecordobject,accessitsfieldvaluesusingthegetValuemethodoftheLogRecordobject,andprocessthe

fieldvaluesasneeded;7. Whenfinished,disposeoftheLogRecordSetobjectby

callingitsclosemethod.

ThefollowingexamplesshowasimpleapplicationparsinganIISwebsite'slogsandprintingtheoutputrecordstotheconsoleoutput.AfterinstantiatingthemainMSUtil.LogQueryobject,theapplicationinstantiatestheMSUtil.IISW3CInputFormatinputformatobject,whichimplementstheIISW3Cinputformat.Then,theapplicationcallstheExecutemethodoftheMSUtil.LogQueryobject,specifyingthequeryandtheinputformatobject,andreceivingtheresultingLogRecordSetobject.TheLogRecordSetobjectisusedinalooptoenumeratetheLogRecordobjectsimplementingthequeryoutputrecords;theapplicationretrievesthefirstfieldfromeachLogRecordobjectandprintsittotheconsoleoutput.Finally,theapplicationdisposesoftheLogRecordSetobjectbycallingitsclosemethod.

JScriptexample:

//CreateInputFormatobjectvaroIISW3CInputFormat=newActiveXObject("MSUtil.LogQuery.IISW3CInputFormat");

//CreatequerytextvarstrQuery="SELECTc-ipFROM<1>WHEREcs-uri-stemLIKE'%hitcount.asp'";

//ExecutequeryandreceiveaLogRecordSetvaroRecordSet=oLogQuery.Execute(strQuery,oIISW3CInputFormat);

//Visitallrecordswhile(!oRecordSet.atEnd())

VBScriptexample:

DimoLogQueryDimoIISW3CInputFormatDimstrQueryDimoRecordSetDimoRecordDimstrClientIp

'CreateInputFormatobjectSetoIISW3CInputFormat=CreateObject("MSUtil.LogQuery.IISW3CInputFo

{ //Getarecord varoRecord=oRecordSet.getRecord();

//Getfirstfieldvalue varstrClientIp=oRecord.getValue(0);

//Printfieldvalue WScript.Echo("ClientIPAddress:"+strClientIp);

//AdvanceLogRecordSettonextrecord oRecordSet.moveNext();}

//CloseLogRecordSetoRecordSet.close();

rmat")

'CreatequerytextstrQuery="SELECTc-ipFROM<1>WHEREcs-uri-stemLIKE'%hitcount.asp'"

'ExecutequeryandreceiveaLogRecordSetSetoRecordSet=oLogQuery.Execute(strQuery,oIISW3CInputFormat)

'VisitallrecordsDOWHILENOToRecordSet.atEnd

'Getarecord SetoRecord=oRecordSet.getRecord

'Getfirstfieldvalue strClientIp=oRecord.getValue(0)

'Printfieldvalue WScript.Echo"ClientIPAddress:"&strClientIp

'AdvanceLogRecordSettonextrecord oRecordSet.moveNext

'CloseRecordSetoRecordSet.close

C#ExampleTheLogParserscriptableCOMcomponentscanbeeasilyconsumedby.NETapplicationsusingtheCOMinteropfeatureofthe.NETFramework.

TheCOMinteropfeatureofthe.NETframeworkallowsuserstoinstantiateanduseCOMobjectsthroughtheuseofRuntimeCallableWrappers(RCW).TheRCWisa.NETclassthatwrapsaCOMobjectandgivesa.NETapplicationthenotionthatit'sinteractingwithamanaged.NETcomponent.RCW'sarecreatedbyeitherusingtheTypeLibraryImporter(tlbimp.exe)tool,orbyimportingareferencetotheLogParserscriptableCOMobjectsthroughtheMicrosoftVisualStudio®.NETuserinterface.Ineithercase,theRCW'saregeneratedandstoredinanassemblynamed"Interop.MSUtil.dll",whichcontainsRuntimeCallableWrappersforalloftheLogParserscriptableCOMcomponents.Byreferencingthisassembly,our.NETapplicationscanusetheLogParserscriptableCOMcomponentsasiftheyweremanaged.NETcomponents.

ThefollowingexampleC#applicationexecutesaLogParserquerythatreturnsthelatest50eventsfromtheSystemeventlog,printingthequeryresultstotheconsoleoutput:

usingSystem;usingLogQuery=Interop.MSUtil.LogQueryClassClass;usingEventLogInputFormat=Interop.MSUtil.COMEventLogInputContextClassClass;usingLogRecordSet=Interop.MSUtil.ILogRecordset;

classLogParserSample{publicstaticvoidMain(string[]Args){try{//InstantiatetheLogQueryobject

Thefollowingstepsdescribehowtobuildthissampleapplication:

1. BuildaninteropassemblycontainingtheRuntimeCallableWrappersfortheLogParserscriptableCOMcomponents.Thisstepcanbyexecutedintwodifferentways:FromwithinaVisualStudio.NETproject,importareferencetotheLogParserscriptableCOMcomponents;Fromacommand-lineshell,executethetlbimp.exetool(generallyavailableinthe"Bin"folderofthe.NETframeworkSDK),specifyingthepathtotheLogParser.dllbinary:

LogQueryoLogQuery=newLogQuery();

//InstantiatetheEventLogInputFormatobjectEventLogInputFormatoEVTInputFormat=newEventLogInputFormat();

//Setits"direction"parameterto"BW"oEVTInputFormat.direction="BW";

//Createthequerystringquery=@"SELECTTOP50SourceName,EventID,MessageFROMSystem";

//ExecutethequeryLogRecordSetoRecordSet=oLogQuery.Execute(query,oEVTInputFormat);

//Browsetherecordsetfor(;!oRecordSet.atEnd();oRecordSet.moveNext()){Console.WriteLine(oRecordSet.getRecord().toNativeString(","));}

//ClosetherecordsetoRecordSet.close();}catch(System.Runtime.InteropServices.COMExceptionexc){Console.WriteLine("Unexpectederror:"+exc.Message);}}}

C:\>tlbimpLogParser.dll/out:Interop.MSUtil.dll

Ineithercase,anassemblynamed"Interop.MSUtil.dll"iscreated.

2. Compilethesamplesourcefileintoanexecutable,referencingthenewlycreated"Interop.MSUtil.dll"assembly.Fromacommand-lineshell,thisstepcanbeexecutedasfollows:

C:\>csc/r:Interop.MSUtil.dll/out:Events.exesample.cs

SecurityConsiderationsWhenusinginputandoutputformatstoretrieveandsenddataoverthenetwork,usersshouldbeawarethatmostoftheprotocolsutilizedfordatatransfer(e.g.SMB,HTTP,andSYSLOG)donotmakeuseofencryption,andcouldthusbevulnerabletointerceptionandtamperingbymaliciousentities.Inordertoprovideasecureenvironmentinwhichthesenetworkconnectionsarelessvulnerabletointerception,usersshouldimplementtheIPSecprotocolontheirnetworks,and/oruseSSLHTTPconnectionswhenretrievingdatafromaWebURL.WhenusingtheIncrementalParsingfeature,usersshouldstoretheircheckpointfilesinasecurelocation,andverifythatcheckpointfileshaveproperACL's(AccessControlLists)preventingmaliciousentitiesfromtamperingwiththedatathattheLogParserinputformatsstoreinthecheckpointfiles.WhenimplementingcustominputformatCOMobjects,usersshouldensurethattheobjectsarenotaccessiblefromlocalandremotelow-privilegedusers,inordertopreventmaliciousentitiesfrominstantiatingandusingthecustominputformatobjectsfromthelocalcomputerorfromaremotecomputer.Inordertodenyaccesstolow-privilegedusers,eithersetproperACL'sonthecustominputformatCOMobjects'binaries,orusethe"DCOMConfiguration"ManagementConsole(availableinthe"AdministrativeTools"folderunderthe"ComponentServices"managementconsole)toexplicitlyallowselectedusersonlylocalaccesstoyourcustominputformatCOMobjects.WhenusingtheSQLoutputformat,usersshouldbeawarethattheODBCconnectionpropertiesprovidedthroughtheSQLoutputformatparameters,whichincludeusernameandpassword,couldbetransmittedoverthenetworkincleartext.Inaddition,thedatatransmittedthroughtheODBCconnectioncouldbeunencryptedandthusvulnerabletointerceptionandtamperingbymaliciousentities.Inordertoprovideamoresecureenvironment,usersshouldcreateaDataSourceName(DSN)onthelocalcomputerspecifyingtheconnectionpropertiestousefortheconnectiontothedatabase,and

specifythenameoftheDataSourceasavaluetothedsnparameteroftheSQLoutputformat.UsingaDataSourceNamefortheconnectionprovidesthefollowingbenefits:TheusernameandpasswordfortheconnectionarestoredsecurelybytheODBCsubsystem;

CertainODBCdrivers,includingMicrosoftSQLServerTMODBCdriversandMicrosoftAccessODBCdrivers,provideanoptionthatallowsuserstoenableencryptionofthenetworktrafficbetweentheODBCconnectionendpoints.

FormoreinformationonsecuringthecommunicationbetweentheODBCconnectionsendpoints,seetheMSDN®DataAccessSecuritytopic.Whenprocessingsensitiveorconfidentialdata,usersshouldprovideproperACL'sonthefilesgeneratedbytheoutputformatsoronthedirectoriesinwhichtheoutputformatsgeneratefiles,inordertopreventmaliciousentitiesfromaccessingand/ortamperingwiththeoutputdatageneratedbyaquery.

FrequentlyAskedQuestions1. HowdoIspecifyyesterday’sdate?2. HowdoIretrievetheeventlogsthathavebeenloggedinthe

past10minutes?3. AfterparsingmyIISlogfiles,Igetamessagesaying"There

havebeen4parseerrors."Whatcausesthis?4. HowdoIchangethecolumnnamesinmyoutputfile?5. HowdoIcombinetheIISW3C"date"and"time"fieldsintoa

singleTIMESTAMPfield?6. HowdoIsplitasingleTIMESTAMPfieldintoadate-onlyfield

andatime-onlyfield?7. WhenIusea"SELECT*"onanIISW3CExtendedlogfile,I

getmanyfieldswithNULLvalues.Whatcausesthis?8. Igetanerrorsaying"UnknownfieldXYZ"whenIexecutemy

query.HowdoIfixthis?9. IamtryingtowriteaquerythatusestheINoperator,butLog

Parserkeepsgivingmeerrors.WhatamIdoingwrong?10. WhenIexecutea"SELECT*"onalogfile,theoutputrecords

contain2extrafieldsthatIcannotfindinthelog.Whatarethesefields?

11. IamdevelopinganASPorASP.NetorScheduledTaskapplicationwithLogParser,andI'mhavingproblemswithpermissions.WhatcanIdo?

12. CanIusetheLogParserscriptableCOMcomponentsfromamulti-threadedapplication?

HowdoIspecifyyesterday’sdate?YouneedtousetheSUBfunctiontosubtractonedayfromthecurrentUTCtimestampreturnedbytheSYSTEM_TIMESTAMPfunction.

TheoriginforTIMESTAMPvaluesisJanuary1,year0at00:00:00.ThismeansthatatimespanofonedayisrepresentedbythetimestampforJanuary2,year0at00:00:00,i.e.24hoursaftertheoriginoftime.Usethefollowingfield-expressiontospecifyyesterday’sdate:

SUB(SYSTEM_TIMESTAMP(),TIMESTAMP('01-02','MM-dd'))

Formoreinformation,seetheTIMESTAMPReference.

HowdoIretrievetheeventlogsthathavebeenloggedinthepast10minutes?

YouneedtousetheSUBfunctiontosubtract10minutesfromthecurrentUTCtimestampreturnedbytheSYSTEM_TIMESTAMPfunction,andconvertthistimestamptolocaltimeusingtheTO_LOCALTIMEfunction:

SELECT*FROMSystemWHERETimeGenerated>=TO_LOCALTIME(SUB(SYSTEM_TIMESTAMP(),TIMESTAMP('10','mm')))AfterparsingmyIISlogfiles,Igetamessagesaying"Therehave

been4parseerrors."Whatcausesthis?Yourlogfilesaresomehowmalformed.Thismighthappen,forexample,ifaclientrequestsaURLorspecifiesausernamecontainingspaces.LogParsercannotprocessthatrowandskipsit.Toseeexactlywhat'sgoingon,setthe-eglobalswitchtoanyvaluegreaterthanorequaltozero.ThismakesLogParserstopthequeryexecutionwhenthatnumberofparseerrorsisencountered,anddumpallthemessagesoftheparseerrorsthatoccurred.Formoreinformation,seeErrors,ParseErrors,andWarnings.

HowdoIchangethecolumnnamesinmyoutputfile?UsetheASkeywordinyourSELECTclausetoaliasthefield.Forexample:

SELECTField1ASnewFieldName,Field2ASnewFieldName2,...

HowdoIcombinetheIISW3C"date"and"time"fieldsintoasingleTIMESTAMPfield?

UsetheTO_TIMESTAMPfunction,asinthefollowingexample:

SELECTTO_TIMESTAMP(date,time),...

HowdoIsplitasingleTIMESTAMPfieldintoadate-onlyfieldandatime-onlyfield?

UsetheTO_DATEandTO_TIMEfunctions,asinthefollowingexample:

SELECTTO_DATE(myTimestamp),TO_TIME(myTimestamp),...

Formoreinformation,seetheTIMESTAMPReference.

WhenIusea"SELECT*"onanIISW3CExtendedlogfile,IgetmanyfieldswithNULLvalues.Whatcausesthis?

TheIISW3Cinputformathas32fields,whichareallthepossiblefieldsthatIIS5.0andIIS6.0canlog.IfyourWebServerisconfiguredtologonlyafewofthesefields,theIISW3CinputformatreturnstheotherfieldvaluesasNULLvalues.

Igetanerrorsaying"UnknownfieldXYZ"whenIexecutemyquery.HowdoIfixthis?

Ifyouhavenotspecifiedaninputformatforyourquery,LogParserchoosesoneautomaticallybasedonthe<from-entity>intheFROMclauseofyourquery.Insomecases,theinputformatmightnotbetheoneyouexpect.Tryspecifyingtheinputformatexplicitlyusingthe-iswitch.Ifyouhavespecifiedthecorrectinputformat,makesurethatyou

havetypedthefieldnamecorrectly.

IamtryingtowriteaquerythatusestheINoperator,butLogParserkeepsgivingmeerrors.WhatamIdoingwrong?

Makesureyouareseparatingthevaluesontheright-sideoftheINoperatorwiththecorrectseparator.IftheINoperatoriscomparingasinglefield-expressionwithalistofvalues,separatethevalueswithasemicolon(;),notwithacomma,asfollows:

WHEREMyFieldIN('VALUE1';'VALUE2';'VALUE3')

Differentvaluesforthesamefield-expression("value-rows")areseparatedbyasemicolon;commacharactersareusedtoseparatevalueswithinasinglevalue-row.Formoreinformation,seetheINOperatorReference.

WhenIexecutea"SELECT*"onalogfile,theoutputrecordscontain2extrafieldsthatIcannotfindinthelog.Whatarethesefields?

Mostoftheinputformatsaddsometrackingfieldstotheinputrecords,suchasthenameofthefilecurrentlyparsed,andtherownumbercurrentlyparsed.Ifyoudonotwantthesefieldstoappearinyouroutputrecords,donotuse"SELECT*".Instead,specifyonlythefieldnamesthatyouwant,asinthefollowingexample:

SELECTField1,Field2,Field3,....

IamdevelopinganASPorASP.NetorScheduledTaskapplicationwithLogParser,andI'mhavingproblemswithpermissions.WhatcanIdo?

ThefirststepintroubleshootingtheseproblemsisidentifyingtheaccountunderwhichLogParserisrunning.Ifyouaredevelopingan

ASPorASP.Netapplication,LogParserwillrunastheaccountoftheuserrequestingthepage.Iftherequestisanonymous,theaccountistheIISAnonymousaccount;iftherequestisauthenticated,theaccountistheauthenticateduser'saccount.IfyouaredevelopingaScheduledTaskapplication,theaccountistheaccountthatyouhavespecifiedforthetask.Oncetheaccounthasbeenidentified,appropriatepermissionsmustbegivenforthisaccounttoaccessboththeLogParserbinaryandtheDynamicLinkLibrariesthatLogParserdependsto,whichincludestandardWindowslibraries(e.g."kernel32.dll","user32.dll",etc.)andasignificantnumberofotherlibraries(e.g."WinInet.dll","odbcint.dll",etc.).Finally,appropriatepermissionsmustbegivenfortheaccounttoaccessthedatathatyourapplicationasksLogParsertoprocess.ThesemayincludeIISlogfiles,theEventLog,textfiles,andwhateverdatayouareprocessing.Note:ItisnotagoodsecuritypracticetochangesystemACL'sandpermissionstograntuseraccountsaccesstoprotectedsystemresources.Thisisespeciallytrueifyouaredevelopinganexternal-facingwebapplicationthatusesLogParsertodisplayinformationtotheusers.Inthesecases,considerinsteaddevelopingaScheduledTaskthatrunsundera"private"account,andthatgeneratesatfrequentintervalsthewebpagesthatyourapplicationwilldisplaytotheuser.

CanIusetheLogParserscriptableCOMcomponentsfromamulti-threadedapplication?

TheLogParserscriptableCOMcomponentsareregisteredtorunwithinasingle-threadedCOMapartment,meaningthattheobjectscanbeusedfrommultiplethreads,butcallstotheobjects'methodswillbeserializedbytheCOMinfrastructuretoguaranteethatonlyonethreadatatimecanaccessthecomponents.

QuerySyntax<query> ::= <select_clause>[<using_clause>]

[<into_clause>]<from_clause>[<where_clause>][<group_by_clause>][<having_clause>][<order_by_clause>]

Remarks:Aquerycanincludecomments,thatis,user-providedtextnotevaluatedbyLogParser,usedtodocumentcodeortemporarilydisablepartsofquerystatements.Formoreinformation,readtheCommentsReference.

Examples:

A.MinimalqueryThefollowingexampleshowstheminimalquerythatcanbewrittenwiththeLogParserSQL-Likelanguage,makinguseoftheSELECTandFROMclausesonly:

SELECTTimeGenerated,SourceNameFROMSystemB.CompletequeryThefollowingexampleshowsacompletequerythatmakesuseofalltheclausesintheLogParserSQL-Likelanguage:

SELECTTypeName,COUNT(*)ASTotalCountUSINGTO_UPPERCASE(EXTRACT_TOKEN(EventTypeName,0,''))ASTypeNameINTOReport.csvFROMSystemWHERETypeNameLIKE'%service%'GROUPBYTypeNameHAVINGTotalCount>5ORDERBYTotalCountDESC

Seealso:SELECTUSINGINTOFROMWHEREGROUPBYHAVINGORDERBY

Comments

SELECT<select_clause> ::= SELECT[TOP<integer>][DISTINCT|ALL

]<selection_list>

<selection_list> ::= <selection_list_el>[,<selection_list_el>...]

<selection_list_el> ::= <field_expr>[AS<alias>]*

TheSELECTclausespecifiesthefieldsoftheoutputrecordstobereturnedbythequery.

Arguments:

TOPnSpecifiesthatonlythefirstnrecordsaretobeoutputfromthequeryresultset.IfthequeryincludesanORDERBYclause,thefirstnrecordsorderedbytheORDERBYclauseareoutput.IfthequeryhasnoORDERBYclause,theorderoftherecordsisarbitrary.Formoreinformation,seeRetrievingaFixedNumberofRecords.

ALLSpecifiesthatduplicaterecordscanappearintheresultset.ALListhedefault.

DISTINCTSpecifiesthatonlyuniquerecordscanappearintheresultset.NULLvaluesareconsideredequalforthepurposesoftheDISTINCTkeyword.Formoreinformation,seeEliminatingDuplicateValues.

<selection_list>Thefieldstobeselectedfortheresultset.Theselectionlistisaseriesoffield-expressionsseparatedbycommas.

*Specifiesthatalltheinputrecordfieldsshouldbereturned.ThefieldsarereturnedintheorderinwhichtheyareexportedbytheInputFormat.

AS<alias>Specifiesanalternativenametoreplacethefieldnameinthequeryresultset.Bydefault,outputformatsthatdisplayfieldnamesusethetextofafield-expressionintheSELECTclauseasthenameofthecorrespondingoutputrecordfield.However,whenafield-expressionintheSELECTclausehasbeenaliased,outputformatswillusethealiasasthenameoftheoutputrecordfield.Thealiasofafield-expressioncanbealsousedanywhereelseinthequeryasashortcutthatreferstotheoriginalfield-expression.

Remarks:Whenafield-expressionisaliasedwithanaliasmatchinganinputrecordfieldname,thealiasingwillaffectthatfield-expressiononly;anyotheroccurrenceofthealiasinthequerywillresolvetotheinputrecordfieldname.Asanexample,theoutputrecordsofthefollowingqueryaremadeupoftwofieldswithanidenticalname("TimeGenerated");thefirstoutputrecordfieldwillcontainvaluesfromthealiasedfield-expression("ADD(EventID,1000)"),whilethesecondoutputrecordfieldwillcontainvaluesfromthe"TimeGenerated"inputformatfield:

SELECTADD(EventID,1000)ASTimeGenerated,TimeGeneratedFROMsystemAfield-expressionintheSELECTclausecanrefertoaliasesdefinedelsewhereintheSELECTclause,aslongasthedefinitionhappens

before(inaleft-to-rightorder)itsuse.ThefollowingexampleisacorrectSELECTclause:

SELECTEventIDASMyAlias,ADD(MyAlias,100)

Ontheotherhand,thefollowingexampleisnotacorrectSELECTclause,sincethe"MyAlias"aliasisusedbeforebeingdefined:

SELECTADD(MyAlias,100),EventIDASMyAlias

Examples:

A.SelectingspecificfieldsThefollowingqueryselectsasubsetofallthefieldsexportedbytheEVTInputFormat:

SELECTTimeGenerated,SourceNameFROMSystemB.Selectingspecificfieldsandfield-expressionsThefollowingqueryselectsaconstantandafunctionthatusesafieldexportedbytheEVTInputFormatasargument:

SELECT'EventType:',EXTRACT_TOKEN(EventTypeName,0,'')FROMSystemC.Selectingallfieldswith*ThefollowingqueryselectsallthefieldsexportedbytheEVTInputFormat:

SELECT*FROMSystemD.UsingTOPThefollowingqueryreturnsthe10mostrequestedUrl'sinthespecifiedIISW3Clogfile:

SELECTTOP10cs-uri-stem,COUNT(*)FROMex040305.logGROUPBYcs-uri-stemORDERBYCOUNT(*)DESCE.UsingDISTINCTThefollowingqueryusestheREGInputFormattoreturnalltheregistrykeyvaluetypesthatarefoundunderthespecifiedkey:

SELECTDISTINCTValueTypeFROM\HKLM\SYSTEM\CurrentControlSetF.Aliasingfield-expressionsThefollowingqueryreturnsabreakdownofpagerequestsperpagetypefromthespecifiedIISW3Clogfile:

SELECTTO_UPPERCASE(EXTRACT_EXTENSION(cs-uri-stem))ASPageType,COUNT(*)ASTotalHitsFROMex040305.logGROUPBYPageTypeORDERBYTotalHitsDESCSeealso:

FieldExpressionsFieldNamesandAliasesUSING

BasicsofaQueryEliminatingDuplicateValuesRetrievingaFixedNumberofRecords

USING<using_clause> ::= USING<field_expr>AS<alias>[,<field_expr>

AS<alias>...]

TheUSINGclausedeclaresaliasedfield-expressionsthatdonotappearintheoutputrecordsbutcanbereferencedanywhereinthequery.TheUSINGclauseisemployedtoimprovequeryreadability.

Remarks:Formoreinformationonaliasingfield-expressions,seetheSELECTClauseReference.

Examples:

A.Declaringaliasedfield-expressionsThefollowingexamplequeryreturnsthe"accountname"portionofthefully-qualifiedaccountnamethatappearsintheresolved"SID"fieldoftheEVTinputformat:

SELECTUsernameUSINGTO_LOWERCASE(RESOLVE_SID(Sid))ASFQAccount,EXTRACT_TOKEN(FQAccount,1,'\\')ASUsernameFROMSecurity

Seealso:FieldExpressionsFieldNamesandAliasesSELECT

ImprovingQueryReadability

INTO<into_clause> ::= INTO<into_entity>

TheINTOclauseisusedtospecifytheoutputformattarget(s)towhichthequeryoutputrecordsaretobewritten.

Remarks:Thesyntaxandinterpretationofthe<into_entity>specifiedintheINTOclausedependsontheoutputformatused.Forinformationonthesyntaxandinterpretationofthe<into_entity>valuessupportedbyeachoutputformat,refertotheOutputFormatsReference.Regardlessoftheoutputformatused,the<into_entity>specifiedintheINTOclausemustcomplywiththefollowinggeneralsyntax:The<into_entity>cannotcontainspaces,unlessitisenclosedbythe'''(singlequote)or'"(doublequotes)characters,asinthefollowingexample:

'C:\ProgramFiles\file3.txt'

Thefollowingcharactersareconsideredparenthesyscharacters,andiftheyappearinan<into_entity>,theymustappearaswell-formedpairsofopeningandclosingparenthesys:

<>()[]{}

Thefollowingexamplesshowvalidinto-entitiescontainingparenthesyscharacters:

entity<value>entity[value]valueThefollowingexamplesshowinvalidinto-entitiescontaining

parenthesyscharacters:

entity>value<entity}valueentity(valueAnycharacter(includingillegalcharactersandnon-printable

characters)inan<into-entity>canbeenteredusingthe\uxxxxnotation,wherexxxxisthe4-digithexadecimalrepresentationoftheUNICODEcharacter,asinthefollowingexample:

C:\Program\u0020Files\file3.txt

Into-entitiesthatrepresentnamesoffilesordirectoriesarenotallowedtocontainthefollowingcharacters,evenwhenenclosedinquotecharactersorenteredusingthe\uxxxxnotation:

tabcarriage-returnline-feed,()"<>

SincetheINTOclauseisnotamandatoryclauseintheLogParserSQL-Likelanguage,mostoutputformatsemploydefault<into_entity>valuesthatareimplicitlyusedwhenaquerydoesnotincludeanINTOclause.Forexample,theNAT,CSV,andTSVoutputformatsassumeSTDOUTwhenanINTOclauseisnotspecified.Formoreinformationonthedefault<into_entity>valuesassumedbyeachoutputformat,refertotheOutputFormatsReference.TheTOclauseusedbyearlierversionsofLogParserhasbeendeprecatedinfavoroftheINTOclause.

Examples:

A.Explicit<into_entity>ThefollowingexamplequeryspecifiesanexplicittargetCSVfilefortheCSVoutputformat:

SELECT*

INTOMyOutput.csvFROMSystemB.Implicit<into_entity>ThefollowingexamplequeryusesanimplicitSTDOUTtargetfortheNAToutputformat:

SELECT*FROMSystemC.Explicit<into_entity>ThefollowingexamplequeryspecifiesanexplicitSTDOUTtargetfortheNAToutputformat:

SELECT*INTOSTDOUTFROMSystem

Seealso:FROM

BasicsofaQueryOutputFormatsReference

FROM<from_clause> ::= FROM<from_entity>

TheFROMclauseisusedtospecifytheinputformatsource(s)fromwhichthequeryinputrecordsaretoberead.

Remarks:Thesyntaxandinterpretationofthe<from_entity>specifiedintheFROMclausedependsontheinputformatused.Forinformationonthesyntaxandinterpretationofthe<from_entity>valuessupportedbyeachinputformat,refertotheInputFormatsReference.Regardlessoftheinputformatused,the<from_entity>specifiedintheFROMclausemustcomplywiththefollowinggeneralsyntax:The<from_entity>mustbeasingleelementoralistofelements,separatedbythe','(comma)or';'(semicolon)characters,asinthefollowingexamples:

file1.txtfile1.txt,file2.txtfile1.txt;D:\file2.txt;file3.txtEachelementcannotcontainspaces,','(comma)characters,or';'(semicolon)characters,unlesstheelementisenclosedbythe'''(singlequote)or'"(doublequotes)characters,asinthefollowingexample:

file2.txt,'C:\ProgramFiles\file3.txt',file4.txt

Thefollowingcharactersareconsideredparenthesyscharacters,andiftheyappearinanelement,theymustappearaswell-formedpairsofopeningandclosingparenthesys:

<>()[]{}

Thefollowingexamplesshowvalidfrom-entitiescontainingparenthesyscharacters:

entity<value>entity[value]valueThefollowingexamplesshowinvalidfrom-entitiescontainingparenthesyscharacters:

entity>value<entity}valueentity(valueAnycharacter(includingillegalcharactersandnon-printable

characters)ina<from-entity>canbeenteredusingthe\uxxxxnotation,wherexxxxisthe4-digithexadecimalrepresentationoftheUNICODEcharacter,asinthefollowingexample:

C:\Program\u0020Files\file3.txt

From-entitiesthatrepresentnamesoffilesordirectoriesarenotallowedtocontainthefollowingcharacters,evenwhenenclosedinquotecharactersorenteredusingthe\uxxxxnotation:

tabcarriage-returnline-feed,()"<>

Examples:

A.<from_entity>withtheREGinputformatThefollowingexamplequeryreadsinputrecordsfromtheregistryusingtheREGinputformat:

SELECT*FROM\HKLM\SOFTWAREB.<from_entity>withtheEVTinputformatThefollowingexamplequeryreadsinputrecordsfromtheSystemandSecurityeventlogsusingtheEVTinputformat:

SELECT*FROMSystem,Security

Seealso:INTO

BasicsofaQueryInputFormatsReference

WHERE<where_clause> ::= WHERE<expression>

TheWHEREclauseisusedtospecifyabooleanconditionthatmustbesatisfiedbyaninputrecordforthatrecordtobeoutput.Inputrecordsthatdonotsatisfytheconditionarediscarded.

Remarks:TheexpressioninaWHEREclausecannotreferenceSQL(aggregate)functions.Tospecifyconditionsonvaluesofaggregatefunctions,usetheHAVINGclause.

Examples:

A.Simpleexpression

WHEREEventID=501

B.Complexexpression

WHEREEXTRACT_TOKEN(Strings,1,'|')LIKE'%logon&'AND(TimeGenerated>SUB(TO_LOCALTIME(SYSTEM_TIMESTAMP()),TIMESTAMP('10','mm'))ORSIDISNOTNULL)Seealso:

ExpressionsHAVING

FilteringInputRecords

GROUPBY<group_by_clause> ::= GROUPBY<field_expr_list>[WITH

ROLLUP]

<field_expr_list> ::= <field_expr>[,<field_expr>...]

TheGROUPBYclausespecifiesthegroupsintowhichoutputrowsaretobeplacedand,ifaggregatefunctionsareincludedintheSELECTorHAVINGclauses,calculatestheaggregatefunctionsvaluesforeachgroup.

Arguments:

WITHROLLUPSpecifiesthatinadditiontotheusualrowsprovidedbyGROUPBY,summaryrowsareintroducedintotheresultset.Groupsaresummarizedinahierarchicalorder,fromthelowestlevelinthegrouptothehighest,andthecorrespondingsummaryrowscontainNULLvaluesforthegroupsthathavebeensummarized.Thegrouphierarchyisdeterminedbytheorderinwhichthegroupingfield-expressionsarespecified.Changingtheorderofthegroupingfield-expressionscanaffectthenumberofrowsproducedintheresultset.TheROLLUPoperatorisoftenusedwiththeGROUPINGaggregatefunction.

Remarks:WhenGROUPBYisspecified,eithereachnon-aggregateandnon-constantfield-expressionintheSELECTclauseshouldbeincludedin

theGROUPBYfield-expressionlist,ortheGROUPBYfield-expressionlistmustmatchexactlytheSELECTclausefield-expressionlist.Formoreinformation,seeAggregatingDataWithinGroups.AggregatefunctionsusingtheDISTINCTkeyword,forexample,"COUNT(DISTINCTfield-expression)",arenotsupportedwhenusingtheGROUPBYclause.IftheORDERBYclauseisnotspecified,groupsreturnedusingtheGROUPBYclausearenotinanyparticularorder.ItisrecommendedthattheORDERBYclauseisalwaysusedtospecifyaparticularorderingofthedata.

Examples:

A.SimpleGROUPBYclauseThefollowingquery,onanIISW3Clogfile,returnsthenumberofrequestsforeachpageoneachday:

SELECTdate,cs-uri-stem,COUNT(*)FROMLogFiles\ex040528.logGROUPBYdate,cs-uri-stemAsampleoutputwouldbe:

datecs-uri-stemCOUNT(ALL*)-----------------------------------------2003-11-18/Default.htm12003-11-18/style.css12003-11-18/images/address.gif12003-11-18/cgi-bin/counts.exe12003-11-18/data/rulesinfo.nsf22003-11-19/data/rulesinfo.nsf62003-11-20/data/rulesinfo.nsf52003-11-20/maindefault.htm12003-11-20/top2.htm12003-11-20/homelog.swf1

B.UsingWITHROLLUPThefollowingexamplequeryisthesameasinthepreviousexample,usingtheWITHROLLUPargumenttodisplayadditionalsummaryrows:

SELECTdate,cs-uri-stem,COUNT(*)FROMLogFiles\ex040528.logGROUPBYdate,cs-uri-stemWITHROLLUPAsampleoutputwouldbe:

datecs-uri-stemCOUNT(ALL*)-----------------------------------------

2003-11-18/Default.htm12003-11-18/style.css12003-11-18/images/address.gif12003-11-18/cgi-bin/counts.exe12003-11-18/data/rulesinfo.nsf22003-11-19/data/rulesinfo.nsf62003-11-20/data/rulesinfo.nsf52003-11-20/maindefault.htm12003-11-20/top2.htm12003-11-20/homelog.swf1--202003-11-18-62003-11-19-62003-11-20-8

Thegroupsummariesthathavebeenintroducedbytherollupoperatorare:

2003-11-18-62003-11-19-62003-11-20-8--20Whichrepresentthenumberofrequestsoneachday,regardlessofthepagerequested,andthetotalnumberofrequestsinthelogfile,regardlessoftheday.

Seealso:FieldExpressionsSELECT

AggregatingDataWithinGroups

HAVING<having_clause> ::= HAVING<expression>

TheHAVINGclauseisusedtospecifyabooleanconditionthatmustbesatisfiedbyagroupforthegrouprecordtobeoutput.Groupsthatdonotsatisfytheconditionarediscarded.

Examples:

A.Simpleexpression

HAVINGEventID=501

B.Complexexpression

HAVINGSUM(sc-bytes)>100000AND(COUNT(*)>1000OREXTRACT_EXTENSION(cs-uri-stem)LIKE'htm')C.ComplexexpressionThefollowingexamplequeryretrievesalltheeventsourcesfromtheSystemeventlogthatgeneratedmorethan10events:

SELECTSourceNameFROMSystemGROUPBYSourceNameHAVINGCOUNT(*)>10

Seealso:ExpressionsWHERE

FilteringGroups

ORDERBY<order_by_clause> ::= ORDERBY<field_expr_list>[ASC|DESC]

TheORDERBYclausespecifieswhichSELECTclausefield-expressionsthequeryoutputrecordsshouldbesortedby.

Arguments:

ASCSpecifiesthatthefield-expressionlistvaluesshouldbesortedinascendingorder,fromlowestvaluetohighestvalue.ASCisthedefault.

DESCSpecifiesthatthefield-expressionlistvaluesshouldbesortedindescendingorder,fromhighestvaluetolowestvalue.

Remarks:TheLogParserSQL-Likelanguagerequiresthateachfield-expressionappearingintheORDERBYclausemustalsoappearintheSELECTclause.DifferentlythanthestandardSQLlanguage,intheLogParserSQL-LikelanguagetheDESCorASCsortdirectionappliestoallthefield-expressionsintheORDERBYclause.Inotherwords,itisnotpossibletospecifydifferentsortdirectionsfordifferentfield-expressions.NULLvaluesaretreatedasthelowestpossiblevalues.

Examples:

A.Sortingbyasinglefield-expression

SELECTdate,cs-uri-stem,cs-uri-query,sc-bytesFROMLogFiles\ex040528.logORDERBYsc-bytesDESCB.Sortingbymultiplefield-expressions

SELECTdate,cs-uri-stem,cs-uri-query,sc-bytesFROMLogFiles\ex040528.logORDERBYdate,sc-bytes

Seealso:FieldExpressionsSELECT

SortingOutputRecords

Expressions<expression> ::= <term1>[OR<expression>]

<term1> ::= <term2>[AND<term1>]

<term2> ::= <field_expr><rel_op><field_expr><field_expr>[NOT]LIKE<like_mask><field_expr>[NOT]BETWEEN<field_expr>AND<field_expr><field_expr>IS[NOT]NULL<field_expr>[NOT]IN(<value_rows>)<field_expr><rel_op>[ALL|ANY](<value_rows>)(<field_expr_list>)[NOT]IN(<value_rows>)(<field_expr_list>)<rel_op>[ALL|ANY](<value_rows>)NOT<term2>(<expression>)

<rel_op> ::= <><>=<=>=

<value_rows> ::= <value_row>[;<value_row>...]

<value_row> ::= <value>[,<value>...]

AnexpressionisusedintheWHEREandHAVINGclausestospecifyconditionsthatmustbesatisfiedforinputrecordsorgrouprecordstobeoutput.

Operators:

<rel_op>Standardcomparisonoperators(lessthan,greatherthan,etc.).

[NOT]LIKEIndicatesthatthesubsequentcharacterstringistobeusedwithpatternmatching.Formoreinformation,seeLIKE.

[NOT]BETWEENSpecifiesaninclusiverangeofvalues.UseANDtoseparatethebeginningandendingvalues.Formoreinformation,seeBETWEEN.

IS[NOT]NULLTheISNULLandISNOTNULLoperatorsdeterminewhetherornotagivenfield-expressionisNULL.

[NOT]INTheINandNOTINoperatorsdeterminewhetherornotagivenfield-expressionorlistoffield-expressionsmatchesanyelementinalistofvalues.Formoreinformation,seeIN.

ALLUsedwithacomparisonoperatorandalistofvalues.ReturnsTRUEifallvaluesinthelistsatisfythecomparisonoperation,orFALSEif

notallvaluessatisfythecomparison.IfnoALLnorANYisspecified,thenANYisassumedbydefault.Formoreinformation,seeALL.

ANYUsedwithacomparisonoperatorandalistofvalues.ReturnsTRUEifanyvalueinthelistsatisfiesthecomparisonoperation,orFALSEifnovaluessatisfythecomparison.IfnoALLnorANYisspecified,thenANYisassumedbydefault.Formoreinformation,seeANY.

Remarks:TheexpressioninaWHEREclausecannotreferenceSQL(aggregate)functions.Tospecifyconditionsonvaluesofaggregatefunctions,usetheHAVINGclause.Thereisnolimittothenumberofoperatorsthatcanbeincludedinanexpression.TheorderofprecedenceforthelogicaloperatorsisNOT(highest),followedbyAND,followedbyOR.Theorderofevaluationatthesameprecedencelevelisfromlefttoright.Parenthesescanbeusedtooverridethisorderinanexpression.

Examples:

A.Simpleexpression

sc-bytes>=1000

B.Complexexpression

EXTRACT_TOKEN(Strings,1,'|')LIKE'%logon&'AND(TimeGenerated>SUB(TO_LOCALTIME(SYSTEM_TIMESTAMP()),TIMESTAMP('10','mm'))ORSIDISNOTNULL

)Seealso:ALLANYBETWEENINLIKE

ConstantValuesFieldExpressionsHAVINGWHERE

ALL<field_expr><rel_op>ALL(<value_rows>)

(<field_expr_list>)<rel_op>ALL(<value_rows>)

TheALLoperatorcomparesagivenfield-expressionwithalistofvalues,returningTRUEifallvaluesinthelistsatisfythecomparisonoperation,orFALSEifnotallvaluessatisfythecomparison.

Examples

A.Singlefield-expressionThefollowingexampleexpressiondetermineswhetherornotthe"Year"fieldisgreaterthanallthevaluesinthespecifiedlist:

Year>ALL(1999;2000;2001)

B.Listoffield-expressionsThefollowingexampleexpressiondetermineswhetherornotthepairof"Year"and"Age"fieldsislessthanallthepairsofvaluesinthespecifiedlist:

(Year,Age)<ALL(1999,30;2001,40;2002,10)

Seealso:ANYExpressionsField-Expressions

ANY<field_expr><rel_op>ANY(<value_rows>)

(<field_expr_list>)<rel_op>ANY(<value_rows>)

TheANYoperatorcomparesagivenfield-expressionwithalistofvalues,returningTRUEifanyvalueinthelistsatisfiesthecomparisonoperation,orFALSEifnovaluessatisfythecomparison.

Examples

A.Singlefield-expressionThefollowingexampleexpressiondetermineswhetherornotthe"Year"fieldisgreaterthananyvalueinthespecifiedlist:

Year>ANY(1999;2000;2001)

B.Listoffield-expressionsThefollowingexampleexpressiondetermineswhetherornotthepairof"Year"and"Age"fieldsislessthananyofthepairsofvaluesinthespecifiedlist:

(Year,Age)<ANY(1999,30;2001,40;2002,10)

Seealso:ALLExpressionsField-Expressions

BETWEEN<field_expr>[NOT]BETWEEN<field_expr>AND<field_expr>

TheBETWEENoperatordeterminesifagivenfield-expressionbelongstoaspecifiedinterval.

Examples

A.BETWEENThefollowingexampleexpressiondeterminesifthe"Year"fieldbelongstothespecifiedinterval:

YearBETWEEN1999AND2004

Thisexampleisequivalenttothefollowingexpression:

Year>=1999ANDYear<=2004

B.NOTBETWEENThefollowingexampleexpressiondeterminesifthe"Year"fielddoesnotbelongtothespecifiedinterval:

YearNOTBETWEEN1999AND2004

Year<1999ORYear>2004

C.TIMESTAMPintervalThefollowingexamplequeryusestheFSInputFormattoreturnallthefilesthathavebeencreatedbetween4hoursagoand1hourago:

SELECTPathFROMC:\MyDir\*.*WHERETO_UTCTIME(CreationTime)BETWEENSUB(SYSTEM_TIMESTAMP(),TIMESTAMP('4','h'))ANDSUB(SYSTEM_TIMESTAMP(),TIMESTAMP('1','h'))Seealso:

ExpressionsField-Expressions

IN<field_expr>[NOT]IN(<value_rows>)

(<field_expr_list>)[NOT]IN(<value_rows>)

TheINandNOTINoperatorsdeterminewhetherornotagivenfield-expressionorlistoffield-expressionsmatchesanyelementinalistofvalues.

Remarks:Usethecommacharacter(,)toseparatevaluesinasinglelistrow,andusethesemicoloncharacter(;)toseparatelistrows.

Examples

A.Singlefield-expressionThefollowingexampleexpressiondeterminesifthe"Age"fieldmatchesanyvalueinthespecifiedlist:

AgeIN(20;30;45;60)

Age=20ORAge=30ORAge=45ORAge=60

B.Listoffield-expressionsThefollowingexampleexpressiondeterminesifthepairof"FirstName"and"State"fieldsmatchesanypairofvaluesinthespecifiedlist:

(FirstName,State)IN('Johnson','OR';'Smith','WA')

(FirstName='Johnson'ANDState='OR')OR(FirstName='Smith'ANDState='WA')

Seealso:ExpressionsField-Expressions

LIKE<field_expr>[NOT]LIKE<like_mask>

Determineswhetherornotagivencharacterstringmatchesaspecifiedpattern.Apatterncanincluderegularcharactersandwildcardcharacters.Duringpatternmatching,regularcharactersmustyieldacase-insensitivematchwiththecharactersspecifiedinthecharacterstring.Wildcardcharacters,however,canbematchedwitharbitraryfragmentsofthecharacterstring.UsingwildcardcharactersmakestheLIKEoperatormoreflexiblethanusingthe=and!=stringcomparisonoperators.

ThewildcardcharactersthatcanbeusedinaLIKEpatternare:

_(underscorecharacter):matchesanysinglecharacterExamples:

LIKE'ab_d':matchesallthefour-letterstringsthatstartwith"ab"andendwith"d"(e.g."abcd","AB+d")LIKE'a_c_':matchesallthefour-letterstringsthathave"a"inthefirstpositionand"c"inthethirdposition(e.g."abcd","Akck")

%(percentcharacter):matchesanystringofzeroormorecharactersExamples:

LIKE'%.asp'matchesallthestringsendingwith".asp"(e.g."/default.asp",".ASP")LIKE'%error%'matchesallthestringscontaining"error"(e.g."anerrorhasbeenfound","ERROR")

Remarks:SimilarlytoSTRINGconstants,charactersinaLIKEpatterncanbeescapedwiththe'\'(backslash)characterorencodedwiththe\uxxxxnotation.Wildcardpatternmatchingcharacterscanbeusedasliteralcharacters.Touseawildcardcharacterasaliteralcharacter,escapethewildcardcharacterwiththe'\'(backslash)character.

Examples:LIKE'ab\_d':matchesthe"ab_d"string(e.g."ab_d","AB_d")LIKE'a\%c%':matchesallthestringsthatstartwith"a%c"(e.g."a%cdefg","A%c")

WhenexecutingaLogParserqueryfromwithinacommand-linebatchfile,usingthe%wildcardcharactermightyeldunexpectedresults.Forexample,considerthefollowingbatchfile:

@echooffLogParser"SELECT*FROMSYSTEMWHEREMessageLIKE'%ERROR%'"Whenthisbatchfileisexecuted,thecommand-lineshellinterpreterwillassumethat"%ERROR%"isareferencetoanenvironmentvariable,anditwilltrytoreplacethisstringwiththevalueoftheenvironmentvariable.Inmostcases,suchanenvironmentvariablewillnotexist,andtheactualcommandexecutedbytheshellwilllooklike:

LogParser"SELECT*FROMSYSTEMWHEREMessageLIKE''"

Whichwouldyeldthefollowingerror:

Error:SyntaxError:<term2>:novalidLIKEmask

Toavoidthisproblem,usedouble%%wildcardcharacterswhenwritingacommand-linebatchfile,asinthefollowingexample:

@echooffLogParser"SELECT*FROMSYSTEMWHEREMessageLIKE'%%ERROR%%'"

Examples

A.LIKEThefollowingexampleWHEREclausefindsalltheURL'sinanIISW3Clogfilethatendwith".htm":

WHEREcs-uri-stemLIKE'%.htm'

B.NOTLIKEThefollowingexampleWHEREclausefindsalltheEventLogmessagesthatdonotcontain"error":

WHEREMessageNOTLIKE'%error%'

Seealso:ExpressionsField-Expressions

Field-Expressions<field_expr> ::= <aggregate_function><function>

<field_name><alias><value>

Field-expressionsareacombinationofsymbolsandfunctionsthatLogParserevaluatestoobtainasingledatavalue.ThesearethebasicargumentsoftheSELECT,USING,WHERE,GROUPBY,HAVING,andORDERBYclauses.

Field-expressionscanbedividedconceptuallyintotwogroups:

Derivedfield-expressions:functionsoraggregatefunctionshavingotherfield-expressionsasarguments;Basicfield-expressions:constantvalues(includingfunctionswithnoarguments),namesofinputrecordfields,oraliasesdefinedintheSELECTorUSINGclauses.

Examples:

A.Basicfield-expressionsTheSELECTclauseinthefollowingexamplequeryspecifies"basic"field-expressionsonly:

SELECT'EventID:',EventID,SYSTEM_TIMESTAMP()FROMSystemB.Derivedfield-expressionsTheSELECTclauseinthefollowingexamplequeryspecifies"derived"field-expressionsonly:

SELECTTO_UPPERCASE(cs-uri-stem),SUM(sc-bytes)FROM\MyLogs\ex042805.logGROUPBYTO_UPPERCASE(cs-uri-stem)

Seealso:AggregateFunctionsFunctionsConstantValuesFieldNamesandAliasesSELECTUSING

BasicsofaQuery

FieldNamesandAliases<field_name> ::= [[]<string>[]]

<alias> ::= [[]<string>[]]

Fieldnamesarenamesoffieldsoftheinputrecordsgeneratedbyaninputformat.

Aliasesarealternativenamesforfield-expressions,assignedintheSELECTorUSINGclauses.Whenafield-expressionintheSELECTclausehasbeenaliased,outputformatswillusethealiasasthenameofthecorrespondingoutputrecordfield.Thealiasofafield-expressioncanbealsousedanywhereelseinthequeryasashortcutthatreferstotheoriginalfield-expression.

Remarks:Thefollowingcharactersarenotallowedinfieldnamesoraliases,unlessthefieldnameoraliasisenclosedinsquarebrackets([and]):

,;<>=!'"@*[]space

Fieldnamesandaliasescontainingspacesorillegalcharacterscanbeenclosedinsquarebrackets([and]),asinthefollowingexample:

SELECT[LastRequestTime],[email@address],CPUTimeas[ElapsedCPUTime]FROMperflog.csvWHERE[ElapsedCPUTime]>0Anycharacter(includingillegalcharactersandnon-printablecharacters)infieldnamesandaliasescanbealsoenteredusingthe\uxxxxnotation,wherexxxxisthe4-digithexadecimalrepresentationoftheUNICODEcharacter:

SELECTLast\u0020Request\u0020TimeFROMperflog.csv

FieldnamesandaliasescannotmatchkeywordsorfunctionnamesoftheLogParserSQL-Likelanguage(e.g."FROM","ADD").Fieldnamesandaliasesarenotcase-sensitive.

Examples:

A.Basicfield-expressionsTheSELECTclauseinthefollowingexamplequeryspecifies"basic"field-expressionsonly:

SELECT'EventID:',EventID,SYSTEM_TIMESTAMP()FROMSystemB.Derivedfield-expressionsTheSELECTclauseinthefollowingexamplequeryspecifies"derived"field-expressionsonly:

SELECTTO_UPPERCASE(cs-uri-stem),SUM(sc-bytes)FROM\MyLogs\ex042805.logGROUPBYTO_UPPERCASE(cs-uri-stem)

Seealso:SELECTUSING

BasicsofaQuery

AggregateFunctions<aggregate_function> ::= COUNT([DISTINCT|ALL]*)COUNT

([DISTINCT|ALL]<field_expr_list>)SUM([DISTINCT|ALL]<field_expr>)AVG([DISTINCT|ALL]<field_expr>)MAX([DISTINCT|ALL]<field_expr>)MIN([DISTINCT|ALL]<field_expr>)PROPCOUNT(*)[ON(<on_field_expr_list>)]PROPCOUNT(<field_expr_list>)[ON(<on_field_expr_list>)]PROPSUM(<field_expr>)[ON(<on_field_expr_list>)]GROUPING(<field_expr>)

Aggregatefunctionsperformacalculationonasetofvaluesbutreturnasingle,summarizingvalue.

AggregatefunctionsareoftenusedwiththeGROUPBYclause.WhenusedwithoutaGROUPBYclause,aggregatefunctionsperformcalculationsontheentiresetofinputrecords,returningasinglesummarizingvalueforthewholeset.WhenusedwithaGROUPBYclause,aggregatefunctionsperformcalculationsoneachsetofgrouprecords,returningasummarizingvalueforeachgroup.

Functions:

Returnsthenumberofitemsinagroup.Formoreinformation,seeCOUNT.

SUMReturnsthesumofthevaluesofthespecifiedfield-expression.Formoreinformation,seeSUM.

AVGReturnstheaverageacrossthevaluesofthespecifiedfield-expression.Formoreinformation,seeAVG.

MAXReturnsthemaximumvalueamongthevaluesofthespecifiedfield-expression.Formoreinformation,seeMAX.

MINReturnstheminimumvalueamongthevaluesofthespecifiedfield-expression.Formoreinformation,seeMIN.

PROPCOUNTReturnstheratiooftheCOUNTaggregatefunctioncalculatedonagrouptotheCOUNTaggregatefunctioncalculatedonahierarchicallyhighergroup.Formoreinformation,seePROPCOUNT.

PROPSUMReturnstheratiooftheSUMaggregatefunctioncalculatedonagrouptotheSUMaggregatefunctioncalculatedonahierarchicallyhighergroup.Formoreinformation,seePROPSUM.

GROUPING

Returnsavalueof1whentherowisaddedbytheROLLUPoperatoroftheGROUPBYclause,or0whentherowisnottheresultofROLLUP.TheGROUPINGaggregatefunctionisallowedonlywhentheGROUPBYclausecontainstheROLLUPoperator.Formoreinformation,seeGROUPING.

Remarks:Aggregatefunctionsareallowedasfield-expressionsonlyintheSELECT,HAVING,andORDERBYclauses.Theargumentsofanaggregatefunctioncannotreferenceotheraggregatefunctions.Theargumentsofanaggregatefunctioncannotreferencethefollowingfunctions:SEQUENCEOUT_ROW_NUMBER

DISTINCTisallowedinaggregatefunctionsonlywhenthereisnoGROUPBYclause.

Examples:

A.COUNT(*)ThefollowingqueryreturnsthetotalnumberofeventsintheSystemeventlog:

SELECTCOUNT(*)FROMSystemB.COUNT(DISTINCT)ThefollowingqueryreturnsthetotalnumberofdistincteventsourcenamesintheSystemeventlog:

SELECTCOUNT(DISTINCTSourceName)FROMSystemC.COUNT(*)andGROUPBYThefollowingqueryreturnsthetotalnumberofeventsgeneratedbyeacheventsourceintheSystemeventlog:

SELECTSourceName,COUNT(*)FROMSystemGROUPBYSourceNameD.SUMandGROUPBYThefollowingqueryreturnsthetotalnumberofbytessentforeachpageextensionloggedinthespecifiedIISW3Clogfile:

SELECTTO_LOWERCASE(EXTRACT_EXTENSION(cs-uri-stem))ASPageType,SUM(sc-bytes)FROMex031118.logGROUPBYPageType

E.PROPCOUNT(*),GROUPBY,andHAVINGThefollowingqueryreturnsthepagesthatrepresentmorethan10%oftherequestsinthespecifiedIISW3Clogfile:

SELECTcs-uri-stemFROMex031118.logGROUPBYcs-uri-stemHAVINGPROPCOUNT(*)>0.1

Seealso:COUNTSUMAVGMAXMINPROPCOUNTPROPSUMGROUPING

FunctionsSELECTHAVING

GROUP_BY

AggregatingDataWithinGroupsCalculatingPercentages

AVGAVG([DISTINCT|ALL]<field_expr>)

Returnstheaverageamongallthevalues,oronlytheDISTINCTvalues,ofthespecifiedfield-expression.

Arguments:

DISTINCTSpecifiesthatAVGreturnstheaverageofuniquevalues.DISTINCTcanonlybeusedwhenthequerydoesnotmakeuseoftheGROUPBYclause.

ALLAppliestheaggregatefunctiontoallvalues.ALListhedefault.

<field_expr>Thefield-expressionwhosevaluesaretobeaveraged.Thefield-expressiondatatypemustbeINTEGERorREAL.

ReturnType:

INTEGERorREAL,dependingontheargumentfield-expression.

Remarks:NULLvaluesareignoredbytheAVGaggregatefunction.Aggregatefunctionsareallowedasfield-expressionsonlyintheSELECT,HAVING,andORDERBYclauses.

Theargumentsofanaggregatefunctioncannotreferenceotheraggregatefunctions.Theargumentsofanaggregatefunctioncannotreferencethefollowingfunctions:SEQUENCEOUT_ROW_NUMBER

Examples:

A.AVGThefollowingqueryreturnstheaveragenumberofbytesforexecutablefilesinthe"system32"directory,usingtheFSinputformat:

SELECTAVG(Size)FROMC:\windows\system32\*.*WHERETO_LOWERCASE(EXTRACT_EXTENSION(Name))='exe'B.AVGandGROUPBYThefollowingqueryreturnstheaveragetimespentbyeachpageextensionloggedinthespecifiedIISW3Clogfile:

SELECTTO_LOWERCASE(EXTRACT_EXTENSION(cs-uri-stem))ASPageType,AVG(time-taken)FROMex031118.logGROUPBYPageTypeSeealso:

COUNTSUMMAXMINPROPCOUNTPROPSUM

GROUPING

AggregateFunctions

COUNTCOUNT([DISTINCT|ALL]*)COUNT([DISTINCT|ALL]<field_expr_list>)

Returnsthenumberofitemsinagroup.

Arguments:

DISTINCTSpecifiesthatCOUNTreturnsthenumberofuniquevalues.DISTINCTcanonlybeusedwhenthequerydoesnotmakeuseoftheGROUPBYclause.

*Specifiesthatallrecordsshouldbecountedtoreturnthetotalnumberofrecords,includingrecordsthatcontainNULLvalues.

<field_expr_list>Specifiesthatonlyrecordsforwhichatleastoneofthespecifiedfield-expressionsisnon-NULLshouldbecounted.

ReturnType:

INTEGER

Remarks:Aggregatefunctionsareallowedasfield-expressionsonlyintheSELECT,HAVING,andORDERBYclauses.Theargumentsofanaggregatefunctioncannotreferenceotheraggregatefunctions.Theargumentsofanaggregatefunctioncannotreferencethefollowingfunctions:SEQUENCEOUT_ROW_NUMBER

Examples:

A.COUNT(*)ThefollowingqueryreturnsthetotalnumberofeventsintheSystemeventlog:

SELECTCOUNT(*)FROMSystemB.COUNT(DISTINCT)ThefollowingqueryreturnsthetotalnumberofdistincteventsourcenamesintheSystemeventlog:

SELECTCOUNT(DISTINCTSourceName)FROMSystemC.COUNT(*)andGROUPBYThefollowingqueryreturnsthetotalnumberofeventsgeneratedby

eacheventsourceintheSystemeventlog:

SELECTSourceName,COUNT(*)FROMSystemGROUPBYSourceNameD.COUNT(field-expression)Thefollowingqueryreturnsthetotalnumberofnon-nullvaluesforthe"cs-username"fieldinthespecifiedIISW3Clogfile:

SELECTCOUNT(cs-username)FROMex040528.logE.COUNT(*)andWHEREThefollowingqueryreturnsthetotalnumberofrequeststoapageloggedinthespecifiedIISW3Clogfile:

SELECTCOUNT(*)FROMex040528.logWHEREcs-uri-stem='/home.asp'F.COUNT(*),GROUPBY,andHAVINGThefollowingqueryreturnsthepagesinthespecifiedIISW3Clogfilethathavebeenrequestedmorethan50times:

SELECTcs-uri-stemFROMex040528.logGROUPBYcs-uri-stemHAVINGCOUNT(*)>50

Seealso:SUMAVGMAXMINPROPCOUNTPROPSUMGROUPING

AggregateFunctions

GROUPINGGROUPING(<field_expr>)

Returnsavalueof1whentherowisaddedbytheROLLUPoperatoroftheGROUPBYclause,or0whentherowisnottheresultofROLLUP.GROUPINGisusedtodistinguishtheNULLvaluesreturnedbyROLLUPfromstandardNULLvalues.TheNULLreturnedastheresultofaROLLUPoperationisaspecialuseofNULL.Itactsasavalueplaceholderintheresultsetandmeans"all".

Arguments:

<field_expr>TheGROUPBYfield-expressioncheckedfornullvalues.

ReturnType:

INTEGER

Remarks:TheGROUPINGaggregatefunctionisallowedonlywhentheGROUPBYclausecontainstheROLLUPoperator.Aggregatefunctionsareallowedasfield-expressionsonlyintheSELECT,HAVING,andORDERBYclauses.Theargumentsofanaggregatefunctioncannotreferenceotheraggregatefunctions.Theargumentsofanaggregatefunctioncannotreferencethefollowingfunctions:

SEQUENCEOUT_ROW_NUMBER

Examples:

A.GROUPINGThefollowingquery,onanIISW3Clogfile,returnsthenumberofrequestsforeachpageoneachday,andusestheROLLUPoperatortoalsodisplaysummaryrowsshowingthenumberofrequestsforeachday,andthetotalnumberofrequests:

SELECTdate,cs-uri-stem,COUNT(*),GROUPING(date)ASGDate,GROUPING(cs-uri-stem)ASGPageFROMex040528.logGROUPBYdate,cs-uri-stemWITHROLLUPAsampleoutputwouldbe:

datecs-uri-stemCOUNT(ALL*)GDateGPage---------------------------------------------------2003-11-18/Default.htm1002003-11-18/style.css1002003-11-18/images/address.gif1002003-11-18/cgi-bin/counts.exe1002003-11-18/data/rulesinfo.nsf2002003-11-19/data/rulesinfo.nsf6002003-11-20/data/rulesinfo.nsf5002003-11-20/maindefault.htm1002003-11-20/top2.htm1002003-11-20/homelog.swf100--20112003-11-18-6012003-11-19-6012003-11-20-801

Thevaluesofthe"GDate"fieldare1onlyfortherowsinwhichthe"date"fieldisNULLduetotheintroductionoftheROLLUPsummaryrows.Similarly,thevaluesofthe"GPage"fieldare1onlyfortherowsinwhichthe"cs-uri-stem"fieldisNULLduetotheintroductionoftheROLLUPsummaryrows.

Seealso:COUNTSUMAVGMAXMINPROPCOUNTPROPSUM

GROUPBYAggregateFunctions

MAXMAX([DISTINCT|ALL]<field_expr>)

Returnsthemaximumvalueamongallthevaluesofthespecifiedfield-expression.

Arguments:

DISTINCTSpecifiesthatMAXreturnsthemaximumvalueofuniquevalues.DISTINCTisnotmeaningfulwithMAXandisavailableforSQL-92compatibilityonly.DISTINCTcanonlybeusedwhenthequerydoesnotmakeuseoftheGROUPBYclause.

<field_expr>Thefield-expressionamongwhosevaluesthemaximumistobefound.Thefield-expressioncanbeofanydatatype.

ReturnType:

Thereturnedtypeisthesameastheargumentfield-expression.

Remarks:

NULLvaluesareignoredbytheMAXaggregatefunction.Aggregatefunctionsareallowedasfield-expressionsonlyintheSELECT,HAVING,andORDERBYclauses.Theargumentsofanaggregatefunctioncannotreferenceotheraggregatefunctions.Theargumentsofanaggregatefunctioncannotreferencethefollowingfunctions:SEQUENCEOUT_ROW_NUMBER

Examples:

A.MAXThefollowingqueryreturnsthesizeofthelargestexecutablefileinthe"system32"directory,usingtheFSinputformat:

SELECTMAX(Size)FROMC:\windows\system32\*.*WHERETO_LOWERCASE(EXTRACT_EXTENSION(Name))='exe'B.MAXandGROUPBYThefollowingqueryreturnsthelongesttimespentbyeachpageextensionloggedinthespecifiedIISW3Clogfile:

SELECTTO_LOWERCASE(EXTRACT_EXTENSION(cs-uri-stem))ASPageType,MAX(time-taken)FROMex031118.logGROUPBYPageTypeSeealso:

COUNTSUMAVG

MINPROPCOUNTPROPSUMGROUPING

AggregateFunctions

MINMIN([DISTINCT|ALL]<field_expr>)

Returnstheminimumvalueamongallthevaluesofthespecifiedfield-expression.

Arguments:

DISTINCTSpecifiesthatMINreturnstheminimumvalueofuniquevalues.DISTINCTisnotmeaningfulwithMINandisavailableforSQL-92compatibilityonly.DISTINCTcanonlybeusedwhenthequerydoesnotmakeuseoftheGROUPBYclause.

<field_expr>Thefield-expressionamongwhosevaluestheminimumistobefound.Thefield-expressioncanbeofanydatatype.

ReturnType:

Thereturnedtypeisthesameastheargumentfield-expression.

Remarks:

NULLvaluesareignoredbytheMINaggregatefunction.Aggregatefunctionsareallowedasfield-expressionsonlyintheSELECT,HAVING,andORDERBYclauses.Theargumentsofanaggregatefunctioncannotreferenceotheraggregatefunctions.Theargumentsofanaggregatefunctioncannotreferencethefollowingfunctions:SEQUENCEOUT_ROW_NUMBER

Examples:

A.MINThefollowingqueryreturnsthesizeofthesmallestexecutablefileinthe"system32"directory,usingtheFSinputformat:

SELECTMIN(Size)FROMC:\windows\system32\*.*WHERETO_LOWERCASE(EXTRACT_EXTENSION(Name))='exe'B.MINandGROUPBYThefollowingqueryreturnstheshortestandthelongesttimespentbyeachpageextensionloggedinthespecifiedIISW3Clogfile:

SELECTTO_LOWERCASE(EXTRACT_EXTENSION(cs-uri-stem))ASPageType,MIN(time-taken),MAX(time-taken)FROMex031118.logGROUPBYPageType

Seealso:COUNTSUMAVG

MAXPROPCOUNTPROPSUMGROUPING

AggregateFunctions

PROPCOUNTPROPCOUNT(*)[ON(<on_field_expr_list>)]PROPCOUNT(<field_expr_list>)[ON(<on_field_expr_list>)]

<on_field_expr_list> ::= <field_expr>[,<field_expr>...]

ReturnstheratiooftheCOUNTaggregatefunctioncalculatedonagrouptotheCOUNTaggregatefunctioncalculatedonahierarchicallyhighergroup.

Arguments:

*Specifiesthatallrecordsshouldbecountedtoreturnthetotalnumberofrecords,includingrecordsthatcontainNULLvalues.

<field_expr_list>Specifiesthatonlyrecordsforwhichatleastoneofthespecifiedfield-expressionsisnon-NULLshouldbecounted.

<on_field_expr_list>ListofGROUPBYfield-expressionsidentifyingthehierarchicallyhighergrouponwhichthedenominatorCOUNTaggregatefunctionistobecalculated.Thislistoffield-expressionsmustbeaproperprefixoftheGROUPBYfield-expressions,thatis,itmustcontain,inthesameorder,asubsetofthefield-expressionsspecifiedintheGROUPBYclause,startingwiththeleftmostGROUPBYfield-expression.

Whenthislistoffield-expressionsisnotspecified,thedenominatorCOUNTaggregatefunctioniscalculatedonthewholesetofinputrecords.

ReturnType:

Remarks:WhenusedwithoutaGROUPBYclause,thePROPCOUNTaggregatefunctionalwaysreturns1.0.Infact,inthiscasetheonlyhierarchicallyhighergroupavailableisthewholesetofinputrecords,andtherationumeratoranddenominatorarecalculatedonthesameset.Toobtainapercentage,multiplythereturnvalueofthePROPCOUNTaggregatefunctionby100.0,usingtheMULfunction.Aggregatefunctionsareallowedasfield-expressionsonlyintheSELECT,HAVING,andORDERBYclauses.Theargumentsofanaggregatefunctioncannotreferenceotheraggregatefunctions.Theargumentsofanaggregatefunctioncannotreferencethefollowingfunctions:SEQUENCEOUT_ROW_NUMBER

Examples:

A.PROPCOUNT(*)ThefollowingqueryreturnsthepercentageofeventsforeachsourceintheSystemeventlog:

SELECTSourceName,MUL(PROPCOUNT(*),100.0)ASPercentFROMSystemGROUPBYSourceNameAsampleoutputofthisqueryis:

SourceNamePercent--------------------------------EventLog10.322979ServiceControlManager63.004172AtiHotKeyPoller3.430691ApplicationPopup0.108175W32Time14.680884DCOM0.046361NtServicePack0.185443Win32k0.324525RemoteAccess2.194406GEMPCC0.509968SCardSvr0.509968Dhcp0.262711i8042prt0.015454Print0.030907Tcpip0.077268Workstation0.015454NETLOGON1.869881DnsApi2.240766Kerberos0.169989

The"Percent"outputrecordfieldshowstheratioofthenumberofeventsloggedbyasourcetothetotalnumberofeventsintheeventlog.

Inthisexample,thecalculationperformedbythePROPCOUNTaggregatefunctionisequivalenttoexecutingthefollowingtwoqueriesandcalculatingtheratioofthetwoaggregatefunctionsforeacheventlogsource:

SELECTSourceName,COUNT(*)ASNumeratorFROMSystemGROUPBYSourceNameSELECTCOUNT(*)ASDenominatorFROMSystemB.UsingONThefollowingqueryusestheIISW3CInputFormattoparseIISlogfilesandcalculatethepercentageofhitsforapagetypeandHTTPstatuscoderelativetothenumberofhitsforthatpagetype(i.e.thedistributionofHTTPstatuscodeswithineachpagetype):

SELECTEXTRACT_EXTENSION(cs-uri-stem)ASPageType,sc-status,MUL(PROPCOUNT(*)ON(PageType),100.0)ASHitsFROMex040528.logGROUPBYPageType,sc-statusORDERBYPageType,sc-status

Asampleoutputofthisqueryis:

PageTypesc-statusHits---------------------------asp200100.000000class20020.000000class30480.000000css20013.636364css30445.454545

ForeachpagetypeandHTTPstatuscode,the"Hits"outputrecordfieldshowstheratioofthenumberofrequestsforthatpagetypeandHTTPstatuscodetothetotalnumberofrequestsforthatpagetype.

Inthisexample,thecalculationperformedbythePROPCOUNT

css40440.909091dll500100.000000exe200100.000000gif20021.025641gif30476.923077gif4042.051282htm20029.565217htm30468.695652htm4041.739130html404100.000000jpg20022.077922jpg30477.922078js20036.363636js30463.636364nsf20090.845070nsf3020.704225nsf3046.338028nsf4032.112676swf20027.272727swf30472.727273

aggregatefunctionisequivalenttoexecutingthefollowingtwoqueriesandcalculatingtheratioofthetwoaggregatefunctionsforeachpagetypeandHTTPstatus:

SELECTEXTRACT_EXTENSION(cs-uri-stem)ASPageType,sc-status,COUNT(*)ASNumeratorFROMex040528.logGROUPBYPageType,sc-statusORDERBYPageType,sc-status

SELECTEXTRACT_EXTENSION(cs-uri-stem)ASPageType,COUNT(*)ASDenominatorFROMex040528.logGROUPBYPageTypeORDERBYPageTypeSeealso:

COUNTSUMAVGMAXMINPROPSUMGROUPING

AggregateFunctions

PROPSUMPROPSUM(<field_expr>)[ON(<on_field_expr_list>)]

<on_field_expr_list> ::= <field_expr>[,<field_expr>...]

ReturnstheratiooftheSUMaggregatefunctioncalculatedonagrouptotheSUMaggregatefunctioncalculatedonahierarchicallyhighergroup.

Arguments:

<field_expr>Thefield-expressionwhosevaluesaretobesummed.Thefield-expressiondatatypemustbeINTEGERorREAL.

<on_field_expr_list>ListofGROUPBYfield-expressionsidentifyingthehierarchicallyhighergrouponwhichthedenominatorSUMaggregatefunctionistobecalculated.Thislistoffield-expressionsmustbeaproperprefixoftheGROUPBYfield-expressions,thatis,itmustcontain,inthesameorder,asubsetofthefield-expressionsspecifiedintheGROUPBYclause,startingwiththeleftmostGROUPBYfield-expression.Whenthislistoffield-expressionsisnotspecified,thedenominatorSUMaggregatefunctioniscalculatedonthewholesetofinputrecords.

ReturnType:

Remarks:WhenusedwithoutaGROUPBYclause,thePROPSUMaggregatefunctionalwaysreturns1.0.Infact,inthiscasetheonlyhierarchicallyhighergroupavailableisthewholesetofinputrecords,andtherationumeratoranddenominatorarecalculatedonthesameset.Toobtainapercentage,multiplythereturnvalueofthePROPSUMaggregatefunctionby100.0,usingtheMULfunction.Aggregatefunctionsareallowedasfield-expressionsonlyintheSELECT,HAVING,andORDERBYclauses.Theargumentsofanaggregatefunctioncannotreferenceotheraggregatefunctions.Theargumentsofanaggregatefunctioncannotreferencethefollowingfunctions:SEQUENCEOUT_ROW_NUMBER

Examples:

A.PROPSUMThefollowingqueryusestheIISW3CInputFormattoparseIISlogfilesandcalculatethepercentageofbytessentforeachpagetype:

SELECTEXTRACT_EXTENSION(cs-uri-stem)ASPageType,MUL(PROPSUM(sc-bytes),100.0)ASPercentBytesFROMex040528.logGROUPBYPageTypeAsampleoutputofthisqueryis:

PageTypePercentBytes--------------------htm7.236737css1.035243gif23.772064

The"PercentBytes"outputrecordfieldshowstheratioofthebytessentforeachpagetypetothetotalnumberofbytessentinthelog.

exe1.398888nsf24.459391swf32.528669jpg8.003440html0.104051dll0.002322asp0.000000js1.260613class0.198582

Inthisexample,thecalculationperformedbythePROPSUMaggregatefunctionisequivalenttoexecutingthefollowingtwoqueriesandcalculatingtheratioofthetwoaggregatefunctionsforeachpagetype:

SELECTEXTRACT_EXTENSION(cs-uri-stem)ASPageType,SUM(sc-bytes)ASNumeratorFROMex040528.logGROUPBYPageTypeSELECTSUM(sc-bytes)ASDenominatorFROMex040528.logB.UsingONThefollowingqueryusestheIISW3CInputFormattoparseIISlogfilesandcalculatethepercentageofbytessentforeachpagetypeandHTTPstatuscoderelativetothetotalbytessentforthatpagetype(i.e.thedistributionofHTTPstatuscoderesponsebyteswithineachpagetype):

SELECTEXTRACT_EXTENSION(cs-uri-stem)ASPageType,sc-status,MUL(PROPSUM(sc-bytes)ON(PageType),100.0)ASPercentBytesFROMex040528.logGROUPBYPageType,sc-statusORDERBYPageType,sc-status

Asampleoutputofthisqueryis:

PageTypesc-statusPercentBytes-----------------------------asp2000.000000class20092.591620class3047.408380css2006.039609css3043.502318css40490.458073dll500100.000000exe200100.000000gif20087.811668gif3046.935887gif4045.252445htm20092.926606htm3044.197755htm4042.875639

ForeachpagetypeandHTTPstatuscode,the"PercentBytes"outputrecordfieldshowstheratiooftheresponsebytesforthatpagetypeandHTTPstatuscodetothetotalresponsebytesforthatpagetype.

Inthisexample,thecalculationperformedbythePROPSUMaggregatefunctionisequivalenttoexecutingthefollowingtwoqueriesandcalculatingtheratioofthetwoaggregatefunctionsforeachpagetypeandHTTPstatus:

SELECTEXTRACT_EXTENSION(cs-uri-stem)ASPageType,sc-status,SUM(sc-bytes)ASNumeratorFROMex040528.logGROUPBYPageType,sc-statusSELECTEXTRACT_EXTENSION(cs-uri-stem)ASPageType,SUM(sc-bytes)ASDenominator

html404100.000000jpg20097.245679jpg3042.754321js20097.963913js3042.036087nsf20099.604883nsf3020.050656nsf3040.281114nsf4030.063347swf20099.910188swf3040.089812

ORDERBYPageType,sc-statusFROMex040528.logGROUPBYPageTypeORDERBYPageType

C.PROPSUM,GROUPBY,andHAVINGThefollowingqueryusestheIISW3CInputFormattoparseIISlogfilesandreturnthepagetypesthatrepresentmorethan10%ofthetotalbytessent:

SELECTEXTRACT_EXTENSION(cs-uri-stem)ASPageTypeFROMex040528.logGROUPBYPageTypeHAVINGPROPSUM(sc-bytes)>0.1

Seealso:COUNTSUMAVGMAXMINPROPCOUNTGROUPING

AggregateFunctions

SUMSUM([DISTINCT|ALL]<field_expr>)

Returnsthesumofallthevalues,oronlytheDISTINCTvalues,ofthespecifiedfield-expression.

Arguments:

DISTINCTSpecifiesthatSUMreturnsthesumofuniquevalues.DISTINCTcanonlybeusedwhenthequerydoesnotmakeuseoftheGROUPBYclause.

<field_expr>Thefield-expressionwhosevaluesaretobesummed.Thefield-expressiondatatypemustbeINTEGERorREAL.

ReturnType:

INTEGERorREAL,dependingontheargumentfield-expression.

Remarks:NULLvaluesareignoredbytheSUMaggregatefunction.Aggregatefunctionsareallowedasfield-expressionsonlyintheSELECT,HAVING,andORDERBYclauses.

Theargumentsofanaggregatefunctioncannotreferenceotheraggregatefunctions.Theargumentsofanaggregatefunctioncannotreferencethefollowingfunctions:SEQUENCEOUT_ROW_NUMBER

Examples:

A.SUMThefollowingqueryreturnsthetotalnumberofbytesforexecutablefilesinthe"system32"directory,usingtheFSinputformat:

SELECTSUM(Size)FROMC:\windows\system32\*.*WHERETO_LOWERCASE(EXTRACT_EXTENSION(Name))='exe'B.SUMandGROUPBYThefollowingqueryreturnsthetotalnumberofbytessentforeachpageextensionloggedinthespecifiedIISW3Clogfile:

SELECTTO_LOWERCASE(EXTRACT_EXTENSION(cs-uri-stem))ASPageType,SUM(sc-bytes)FROMex031118.logGROUPBYPageTypeSeealso:

COUNTAVGMAXMINPROPCOUNTPROPSUMGROUPING

AggregateFunctions

Functions<function> ::= <function_name>(<argument_list>)

<argument_list> ::= <field_expr>[,<field_expr>...]

<empty>

LogParserfunctionstakezeroormorefield-expressionsasarguments,processthearguments,andreturnasinglevalue.

Remarks:Generally,functionsthattakenoargumentsandfunctionswhoseargumentsareconstantvaluesareexecutedandreplacedwiththereturnvaluebeforethequeryisprocessed.Asanexample,thefollowingqueryusesafunctionwithnoargumentsandafunctionwithconstantarguments:

SELECTCOMPUTER_NAME(),SUM(4,5),TimeGeneratedFROMSystemBeforebeingprocessed,thequeryismodifiedasfollows:

SELECT'MYSERVER0',9,TimeGeneratedFROMSystemTheonlyzero-argumentfunctionsthatarenotreplacedwiththeirreturnvaluebeforethequeryisprocessedare:SEQUENCEIN_ROW_NUMBEROUT_ROW_NUMBER

Functions:

ArithmeticalADDBIT_ANDBIT_NOTBIT_ORBIT_SHLBIT_SHRBIT_XORDIVEXPEXP10FLOORLOGLOG10MODMULQNTFLOOR_TO_DIGITQNTROUND_TO_DIGITQUANTIZEROUNDSQRSQRROOTSUB

ConversionHEX_TO_INTINT_TO_IPV4IPV4_TO_INTTO_DATETO_HEXTO_INTTO_LOCALTIME

TO_REALTO_STRINGTO_TIMETO_TIMESTAMPTO_UTCTIME

StringManipulationEXTRACT_EXTENSIONEXTRACT_FILENAMEEXTRACT_PATHEXTRACT_PREFIXEXTRACT_SUFFIXEXTRACT_TOKENEXTRACT_VALUEHEX_TO_ASCHEX_TO_HEX16HEX_TO_HEX32HEX_TO_HEX8HEX_TO_PRINTINDEX_OFLAST_INDEX_OFLTRIMREPLACE_CHRREPLACE_STRROT13RTRIMSTRCATSTRCNTSTRLENSTRREPEATSTRREVSUBSTRTO_LOWERCASETO_UPPERCASETRIM

URLESCAPEURLUNESCAPE

SystemInformationCOMPUTER_NAMERESOLVE_SIDREVERSEDNSSYSTEM_DATESYSTEM_TIMESYSTEM_TIMESTAMPSYSTEM_UTCOFFSET

MiscellaneousCASECOALESCEHASHMD5_FILEHASHSEQIN_ROW_NUMBEROUT_ROW_NUMBERREPLACE_IF_NOT_NULLSEQUENCEWIN32_ERROR_DESCRIPTION

Note:TheREPLACE_IF_NULLfunctionhasbeendeprecatedinfavoroftheCOALESCEfunction.

Seealso:AggregateFunctions

ConstantValuesFieldExpressions

ConstantValues<value> ::= <integer_constant>

<real_constant><string_constant><timestamp_constant><null_constant>

<integer_constant> ::= integer0xhexadecimal

<real_constant> ::= integer_part.fractional_part

<string_constant> ::= 'string'

<timestamp_constant> ::= TIMESTAMP('timestamp','format')

<null_constant> ::= NULL

Constantsareimmutablefield-expressions,andtheyaremostlyusedinexpressionsorasargumentsoffunctions.

Constants:

<integer_constant>ConstantvaluesoftheINTEGERtypecanbeenteredasdecimalnumbers,orashexadecimalnumbersprecededbythe"0x"prefix.FormoreinformationabouttheLogParserINTEGERdatatype,seeINTEGERDataType.

<real_constant>

ConstantvaluesoftheREALtypeareenteredasdecimalnumberscontainingadecimalpoint.FormoreinformationabouttheLogParserREALdatatype,seeREALDataType.

<string_constant>ConstantvaluesoftheSTRINGtypeareenteredasstringsenclosedbysinglequotecharacters(').Thesinglequotecharacter(')andthebackslashcharacter(\)areconsideredspecialcharactersinastringconstant,andtheycanonlybeenteredasescapesequencesprecededbyabackslashcharacter(\'and\\),asinthefollowingexample:

'Contains\'singlequoteand\\backslash'

Inaddition,anycharacter(includingillegalcharactersandnon-printablecharacters)canbeenteredusingthe\uxxxxnotation,wherexxxxisthe4-digithexadecimalrepresentationofthedesiredUNICODEcharacter,asinthefollowingexample:

'Contains\u0009tabs'

FormoreinformationabouttheLogParserSTRINGdatatype,seeSTRINGDataType.

<timestamp_constant>ConstantvaluesoftheTIMESTAMPtypeareenteredwiththespecialTIMESTAMPkeyword,followedbyastringrepresentationofthedesiredtimestamp,andbytheformatofthestringrepresentationofthedesiredtimestamp,usingtheLogParserTimestampFormatSpecifiers.Ifthetimestampformatspecifiersincludedatespecifiersonly,theresultingTIMESTAMPvaluewillbeadate-onlytimestamp.Similarly,ifthetimestampformatspecifiersincludetimeofdayspecifiersonly,theresultingTIMESTAMPvaluewillbeatime-onlytimestamp.FormoreinformationabouttheLogParserTIMESTAMPdatatype,

seeTIMESTAMPDataType.

<null_constant>ConstantvaluesoftheNULLtypeareenteredwiththespecialNULLkeyword.FormoreinformationabouttheLogParserNULLdatatype,seeNULLDataType.

Remarks:Integerconstantsenteredashexadecimalnumbersareconvertedinternallytodecimalvalues.Toforceanoutputformattodisplayanintegerfield-expressionasanhexadecimalvalue,usetheTO_HEXfunction.

Examples:

A.Integerconstantenteredasdecimalnumber

sc-bytes>=1000

B.Integerconstantenteredashexadecimalnumber

BIT_AND(Flags,0x1000)

C.Realconstant

AVG(time-taken)<75.45

D.Stringconstant

'Somestring'

E.Stringconstantcontainingspecialcharacters

'Contains\'singlequoteand\\backslash'

F.StringconstantcontainingUNICODEcharacters

'Containsa\u2530UNICODEcharacter'

G.Timestampconstant

TimeGenerated>TIMESTAMP('2004-05-2819:12:43','yyyy-MM-ddhh:mm:ss')H.Date-onlytimestampconstant

date>TIMESTAMP('2004-05-28','yyyy-MM-dd')

I.Time-onlytimestampconstant

time>TIMESTAMP('19:12:43','hh:mm:ss')

J.NULLconstant

Message<>NULL

Seealso:FieldExpressionsINTEGERDataTypeREALDataTypeSTRINGDataTypeTIMESTAMPDataTypeNULLDataType

BasicsofaQuery

Comments<comment> ::= /*text_of_comment*/

--text_of_comment

Commentsareuser-providedtextnotevaluatedbyLogParser,usedtodocumentcodeortemporarilydisablepartsofquerystatements.

Remarks:Use--forsingle-lineornestedcomments.Commentsinsertedwith--aredelimitedbythenewlinecharacter.Multiple-linecommentsmustbeindicatedby/*and*/.Thereisnomaximumlengthforcomments.

Examples:

A.Single-linecomments

SELECTTimeGenerated,SourceNameFROMSystem--WeareusingtheSYSTEMeventlogB.Multiple-linecomments

SELECTTypeName,COUNT(*)ASTotalCountUSINGTO_UPPERCASE(EXTRACT_TOKEN(EventTypeName,0,''))ASTypeNameINTOReport.csvFROMSystem/*Weonlywanttoretrieveeventlogswhosetypenamecontains'service'

*/WHERETypeNameLIKE'%service%'GROUPBYTypeNameHAVINGTotalCount>5ORDERBYTotalCountDESC

DataTypesIntheLogParserSQL-Likelanguage,eachfield-expressionhasarelateddatatype,whichisanattributethatspecifiesthetypeofdatathatthefield-expressioncanhold.LogParsersuppliesasetofsystemdatatypesthatdefineallofthetypesofdatathatcanbeusedwithLogParser.Thesetofsystem-supplieddatatypesis:

INTEGER:integernumericdata;REAL:floatingprecisionnumericdata;STRING:variablelengthUNICODEcharacterstringdata;TIMESTAMP:dateandtimedata;NULL:unknownorunavailabledata.

INTEGERDataTypeTheINTEGERdatatyperepresentsinteger(wholenumber)numericdata.

Valuerange:

INTEGERvaluesarerepresentedassigned64-bit(8-byte)integernumbers,withvaluesrangingfrom-2^63(-9,223,372,036,854,775,808)through2^63-1(9,223,372,036,854,775,807).

ConversionFunctions:

OtherdatatypestoINTEGERdatatype:TO_INT

INTEGERdatatypetootherdatatypes:TO_REALTO_STRINGTO_TIMESTAMP

Seealso:ConstantValues

REALDataTypeTheREALdatatyperepresentsfloatingpointnumericdata.Floatingpointdataisapproximate;notallvaluesinthedatatyperangecanbepreciselyrepresented.

Valuerange:

REALvaluesarerepresentedassigned64-bit(8-byte)floatingpointnumbers,withvaluesrangingfrom±5.0×10-324through±1.7×10308,withatleast15digitsofprecision.

OtherdatatypestoREALdatatype:TO_REAL

REALdatatypetootherdatatypes:TO_INTTO_STRINGTO_TIMESTAMP

STRINGDataTypeTheSTRINGdatatyperepresentsvariablelengthUNICODEcharacterstringdata.

OtherdatatypestoSTRINGdatatype:TO_STRING

STRINGdatatypetootherdatatypes:TO_INTTO_REALTO_TIMESTAMP

TIMESTAMPDataTypeTheTIMESTAMPdatatyperepresentsdateandtimeofdaydata.

Valuerange:

TIMESTAMPvaluesrangefromJanuary1,-8192throughDecember31,8191,toanaccuracyofonehundrednanoseconds(oneten-thousandthofamillisecond).

Date-onlyandTime-onlyTimestamps

TIMESTAMPvaluescanberestrictedtorepresentdatedataonlyortimeofdaydataonly.AsexplainedintheRemarkssectionbelow,aTIMESTAMPvaluethathasbeenrestrictedtorepresentdatedataonlyortimeofdaydataonlywillbeformattedtodisplaydateelementsonly(year,month,andday)ortimeofdayelementsonly(hour,minute,second,millisecond,andnanosecond).TIMESTAMPvaluescanberestrictedtodate-onlyortime-onlytimestampsindifferentways.SomeinputformatsreturnTIMESTAMPinputrecordfieldswhosevaluesrepresentonlydatesortimesofday.Forexample,the"date"and"time"fieldsoftheIISW3Cinputformathavevaluesrepresentingonlydatesandtimesofday,respectively.TIMESTAMPconstantscanalsobeenteredasdate-onlyortime-onlytimestampvalues,dependingontheTimestampFormatSpecifiersused.Inaddition,theTO_DATE,TO_TIME,SYSTEM_DATE,andSYSTEM_TIMEfunctionsallreturnTIMESTAMPvaluesrepresentingdatesortimesofdayonly.Formoreinformation,refertotheRemarkssectionbelow.

Remarks:

TIMESTAMPvaluesareformattedandparsedusingTimestampFormatSpecifiers.Timestampformatspecifiersarestringsthatusespecialcharacterstodescribedateand/ortimeelementsinastringrepresentationofatimestamp.Formoreinformation,refertotheTimestampFormatSpecifiersreference.Althoughthedistinctionbetweendate-onlyortime-onlyTIMESTAMPvaluesandfullTIMESTAMPvaluesisoftentransparenttotheuser,date-onlyortime-onlyvaluesbehavedifferentlythanfullTIMESTAMPvaluesinthefollowingcircumstances:Comparisonoperatorsinexpressions:Whencomparingadate-onlyTIMESTAMPvaluewithanotherTIMESTAMPvalue,thetimeofdaydataofthedate-onlyvalueisassumedtobetimezero.Similarly,whencomparingatime-onlyTIMESTAMPvaluewithanotherTIMESTAMPvalue,thedatedataofthetime-onlyvalueisassumedtobeJanuary1,year0.FormattingTIMESTAMPvalues:wheneveradate-onlyortime-onlyTIMESTAMPvalueisformattedtoaSTRINGvaluebyeitherexplicitlyusingtheTO_STRINGfunctionorasimplicitlydonebyanoutputformat,theresultingSTRINGwillonlycontainthedateortimeofdaydata,andthenon-applicableTimestampFormatSpecifierswillbeignored.Asanexample,thefollowingqueryusestheTO_STRINGfunctionwithdateandtimeofdayformatspecifierstoformatthe"time"fieldoftheIISW3Cinputformat:

SELECTTO_STRING(time,'yyyy-MM-ddhh:mm:ss')FROM<1>Sincethevaluesofthe"time"fieldaretime-onlyTIMESTAMPvalues,theresultingSTRINGvalueswillbeformattedaccordingtothetimeofdayformatspecifiersonly,andthedateformatspecifierswillbeignored:

18:48:0418:48:2718:48:2718:48:29

ValuesoftypeTIMESTAMPcanalsobeusedtorepresenttimeintervals,forexamplewiththeADDandSUBfunctions.SincetheoriginoftimeintheLogParserSQL-Likelanguageis

January1,year0,timeintervalsshouldbeexpressedastimestampsrelativetothisoriginoftime.Forexample,atimeintervalofonedayshouldbespecifiedasJanuary2,year0,i.e.24hoursaftertheoriginoftime.Thefollowingexamplequeryselectsalltheeventlogrecordsthathavebeenwritteninthepast2days:

SELECT*FROMSYSTEMWHERETimeWritten>TO_LOCALTIME(SUB(SYSTEM_TIMESTAMP(),TIMESTAMP('0000-01-03','yyyy-MM-dd')))TIMESTAMPvaluesdonotcarryinformationonthetimezonethetimestampisrelativeto.WhenworkingwithTIMESTAMPfieldsgeneratedbyaninputformat,usersshouldbeawareofthetimezonethesefieldsarerelativeto,andhandletheirvaluesaccordingly.Forexample,valuesofthe"TimeGenerated"fieldoftheEVTInputFormatarerelativetothelocaltimezone.IfUniversalTimeCoordinates(UTC)aredesired,theTO_UTCTIMEfunctionshouldbeusedtoconverttheselocaltimestampstoUTCtimestamps.

OtherdatatypestoTIMESTAMPdatatype:TO_TIMESTAMP

TIMESTAMPdatatypetootherdatatypes:TO_INTTO_REALTO_STRING

FullTIMESTAMPvaluestodate-onlyTIMESTAMPvalues:TO_DATE

FullTIMESTAMPvaluestotime-onlyTIMESTAMPvalues:

TO_TIME

Date-onlyandtime-onlyTIMESTAMPvaluestofullTIMESTAMPvalues:TO_TIMESTAMP

LocaltimezoneTIMESTAMPvaluestoUTCTIMESTAMPvalues:TO_UTCTIME

UTCTIMESTAMPvaluestolocaltimezoneTIMESTAMPvalues:TO_LOCALTIME

Seealso:ConstantValuesTimestampFormatSpecifiers

TimestampFormatSpecifiersTIMESTAMPvaluesareformattedandparsedusingTimestampFormatSpecifiers.Timestampformatspecifiersarestringsthatusespecialcharacterstodescribedateand/ortimeelementsinastringrepresentationofatimestamp.

Timestampformatspecifiersareusedinthefollowingcircumstances:

WhenenteringaTIMESTAMPconstantwiththeTIMESTAMPkeyword.Inthiscase,timestampformatspecifiersareusedtodescribehowthestringenteredshouldbeparsedinordertoobtainaTIMESTAMPvalue,asinthefollowingexample:

TimeGenerated>TIMESTAMP('2004-05-2810:23:15','yyyy-MM-ddhh:mm:ss')WhenconvertingaTIMESTAMPvaluetoaSTRINGvalueusingtheTO_STRINGfunction.Inthiscase,timestampformatspecifiersareusedtodescribehowtheTIMESTAMPvalueshouldbeformattedinordertoobtainaSTRINGvalue,asinthefollowingexample:

TO_STRING(TimeGenerated,'yyyyMMM,ddh:m:s')

WhenconvertingaSTRINGvaluetoaTIMESTAMPvalueusingtheTO_TIMESTAMPfunction.Inthiscase,timestampformatspecifiersareusedtodescribehowtheSTRINGvalueshouldbeparsedinordertoobtainaTIMESTAMPvalue,asinthefollowingexample:

TO_TIMESTAMP(Text,'MMMdddyyyy')

WhenspecifyinghowaninputformatshouldparseTIMESTAMPfields,usingthe"iTsFormat"parameter.Inthiscase,timestampformatspecifiersareusedtodescribehowtimestampvaluesarerepresentedbytheselecteddatasource,sothattheinputformatiscapabletoparsethesefieldsandrepresentthemasvaluesoftypeTIMESTAMP.Thefollowingexamplesetsaspecificvalueforthe"iTsFormat"

parameteroftheCSVInputFormat:

C:\>logparser"SELECTMyFieldFROMfile.csv"-i:CSV-iTsFormat:"yyyy-MM-dd"WhenspecifyinghowanoutputformatshouldformatanddisplayTIMESTAMPfields,usingthe"oTsFormat"parameter.Inthiscase,timestampformatspecifiersareusedtodescribehowTIMESTAMPvaluesshouldbeformattedbytheoutputformat,asinthefollowingexampleusingtheTSVOutputFormat:

C:\>logparser"SELECTTimeGeneratedINTOfile.txtFROMSystem"-i:EVT-o:TSV-oTsFormat:"yyyy-MM-dd"

ThefollowingtabledescribesthetimestampformatspecifierssupportedbytheLogParserSQL-Likelanguage:

Specifier Description

Examplespecifierstrings Exampleformats

y year,lastdigit(whenparsing,assumedtoberelativetoyear2000)

yMMdd 40528

yy year,last2digits(whenparsing,assumedtoberelativetoyear2000)

yyMMdd 040528

yyy year,last3digits(whenparsing,assumedtoberelativetoyear2000)

yyyMMdd 0040528

yyyy year,4digits yyyyMMdd 20040528M month,noleadingzero yyyy-M-dd 2004-5-28

2004-12-01MM month,leadingzero yyyy-MM-dd 2004-05-28

2004-12-01

MP month,leadingspace yyyy-MP-dd 2004-5-282004-12-01

MX month,withorwithoutleadingzero(whenparsing)month,withoutleadingzero(whenformatting)

yyyy-MX-dd 2004-05-28(whenparsing)2004-5-282004-12-01

MMM month,3-characterabbreviationofname(1)

MMMd,yyyy Dec1,2004

MMMM month,fullname(1) MMMMd,yyyy

December1,2004

d day,noleadingzero yyyy-MM-d 2004-12-12004-05-28

dd day,leadingzero yyyy-MM-dd 2004-12-012004-05-28

dp day,leadingspace yyyy-MM-dp 2004-12-12004-05-28

dx day,withorwithoutleadingzero(whenparsing)day,withoutleadingzero(whenformatting)

yyyy-MM-dx 2004-12-01(whenparsing)2004-12-12004-05-28

ddd weekday,3-characterabbreviationofname(1)

dddMMMMd,yyyy

WedDecember1,2004

dddd weekday,fullname(1)

ddddMMMMd,yyyy

WednesdayDecember1,2004

h,H hour,noleadingzero h:mm:ss 3:12:0521:04:15

hh,HH hour,leadingzero hh:mm:ss 03:12:0521:04:15

hp,HP hour,leadingspace hp:mm:ss 3:12:0521:04:15

hx,HX hour,withorwithoutleadingzero(whenparsing)hour,withoutleadingzero(whenformatting)

hx:mm:ss 03:12:05(whenparsing)3:12:0521:04:15

m minute,noleadingzero

hh:m:ss 21:4:1503:12:05

mm minute,leadingzero hh:mm:ss 21:04:1503:12:05

mp minute,leadingspace hh:mp:ss 21:4:1503:12:05

mx minute,withorwithoutleadingzero(whenparsing)minute,withoutleadingzero(whenformatting)

hh:mx:ss 21:04:15(whenparsing)21:4:153:12:05

s second,noleadingzero

hh:mm:ss 03:12:521:04:15

ss second,leadingzero hh:mm:ss 03:12:0521:04:15

sp second,leadingspace hh:mm:sp 03:12:521:04:15

sx second,withorwithoutleadingzero(whenparsing)second,withoutleadingzero(whenformatting)

hh:mm:ss 03:12:05(whenparsing)03:12:521:04:15

l millisecond,noleadingzeroes

hh:mm:ss.l 21:4:15.503:12:05.395

ll millisecond,leadingzeroes

hh:mm:ss.ll 21:04:15.00503:12:05.395

lp millisecond,leadingspaces

hh:mm:ss.lp 21:04:15.503:12:05.395

lx millisecond,withorwithoutleadingzero(whenparsing)millisecond,withoutleadingzero(whenformatting)

hh:mm:ss.lx 21:04:15.005(whenparsing)21:04:15.53:12:05.395

n nanosecond,noleadingzeroes

hh:mm:ss.ll.n 21:4:15.005.40003:12:05.395.1900

nn nanosecond,leadingzeroes

hh:mm:ss.ll.nn 21:04:15.005.0000040003:12:05.395.001900

np nanosecond,leadingspaces

hh:mm:ss.ll.np 21:04:15.005.40003:12:05.395.1900

nx nanosecond,withorwithoutleadingzero(whenparsing)nanosecond,withoutleadingzero(whenformatting)

hh:mm:ss.ll.nx 21:04:15.005.00000400(whenparsing)21:04:15.005.4003:12:05.395.1900

tt AM/PMnotation hh:mm:sstt 09:04:15PM03:12.05AM

? anycharacter(whenparsing)space(whenformatting)

yyyy-MM-dd?hh:mm:ss

2004-05-28T21:04:15(whenparsing)2004-05-2821:04:15(whenformatting)

anyother

characterverbatimcharacter hh:mm:ss---

yyyy.MM+dd09:04:15---2004.05+28

Notes:(1):elementnamesareobtainedfromthecurrentsystemlocale.

Date-onlyandTime-onlyTimestampsWhenparsingatimestampstring,thefollowingassumptionsaremade:

Ifthetimestampformatspecifiersincludedateelementsonly,theresultingTIMESTAMPvaluewillbeadate-onlytimestamp;forexample,thefollowingstatementcreatesadate-onlyTIMESTAMPconstantvalue:

TIMESTAMP('2004-05-28','yyyy-MM-dd')

Ifthetimestampformatspecifiersincludetimeofdayelementsonly,theresultingTIMESTAMPvaluewillbeatime-onlytimestamp;forexample,thefollowingstatementcreatesatime-onlyTIMESTAMPconstantvalue:

TIMESTAMP('21:04:15','hh:mm:ss')

UnspecifieddateelementsarereplacedwiththecorrespondingelementsoftheLogParserorigindate(January1,year0),unlessthetimestampisatime-onlytimestampvalue;forexample,thefollowingstatementcreatesadate-onlytimestamprepresentingthedateFebruary1,year0:

TIMESTAMP('2','M')

Similarly,unspecifiedtimeelementsarereplacedwithzerovalues,unlessthetimestampisadate-onlytimestampvalue;forexample,thefollowingstatementcreatesatime-onlytimestamprepresentingthetime10:00:00.0.0:

TIMESTAMP('10','h')

Asanotherexample,thefollowingstatementcreatesafulltimestampvaluerepresentingthetime10:00:00.0.0onFebruary1,year0:

TIMESTAMP('210','Mh')

Formoreinformationondate-onlyandtime-onlytimestampvalues,refertotheTimestampDataTypereference.

Seealso:ConstantValuesTimestampDataType

NULLDataTypeTheNULLdatatyperepresentsunknownorunavailabledata.

Remarks:InputformatsoftenreturnNULLvaluesforinputrecordfieldstoindicatethatthefielddataisnotavailableinthecurrentlog.AvalueofNULLisdifferentfromazerovalue.IntheLogParserSQL-Likelanguage,comparisonoperatorsinexpressionstreatNULLvaluesastheminimumpossiblevalues.Inotherwords,allnon-NULLvalues,evennegativenumericvalues,arealwaysgreaterthanaNULLvalue.Ontheotherhand,theMINandMAXaggregatefunctionstreatNULLvaluesasrespectivelythemaximumandminimumpossiblevalues.Inotherwords,theMINorMAXvaluebetweenanon-NULLvalueandaNULLvalueisalwaysthenon-NULLvalue.TotestforNULLvaluesinaqueryuseISNULLorISNOTNULLintheWHEREorHAVINGclauses.

Seealso:ConstantValuesExpressions

InputFormatsIISLogFileInputFormatsIISW3C:parsesIISlogfilesintheW3CExtendedLogFileFormat.IIS:parsesIISlogfilesintheMicrosoftIISLogFileFormat.BIN:parsesIISlogfilesintheCentralizedBinaryLogFileFormat.IISODBC:returnsdatabaserecordsfromthetablesloggedtobyIISwhenconfiguredtologintheODBCLogFormat.HTTPERR:parsesHTTPerrorlogfilesgeneratedbyHttp.sys.URLSCAN:parseslogfilesgeneratedbytheURLScanIISfilter.

GenericTextFileInputFormatsCSV:parsescomma-separatedvaluestextfiles.TSV:parsestab-separatedandspace-separatedvaluestextfiles.XML:parsesXMLtextfiles.W3C:parsestextfilesintheW3CExtendedLogFileFormat.NCSA:parseswebserverlogfilesintheNCSACommon,Combined,andExtendedLogFileFormats.TEXTLINE:returnslinesfromgenerictextfiles.TEXTWORD:returnswordsfromgenerictextfiles.

SystemInformationInputFormatsEVT:returnseventsfromtheWindowsEventLogandfromEventLogbackupfiles(.evtfiles).FS:returnsinformationonfilesanddirectories.REG:returnsinformationonregistryvalues.ADS:returnsinformationonActiveDirectoryobjects.

Special-purposeInputFormats

NETMON:parsesnetworkcapturefilescreatedbyNetMon.ETW:parsesEnterpriseTracingforWindowstracelogfilesandlivesessions.COM:providesaninterfacetoCustomInputFormatCOMPlugins.

ADSInputFormatTheADSinputformatreturnspropertiesofActiveDirectoryobjects.

TheADSinputformatenumeratestheActiveDirectoryobjectsintheActiveDirectoryContainerwhoseLDAPpathisspecifiedinthefrom-entity,eventuallyrecursingintoadditionalContainerobjectsfoundduringtheenumeration.TheinformationreturnedforeachobjectdependsonthevaluespecifiedfortheobjClassparameter.

WhentheobjClassparameterisleftunspecified,theADSinputformatworksin"propertymode",returningarecordforeachpropertyofeachobjectvisitedduringtheenumeration.Inthiscase,inputrecordshaveafixednumberoffieldswhosevaluesdescribethepropertiesbeingreturned,includinga"PropertyName"fieldanda"PropertyValue"fieldcontainingthenameandthevalueofthepropertybeingprocessed.Queriesoperatingin"propertymode"canworkonActiveDirectoryobjectsofdifferenttypes,andsinceeachinputrecordrepresentsasingleobjectproperty,theycanonlyreferenceasinglepropertyatatime.

Forexample,thefollowingcommandreturnsthevaluesofallthepropertiesnamed"comment"fromalltheobjectsinthespecifiedpath:

LogParser"SELECTPropertyValueFROMLDAP://mydomain.mycompany.comWHEREPropertyName='comment'"-i:ADSTheoutputwouldlooklikethefollowingexample:

PropertyValue-----------------BuiltinBuiltinAccountOperatorsAccountOperatorsAdministratorsAdministrators

WhenthenameofanActiveDirectoryobjectclassisspecifiedfortheobjClassparameter,theADSinputformatworksin"objectmode",returningarecordforeachobjectvisitedduringtheenumerationthatisaninstanceofthespecifiedclass.Inthiscase,thereisaninputrecordfieldforeachofthepropertiesofthe

BackupOperatorsBackupOperatorsobjectbeingreturned.Queriesoperatingin"objectmode"canonlyworkonActiveDirectoryobjectsofasingletype,andsinceeachinputrecordrepresentsasingleobject,theycanreferencemultiplepropertiesofthesameobjectatthesametime.

Forexample,thefollowingcommandreturnsthespecifiedpropertiesfromalltheobjectsoftype"Computer":

LogParser"SELECTcn,operatingSystem,operatingSystemServicePackFROMLDAP://mydomain.mycompany.com/CN=Computers,DC=mydomain,DC=mycompany,DC=com"-i:ADS-objClass:ComputerTheoutputwouldlooklikethefollowingexample:

cnoperatingSystemoperatingSystemServicePack-------------------------------------------------------------SERVER01WindowsXPProfessionalServicePack1SERVER02WindowsXPProfessionalServicePack2TESTMACHINE1WindowsServer2003-TESTMACHINE2WindowsXPProfessionalServicePack2TESTMACHINE3WindowsXPProfessionalServicePack1TESTMACHINE4Windows2000ServerServicePack4

From-EntitySyntaxFieldsParametersExamples

ADSInputFormatFrom-EntitySyntax<from-entity>

::= [[<provider>:]//[<username>:<password>@]<domain>]/<path>[;...]

The<from-entity>specifiedinqueriesusingtheADSinputformatisasemicolon-separatedlistofLDAPpaths.EachLDAPpathbeginswithanoptionalprovidername(e.g."IIS","LDAP"),followedbyanoptionaldomainorcomputername.Ifaprovidernameisnotspecified,then"IIS"isassumedbydefault.Ifadomainnameorcomputernameisnotspecified,then"localhost"isassumedbydefault.

Thefrom-entitycanoptionallyincludeausernameandapasswordtobeusedfortheconnectiontotheActiveDirectoryprovider.Whenthesearenotspecified,theADSinputformatusesthecurrentuser'scredentials.

Note:LDAPpathscontainingcomma(,)charactersshouldbeenclosedwithinsingle-quote(')characters.

Examples:

FROMIIS://COMPUTER01/W3SVC/1

FROMIIS://MyUsername:MyPassword@COMPUTER01/W3SVC/1

FROM'LDAP://MyDomain/CN=Users,DC=MyDomain,DC=com'

FROM'LDAP://MyUsername:MyPassword@MyDomain/CN=Users,DC=MyDomain,DC=com'FROM/W3SVC/1;/W3SVC/2;//COMPUTER02/W3SVC/1

ADSInputFormatFieldsThestructureoftheinputrecordsgeneratedbytheADSinputformatdependsonthevaluespecifiedfortheobjClassparameter.

PropertyModeWhentheobjClassparameterisleftunspecified,theADSinputformatworksin"propertymode",returningarecordforeachpropertyofeachobjectvisitedduringtheenumeration.Inthiscase,inputrecordshavethefollowingfixedstructure:

Name Type Description

ObjectPath STRING FullActiveDirectorypathoftheobjectcontainingthisproperty

ObjectName STRING Nameoftheobjectcontainingthisproperty

ObjectClass STRING Classnameoftheobjectcontainingthisproperty

PropertyName STRING Nameofthepropertybeingprocessed

PropertyValue STRING Valueofthepropertybeingprocessed

PropertyType STRING Typeofthepropertybeingprocessed

Queriesoperatingin"propertymode"canworkonActiveDirectoryobjectsofdifferenttypes,andsinceeachinputrecordrepresentsasingleobjectproperty,theycanonlyreferenceasinglepropertyatatime.

ObjectModeWhenthenameofanActiveDirectoryobjectclassisspecifiedfortheobjClassparameter,theADSinputformatworksin"objectmode",returningarecordforeachobjectvisitedduringtheenumerationthatisaninstanceofthespecifiedclass.Inthiscase,thefirstinputrecordfieldisfixed,anditisdescribedinthefollowingtable:

ObjectPath STRING FullActiveDirectorypathoftheobjectbeingprocessed

Thisfieldisfollowedbyfieldsrepresentingallthepropertiesofthespecifiedobjectclass.Eachfieldisnamedafterthecorrespondingpropertyname,anditsdatatypeisdeterminedbythepropertytypedeclaredbytheActiveDirectoryschemaobjectforthespecifiedclass.

Queriesoperatingin"objectmode"canonlyworkonActiveDirectoryobjectsofasingletype,andsinceeachinputrecordrepresentsasingleobject,theycanreferencemultiplepropertiesofthesameobjectatthesametime.

ADSInputFormatParametersTheADSinputformatsupportsthefollowingparameters:

objClass

Values: ActiveDirectoryobjectclassname

Default: notspecified

Description: Objectclassnamefor"objectmode"operation.

Details: Whenthisparameterisleftunspecified,theADSinputformatworksin"propertymode",returningarecordforeachpropertyofeachobjectvisitedduringtheenumeration.Ontheotherhand,whenthenameofanActiveDirectoryobjectclassisspecifiedforthisparameter,theADSinputformatworksin"objectmode",returningarecordforeachobjectvisitedduringtheenumerationthatisaninstanceofthespecifiedclass.Formoreinformationonthedifferentmodesofoperation,seeFormatFields.

Example: -objClass:Userusername

Values: username

Description: UsernamefortheActiveDirectoryconnection.

Details: Whenausernameisnotspecifiedforthisparameter,theADSinputformatusestheusernamespecifiedinthefrom-entityofthequery.Ifthefrom-entitydoesnotincludeausername,theADSinputformatwillusethecurrentuser'scredentials.

Note:Forsecurityreasons,valuesspecifiedforthisparameterarenotpersistedwhenusingtheLogParsercommand-lineDefaultsOverrideMode.

Example: -username:MyUserpassword

Values: password

Description: PasswordfortheActiveDirectoryconnection.

Details: Passwordfortheusernamespecifiedwiththe"username"parameter.

Example: -password:MyPasswordrecurse

Values: recursionlevel(number)

Default: -1

Description: MaxADScontainerrecursionlevel.

Details: 0disablescontainerrecursion;-1enablesunlimitedrecursion.

Example: -recurse:2multiValuedSep

Values: anystring

Default: |

Description: Separatorbetweenvaluesofmulti-valuedtypes.

Details: Multi-valuedpropertyvaluesarereturnedasasinglestring,builtbyconcatenatingthemultiplevaluesoneaftertheotherusingthevalueofthisparameterasaseparatorbetweentheelements.

Example: -multiValuedSep:,

ignoreDSErrors

Values: ON|OFF

Default: ON

Description: IgnoreDirectoryServiceerrors.

Details: Whenthisparameterissetto"OFF",DirectoryServiceerrorsoccurringduringtheenumerationofobjectsandpropertiesarereturnedasErrors.Whenthisparameterissetto"ON",DirectoryServiceerrorsaresilentlyignored,andinputrecordfieldscorrespondingtounretrievableobjectsorpropertiesarereturnedasNULLvalues.

Example: -ignoreDSErrors:OFFparseBinary

Values: ON|OFF

Default: OFF

Description: Returnvalueofbinaryproperties.

Details: Thisparameterspecifieswhetherpropertiescontainingbinaryvaluesarereturnedornot.Whenthisparameterissetto"ON",binaryvaluesarereturnedasSTRINGvaluesformattedaccordingtothevaluespecifiedforthe"binaryFormat"parameter.

Example: -parseBinary:ONbinaryFormat

Values: ASC|PRINT|HEX

Default: HEX

Description: Formatofbinaryproperties.

Details: Whenthe"parseBinary"propertyissetto"ON",theADSinputformatreturnspropertiescontainingbinaryvalues.Inthiscase,binaryvaluesarereturnedasSTRINGvaluesformattedaccordingtothevaluespecifiedforthisparameter.Whenthisparameterissetto"ASC",databytesbelongingtothe0x20-0x7FrangearereturnedasASCIIcharacters,whiledatabytesoutsidetherangearereturnedasperiod(.)characters,asshowninthefollowingexample:

Bucket:02096553..rundll32.exe

Whenthisparameterissetto"PRINT",databytesrepresentingprintableASCIIcharactersarereturnedasASCIIcharacters,whiledatabytesthatdonotrepresentprintableASCIIcharactersarereturnedasperiod(.)characters,asshowninthefollowingexample:

Bucket:02096553rundll32.exeWhenthisparameterissetto"HEX",alldatabytesarereturnedastwo-digithexadecimalvalues,asshowninthefollowingexample:

4275636B65743A2030323039363535330D0A72756E646C6C33322E657865

Example: -binaryFormat:PRINT

ADSInputFormatExamplesUsers'JobTitlesRetrieveusers'jobtitlebreakdownfromActiveDirectory:

LogParser"SELECTtitle,MUL(PROPCOUNT(*),100.0)ASPercentageINTODATAGRIDFROM'LDAP://MyUsername:MyPassword@mydomain/CN=Users,DC=mydomain,DC=com'WHEREtitleISNOTNULLGROUPBYtitleORDERBYPercentageDESC"-objClass:UserIISAccessFlagsMetaBasePropertiesRetrievealltheAccessFlagspropertiesfromIISmetabaseobjects:

LogParser"SELECTObjectPath,PropertyValueFROMIIS://localhostWHEREPropertyName='AccessFlags'"

BINInputFormatTheBINinputformatparsesIISlogfilesintheCentralizedBinaryLogFileFormat.

WhenanIIS6.0webserverisconfiguredtologintheCentralizedBinaryLogFileFormat,alltheIISvirtualsiteshostedbytheserverloginasingle,server-widelogfile.Logfilesinthisformatarebinaryfiles,andtheinformationcontainedintheselogscannotbevisualizedbystandardtextfileprocessors.

From-EntitySyntaxFieldsExamples

BINInputFormatFrom-EntitySyntax<from-entity> ::= <filename>|<SiteID>[,<filename>|<SiteID>...]

The<from-entity>specifiedinqueriesusingtheBINinputformatisacomma-separatedlistof:

PathsofIISCentralizedBinarylogfiles;IISVirtualSite"identifiers".

"Siteidentifiers"mustbeenclosedwithinanglebrackets(<and>),andcanhaveoneofthefollowingvalues:ThenumericsiteID(e.g."<1>","<28163489>");Thetextvalueofthe"ServerComment"propertyofthesite(e.g."<MyExternalSite>","<www.margiestravel.com>");Thefully-qualifiedADSImetabasepathtothesite(e.g."<//MYSERVER/W3SVC/1>"),usingeitherthenumericsiteIDorthetextvalueofthe"ServerComment"propertyofthesite.

Whena"siteidentifier"isused,theBINinputformatconnectstothespecifiedmachine'smetabase,gathersinformationontheserver'scurrentloggingproperties,andparsesallthelogfilesintheserver'scurrentlogfiledirectory,returningonlytheentriescorrespondingtorequeststothespecifiedvirtualsite.

Filenamesand"Siteidentifiers"canalsoincludewildcards(e.g."LogFiles\ra04*.ibl","<www.*.com>").

Examples:

FROMLogFiles\ra04*.ibl,LogFiles\ra03*.ibl,\\MyServer\LoggingShare\W3SVC\ra04*.ibl

FROM<1>,<2>,<MyExternalSite>,raw9.ibl

FROM<www.net*home.com>,<//MyServer2/W3SVC/www.net*home.com>,<*>

BINInputFormatFieldsTheinputrecordsgeneratedbytheBINinputformatcontainthefollowingfields:

LogFilename STRING Fullpathofthelogfilecontainingthisentry

LogRow INTEGER Lineinthelogfilecontainingthisentry

ComputerName STRING Thenameoftheserverthatservedtherequest

SiteID INTEGER TheIISvirtualsiteinstancenumberthatservedtherequest

DateTime TIMESTAMP Thedateandtimeatwhichtherequestwasserved(UniversalTimeCoordinates(UTC)time)

ClientIpAddress STRING TheIPaddressoftheclientthatmadetherequest

ServerIpAddress STRING TheIPaddressoftheserverthatservedtherequest

ServerPort INTEGER Theserverportnumberthatreceivedtherequest

Method STRING TheHTTPrequestverb

ProtocolVersion STRING TheHTTPversionoftheclientrequest

ProtocolStatus INTEGER TheresponseHTTPstatuscode

SubStatus INTEGER TheresponseHTTPsub-statuscode

TimeTaken INTEGER Thenumberofmillisecondselapsedsincethemomenttheserverreceivedtherequesttothemomenttheserversentthelastresponsechunktotheclient

BytesSent INTEGER Thenumberofbytesintheresponsesentbytheserver

BytesReceived INTEGER Thenumberofbytesintherequestsentbytheclient

Win32Status INTEGER TheWindowsstatuscodeassociatedwiththeresponseHTTPstatuscode

UriStem STRING TheHTTPrequesturi-stem

UriQuery STRING TheHTTPrequesturi-query,orNULLiftherequestedURIdidnotincludeauri-query

UserName STRING Thenameoftheauthenticateduserthatmadetherequest,orNULLiftherequestwasfromananonymoususer

BINInputFormatExamplesTop20URL'sforaSiteCreateachartcontainingtheTOP20URL'sinthe"www.margiestravel.com"website(assumedtobeloggingintheCentralizedBinarylogformat):

LogParser"SELECTTOP20UriStem,COUNT(*)ASHitsINTOMyChart.gifFROM<www.margiestravel.com>GROUPBYUriStemORDERBYHitsDESC"-chartType:Column3D-groupSize:1024x768

COMInputFormatTheCOMinputformatprovidesaninterfacetoCustomInputFormatCOMPlugins.

WiththeLogParsercommand-lineexecutable,CustomInputFormatCOMPluginsareusedthroughtheCOMinputformat.ThisinputformattakestheProgIDofthepluginCOMobjectasavalueoftheiProgIDparameter,anditprovidesaninterfaceforcommand-lineoperationstousethecustominputformat.

WiththeLogParserscriptableCOMcomponents,CustomInputFormatCOMPluginobjectscanbeuseddirectlyasargumentstotheExecuteorExecuteBatchmethodsoftheLogQueryobject.Forthisreason,theCOMinputformatisnotprovidedasaLogParserscriptableCOMcomponent.

Seealso:CustomPluginsCOMInputFormatPluginsReference

COMInputFormatFrom-EntitySyntaxThe<from-entity>specifiedinqueriesusingtheCOMinputformatisdeliveredas-istothecustominputformatCOMobjectasanargumenttotheOpenInputmethodoftheILogParserInputContextinterface,anditssyntaxandinterpretationisprovidedbythecustominputformatselected.The<from-entity>specifiedinqueriesusingtheCOMinputformatmusthoweverobeythegeneralsyntaxfor<from-entity>languageelements.

COMInputFormatFieldsTheinputrecordsgeneratedbytheCOMinputformatcontainthefieldsprovidedbythecurrentlyselectedCustomInputFormatCOMplugin.

Thenumberoffields,theirnames,andtheirdatatypesareretrievedthroughtheGetFieldCount,GetFieldName,andGetFieldTypemethodsoftheILogParserInputContextinterface.

COMInputFormatParametersTheCOMinputformatsupportsthefollowingparameters:

iProgID

Values: COMProgID

Description: ProgIDoftheCustomInputFormatCOMPlugin.

Details: Thisparameterisusedtospecifytheversion-independentProgIDofthecustominputformatCOMobjectselectedforthecurrentquery.

Example: -iProgID:MSUtil.LogQuery.Sample.QFEiCOMParams

Values: name=value[,name=value...]

Description: ParametersfortheCustomInputFormatCOMPlugin.

Details: Thevalueofthisparameterisacomma-separatedlistofname-valuepairsspecifyingpropertynamesandvaluesforCustomInputFormatCOMPluginsimplementedthroughtheIDispatchCOMinterface.Ifpropertynamesortheirvaluescontainspacecharacters,thevalueofthisparametershouldbesurroundedbydouble-quote(")characters.FormoreinformationoncustompropertiesexposedbyCOMplugins,seeCustomPropertiesintheCOMInputFormatPluginsreference.

Example: -iCOMParams:TargetMachine=localhost,ExtendedFields=on

iCOMServer

Values: computername

Default: localhost

Description: ComputernameonwhichtheCustomInputFormatCOMPluginistobeinstantiated.

Details: PluginCOMobjectssupportingDistributedCOM(DCOM)canbeinstantiatedonaremotecomputer,thusprovidingameansforthecustominputformattoprocessdataonacomputerdifferentthanthecomputerrunningtheLogParserquery.

Example: -iCOMServer:MYSERVER01

COMInputFormatExamplesQFEInformationReturnQFEinformationfromthelocalmachine,usingthe"QFE"sampleCustomInputFormatCOMPlugin:

LogParser"SELECT*FROM."-i:COM-iProgID:MSUtil.LogQuery.Sample.QFE-iCOMParams:ExtendedFields=on

CSVInputFormatTheCSVinputformatparsescomma-separatedvaluestextfiles.

CSVtextfilesaregeneratedandhandledbyalargenumberofapplicationsandtools,including:

MicrosoftExcelPerfMonGenericspreadsheetapplications

InaCSVtextfile,eachlineconsistsofonerecord,andfieldsinarecordareseparatedbycommas.Dependingontheapplication,thefirstlineinaCSVfilemightbea"header",containingthelabelsoftherecordfields.ThefollowingexampleshowsaCSVfilebeginningwithaheader:

DateTime,PID,Comment5/28/200413:56:12,2956,Applicationstarted5/28/200413:59:02,2956,Waitingforinput5/28/200414:12:45,3104,Applicationstarted5/28/200415:24:42,1048,Applicationstarted

Moreover,fieldvaluesandlabelsmightbeenclosedwithindouble-quote(")characters,asshownbythefollowingPerfMonCSVlogfileexample:

"\\GAB1\Processor(_Total)\%ProcessorTime","\\GAB1\System\Processes""99.999993086289507","33""2.0000000000000018","33""1.0000000000000009","33""0.33333333333332993","33""0.33333333333332993","33""0","33""4.0000000000000036","33""4.3333333333333339","33"

Seealso:TSVInputFormatCSVOutputFormat

CSVInputFormatFrom-EntitySyntax<from-entity> ::= <filename>[,<filename>...]|

http://<url>|STDIN

The<from-entity>specifiedinqueriesusingtheCSVinputformatiseither:

Acomma-separatedlistofpathsofCSVfiles,eventuallyincludingwildcards;TheURLofafileintheCSVformat;The"STDIN"keyword,whichspecifiesthattheinputdataisavailablefromtheinputstream(commonlyusedwhenpipingcommandexecutions).

Examples:

FROMLogFiles1\*.csv,LogFiles2\*.csv,\\MyServer\FileShare\*.csv

FROMhttp://www.microsoft.adatum.com/MyCSVFiles/example.csv

typedata.csv|LogParser"SELECT*FROMSTDIN"-i:CSV

CSVInputFormatFieldsThestructureoftheinputrecordsgeneratedbytheCSVinputformatisdeterminedatruntime,dependingonthedatabeingparsed,andonthevaluesspecifiedfortheinputformatparameters.

Thefirsttwoinputrecordfieldsarefixed,andtheyaredescribedinthefollowingtable:

Filename STRING Fullpathofthefilecontainingthisentry

RowNumber INTEGER Lineinthefilecontainingthisentry

ThesetwofieldsarethenfollowedbythefieldsdetectedbytheCSVinputformatintheCSVfile(s)beingparsed.Thenumber,names,anddatatypesofthefieldsaredeterminedbyexamininginitiallytheCSVdataaccordingtothevaluesspecifiedfortheinputformatparameters.

ThenumberoffieldsdetectedbytheCSVinputformatduringtheinitialinspectionphasedictateshowtheCSVrecordfieldswillbeextractedfromtheinputdataduringthesubsequentparsingstage.IfaCSVlinecontainslessfieldsthanthenumberoffieldsestablished,themissingfieldsarereturnedasNULLvalues.Ontheotherhand,ifaCSVlinecontainsmorefieldsthanthenumberoffieldsestablished,theextrafieldsareparsedasiftheywerepartofthevalueofthelastfieldexpectedbytheCSVinputformat.

NumberofFieldsThenumberoffieldsinaninputrecordisdeterminedbytheinputCSVdataandbythevaluesofthenFieldsandfixedFieldsparameters.

Whenthe"nFields"parameterissetto-1,theCSVinputformatdeterminesthenumberoffieldsbyinspectingtheinputCSVdata.

Ifthe"fixedFields"parameterissetto"ON",indicatingthatalltherowsintheCSVfilehavethesamefixednumberoffields,thenthenumberoffieldsisdeterminedbyparsingeitherthefirstlineoftheCSVinputdata,orthefirstlineoftheheaderfilespecifiedwiththe"iHeaderFile"parameter.Ontheotherhand,ifthe"fixedFields"parameterissetto"OFF",indicatingthattherowsintheCSVfilehaveavariablenumberoffields,thenthenumberoffieldsisassumedtobethelargestnumberoffieldsfoundamongthefirstnlinesoftheCSVinputdata(eventuallyincludingthefirstlineoftheheaderfilespecifiedwiththe"iHeaderFile"parameter),wherenisthevalueofthe"dtLines"parameter.

Asanexample,thefollowingCSVfilecontainsavariablenumberoffields:

Name,City,AreaCodeJeff,Redmond,425Steve,Seattle,206,98101Edward,Olympia,360Whenparsedwiththe"nFields"parametersetto-1andthe"fixedFields"parametersetto"ON",thisCSVfilewouldyieldthreefields("Name","City",and"AreaCode").Inthiscase,theextrafourthfieldinthesecondrecordwouldbeparsedaspartofthethird"AreaCode"field,whosevaluewouldthenbe"206,98101".Ontheotherhand,ifthe"fixedFields"parameterissetto"OFF",andthe"dtLines"parameterissettoanyvaluegreaterthan2,thenthesameCSVfilewouldyieldfourfields("Name","City","AreaCode",andanadditionalfourthfielddetectedinthesecondCSVrecord).Inthiscase,thefirstandthirdrecordswouldhaveaNULLvalueforthefourthfield,andthesecondrecordwouldhavea"98101"valueforthefourthfield.

Whenthe"nFields"parameterissettoavaluegreaterthanzero,theCSVinputformatusesthespecifiedvalueasthenumberoffieldsintheinputdata.However,ifthe"fixedFields"parameterissetto"OFF",indicatingthattherowsintheCSVfilehaveavariablenumberoffields,thentheCSVinputformatusesthevalueofthe"nFields"parameterasa"suggestedminimum"numberoffields,anditexaminesthefirstnlinesoftheCSV

inputdata(eventuallyincludingthefirstlineoftheheaderfilespecifiedwiththe"iHeaderFile"parameter),wherenisthevalueofthe"dtLines"parameter,todeterminethenumberoffieldsamongtheselines.Iflinesarefoundcontainingmorefieldsthanthevaluespecifiedforthe"nFields"parameter,thenthenumberoffieldsisadjustedtothelargestnumberoffieldsfoundamongthefirstnlines.

ConsideringagainthepreviousCSVexamplefile,parsingthefilewiththe"nFields"parametersetto3andthe"fixedFields"parametersetto"ON"wouldyieldthreefields.However,settingthe"fixedFields"parameterto"OFF"andthe"dtLines"parametertoanyvaluegreaterthan2wouldyieldfourfields,detectingtheextrafieldinthesecondrecord.

FieldNamesThenamesofthefieldsinaninputrecordisdeterminedbytheinputCSVdataandbythevaluesoftheheaderRowandiHeaderFileparameters.

Whenthe"headerRow"parameterissetto"ON",theCSVinputformatassumesthatthefirstlineintheCSVfilebeingparsedisaheadercontainingthefieldnames.Inthiscase,ifthe"iHeaderFile"parameterisleftunspecified,theCSVinputformatextractsthefieldnamesfromtheheaderline.Ontheotherhand,ifthe"iHeaderFile"parameterissettothepathofaCSVfilecontainingatleastoneline,thentheCSVinputformatassumesthatthespecifiedfilecontainsaheader,parsesitsfirstlineonly,andextractsthefieldnamesfromthisline,ignoringthefirstlineoftheCSVfilebeingparsed.

Ifthenumberoffieldnamesextractedislessthanthenumberoffieldsdetected,theadditionalfieldsareautomaticallynamed"FieldN",withNbeingaprogressiveindexindicatingthefieldpositionintheinputrecord.

ConsideringthepreviousexampleCSVfile,settingthe"headerRow"parameterto"ON"wouldcausetheCSVinputformattousethefirstlineoftheCSVfileasaheadercontainingthefieldnames.Withthe"fixedFields"parametersetto"ON",theCSVinputformatwoulddetectthreefields,whosenameswouldbe"Name","City",and

"AreaCode".Ontheotherhand,withthe"fixedFields"parametersetto"OFF",theCSVinputformatwoulddetectfourfields,named"Name","City","AreaCode",and"Field4".

Whenthe"headerRow"parameterissetto"OFF",theCSVinputformatassumesthattheCSVfilebeingparseddoesnotcontainaheader,andthatitsfirstlineisthefirstdatarecordinthefile.Inthiscase,ifthe"iHeaderFile"parameterissettothepathofaCSVfilecontainingatleastoneline,thentheCSVinputformatassumesthatthespecifiedfilecontainsaheader,parsesitsfirstlineonly,andextractsthefieldnamesfromthisline.Ontheotherhand,ifthe"iHeaderFile"parameterisleftunspecified,thefieldsareautomaticallynamed"FieldN",withNbeingaprogressivenumberindicatingthefieldpositionintheinputrecord.

Asanexample,thefollowingCSVfiledoesnotcontainaheaderline:

Jeff,Redmond,425Steve,Seattle,206Edward,Olympia,360Whenparsedwiththe"headerRow"parameterto"OFF",theCSVinputformatassumesthatthefirstlineoftheCSVfileisthefirstdatarecordinthefile.Inthiscase,thethreefieldswouldbenamed"Field1","Field2",and"Field3".

FieldTypesThedatatypeofeachfieldextractedfromtheinputdataisdeterminedbyexaminingthefirstnCSVdatalines,wherenisthevaluespecifiedforthedtLinesparameter,inthefollowingway:Ifallthenon-emptyfieldvaluesinthefirstnlinesareformattedasdecimalnumbers,thenthefieldisassumedtobeoftheREALtype.Ifallthenon-emptyfieldvaluesinthefirstnlinesareformattedasintegernumbers,thenthefieldisassumedtobeoftheINTEGERtype.Ifallthenon-emptyfieldvaluesinthefirstnlinesareformattedastimestampsintheformatspecifiedbytheiTsFormatparameter,thenthefieldisassumedtobeoftheTIMESTAMPtype.Otherwise,thefieldisassumedtobeoftheSTRINGtype.

EmptyfieldvaluesarereturnedasNULLvalues.

CSVInputFormatParametersTheCSVinputformatsupportsthefollowingparameters:

headerRow

Values: ON|OFF

Default: ON

Description: SpecifieswhetherornottheinputCSVfile(s)beginwithaheaderline.

Details: Whenthisparameterissetto"ON",theCSVinputformatassumesthateachfilebeingparsedbeginswithaheaderline,containingthelabelsofthefieldsinthefile.Ifthe"iHeaderFile"parameterisleftunspecified,theCSVinputformatwillusethefieldnamesinthefirstfile'sheaderasthenamesoftheinputrecordfields.Ifavalueisspecifiedforthe"iHeaderFile"parameter,theCSVinputformatwillignoretheheaderlineineachfilebeingparsed.Whenthisparameterissetto"OFF",theCSVinputformatassumesthatthefile(s)beingparseddonotcontainaheader,andparsestheirfirstlineasdatarecords.Formoreinformationonheadersandfieldnames,seeCSVInputFormatFields.

Example: -headerRow:OFFiHeaderFile

Values: pathtoaCSVfile

Description: Filecontainingfieldnames.

Details: WhenparsingCSVfilesthatdonotcontainaheader

line,thefieldsoftheinputrecordsproducedbytheCSVinputformatarenamed"Field1","Field2",...Tooverridethisbehaviorandusemeaningfulfieldnames,thisparametercanbesettotothepathofaCSVfilecontainingaheaderline,causingtheCSVinputformattousethefieldnamesinthespecifiedCSVfile'sheaderlineasthenamesoftheinputrecordfields.OnlythefirstlineofthespecifiedCSVfileisparsed,andeventualadditionallinesareignored.Formoreinformationonheadersandfieldnames,seeCSVInputFormatFields.

Example: -iHeaderFile:"C:\MyFolder\header.csv"fixedFields

Values: ON|OFF

Default: ON

Description: SpecifieswhetherornotalltherecordsintheinputCSVfile(s)haveafixednumberoffields.

Details: Whenthisparameterissetto"ON",theCSVinputformatassumesthatthenumberoffieldsinalltheinputCSVrecordsequalsthenumberoffieldsfoundinthefirstCSVlineparsed,orthenumberoffieldsspecifiedforthe"nFields"parameter.Whenthisparameterissetto"OFF",theCSVinputformatassumesthattheinputCSVrecordshaveavariablenumberoffields,anditparsesthefirstnlinesoftheinputCSVdatatodeterminethemaximumnumberoffieldsintherecords,wherenisthevaluespecifiedforthe"dtLines"parameter.Formoreinformationonhowthenumberoffieldsisdetermined,seeCSVInputFormatFields.

Example: -fixedFields:OFF

nFields

Values: numberoffields(number)

Default: -1

Description: NumberoffieldsintheCSVdatarecords.

Details: Whenthe"fixedFields"parameterissetto"ON",thisparameterspecifiesthenumberoffieldsintheinputCSVdata.Whenthe"fixedFields"parameterissetto"OFF",thisparameterspecifiestheminimumnumberoffieldsintheinputCSVdata.Ifthefirstnlinesofinputdatacontainmorefieldsthanthespecifiednumberoffields,wherenisthevalueofthe"dtLines"parameter,thenthenumberoffieldsisassumedtobethemaximumnumberoffieldsfoundwithinthenlinesofdata.Thespecial"-1"valuespecifiesthatthenumberoffieldsistobedeductedbyinspectingthefirstnlinesofinputdata,wherenisthevalueofthe"dtLines"parameter.Formoreinformationonhowthenumberoffieldsisdetermined,seeCSVInputFormatFields.

Example: -nFields:3dtLines

Values: numberoflines(number)

Default: 10

Description: Numberoflinesexaminedtodeterminenumberoffieldsandfieldtypesatruntime.

Details: ThisparameterspecifiesthenumberofinitiallinesthattheCSVinputformatexaminestodeterminethenumberoftheinputrecordfieldsandthedatatypeofeachfield.

Ifthevalueis0,allfieldswillbeassumedtobeoftheSTRINGdatatype.Formoreinformationonhowthenumberoffieldsandtheirdatatypesaredetermined,seeCSVInputFormatFields.

Example: -dtLines:50iDQuotes

Values: Auto|Ignore

Default: Auto

Description: Behaviorwithdouble-quotedfields.

Details: Whenthisparameterissetto"Auto"andafieldvalueisenclosedwithindouble-quotecharacters("),theCSVinputformatparsesthefieldignoringcommacharacters(,)withinthedouble-quotes,andreturnstheenclosedvaluestrippingoffthesurroundingdouble-quotecharacters.Whensetto"Ignore",theCSVinputformatdoesnotperformanydouble-quoteprocessing,andfieldvaluesarereturnedverbatim,includingdouble-quotecharacters.

Example: -iDQuotes:IgnorenSkipLines

Default: 0

Description: Numberofinitiallinestoskip.

Details: Whenthisparameterissettoavaluegreaterthanzero,theCSVinputformatskipsthefirstnlinesofeachinputfilebeforeparsingitsheaderline,wherenisthevaluespecifiedforthisparameter.

Example: -nSkipLines:5comment

Values: anystring

Description: Skiplinesbeginningwiththisstring.

Details: Whenthisparameterissettoanon-emptystring,theCSVinputformatskipsalltheinputCSVlinesthatbeginwiththisstring.

Example: -comment:"MetaData:"iCodepage

Values: codepageID(number)

Default: 0

Description: CodepageoftheCSVfile.

Details: 0isthesystemcodepage,-1isUNICODE.

Example: -iCodepage:1245iTsFormat

Values: timestampformat

Default: yyyy-MM-ddhh:mm:ss

Description: FormatoftimestampvaluesintheinputCSVdata.

Details: Thisparameterspecifiesthedateand/ortimeformatusedintheCSVdatabeingparsed.ValuesoffieldsmatchingthespecifiedformatarereturnedasvaluesoftheTIMESTAMPdatatype.Formoreinformationondateandtimeformats,seeTimestampFormat

Specifiers.

Example: -iTsFormat:"MMMdd,yyyy"iCheckpoint

Values: checkpointfilename

Description: Loadandsavecheckpointinformationtothisfile.

Details: Thisparameterenablesthe"IncrementalParsing"featurethatallowssequentialexecutionsofthesamequerytoonlyprocessneweventsthathavebeenloggedsincethelastexecution.Formoreinformation,seeParsingInputIncrementally.

Example: -iCheckpoint:C:\Temp\myCheckpoint.lpc

CSVInputFormatExamplesAverageProcessorUsageperMinuteParseaPerfMonCSVlogfileandcalculatetheaverageprocessorusageperminute:

LogParser"SELECTQUANTIZE([(PDH-CSV4.0)(PacificDaylightTime)(420)],60)ASMinute,AVG([\\GAB1\Processor(_Total)\%ProcessorTime])ASAVGProcessorFROMPerfMon_000001.csvGROUPBYMinute"-i:CSV-iTsFormat:"MM/dd/yyyyhh:mm:ss.ll"

ETWInputFormatTheETWinputformatparsesEnterpriseTracingforWindowstracelogfiles(.etlfiles)andliveETWtracesessions.

EnterpriseTracingforWindows(ETW)isaframeworkforimplementingtracingprovidersthatcanbeusedfordebuggingandcapacityplanning.AnETWtracelogorlivesessionconsistsofastreamof"Events",eachpublishedbya"Provider".WindowseventprovidersincludetheKernel,IIS,COM+,andmanyotherWindowscomponents.Eacheventhasitsownsetofnamedproperties,orfields,containingtheeventdata.ThestructureofeacheventisdescribedbyaWMIclassderivedfromthe"EventTrace"classandregisteredwiththeWMIrepositoryduringthesetupoftheprovidercomponent.TheETWinputformatqueriestheWMIrepositoryfortheseclassesinordertoretrieveinformationaboutthestructureofeachevent.

ETWtracelogfilesandlivesessionscanbecontrolledthrougheitherthePerfMonutility,orthroughthetracelog.exeorlogman.execommand-linetools.

ETWInputFormatFrom-EntitySyntax<from-entity> ::= <etl_file_name>[,<etl_file_name>...]|

<live_session_name>

The<from-entity>specifiedinqueriesusingtheETWinputformatcanassumeoneofthefollowingvalues:

Acomma-separatedlistofpathsto.etlETWtracelogfiles;ThenameofanETWlivetracingsession.

Examples:

FROMMyTrace1.etl,MyTrace2.etl,MyTrace3.etl

FROM\\COMPUTER01\TraceFiles\MyTrace.etl,\\COMPUTER02\TraceFiles\MyTrace.etlFROMMyLiveSession

ETWInputFormatFieldsThestructureoftheinputrecordsgeneratedbytheETWinputformatisdeterminedatruntime,dependingontheETWtracebeingparsed,andonthevaluespecifiedforthefMode("fieldmode")parameter,whichcanbesetto"Compact","FNames","Full",or"Meta".

CompactFieldModeWhenthe"fMode"parameterissetto"Compact",theETWinputformatgeneratesaninputrecordforeacheventinthetracebeingparsed.Inthismode,inputrecordscontainfourfieldscommontoalltheevents,plusanadditional"UserData"fieldcontainingthevaluesofallthepropertiesspecifictotheeventbeingprocessed,concatenatedintoasinglestringvalueusingthecharacterspecifiedforthecompactModeSepparameterasaseparatorbetweenthevalues.Thefollowingtableshowsthefieldsoftheinputrecordsgeneratedinthe"Compact"fieldmode:

EventNumber INTEGER Indexofthiseventinthetracebeingparsed

EventName STRING Nameoftheevent

EventTypeName STRING Nameoftheeventtype

Timestamp TIMESTAMP Dateandtimeatwhichtheeventwastraced

UserData STRING Event-specificpropertyvalues

Thefollowingexampleshowssomesample"UserData"fieldvalues

generatedinthe"Compact"fieldmode:

UserData----------------------------------------------------DefaultAppPool|0|http://localhost:80/|GET{00000000-0000-0000-1200-0060000000fc}|/DefaultAppPool|0|http://localhost:80/default.htm|GET

The"Compact"fieldmodeprovidesaneasilyreadablewaytodisplaytheeventscontainedinanETWtrace,butqueriesoperatinginthismodecannotreferencepropertiesofaspecificevent.

FNamesFieldModeThe"FNames"fieldmodeoperatessimilartothe"Compact"fieldmode,buteachpropertyvalueinthe"UserData"fieldisprecededbythenameofthepropertyforbetterreadability.

Thefollowingexampleshowssomesample"UserData"fieldvaluesgeneratedinthe"FNames"fieldmode:

UserData-----------------------------------------------------------------------------------------------AppPoolId=DefaultAppPool|RawConnId=0|RequestURL=http://localhost:80/|RequestVerb=GETContextId={00000000-0000-0000-1200-0060000000fc}|RequestURL=/AppPoolId=DefaultAppPool|RawConnId=0|RequestURL=http://localhost:80/default.htm|RequestVerb=GET

FullFieldModeIn"Full"fieldmode,theETWinputformatgeneratesaninputrecordforeacheventinthetracebeingparsed.Inthismode,inputrecordscontainafieldforeachpropertyofeacheventgeneratedbytheprovidersinthetracebeingparsed.

Whenoperatingin"Full"fieldmode,theETWinputformatworkswithatwo-stageapproach.Duringthefirststage,theETWinputformatexaminestheinputtracetodeterminewhichprovidershaveloggedeventsinthetracebeingparsed.Whentheprovidersparameterisleftunspecified,theETWinputformatpre-processesanumberofeventsequaltothevaluespecifiedforthedtEventsLogordtEventsLiveparameters,dependingonwhetherornotthetracebeingparsedisatracelogfileoralivetracesession.Afterparsingtheseinitialevents,theETWinputformatassumesthatthetrace

beingparsedcontainsalltheeventsthatcanbeloggedbytheprovidersfoundamongtheseinitialevents.Ontheotherhand,whenthe"providers"parameterissettoeitheracomma-separatedlistofprovidernamesorGUIDsortothepathtoatextfilecontainingalistofprovidernamesorGUIDs,theETWinputformatassumesthatthetracebeingparsedcontainsalltheeventsthatcanbeloggedbythespecifiedproviders.

Oncethesetofprovidersloggingintheinputtracehasbeenidentified,theETWinputformat"constructs"theinputrecordstructure.Thefirst20inputrecordfieldsarecommontoalltheevents,andtheyaredescribedinthefollowingtable:

TraceName STRING Tracefileorsessionnamecontainingthisevent

EventNumber INTEGER Indexofthiseventinthetracebeingparsed

Timestamp TIMESTAMP Dateandtimeatwhichtheeventwastraced

InstanceID INTEGER InstanceIDfieldofthisevent

ParentInstanceID INTEGER ParentInstanceIDfieldofthisevent

ParentGUID STRING ParentGUIDfieldofthisevent

ProviderDescription STRING Nameoftheproviderofthisevent

ProviderGUID STRING GUIDoftheproviderofthisevent

EventName STRING Nameofthisevent

EventDescription STRING Descriptionofthisevent

EventVersion INTEGER Versionofthisevent

EventGUID STRING GUIDofthisevent

EventType INTEGER Typeofthisevent

EventTypeName STRING Nameofthiseventtype

EventTypeDescription STRING Descriptionofthiseventtype

EventTypeLevel INTEGER Levelofthiseventtype

ThreadID INTEGER IDofthethreadthatloggedthisevent

ProcessID INTEGER IDoftheprocessthatloggedthisevent

KernelTime INTEGER Elapsedexecutiontimeforkernelmodeinstructions,inCPUticks

UserTime INTEGER Elapsedexecutiontimeforusermodeinstructions,inCPUticks

These20fieldsarethenfollowedbytheunionofallthepropertiesofall

theeventsthatcanbeloggedbytheprovidersidentifiedduringthisstage.

Duringthesecondstage,theETWinputformatparsesthetraceeventsfrombeginningtoend,generatinganinputrecordforeachevent.Foranygivenevent,onlythefirst20inputrecordfieldsandthefieldscorrespondingtotheeventpropertiesarepopulatedwithavalue;alltheotherinputrecordfieldscorrespondingtopropertiesofothereventsaresettoNULLvalues.

Thefollowingsampleoutputshowsselectedfieldsfromtheinputrecordsgeneratedwhenparsingthepreviousexamplein"Full"fieldmode:

AppPoolIdRawConnIdContextIdRequestURLRequestVerb-------------------------------------------------------------------------------------------------------DefaultAppPool0-http://localhost:80/GET--{00000000-0000-0000-1200-0060000000fc}/-DefaultAppPool0-http://localhost:80/default.htmGET

Queriesoperatingin"Full"modecanrefertoindividualpropertiesofevents,buttheinputrecordsgeneratedcontaintoomanyfieldsfortheresultstobeeailyredable.

MetaFieldModeIn"Meta"fieldmode,theETWinputformatreturnsmeta-informationaboutevents,generatinganinputrecordforeachpropertyofeacheventthatcanbeloggedbyeachproviderinthetrace(s)beingparsed.Inputrecordscontainmeta-dataabouttheeventproperties,includinginformationaboutthepropertytype,informationabouttheeventcontainingtheproperty,andinformationabouttheprovidergeneratingtheevent.

The"Meta"fieldmodeemploysatwo-stageparsingschemasimilartothe"Full"fieldmode.Duringthefirststage,theETWinputformatpre-processestheinputtracetodeterminethesetofprovidersthatgeneratedeventsinthetrace.Inthismode,however,oncethesetofprovidershasbeenidentified,theETWinputformatdoesnotprocessthetrace,butratherreturnstheeventmeta-informationpopulatingtheinputrecordfieldsdescribedinthefollowingtable:

ProviderDescription STRING Descriptionoftheprovider

ProviderClassName STRING WMIclassnameoftheprovider

ProviderGUID STRING GUIDoftheprovider

EventName STRING Nameoftheevent

EventDescription STRING Descriptionoftheevent

EventVersion INTEGER Versionoftheevent

EventClassName STRING WMIclassnameoftheevent

EventGUID STRING GUIDoftheEvent

EventType INTEGER Typeoftheevent

EventTypeName STRING Nameoftheeventtype

EventTypeDescription STRING Descriptionoftheeventtype

EventTypeClassName STRING WMIclassnameoftheeventtype

EventTypeLevel INTEGER Leveloftheeventtype

FieldName STRING Nameofthiseventfield

FieldDescription STRING Descriptionofthiseventfield

FieldIndex INTEGER Indexofthisfieldamongtheevent'sfields

FieldType STRING WMItypeofthisfield

ETWInputFormatParametersTheEVTinputformatsupportsthefollowingparameters:

Values: Full|Compact|FNames|Meta

Default: FNames

Description: Operationmode.

Details: ThisparameterspecifieshowtheETWinputformatshouldreturntheinformationcontainedinthetrace(s)beingparsed.Formoreinformationonthedifferentfieldmodes,seeETWInputFormatFields.

Example: -fMode:Fullproviders

Values: filenameorcomma-separatedlistofprovidernamesorGUIDs

Description: Listofprovidersforthe"Full"or"Meta"fieldmodes.

Details: Thisparameterspecifiesthesetofprovidersloggingtotheinputtrace(s)toallowthe"Full"or"Meta"fieldmodestoearlydetecttheproviderstoprocess.Thevalueofthisparametercaneitherbythepathtoatextfilecontainingtheproviders'GUIDs(inthesameformatacceptedbythe"pf"argumentofthelogman.exetool),oracomma-separatedlistofprovidernamesorGUIDs.IfthisparameterisnotspecifiedwhentheETWinputformatoperatesin"Full"or"Meta"fieldmode,thenthesetofproviderswillbedetectedbypre-processingthefirstnevents,wherenisthevaluespecifiedforthe

"dtEventsLog"or"dtEventsLive"parameters.Formoreinformationaboutthedifferentfieldmodes,seeETWInputFormatFields.

Examples: -providers:MyProviders.guid -providers:"IIS:WWWServer,IIS:ActiveServerPages

(ASP)"dtEventsLog

Values: numberofevents(number)

Default: 3000

Description: Numberoftracelogfileeventsexaminedtodetectthesetofprovidersin"Full"or"Meta"fieldmodes.

Details: ThisparameterspecifiesthenumberofinitialeventsthattheETWinputformatexaminestodetectthesetofproviderslogginginaninputtracelogfilewhenoperatinginthe"Full"or"Meta"fieldmodes.Thevalueofthisparameterisonlyusedwhenthe"providers"parameterisleftunspecified.Formoreinformationaboutthedifferentfieldmodes,seeETWInputFormatFields.

Example: -dtEventsLog:100dtEventsLive

Values: numberofevents(number)

Default: 20

Description: Numberoflivetracesessioneventsexaminedtodetectthesetofprovidersin"Full"or"Meta"fieldmodes.

Details: ThisparameterspecifiesthenumberofinitialeventsthattheETWinputformatexaminestodetectthesetofproviderslogginginaninputlivetracesessionwhen

operatinginthe"Full"or"Meta"fieldmodes.Thevalueofthisparameterisonlyusedwhenthe"providers"parameterisleftunspecified.Formoreinformationaboutthedifferentfieldmodes,seeETWInputFormatFields.

Example: -dtEventsLive:100flushPeriod

Values: milliseconds

Default: 500

Description: Numberofmillisecondsbetweenlivetracesessionflushes.

Details: Whenprocessingalivetracesession,theinternalbufferingmechanismsoftheETWinfrastructuremightcauseeventstoappearwithanoticeabledelay.ThisparameterspecifieshowoftentheETWinputformatshouldforceabufferflushtoretrievereal-timeevents.

Example: -flushPeriod:2000ignoreEventTrace

Values: ON|OFF

Default: ON

Description: IgnoreEventTraceevents.

Details: Theveryfirsteventinanytracesessionisthe"EventTrace"event,whichcontainsmeta-dataaboutthetracesession.ThisparameterspecifieswhetherornotthiseventshouldbeprocessedandreturnedbytheETWinputformat.

Example: -ignoreEventTrace:OFF

compactModeSep

Values: anystring

Default: |

Description: Separatorbetweenthevaluesofthe"UserData"fieldinthe"Compact"or"FNames"fieldmodes.

Details: Whenoperatinginthe"Compact"or"FNames"fieldmodes,the"UserData"fieldcontainsallthepropertiesoftheeventbeingprocessedconcatenatedoneaftertheother,usingthevalueofthisparameterasaseparatorbetweentheelements.

Example: -compactModeSep:,expandEnums

Values: ON|OFF

Default: ON

Description: Expandenumerationeventproperties.

Details: ManyETWeventscontainnumericpropertieswhosevaluesdescribeenumerations.Thisparameterspecifieswhetherornotthenumericvaluesofpropertiesofthistypeshouldbeexpandedtoreturnthetextrepresentationoftheenumerationvalues.

Example: -expandEnums:OFFignoreLostEvents

Values: ON|OFF

Default: ON

Description: Ignorelostevents.

Details: ETWtracescontaininformationabouteventsthatmighthavebeenlostduringthetracingsession.Ifthisparameterissetto"OFF"andtheinputtraceindicatesthepresenceoflostevents,theETWinputformatgeneratesawarningwhenthetracehasbeencompletelyprocessedshowingthenumberofeventsthathavebeenlost.

Example: -ignoreLostEvents:OFFschemaServer

Values: computername

Description: Nameofcomputerwitheventschemainformation.

Details: ThisparameterspecifiesthenameofthecomputerwhoseWMIrepositorycontainstheschemainformationfortheeventsbeingparsed.Whenthisparameterisnotspecified,theETWinputformatconnectstothecomputerspecifiedinthefrom-entityifparsingatracefilefromaremotecomputer,ortothelocalcomputerifparsingalocaltracefileorlivetracingsession.

Example: -schemaServer:MYCOMPUTER02

ETWInputFormatExamplesParsinganIIS6.0ETWTraceLogFileThisexampleshowshowtostartatracesessioncontainingeventsfromtheIIS6.0providers,howtostopthesession,andhowtoparsetheresultingtracelogfile.TheexamplecommandsshownhereapplytoWindowsServer2003.

1. ListtheGUIDsoftheprovidersregisteredwiththesystemusingthefollowingcommandfromacommand-linewindow:

C:\>logmanqueryproviders

Theoutputofthiscommandwilllooklikethefollowingsample:

ProviderGUID-------------------------------------------------------------------------------IIS:WWWGlobal{d55d3bc9-cba9-44df-827e-132d3a4596c2}ACPIDriverTraceProvider{dab01d4d-2d48-477d-b1c3-daad0ce6f06b}ActiveDirectory:Kerberos{bba3add2-c229-4cdb-ae2b-57eb6966b0c4}IIS:SSLFilter{1fbecc45-c060-4e7c-8a0e-0dbd6116181b}IIS:RequestMonitor{3b7b0b4b-4b01-44b4-a95e-3c755719aebf}IIS:WWWServer{3a2a4e84-4c21-4981-ae10-3fda0d9b0f83}IIS:ActiveServerPages(ASP){06b94d9a-b15e-456e-a4ef-37c984a2cb4b}LocalSecurityAuthority(LSA){cc85922f-db41-11d2-9244-006008269001}IIS:IISADMINGlobal{DC1271C2-A0AF-400f-850

2. Identifytheprovidersneededforthetracesession;inthisexample,thetracesessionwillbeenabledforthe"IIS:WWWServer"and"IIS:ActiveServerPages(ASP)"providers.

3. CreateatextfilecontainingtheGUIDofeachselectedprovideronaline,followedbythetracingflagsandtracinglevelvaluesfortheprovider.Formoreinformationontheavailableflagsandlevelsforaprovider,consultthecomponentdocumentation.Thefollowingexampleshowsatextfilenamed"MyProviders.guid"containingthe"IIS:WWWServer"and"IIS:ActiveServerPages(ASP)"providers:

{3a2a4e84-4c21-4981-ae10-3fda0d9b0f83}0xfffffffe5{06b94d9a-b15e-456e-a4ef-37c984a2cb4b}0xffffffff5

4. Startthetracingsessionusingtheproviderstextfileastheargumentofthe"-pf"logmancommand-lineparameter:

C-4E42FE16BE1C}WindowsKernelTrace{9e814aad-3204-11d2-9a82-006008a86939}ASP.NETEvents{AFF081FE-0247-4275-9C4E-021F3DC1DA35}NTLMSecurityProtocol{C92CF544-91B3-4dc0-8E11-C580339A0BF8}IIS:WWWIsapiExtension{a1c2040e-8840-4c31-ba11-9871031a19ea}ActiveDirectory:SAM{8e598056-8993-11d2-819e-0000f875a064}HTTPServiceTrace{dd5ef90a-6398-47a4-ad34-4dcecdef795f}ActiveDirectory:NetLogon{f33959b4-dbec-11d2-895b-00c04f79ab69}SpoolerTraceControl{94a984ef-f525-4bf1-be3c-ef374056a592}

Thecommandcompletedsuccessfully.

C:\>logmanstartExampleTrace-pfMyProviders.guid-ets

5. Thetracingsessionhasnowstarted,andtheselectedproviderswillbeloggingeventsforeachrequesttotheIISWebServer.

6. Whendesired,thetracingsessioncanbestoppedwiththefollowingcommand:

C:\>logmanstopExampleTrace-ets

7. Afterthetracingsessionhasbeenstopped,theETWtracelogfilenamed"ExampleTrace.etl"isavailableforuse.ThefollowingLogParsercommandparsestheETWtracelogfileanddisplaystheloggedevents:

C:\>LogParser"SELECT*FROMExampleTrace.etl"-i:ETW

Theoutputofthiscommandwilllooklikethefollowingsample:

EventNumberEventNameEventTypeNameTimestampUserData--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------2IISGeneralGENERAL_REQUEST_START2004-10-1420:27:26.624.399000ContextId={00000000-0000-0000-1200-0060000000fc}|SiteId=1|AppPoolId=DefaultAppPool|ConnId=-288230375077969904|RawConnId=0|RequestURL=http://localhost:80/|RequestVerb=GET3IISFilterFILTER_START2004-10-1420:27:26.624.399000ContextId={00000000-0000-0000-1200-0060000000fc}|FilterName=C:\WINNT\Microsoft.NET\Framework\v1.1.4322\aspnet_filter.dll4IISFilterFILTER_PREPROC_HEADERS_START2004-10-1420:27:26.624.399000ContextId={00000000-0000-0000-12

ParsingaliveIIS6.0ETWTraceSessionThisexampleshowshowtostartalivetracesessioncontainingeventsfromtheIIS6.0providers,howtostartaLogParsercommandthatshowstheeventsinreal-time,andhowtostopthesession.TheexamplecommandsshownhereapplytoWindowsServer2003.

1. Executesteps1-3fromtheexampleabove.4. Startthetracingsessionusingtheproviderstextfileasthe

argumentofthe"-pf"logmancommand-lineparameter,specifyingalsothe"-rt"flagtoenableareal-timetracingsession:

C:\>logmanstartExampleTrace-pfMyProviders.guid-ets-rt

00-0060000000fc}5IISFilterFILTER_PREPROC_HEADERS_END2004-10-1420:27:26.624.399000ContextId={00000000-0000-0000-1200-0060000000fc}6IISFilterFILTER_END2004-10-1420:27:26.624.399000ContextId={00000000-0000-0000-1200-0060000000fc}7IISFilterFILTER_START2004-10-1420:27:26.624.399000ContextId={00000000-0000-0000-1200-0060000000fc}|FilterName=C:\ProgramFiles\CommonFiles\MicrosoftShared\WebServerExtensions\50\bin\fpexedll.dll8IISFilterFILTER_PREPROC_HEADERS_START2004-10-1420:27:26.624.399000ContextId={00000000-0000-0000-1200-0060000000fc}9IISFilterFILTER_PREPROC_HEADERS_END2004-10-1420:27:26.624.399000ContextId={00000000-0000-0000-1200-0060000000fc}10IISFilterFILTER_END2004-10-1420:27:26.624.399000ContextId={00000000-0000-0000-1200-0060000000fc}11IISCacheURL_CACHE_ACCESS_START2004-10-1420:27:26.624.399000ContextId={00000000-0000-0000-1200-0060000000fc}|RequestURL=/

5. Thetracingsessionhasnowstarted,andtheselectedproviderswillbeloggingeventsforeachrequesttotheIISWebServer.

6. Fromaseparatecommand-lineshellwindow,executethefollowingLogParsercommandtoparsethelivetracingsessioninreal-time:

C:\>LogParser"SELECT*FROMExampleTrace"-i:ETW

ThisLogParsercommandwilloutputthetraceeventsindefinitely,untilthecommandismanuallyaborted,oruntilthetracingsessionisstopped.

7. Whendesired,thetracingsessioncanbestoppedwiththefollowingcommand:

C:\>logmanstopExampleTrace-ets

EVTInputFormatTheEVTinputformatreturnseventsfromtheWindowsEventLogandfromEventLogbackupfiles(.evtfiles).

ThisinputformatreadseventinformationfromtheWindowsEventLog,includinglocalandremoteSystem,Application,Security,andcustomeventlogs,aswellasfromEventLogbackupfiles.

EVTInputFormatFrom-EntitySyntax<from-entity> ::= <event_log>[,<event_log>...]

<event_log> ::= [\\<computer_name>\]<event_log_name>|<event_log_backup_filename>

The<from-entity>specifiedinqueriesusingtheEVTinputformatisacomma-separatedlistof:

NamesofEventLogs("System","Application","Security",oracustomeventlog),optionallyprecededbythenameofaremotecomputerintheUNCnotation;PathsofEventLogbackupfiles(.evtfiles),optionallyincludingwildcards.

Namesofcustomeventlogsthatincludespacecharactersmustbespecifiedwithinsingle-quotecharacters.

Examples:

FROMSystem,Application,\\SERVER2\System,\\SERVER2\Application

FROMSystem,Application,'MyCustomEventLog'

FROMD:\MyEVTLogs\*.evt,\\SERVER2\D$\MyEVTLogs\*.evt

FROMSystem,D:\MyEVTLogs\System.evt

EVTInputFormatFieldsTheinputrecordsgeneratedbytheEVTinputformatcontainthefollowingfields:

EventLog STRING NameoftheEventLogorEventLogbackupfilecontainingthisevent

RecordNumber INTEGER IndexofthiseventintheEventLogorEventLogbackupfilecontainingthisevent

TimeGenerated TIMESTAMP Thedateandtimeatwhichtheeventwasgenerated(localtime)

TimeWritten TIMESTAMP Thedateandtimeatwhichtheeventwaslogged(localtime)

EventID INTEGER TheIDoftheevent

EventType INTEGER Thenumerictypeoftheevent

EventTypeName STRING Thedescriptivetypeoftheevent

EventCategory INTEGER Thenumericcategoryofthe

EventCategoryName STRING Thedescriptivecategoryoftheevent

SourceName STRING Thesourcethatgeneratedtheevent

Strings STRING Thetextualdataassociatedwiththeevent

ComputerName STRING Thenameofthecomputeronwhichtheeventwasgenerated

SID STRING TheSecurityIdentifierassociatedwiththeevent

Message STRING Thefulleventmessage

Data STRING Thebinarydataassociatedwiththeevent

EVTInputFormatParametersTheEVTinputformatsupportsthefollowingparameters:

fullText

Values: ON|OFF

Default: ON

Description: Retrievethefulltextmessage.

Details: Thisparameterenables/disablestheretrievalofEventLogtextmessages.

Example: -fullText:OFFresolveSIDs

Values: ON|OFF

Default: OFF

Description: ResolveSIDvaluesintofullaccountnames.

Details: Whensetto"ON",thisparametercausestheEVTinputformattoperformanaccountnamelookupforeachSIDvalueintheeventsbeingparsed,andreturntheaccountnameinsteadoftheSIDalphanumericalvalue.

Example: -resolveSIDs:ONformatMsg

Values: ON|OFF

Default: ON

Description: Formatthetextmessageasasingleline.

Details: Eventtextmessagesoftenspanmultiplelines.Whenthisparameteris

setto"ON",theEVTinputformatpreservesreadabilityofthebyremovingcarriage-return,line-feed,andmultiplespacecharactersfromthemessagetext.Whenthisparameterissetto"OFF",theEVTinputformatreturnstheoriginalmessagetextwithnointerveningpost-processing.

Example: -formatMsg:OFFmsgErrorMode

Values: NULL|ERROR|MSG

Default: MSG

Description: Behaviorwheneventmessagesoreventcategorynamescannotberesolved.

Details: Thetextofaneventlogmessageandthetextualnameofitscategoryarestoredinbinaryfilesinstalledwiththeapplicationthatgeneratestheeventlog.Insomecases,uninstallingtheapplicationorreconfiguringtheapplicationmightcausethelossofthenecessarybinaryfiles,thusmakingitimpossibletoretrievethetextdataforthoseeventsthathadbeenloggedpriortothereconfiguration.ThisparameterspecifiesthedesiredbehaviorfortheEVTinputformatwhenaneventlogmessagetextoritscategorynamecannotberetrieved.Whenthisparameterissetto"NULL",the"Message"or"EventCategoryName"fieldvalueisreturnedasaNULLvalue.Whensetto"ERROR",aparseerrorisreturned.Whensetto"MSG",amessageisreturnedforthefield,specifyingthatthetextofthemessageorthecategorynamecouldnotbefound.

Example: -msgErrorMode:NULLfullEventCode

Values: ON|OFF

Default: OFF

Description: ReturnthefulleventIDcodeinsteadofthefriendlycode.

Details: Whenthisparameterissetto"ON",theEVTinputformatreturnsthefull32-bitvalueoftheeventIDcode.Whensetto"OFF",theEVTinputformatreturnsthelower16-bitvalueofthecode(asdisplayedbytheEventViewer).

Example: -fullEventCode:ONdirection

Values: FW|BW

Default: FW

Description: Chronologicaldirectioninwhicheventsareretrieved.

Details: Whensetto"FW",eventsareretrievedfromtheoldesttothenewest.Whensetto"BW",eventsareretrievedfromthenewesttotheoldest.Thisparameterisespeciallyusefulwithqueriesthatusethekeywordtoretrievethelastnloggedevents.

Example: -direction:BWstringsSep

Values: anystring

Default: |

Description: Separatorbetweenvaluesofthe"Strings"field.

Details: The"Strings"fieldcontainsanarrayoftextdataassociatedwiththeevent.Thevalueofthisfieldisbuiltbyconcatenatingtheoneaftertheother,usingthevalueofthisparameterasaseparatorbetweentheelements.

Example: -stringsSep:,iCheckpoint

Example: -iCheckpoint:C:\Temp\myCheckpoint.lpcbinaryFormat

Default: HEX

Description: Formatofthe"Data"binaryfield.

Details: The"Data"fieldcontainsbinarydatathatisoftennotsuitabletobetextuallyrepresented.Whenthisparameterissetto"ASC",databytesbelongingtothe0x20-0x7FrangearereturnedasASCIIcharacters,whiledatabytesoutsidetherangearereturnedasperiod(.)characters,asshowninthefollowingexample:

4275636B65743A2030323039363535330D0A72756E646C6C33322E657865

EVTInputFormatExamplesLogonsCreateanXMLreportfilecontaininglogonaccountnamesanddatesfromtheSecurityEventLog:

LogParser"SELECTTimeGeneratedASLogonDate,EXTRACT_TOKEN(Strings,0,'|')ASAccountINTOReport.xmlFROMSecurityWHEREEventIDNOTIN(541;542;543)ANDEventType=8ANDEventCategory=2"

EventDistributionRetrievethedistributionofEventIDvaluesforeachEventSource:

LogParser"SELECTSourceName,EventID,MUL(PROPCOUNT(*)ON(SourceName),100.0)ASPercentFROMSystemGROUPBYSourceName,EventIDORDERBYSourceName,PercentDESC"

EventMessageReportCreateTSVfilescontainingEventMessagesforeachSourceintheApplicationEventLog:

LogParser"SELECTSourceName,MessageINTOmyFile_*.tsvFROM\\MYSERVER1\Application,\\MYSERVER2\Application"

FSInputFormatTheFSinputformatreturnsinformationonfilesanddirectories.

TheFSinputformatenumeratesthefilesanddirectoriesmatchingthesearchpath(s)specifiedinthefrom-entity,muchliketheWindowsshell"dir"command,returninganinputrecordforeachfileanddirectoryintheenumeration.

Seealso:REGInputFormat

FSInputFormatFrom-EntitySyntax<from-entity> ::= <path>[,<path>...]

The<from-entity>specifiedinqueriesusingtheFSinputformatisacomma-separatedlistofpaths,eventuallycontainingwildcards.

Examples:

FROMC:\Windows\*.dll,\\MYSERVER\C$\Windows\*.dll

FROM*.*

FROMC:\*.*,D:\*.*

FROMC:\Windows\Explorer.exe

FSInputFormatFieldsTheinputrecordsgeneratedbytheFSinputformatcontainthefollowingfields:

Path STRING Fullpathofthefileordirectory

Name STRING Nameofthefileordirectory

Size INTEGER Sizeofthefile,inbytes

Attributes STRING Attributesofthefileordirectory

CreationTime TIMESTAMP Dateandtimeatwhichthefileordirectoryhasbeencreated(localorUTCtime,dependingonthevalueoftheuseLocalTimeparameter)

LastAccessTime TIMESTAMP Dateandtimeatwhichthefileordirectoryhasbeenlastaccessed(localorUTCtime,dependingonthevalueoftheuseLocalTimeparameter)

LastWriteTime TIMESTAMP Dateandtimeatwhichthefileordirectoryhasbeenlastmodified(localorUTCtime,dependingonthevalueoftheuseLocalTimeparameter)

FileVersion STRING Versionofthefile

ProductVersion STRING Versionoftheproductthefileisdistributedwith

InternalName STRING Internalnameofthefile

ProductName STRING Nameoftheproductthefileisdistributedwith

CompanyName STRING Nameofthevendorcompanythatproducedthefile

LegalCopyright STRING Copyrightnoticesthatapplytothefile

LegalTrademarks STRING Trademarksandregisteredtrademarksthatapplytothefile

PrivateBuild STRING Privateversioninformationofthefile

SpecialBuild STRING Specialfilebuildnotes

Comments STRING Commentsassociatedwiththefile

FileDescription STRING Descriptionofthefile

OriginalFilename STRING Originalnameofthefile

FSInputFormatParametersTheFSinputformatsupportsthefollowingparameters:

recurse

Default: -1

Description: Maxsubdirectoryrecursionlevel.

Details: 0disablessubdirectoryrecursion;-1enablesunlimitedrecursion.

Example: -recurse:2preserveLastAccTime

Values: ON|OFF

Default: OFF

Description: Preservethelastaccesstimeofvisitedfiles.

Details: Enumeratingfilesanddirectoriescausestheirlastaccesstimetobeupdated.Settingthisparameterto"ON"causestheFSinputformattorestorethelastaccesstimeofthefilesbeingvisited.

Example: -preserveLastAccTime:ONuseLocalTime

Values: ON|OFF

Default: ON

Description: Uselocaltimefortimestampfields.

Details: Whensetto"ON",thevaluesofthe"CreationTime",

"LastAccessTime",and"LastWriteTime"fieldsareexpressedinlocaltime.Whensetto"OFF",thevaluesofthesefieldsareexpressedinUniversalTimeCoordinates(UTC)time.

Example: -useLocalTime:OFF

FSInputFormatExamplesTenLargestFilesPrintthe10largestfilesontheC:drive:

LogParser"SELECTTOP10Path,Name,SizeFROMC:\*.*ORDERBYSizeDESC"-i:FS

MD5HashesofSystemFilesReturntheMD5hashofsystemexecutablefiles:

LogParser"SELECTPath,HASHMD5_FILE(Path)FROMC:\Windows\System32\*.exe"-i:FS-recurse:0

IdenticalFilesFindoutifthereareidenticalcopiesofthesamefileontheC:drive:

LogParser"SELECTHASHMD5_FILE(Path)ASHash,COUNT(*)ASNumberOfCopiesFROMC:\*.*GROUPBYHashHAVINGNumberOfCopies>1"-i:FS

HTTPERRInputFormatTheHTTPERRinputformatparsesHTTPErrorlogfilescreatedbytheHttp.sysdriver.

HTTPErrorlogfilesareserver-widetextlogfilescontaininglogentriesforHttp.sys-initiatederrorresponsestomalformedclientrequestsortovalidrequeststhatareabortedduetoabnormalcircumstances.

DependingontheversionofHttp.sys,HTTPErrorlogfilescanbeloggedintwodifferentformats.EarlierversionsofHttp.syslogHTTPErrorlogentriesasrawlinesconsistingofspace-separatedvalues.ThefollowingexampleshowsaportionofanHTTPErrorlogfilegeneratedbyearlierversionsofHttp.sys:

2002-06-2719:11:28172.30.92.883405172.30.162.21380HTTP/1.0GET/msadc/..%255c../..%255c../..%255c/..%c1%1c../..%c1%1c../..%c1%1c../winnt/system32/cmd.exe?/c+dir400-URL2002-06-2719:11:28172.30.92.883407172.30.162.21380HTTP/1.0GET/scripts/..%c1%1c../winnt/system32/cmd.exe?/c+dir400-URL2002-06-2719:11:28172.30.92.883412172.30.162.21380HTTP/1.0GET/scripts/..%%35%63../winnt/system32/cmd.exe?/c+dir400-URL

LaterversionsofHttp.syslogHTTPErrorlogfilesintheW3CExtendedlogfileformat.Logfilesinthisformatbeginwithsomeinformativeheaders("directives"),themostimportantofwhichisthe"#Fields"directive,describingwhichfieldsareloggedatwhichpositioninalogrow.Afterthedirectives,thelogentriesfollow.Eachlogentryisaspace-separatedlistoffieldvalues.ThefollowingexampleshowsaportionofanHTTPErrorlogfilegeneratedbylaterversionsofHttp.sys:

#Software:MicrosoftHTTPAPI1.0#Version:1.0#Date:2003-08-0803:12:41#Fields:datetimec-ipc-ports-ips-portcs-versioncs-methodcs-urisc-statuss-siteids-reasons-queuename2003-08-0803:12:4110.193.50.9354410.193.50.980HTTP/1.1GET/ISAPI_OOP/ISAPIExtTest.dll?Action=Crash&Action;=Print&Data;=Req17769_0-1Connection_Abandoned_By_AppPoolDefaultAppPool2003-08-0803:12:4110.193.50.9354510.193.50.980HTTP/1.1GET/ISAPI

_OOP/ISAPIExtTest.dll?Action=Crash&Action;=Print&Data;=Req17769_1-1Connection_Abandoned_By_AppPoolDefaultAppPool2003-08-0803:12:4310.193.50.9354610.193.50.980HTTP/1.1GET/ISAPI_OOP/ISAPIExtTest.dll?Action=Crash&Action;=Print&Data;=Req17769_2-1Connection_Abandoned_By_AppPoolDefaultAppPool

HTTPERRInputFormatFrom-EntitySyntax<from-entity> ::= HTTPERR|

<filename>[,<filename>...]

The<from-entity>specifiedinqueriesusingtheHTTERRinputformatiseitherthe"HTTPERR"keywordoracomma-separatedlistofpathsofHTTPErrorlogfiles.Whenthe"HTTPERR"keywordisused,theHTTPERRinputformatreadstheHTTPErrorlogconfigurationfromtheregistryandparsesalltheHTTPErrorlogfilescurrentlyavailableintheHTTPErrorlogfiledirectory.

Filenamescanincludewildcards(e.g."LogFiles\HTTPERR\httperr*.log").

Examples:

FROMLogFiles\HTTPERR\httperr1.log,LogFiles\HTTPERR\httperr2.log

FROM\\MYMACHINE\LogFiles\HTTPERR\httperr*.log

FROMHTTPERR

HTTPERRInputFormatFieldsTheinputrecordsgeneratedbytheHTTPERRinputformatcontainthefollowingfields:

date TIMESTAMP Thedateonwhichtherequestwasserved(UniversalTimeCoordinates(UTC)time)

time TIMESTAMP Thetimeatwhichtherequestwasserved(UniversalTimeCoordinates(UTC)time)

s-computername

STRING Thenameoftheserverthatservedtherequest(thisfieldisloggedbylaterversionsofHttp.sysonly)

c-ip STRING TheIPaddressoftheclientthatmadetherequest

c-port INTEGER Theclientportnumberthatsenttherequest

s-ip STRING TheIPaddressoftheserverthatservedtherequest

s-port INTEGER Theserverportnumberthatreceivedtherequest

cs-version STRING TheHTTPversionoftheclientrequest

cs-method STRING TheHTTPrequestverb

cs-uri STRING TheHTTPrequesturi

cs(User-Agent)

STRING TheclientrequestUser-Agentheader(thisfieldisloggedbylaterversionsofHttp.sysonly)

cs(Cookie) STRING TheclientrequestCookieheader(thisfieldisloggedbylaterversionsofHttp.sysonly)

cs(Referer) STRING TheclientrequestRefererheader(thisfieldisloggedbylaterversionsofHttp.sysonly)

cs-host STRING TheclientrequestHostheader(thisfieldisloggedbylaterversionsofHttp.sysonly)

sc-status INTEGER TheresponseHTTPstatuscode

sc-bytes INTEGER Thenumberofbytesintheresponsesentbytheserver(thisfieldisloggedbylaterversionsofHttp.sysonly)

cs-bytes INTEGER Thenumberofbytesintherequest

sentbytheclient(thisfieldisloggedbylaterversionsofHttp.sysonly)

time-taken INTEGER Thenumberofmillisecondselapsedsincethemomenttheserverreceivedtherequesttothemomenttheserversenttheresponsetotheclient(thisfieldisloggedbylaterversionsofHttp.sysonly)

s-siteid INTEGER TheIISsiteinstancenumberthatservedtherequest

s-reason STRING Informationaboutwhytheerroroccurred

s-queuename STRING ThenameoftheapplicationpoolhostingtheIISworkerprocessthatprocessedtherequest(thisfieldisloggedbylaterversionsofHttp.sysonly)

HTTPERRInputFormatParametersTheHTTPERRinputformatsupportsthefollowingparameters:

iCodepage

Default: 0

Description: Codepageofthelogfile.

Example: -iCodepage:1245minDateMod

Values: date/time(in"yyyy-MM-ddhh:mm:ss"format)

Description: Minimumfilelastmodifieddate,inlocaltimecoordinates.

Details: Whenthisparameterisspecified,theHTTPERRinputformatprocessesonlylogfilesthathavebeenmodifiedafterthespecifieddate.

Example: -minDateMod:"2004-05-2822:05:10"dirTime

Values: ON|OFF

Default: OFF

Description: Usethevalueofthe"#Date"directiveforthe"date"and/or"time"fieldvalueswhenthesefieldsarenotlogged.

Details: Whenalogfileisconfiguredtonotlogthe"date"and/or"time"fields,specifying"ON"forthisparameterscausestheHTTPERRinputformattogenerate"date"and"time"valuesusingthevalueofthelastseen"#Date"directive.

Example: -dirTime:ONiCheckpoint

Details: Thisparameterenablesthe"IncrementalParsing"featurethatallowssequentialexecutionsofthesamequerytoonlyprocessnewlogentriesthathavebeenloggedsincethelastexecution.Formoreinformation,seeParsingInputIncrementally.

HTTPERRInputFormatExamplesErrorsDistributionChartCreateapiechartcontainingthedistributionoferrorsintheHTTPErrorlogs:

LogParser"SELECTsc-status,PROPCOUNT(*)ASPercentageINTOPie.gifFROMHTTPERRGROUPBYsc-statusORDERBYPercentageDESC"-chartType:PieExploded-chartTitle:"ErrorsDistribution"-categories:off

IISInputFormatTheIISinputformatparsesIISlogfilesintheMicrosoftIISLogFileFormat.

TheMicrosoftIISLogFileFormatisatext-based,fixed-fieldformat.Logentriesareloggedonasingleline,consistingofacomma-separatedlistoffieldvalues.

ThefollowingexampleshowsaportionofaMicrosoftIISLogFileFormatlogfile:

192.168.114.201,-,03/20/01,7:55:20,W3SVC2,SERVER,172.21.13.45,4502,163,3223,200,0,GET,/DeptLogo.gif,-,192.168.110.54,-,03/20/01,7:57:20,W3SVC2,SERVER,172.21.13.45,411,221,1967,200,0,GET,/style.css,-,From-EntitySyntaxFieldsParametersExamples

Seealso:IISOutputFormat

IISInputFormatFrom-EntitySyntax<from-entity> ::= <filename>|<SiteID>[,<filename>|<SiteID>...]

The<from-entity>specifiedinqueriesusingtheIISinputformatisacomma-separatedlistof:

PathsofMicrosoftIISLogFileFormatlogfiles;IISVirtualSite"identifiers".

Whena"siteidentifier"isused,theIISinputformatconnectstothespecifiedmachine'smetabase,gathersinformationonthesite'scurrentloggingproperties,andparsesallthelogfilesinthesite'scurrentlogfiledirectory.

Filenamesand"Siteidentifiers"canalsoincludewildcards(e.g."LogFiles\in04*.log","<www.*.com>").

Examples:

FROMLogFiles\in04*log,LogFiles\in03*.log,\\MyServer\LoggingShare\W3SVC2\in04*.logFROM<1>,<2>,<MyExternalSite>,inetsv9.log

FROM<www.net*home.com>,<//MyServer2/W3SVC/www.net*home.com>,<//MyServer2/MSFTPSVC/*>,<*>

IISInputFormatFieldsTheinputrecordsgeneratedbytheIISinputformatcontainthefollowingfields:

UserIP STRING TheIPaddressoftheclientthatmadetherequest

Date TIMESTAMP Thedateonwhichtherequestwasserved(localtime)

Time TIMESTAMP Thetimeatwhichtherequestwasserved(localtime)

ServiceInstance STRING TheIISservicenameandsiteinstancenumberthatservedtherequest

HostName STRING Thenameoftheserverthatservedtherequest

ServerIP STRING TheIPaddressoftheserverthatservedtherequest

TimeTaken INTEGER Thenumberofmillisecondselapsedsincethemomenttheserverreceivedtherequesttothemomenttheserversentthelastresponsechunktotheclient

BytesSent INTEGER Thenumberofbytesintherequestsentbytheclient

BytesReceived INTEGER Thenumberofbytesintheresponsesentbytheserver

StatusCode INTEGER TheresponseHTTPorFTPstatuscode

Win32StatusCode INTEGER TheWindowsstatuscodeassociatedwiththeresponseHTTPorFTPstatuscode

RequestType STRING TheHTTPrequestverborFTPoperation

Target STRING TheHTTPrequesturi-stemorFTPoperationtarget

Parameters STRING TheHTTPrequesturi-query,orNULLiftherequestedURIdidnotincludeauri-query

IISInputFormatParametersTheIISinputformatsupportsthefollowingparameters:

iCodepage

Default: -2

Details: 0isthesystemcodepage;-2specifiesthatthecodepageisautomaticallydeterminedbyinspectingthefilenameand/orthesite's"LogInUTF8"property.

Example: -iCodepage:1245recurse

Default: 0

Example: -recurse:-1minDateMod

Details: Whenthisparameterisspecified,theIISinputformat

processesonlylogfilesthathavebeenmodifiedafterthespecifieddate.

Example: -minDateMod:"2004-05-2822:05:10"locale

Values: 3-characterlocaleID

Default: DEF

Description: IDofthelocaleinwhichthelogfilewasgenerated.

Details: IISversionsearlierthan6.0logthe"Date"and"Time"fieldsusingthecurrentsystemlocaledateandtimeformats.IIS6.0andlaterversionsusetheENUlocaleinstead,regardlessofthesystemlocalesettings.Forthesereasons,whenparsingMicrosoftIISLogFileFormatlogfilesonalocalewhosedateandtimeformatsdonotmatchtheformatsofthelocaleofthecomputerwherethelogfilehasbeencreated,usersneedtospecifytheIDofthesystemlocaleofthecomputerthatcreatedthelogfile.Thespecial"DEF"valuemeansthecurrentsystemlocale.

Example: -locale:JPNiCheckpoint

IISInputFormatExamplesTop20URL'sforaSiteCreateachartcontainingtheTOP20URL'sinthe"www.margiestravel.com"website(assumedtobeloggingintheIISlogformat):

LogParser"SELECTTOP20Target,COUNT(*)ASHitsINTOMyChart.gifFROM<www.margiestravel.com>GROUPBYTargetORDERBYHitsDESC"-chartType:Column3D-groupSize:1024x768

ExportErrorstoSYSLOGSenderrorentriesintheIISlogtoaSYSLOGserver:

LogParser"SELECTTO_TIMESTAMP(Date,Time),CASEStatusCodeWHEN500THEN'emerg'ELSE'err'ENDASMySeverity,HostNameASMyHostname,TargetINTO@myserverFROM<1>WHEREStatusCode>=400"-o:SYSLOG-severity:$MySeverity-hostName:$MyHostnameBytesbyExtensionChartCreateapiechartwiththetotalnumberofbytesgeneratedbyeachextension:

LogParser"SELECTEXTRACT_EXTENSION(Target)ASExtension,MUL(PROPSUM(BytesReceived),100.0)ASBytesINTOPie.gifFROM<1>GROUPBYExtensionORDERBYBytesDESC"-chartType:PieExploded-chartTitle:"Bytesperextension"-categories:off

IISODBCInputFormatTheIISODBCinputformatreturnsdatabaserecordsfromthetablesloggedtobyIISwhenconfiguredtologintheODBCLogFormat.

From-EntitySyntaxFieldsExamples

IISODBCInputFormatFrom-EntitySyntax<from-entity>

::= <SiteID>[,<SiteID>...]|table:<tablename>;username:<username>;password:<password>;dsn:<dsn>

The<from-entity>specifiedinqueriesusingtheIISODBCinputformatiseitheracomma-separatedlistofIISVirtualSite"identifiers",orasinglespecificationoftheODBCparametersneededtoaccessthetable.

"Siteidentifiers"mustbeenclosedwithinanglebrackets(<and>),andcanhaveoneofthefollowingvalues:

ThenumericsiteID(e.g."<1>","<28163489>");Thetextvalueofthe"ServerComment"propertyofthesite(e.g."<MyExternalSite>","<www.margiestravel.com>");Thefully-qualifiedADSImetabasepathtothesite(e.g."<//MYSERVER/W3SVC/1>"),usingeitherthenumericsiteIDorthetextvalueofthe"ServerComment"propertyofthesite.

Whena"siteidentifier"isused,theIISODBCinputformatconnectstothespecifiedmachine'smetabase,gathersinformationonthesite'scurrentODBCloggingproperties,andusesthisinformationtoconnecttothedatabasetable.

"Siteidentifiers"canalsoincludewildcards(e.g."<www.*.com>").

Examples:

FROM<1>,<2>,<MyExternalSite>

FROMtable:MYLOGTABLE;username:IISLOGUSER;password:IISLOGUSERPW;dsn:IISLOGDSN

IISODBCInputFormatFieldsTheinputrecordsgeneratedbytheIISODBCinputformatcontainthefollowingfields:

ClientHost STRING TheIPaddressoftheclientthatmadetherequest

Username STRING Thenameoftheauthenticateduserthatmadetherequest,orNULLiftherequestwasfromananonymoususer

LogTime TIMESTAMP Thedateandtimeatwhichtherequestwasserved(localtime)

Service INTEGER TheIISservicenameandsiteinstancenumberthatservedtherequest

Machine STRING Thenameoftheserverthatservedtherequest

ServerIP STRING TheIPaddressoftheserverthatservedtherequest

ProcessingTime INTEGER Thenumberofmillisecondselapsedsincethemomenttheserverreceivedtherequesttothemomenttheserversentthelast

responsechunktotheclient

BytesRecvd INTEGER Thenumberofbytesintherequestsentbytheclient

ServiceStatus INTEGER TheresponseHTTPorFTPstatuscode

Win32Status INTEGER TheWindowsstatuscodeassociatedwiththeresponseHTTPorFTPstatuscode

Operation STRING TheHTTPrequestverborFTPoperation

Target STRING TheHTTPrequesturi-stemorFTPoperationtarget

Parameters STRING TheHTTPrequesturi-query,orNULLiftherequestedURIdidnotincludeauri-query

IISODBCInputFormatExamplesTop20URL'sforaSiteCreateachartcontainingtheTOP20URL'sinthe"www.margiestravel.com"website(assumedtobeloggingintheODBClogformat):

LogParser"SELECTTOP20Target,COUNT(*)ASHitsINTOMyChart.gifFROM<www.margiestravel.com>GROUPBYTargetORDERBYHitsDESC"-chartType:Column3D-groupSize:1024x768

IISW3CInputFormatTheIISW3CinputformatparsesIISlogfilesintheW3CExtendedLogFileFormat.

IISwebsitesloggingintheW3CExtendedformatcanbeconfiguredtologonlyaspecificsubsetoftheavailablefields.Logfilesinthisformatbeginwithsomeinformativeheaders("directives"),themostimportantofwhichisthe"#Fields"directive,describingwhichfieldsareloggedatwhichpositioninalogrow.Afterthedirectives,thelogentriesfollow.Eachlogentryisaspace-separatedlistoffieldvalues.

IftheloggingconfigurationofanIISvirtualsiteisupdated,thestructureofthefieldsinthefilethatiscurrentlyloggedtomightchangeaccordingtothenewconfiguration.Inthiscase,anew"#Fields"directiveisloggeddescribingthenewfieldsstructure,andtheIISW3Cinputformatkeepstrackofthestructurechangeandparsesthenewlogentriesaccordingly.

ThefollowingexampleshowsaportionofaW3CExtendedLogFileFormatlogfile:

#Software:MicrosoftInternetInformationServices5.0#Version:1.0#Date:2003-11-1800:28:33#Fields:datec-ipcs-uri-stemcs-bytes2003-11-18192.168.1.101/Default.htm1002003-11-18192.168.1.104/hitcount.asp2002003-11-18192.168.1.102/images/address.gif2003-11-18192.168.1.102/cgi-bin/counts.exe400

Seealso:W3CInputFormatW3COutputFormat

IISW3CInputFormatFrom-EntitySyntax<from-entity> ::= <filename>|<SiteID>[,<filename>|<SiteID>...]

The<from-entity>specifiedinqueriesusingtheIISW3Cinputformatisacomma-separatedlistof:

PathsofIISW3CExtendedlogfiles;IISVirtualSite"identifiers".

Whena"siteidentifier"isused,theIISW3Cinputformatconnectstothespecifiedmachine'smetabase,gathersinformationonthesite'scurrentloggingproperties,andparsesallthelogfilesinthesite'scurrentlogfiledirectory.

Filenamesand"Siteidentifiers"canalsoincludewildcards(e.g."LogFiles\ex04*.log","<www.*.com>").

Examples:

FROMLogFiles\ex04*log,LogFiles\ex03*.log,\\MyServer\LoggingShare\W3SVC2\ex04*.logFROM<1>,<2>,<MyExternalSite>,extend9.log

FROM<www.net*home.com>,<//MyServer2/W3SVC/www.net*home.com>,<//MyServer2/MSFTPSVC/*>,<*>

IISW3CInputFormatFieldsTheinputrecordsgeneratedbytheIISW3Cinputformatcontainthefollowingfields:

date TIMESTAMP Thedateonwhichtherequestwasserved(UniversalTimeCoordinates(UTC)time)

time TIMESTAMP Thetimeatwhichtherequestwasserved(UniversalTimeCoordinates(UTC)time)

c-ip STRING TheIPaddressoftheclientthatmadetherequest

cs-username STRING Thenameoftheauthenticateduserthatmadetherequest,orNULLiftherequestwasfromananonymoususer

s-sitename STRING TheIISservicenameandsiteinstancenumberthatservedtherequest

s-computername

STRING Thenameoftheserverthatservedtherequest

s-ip STRING TheIPaddressoftheserverthatservedtherequest

s-port INTEGER Theserverportnumberthatreceivedtherequest

cs-method STRING TheHTTPrequestverborFTPoperation

cs-uri-stem STRING TheHTTPrequesturi-stemorFTPoperationtarget

cs-uri-query STRING TheHTTPrequesturi-query,orNULLiftherequestedURIdidnotincludeauri-query

sc-status INTEGER TheresponseHTTPorFTPstatuscode

sc-substatus INTEGER TheresponseHTTPsub-statuscode(thisfieldisloggedbyIISversion6.0andlateronly)

sc-win32-status

INTEGER TheWindowsstatuscodeassociatedwiththeresponseHTTPorFTPstatuscode

sc-bytes INTEGER Thenumberofbytesintheresponsesentbytheserver

cs-bytes INTEGER Thenumberofbytesintherequest

sentbytheclient

time-taken INTEGER Thenumberofmillisecondselapsedsincethemomenttheserverreceivedtherequesttothemomenttheserversentthelastresponsechunktotheclient

cs-version STRING TheHTTPversionoftheclientrequest

cs-host STRING TheclientrequestHostheader

cs(User-Agent)

STRING TheclientrequestUser-Agentheader

cs(Cookie) STRING TheclientrequestCookieheader

cs(Referer) STRING TheclientrequestRefererheader

s-event STRING Thetypeoflogevent(thisfieldisloggedbyIISversion5.0onlywhenthe"ProcessAccountingLogging"featureisenabled)

s-process-type STRING Thetypeofprocessthattriggeredthelogevent(thisfieldisloggedbyIISversion5.0onlywhenthe"ProcessAccountingLogging"featureisenabled)

s-user-time REAL ThetotalaccumulatedUserModeprocessortime,inpercentage,that

thesiteusedduringthecurrentinterval(thisfieldisloggedbyIISversion5.0onlywhenthe"ProcessAccountingLogging"featureisenabled)

s-kernel-time REAL ThetotalaccumulatedKernelModeprocessortime,inpercentage,thatthesiteusedduringthecurrentinterval(thisfieldisloggedbyIISversion5.0onlywhenthe"ProcessAccountingLogging"featureisenabled)

s-page-faults INTEGER Thetotalnumberofmemoryreferencesthatresultedinmemorypagefaultsduringthecurrentinterval(thisfieldisloggedbyIISversion5.0onlywhenthe"ProcessAccountingLogging"featureisenabled)

s-total-procs INTEGER Thetotalnumberofapplicationscreatedduringthecurrentinterval(thisfieldisloggedbyIISversion5.0onlywhenthe"ProcessAccountingLogging"featureisenabled)

s-active-procs INTEGER Thetotalnumberofapplications

runningwhenthelogeventwastriggered(thisfieldisloggedbyIISversion5.0onlywhenthe"ProcessAccountingLogging"featureisenabled)

s-stopped-procs

INTEGER Thetotalnumberofapplicationsstoppedduetoprocessthrottlingduringthecurrentinterval(thisfieldisloggedbyIISversion5.0onlywhenthe"ProcessAccountingLogging"featureisenabled)

IISW3CInputFormatParametersTheIISW3Cinputformatsupportsthefollowingparameters:

iCodepage

Default: -2

Default: 0

Details: Whenthisparameterisspecified,theIISW3Cinput

formatprocessesonlylogfilesthathavebeenmodifiedafterthespecifieddate.

Example: -minDateMod:"2004-05-2822:05:10"dQuotes

Values: ON|OFF

Default: OFF

Description: Specifiesthatstringvaluesinthelogaredouble-quoted.

Details: LogprocessorsmightgenerateW3Clogswhosestringvaluesareenclosedindouble-quotes.

Example: -dQuotes:ONdirTime

Values: ON|OFF

Default: OFF

Description: Usethevalueofthe"#Date"directiveforthe"date"and/or"time"fieldvalueswhenthesefieldsarenotlogged.

Details: Whenalogfileisconfiguredtonotlogthe"date"and/or"time"fields,specifying"ON"forthisparameterscausestheIISW3Cinputformattogenerate"date"and"time"valuesusingthevalueofthelastseen"#Date"directive.

Example: -dirTime:ONconsolidateLogs

Values: ON|OFF

Default: OFF

Description: Returnentriesfromalltheinputlogfilesorderingbydateandtime.

Details: Whenafrom-entityreferstologfilesfrommultipleIISvirtualsites,specifyingONforthisparametercausestheIISW3Cinputformattoparsealltheinputlogfilesinparallel,returningentriesorderedbythevaluesofthe"date"and"time"fieldsinthelogfiles;theinputrecordsreturnedwillthusappearasifasingleIISW3Clogfilewasbeingparsed.Enablingthisfeatureisequivalenttoexecutingaquerywithan"ORDERBYdate,time"clauseonallthelogfiles.However,theimplementationofthisfeatureleveragesthepre-existingchronologicalorderofentriesineachlogfile,anditdoesnotrequiretheextensivememoryresourcesotherwiserequiredbytheORDERBYqueryclause.

Example: -consolidateLogs:ONiCheckpoint

IISW3CInputFormatExamplesTop20URL'sforaSiteCreateachartcontainingtheTOP20URL'sinthe"www.margiestravel.com"website(assumedtobeloggingintheW3Clogformat):

LogParser"SELECTTOP20cs-uri-stem,COUNT(*)ASHitsINTOMyChart.gifFROM<www.margiestravel.com>GROUPBYcs-uri-stemORDERBYHitsDESC"-chartType:Column3D-groupSize:1024x768

ExportErrorstoSYSLOGSenderrorentriesintheW3ClogtoaSYSLOGserver:

LogParser"SELECTTO_TIMESTAMP(date,time),CASEsc-statusWHEN500THEN'emerg'ELSE'err'ENDASMySeverity,s-computernameASMyHostname,cs-uri-stemINTO@myserverFROM<1>WHEREsc-status>=400"-o:SYSLOG-severity:$MySeverity-hostName:$MyHostnameBytesbyExtensionChartCreateapiechartwiththetotalnumberofbytesgeneratedbyeachextension:

LogParser"SELECTEXTRACT_EXTENSION(cs-uri-stem)ASExtension,MUL(PROPSUM(sc-bytes),100.0)ASBytesINTOPie.gifFROM<1>GROUPBYExtensionORDERBYBytesDESC"-chartType:PieExploded-chartTitle:"Bytesperextension"-categories:off

NCSAInputFormatTheNCSAinputformatparseslogfilesintheNCSACommon,Combined,andExtendedLogFileFormats.

TheNCSALogFileFormatisatext-based,fixed-fieldformat.Logentriesareloggedonasingleline,consistingofaspace-separatedlistoffieldvalues.TherearethreeversionsoftheNCSALogFileFormat:"Common","Combined",and"Extended".Thethreeversionsdifferbythenumberoffieldsthatareloggedforeachrequest.IIScanlogNCSACommonLogFileFormatlogfiles,whileotherwebserverscanbeconfiguredtologwiththeCombinedandExtendedformats.

ThefollowingexampleshowsaportionofanNCSACommonLogFileFormatlogfile:

172.21.13.45-Microsoft\User[08/Apr/2001:17:39:04-0800]"GET/scripts/iisadmin/ism.dll?http/servHTTP/1.0"2003401172.21.201.112--[08/Apr/2001:21:01:19-0800]"GET/style.cssHTTP/1.0"2003401ThefollowingexampleshowsaportionofanNCSACombinedLogFileFormatlogfile:

172.21.13.45-Microsoft\User[08/Apr/2001:17:39:04-0800]"GET/scripts/iisadmin/ism.dll?http/servHTTP/1.0"2003401"http://www.microsoft.com/""Mozilla/4.05[en](WinNT;I)""USERID=CustomerA"172.21.201.112--[08/Apr/2001:21:01:19-0800]"GET/style.cssHTTP/1.0"2001937"http://www.microsoft.com/""Mozilla/4.05[en](WinNT;I)""USERID=CustomerA"

NCSAInputFormatFrom-EntitySyntax<from-entity> ::= <filename>|<SiteID>[,<filename>|<SiteID>...]

The<from-entity>specifiedinqueriesusingtheNCSAinputformatisacomma-separatedlistof:

PathsofNCSALogFileFormatlogfiles;IISVirtualSite"identifiers".

Whena"siteidentifier"isused,theNCSAinputformatconnectstothespecifiedmachine'smetabase,gathersinformationonthesite'scurrentloggingproperties,andparsesallthelogfilesinthesite'scurrentlogfiledirectory.

Filenamesand"Siteidentifiers"canalsoincludewildcards(e.g."LogFiles\nc04*.log","<www.*.com>").

Examples:

FROMLogFiles\nc04*log,LogFiles\nc03*.log,\\MyServer\LoggingShare\W3SVC2\nc04*.logFROM<1>,<2>,<MyExternalSite>,ncsa9.log

FROM<www.net*home.com>,<//MyServer2/W3SVC/www.net*home.com>,<*>

NCSAInputFormatFieldsTheinputrecordsgeneratedbytheNCSAinputformatcontainthefollowingfields:

RemoteHostName STRING TheIPaddressoftheclientthatmadetherequest

RemoteLogName STRING TheidentifierusedtoidentifytheclientmakingtheHTTPrequest,orNULLifnoidentifierisused(alwaysNULLinNCSAlogfilesgeneratedbyIIS)

DateTime TIMESTAMP Thedateandtimeatwhichtherequestwasserved(UniversalTimeCoordinates(UTC)time)

Request STRING TheHTTPrequestline(verb,

URI,andHTTPversion)

StatusCode INTEGER TheresponseHTTPstatuscode

Referer STRING TheclientrequestRefererheader(notloggedinNCSACommonLogFileFormatlogfiles)

User-Agent STRING TheclientrequestUser-Agentheader(notloggedinNCSACommonLogFileFormatlogfiles)

Cookie STRING TheclientrequestCookieheader(notloggedinNCSACommonLogFileFormatlogfiles)

NCSAInputFormatParametersTheNCSAinputformatsupportsthefollowingparameters:

iCodepage

Default: -2

Default: 0

Details: Whenthisparameterisspecified,theNCSAinput

formatprocessesonlylogfilesthathavebeenmodifiedafterthespecifieddate.

Example: -minDateMod:"2004-05-2822:05:10"iCheckpoint

NCSAInputFormatExamplesSliceRequestfieldintocomponentsReturntheverb,URI,andHTTPversionforeachrequest:

LogParser"SELECTEXTRACT_TOKEN(Request,0,'')ASVerb,EXTRACT_TOKEN(Request,1,'')ASURI,EXTRACT_TOKEN(Request,2,'')ASVersionFROMncsa9.log"

Top20URL'sforaSiteCreateachartcontainingtheTOP20URL'sinthe"www.margiestravel.com"website(assumedtobeloggingintheNCSAlogformat):

LogParser"SELECTTOP20EXTRACT_TOKEN(Request,1,'')ASURI,COUNT(*)ASHitsINTOMyChart.gifFROM<www.margiestravel.com>GROUPBYURIORDERBYHitsDESC"-chartType:Column3D-groupSize:1024x768

NETMONInputFormatTheNETMONinputformatparsesnetworkcapturefiles(.capfiles)createdbytheNetMonNetworkMonitorapplication.

TheNETMONinputformatworksintwodifferentmodes,selectablethroughthefModeparameter.

Whenthe"fMode"parameterissetto"TCPIP",theNETMONinputformatreturnsaninputrecordforeachTCP/IPpacketfoundinthecapturefile.Inthiscase,inputrecordscontainfieldsfromtheTCPandIPpacketheaders,togetherwiththepayloadofeachpacket.Forexample,thefollowingcommandreturnsthespecifiedfieldsfromtheTCP/IPpacketsinthecapturefile:

LogParser"SELECTSrcPort,TCPFlags,PayloadBytesFROMMyCapture.cap"-fMode:TCPIPTheoutputofthiscommandwouldlooklikethefollowingsample:

SrcPortTCPFlagsPayloadBytes---------------------------445A11146A01336S080AS01336A01336AP2831336A143180A01336A14311336AP549

Whenthe"fMode"parameterissetto"TCPConn",theNETMONinputformatreturnsaninputrecordforeachTCPconnectionfoundinthecapturefile.Inthiscase,inputrecordscontainfieldscalculatedbyaggregatingalltheTCPpacketsintheconnection,includingthereconstructedpayloadsentbybothendpoints.Forexample,thefollowingcommandreturnsthespecifiedfieldsfromtheTCPconnectionsinthecapturefile:

LogParser"SELECTSrcPort,TimeTaken,SrcPayloadBytes,DstPayloadBytesFROMMyCapture.cap"-fMode:TCPConnTheoutputofthiscommandwouldlooklikethefollowingsample:

SrcPortTimeTakenSrcPayloadBytesDstPayloadBytes-------------------------------------------------

1336150.216000369436731284450.64800031213621286711.0230000012871001.440000001288851.22400000128915120.24000000128366619.38800018863718129113663.102000312636128547883.357000312708129021203.9460003121362

NETMONInputFormatFrom-EntitySyntax<from-entity> ::= <filename>[,<filename>...]

The<from-entity>specifiedinqueriesusingtheNETMONinputformatisacomma-separatedlistofNetMoncapturefiles(.capfiles).

Examples:

FROMMyCapture1.cap

FROMMyCapture1.cap,MyCapture2.cap

NETMONInputFormatFieldsThestructureoftheinputrecordsgeneratedbytheNETMONinputformatdependsonthevaluespecifiedforthefModeparameter.

TCPIPModeWhenthefModeparameterissetto"TCPIP",theNETMONinputformatreturnsaninputrecordforeachTCP/IPpacketfoundinthecapturefile.Inthismode,inputrecordscontainthefollowingfields:

CaptureFilename STRING Thefullpathofthecapturefilecontainingthispacket

Frame INTEGER Theframenumbercontainingthispacket

DateTime TIMESTAMP Dateandtimeatwhichthepacketwassent

FrameBytes INTEGER Totalnumberofbytesintheframe

SrcMAC STRING MACaddressofthesenderofthispacket

SrcIP STRING IPaddressofthesenderofthispacket

SrcPort INTEGER TCPportnumberofthesenderofthispacket

DstMAC STRING MACaddressofthedestinationofthispacket

DstIP STRING IPaddressofthedestinationofthispacket

DstPort INTEGER TCPportnumberofthedestinationofthispacket

IPVersion INTEGER IPversionofthispacket

TTL INTEGER Time-To-LivefieldoftheIPheaderofthispacket

TCPFlags STRING TCPflagsfieldoftheTCPheaderofthispacket

Seq INTEGER TCPsequencenumberofthispacket

Ack INTEGER TCPacknowledgenumberofthispacket

WindowSize INTEGER WindowsizefieldoftheTCPheaderofthispacket

PayloadBytes INTEGER NumberofbytesintheTCPpayloadofthispacket

Payload STRING TCPpayloadofthispacket

Connection INTEGER UniqueidentifieroftheTCPconnectiontowhichthispacketbelongs

TCPConnModeWhenthefModeparameterissetto"TCPConn",theNETMONinputformatreturnsaninputrecordforeachTCPconnectionfoundinthecapturefile.Inthismode,inputrecordscontainthefollowingfields:

CaptureFilename STRING Thefullpathofthecapturefilecontainingthisconnection

StartFrame INTEGER Framenumbercontainingthefirstpacketofthisconnection

EndFrame INTEGER Framenumbercontainingthelastpacketofthisconnection

Frames INTEGER Totalnumberofframescontainingpacketsbelongingtothisconnection

DateTime TIMESTAMP Dateandtimeofatwhichthefirstpacketofthisconnectionwassent

TimeTaken INTEGER Totalnumberofmillisecondselapsedsincethefirstpacketofthisconnectiontothelastpacket

SrcMAC STRING MACaddressoftheinitiatorofthisconnection

SrcIP STRING IPaddressoftheinitiatorofthisconnection

SrcPort INTEGER TCPportnumberoftheinitiatorofthisconnection

SrcPayloadBytes INTEGER TotalnumberofbytesinthereconstructedTCPpayloadsentbytheinitiatorofthisconnection

SrcPayload STRING ReconstructedTCPpayloadsentbytheinitiatorofthisconnection

DstMAC STRING MACaddressofthereceiverofthisconnection

DstIP STRING IPaddressofthereceiverofthisconnection

DstPort INTEGER TCPportnumberofthereceiverofthisconnection

DstPayloadBytes INTEGER TotalnumberofbytesinthereconstructedTCPpayloadsentbythereceiverofthisconnection

DstPayload STRING ReconstructedTCPpayloadsentbythereceiverofthisconnection

NETMONInputFormatParametersTheNETMONinputformatsupportsthefollowingparameters:

Values: TCPIP|TCPConn

Default: TCPIP

Description: Operationmode.

Details: Whenthisparameterissetto"TCPIP",theNETMONinputformatreturnsaninputrecordforeachTCP/IPpacketfoundinthecapturefile.Inthiscase,inputrecordscontainfieldsfromtheTCPandIPpacketheaders,togethereachpacket.Whenthisparameterissetto"TCPConn",theNETMONinputformatreturnsaninputrecordforeachTCPconnectionfoundinthecapturefile.Inthiscase,inputrecordscontainfieldscalculatedbyaggregatingalltheTCPpacketsconnection,includingthereconstructedpayloadsentbybothendpoints.Formoreinformationonthedifferentmodesofoperation,seeFormatFields.

Example: -fMode:TCPConnbinaryFormat

Default: ASC

Description: Formatofbinaryfields.

Details: TCPpacketpayloadsarereturnedasSTRINGvaluesformattedaccordingtothevaluespecifiedforthisparameter.Whenthisparameterissetto"ASC",databytesbelongingtothe0x20-0x7FrangearereturnedasASCIIcharacters,whiledatabytesoutsidetherangearereturnedasperiod(.)characters,asshowninthefollowingexample:

POST/test_system/requestHTTP/1.1..Content-Length:3411..Connection:Keep-Alive..

POST/test_system/requestHTTP/1.1Content-Length:3411Connection:Keep-AliveWhenthisparameterissetto"HEX",alldatabytesarereturnedastwo-digithexadecimalvalues,asshowninthefollowingexample:

504F5354202F63636D5F73797374656D2F7265717565737420485454502F312E310D0A

NETMONInputFormatExamplesNetworkTrafficperSecondDisplaytotalnetworktrafficbytespersecond:

LogParser"SELECTQUANTIZE(DateTime,1)ASSecond,SUM(FrameBytes)INTODATAGRIDFROMMyCapture.capGROUPBYSecond"

REGInputFormatTheREGinputformatreturnsinformationonregistryvalues.

TheREGinputformatenumerateslocalorremoteregistrykeysandvalues,returninganinputrecordforeachregistryvaluefoundintheenumeration.

Seealso:FSInputFormat

REGInputFormatFrom-EntitySyntax<from-entity> ::= <registry_key>[,<registry_key>...]

<registry_key> ::= [\\<computer_name>]\[<root_name>[\<subkey_path>]]

<root_name> ::= HKCR|HKCU|HKLM|HKCC|HKU

The<from-entity>specifiedinqueriesusingtheREGinputformatisacomma-separatedlistofregistrykeys.Validregistrykeysare:

Theregistryroot(e.g."\");Asystemregistryroot(e.g."\HKLM");Anykeybelowasystemregistryroot(e.g."\HKLM\Software\Microsoft").

RegistrykeyscanbeoptionallyprecededbyaremotecomputernameintheUNCnotation.

Examples:

FROM\HKLM,\HKCU

FROM\\SERVER1\HKLM\Software,\\SERVER2\HKLM\Software

REGInputFormatFieldsTheinputrecordsgeneratedbytheREGinputformatcontainthefollowingfields:

ComputerName STRING Nameofthecomputerhostingtheregistrycontainingthisvalue

Path STRING Pathoftheregistrykeycontainingthisvalue

KeyName STRING Nameoftheregistrykeycontainingthisvalue

ValueName STRING Nameoftheregistryvalue

ValueType STRING Nameofthetypeoftheregistryvalue

Value STRING Textrepresentationofthecontentoftheregistryvalue

LastWriteTime TIMESTAMP Dateandtimeatwhichtheregistryvaluehasbeenlastmodified(UniversalTimeCoordinates(UTC)time)

REGInputFormatParametersTheREGinputformatsupportsthefollowingparameters:

recurse

Default: -1

Description: Maxsubkeyrecursionlevel.

Details: 0disablessubkeyrecursion;-1enablesunlimitedrecursion.

Example: -recurse:2multiSZSep

Values: anystring

Default: |

Description: SeparatorbetweenelementsofMULTI_SZregistryvalues.

Details: RegistryvaluesoftheMULTI_SZtypecontainarraysofstrings.Inthesecases,thecontentofthe"Value"fieldisbuiltbyconcatenatingthearrayelementsoneaftertheother,usingthevalueofthisparameterasaseparatorbetweentheelements.

Example: -multiSZSep:,binaryFormat

Default: ASC

Description: FormatofREG_BINARYregistryvalues.

Details: RegistryvaluesoftheREG_BINARYtypecontainbinarydataoftennotsuitabletobetextuallyrepresented.Thisparameterspecifies

howbinarydataisformattedtoaSTRINGwhenreturnedascontentofthe"Value"field.Whenthisparameterissetto"ASC",databytesbelongingtothe0x20-0x7FrangearereturnedasASCIIcharacters,whiledatabytesoutsidetherangearereturnedasperiod(.)characters,asshowninthefollowingexample:

4275636B65743A2030323039363535330D0A72756E646C6C33322E657865

REGInputFormatExamplesUploadRegistrytoSQLTableLoadaportionoftheregistryintoaSQLtable:

LogParser"SELECT*INTOMyTableFROM\HKLM"-i:REG-o:SQL-server:MyServer-database:MyDatabase-driver:"SQLServer"-username:TestSQLUser-password:TestSQLPassword-createTable:ON

RegistryTypeDistributionDisplaythedistributionofregistryvaluetypes:

LogParser"SELECTValueType,COUNT(*)INTODATAGRIDFROM\HKLMGROUPBYValueType"

TEXTLINEInputFormatTheTEXTLINEinputformatreturnslinesfromgenerictextfiles.

TheTEXTLINEinputformatmakesitpossibletoparsetextfilesinanyformatnotsupportednativelybyLogParser,andretrieveentirelinesoftextasasinglefield.ThefieldcanthenbeprocessedbytheSQL-likequerybymakinguseofstringmanipulationfunctions,suchastheEXTRACT_TOKENfunction.

Seealso:TEXTWORDInputFormatTSVInputFormat

TEXTLINEInputFormatFrom-EntitySyntax<from-entity> ::= <filename>[,<filename>...]|

http://<url>|STDIN

The<from-entity>specifiedinqueriesusingtheTEXTLINEinputformatiseither:

Acomma-separatedlistofpathstotextfiles,eventuallyincludingwildcards;TheURLofatextfile;The"STDIN"keyword,whichspecifiesthattheinputdataisavailablefromtheinputstream(commonlyusedwhenpipingcommandexecutions).

Examples:

FROM*.txt,\\MyServer\FileShare\*.tsv

FROMhttp://www.microsoft.adatum.com/example.tsv

typedata.txt|LogParser"SELECT*FROMSTDIN"-i:TEXTLINE

TEXTLINEInputFormatFieldsTheinputrecordsgeneratedbytheTEXTLINEinputformatcontainthefollowingfields:

LogFilename STRING Fullpathofthefilecontainingthisline

Index INTEGER Linenumber

Text STRING Textlinecontent

TEXTLINEInputFormatParametersTheTEXTLINEinputformatsupportsthefollowingparameters:

iCodepage

Default: 0

Description: Codepageofthetextfile.

Default: 0

Example: -recurse:-1splitLongLines

Values: ON|OFF

Default: OFF

Description: Splitlineswhenlongerthanmaximumallowed.

Details: Whenatextlineislongerthan128Kcharacters,theTEXTLINEinputformattruncatesthelineandeitherdiscardstheremainingoftheline(whenthisparameterissetto"OFF"),orprocessestheremainderoftheline

asanewline(whenthisparameterissetto"ON").

Example: -dQuotes:ONiCheckpoint

TEXTLINEInputFormatExamplesHTMLLinksReturnthelinesinanHTMLdocumentthatcontainlinkstootherpages:

LogParser"SELECTTextFROMhttp://www.microsoft.adatum.comWHERETextLIKE'%href%'"-i:TEXTLINE

TEXTWORDInputFormatTheTEXTWORDinputformatreturnswordsfromgenerictextfiles.

TheTEXTWORDinputformatmakesitpossibletoparsetextfilesinanyformatnotsupportednativelybyLogParser,andretrieveeachword(i.e.eachstringdelimitedbywhitespacecharacters)asasinglefield.

Seealso:TEXTLINEInputFormatTSVInputFormat

TEXTWORDInputFormatFrom-EntitySyntax<from-entity> ::= <filename>[,<filename>...]|

http://<url>|STDIN

The<from-entity>specifiedinqueriesusingtheTEXTWORDinputformatiseither:

Acomma-separatedlistofpathstotextfiles,eventuallyincludingwildcards;TheURLofatextfile;The"STDIN"keyword,whichspecifiesthattheinputdataisavailablefromtheinputstream(commonlyusedwhenpipingcommandexecutions).

Examples:

FROM*.txt,\\MyServer\FileShare\*.tsv

FROMhttp://www.microsoft.adatum.com/example.tsv

typedata.txt|LogParser"SELECT*FROMSTDIN"-i:TEXTWORD

TEXTWORDInputFormatFieldsTheinputrecordsgeneratedbytheTEXTWORDinputformatcontainthefollowingfields:

LogFilename STRING Fullpathofthefilecontainingthisword

Index INTEGER Wordnumber

Text STRING Word

TEXTWORDInputFormatParametersTheTEXTWORDinputformatsupportsthefollowingparameters:

iCodepage

Default: 0

Description: Codepageofthetextfile.

Default: 0

Example: -recurse:-1iCheckpoint

Details: Thisparameterenablesthe"IncrementalParsing"featurethatallowssequentialexecutionsofthesamequerytoonlyprocessnewlogentriesthathavebeenloggedsincethelastexecution.Formoreinformation,

seeParsingInputIncrementally.

TEXTWORDInputFormatExamplesWordDistributionReturnthedistributionofwordsinthespecifiedtextfile:

LogParser"SELECTText,COUNT(*)FROMMyFile.txtGROUPBYTextORDERBYCOUNT(*)DESC"-i:TEXTWORD

TSVInputFormatTheTSVinputformatparsestab-separatedandspace-separatedvaluestextfiles.

TSVtextfiles,usuallycalled"tabular"files,aregenerictextfilescontainingvaluesseparatedbyeitherspacesortabs.Thisitalsotheformatoftheoutputofmanycommand-linetools.Forexample,theoutputofthe"netstat"toolisaseriesoflines,eachlineconsistingofvaluesseparatedbyspaces:

ActiveConnections

ProtoLocalAddressForeignAddressStateTCPGABRIEGI-M:epmapGABRIEGI-M.redmond.corp.microsoft.com:0LISTENINGTCPGABRIEGI-M:microsoft-dsGABRIEGI-M.redmond.corp.microsoft.com:0LISTENINGTCPGABRIEGI-M:1025GABRIEGI-M.redmond.corp.microsoft.com:0LISTENINGTCPGABRIEGI-M:1036GABRIEGI-M.redmond.corp.microsoft.com:0LISTENINGTCPGABRIEGI-M:3389GABRIEGI-M.redmond.corp.microsoft.com:0LISTENINGTCPGABRIEGI-M:5000GABRIEGI-M.redmond.corp.microsoft.com:0LISTENINGTCPGABRIEGI-M:42510GABRIEGI-M.redmond.corp.microsoft.com:0LISTENINGTCPGABRIEGI-M:netbios-ssnGABRIEGI-M.redmond.corp.microsoft.com:0LISTENINGUDPGABRIEGI-M:microsoft-ds*:*UDPGABRIEGI-M:isakmp*:*UDPGABRIEGI-M:1026*:*UDPGABRIEGI-M:1027*:*UDPGABRIEGI-M:1028*:*UDPGABRIEGI-M:ntp*:*

Dependingontheapplication,thefirstlineinaTSVfilemightbea"header",containingthelabelsoftherecordfields.ThefollowingexampleshowsaTSVfilebeginningwithaheader:

YearPIDComment2004 2956 Applicationstarted2004 Waitingforinput2004 3104 Applicationstarted2004 1048 ApplicationstartedAmongalltheparameterssupportedbytheTSVinputformat,theiSeparator,nSep,andfixedSepparametersplayacrucialroleinprovidingtheflexibilityoftheTSVinputformatontheformatofthefilesbeingparsed.

TheiSeparatorparameterspecifiesthecharacterusedasaseparatorbetweenthefieldsinthefilesbeingparsed.Sometextfiles,likethepreviousnetstatexample,usesimplespacecharactersasseparatorcharacters,whileothertextfiles,likethesecondexampleabove,usetabcharacters.

ThenSepparameterspecifieshowmanyseparatorcharactersmustappearforthecharacterstosignifyafieldseparator.Inthenetstatexampleabove,fieldsareseparatedbyatleasttwospacecharacters,whileasinglespacecharacterisallowedtoappearinthevalueofafield(asisthecasewiththe"LocalAddress"fieldname).Ontheotherhand,intheprevioustab-separatedexamplefile,fieldsare

UDPGABRIEGI-M:1900*:*UDPGABRIEGI-M:ntp*:*UDPGABRIEGI-M:netbios-ns*:*UDPGABRIEGI-M:netbios-dgm*:*UDPGABRIEGI-M:1900*:*UDPGABRIEGI-M:42508*:*

separatedbyasingletabcharacter.

ThefixedSepparameterspecifieswhetherornotthefieldsintheinputfilesareseparatedbyafixednumberofseparatorcharacters.Inthenetstatexampleabove,fieldsareseparatedbyatleasttwospacecharacters,butthreeormorespacecharactersstillsignifyasinglefieldseparator.Ontheotherhand,intheprevioustab-separatedexamplefile,fieldsareseparatedbyexactlyasingletabcharacter,andthepresenceoftwoconsecutivetabcharacterssignifiesanemptyfield.

Seealso:CSVInputFormatTSVOutputFormat

TSVInputFormatFrom-EntitySyntax<from-entity> ::= <filename>[,<filename>...]|

http://<url>|STDIN

The<from-entity>specifiedinqueriesusingtheTSVinputformatiseither:

Acomma-separatedlistofpathsofTSVfiles,eventuallyincludingwildcards;TheURLofafileintheTSVformat;The"STDIN"keyword,whichspecifiesthattheinputdataisavailablefromtheinputstream(commonlyusedwhenpipingcommandexecutions).

Examples:

FROMLogFiles1\*.txt,LogFiles2\*.txt,\\MyServer\FileShare\*.txt

FROMhttp://www.microsoft.adatum.com/MyTSVFiles/example.tsv

typedata.tsv|LogParser"SELECT*FROMSTDIN"-i:TSV

TSVInputFormatFieldsThestructureoftheinputrecordsgeneratedbytheTSVinputformatisdeterminedatruntime,dependingonthedatabeingparsed,andonthevaluesspecifiedfortheinputformatparameters.

Filename STRING Fullpathofthefilecontainingthisentry

RowNumber INTEGER Lineinthefilecontainingthisentry

ThesetwofieldsarethenfollowedbythefieldsdetectedbytheTSVinputformatinthefile(s)beingparsed.Thenumber,names,anddatatypesofthefieldsaredeterminedbyexamininginitiallytheinputdataaccordingtothevaluesspecifiedfortheinputformatparameters.

ThenumberoffieldsdetectedbytheTSVinputformatduringtheinitialinspectionphasedictateshowtherecordfieldswillbeextractedfromtheinputdataduringthesubsequentparsingstage.Ifalinecontainslessfieldsthanthenumberoffieldsestablished,themissingfieldsarereturnedasNULLvalues.Ontheotherhand,ifalinecontainsmorefieldsthanthenumberoffieldsestablished,theextrafieldsareparsedasiftheywerepartofthevalueofthelastfieldexpectedbytheTSVinputformat.

NumberofFieldsThenumberoffieldsinaninputrecordisdeterminedbytheinputdataandbythevalueofthenFieldsparameter.

Whenthe"nFields"parameterissetto-1,theTSVinputformatdeterminesthenumberoffieldsbyinspectingthefirstlineoftheinput

data,orthefirstlineoftheheaderfilespecifiedwiththe"iHeaderFile"parameter.Asanexample,thefollowingTSVfilecontainsavariablenumberoffields:

NameCityAreaCodeJeffRedmond425SteveSeattle20698101EdwardOlympia360Whenparsedwiththe"nFields"parametersetto-1,thisTSVfilewouldyieldthreefields("Name","City",and"AreaCode").Inthiscase,theextrafourthfieldinthesecondrecordwouldbeparsedaspartofthethird"AreaCode"field,whosevaluewouldthenbe"20698101".

Whenthe"nFields"parameterissettoavaluegreaterthanzero,theTSVinputformatusesthespecifiedvalueasthenumberoffieldsintheinputdata.Consideringagainthepreviousexamplefile,parsingthefilewiththe"nFields"parametersetto4wouldyieldfourfields.

FieldNamesThenamesofthefieldsinaninputrecordisdeterminedbytheinputdataandbythevaluesoftheheaderRowandiHeaderFileparameters.

Whenthe"headerRow"parameterissetto"ON",theTSVinputformatassumesthatthefirstlineinthefilebeingparsedisaheadercontainingthefieldnames.Inthiscase,ifthe"iHeaderFile"parameterisleftunspecified,theTSVinputformatextractsthefieldnamesfromtheheaderline.Ontheotherhand,ifthe"iHeaderFile"parameterissettothepathofaTSVfilecontainingatleastoneline,thentheTSVinputformatassumesthatthespecifiedfilecontainsaheader,parsesitsfirstlineonly,andextractsthefieldnamesfromthisline,ignoringthefirstlineofthefilebeingparsed.

Ifthenumberoffieldnamesextractedislessthanthenumberoffieldsdetected,theadditionalfieldsareautomaticallynamed"FieldN",withNbeingaprogressiveindexindicatingthefieldpositionintheinputrecord.

Consideringthepreviousexamplefile,settingthe"headerRow"

parameterto"ON"wouldcausetheTSVinputformattousethefirstlineofthefileasaheadercontainingthefieldnames.Withthe"nFields"parametersetto-1,theTSVinputformatwoulddetectthreefields,whosenameswouldbe"Name","City",and"AreaCode".Ontheotherhand,withthe"nFields"parametersetto4,theTSVinputformatwoulddetectfourfields,named"Name","City","AreaCode",and"Field4".

Whenthe"headerRow"parameterissetto"OFF",theTSVinputformatassumesthatthefilebeingparseddoesnotcontainaheader,andthatitsfirstlineisthefirstdatarecordinthefile.Inthiscase,ifthe"iHeaderFile"parameterissettothepathofaTSVfilecontainingatleastoneline,thentheTSVinputformatassumesthatthespecifiedfilecontainsaheader,parsesitsfirstlineonly,andextractsthefieldnamesfromthisline.Ontheotherhand,ifthe"iHeaderFile"parameterisleftunspecified,thefieldsareautomaticallynamed"FieldN",withNbeingaprogressivenumberindicatingthefieldpositionintheinputrecord.

Asanexample,thefollowingTSVfiledoesnotcontainaheaderline:

JeffRedmond425SteveSeattle206EdwardOlympia360Whenparsedwiththe"headerRow"parameterto"OFF",theTSVinputformatassumesthatthefirstlineoftheTSVfileisthefirstdatarecordinthefile.Inthiscase,thethreefieldswouldbenamed"Field1","Field2",and"Field3".

FieldTypesThedatatypeofeachfieldextractedfromtheinputdataisdeterminedbyexaminingthefirstndatalines,wherenisthevaluespecifiedforthedtLinesparameter,inthefollowingway:Ifallthenon-emptyfieldvaluesinthefirstnlinesareformattedasdecimalnumbers,thenthefieldisassumedtobeoftheREALtype.Ifallthenon-emptyfieldvaluesinthefirstnlinesareformattedasintegernumbers,thenthefieldisassumedtobeoftheINTEGERtype.Ifallthenon-emptyfieldvaluesinthefirstnlinesareformattedas

timestampsintheformatspecifiedbytheiTsFormatparameter,thenthefieldisassumedtobeoftheTIMESTAMPtype.Otherwise,thefieldisassumedtobeoftheSTRINGtype.

EmptyfieldvaluesarereturnedasNULLvalues.

TSVInputFormatParametersTheTSVinputformatsupportsthefollowingparameters:

iSeparator

Values: asinglecharacter|spaces|space|tab

Default: tab

Description: Separatorcharacterbetweenfields.

Details: The"spaces"valueinstructstheTSVinputformattoconsideranyspacingcharacter(spaceandtab)asaseparatorcharacter.

Example: -iSeparator:spacenSep

Values: numberofseparators(number)

Default: 1

Description: Numberofseparatorcharactersbetweenfieldsinthedatarecords.

Details: Thisparameterspecifieshowmanyseparatorcharactersmustappearforthecharacterstosignifyafieldseparator.Thisparameterisusuallysettoavaluegreaterthanonewhenparsingspace-separatedtextfilesinwhichfieldvaluescancontainasinglespacecharacter.Inthesecases,fieldsareusuallyseparatedbymorethanasinglespacecharacter.Whenthe"fixedSep"parameterissetto"OFF",thevalueofthe"nSep"parameterisassumedtobetheminimumnumberofseparatorcharacterssignifyingafieldseparator.

Example: -nSep:2fixedSep

Values: ON|OFF

Default: OFF

Description: SpecifieswhetherornotthefieldsintheinputTSVfile(s)areseparatedbyafixednumberofseparatorcharacters.

Details: Whenthisparameterissetto"ON",theTSVinputformatassumesthatthenumberofseparatorcharactersbetweenthefieldsintheinputdataequalsexactlythevaluespecifiedforthe"nSep"parameter.Inthiscase,thepresenceofmoreseparatorcharacterssignifiesanemptyvalue,whichisreturnedasaNULLvalue.Whenthisparameterissetto"OFF",theTSVinputformatassumesthatthefieldsintheinputdataareseparatedbyavariablenumberofseparatorcharacters,andthevalueofthe"nSep"parameterisassumedtobetheminimumnumberofseparatorcharacterssignifyingafieldseparator.Inthiscase,additionalseparatorcharactersareignoredandparsedasasinglefieldseparator,thusmakingitimpossibleforavaluetobeinterpretedasaNULLvalue.

Example: -fixedSep:ONheaderRow

Values: ON|OFF

Default: ON

Description: Specifieswhetherornottheinputfile(s)beginwithaheaderline.

Details: Whenthisparameterissetto"ON",theTSVinputformatassumesthateachfilebeingparsedbeginswithaheaderline,containingthelabelsofthefieldsinthefile.Ifthe"iHeaderFile"parameterisleftunspecified,theTSVinputformatwillusethefieldnamesinthefirstfile'sheaderasthenamesoftheinputrecordfields.Ifavalueisspecifiedforthe"iHeaderFile"parameter,theTSVinputformatwillignoretheheaderlineineachfilebeingparsed.Whenthisparameterissetto"OFF",theTSVinputformatassumesthatthefile(s)beingparseddonotcontainaheader,andparsestheirfirstlineasdatarecords.Formoreinformationonheadersandfieldnames,seeTSVInputFormatFields.

Example: -headerRow:OFFiHeaderFile

Values: pathtoaTSVfile

Description: Filecontainingfieldnames.

Details: WhenparsingTSVfilesthatdonotcontainaheaderline,thefieldsoftheinputrecordsproducedbytheTSVinputformatarenamed"Field1","Field2",...Tooverridethisbehaviorandusemeaningfulfieldnames,thisparametercanbesettotothepathofaTSVfilecontainingaheaderline,causingtheTSVinputformattousethefieldnamesinthespecifiedTSVfile'sheaderlineasthenamesoftheinputrecordfields.OnlythefirstlineofthespecifiedTSVfileisparsed,andeventualadditionallinesareignored.Formoreinformationonheadersandfieldnames,seeTSVInputFormatFields.

Example: -iHeaderFile:"C:\MyFolder\header.tsv"nFields

Values: numberoffields(number)

Default: -1

Description: Numberoffieldsinthedatarecords.

Details: Thisparameterspecifiesthenumberoffieldsintheinputdata.Thespecial"-1"valuespecifiesthatthenumberoffieldsistobedeductedbyinspectingthefirstlineofinputdata.Formoreinformationonhowthenumberoffieldsisdetermined,seeTSVInputFormatFields.

Example: -nFields:3dtLines

Default: 100

Description: Numberoflinesexaminedtodeterminefieldtypesatruntime.

Details: ThisparameterspecifiesthenumberofinitiallinesthattheTSVinputformatexaminestodeterminethedatatypeofeachinputfield.Ifthevalueis0,allfieldswillbeassumedtobeoftheSTRINGdatatype.Formoreinformationonhowfielddatatypesaredetermined,seeTSVInputFormatFields.

Example: -dtLines:10nSkipLines

Default: 0

Description: Numberofinitiallinestoskip.

Details: Whenthisparameterissettoavaluegreaterthanzero,theTSVinputformatskipsthefirstnlinesofeachinputfilebeforeparsingitsheaderline,wherenisthevaluespecifiedforthisparameter.

Example: -nSkipLines:5lineFilter

Values: +|-<any_string>[,<any_string>...]

Description: Skiporconsideronlylinesbeginningwiththesestrings.

Details: Whenthevalueofthisparameterbeginswitha"+"character,theTSVinputformatwillonlyparsethoselinesbeginningwithoneofthestringsfollowingthe"+"characterinthespecifiedvalue.Forexample,thevalue"+Data:,Summary:"causestheTSVinputformattoparseonlylinesbeginningwitheither"Data:"or"Summary:".Whenthevalueofthisparameterbeginswitha"-"character,theTSVinputformatwillignorethoselinesbeginningwithoneofthestringsthatfollowthe"-"characterinthespecifiedvalue.Forexample,thevalue"-Comment,Marker"causestheTSVinputformattoignorelinesbeginningwitheither"Comment"or"Marker".

Example: -lineFilter:"-MetaData:,Summary:"iCodepage

Default: 0

Description: CodepageoftheTSVfile.

Example: -iCodepage:1245iTsFormat

Description: Formatoftimestampvaluesintheinputdata.

Details: Thisparameterspecifiesthedateand/ortimeformatusedintheinputdatabeingparsed.ValuesoffieldsmatchingthespecifiedformatarereturnedasvaluesoftheTIMESTAMPdatatype.Formoreinformationondateandtimeformats,seeTimestampFormatSpecifiers.

Example: -iTsFormat:"MMMdd,yyyy"iCheckpoint

Example:

-iCheckpoint:C:\Temp\myCheckpoint.lpc

TSVInputFormatExamplesNetStatoutputParsetheoutputofa'netstat'command:

netstat-a|LogParser"SELECT*FROMSTDIN"-i:TSV-iSeparator:space-nSep:2-fixedSep:OFF-nSkipLines:3

URLSCANInputFormatTheURLSCANinputformatparseslogfilescreatedbytheURLScanIISfilter.

URLScanisanISAPIfilterthatallowsadministratorsofwebserverstorestrictthekindofHTTPrequeststhattheserverwillprocess.ByblockingspecificHTTPrequests,theURLScanfilterpreventspotentiallyharmfulrequestsfromreachingtheserverandcausingdamage.TheURLScanfiltermaintainsalogfiledescribingtheactionstakenwhenHTTPrequestsmatchtheadministrator-specifiedfilters.

LogfilescreatedbytheURLScanfilterlooklikethefollowingexample:

[04-30-2002-17:09:48]----------------InitializingUrlScan.log----------------[04-30-2002-17:09:48]--Filterinitializationtime:[04-30-2002-17:09:48]--[04-30-2002-17:09:48]----------------UrlScan.dllInitializing----------------[04-30-2002-17:09:49]UrlScanwillreturnthefollowingURLforrejectedrequests:"/<Rejected-By-UrlScan>"[04-30-2002-17:09:49]URLswillbenormalizedbeforeanalysis.[04-30-2002-17:09:49]URLnormalizationwillbeverified.[04-30-2002-17:09:49]URLsmustcontainonlyANSIcharacters.[04-30-2002-17:09:49]URLsmustnotcontainanydotexceptforthefileextension.[04-30-2002-17:09:49]URLswillbeloggedupto128Kbytes.[04-30-2002-17:09:49]RequestswithContent-Lengthexceeding30000000willberejected.[04-30-2002-17:09:49]RequestswithURLlengthexceeding260willberejected.[04-30-2002-17:09:49]RequestswithQueryStringlengthexceeding4096willberejected.[04-30-2002-17:09:49]Onlythefollowingverbswillbeallowed(casesensitive):[04-30-2002-17:09:49]'GET'[04-30-2002-17:09:49]Requestscontainingthefollowingcharactersequenceswillberejected:

[04-30-2002-17:09:49]'jj'[04-30-2002-17:10:08]Clientat192.168.1.81:URLcontainssequence'jj',whichisdisallowed.Requestwillberejected.SiteInstance='1',RawURL='/jj/LogLongUrlsTest_2_124_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'[04-30-2002-17:10:08]Clientat192.168.1.81:URLlengthexceededmaximumallowed.Requestwillberejected.SiteInstance='1',RawURL='/jj/LogLongUrlsTest_2_800_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'[04-30-2002-17:10:09]Clientat192.168.1.81:URLlengthexceededmaximumallowed.Requestwillberejected.SiteInstance='1',RawURL='/jj/LogLongUrlsTest_2_1000_aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa'

URLSCANInputFormatFrom-EntitySyntax<from-entity> ::= URLSCAN|

<filename>[,<filename>...]

The<from-entity>specifiedinqueriesusingtheURLSCANinputformatiseitherthe"URLSCAN"keywordoracomma-separatedlistofpathsofURLScanlogfiles.Whenthe"URLSCAN"keywordisused,theURLSCANinputformatextractstheURLScanlogconfigurationparametersfromtheUrlScan.iniconfigurationfileandparsesalltheURLScanlogfilescurrentlyavailableintheURLScanlogfiledirectory.

Filenamescanincludewildcards(e.g."URLSCAN\UrlScan*.log").

Examples:

FROMURLSCAN\UrlScan1.log,URLSCAN\UrlScan2.log

FROM\\MYMACHINE\URLSCAN\UrlScan*.log

FROMURLSCAN

URLSCANInputFormatFieldsTheinputrecordsgeneratedbytheURLSCANinputformatcontainthefollowingfields:

Date TIMESTAMP Thedateandtimeatwhichtherequestwasserved(localtime)

ClientIP STRING TheIPaddressoftheclientthatmadetherequest

Comment STRING ThefilterthatmatchedtherequestandtheactionexecutedbyURLScan

SiteInstance INTEGER TheIISvirtualsiteinstancenumberthatservedtherequest

Url STRING TheHTTPrequesturl

URLSCANInputFormatParametersTheURLSCANinputformatsupportsthefollowingparameters:

iCheckpoint

URLSCANInputFormatExamplesClientssendingsuspiciousrequestsRetrievetheDNSnamesoftheclientsthatsentrequestsmatchingtheURLScanfilters:

LogParser"SELECTDISTINCTREVERSEDNS(ClientIP)FROMURLSCAN"

W3CInputFormatTheW3CinputformatparseslogfilesintheW3CExtendedLogFileFormat.

Examplesoflogfilesinthisformatinclude:

PersonalFirewalllogfilesMicrosoftInternetSecurityandAccelerationServer(ISAServer)logfilesWindowsMediaServiceslogfilesExchangeTrackinglogfilesSimpleMailTransferProtocol(SMTP)logfiles

Logfilesinthisformatbeginwithsomeinformativeheaders("directives"),themostimportantofwhichisthe"#Fields"directive,describingwhichfieldsareloggedatwhichpositioninalogrow.Afterthedirectives,thelogentriesfollow.Eachlogentryisaspace-separatedlistoffieldvalues.

ThefollowingexampleshowsaportionofaPersonalFirewallW3CExtendedLogFileFormatlogfile:

#Verson:1.0#Software:MicrosoftInternetConnectionFirewall#TimeFormat:Local#Fields:datetimeactionprotocolsrc-ipdst-ipsrc-portdst-portsizetcpflagstcpsyntcpacktcpwinicmptypeicmpcodeinfo

2004-09-0307:11:54OPENUDP192.168.1.103192.168.1.108102653--------2004-09-0307:11:54OPENTCP192.168.1.101192.168.1.108300580--------2004-09-0307:11:55OPENTCP192.168.1.103192.168.1.1081104139--------2004-09-0307:11:55OPENTCP192.168.1.104192.168.1.1081103445--------

Note:DifferentlythantheIISW3Cinputformat,theW3Cinputformatdoesnotsupportlogfileswithvaryingnumberand/orpositionoffields.Inotherwords,whenparsingasetofW3Clogfiles,allthelogentriesinallthelogfilesmustbestructuredidenticallyasdeclaredbythefirst"#Fields"directiveencounteredinthefirstlogfile.

Seealso:IISW3CInputFormatW3COutputFormat

W3CInputFormatFrom-EntitySyntax<from-entity> ::= <filename>[,<filename>...]|

http://<url>|STDIN

The<from-entity>specifiedinqueriesusingtheW3Cinputformatiseither:

Acomma-separatedlistofpathsofW3CExtendedlogfiles,eventuallyincludingwildcards;TheURLofafileintheW3CExtendedLogFileFormat;The"STDIN"keyword,whichspecifiesthattheinputdataisavailablefromtheinputstream(commonlyusedwhenpipingcommandexecutions).

Examples:

FROMLogFiles1\pf*.log,LogFiles2\pf*.log,\\MyServer\LoggingShare\pf*.logFROMhttp://www.microsoft.adatum.com/MyLogFiles/example.log

typemylog.log|LogParser"SELECT*FROMSTDIN"-i:W3C

W3CInputFormatFieldsThestructureoftheinputrecordsgeneratedbytheW3Cinputformatisdeterminedatruntime,dependingontheinputdata.

RowNumber INTEGER Lineinthelogfilecontainingthisentry

Followingthesetwofieldsareallthefieldsdeclaredbythefirst"#Fields"directiveencounteredintheinputdata.Thedatatypeofeachfieldextractedfromtheinputdataisdeterminedbyexaminingthefirstnlogentries,wherenisthevaluespecifiedforthedtLinesparameter,inthefollowingway:

Ifallthenon-emptyfieldvaluesinthefirstnlogentriesareformattedasdecimalnumbers,thenthefieldisassumedtobeoftheREALtype.Ifallthenon-emptyfieldvaluesinthefirstnlogentriesareformattedasintegernumbers,thenthefieldisassumedtobeoftheINTEGERtype.Ifallthenon-emptyfieldvaluesinthefirstnlogentriesareformattedastimestampsinthe"yyyy-MM-ddhh:mm:ss"format,thenthefieldisassumedtobeoftheTIMESTAMPtype.Inparticular,ifafieldvalueisformattedasadateinthe"yyyy-MM-dd"format,thenthevalueisreturnedasadate-onlyTIMESTAMPvalue.Ifthefieldvalueisformattedasatimeofdayinthe"hh:mm:ss"format,thenthevalueisreturnedasatime-onlyTIMESTAMPvalue.Otherwise,thefieldisassumedtobeoftheSTRINGtype.

Emptyvalues,representedbyahyphen(-)intheW3CExtendedLogFileFormat,arereturnedasNULLvalues.

Asanexample,thefollowinghelpcommanddisplaystheinputrecordstructuredeterminedbytheW3CinputformatwhenparsingthespecifiedPersonalFirewalllogfile:

C:\>LogParser-h-i:W3Cpfirewall.log

Thestructuredisplayedbythishelpcommandwillbe:

Fields:

LogFilename(S)RowNumber(I)date(T)time(T)action(S)protocol(S)src-ip(S)dst-ip(S)src-port(I)dst-port(I)size(I)tcpflags(S)tcpsyn(I)tcpack(I)tcpwin(I)icmptype(S)icmpcode(S)info(S)

W3CInputFormatParametersTheW3Cinputformatsupportsthefollowingparameters:

iCodepage

Default: 0

Example: -iCodepage:1245dtLines

Default: 10

Description: Numberoflinesexaminedtodeterminefieldtypesatruntime.

Details: ThisparameterspecifiesthenumberofinitialloglinesthattheW3Cinputformatexaminestodeterminethedatatypeoftheinputrecordfields.Ifthevalueiszero,allfieldswillbeassumedtobeoftheSTRINGdatatype.Formoreinformationonhowfielddatatypesaredetermined,seeW3CInputFormatFields.

Example: -dtLines:50dQuotes

Values: ON|OFF

Default: OFF

Description: Specifiesthatstringvaluesinthelogaredouble-quoted.

Details: SomeW3Clogfilesenclosestringvalueswithindouble-quotecharacters(").

Example: -dQuotes:ONseparator

Values: asinglecharacter|space|tab|auto

Default: auto

Description: Separatorcharacterbetweenfields.

Details: DifferentW3Clogfilescanusedifferentseparatorcharactersbetweenthefields;forexample,ExchangeTrackinglogfilesusetabcharacters,whilePersonalFirewalllogfilesusespacecharacters.The"auto"valueinstructstheW3Cinputformattodetectautomaticallytheseparatorcharacterusedintheinputlog(s).

Example: -separator:tab

W3CInputFormatExamplesClientsSendingDroppedPacketsReturnalltheclientsthatsentapacketdroppedbyPersonalFirewall:

LogParser"SELECTDISTINCTsrc-ipFROMpfirewall.logWHEREaction='DROP'"-i:W3C

XMLInputFormatTheXMLinputformatparsesXMLtextfiles.

XMLfiles(alsocalled"XMLdocuments")arehierarchiesofnodes.Nodescanincludeothernodes,andeachnodecanhaveanodevalueandasetofattributes.Forexample,thefollowingXMLnodehasavalue(inthisinstance,"Rome"),andasingleattribute("Population",whosevalueis,inthisexample,"3350000"):

<CITYPopulation='3350000'>Rome</CITY>

XMLdocumentscanbeparsedindifferentways,andtheXMLinputformatoffersthreedistinctusageswhoseapplicabilitydependsonthestructureofthedocuments,andonthestructureoftheinformationthatneedstobeextracted.

Note:TheXMLinputformatrequirestheMicrosoftXMLparser(MSXML)tobeinstalledonthecomputerrunningLogParser.

Seealso:XMLOutputFormat

XMLInputFormatFrom-EntitySyntax<from-entity>

::= <document>[#<XPath>][,<document>[#<XPath>]...]

The<from-entity>specifiedinqueriesusingtheXMLinputformatisacomma-separatedlistofpathsorURLsofXMLfiles.FilenamesorURLscanbeoptionallyfollowedbyanXPaththatspecifieswhichnode(s)inthedocumentaretobeconsideredrootnode(s).

Filenamescanincludewildcards(e.g."LogFiles\doc*.xml").

Examples:

FROMDocument1.xml,http://blogs.msdn.com/MainFeed.aspx

FROMDocument1.xml#/rss/channel/item,http://blogs.msdn.com/MainFeed.aspx#/rss/channel/item

XMLInputFormatFieldsThestructureoftheinputrecordsgeneratedbytheXMLinputformatisdeterminedatruntime,dependingonthedocumentbeingparsed,andonthevaluesspecifiedfortheinputformatparameters.

TheXMLinputformatparsesanXMLdocumentby"visiting"thenodesinthedocument,andtheinputrecordfieldsaretheattributesandvaluesofthenodesthatarevisitedbytheXMLinputformat.

Bydefault,nodesarevisitedfromthedocumentroot,thatis,thesingletop-levelnodeinanXMLdocumentthatcontainsalltheothernodesinthedocument.However,bysupplyinganXPathineitherthefrom-entityorasavalueoftherootXPathparameter,userscanspecifythatthedocumentnodesaretobevisitedstartingfromthenode(s)selectedbytheXPath.

BeforeparsingtheXMLdocumentandreturntheinputrecords,theXMLinputformatinitiallyexaminesthenodesfoundalongthepathsfromtherootnodeorfromthenode(s)selectedbytheuser-suppliedrootXPathtothefirstnleafnodes,wherenisthevalueofthedtNodesparameter.Duringthisphase,theXMLinputformatcreatesarepresentationofthetreestructure("schema"tree)bymergingnodeswiththesamenameandhierarchicalposition.Whencompleted,theschematreecontainsonesingleinstanceofeachnodetype,andeachnodecontainsanattributesetequaltotheunionofalltheattributesfoundinthenodesofthattype.Atthismoment,aninputrecordfieldiscreatedforeachattributebelongingtoanodetypeandforeachnodetypehavingavalue.

Oncetheschematreehasbeendeterminedandtheinputrecordstructurehasbeencreated,theXMLinputformatparsestheXMLdocumentandgeneratesinputrecords,visitingthedocumentnodesandextractingtheirvaluesandattributes.TheXMLinputformatimplementsthreedifferentalgorithmstodecidehowdocumentnodeswillbevisited.ThethreealgorithmsrepresentthreedifferentwaysinwhichtheinformationcontainedinanXMLdocumentcanberetrieved,andthechoiceofanalgorithmdependsonthestructureofthedocumentandonthestructureoftheinformationthatneedstobe

extracted.Sincedifferentalgorithmsvisitdifferentsetsofnodes,thechoiceofanalgorithmaffectswhichfields(i.e.whichnodeattributesandvalues)willbecontainedintheinputrecords.UserscanspecifythealgorithmtousethroughthefMode("fieldmode")parameter,whichcanbesetto"Branch","Tree",or"Node".

BranchFieldModeInthismode,inputrecordscontaintheattributesandvaluesofthenodesthatarevisitedalongallthepossiblepathsfromthedocumentrootorfromthenode(s)selectedbytheuser-suppliedrootXPathtoalltheleafnodes.

Thismodeisappropriatefordocumentsinwhicheachhierarchicallevelconsistsofnodesofthesametype,asdepictedinthefollowingdiagram:

Inthisstructure,therootnodecontainsonlynodesoftype"A",andeach"A"nodecontainsonlynodesoftype"B".Forexample,therootofthefollowingXMLdocumentcontains"Continent"nodesonly;each"Continent"nodecontains"Country"nodesonly,andeach"Country"nodecontains"City"nodesonly:

<?xmlversion="1.0"?><World>

<ContinentContinentName='NorthAmerica'>

<CountryCountryName='USA'><City>Redmond</City><City>SanFrancisco</City></Country>

Thisdocumentcanbethoughtofascontainingsix"entries",theleaf"City"nodes,withtheinformationassociatedwitheachentrybeingcontainedinthenodesthatareencounteredalongapathfromtherootnodetotheleafnode.Inthisexample,theinformationabout"Roma"includestheattributesandvalueofthe"City"node(the"Roma"nodevalueandthe"3350000"valueofits"Population"attribute),theattributesandvalueofitsparent

<CountryCountryName='Canada'><City>Vancouver</City><City>Toronto</City></Country>

</Continent>

<ContinentContinentName='Europe'>

<CountryCountryName='Italia'><CityPopulation='3350000'>Roma</City><City>Milano</City></Country>

</Continent>

</World>

"Country"node(the"Italia"valueofthe"CountryName"attribute),andtheattributesandvalueofitsgrandparent"Continent"node(the"Europe"valueofthe"ContinentName"attribute).

Theschematreeextractedfromthisexampledocumentspecifiesthatthedocumentrootnodecontainsnodesofthe"Continent"type,andthatnodesofthistypehavea"ContinentName"attribute."Continent"nodes,inturn,containnodesofthe"Country"type,witha"CountryName"attribute;finally,"Country"nodescontainnodesofthe"City"type,andnodesofthistypehaveavalue,anda"Population"attribute.Theinputrecordsgeneratedaftertheschematreewouldthuscontainfourfields:"ContinentName","CountryName","City",and"Population".

Whenusingthe"Branch"fieldmode,theXMLinputformatgeneratesaninputrecordforeachpathfromthedocumentrootnodeorfromthenode(s)selectedbytheuser-suppliedrootXPathtoalltheleafnodes.Eachinputrecordcontainstheattributesandvaluesofthenodesencounteredalongthepath:

Record1 Record2

Record3 Record4

Record5

Ifanodedoesnotspecifyanattributethatiscontainedintheattributesupersetofthecorrespondingschematreenode,orifanodedoesnotsupplyavaluewhilethecorrespondingschematreenodespecifiesthatatleastonenodeofthattypehasavalue,thenthecorrespondingfieldvalueissettoNULL.Forexample,parsingtheaboveexampleXMLdocumentin"Branch"fieldmodewouldproducethefollowingoutput:

ContinentNameCountryNameCityPopulation-----------------------------------------------NorthAmericaUSARedmond-NorthAmericaUSASanFrancisco-NorthAmericaCanadaVancouver-NorthAmericaCanadaToronto-EuropeItaliaRoma3350000EuropeItaliaMilano-

TreeFieldModeInthismode,inputrecordscontaintheattributesandvaluesofthenodesfoundinsubtreesthatincludeallnodesofdistincttypes.

Thismodeisappropriatefordocumentsinwhichaspecifichierarchicallevelcontainschildnodesallhavingdifferenttypes,asdepictedinthefollowingdiagram:

Inthisstructure,therootnodecontainsonlynodesoftype"A";each"A"nodehowevercontainsnodesallhavingdifferenttypes(asingle"B"

node,asingle"C"node,andasingle"D"node).Forexample,therootofthefollowingXMLdocumentcontains"Message"nodes;each"Message"nodecontainsasingle"From"node,asingle"To"node,andasingle"Body"node:

<?xmlversion="1.0"?><Messages>

<MessageDate='2004-05-28T12:24:05'><From>Gabriele</From><To>Monica</To><Body>How'sgoing?</Body></Message>

<MessageDate='2004-05-28T13:01:14'><From>Monica</From><To>Gabriele</To><Body>Fine,thanks.</Body></Message>

</Messages>

Thisdocumentcanbethoughtofascontainingtwo"entries",the"Message"subtrees,withtheinformationassociatedwitheachentrybeingcontainedinallthenodesinthesubtreeandinthenodesthatareencounteredalongapathfromtherootnodetothesubtreeroot.Inthisexample,theinformationaboutamessageincludestheattributesandvaluesofallthenodesincludedinthesubtree("From","To",and"Body"nodes),andtheattributesandvaluesofallthenodesencounteredalongthepathfromthedocumentroottothesubtreeroot("Date"attributeofthe"Message"node).

Theschematreeextractedfromthisexampledocumentspecifiesthatthedocumentrootnodecontainsnodesofthe"Message"type,andthatnodesofthistypehavea"Date"attribute."Message"nodes,inturn,containnodesofthe"From","To",and"Body"types,eachtypehavinganodevalue.Theinputrecordsgeneratedaftertheschematreewouldthuscontainfourfields:"Date","From","To",and"Body".

Whenusingthe"Tree"fieldmode,theXMLinputformatgeneratesaninputrecordforeachsubtreethatincludesallnodesofdistincttypes.Eachinputrecordcontainstheattributesandvaluesofthenodesfoundinthesubtrees,togetherwiththeattributesandvaluesofthenodesencounteredalongthepathsfromthedocumentrootnodeorfromthenode(s)selectedbytheuser-suppliedrootXPathtothesubtreerootnodes:

Record1 Record2

Forexample,parsingtheaboveexampleXMLdocumentin"Tree"fieldmodewouldproducethefollowingoutput:

DateFromToBody------------------------------------------------2004-05-2812:24:05GabrieleMonicaHow'sgoing?2004-05-2813:01:14MonicaGabrieleFine,thanks.WhileparsinganXMLdocumentin"Tree"mode,ifasubtreeisfoundcontainingmultipleinstancesofthesamenodetype,thatsubtreeis"replicated"combinatoriallytogenerateallthepossiblesubtreescontainingonesingleinstanceofeachnodetype.ThefollowingdiagramdepictsanXMLdocumentinwhichasubtreecontainsmultipleinstancesofthesamenodetype:

Inthisdiagram,the"A"nodecontainsoneinstanceofthe"B"nodetype,twoinstancesofthe"C"nodetype,andtwoinstancesofthe"D"notetype.Forexample,the"Message"nodeinthefollowingXMLdocumentcontainsasingle"From"node,two"To"nodes,andtwo"Body"nodes:

<?xmlversion="1.0"?><Messages>Thisdocumentcanbethoughtofasa"compact"representationoffour

<MessageDate='2004-05-28T12:24:05'><From>Gabriele</From><To>Jeff</To><To>Steve</To><BodyLanguage='ENU'>Reviewready?</Body><BodyLanguage='ITA'>E'prontalareview?</Body></Message>

</Messages>

differentmessages:From"Gabriele"to"Jeff"inthe"ENU"language;From"Gabriele"to"Jeff"inthe"ITA"language;From"Gabriele"to"Steve"inthe"ENU"language;From"Gabriele"to"Steve"inthe"ITA"language;

Whenusingthe"Tree"fieldmode,these"Message"subtreesarereplicatedcombinatoriallytogenerateallthepossiblesubtreescontainingonesingleinstanceofeachofthe"From","To",and"Body"nodetypes:

Record1 Record2

Record3 Record4

Forexample,parsingtheaboveexampleXMLdocumentin"Tree"fieldmodewouldproducethefollowingoutput:

DateFromToBodyLanguage------------------------------------------------------------2004-05-2812:24:05GabrieleJeffReviewready?ENU2004-05-2812:24:05GabrieleJeffE'prontalareview?ITA2004-05-2812:24:05GabrieleSteveReviewready?ENU2004-05-2812:24:05GabrieleSteveE'prontalareview?ITANodeFieldModeInthismode,inputrecordscontainonlytheattributesandvaluesofthedocumentrootnodeorofthenode(s)selectedbytheuser-suppliedroot

XPath.

Thismodeisappropriateforsituationsinwhichtheinformationtoberetrievedisassociatedwithaspecificnodetypeonly.Forexample,therelevantinformationinthedocumentdepictedbythefollowingdiagrammightbeassociatedwith"B"nodetypesonly:

Whenusingthe"Node"fieldmode,theXMLinputformatgeneratesaninputrecordforeachrootnode,eitherthedocumentrootorthenode(s)selectedbytheuser-suppliedrootXPath.Eachinputrecordcontainstheattributesandvaluesofthatnodeonly:

Record1 Record2

Forexample,parsingtheprevious"Cities"exampleXMLdocumentin"Node"fieldmodespecifying"/World/Continent/Country"astherootXPathwouldproducethefollowingoutput:

CountryName-----------USACanadaItaliaFieldTypesThedatatypeofeachfieldextractedfromtheschematreeisdetermined

inthefollowingway:Ifallthenon-emptyfieldvalues(nodevaluesorattributevalues)encounteredwhileconstructingtheschematreeareformattedasdecimalnumbers,thenthefieldisassumedtobeoftheREALtype.Ifallthenon-emptyfieldvalues(nodevaluesorattributevalues)encounteredwhileconstructingtheschematreeareformattedasintegernumbers,thenthefieldisassumedtobeoftheINTEGERtype.Ifallthenon-emptyfieldvalues(nodevaluesorattributevalues)encounteredwhileconstructingtheschematreeareformattedastimestampsintheformatspecifiedbytheiTsFormatparameter,thenthefieldisassumedtobeoftheTIMESTAMPtype.Otherwise,thefieldisassumedtobeoftheSTRINGtype.

Asanexample,thefollowinghelpcommanddisplaystheinputrecordstructuredeterminedbytheXMLinputformatwhenparsingtheprevious"Cities"exampleXMLdocument:

C:\>LogParser-h-i:XMLCities.xml

Thestructuredisplayedbythishelpcommandwillbe:

Fields:

XMLInputFormatParametersTheXMLinputformatsupportsthefollowingparameters:

rootXPath

Values: XPathquery

Description: XPathqueryofdocumentnode(s)tobeconsideredrootnode(s).

Details: Thenode(s)selectedbythespecifiedXPathreplacethedocumentrootnodeasthestartingnode(s)fromwhichallthedocumentnodesarevisited.

Note:ThisparameterisignoredforXMLdocumentswhosefilenameorURLhasbeenspecifiedtogetherwithanoptionalXPathinthefrom-entity.

Note:TheXPathspecifiedforthisparameteriscase-sensitive.IfanXPathisspecifiedcontainingnon-existingnodeorattributenames,orcontainingnodeorattributenameswiththewrongcapitalization,norootnodeisselectedandanerrorisreturned.

Example: -rootXPath:/World/Continent/CountryfMode

Values: Branch|Tree|Node|Auto

Default: Auto

Description: Algorithmtousewhenvisitingthedocumentnodes.

Details: Forinformationonthe"Branch","Tree",and"Node"visitalgorithmsseeXMLInputFormatFields.The"Auto"valueinstructstheXMLinputformatto

determineautomaticallythebestalgorithmafterinspectingthestructureoftheinputdocument(s).

Example: -fMode:TreeiTsFormat

Default: yyyy-MM-dd?hh:mm:ss

Description: Formatoftimestampvaluesinthedocument.

Details: Thisparameterspecifiesthedateand/ortimeformatusedinthedocumentbeingparsed.ValuesofnodesorattributesmatchingthespecifiedformatarereturnedasvaluesoftheTIMESTAMPdatatype.Formoreinformationondateandtimeformats,seeTimestampFormatSpecifiers.

Example: -iTsFormat:"MMMdd,yyyy"dtNodes

Values: numberofleafnodes(number)

Default: -1

Description: Numberofleafnodestobeexaminedwhendeterminingthedocumentstructure.

Details: Inordertodeterminetheinputdocumentstructure,theXMLinputformatinitiallyexaminesthenodesfoundalongthepathsfromtherootnodeorfromthenode(s)selectedbytheuser-suppliedrootXPathtothefirstnleafnodes,wherenisthevaluespecifiedforthisparameter.Specifying-1causestheXMLinputformattoexamineallthenodesintheinputdocument.

Example: -dtNodes:50

fNames

Values: Compact|XPath

Default: Compact

Description: Fieldnamingschema.

Details: Specifying"Compact"causestheXMLinputformattocreatefieldnamesusingthenamesofthecorrespondingnodesorattributes.Ifafieldnameisnotunique,asequentialnumberisappendedtothenametorenderitunique.Examplefieldnamesinthe"Compact"modeare:

ContinentNameCountryNameCityPopulationSpecifying"XPath"causestheXMLinputformattocreatefieldnamesusingtheXPathqueriesforthecorrespondingnodesorattributes.Examplefieldnamesinthe"XPath"modeare:

/World/Continent/@ContinentName/World/Continent/Country/@CountryName/World/Continent/Country/City/World/Continent/Country/City/@Population

Example: -fNames:XPath

XMLInputFormatExamplesMSDNBLogsChannelTitlesDisplaytitlesofcurrentchannelsonMSDNBLogs:

LogParser"SELECTtitleFROMhttp://blogs.msdn.com/MainFeed.aspx#/rss/channel/item"-i:XML-fMode:Tree

CheckNamesfromMBSAreportDisplaythechecksinanMBSAreport:

LogParser"SELECTNameFROMMYMACHINE.xml#/SecScan/Check"-fMode:Node

OutputFormatsGenericTextFileOutputFormatsNAT:formatsoutputrecordsasreadabletabulatedcolumns.CSV:formatsoutputrecordsascomma-separatedvaluestext.TSV:formatsoutputrecordsastab-separatedorspace-separatedvaluestext.XML:formatsoutputrecordsasXMLdocuments.W3C:formatsoutputrecordsintheW3CExtendedLogFileFormat.TPL:formatsoutputrecordsfollowinguser-definedtemplates.IIS:formatsoutputrecordsintheMicrosoftIISLogFileFormat.

Special-purposeOutputFormatsSQL:uploadsoutputrecordstoatableinaSQLdatabase.SYSLOG:sendsoutputrecordstoaSyslogserver.DATAGRID:displaysoutputrecordsinagraphicaluserinterface.CHART:createsimagefilescontainingcharts.

CHARTOutputFormatTheCHARToutputformatcreatesimagefilescontainingchartsoftheoutputrecordfieldvalues.

WhenusingtheCHARToutputformat,outputrecordfieldsmustbeoftheINTEGERorREALdatatypes,inorderfortheirvaluestobeplottedinachart.ThefirstfieldonlycanoptionallybeoftheSTRINGorTIMESTAMPdatatypes,inwhichcaseitsvaluesareusedasthenamesofthecategoriesontheX-axisofthechart.

ThefollowingexamplecommandcreatesachartplottingthenumberofeventsloggedintheSystemEventLogbyeacheventsource.Thefirstfieldintheoutputrecordsofthisqueryisthenameoftheeventsource,andtheCHARToutputformatwilluseitsvaluestolabelthecategoriesalongtheX-axisofthechart.Thesecondfieldintheoutputrecordsisthenumberofevents,whichwillbeplottedonthechart:

LogParser"SELECTSourceName,COUNT(*)AS[NumberofEvents]INTOEvents.gifFROMSystemGROUPBYSourceNameORDERBY[NumberofEvents]DESC"-o:CHART-chartType:Column3DTheresultingchartwilllooklikethefollowingexample:

Chartscanalsocontainmultipleseriesplottedfromthevaluesofdifferentoutputrecordfields.Forexample,thefollowingcommandcalculatestheaverage,minimum,andmaximumnumberofbytesservedforeachwebpagetype:

LogParser"SELECTTO_UPPERCASE(EXTRACT_EXTENSION(cs-uri-stem))ASPageType,MIN(sc-bytes)ASMinimum,AVG(sc-bytes)ASAverage,MAX(sc-bytes)ASMaximumINTOBytesChart.gifFROM<1>GROUPBYPageTypeORDERBYAverageASC"-o:CHART-chartType:Column3DTheresultingchartwilllooklikethefollowingexample:

TheCHARToutputformatrequirestheMicrosoftOfficeWebComponents,whicharegenerallyinstalledwithMicrosoftOffice2000,MicrosoftOfficeXP,andMicrosoftOffice2003.InordertousetheCHARToutputformat,usersmusthaveavalidlicenseofMicrosoftOfficeforthecomputerexecutingtheLogParserquery.

ConfigurationScriptsInto-EntitySyntaxParametersExamples

CHARTOutputFormatConfigurationScriptsChartscreatedbytheCHARToutputformatcanbecustomizedbyuser-providedscriptsintheJScriptorVBScriptlanguagesthatareexecutedbytheCHARToutputformatpriortogeneratingtheoutputimagefile.

Thesescriptscanrefertotwoglobalobjectswhichexposemethodsandpropertiesthatcanbeusedtomodifyparameterssuchasthechartcolors,thechartfonts,andmanyotherattributes.ThetwoglobalobjectsavailabletoconfigurationscriptsareinstancesofthechartSpaceandchartobjectsoftheMicrosoftOfficeWebComponentsChartSpaceobjectmodel,andtheyarenamed"chartSpace"and"chart",respectively.ForinformationontheOfficeWebComponentsChartSpaceobjectmodel,andonthechartSpaceandchartobjects,visittheMSDNChartSpaceObjectModeldocumentation.

ThefollowingexamplescriptintheJScriptlanguagemanipulatesthechartSpaceandchartobjectstoaddacaptiontothechartandtosetthebackgroundcolortothetransparentcolor:

//AddacaptionchartSpace.HasChartSpaceTitle=true;chartSpace.ChartSpaceTitle.Caption="GeneratedbyLogParser2.2";chartSpace.ChartSpaceTitle.Font.Size=6;chartSpace.ChartSpaceTitle.Position=chartSpace.Constants.chTitlePositionBottom;

//Changethebackgroundcolorchart.PlotArea.Interior.Color=chartSpace.Constants.chColorNone;

ConfigurationscriptsareusedwiththeCHARToutputformatbyspecifyingtheirpathasavaluetotheconfigparameter,asshowninthefollowingexample:

LogParser"SELECTSourceName,COUNT(*)AS[NumberofEvents]INTOEvents.gifFROMSystemGROUPBYSourceNameORDERBY[NumberofEvents]DESC"-o:CHART-chartType:Column3D-config:MyScript.jsTheresultingchartwilllooklikethefollowingexample:

CHARTOutputFormatInto-EntitySyntax<into-entity> ::= <filename>

The<into-entity>specifiedinqueriesusingtheCHARToutputformatisthepathtotheoutputimagefile.

Examples:

INTOMyChart.gif

INTO\\COMPUTER01\Charts\Chart02.jpg

CHARTOutputFormatParametersTheCHARToutputformatsupportsthefollowingparameters:

chartType

Values: nameofcharttype

Default: Line

Description: Charttype.

Details: ThesetofavailablecharttypesdependsontheversionoftheMicrosoftOfficeWebComponentsinstalledonthelocalcomputer.Foralistoftheavailablecharttypes,typethefollowinghelpcommandfromthecommand-lineshell:

LogParser-h-o:CHART

Example: -chartType:Pie3Dcategories

Values: ON|OFF|AUTO

Default: AUTO

Description: Displaycategorylabelsalongthecategoryaxis.

Details: Whenthisparameterissetto"ON",theCHARToutputformatusesthevaluesofthefirstoutputrecordfieldtodisplaycategorylabelsalongthecategoryaxis.Settingthisparameterto"AUTO"causestheCHARToutputformattodisplaycategorylabelsonlywhenthefirstoutputrecordfieldisoftheSTRINGorTIMESTAMPdatatypes.Settingthisparameterto"OFF"preventstheCHARToutputformatfromdisplayingcategorylabels.

Example: -categories:ONmaxCategoryLabels

Values: number

Default: 0

Description: Maximumnumberofcategorylabelsdisplayedalongthecategoryaxis.

Details: Thisparameterisusedtolimitthenumberofcategorylabelsdisplayedalongthecategoryaxis,inordertopreventclutterintheoutputimage.Whenthisparameterissetto"0",theCHARToutputformatcalculatesthemaximumnumberofcategorylabelstodisplayasafunctionofthedimensionsofthetargetimage.Settingthisparameterto"-1"causesthenumberofcategorylabelsdisplayedalongthecategoryaxistobeunlimited.

Example: -maxCategoryLabels:20legend

Values: ON|OFF|AUTO

Default: AUTO

Description: Displayalegenddescribingtheseries.

Details: Whenthisparameterissetto"ON",theCHARToutputformatdisplaysalegendonthechartthatdescribestheseriesbeingplotted.Settingthisparameterto"AUTO"causestheCHARToutputformattodisplayalegendonlywhen2ormoreseriesarebeingplotted.Settingthisparameterto"OFF"preventstheCHARToutputformatfromdisplayingalegend.

Example:

-legend:ONvalues

Values: ON|OFF|AUTO

Default: AUTO

Description: Displayvaluelabels.

Details: Whenthisparameterissetto"ON",theCHARToutputformatdisplaysalabelalongeachvaluebeingplotted,showingitsnumericvalue.Settingthisparameterto"AUTO"causestheCHARToutputformattodisplayvaluelabelsdependingonthetypeofchartselected.Settingthisparameterto"OFF"preventstheCHARToutputformatfromdisplayingvaluelabels.

Example: -values:ONgroupSize

Values: widthxheight

Default: 640x480

Description: Dimensionsofthetargetimage,inpixels.

Details: Thisparameterspecifiesthewidthandheightofthetargetimage,inpixels.

Example: -groupSize:400x260fileType

Values: GIF|JPG|AUTO

Default: AUTO

Description: Formatoftheoutputimagefile.

Details: Whenthisparameterissetto"AUTO",theCHARToutputformatdeterminestheoutputimagefileformatbyinspectingtheextensionofthefilespecifiedfortheinto-entity.

Example: -fileType:JPGconfig

Values: comma-separatedlistoffilepaths

Description: Configurationscriptstouseforchartcustomization.

Details: Thisparameterspecifiesacomma-separatedlistofscriptsintheJScriptorVBScriptlanguagesthatcanbeusedtofurthercustomizethechartgeneratedbytheCHARToutputformat.Formoreinformationonconfigurationscripts,seeCHARTOutputFormatConfigurationScripts.

Example: -config:C:\MyScripts\MyConfig1.js,C:\MyScripts\MyConfig2.vbs

chartTitle

Values: charttitle

Default: Auto

Description: Titleofthechart.

Details: Whenthisparameterissetto"Auto"andtheoutputrecordscontain1seriesonly,theCHARToutputformatusestheseries'fieldnameasthetitleofthechart.

Example: -chartTitle:"BytesPerPage"oTsFormat

Description: Formatoftimestampvaluesinthecategorylabels.

Details: Thisparameterspecifiesthedateand/ortimeformattousewhenformattingvaluesoftheTIMESTAMPdatatypetogeneratecategorylabels.Formoreinformationondateandtimeformats,seeTimestampFormatSpecifiers.

Example: -oTsFormat:"MMMdd,yyyy"view

Values: ON|OFF

Default: OFF

Description: Displaychartimage.

Details: Settingthisparameterto"ON"causestheCHARToutputformattoopenawindowdisplayingthegeneratedoutputimagefile.

Example: -view:ON

CHARTOutputFormatExamplesTop20URL'sCreateachartcontainingtheTOP20URL'sinthe"www.margiestravel.com"website:

LogParser"SELECTTOP20cs-uri-stem,COUNT(*)ASHitsINTOMyChart.gifFROM<www.margiestravel.com>GROUPBYcs-uri-stemORDERBYHitsDESC"-chartType:Column3D-groupSize:1024x768

BytesperPageTypeCreateapiechartwiththedistributionofbytesservedforeachpagetype:

LogParser"SELECTTO_UPPERCASE(EXTRACT_EXTENSION(cs-uri-stem))ASPageType,MUL(PROPSUM(sc-bytes),100.0)ASBytesINTOPie.gifFROM<1>GROUPBYPageTypeORDERBYBytesDESC"-chartType:PieExploded-chartTitle:"Bytesperpagetype"-categories:off©2004MicrosoftCorporation.Allrightsreserved.

CSVOutputFormatTheCSVoutputformatwritesoutputrecordsascomma-separatedvaluestext.

TheoutputoftheCSVoutputformatconsistsofmultiplelinesoftext,onelineforeachoutputrecord.Eachlinecontainsthevaluesoftheoutputrecordfields,separatedbyacomma(,)character.DependingonthevalueoftheoDQuotesparameter,fieldvaluescanbeenclosedwithindouble-quotecharacters(").Ifenabledthroughtheheadersparameter,thefirstlineintheoutputisa"header"thatcontainsthenamesofthefields.

ThefollowingsampleshowstheoutputoftheCSVoutputformatwhenusingthedefaultvaluesforitsparameters:

EventID,SourceName,EventType,TimeGenerated6009,EventLog,4,2004-04-1818:48:046005,EventLog,4,2004-04-1818:48:047024,ServiceControlManager,1,2004-04-1818:48:277035,ServiceControlManager,4,2004-04-1818:48:277035,ServiceControlManager,4,2004-04-1818:48:277036,ServiceControlManager,4,2004-04-1818:48:277036,ServiceControlManager,4,2004-04-1818:48:277035,ServiceControlManager,4,2004-04-1818:48:277036,ServiceControlManager,4,2004-04-1818:48:277035,ServiceControlManager,4,2004-04-1818:48:277036,ServiceControlManager,4,2004-04-1818:48:277035,ServiceControlManager,4,2004-04-1818:48:277036,ServiceControlManager,4,2004-04-1818:48:277035,ServiceControlManager,4,2004-04-1818:48:277036,ServiceControlManager,4,2004-04-1818:48:277036,ServiceControlManager,4,2004-04-1818:48:277035,ServiceControlManager,4,2004-04-1818:48:367036,ServiceControlManager,4,2004-04-1818:51:267036,ServiceControlManager,4,2004-04-1818:51:29

FilescreatedwiththeCSVoutputformataresuitabletobeconsumedbyalargenumberofapplicationsthathandleCSVtextfiles,includingMicrosoftExcelandgenericspreadsheetapplications.

Into-EntitySyntaxParametersExamples

Seealso:TSVOutputFormatCSVInputFormat

6006,EventLog,4,2004-04-1818:51:37

CSVOutputFormatInto-EntitySyntax<into-entity> ::= <filename>|

STDOUT

The<into-entity>specifiedinqueriesusingtheCSVoutputformatiseither:

Afilename;The"STDOUT"keyword,whichspecifiesthattheoutputdataistobewrittentotheoutputstream(theconsoleoutput).

Thedefaultinto-entityforqueriesthatdonotspecifyanINTOclauseis"STDOUT".

TheCSVoutputformatsupportsthemultiplexfeature,whichcanbeenabledbyspecifying'*'wildcardsintheinto-entityfilename.Thisfeatureallowsoutputrecordstobewrittentodifferentfilesdependingonthevaluesoftheirfields.Formoreinformationonthemultiplexfeature,seeMultiplexingOutputRecords.

Examples:

INTOreport.csv

INTO\\COMPUTER01\Reports\report.csv

INTOSTDOUT

INTOReports_*_*\Report*.csv

CSVOutputFormatParametersTheCSVoutputformatsupportsthefollowingparameters:

headers

Values: ON|OFF|AUTO

Default: AUTO

Description: Writeaheaderlinecontainingthefieldnames.

Details: ThisparametercontrolstheCSVheaderlinethatisoutputatthebeginningofeachfile.Thepossiblevaluesforthisparameterare:ON:alwayswritetheheader;OFF:neverwritetheheader;AUTO:writetheheaderonlywhennotappendingtoanexistingfile.

Example: -headers:OFFoDQuotes

Values: ON|OFF|AUTO

Default: AUTO

Description: Enclosefieldvalueswithindouble-quotecharacters(").

Details: ThisparametercontrolswhetherornottheCSVoutputformatshouldenclosefieldvalueswithindouble-quotecharacters(").Thepossiblevaluesforthisparameterare:ON:alwaysenclosefieldvalueswithindouble-quotecharacters;OFF:neverenclosefieldvalueswithindouble-quotecharacters;

AUTO:enclosewithindouble-quotecharactersonlythosefieldvaluesthatcontaincomma(,)characters.

Example: -oDQuotes:ONtabs

Values: ON|OFF

Default: OFF

Description: Writeatabcharacteraftereachcommaseparator.

Details: Settingthisparameterto"ON"causestheCSVoutputformattowriteatabcharacteraftereachcommafieldseparator,inordertoimprovereadabilityoftheCSVoutput.Notethatusingtabsbetweenfieldvaluesmightgenerateoutputthatisnotcompatiblewithcertainspreadsheetapplications.

Example: -tabs:ONoTsFormat

Description: FormatoftimestampvaluesintheoutputCSVdata.

Details: Thisparameterspecifiesthedateand/ortimeformattousewhenformattingvaluesoftheTIMESTAMPdatatype.Formoreinformationondateandtimeformats,seeTimestampFormatSpecifiers.

Example: -oTsFormat:"MMMdd,yyyy"

oCodepage

Default: 0

Description: Codepageoftheoutputtext.

Example: -oCodepage:1245fileMode

Values: 0|1|2

Default: 1

Description: Actiontoperformwhenanoutputfilealreadyexists.

Details: ThisparametercontrolsthebehavioroftheCSVoutputformatwhentheinto-entityspecifiesdirectlyorindirectlythroughthe"multiplex"featurethenameofafilethatalreadyexists.Thepossiblevaluesforthisparameterare:0:existingfilesareappendedwiththeoutput;1:existingfilesareoverwrittenwiththeoutput;2:existingfilesareleftintact,discardingtheoutput.

Example: -fileMode:0

CSVOutputFormatExamplesFileInformationCreateaCSVfilecontaininginformationonthefilescontainedinthespecifieddirectory:

LogParser"SELECTPath,Name,Size,AttributesINTOFiles.csvFROMC:\Test\*.*"-i:FS-o:CSV-recurse:0

SecurityEventsRetrievethe10latesteventsfromtheSecurityeventlogandwritetheirinformationtoaCSVfileforeacheventID:

LogParser"SELECTTOP10EventID,EventTypeName,MessageINTOEvents_*.csvFROMSecurity"-i:EVT-direction:BW-o:CSV

DATAGRIDOutputFormatTheDATAGRIDoutputformatdisplaysoutputrecordsinagraphicaluserinterface.

Outputrecordsaredisplayedinascrollablegridthatallowsuserstobrowsethroughthequeryresults.IndividualoutputrecordscanbeselectedandcopiedtotheclipboardasCSV-formatteddatathatcanbepastedintoanotherapplication.

ThefollowingscreenshotshowstheDATAGRIDwindowdisplayingtheresultsofaquery:

ControlsintheDATAGRIDuserinterfaceallowuserstoresizethewindowandtheindividualoutputrecordcolumns,andtochangethepropertiesofthefontusedtodisplaythedata.

Seealso:NATOutputFormat

DATAGRIDOutputFormatInto-EntitySyntax<into-entity> ::= DATAGRID

QueriesusingtheDATAGRIDoutputformatarenotrequiredtospecifyanINTOclause.IfanINTOclauseisused,its<into-entity>mustbespecifiedas"DATAGRID".

Usingthe"DATAGRID"keywordinthe<into-entity>allowsLogParsertoselecttheDATAGRIDoutputformatautomaticallywhennooutputformatisexplicitlyspecified.

Examples:

INTODATAGRID

DATAGRIDOutputFormatParametersTheDATAGRIDoutputformatsupportsthefollowingparameters:

Values: numberofrows

Default: 10

Description: Rowstoprintbeforepausing.

Details: TheDATAGRIDoutputformatdisplaysoutputrecordsinbatchesmadeupofanumberofrowsequaltothevaluespecifiedforthisparameter.Onceabatchofrowshasbeendisplayed,the"Nextnrows"buttonisenabled,andtheDATAGRIDoutputformatwaitsfortheusertopressthebuttonbeforedisplayingthenextbatchofrows.Specifying"-1"forthisparameterdisablesbatchingaltogether.

Example: -rtp:-1autoScroll

Values: ON|OFF

Default: ON

Description: Automaticallyscrollwindowwhennewrowsareoutput.

Details: Whenthisparameterissetto"ON",theDATAGRIDwindowscrollsdownautomaticallywhenevernewoutputrecordsaredisplayed,inordertopositionthedisplaygridoverthelatestoutputrecords.Settingthisparameterto"OFF"causesthegridpositiontoremainunalteredwhennewoutputrecordsaredisplayed.ThisparameterisalsoaccessiblefromtheViewmenu

intheDATAGRIDwindow.

Example: -autoScroll:OFF

DATAGRIDOutputFormatExamplesUsers'JobTitlesRetrieveusers'jobtitlebreakdownfromActiveDirectory:

LogParser"SELECTtitle,MUL(PROPCOUNT(*),100.0)ASPercentageINTODATAGRIDFROM'LDAP://MyUsername:MyPassword@mydomain/CN=Users,DC=mydomain,DC=com'WHEREtitleISNOTNULLGROUPBYtitleORDERBYPercentageDESC"-objClass:UserRegistryTypeDistributionDisplaythedistributionofregistryvaluetypes:

LogParser"SELECTValueType,COUNT(*)FROM\HKLMGROUPBYValueType"-o:DATAGRID

IISOutputFormatTheIISoutputformatwritesoutputrecordsintheMicrosoftIISLogFileFormat.

ThefollowingexampleshowsasampleoutputfilegeneratedbytheIISoutputformat:

192.168.1.1,-,11/18/2003,0:28:33,-,-,192.168.1.100,15,194,345,304,-,GET,/Default.htm,-,192.168.1.1,-,11/18/2003,0:28:33,-,-,192.168.1.100,0,139,323,304,-,GET,/style.css,-,192.168.1.1,-,11/18/2003,0:28:33,-,-,192.168.1.100,0,139,334,304,-,GET,/images/address.gif,-,192.168.1.1,-,11/18/2003,0:28:33,-,-,192.168.1.100,31,2285,273,200,-,GET,/cgi-bin/counts.exe,test=npa&style;=14,192.168.1.2,-,11/18/2003,0:28:42,-,-,192.168.1.100,1828,666,442,200,-,GET,/home/rules.htm,-,192.168.1.2,-,11/18/2003,0:28:42,-,-,192.168.1.100,47,2018,463,200,-,GET,/home/rules.htm,-,192.168.1.2,-,11/18/2003,0:28:42,-,-,192.168.1.100,62,8903,308,200,-,GET,/home/rules.htm,-,

Seealso:IISInputFormat

IISOutputFormatInto-EntitySyntax<into-entity> ::= <filename>|

STDOUT

The<into-entity>specifiedinqueriesusingtheIISoutputformatiseither:

TheIISoutputformatsupportsthemultiplexfeature,whichcanbeenabledbyspecifying'*'wildcardsintheinto-entityfilename.Thisfeatureallowsoutputrecordstobewrittentodifferentfilesdependingonthevaluesoftheirfields.Formoreinformationonthemultiplexfeature,seeMultiplexingOutputRecords.

Examples:

INTOinetsv1.log

INTO\\COMPUTER01\Logs\in040528.log

INTOSTDOUT

INTOLogs_*_*\in*.log

IISOutputFormatParametersTheIISoutputformatsupportsthefollowingparameters:

Default: 10

Details: WhenwritingtoSTDOUT,theIISoutputformatdisplaysoutputrecordsinbatchesmadeupofanumberofrowsequaltothevaluespecifiedforthisparameter.Onceabatchofrowshasbeendisplayed,theIISoutputformatpromptstheusertopressakeytodisplaythenextbatchofrows.Specifying"-1"forthisparameterdisablesbatchingaltogether.

Example: -rtp:-1oCodepage

Default: 0

Values: 0|1|2

Default: 1

Details: ThisparametercontrolsthebehavioroftheIISoutputformatwhentheinto-entityspecifiesdirectlyorindirectlythroughthe"multiplex"featurethenameofafilethatalreadyexists.Thepossiblevaluesforthisparameterare:0:existingfilesareappendedwiththeoutput;1:existingfilesareoverwrittenwiththeoutput;2:existingfilesareleftintact,discardingtheoutput.

IISOutputFormatExamplesW3CtoIISConversionConvertthespecifiedW3ClogfiletoanIISlogfile:

LogParser"SELECTc-ip,cs-username,TO_DATE(TO_LOCALTIME(TO_TIMESTAMP(date,time))),TO_TIME(TO_LOCALTIME(TO_TIMESTAMP(date,time))),s-sitename,s-computername,s-ip,time-taken,sc-bytes,cs-bytes,sc-status,sc-win32-status,cs-method,cs-uri-stem,cs-uri-queryINTOinetsv1.logFROMextend1.log"-i:IISW3C-o:IIS©2004MicrosoftCorporation.Allrightsreserved.

NATOutputFormatTheNAToutputformatwritesoutputrecordsinareadabletabulatedcolumnformat.

TheprimaryintendeduseoftheNAToutputformatistodisplayoutputrecordstotheconsoleoutput.ThisisthedefaultoutputformatselectedbyLogParserwhenacommanddoesnotexplicitlyspecifyanoutputformatandthequerydoesnotspecifyanINTOclause.

ThefollowingexampleshowsasampleoutputgeneratedbytheNAToutputformat:

TimeGeneratedSourceNameEventID-------------------------------------------------2004-04-1818:48:04EventLog60092004-04-1818:48:04EventLog60052004-04-1818:48:27ServiceControlManager70242004-04-1818:48:27ServiceControlManager70352004-04-1818:48:27ServiceControlManager70352004-04-1818:48:27ServiceControlManager70362004-04-1818:48:27ServiceControlManager70362004-04-1818:48:27ServiceControlManager70352004-04-1818:48:27ServiceControlManager70362004-04-1818:48:27ServiceControlManager7035

Seealso:DATAGRIDOutputFormat

NATOutputFormatInto-EntitySyntax<into-entity> ::= <filename>|

STDOUT

The<into-entity>specifiedinqueriesusingtheNAToutputformatiseither:

TheNAToutputformatsupportsthemultiplexfeature,whichcanbeenabledbyspecifying'*'wildcardsintheinto-entityfilename.Thisfeatureallowsoutputrecordstobewrittentodifferentfilesdependingonthevaluesoftheirfields.Formoreinformationonthemultiplexfeature,seeMultiplexingOutputRecords.

Examples:

INTOreport.txt

INTO\\COMPUTER01\Reports\report.txt

INTOSTDOUT

INTOReports_*_*\Report*.txt

NATOutputFormatParametersTheNAToutputformatsupportsthefollowingparameters:

Default: 10

Details: WhenwritingtoSTDOUT,theNAToutputformatdisplaysoutputrecordsinbatchesmadeupofanumberofrowsequaltothevaluespecifiedforthisparameter.Onceabatchofrowshasbeendisplayed,theNAToutputformatpromptstheusertopressakeytodisplaythenextbatchofrows.Specifying"-1"forthisparameterdisablesbatchingaltogether.

Example: -rtp:-1headers

Values: ON|OFF

Default: ON

Description: Printcolumnheaders.

Details: Thisparameterenablesordisablesthecolumnheadersdisplayedbeforeeachbatchofoutputrows.

Example: -headers:OFFspaceCol

Values: ON|OFF

Default: ON

Description: Spacecolumnsuniformly.

Details: Whenthisparameterissetto"ON",theNAToutputformatpadsvalueswithenoughspacecharacterstocreatecolumnshavingauniformwidthwithineachbatchofoutputrows.Whenthisparameterissetto"OFF",theNAToutputformatdisplaysunalignedvaluesseparatedbyasinglespacecharacter.

Example: -spaceCol:OFFrAlign

Values: ON|OFF

Default: OFF

Description: Aligncolumnstotheright.

Details: Whenthisparameterissetto"ON",theNAToutputformatalignsvaluestotherightsideofeachcolumn.Whenthisparameterissetto"OFF",valuesarealignedtotheleftsideofeachcolumn.

Example: -rAlign:ONcolSep

Values: anystring

Default: singlespacecharacter

Description: Columnseparator.

Details: Thisparameterspecifiestheseparatortobeusedbetweenthecolumns.

Example: -colSep:","

direct

Values: ON|OFF

Default: OFF

Description: Enable"directmode".

Details: When"directmode"isenabled,theNAToutputformatdisplaysoutputrecordsastheyaremadeavailable,disablingtheinternalbufferingmechanismusedforcolumnspacingandoutputrowbatching.In"directmode"columnsarenotuniformlyspaced,headersareprintedonlyatthebeginningoftheoutput,andoutputrecordsaredisplayedwithoutinterruption.

Example: -direct:ONoCodepage

Default: 0

Values: 0|1|2

Default: 1

Details: ThisparametercontrolsthebehavioroftheNAToutputformatwhentheinto-entityspecifiesdirectlyorindirectlythroughthe"multiplex"featurethenameofafilethatalreadyexists.

Thepossiblevaluesforthisparameterare:0:existingfilesareappendedwiththeoutput;1:existingfilesareoverwrittenwiththeoutput;2:existingfilesareleftintact,discardingtheoutput.

NATOutputFormatExamplesTenLargestFilesPrintthe10largestfilesontheC:drive:

LogParser"SELECTTOP10*FROMC:\*.*ORDERBYSizeDESC"-i:FS

SQLOutputFormatTheSQLoutputformatuploadsoutputrecordstoatableinaSQLdatabase.

ThisoutputformatcanuploadrecordstoatableinanyODBC-compliantdatabase,includingMicrosoftSQLServerandMicrosoftAccessdatabases.

Whenthetargettabledoesnotalreadyexistinthespecifieddatabase,theSQLoutputformatcreatesatablewithasmanycolumnsasthenumberoffieldsintheSELECTclauseofthequery.Inthiscase,theSQLtypeofeachcolumnisdeterminedbythedatatypeofthecorrespondingoutputrecordfield,asdescribedinColumnTypeMappings.

Ifthetargettablealreadyexists,thenumberofcolumnsinthetablemustmatchexactlythenumberoffieldsintheSELECTclauseofthequery,andtheSQLtypeofeachcolumnmustbecompatiblewiththedatatypeoftheoutputrecordfieldinthesameposition,asdescribedinColumnTypeMappings.

ColumnTypeMappingsInto-EntitySyntaxParametersExamples

SQLOutputFormatColumnTypeMappingsThefollowingtableshowsthemappingsbetweenthedatatypesofthequeryoutputrecordfieldsandtheSQLtypesofthecolumnsinthetargettable.

Thecolumnlabeled"NewTable"showstheSQLtypesdeclaredforthetablecolumnswhentheSQLoutputformatcreatesthetable.Thecolumnlabeled"ExistingTable"showstheSQLtypesthatarecompatiblewiththecorrespondingLogParserdatatypewhentheSQLoutputformatuploadsrecordstoanexistingtable.

LogParserDataType NewTable ExistingTable

INTEGER int int,bigint,smallint,tinyint,bit1

REAL real real,decimal,float

STRING varchar(n2) varchar(n),nvarchar(n),charTIMESTAMP datetime datetime,smalldatetime,date,timeNULL varchar anytype

Notes:(1):whenuploadingtoafieldofthebittype,thetargetvalueissettotruewhentheINTEGERvalueisdifferentthanzero,andtofalsewhenthevalueisNULLorzero.

(2):themaximumlengthofnewfieldsofthevarchartypecanbecontrolledthroughthemaxStrFieldLenparameter.

SQLOutputFormatInto-EntitySyntax<into-entity> ::= <table_name>

The<into-entity>specifiedinqueriesusingtheSQLoutputformatisthenameofthetablewheretheresultsaretobeuploadedto.

Ifthespecifiedtabledoesnotalreadyexist,theSQLoutputformatcreatesatablewithasmanycolumnsasthenumberoffieldsintheSELECTclauseofthequery.Inthiscase,theSQLtypeofeachcolumnisdeterminedbythedatatypeofthecorrespondingoutputrecordfield,asdescribedinColumnTypeMappings.Ifthespecifiedtablealreadyexists,thenumberofcolumnsinthetablemustmatchexactlythenumberoffieldsintheSELECTclauseofthequery,andtheSQLtypeofeachcolumnmustbecompatiblewiththedatatypeoftheoutputrecordfieldinthesameposition,asdescribedinColumnTypeMappings.

Examples:

INTOReportTable

SQLOutputFormatParametersTheSQLoutputformatsupportsthefollowingparameters:

server

Values: servername

Default: .

Description: Nameofthedatabaseserver.

Details: Settingavalueforthe"oConnString"parametercausesthisparametertobeignored.

Example: -server:SQLREPORTSdatabase

Values: databasename

Description: Nameofthetargetdatabase.

Example: -database:LogParserLogsdriver

Values: ODBCdrivername

Default: SQLServer

Description: NameoftheODBCdrivertouse.

Example: -driver:"MicrosoftAccessDriver(*.mdb)"

Values: DSNname

Description: NameoftheDSNtouse.

Details: ThisparametercanbeusedtospecifyaDataSourceNamethatcontainsinformationabouttheconnectiontothetargetdatabase.Settingavalueforthe"oConnString"parametercausesthisparametertobeignored.

Example: -dsn:"MyDSN"username

Values: SQLusername

Description: Databaseusername.

Details: Whenthisparameterisnotspecified,theSQLoutputformatusesthecurrentuser'scredentialsthroughWindowsIntegratedAuthentication.Settingavalueforthe"oConnString"parametercausesthisparametertobeignored.

Example: -username:MyDBUserpassword

Values: SQLpassword

Description: Databaseuserpassword.

Example: -password:MyPasswordoConnString

Values: connectionstring

Description: ODBCconnectionstringcontainingtheparametersfortheconnectiontothedatabase.

Details: SettingavalueforthisparametercausestheSQLoutputformattoignoreanyvaluesetforthe"server","database","driver","dsn","username",and"password"parameters.TheSQLoutputformatdoesnotenforceanysyntaxontheconnectionstring.ThevaluespecifiedforthisparameterishandeddirectlytotheODBCsubsystemwheninitiatingtheconnectiontothedatabase.

Note:Forsecurityreasons,valuesspecifiedforthisparameterthatcontainausernameand/orapasswordarenotpersistedwhenusingtheLogParsercommand-lineDefaultsOverrideMode.

Example: -oConnString:"Driver={SQLServer};Server=MyServer;db=pubs;uid=sa;pwd=MyPassword"

createTable

Values: ON|OFF

Default: OFF

Description: Createanewtablewhenthetablespecifiedintheinto-entitydoesnotexist.

Details: Whenthisparameterissetto"ON"andthetargettabledoesnotalreadyexistinthespecifieddatabase,theSQLoutputformatcreatesatablewithasmanycolumnsasthenumberoffieldsintheSELECTclauseofthequery.Inthiscase,theSQLtypeofeachcolumnisdeterminedbythedatatypeofthecorrespondingoutputrecordfield,asdescribedinColumnTypeMappings.Whenthisparameterissetto"OFF"andthetargettabledoesnotalreadyexistinthespecifieddatabase,theSQLoutputformatgeneratesanerror,causingthecurrentlyexecutingquerytoabort.

Example: -createTable:ONclearTable

Values: ON|OFF

Default: OFF

Description: Clearexistingtablebeforeinsertingnewrows.

Details: Settingthisparameterto"ON"causestheSQLoutputformattodeleteexistingrowsinthetargettablebeforeinsertingthequeryoutputrecords.

Example: -clearTable:ONfixColNames

Values: ON|OFF

Default: ON

Description: Automaticallyremoveinvalidcharactersfromcolumnnameswhencreatingthetargettable.

Details: Whenthe"createTable"parameterissetto"ON"andthe

targettabledoesnotalreadyexistinthespecifieddatabase,theSQLoutputformatcreatesthetablenamingitscolumnswiththenamesofthequeryoutputrecordfields.Whenthisparameterissetto"ON",theSQLoutputformatprocessesthefieldnamesandremovesorsubstitutesthosecharactersthatareconsideredillegalbymostdatabases,includingspacecharacters,parenthesyscharacters,anddash(-)characters.

Example: -fixColNames:OFFmaxStrFieldLen

Values: numberofcharacters

Default: 255

Description: Maximumnumberofcharactersdeclaredforstringcolumnswhencreatingatable.

Details: Whenthe"createTable"parameterissetto"ON"andthetargettabledoesnotalreadyexistinthespecifieddatabase,theSQLoutputformatcreatesthetabledeterminingtheSQLtypeofeachcolumnfromthedatatypeofthecorrespondingoutputrecordfield,asdescribedinColumnTypeMappings.ColumnscorrespondingtooutputrecordfieldsoftheSTRINGdatatypearedeclaredasSQLstringshavingamaximumlengthequaltothevaluespecifiedforthisparameter.

Example: -maxStrFieldLen:511transactionRowCount

Default: 0

Description: NumberofrowsenclosedinaSQLtransaction.

Details: Whenthisparameterissetto"0",theSQLoutputformatworksin"autocommit"mode,whereeachsingleoutputrecorduploadedtothetargettableisautomaticallycommitted.Whenthisparameterissetto"-1",theSQLoutputformatinitiatesaSQLtransactionwhenuploadingthefirstoutputrecord,andcommitsorrollbacksthetransactionafteruploadingthelastrecordorwhenanerrorcausesthequeryexecutiontoabort.SettingthisparametertoanyothervaluecausestheSQLoutputformattocreatemultipleSQLtransactions,eachcontaininganumberofrecordsequaltothespecifiedvalue.

Example: -transactionRowCount:200ignoreMinWarns

Values: ON|OFF

Default: ON

Description: Ignoreminorwarnings.

Details: Whenthisparameterissetto"ON",theSQLoutputformatignoresminorwarningsthatmightoccurwhileuploadingrecordstothetargettable,includingdatatruncationwarningsandinvalidescapecharactererrors.Whenthisparameterissetto"OFF",allminorwarningsarereportedaswarningswhenthequeryexecutioniscomplete.

Example: -ignoreMinWarns:OFFignoreIdCols

Values: ON|OFF

Default: OFF

Description: Ignore"identity"columnsinthetargettable.

Details: Whenthisparameterissetto"OFF"andthetargettablespecifiedintheinto-entityalreadyexists,theSQLoutputformatexpectsa1-to-1matchbetweenthecolumnsinthetargettableandthefieldsinthequeryoutputrecords,regardlessofwhetherornotanycolumninthetargettableisan"identity"column.Inthiscase,thevaluesoftheoutputrecordfieldswillbeuploadedtoallthecolumnsinthetable,includingeventual"identity"columns.Whenthisparameterissetto"ON"andthetargettablespecifiedintheinto-entityalreadyexists,theSQLoutputformatignores"identity"columnsinthetargettable,checkingfora1-to-1matchonlybetweenthenon-identitycolumnsandthefieldsinthequeryoutputrecords,anduploadingoutputrecordfieldvaluestonon-identitycolumnsonly.

Example: -ignoreIdCols:ON

SQLOutputFormatExamplesUploadRegistryValuestoaSQLtableUploadaportionoftheregistryintoanewly-createdSQLtable:

LogParser"SELECTPath,KeyName,ValuleNameINTOMyTableFROM\HKLM"-i:REG-o:SQL-server:MyServer-database:MyDatabase-driver:"SQLServer"-username:TestSQLUser-password:TestSQLPassword-createTable:ONUploadIISW3ClogfilestoanAccessdatabaseUploadselectedfieldsofanIISW3ClogfileintoanexistingtableinMicrosoftAccess:

LogParser"SELECTTO_TIMESTAMP(date,time),c-ip,cs-uri-stem,sc-statusINTOMyTableFROMextend1.log"-i:IISW3C-o:SQL-oConnString:"Driver={MicrosoftAccessDriver(*.mdb)};Dbq=C:\MyDB\MyDB.mdb;Uid=MyUsername;Pwd=MyPassword"©2004MicrosoftCorporation.Allrightsreserved.

SYSLOGOutputFormatTheSYSLOGoutputformatcanbeusedtosendmessagestoaSyslogserver,tocreatetextfilescontainingSyslogmessages,andtosendSyslogmessagestousers.

TheSYSLOGoutputformatgeneratesmessagesformattedaccordingtotheSyslogspecificationsdescribedinRFC3164.Syslogmessagesconsistofsixparts,andtheSYSLOGoutputformatprovidesparametersthatallowuserstoassignconstantsoroutputrecordfieldstothedifferentpartsofamessage.

ThefollowingexampleshowsSyslogmessagescontaininginformationgatheredfromtheSystemeventlog:

<46>Apr1818:48:04MYSERVER-MLogParser:EventLog:TheEventlogservicewasstarted.<30>Apr1818:48:27MYSERVER-MLogParser:ServiceControlManager:TheTelephonyserviceenteredtherunningstate.<46>Apr1818:51:37MYSERVER-MLogParser:EventLog:TheEventlogservicewasstopped.<134>Apr1819:20:23MYSERVER-MLogParser:AtiHotKeyPoller:Theservicewasstarted.<46>Apr1819:20:07MYSERVER-MLogParser:EventLog:TheEventlogservicewasstarted.<30>Apr1819:20:47MYSERVER-MLogParser:ServiceControlManager:TheTelephonyserviceenteredtherunningstate.<46>Apr1819:33:17MYSERVER-MLogParser:EventLog:TheEventlogservicewasstopped.<134>Apr1907:01:57MYSERVER-MLogParser:AtiHotKeyPoller:Theservicewasstarted.<46>Apr1907:01:41MYSERVER-MLogParser:EventLog:TheEventlogservicewasstarted.<30>Apr1907:02:07MYSERVER-MLogParser:ServiceControlManager:TheTelephonyserviceenteredtherunningstate.

TheSYSLOGoutputformatcanbeoptionallyconfiguredwithaSyslogserverconfigurationfile,whichdescribestherulesusedtoforwardmessagestofiles,Syslogservers,orusers.

MessageStructureConfigurationFilesInto-EntitySyntaxParametersExamples

SYSLOGOutputFormatMessageStructureTheSYSLOGoutputformatgeneratesmessagesformattedaccordingtotheSyslogspecificationsdescribedinRFC3164.Syslogmessagesconsistofsixparts,andtheSYSLOGoutputformatprovidesparametersthatallowuserstoassignconstantsoroutputrecordfieldstothedifferentpartsofamessage.

AsampleSyslogmessageisformattedasfollows:

<14>Nov1116:05:33MYSERVER-MLogParser:Theservicewasstarted.

Thismessageconsistsofthefollowingparts:

PRI: <14>

ThePRIpartisboundwithanglebracketsandcontainsadecimalPriorityvalue,whichinturnisbuiltasfollows:

Thefirst7bitscontainthefacilityvalue,describingtheoriginofthemessage;Thelast3bitscontaintheseverityvalue,describingtheimportanceofthemessage.

HEADER: Nov1116:05:33MYSERVER-M

TheHEADERpartconsistsofthefollowingtwoelements:

Atimestampvalue,indicatingthelocaltimeatwhichthemessagewasgenerated;Ahostnamevalue,indicatingthehostonwhichthemessageoriginated.

MSG: LogParser:Theservicewasstarted.

TheMSGpartconsistsofthefollowingtwoelements:

Atagvalue,indicatingthenameoftheprogramorprocessthatgeneratedthemessage,followedbyacoloncharacter(":");Acontentvalue,containingthedetailsofthemessage.

FacilityThefacilityvalueisrepresentedbytheupper7bitsofthepriorityvalueinthePRIpartofthemessage,anditdescribestheapplicationoroperatingsystemcomponentthatoriginatedthemessage.Foradetailedlistofthenumericvaluesdesignatedforwell-knownoperatingsystemcomponents,refertoRFC3164.Thefollowingtableshowsthenamesassignedtothemostcommonfacilityvalues:

NumericalValue FacilityName

0 kern

1 user

2 mail

3 daemon

4 auth

5 mark

7 news

8 uucp

9 cron

10 auth2

11 ftp

12 ntp

13 logaudit

14 logalert

15 clock

16 local0

17 local1

18 local2

19 local3

20 local4

21 local5

22 local6

23 local7

Inthepreviousexamplemessage,thepriorityvalue"14"indicatesafacilityvalueof1("user").

facilityparameteroftheSYSLOGoutputformatallowsuserstocontrolthevalueofthefacilityfieldintheoutputmessages.Thisparametercanbesettoanyofthefollowingvalues:Anumericvalue,suchas"1"or"23";Thenameofafacilityvalue,suchas"user"or"local7";

Thenameorthe1-basedindexofanoutputrecordfieldprependedwithadollarcharacter("$"),suchas"$MyFacility"or"$2".ThespecifiedoutputrecordfieldmustbeofeithertheINTEGERdatatype-inwhichcaseitsvaluesareassumedtobenumericalfacilityvalues,oroftheSTRINGdatatype-inwhichcaseitsvaluesareassumedtobefacilitynamesamongthosedescribedintheprevioustable.Whenanoutputrecordfieldvaluedoesnotcontainarecognizedfacilitynameoritcontainsafacilityvaluegreaterthan23,theSYSLOGoutputformatusesadefaultfacilityvalueof1("user").

ThefollowingexamplequeryreturnseventmessagesfromtheSystemeventlogtogetherwitha"MyFacility"fieldthatmapseacheventsourcetoaSyslogfacilityname:

SELECTCASESourceNameWHEN'EventLog'THEN'mark'WHEN'ServiceControlManager'THEN'daemon'WHEN'Print'THEN'lpr'WHEN'Kerberos'THEN'auth'WHEN'NETLOGON'THEN'logaudit'WHEN'ApplicationPopup'THEN'local7'ELSE'local0'ENDASMyFacility,MessageINTOSYSLOGFROMSystem

Thisquerycanbeexecutedwiththefollowingcommand,whichspecifiesthatthefacilityvalueofeachoutputmessageistoberetrievedfromthe"MyFacility"outputrecordfield:

LogParserfile:MyQuery.sql-o:SYSLOG-conf:Myconfig.conf-facility:$MyFacilityTheSyslogmessagesgeneratedbythiscommandwilllooklikethefollowingexamples:

<134>Nov1318:17:25MYSERVER-MLogParser:Theservicewasstarted.<46>Nov1318:17:46MYSERVER-MLogParser:TheEventlogservicewasstarted.<30>Nov1318:17:46MYSERVER-MLogParser:TheTelephonyserviceenteredtherunningstate.<46>Nov1318:17:46MYSERVER-MLogParser:TheEventlogservicewasstopped.<134>Nov1318:17:46MYSERVER-MLogParser:Theservicewasstarted.<46>Nov1318:17:46MYSERVER-MLogParser:TheEventlogservicewasstarted.<30>Nov1318:17:46MYSERVER-MLogParser:TheTelephonyserviceenteredtherunningstate.

Theupper7bitsofthepriorityfieldofeachofthesemessagescontainthefacilityvalueprovidedbythe"MyFacility"outputrecordfield.

SeverityTheseverityvalueisrepresentedbythelower3bitsofthepriorityvalueinthePRIpartofthemessage,anditdescribestheimportanceofthemessage.Foradetaileddescriptionofthedifferentvaluesoftheseverityfield,refertoRFC3164.

<46>Nov1318:17:46MYSERVER-MLogParser:TheEventlogservicewasstopped.<134>Nov1318:17:46MYSERVER-MLogParser:Theservicewasstarted.<46>Nov1318:17:46MYSERVER-MLogParser:TheEventlogservicewasstarted.<30>Nov1318:17:46MYSERVER-MLogParser:TheTelephonyserviceenteredtherunningstate.

Thefollowingtableshowsthenamescommonlyassignedtothedifferentseverityvalues:

NumericalValue SeverityName

0 emerg

1 alert

2 crit

4 warning

5 notice

6 info

7 debug

Forexample,apriorityvalueof"14"indicatesaseverityvalueof6("info").

severityparameteroftheSYSLOGoutputformatallowsuserstocontrolthevalueoftheseverityfieldintheoutputmessages.Thisparametercanbesettoanyofthefollowingvalues:Anumericvalue,suchas"1"or"7";Thenameofaseverityvalue,suchas"alert"or"debug";Thenameorthe1-basedindexofanoutputrecordfieldprependedwithadollarcharacter("$"),suchas"$MySeverity"or"$2".ThespecifiedoutputrecordfieldmustbeofeithertheINTEGERdatatype-inwhichcaseitsvaluesareassumedtobenumericalseverityvalues,oroftheSTRINGdatatype-inwhichcaseitsvaluesareassumedtobeseveritynamesamongthosedescribedintheprevioustable.Whenanoutputrecordfieldvaluedoesnotcontainarecognized

severitynameoritcontainsaseverityvaluegreaterthan7,theSYSLOGoutputformatusesadefaultseverityvalueof6("info").

ThefollowingexamplequeryreturnseventmessagesfromtheSystemeventlogtogetherwitha"MySeverity"fieldthatmapseacheventtypetoaSyslogseverityname:

SELECTCASEEventTypeNameWHEN'Errorevent'THEN'err'WHEN'Warningevent'THEN'warning'WHEN'Informationevent'THEN'info'ELSE'info'ENDASMySeverity,MessageINTOSYSLOGFROMSystem

Thisquerycanbeexecutedwiththefollowingcommand,whichspecifiesthattheseverityvalueofeachoutputmessageistoberetrievedfromthe"MySeverity"outputrecordfield:

LogParserfile:MyQuery.sql-o:SYSLOG-conf:Myconfig.conf-severity:$MySeverityTheSyslogmessagesgeneratedbythiscommandwilllooklikethefollowingexamples:

<14>Nov1321:42:15MYSERVER-MLogParser:TheEventlogservicewasstarted.<11>Nov1321:42:15MYSERVER-MLogParser:TheComputerBrowserserviceterminatedwithservice-specificerror2550(0x9F6).<14>Nov1321:42:15MYSERVER-MLogParser:TheTerminalServicesservicewassuccessfullysentastartcontrol.<12>Nov1321:42:15MYSERVER-MLogParser:Arequesttosuspendpowerwasdeniedbywinlogon.exe.<14>Nov1321:42:15MYSERVER-MLogParser:TheEventlogservicewasstopped.

Thelower3bitsofthepriorityfieldofeachofthesemessagescontaintheseverityvalueprovidedbythe"MySeverity"outputrecordfield.

TimestampThetimestampfieldindicatesthelocaltimeatwhichthemessagewasoriginated,anditisusuallyformattedasfollows:

Nov1116:05:33

Ifthefirstfieldinthequeryoutputrecordsisofthe

TIMESTAMPdatatype,theSYSLOGoutputformatwillusethefieldvaluestopopulatethetimestampfieldintheoutputmessages.Ontheotherhand,ifthefirstfieldisnotoftheTIMESTAMPdatatype,theSYSLOGoutputformatwillusethecurrentlocaltime.

ThefollowingexamplequeryreturnseventmessagesfromtheSystem

eventlogtogetherwiththedateandtimeatwhichtheeventshavebeengenerated:

SELECTTimeGenerated,MessageINTOSYSLOGFROMSystemWHERESourceName='EventLog'

TheSyslogmessagesgeneratedbythisquerywilllooklikethefollowingexamples:

<14>Apr1818:48:04MYSERVER-MLogParser:TheEventlogservicewasstarted.<14>Apr1818:51:37MYSERVER-MLogParser:TheEventlogservicewasstopped.<14>Apr1819:20:07MYSERVER-MLogParser:Microsoft(R)Windows(R)5.01.2600ServicePack1UniprocessorFree.<14>Apr1819:20:07MYSERVER-MLogParser:TheEventlogservicewasstarted.<14>Apr1819:33:17MYSERVER-MLogParser:TheEventlogservicewasstopped.<14>Apr1907:01:41MYSERVER-MLogParser:Microsoft(R)Windows(R)5.01.2600ServicePack1UniprocessorFree.<14>Apr1907:01:41MYSERVER-MLogParser:TheEventlogservicewasstarted.<14>Apr1907:29:19MYSERVER-MLogParser:TheEventlogservicewasstopped.

HostnameThehostnamefieldindicatestheserveronwhichthemessageoriginated.

hostNameparameteroftheSYSLOGoutputformatallowsuserstocontrolthevalueofthehostnamefieldintheoutputmessages.Thisparametercanbesettoanyofthefollowingvalues:The"localhost"keyword,specifyingthatthefieldshouldbepopulatedwiththelocalcomputername;Agenericstringindicatingthedesiredhostname,suchas"MYCOMPUTER";Thenameorthe1-basedindexofanoutputrecordfieldprependedwithadollarcharacter("$"),suchas"$MyHostname"or"$2".ThespecifiedoutputrecordfieldmustbeoftheSTRINGdatatype,anditsvalueswillbeusedtopopulatethehostnamefieldintheoutputmessages.

Whennovalueisspecifiedforthe"hostName"parameter,thehostnamefieldisautomaticallypopulatedwiththelocalcomputername.

ThefollowingexamplequeryreturnseventmessagesfromtheSystemeventlogofdifferentcomputers,togetherwiththecomputernameonwhichtheeventoriginated:

SELECTMessage,ComputerNameINTOSYSLOGFROM\\MYSERVER01\System,\\MYSERVER02\System,\\MYSERVER03\System

Thisquerycanbeexecutedwiththefollowingcommand,whichspecifiesthatthehostnamefieldofeachoutputmessageistoberetrievedfromthesecondoutputrecordfield:

LogParserfile:MyQuery.sql-o:SYSLOG-conf:Myconfig.conf-hostName:$2

TheSyslogmessagesgeneratedbythiscommandwilllooklikethefollowingexamples:

<14>Nov1322:07:11MYSERVER03LogParser:Microsoft(R)Windows(R)5.01.2600ServicePack1UniprocessorFree.<14>Nov1322:07:11MYSERVER03LogParser:TheEventlogservicewasstarted.<14>Nov1322:07:11MYSERVER01LogParser:TheTerminalServicesservicewassuccessfullysentastartcontrol.<14>Nov1322:07:11MYSERVER02LogParser:TheNetworkConnectionsservicewassuccessfullysentastartcontrol.<14>Nov1322:07:11MYSERVER01LogParser:TheTerminalServicesserviceenteredtherunningstate.<14>Nov1322:07:11MYSERVER02LogParser:TheNetworkConnectionsserviceenteredtherunningstate.<14>Nov1322:07:11MYSERVER02LogParser:TheSSDPDiscoveryServiceservicewassuccessfullysentastartcontrol.<14>Nov1322:07:11MYSERVER03LogParser:TheSSDPDiscoveryServiceservicewassuccessfullysentastartcontrol.

TagThetagfieldindicatesthenameoftheprogramorprocessthatgeneratedthemessage.

processNameparameteroftheSYSLOGoutputformatallowsuserstocontrolthevalueofthetagfieldintheoutputmessages.Thisparametercanbesettoanyofthefollowingvalues:Agenericstringindicatingthedesiredtagfieldvalue,suchas"MyReports";Thenameorthe1-basedindexofanoutputrecordfieldprependedwithadollarcharacter("$"),suchas"$MyProgram"or"$2".ThespecifiedoutputrecordfieldmustbeoftheSTRINGdatatype,anditsvalueswillbeusedtopopulatethetagfieldintheoutputmessages.

Whennovalueisspecifiedforthe"processName"parameter,thetagfieldisautomaticallypopulatedwith"LogParser:".

ContentThecontentfieldcontainsthedetailsofthemessage,anditsvalueisbuiltbytheSYSLOGoutputformatbyconcatenatingthevaluesofallthe

outputrecordfields,excludingthosefieldsthatareusedforthevaluesofthe

facility,severity,timestamp,hostname,andtagmessagefields.

ThefollowingexamplequeryreturnsinformationfromtheSystemeventlog:

SELECTSourceName,EventTypeName,EventCategoryName,MessageINTOSYSLOGFROMSystem

TheSyslogmessagesgeneratedbythisquerywilllooklikethefollowingexamples:

<14>Nov1322:27:17MYSERVER-MLogParser:EventLogInformationeventNoneMicrosoft(R)Windows(R)5.01.2600ServicePack1UniprocessorFree.<14>Nov1322:27:17MYSERVER-MLogParser:EventLogInformationeventNoneTheEventlogservicewasstarted.<14>Nov1322:27:17MYSERVER-MLogParser:ServiceControlManagerErroreventNoneTheComputerBrowserserviceterminatedwithservice-specificerror2550(0x9F6).<14>Nov1322:27:17MYSERVER-MLogParser:EventLogInformationeventNoneTheEventlogservicewasstopped.<14>Nov1322:27:17MYSERVER-MLogParser:AtiHotKeyPollerInformationeventNoneTheservicewasstarted.<14>Nov1322:27:17MYSERVER-MLogParser:EventLogInformationeventNoneMicrosoft(R)Windows(R)5.01.2600ServicePack1UniprocessorFree.<14>Nov1322:27:17MYSERVER-MLogParser:EventLogInformationeventNoneTheEventlogservicewasstarted.<14>Nov1322:27:17MYSERVER-MLogParser:EventLogInformationeventNoneTheEventlogservicewasstopped.

SYSLOGOutputFormatConfigurationFilesMessagesgeneratedbytheSYSLOGoutputformatcanbeforwardedtoanyofthefollowingthreepossibledestinations:

ASyslogserver;Atextfile;Auser,throughtheWindowsalerterandmessengerservices.

TheconfparameteroftheSYSLOGoutputformatallowsuserstospecifyaconfigurationfileresemblingthestandard"syslog.conf"filethatdescribestherulesusedtoforwardmessagestodifferentdestinations.TheserulesassociatevaluesofthefacilityandseveritymessagefieldswithspecificSyslogservers,textfiles,orusers.

Eachlineinaconfigurationfileiseitheracommentbeginningwiththepoundcharacter("#"),oraconfigurationentry.Configurationentrieshavethefollowingsyntax:

<config_entry> ::= <selector><action>

<facilities> ::= <facility>[,<facility>...]

<action> ::= <send_server>|<send_file>|

<send_user>

<send_server> ::= @<server_name>[:<port>]

<send_file> ::= <filepath>|STDOUT

<send_user> ::= <user_name>

Aconfigurationentryiscomposedofaselectorandanaction,separatedbyspacesortabcharacters.Aselectorisacomma-separatedlistoffacilitynamesfollowedbyadot(".")andfollowedbyaseverityname.Thespecial"*"wildcardmeans"allfacilities".Messageswhosefacilityisincludedintheselector'ssetoffacilitiesandwhoseseverityisgreaterthanorequaltotheselector'sseverityareforwardedtothedestinationspecifiedintheaction.

Anactioncanspecifyanyofthefollowingdestinations:

ThenameoraddressofaSyslogserver,precededbyanatcharacter("@")andoptionallyfollowedbyaportnumber;whennoportnumberisspecified,theSYSLOGoutputformatwilluseport514;Thepathofanoutputfilename;TheSTDOUTkeyword,whichspecifiesthattheoutputdataistobewrittentotheoutputstream(theconsoleoutput);Thenameofauser.

ThefollowingexampleshowsaSYSLOGoutputformatconfigurationfile:

##SampleSYSLOGoutputformatconfigurationfile#auth.err@MYSERVER01*.debugSTDOUT*.infoC:\MyLogs\Infos.txt

Thisconfigurationfiledefinesthefollowingrules:Messagesfromthe"auth"facilitywithaseveritygreaterthanorequalto"err"areforwardedtothe"MYSERVER01"Syslogserveronport514;

kern.emergMYUSERlocal0,local1.emerg@192.168.1.100:515Allmessageshavingaseveritygreaterthanorequalto"debug"aredisplayedintheconsoleoutput;Allmessageshavingaseveritygreaterthanorequalto"info"arewrittentothe"C:\MyLogs\Infos.txt"textfile;Messagesfromthe"kern"facilitywithaseveritygreaterthanorequalto"emerg"aresenttothe"MYUSER"user;Messagesfromthe"local0"or"local1"facilitieswithaseveritygreaterthanorequalto"emerg"areforwardedtotheSyslogserverwithaddress192.168.1.100onport515.

Messagesmatchingmorethanoneruleareforwardedtoallthespecifieddestinations.Forexample,withtheaboveconfigurationfile,messageshavingaseveritygreaterthanorequalto"debug"arebothdisplayedintheconsoleoutputandwrittentothe"C:\MyLogs\Infos.txt"textfile.

Actionscanalsobespecifiedintheinto-entityofthequery.Theseactionsareprocessedasruleshavingaselectorthatmatchesallmessages,witha"*"facilityvalueandan"emerg"severityvalue.

SYSLOGOutputFormatInto-EntitySyntax<into-entity> ::= <action>[,<action>...]|

SYSLOG

<send_server> ::= @<server_name>[:<port>]

<send_file> ::= <filepath>|STDOUT

<send_user> ::= <user_name>

The<into-entity>specifiedinqueriesusingtheSYSLOGoutputformatiseitherthe"SYSLOG"keyword,whichspecifiesthatmessagesshouldbeforwardedaccordingtotherulesintheconfigurationfilespecifiedfortheconfparameter,oracomma-separatedlistofactions,whereeachactioniseither:

ThenameoraddressofaSyslogserver,precededbyanatcharacter("@")andoptionallyfollowedbyaportnumber;whennoportnumberisspecified,theSYSLOGoutputformatwilluseport514;Thepathofanoutputfilename;TheSTDOUTkeyword,whichspecifiesthattheoutputdataistobewrittentotheoutputstream(theconsoleoutput);Thenameofauser,towhichSyslogmessageswillbesentthroughtheWindowsalerterandmessengerservices.

Whenaconfigurationfilehasbeenspecifiedthroughthe"conf"parameter,queriesareallowedtonotprovideanINTOclauseatall;ifanINTOclauseisused,itsinto-entitymustbespecifiedas"SYSLOG".

Whenaconfigurationfilehasnotbeenspecified,theINTOclauseismandatoryanditmustcontainatleastonevalidaction.

Actionsspecifiedintheinto-entityareprocessedasconfigurationruleshavingaselectorthatmatchesallmessages,witha"*"facilityvalueandan"emerg"severityvalue.

Examples:

INTOSYSLOG

INTO@MYSERVER02:515

INTOMYUSER

INTO@MYSERVER01,C:\MyLogs\Infos.txt,STDOUT,MYUSER,@192.168.1.100:515

SYSLOGOutputFormatParametersTheSYSLOGoutputformatsupportsthefollowingparameters:

Values: filepath

Description: Syslogconfigurationfile.

Details: Thisparameterspecifiesthepathtoaconfigurationfilethatdescribestherulesusedtoforwardmessagestodifferentdestinations.Whenthisparameterisused,queriesareallowedtonotprovideanINTOclauseatall;ifanINTOclauseisused,itsinto-entitymustbespecifiedas"SYSLOG".Formoreinformationonconfigurationfiles,seeSYSLOGOutputFormatConfigurationFiles.

Example: -conf:C:\mysyslog.confseverity

Values: <numeric_value>|<name>|$<field_name>|$<field_index>

Default: info

Description: Messageseveritylevel.

Details: Thisparametercontrolsthevalueoftheseverityfieldoftheoutputmessages.Thepossiblevaluesforthisparameterare:Anumericvalue,suchas"1"or"7";Thenameofaseverityvalue,suchas"alert"or"debug";Thenameorthe1-basedindexofanoutputrecord

fieldprependedwithadollarcharacter("$"),suchas"$MySeverity"or"$2".ThespecifiedoutputrecordfieldmustbeofeithertheINTEGERdatatype-inwhichcaseitsvaluesareassumedtobenumericalseverityvalues,oroftheSTRINGdatatype-inwhichcaseitsvaluesareassumedtobeseveritynamesamongthosedescribedintheprevioustable.Whenanoutputrecordfieldvaluedoesnotcontainarecognizedseveritynameoritcontainsaseverityvaluegreaterthan7,theSYSLOGoutputformatusesadefaultseverityvalueof6("info").

Formoreinformationontheseverityfieldoftheoutputmessages,seeSYSLOGOutputFormatMessageStructure.

Examples: -severity:1-severity:alert-severity:$MySeverity-severity:$2

facility

Values: <numeric_value>|<name>|$<field_name>|$<field_index>

Default: user

Description: Messagefacility.

Details: Thisparametercontrolsthevalueofthefacilityfieldoftheoutputmessages.Thepossiblevaluesforthisparameterare:Anumericvalue,suchas"1"or"23";Thenameofafacilityvalue,suchas"user"or"local7";Thenameorthe1-basedindexofanoutputrecordfieldprependedwithadollarcharacter("$"),suchas

"$MyFacility"or"$2".ThespecifiedoutputrecordfieldmustbeofeithertheINTEGERdatatype-inwhichcaseitsvaluesareassumedtobenumericalfacilityvalues,oroftheSTRINGdatatype-inwhichcaseitsvaluesareassumedtobefacilitynamesamongthosedescribedintheprevioustable.Whenanoutputrecordfieldvaluedoesnotcontainarecognizedfacilitynameoritcontainsafacilityvaluegreaterthan23,theSYSLOGoutputformatusesadefaultfacilityvalueof1("user").

Formoreinformationonthefacilityfieldoftheoutputmessages,seeSYSLOGOutputFormatMessageStructure.

Examples: -facility:23-facility:local7-facility:$MyFacility-facility:$2

oTsFormat

Default: MMMdphh:mm:ss

Description: Formatofthetimestampfield.

Details: Thisparameterspecifiestheformatofthetimestampfieldoftheoutputmessages.Formoreinformationondateandtimeformats,seeTimestampFormatSpecifiers.Formoreinformationonthetimestampfieldoftheoutputmessages,seeSYSLOGOutputFormatMessageStructure.

Example: -oTsFormat:"MMMdd,yyyy"

hostName

Values: localhost|<name>|$<field_name>|$<field_index>

Default: localhost

Description: Valueofthehostnamefield.

Details: Thisparametercontrolsthevalueofthehostnamefieldoftheoutputmessages.Thepossiblevaluesforthisparameterare:The"localhost"keyword,specifyingthatthefieldshouldbepopulatedwiththelocalcomputername;Agenericstringindicatingthedesiredhostname,suchas"MYCOMPUTER";Thenameorthe1-basedindexofanoutputrecordfieldprependedwithadollarcharacter("$"),suchas"$MyHostname"or"$2".ThespecifiedoutputrecordfieldmustbeoftheSTRINGdatatype,anditsvalueswillbeusedtopopulatethehostnamefieldintheoutputmessages.

Formoreinformationonthehostnamefieldoftheoutputmessages,seeSYSLOGOutputFormatMessageStructure.

Examples: -hostName:MYCOMPUTER-hostName:$MyHostname-hostName:$2

processName

Values: <name>|$<field_name>|$<field_index>

Default: LogParser:

Description: Valueofthetagfield.

Details: Thisparametercontrolsthevalueofthetagfieldofthe

outputmessages.Thepossiblevaluesforthisparameterare:Agenericstringindicatingthedesiredtagfieldvalue,suchas"MyReports";Thenameorthe1-basedindexofanoutputrecordfieldprependedwithadollarcharacter("$"),suchas"$MyProgram"or"$2".ThespecifiedoutputrecordfieldmustbeoftheSTRINGdatatype,anditsvalueswillbeusedtopopulatethetagfieldintheoutputmessages.

Formoreinformationonthetagfieldoftheoutputmessages,seeSYSLOGOutputFormatMessageStructure.

Examples: -processName:MyReports-processName:$MyProgram-processName:$2

separator

Values: anystring|space|tab

Default: space

Description: Separatorbetweenfields.

Details: Thisparametercontrolstheseparatortobeusedbetweenthemessagefields.The"tab"keywordcausestheSYSLOGoutputformattouseasingletabcharacterbetweenthefields,whilethe"space"keywordcausestheSYSLOGoutputformattouseasinglespacecharacter.

Example: -separator:tabmaxPacketSize

Values: numberofbytes

Default: 1024

Description: Maximummessagesize.

Details: ThisparametercontrolsthemaximumsizeofthemessagesgeneratedbytheSYSLOGoutputformat.Messageswhosesizeexceedsthevaluespecifiedforthisparameterareeithertruncatedordiscarded,dependingonthevalueofthe"discardOversized"parameter.

Example: -maxPacketSize:8192discardOversized

Values: ON|OFF

Default: OFF

Description: Discardoversizedmessages.

Details: Whenthisparameterissetto"ON",theSYSLOGoutputformatdiscardsmessageswhosesizeexceedsthevaluespecifiedforthe"maxPacketSize"parameter.Whenthisparameterissetto"OFF",theSYSLOGoutputformattruncatesoversizedmessagestothesizespecifiedwiththe"maxPacketSize"parameter.

Example: -discardOversized:ONprotocol

Values: UDP|TCP

Default: UDP

Description: Protocolusedfortransmission.

Details: ThisparameterspecifiestheprotocoltousewhensendingmessagestoSyslogservers.

Example: -protocol:TCP

sourcePort

Values: portnumber|*

Default: *

Description: Sourceporttousefortransmission.

Details: ThisparameterspecifiesthesourceporttousewhensendingmessagestoSyslogservers.Specifying"*"causestheSYSLOGoutputformattochooseanyavailableportnumber.

Example: -sourcePort:514ignoreDspchErrs

Values: ON|OFF

Default: OFF

Description: Ignoredispatcherrors.

Details: Settingthisparameterto"ON"causestheSYSLOGoutputformattobuffererrorsoccurringwhiletransmittingmessagestoSyslogserversorusers,reportingalltheerrorsaswarningswhenthequeryexecutionhascompleted.Settingthisparameterto"OFF"causestheSYSLOGoutputformattoreporterrorsastheyoccur,abortingtheexecutionofthequery.

Example: -ignoreDspchErrs:ONoCodepage

Default: 0

Description: Codepageoftheoutputmessagetext.

Example: -oCodepage:1245

SYSLOGOutputFormatExamplesExportSystemEventLogExporteventsfromtheSystemeventlogtoaSyslogserverandtoalocalfile:

SELECTTimeGenerated,CASESourceNameWHEN'EventLog'THEN'mark'WHEN'ServiceControlManager'THEN'daemon'WHEN'Print'THEN'lpr'WHEN'Kerberos'THEN'auth'WHEN'NETLOGON'THEN'logaudit'WHEN'ApplicationPopup'THEN'local7'ELSE'local0'ENDASMyFacility,CASEEventTypeNameWHEN'Errorevent'THEN'err'WHEN'Warningevent'THEN'warning'WHEN'Informationevent'THEN'info'ELSE'info'ENDASMySeverity,ComputerName,STRCAT(SourceName,':'),MessageINTO@MYSERVER04,Log.txtFROMSystem

Thisquerycanbeexecutedwiththefollowingcommand:

LogParserfile:MyQuery.sql-o:SYSLOG-facility:$MyFacility-severity:$MySeverity-hostName:$ComputerNameTheoutputwilllooklikethefollowingsample:

<46>Apr1818:48:04MYSERVER-MLogParser:EventLog:TheEventlogservicewasstarted.<30>Apr1818:48:27MYSERVER-MLogParser:ServiceControlManager:TheTelephonyserviceenteredtherunningstate.<46>Apr1818:51:37MYSERVER-MLogParser:EventLog:TheEventlogservicewasstopped.<134>Apr1819:20:23MYSERVER-MLogParser:AtiHotKeyPoller:Theservicewasstarted.<46>Apr1819:20:07MYSERVER-MLogParser:EventLog:TheEventlogservicewasstarted.<30>Apr1819:20:47MYSERVER-MLogParser:ServiceControlManager:TheTelephonyserviceenteredtherunningstate.<46>Apr1819:33:17MYSERVER-MLogParser:EventLog:TheEventlogservicewasstopped.<134>Apr1907:01:57MYSERVER-MLogParser:AtiHotKeyPoller:Theservicewasstarted.<46>Apr1907:01:41MYSERVER-MLogParser:EventLog:TheEventlogservicewasstarted.<30>Apr1907:02:07MYSERVER-MLogParser:ServiceControlManager:TheTelephonyserviceenteredtherunningstate.

IISLogErrorEntriesSenderrorentriesintheIISlogtoaSyslogserver:

SELECTTO_TIMESTAMP(date,time),CASEsc-statusWHEN500THEN'emerg'ELSE'err'ENDASMySeverity,s-computernameASMyHostname,cs-uri-stem,sc-statusINTO@MYSERVER04FROM<1>WHEREsc-status>=400

Thisquerycanbeexecutedwiththefollowingcommand:

LogParserfile:MyQuery.sql-o:SYSLOG-facility:logalert-severity:$MySeverity-hostName:$MyHostname-processName:IIS:Themessageswilllooklikethefollowingsamples:

<115>Nov1800:28:43MYSERVER04IIS:/images/tibg.gif404<115>Nov1800:28:44MYSERVER04IIS:/aa.css404<115>Nov1800:28:59MYSERVER04IIS:/images/tibg.gif404<115>Nov1800:29:00MYSERVER04IIS:/aa.css404<115>Nov1800:29:01MYSERVER04IIS:/images/tibg.gif404<115>Nov1800:29:02MYSERVER04IIS:/images/tibg.gif404

<115>Nov1800:29:04MYSERVER04IIS:/gorice/rulesinfo.nsf403<115>Nov1800:29:05MYSERVER04IIS:/_vti_inf.html404<112>Nov1800:29:05MYSERVER04IIS:/_vti_bin/shtml.dll500<115>Nov1800:31:51MYSERVER04IIS:/na/index.html404

TPLOutputFormatTheTPLoutputformatwritesoutputrecordsformattedaccordingtouser-definedtemplates.

Templatesaretextfilesdividedintothreesections-aheader,abody,andafooter-containingvariablesthatrefertothevaluesandnamesoftheoutputrecordfields.Duringtheoutputgenerationstage,theTPLoutputformatsubstitutesthevariableswiththevaluesoftheoutputrecordfields,generatingtextfilesformattedaccordingtotheuserspecifications.

TheflexibilityoftheTPLoutputformatallowsuserstogenerateHTMLfiles,XMLfiles,andgenerictextfilesinalmostanyformat.

TemplateFilesInto-EntitySyntaxParametersExamples

TPLOutputFormatTemplateFilesTemplatefilesaredividedintothreesections:anoptionalheadersectionthatiswrittenonceatthebeginningoftheoutput,abodysectionthatiswrittenrepeatedlyforeachoutputrecord,andanoptionalfootersectionthatiswrittenonceattheendoftheoutput.Thebodysectioncancontainspecialvariablesthataresubstitutedatruntimewithvaluescomputedduringtheexecutionofthequery,suchasvaluesandnamesofoutputrecordfields,andthenumberoffieldsintheoutputrecords.Theheaderandfootersectionscancontainthesamevariablesavailabletothebodysection,exceptforthosethatrefertovaluesofoutputrecordfields.

Templatefilescanbespecifiedintwodifferentways:asrawformattemplates,orasstructuredformattemplates.

RawFormatTemplatesIntherawformat,thethreetemplatesectionsarespecifiedasthreedifferentfiles.Thetemplatefilecontainingthebodysectionisspecifiedusingthetplparameter,whiletheoptionalheaderandfootersectionsarespecifiedwiththetplHeaderandtplFooterparameters,respectively.

Thefollowingisasamplerawformattemplatefilecontainingthebodysection:

TheUrl%cs-uri-stem%,requestedby%c-ip%,took%time-taken%millisecondstoexecute.Itwasrequestedat%time%o’clock.ThefollowingcommandparsesanIISlogfileandcreatesatextfileformattedaccordingtothetemplatefile:

LogParser"SELECT*INTOout.txtFROMextend1.log"-o:TPL-tpl:mytemplate.tplTheresultingoutputwilllooklikethefollowingexample:

TheUrl/default.htm,requestedby192.168.1.102,took24millisecondstoexecute.Itwasrequestedat04:23:45o’clock.TheUrl/mydocuments/index.html,requestedby192.168.1.104,took134millisecondstoexecute.Itwasrequestedat04:23:47o’clock.TheUrl/mydocuments/styles/style.css,requestedby192.168.1.101,took49millisecondstoexecute.Itwasrequestedat04:23:48o’clock.

StructuredFormatTemplatesInthestructuredformat,asingletemplatefilecontainstheheader,body,andfootersections,eachenclosedwithinspecial<LPHEADER>,<LPBODY>,and<LPFOOTER>tagsthatmarktheboundariesofeachsection.Structuredformattemplatefilesarespecifiedusingthetplparameter.

Thefollowingisasamplestructuredformattemplatefile:

<LPHEADER>Thisismytemplate,foraquerycontaining%FIELDS_NUM%fields,executedby%USERNAME%.</LPHEADER>Someignoredcommenthere.<LPBODY>TheUrl%cs-uri-stem%,requestedby%c-ip%,took%time-taken%millisecondstoexecute.Itwasrequestedat%time%o’clock.</LPBODY><LPFOOTER>Endofreport.</LPFOOTER>

ThefollowingcommandparsesanIISlogfileandcreatesatextfileformattedaccordingtothetemplatefile:

LogParser"SELECT*INTOout.txtFROMextend1.log"-o:TPL-tpl:mytemplate.tplTheresultingoutputwilllooklikethefollowingexample:

Thisismytemplate,foraquerycontaining32fields,executedbyTestUser.TheUrl/default.htm,requestedby192.168.1.102,took24millisecondstoexecute.Itwasrequestedat04:23:45o’clock.TheUrl/mydocuments/index.html,requestedby192.168.1.104,took134millisecondstoexecute.Itwasrequestedat04:23:47o’clock.TheUrl/mydocuments/styles/style.css,requestedby192.168.1.101,took49millisecondstoexecute.Itwasrequestedat04:23:48o’clock.Endofreport.

Note:TheTPLoutputformatassumesthatthecharacterimmediatelyfollowingtheopeningtagforasection,suchas<LPBODY>,belongstothatsection.

TemplateVariablesThefollowingtableliststhevariablesthatareavailabletotemplatefiles:

Variable Description ExampleTemplate

%FIELD_n% Valueoftheoutput

Firstfieldvalue:%FIELD_1%

recordfieldwiththespecified1-basedindex

%field_name% Valueofthespecifiedoutputrecordfield

Firstfieldvalue:%SourceName%

%FIELDNAME_n% Nameoftheoutputrecordfieldwiththespecified1-basedindex

%FIELDNAME_1%value:%FIELD_1%

%FIELDS_NUM% Numberofoutputrecordfields

Thereare%FIELDS_NUM%fields.

%SYSTEM_TIMESTAMP% Currentsystemdateandtime,inUTCcoordinates

Generatedat%SYSTEM_TIMESTAMP%

%environment_variable% Valueofthespecifiedenvironment

variable1

Generatedby%USERNAME%

Notes:(1):Whenavariablematchesbothafieldnameandanenvironmentvariable,thefieldvalueissubstituted.

TPLOutputFormatInto-EntitySyntax<into-entity> ::= <filename>|

STDOUT

The<into-entity>specifiedinqueriesusingtheTPLoutputformatiseither:

TheTPLoutputformatsupportsthemultiplexfeature,whichcanbeenabledbyspecifying'*'wildcardsintheinto-entityfilename.Thisfeatureallowsoutputrecordstobewrittentodifferentfilesdependingonthevaluesoftheirfields.Formoreinformationonthemultiplexfeature,seeMultiplexingOutputRecords.

Examples:

INTOMyPage.html

INTOSTDOUT

INTOReports_*_*\Report*.txt

TPLOutputFormatParametersTheTPLoutputformatsupportsthefollowingparameters:

Values: filepath

Description: Templatefile.

Details: Whenusingrawformattemplatefiles,thisparameterspecifiesthetemplatefilecontainingthebodysection.Whenusingstructuredformattemplatefiles,thisparameterspecifiesthesingletemplatefilethatcontainstheheader,body,andfootersections.Formoreinformationontemplatefiles,seeTemplateFiles.

Example: -tpl:MyTemplate.tpltplHeader

Values: filepath

Description: Templateheaderfile.

Details: Whenusingrawformattemplatefiles,thisparameterspecifiesthetemplatefilecontainingtheheadersection.Whenusingstructuredformattemplatefiles,thisparameterspecifiesarawformattemplatefilethatoverridesthe<LPHEADER>sectionofthestructuredformattemplatefilespecifiedwiththe"tpl"parameter.Formoreinformationontemplatefiles,seeTemplateFiles.

Example: -tplHeader:MyTemplateHeader.tpltplFooter

Values: filepath

Description: Templatefooterfile.

Details: Whenusingrawformattemplatefiles,thisparameterspecifiesthetemplatefilecontainingthefootersection.Whenusingstructuredformattemplatefiles,thisparameterspecifiesarawformattemplatefilethatoverridesthe<LPFOOTER>sectionofthestructuredformattemplatefilespecifiedwiththe"tpl"parameter.Formoreinformationontemplatefiles,seeTemplateFiles.

Example: -tplFooter:MyTemplateFooter.tplnoEmptyFile

Values: ON|OFF

Default: ON

Description: Donotgenerateemptyfiles.

Details: Whenaquerydoesnotproduceoutputrecords,theTPLoutputformatdoesnotwriteabodysection,andtheresultingoutputfilecouldbeempty.Settingthisparameterto"ON"causestheTPLoutputformattoavoidgeneratinganemptyfileinthesesituations.

Example: -noEmptyFile:OFFoCodepage

Default: 0

Values: 0|1|2

Default: 1

Details: ThisparametercontrolsthebehavioroftheTPLoutputformatwhentheinto-entityspecifiesdirectlyorindirectlythroughthe"multiplex"featurethenameofafilethatalreadyexists.Thepossiblevaluesforthisparameterare:0:existingfilesareappendedwiththeoutput;1:existingfilesareoverwrittenwiththeoutput;2:existingfilesareleftintact,discardingtheoutput.

TPLOutputFormatExamplesLast50SecurityEventsCreateanHTMLpagecontainingthemostrecent50eventsfromtheSecurityeventlog:

LogParser"SELECTTOP50TimeGenerated,SourceName,EventID,MessageINTOEvents.htmlFROMSecurity"-i:EVT-direction:BW-o:TPL-tpl:HTMLBody.txt-tplHeader:HTMLHeader.txt-tplFooter:HTMLFooter.txt

MSDNBLogsChannelTitlesDisplaytitlesofcurrentchannelsonMSDNBLogs:

LogParser"SELECTtitleINTOchannels.txtFROMhttp://blogs.msdn.com/MainFeed.aspx#/rss/channel/item"-i:XML-fMode:Tree-o:TPL-tpl:mytemplate.tpl

TSVOutputFormatTheTSVoutputformatwritesoutputrecordsastab-separatedorspace-separatedvaluestext.

TheoutputoftheTSVoutputformatconsistsofmultiplelinesoftext,onelineforeachoutputrecord.Eachlinecontainsthevaluesoftheoutputrecordfields,separatedbyeitheratabcharacteroraspacecharacter,dependingonthevalueoftheoSeparatorparameter.Ifenabledthroughtheheadersparameter,thefirstlineintheoutputisa"header"thatcontainsthenamesofthefields.

ThefollowingsampleshowstheoutputoftheTSVoutputformatwhenusingthedefaultvaluesforitsparameters:

EventID SourceName EventType TimeGenerated6009 EventLog4 2004-04-1818:48:046005 EventLog4 2004-04-1818:48:047024 ServiceControlManager 1 2004-04-1818:48:277035 ServiceControlManager 4 2004-04-1818:48:277035 ServiceControlManager 4 2004-04-1818:48:277036 ServiceControlManager 4 2004-04-1818:48:277036 ServiceControlManager 4 2004-04-1818:48:277035 ServiceControlManager 4 2004-04-1818:48:277036 ServiceControlManager 4 2004-04-1818:48:277035 ServiceControlManager 4 2004-04-1818:48:277036 ServiceControlManager 4 2004-04-1818:48:277035 ServiceControlManager 4 2004-04-1818:48:277036 ServiceControlManager 4 2004-04-1818:48:277035 ServiceControlManager 4 2004-04-1818:48:277036 ServiceControlManager 4 2004-04-1818:48:277036 ServiceControlManager 4 2004-04-1818:48:277035 ServiceControlManager 4 2004-04-1818:48:367036 ServiceControlManager 4 2004-04-1818:51:267036 ServiceControlManager 4 2004-04-1818:51:296006 EventLog4 2004-04-1818:51:37

Seealso:CSVOutputFormatTSVInputFormat

TSVOutputFormatInto-EntitySyntax<into-entity> ::= <filename>|

STDOUT

The<into-entity>specifiedinqueriesusingtheTSVoutputformatiseither:

TheTSVoutputformatsupportsthemultiplexfeature,whichcanbeenabledbyspecifying'*'wildcardsintheinto-entityfilename.Thisfeatureallowsoutputrecordstobewrittentodifferentfilesdependingonthevaluesoftheirfields.Formoreinformationonthemultiplexfeature,seeMultiplexingOutputRecords.

Examples:

INTOreport.tsv

INTO\\COMPUTER01\Reports\report.tsv

INTOSTDOUT

INTOReports_*_*\Report*.tsv

TSVOutputFormatParametersTheTSVoutputformatsupportsthefollowingparameters:

headers

Values: ON|OFF|AUTO

Default: AUTO

Description: Writeaheaderlinecontainingthefieldnames.

Details: Thisparametercontrolstheheaderlinethatisoutputatthebeginningofeachfile.Thepossiblevaluesforthisparameterare:ON:alwayswritetheheader;OFF:neverwritetheheader;AUTO:writetheheaderonlywhennotappendingtoanexistingfile.

Example: -headers:OFFoSeparator

Values: anystring|space|tab

Default: tab

Description: Separatorbetweenfields.

Details: Thisparametercontrolstheseparatortobeusedbetweenfieldvalues.The"tab"keywordcausestheTSVoutputformattouseasingletabcharacterbetweenthefields,whilethe"space"keywordcausestheTSVoutputformattouseasinglespacecharacter.

Example: -oSeparator:space

oTsFormat

Description: FormatoftimestampvaluesintheoutputTSVdata.

Details: Thisparameterspecifiesthedateand/ortimeformattousewhenformattingvaluesoftheTIMESTAMPdatatype.Formoreinformationondateandtimeformats,seeTimestampFormatSpecifiers.

Example: -oTsFormat:"MMMdd,yyyy"oCodepage

Default: 0

Values: 0|1|2

Default: 1

Details: ThisparametercontrolsthebehavioroftheTSVoutputformatwhentheinto-entityspecifiesdirectlyorindirectlythroughthe"multiplex"featurethenameofafilethatalreadyexists.Thepossiblevaluesforthisparameterare:0:existingfilesareappendedwiththeoutput;

1:existingfilesareoverwrittenwiththeoutput;2:existingfilesareleftintact,discardingtheoutput.

TSVOutputFormatExamplesFileInformationCreateaTSVfilecontaininginformationonthefilescontainedinthespecifieddirectory:

LogParser"SELECTPath,Name,Size,AttributesINTOFiles.tsvFROMC:\Test\*.*"-i:FS-o:TSV-recurse:0

SecurityEventsRetrievethe10latesteventsfromtheSecurityeventlogandwritetheirinformationtoaTSVfileforeacheventID:

LogParser"SELECTTOP10EventID,EventTypeName,MessageINTOEvents_*.tsvFROMSecurity"-i:EVT-direction:BW-o:TSV

W3COutputFormatTheW3CoutputformatwritesoutputrecordsintheW3CExtendedLogFileFormat.

ThefollowingexampleshowsasampleoutputgeneratedbytheW3Coutputformat:

#Software:MicrosoftLogParser#Version:1.0#Date:2004-10-2514:20:40#Fields:datetimes-ids-types-category2004-04-1818:48:046009402004-04-1818:48:046005402004-04-1818:48:277024102004-04-1818:48:277035402004-04-1818:48:277035402004-04-1818:48:277036402004-04-1818:48:277036402004-04-1818:48:277035402004-04-1818:48:27703640

Seealso:W3CInputFormat

W3COutputFormatInto-EntitySyntax<into-entity> ::= <filename>|

STDOUT

The<into-entity>specifiedinqueriesusingtheW3Coutputformatiseither:

TheW3Coutputformatsupportsthemultiplexfeature,whichcanbeenabledbyspecifying'*'wildcardsintheinto-entityfilename.Thisfeatureallowsoutputrecordstobewrittentodifferentfilesdependingonthevaluesoftheirfields.Formoreinformationonthemultiplexfeature,seeMultiplexingOutputRecords.

Examples:

INTOreport.log

INTO\\COMPUTER01\Reports\report.log

INTOSTDOUT

INTOReports_*_*\Report*.log

W3COutputFormatParametersTheW3Coutputformatsupportsthefollowingparameters:

Default: 10

Details: WhenwritingtoSTDOUT,theW3Coutputformatdisplaysoutputrecordsinbatchesmadeupofanumberofrowsequaltothevaluespecifiedforthisparameter.Onceabatchofrowshasbeendisplayed,theW3Coutputformatpromptstheusertopressakeytodisplaythenextbatchofrows.Specifying"-1"forthisparameterdisablesbatchingaltogether.

Example: -rtp:-1oDQuotes

Values: ON|OFF

Default: OFF

Description: Enclosestringvaluesindouble-quotecharacters.

Details: Whenthisparameterissetto"ON",theW3Coutputformatwritesstringvalueswithdouble-quote(")charactersaroundthem.

Example: -oDQuotes:ONoDirTime

Values: anystring

Description: Contentofthe"#Date"directiveheader.

Details: TheW3Coutputformatusesthevaluespecifiedforthisparameterasthecontentofthe"#Date"directivewrittentotheheaderoftheoutputfile.Whenavalueisnotspecified,theW3Coutputformatusesthecurrentdateandtime.

Example: -oDirTime:"1973-05-2803:02:42"encodeDelim

Values: ON|OFF

Default: OFF

Description: Substitutespacecharacterswithinfieldvalueswithpluscharacters.

Details: Whenthisparameterissetto"ON",theW3Coutputformatsubstitutesspacecharactersfoundinstringvalueswithplus(+)characters,inordertogenerateW3Coutputthatisformattedcorrectly.Whenthisparameterissetto"OFF",spacecharacterswithinfieldvaluesarepreserved,potentiallygeneratinginvalidW3Coutput.

Example: -encodeDelim:ONoCodepage

Default: 0

Example: -oCodepage:1245

fileMode

Values: 0|1|2

Default: 1

Details: ThisparametercontrolsthebehavioroftheW3Coutputformatwhentheinto-entityspecifiesdirectlyorindirectlythroughthe"multiplex"featurethenameofafilethatalreadyexists.Thepossiblevaluesforthisparameterare:0:existingfilesareappendedwiththeoutput;1:existingfilesareoverwrittenwiththeoutput;2:existingfilesareleftintact,discardingtheoutput.

W3COutputFormatExamplesEventLogReportCreateaW3CfilewithinformationfromtheSystemeventlog:

LogParser"SELECTTO_DATE(TimeGenerated)ASdate,TO_TIME(TimeGenerated)AStime,SourceNameASs-source,EventIDASs-event-id,EventCategoryASs-event-categoryINTOreport.logFROMSystem"-i:EVT-o:W3C-encodeDelim:ON

XMLOutputFormatTheXMLoutputformatwritesoutputrecordsasXMLdocumentnodes.

UserscanchoosebetweenfourdifferentstructuresfortheoutputXMLdocument.Differentstructuresformattheoutputrecordfieldsindifferentways,givinguserstheabilitytofine-tunethegeneratedXMLfortheirapplications.

ThefollowingexamplecommandgeneratesanXMLdocumentcontainingfieldsfromtheSystemeventlog:

LogParser"SELECTTimeGenerated,SourceName,EventID,MessageINTOEvents.xmlFROMSystem"TheoutputXMLwilllooklikethefollowingexample:

<?xmlversion="1.0"encoding="ISO-10646-UCS-2"standalone="yes"?><!DOCTYPEROOT[<!ATTLISTROOTDATE_CREATEDCDATA#REQUIRED><!ATTLISTROOTCREATED_BYCDATA#REQUIRED><!ELEMENTTimeGenerated(#PCDATA)><!ELEMENTSourceName(#PCDATA)><!ELEMENTEventID(#PCDATA)><!ELEMENTMessage(#PCDATA)><!ELEMENTROW(TimeGenerated,SourceName,EventID,Message)><!ELEMENTROOT(ROW*)>]><ROOTDATE_CREATED="2004-11-0816:26:54"CREATED_BY="MicrosoftLogParserV2.2"><ROW><TimeGenerated>2004-04-1818:48:04</TimeGenerated><SourceName>EventLog</SourceName>

DocumentStructuresInto-EntitySyntaxParametersExamples

Seealso:XMLInputFormat

<EventID>6009</EventID><Message>Microsoft(R)Windows(R)5.01.2600ServicePack1UniprocessorFree.</Message></ROW><ROW><TimeGenerated>2004-04-1818:48:04</TimeGenerated><SourceName>EventLog</SourceName><EventID>6005</EventID><Message>TheEventlogservicewasstarted.</Message></ROW><ROW><TimeGenerated>2004-04-1818:48:27</TimeGenerated><SourceName>ServiceControlManager</SourceName><EventID>7035</EventID><Message>TheNetworkConnectionsservicewassuccessfullysentastartcontrol.</Message></ROW></ROOT>

XMLOutputFormatDocumentStructuresTheXMLoutputformatgeneratesXMLdocumentsthatcanbestructuredinfourdifferentways,dependingonthevaluespecifiedforthestructureparameter.

Structure1Whenthe"structure"parameterissetto"1",theXMLoutputformatcreatesanodenamed"ROW"foreachoutputrecord.Thisnodeinturncontainsnodesforeachfieldintheoutputrecord,namedafterthefieldnamesandwithnodevaluescontainingthefieldvalues.

ThefollowingexampleshowsanXMLdocumentcreatedwithstructure"1":

<?xmlversion="1.0"encoding="ISO-10646-UCS-2"standalone="yes"?><!DOCTYPEROOT[<!ATTLISTROOTDATE_CREATEDCDATA#REQUIRED><!ATTLISTROOTCREATED_BYCDATA#REQUIRED><!ELEMENTTimeGenerated(#PCDATA)><!ELEMENTSourceName(#PCDATA)><!ELEMENTEventID(#PCDATA)><!ELEMENTMessage(#PCDATA)><!ELEMENTROW(TimeGenerated,SourceName,EventID,Message)><!ELEMENTROOT(ROW*)>]><ROOTDATE_CREATED="2004-11-0817:36:44"CREATED_BY="MicrosoftLogParserV2.2"><ROW><TimeGenerated>2004-04-1818:48:04</TimeGenerated><SourceName>

Structure2Settingthe"structure"parameterto"2"causestheXMLoutputformattogenerateXMLdocumentsthatareformattedaccordingtostructure"1",andinwhichfieldnodeshavea"TYPE"attributethatspecifiesthedatatypeofthecorrespondingoutputrecordfield.

<?xmlversion="1.0"encoding="ISO-10646-UCS-2"standalone="yes"?><!DOCTYPEROOT[<!ATTLISTROOTDATE_CREATEDCDATA#REQUIRED><!ATTLISTROOTCREATED_BYCDATA#REQUIRED><!ELEMENTTimeGenerated(#PCDATA)><!ATTLISTTimeGeneratedTYPECDATA#REQUIRED><!ELEMENTSourceName(#PCDATA)>

Structure3Whenthe"structure"parameterissetto"3",theXMLoutputformatcreatesanodenamed"ROW"foreachoutputrecord.Thisnodeinturncontainsnodesnamed"FIELD"foreachfieldinthe

EventLog</SourceName><EventID>6009</EventID><Message>Microsoft(R)Windows(R)5.01.2600ServicePack1UniprocessorFree.</Message></ROW><ROW><TimeGenerated>2004-04-1818:48:04</TimeGenerated><SourceName>EventLog</SourceName><EventID>6005</EventID><Message>TheEventlogservicewasstarted.</Message></ROW></ROOT>

<!ATTLISTSourceNameTYPECDATA#REQUIRED><!ELEMENTEventID(#PCDATA)><!ATTLISTEventIDTYPECDATA#REQUIRED><!ELEMENTMessage(#PCDATA)><!ATTLISTMessageTYPECDATA#REQUIRED><!ELEMENTROW(TimeGenerated,SourceName,EventID,Message)><!ELEMENTROOT(ROW*)>]><ROOTDATE_CREATED="2004-11-0817:30:25"CREATED_BY="MicrosoftLogParserV2.2"><ROW><TimeGeneratedTYPE="TIMESTAMP">2004-04-1818:48:04</TimeGenerated><SourceNameTYPE="STRING">EventLog</SourceName><EventIDTYPE="INTEGER">6009</EventID><MessageTYPE="STRING">Microsoft(R)Windows(R)5.01.2600ServicePack1UniprocessorFree.</Message></ROW><ROW><TimeGeneratedTYPE="TIMESTAMP">2004-04-1818:48:04</TimeGenerated><SourceNameTYPE="STRING">EventLog</SourceName><EventIDTYPE="INTEGER">6005</EventID><MessageTYPE="STRING">TheEventlogservicewasstarted.</Message>

outputrecord;each"FIELD"nodehasanodevalueequaltothefieldvalue,anda"NAME"attributethatspecifiesthefieldname.

<?xmlversion="1.0"encoding="ISO-10646-UCS-2"standalone="yes"?><!DOCTYPEROOT[<!ATTLISTROOTDATE_CREATEDCDATA#REQUIRED><!ATTLISTROOTCREATED_BYCDATA#REQUIRED><!ELEMENTFIELD(#PCDATA)><!ATTLISTFIELDNAMECDATA#REQUIRED><!ELEMENTROW(FIELD,FIELD,FIELD,FIELD)><!ELEMENTROOT(ROW*)>]><ROOTDATE_CREATED="2004-11-0817:32:41"CREATED_BY="MicrosoftLogParserV2.2"><ROW><FIELDNAME="TimeGenerated">2004-04-1818:48:04</FIELD><FIELDNAME="SourceName">EventLog</FIELD><FIELDNAME="EventID">6009</FIELD><FIELDNAME="Message">Microsoft(R)Windows(R)5.01.2600ServicePack1UniprocessorFree.</FIELD></ROW><ROW><FIELDNAME="TimeGenerated">2004-04-1818:48:04</FIELD><FIELDNAME="SourceName">EventLog

Structure4Settingthe"structure"parameterto"4"causestheXMLoutputformattogenerateXMLdocumentsthatareformattedaccordingtostructure"3",andinwhich"FIELD"nodeshaveanadditional"TYPE"attributethatspecifiesthedatatypeofthecorrespondingoutputrecordfield.

<?xmlversion="1.0"encoding="ISO-10646-UCS-2"standalone="yes"?><!DOCTYPEROOT[<!ATTLISTROOTDATE_CREATEDCDATA#REQUIRED><!ATTLISTROOTCREATED_BYCDATA#REQUIRED><!ELEMENTFIELD(#PCDATA)><!ATTLISTFIELDNAMECDATA#REQUIRED><!ATTLISTFIELDTYPECDATA#REQUIRED><!ELEMENTROW(FIELD,FIELD,FIELD,FIELD)><!ELEMENTROOT(ROW*)>]><ROOTDATE_CREATED="2004-11-0817:35:04"CREATED_BY="MicrosoftLogParserV2.2"><ROW><FIELDNAME="TimeGenerated"TYPE="TIMESTAMP">2004-04-1818:48:04</FIELD><FIELDNAME="SourceName"TYPE="STRING">EventLog</FIELD><FIELDNAME="EventID"TYPE="INTEGER">

</ROW></ROOT></FIELD><FIELDNAME="EventID">6005</FIELD><FIELDNAME="Message">TheEventlogservicewasstarted.</FIELD></ROW></ROOT>

6009</FIELD><FIELDNAME="Message"TYPE="STRING">Microsoft(R)Windows(R)5.01.2600ServicePack1UniprocessorFree.</FIELD></ROW><ROW><FIELDNAME="TimeGenerated"TYPE="TIMESTAMP">2004-04-1818:48:04</FIELD><FIELDNAME="SourceName"TYPE="STRING">EventLog</FIELD><FIELDNAME="EventID"TYPE="INTEGER">6005</FIELD><FIELDNAME="Message"TYPE="STRING">TheEventlogservicewasstarted.</FIELD></ROW></ROOT>

XMLOutputFormatInto-EntitySyntax<into-entity> ::= <filename>|

STDOUT

The<into-entity>specifiedinqueriesusingtheXMLoutputformatiseither:

TheXMLoutputformatsupportsthemultiplexfeature,whichcanbeenabledbyspecifying'*'wildcardsintheinto-entityfilename.Thisfeatureallowsoutputrecordstobewrittentodifferentfilesdependingonthevaluesoftheirfields.Formoreinformationonthemultiplexfeature,seeMultiplexingOutputRecords.

Examples:

INTOreport.xml

INTO\\COMPUTER01\Reports\report.xml

INTOSTDOUT

INTOReports_*_*\Report*.xml

XMLOutputFormatParametersTheXMLoutputformatsupportsthefollowingparameters:

structure

Values: 1|2|3|4

Default: 1

Description: Structureoftheoutputdocument.

Details: Foradescriptionofthedifferentstructuresavailable,seeDocumentStructures.

Example: -structure:4rootName

Values: string

Default: ROOT

Description: Nameofthedocumentrootnode.

Details: Thisparameterallowsuserstocustomizethenameofthesinglerootnodethatcontainsalltheothernodesintheoutputdocument.

Example: -rootName:REPORTrowName

Values: string

Default: ROW

Description: Nameofthenodecontainingtheoutputrecordfields.

Details: Thisparameterallowsuserstocustomizethenameofthenodethatisgeneratedforeachoutputrecord.

Example: -rowName:ENTRYfieldName

Values: string

Default: FIELD

Description: Nameofthenodecontainingtheoutputrecordfieldvalues.

Details: Thisparameterallowsuserstocustomizethenameofthenodethatisgeneratedforeachoutputrecordfieldwhenthe"structure"parameterissetto"3"or"4".

Example: -fieldName:DATAxslLink

Values: pathtoXSLdocument

Description: XSLdocumenttobereferencedbytheoutputXMLdocument.

Details: SpecifyingavalueforthisparametercausestheXMLoutputformattoplacealinktothespecifiedXSLstylesheetintheheaderoftheoutputXMLdocument.XSL-enabledXMLbrowserswillfollowthespecifiedlinkandformattheoutputXMLdocumentaccordingly.Thelinkplacedinthedocumentheaderisformattedasfollows:

<?xml-stylesheettype="text/xsl"href="C:\XSL\MyXSL.xsl"?>

Example: -xslLink:C:\XSL\MyXSL.xslschemaType

Values: 0|1

Default: 1

Description: Typeofinlineschema.

Details: Whenthisparameterissetto"1",theoutputXMLdocumentcontainsaninlineDTDschema.Settingthisparameterto"0"preventstheXMLoutputformatfromgeneratinganinlineschema.

Example: -schemaType:0compact

Values: ON|OFF

Default: OFF

Description: Suppressindentationsandextralinesinoutput.

Details: Whenthisparameterissetto"OFF",theXMLoutputformatgeneratesXMLdocumentsthatareoptimizedforhumanreadability,indentingnodesaccordingtotheirdepth,andwritingnodesonmultiplelines.Settingthisparameterto"ON"causestheXMLoutputformattowriteeach"ROW"nodeonasinglelinewithoutindentation.

Example: -compact:ONnoEmptyField

Values: ON|OFF

Default: OFF

Description: AvoidwritingemptynodesforNULLfieldvalues.

Details: Whenthisparameterissetto"OFF",outputrecordfieldshavingNULLvaluesarerenderedasemptynodes.Settingthisparameterto"ON"preventstheXMLoutputformatfromgeneratinganodewhenthecorresponding

outputrecordfieldhasaNULLvalue.

Example: -noEmptyField:ONstandAlone

Values: ON|OFF

Default: ON

Description: Createawell-formed,stand-aloneXMLdocument.

Details: Whenthisparameterissetto"ON",theXMLoutputformatgenerateswell-formedXMLdocumentshavinganXMLheaderandasingledocumentrootnode.Whenthisparameterissetto"OFF",theXMLoutputformatgeneratesXMLtextthatonlycontainstheoutputrecordnodes,withnoXMLheaderandnodocumentrootnode.

Example: -standAlone:OFFoCodepage

Default: 0

Values: 0|1|2

Default: 1

Details: ThisparametercontrolsthebehavioroftheXMLoutputformatwhentheinto-entityspecifiesdirectlyorindirectlythroughthe"multiplex"featurethenameofafilethatalreadyexists.Thepossiblevaluesforthisparameterare:0:existingfilesareappendedwiththeoutput;1:existingfilesareoverwrittenwiththeoutput;2:existingfilesareleftintact,discardingtheoutput.

XMLOutputFormatExamplesAccountLogonsCreateanXMLdocumentcontaininglogonaccountnamesanddatesfromtheSecurityEventLogmessages:

LogParser"SELECTTimeGeneratedASLogonDate,EXTRACT_TOKEN(Strings,0,'|')ASAccountINTOReport.xmlFROMSecurityWHEREEventIDNOTIN(541;542;543)ANDEventType=8ANDEventCategory=2"

Command-LineOperationTheLogParsercommand-lineexecutableisasingle,standalonebinaryfile("LogParser.exe")thatcanbeusedfromtheWindowscommand-lineshelltoexecutequeriesandperformotherLogParsertasks.Theexecutablebinarydoesnotrequireanyinstallation;oncecopiedtoacomputer,itisreadytouse.

Tip:IfyouwanttorunLogParser.exefromanydirectorywithouthavingtospecifytheabsoluteorrelativepath,youcanaddtheLogParserdirectorylocationtothe"PATH"environmentvariable.

TheLogParsercommand-lineexecutableworksoncommandssuppliedbytheuser.Commandsarecombinationsofswitches,orarguments,thatspecifyparametersforthetaskthatneedstobeexecuted.TheswitchesusedwiththeLogParsercommand-lineexecutablemustbeenteredwithadashcharacter(-)followedbytheswitchname,asinthefollowingexample:

C:\>LogParser-h

Mostswitchesrequireauser-suppliedvalue;inthesecases,theswitchnamemustbefollowedbyacoloncharacter(:)andbytheuser-suppliedvaluewithnointerveningspaces,asinthefollowingexample:

C:\>LogParser-iCodepage:931

Iftheuser-suppliedvaluecontainsspaces,thevaluecanbesurroundbydouble-quotecharacters("),asinthefollowingexample:

C:\>LogParser-chartTitle:"Top20Pages"

Dependingontheswitchesusedinacommand,theLogParsercommand-lineexecutablecanbeusedinfourdifferentmodesofoperation:

QueryExecutionMode:thisisthedefaultmodeofoperation;inthis

mode,LogParserisusedtoexecutequeriesreadinginputrecordsfromaninputformatandwritingoutputrecordstoanoutputformat.ConversionMode:inthismode,activatedbythe"-c"switch,LogParserisusedtoexecutebuilt-inqueriesthatconvertlogfilesbetweensupportedlogfileformats.DefaultsOverrideMode:inthismode,activatedbythe"-saveDefaults"switch,userscanoverridethedefaultbehaviorofLogParserbyspecifyingcustomdefaultvaluesfortheexecutionparameters.HelpMode:inthismode,activatedbythe"-h"switch,thecommand-lineexecutablecanbeusedtodisplaytotheconsolewindowa"quickreference"helponselectedtopics,suchasinformationoninputandoutputformats,syntaxoffunctions,andsyntaxoftheLogParserSQL-Likequerylanguage.

Seealso:GlobalSwitchesReferenceCommandsandQueries

QueryExecutionMode"QueryExecutionMode"isthedefaultoperationalmodeoftheLogParsercommand-lineexecutable.Inthismode,LogParserisusedtoexecutequeriesreadinginputrecordsfromaninputformatandwritingoutputrecordstoanoutputformat.

Thegeneralsyntaxofcommandsinqueryexecutionmodeis:

LogParser [-i:<input_format>][<input_format_options>][-o:<output_format>][<output_format_options>]<SQLquery>|file:<query_filename>[?param1=value1+...][<global_switches>][-queryInfo]

-i:<input_format>

Specifiestheinputformatforthequery.The"-i:"switchisfollowedbythenameoftheselectedinputformat,asinthefollowingexample:

C:\>LogParser-i:IISW3C"SELECT*FROMextend1.log"

Whenaninputformatisnotspecified,LogParserwillattempttoselectautomaticallyaninputformatuponinspectionofthe<from-entity>intheFROMclause.Forexample,"System"suggeststheuseoftheEVTInputFormat,while"ex040302.log"suggeststheuseoftheIISW3CInputFormat.Ifthe<from-entity>doesnotsuggestaspecificinputformat,theTextLineInputFormatwillbeselectedbydefault.

<input_format_options>

Specifyvaluesforinputformatparameters.Theseareenteredasswitcheswithnamesmatchingtheinputformat'sparameternames,followedbyacolonandbythevaluefor

theparameter,asinthefollowingexamples:

C:\>LogParser-i:IISW3C-iCodepage:932-iCheckpoint:MyCheckpoint.lpc"SELECT*FROMextend1.log"C:\>LogParser-i:EVT-binaryFormat:ASC"SELECT*FROMSystem"

Parametervaluescontainingspacesmustbeenclosedwithindouble-quotecharacters("),asinthefollowingexample:

C:\>LogParser-i:EVT-stringsSep:"MYSEPARATOR""SELECT*FROMSystem"Formoreinformationoninputformatparameters,refertotheInputFormatReference.

-o:<output_format>

Specifiestheoutputformatforthequery.The"-o:"switchisfollowedbythenameoftheselectedoutputformat,asinthefollowingexample:

C:\>LogParser-o:CSV"SELECT*FROMSystem"

Whenanoutputformatisnotspecified,LogParserwillattempttoselectautomaticallyanoutputformatuponinspectionofthe<into-entity>intheINTOclause.Forexample,"chart.gif"suggeststheuseoftheCHARTOutputFormat,while"MyFile.csv"suggeststheuseoftheCSVOutputFormat.Ifthe<into-entity>doesnotsuggestaspecificoutputformat,orthequerydoesnotspecifyanINTOclause,theNATOutputFormatwillbeselectedbydefault.

<output_format_options>

Specifyvaluesforoutputformatparameters.Theseareenteredasswitcheswithnamesmatchingtheoutputformat'sparameternames,followedbyacolonandbythevaluefor

theparameter,asinthefollowingexamples:

C:\>LogParser-o:NAT-rtp:-1-fileMode:1"SELECT*FROMSystem"

C:\>LogParser-o:CSV-tabs:ON"SELECT*FROMSystem"

Parametervaluescontainingspacesmustbeenclosedwithindouble-quotecharacters("),asinthefollowingexample:

C:\>LogParser-o:CHART-chartTitle:"PageHitsperDay""SELECTdate,COUNT(*)FROMextend1.logGROUPBYdate"Formoreinformationonoutputformatparameters,refertotheOutputFormatReference.

SpecifiesthetextoftheLogParserSQL-Likequery.Sinceaqueryalwayscontainsspaces,thetextofthequerymustbeenclosedwithindouble-quotecharacters("),asinthefollowingexample:

C:\>LogParser"SELECT*FROMSystem"

Alternatively,aquerycanbespecifiedthroughatextfilewiththe"file:"switch,asshowninthenextsection.Commandscontainingbothaquerytextargumentanda"file:"switchareconsideredillegalandreturnanerror.

file:<query_filename>[?param1=value1+...]

SpecifiesthenameofatextfilecontainingaLogParserSQL-Likequery.ThetextfilespecifiedmustcontainavalidqueryintheLogParserSQL-Likelanguage.Multiplespaces,comments,andnew-linecharactersinthetextfileareignored,allowingthequerytexttobeformattedasdesiredforreadability.

Thefollowingexampleshowsanexamplecontentofaquerytextfile:

SELECTTimeGenerated,EXTRACT_TOKEN(ResolvedSid,1,'\\')ASUsername--onlythe'username'portion/*Wewanttoretrievethefullusername*/USINGRESOLVE_SID(Sid)ASResolvedSidFROMSecurity

Thefollowingexampleshowshowthequeryisexecuted,assumingthatthequerytexthasbeensavedtoafilenamed"MyQuery.sql":

C:\>LogParser-i:EVTfile:Myquery.sql

Querytextfilescanincludeparameters,whicharesubstitutedatruntimewithuser-suppliedtextorenvironmentvariablevalues.Parametersareuser-definednamesinthequerytextenclosedwithinpercentcharacters(%),suchas"%MyParameter%".WhenissuingaLogParsercommandtoexecuteaquerytextfilecontainingparameters,userscanspecifythevaluesoftheparametersbyappendingthequestion-markcharacter(?)tothequeryfilename,followedbyalistofpairsintheformof"parameter_name=parameter_value",separatedbythepluscharacter(+).Forexample,thefollowingquerycontainstwoparameters:

SELECTEventIDFROM%InputEventLog%WHERESourceName='%InputSourceName%'Thefollowingexamplecommandexecutesthequerysubstitutinguser-suppliedvaluesfortheparameters:

C:\>LogParser-i:EVTfile:Myquery.sql?InputEventLog=System+InputSourceName=EventLogIfaparameternameorvaluecontainsspaces,thenameorvaluemustbeenclosedwithindouble-quotecharacters("),asinthefollowingexample:

C:\>LogParser-i:EVTfile:Myquery.sql?InputEventLog=System+InputSourceName="ServiceControlManager"Ifthevalueofaquerytextfileparameterisnotsuppliedbytheuser,LogParserwillsearchfortheparameternameinthecurrentenvironmentvariableset.Ifanenvironmentvariableisfound

matchingtheparametername,itsvaluewillbesubstitutedfortheparameter;otherwise,theparameternameisleftas-isinthequerytext.

Thetextofthequerycanalsobespecifieddirectlyasacommand-lineargument,asshownintheprevioussection.Commandscontainingbothaquerytextargumentanda"file:"switchareconsideredillegalandreturnanerror.

<global_switches>

Globalswitchescontroloverallbehaviorsofthecommand,suchaserrorhandlingandcommandstatisticsverbosity.Formoreinformationonglobalswitches,refertotheGlobalSwitchesReference.

-queryInfo

Displaysdiagnosticinformationaboutthecommand.When"-queryInfo"isspecified,thecommandisnotexecuted,andthefollowingdiagnosticinformationisdisplayedtotheconsolewindow:Thetextoftheprovidedquery,afterbeingparsedandinterpretedbytheLogParserSQL-Likeenginecore;Namesoftheinputandoutputformatsselected;Structureofthequeryoutputrecords,includingfieldnamesandfielddatatypes.

Thisinformationcanbeusedtotroubleshootavarietyofproblems,includingunexpectedqueryexecutionresults,andqueryparametersubtitution.

Thefollowingexampleusesthe"-queryInfo"switchtodisplaydiagnosticinformationaboutthespecifiedcommand:

C:\>LogParser"SELECTTO_UTCTIME(TimeGenerated)ASUTCTimeGenerated,SourceNameFROMSystemWHEREEventID>20"-queryIn

foTheoutputofthiscommandis:

Query:SELECTTO_UTCTIME([TimeGenerated])ASUTCTimeGenerated,[SourceName]FROMSystemWHERE[EventID]>ANY(20)

Formatsselected:Inputformat:EVT(WindowsEventLog)Outputformat:NAT(NativeFormat)

Queryfields:UTCTimeGenerated(T)SourceName(S)

Seealso:Command-LineOperationReferenceGlobalSwitchesReferenceCommandsandQueries

ConversionModeIn"ConversionMode",LogParserisusedtoexecutebuilt-inqueriestoconvertlogfilesbetweenthefollowingformats:

BINtoW3CIIStoW3CBINtoIISIISW3CtoIIS

Conversionmodeisactivatedbythe"-c"switch.

Thegeneralsyntaxofcommandsinconversionmodeis:

LogParser -c-i:<input_format>-o:<output_format><from_entity><into_entity>[<where_clause>][<input_format_options>][<output_format_options>][-multiSite[:ON|OFF]][<global_switches>][-queryInfo]

Formoreinformationonlogfileformatconversions,refertoConvertingFileFormats.

-i:<input_format>

Specifiestheinputformatfortheconversion.The"-i:"switchisfollowedbythenameoftheselectedinputformat,asinthefollowingexample:

C:\>LogParser-c-i:IISW3C-o:IISextend1.loginetsv1.log

DifferentlythanQueryExecutionMode,theinputformatspecificationisamandatoryargumentforcommandsinconversionmode.Thespecifiedinputformatnamemustbeoneoftheinputformatsinthetableaboveforwhichaconversionissupported.

-o:<output_format>

Specifiestheoutputformatfortheconversion.The"-o:"switchisfollowedbythenameoftheselectedoutputformat,asinthefollowingexample:

C:\>LogParser-c-i:IISW3C-o:IISextend1.loginetsv1.log

DifferentlythanQueryExecutionMode,theoutputformatspecificationisamandatoryargumentforcommandsinconversionmode.Thespecifiedoutputformatnamemustbeoneoftheoutputformatsinthetableaboveforwhichaconversionissupported.

<from_entity>

Specifiestheinputfile(s)tobeconverted.Thisargumentmustconformtothe<from_entity>syntaxoftheselectedinputformat.Forinformationonthesyntaxandinterpretationofthe<from_entity>valuessupportedbyeachinputformat,refertotheInputFormatsReference.Iftheargumentcontainsspaces,itmustbeenclosedwithindouble-quotecharacters("),asinthefollowingexample:

C:\>LogParser-c-i:IISW3C-o:IIS"extend1.log;,<1>"inetsv1.log

<into_entity>

Specifiestheconversiontargetoutputfile.Thisargumentmustconformtothe<into_entity>syntaxoftheselectedoutputformat.Forinformationonthesyntaxandinterpretationofthe<into_entity>valuessupportedbyeachoutputformat,refertotheOutputFormatsReference.Iftheargumentcontainsspaces,itmustbeenclosedwithindouble-quotecharacters("),asinthefollowingexample:

C:\>LogParser-c-i:IISW3C-o:IISextend1.log"C:\MyFolder\inetsv1.log"

<where_clause>

SpecifiesanoptionalWHEREclausetoperformfilteringontheinputformatentries.

ThefollowingexampleconvertsonlytheIISW3Clogfileentriesthatrepresentsuccessfulrequests:

C:\>LogParser-c-i:IISW3C-o:IISextend1.loginetsv1.log"WHEREsc-statusBETWEEN200AND399"

<input_format_options>

Specifyvaluesforinputformatparameters.Theseareenteredasswitcheswithnamesmatchingtheinputformat'sparameternames,followedbyacolonandbythevaluefortheparameter,asinthefollowingexample:

C:\>LogParser-c-i:IISW3C-o:IISextend1.loginetsv1.log-iCodepage:932Formoreinformationoninputformatparameters,refertotheInputFormatReference.

<output_format_options>

Specifyvaluesforoutputformatparameters.Theseareenteredasswitcheswithnamesmatchingtheoutputformat'sparameternames,followedbyacolonandbythevaluefortheparameter,asinthefollowingexample:

C:\>LogParser-c-i:IISW3C-o:IISextend1.loginetsv1.log-fileMode:1

Formoreinformationonoutputformatparameters,refertotheOutputFormatReference.

-multiSite[:ON|OFF]

SpecifiesthatanIISCentralBinarylogfileistobeconvertedtomultiplelogfiles,oneforeachIISVirtualSite.ThisoptionisonlyavailablewhentheconversionisfromtheBINinputformat,andwhenthespecified<into-entity>containsone"*"wildcardenablingtheMultiplexOuputMode.ThewildcardwillbereplacedwiththenumericidentifiersoftheIISVirtualSitesthatservedtherequestsloggedinthecentralbinarylogfile.

ThefollowingexampleconvertsasingleIISCentralBinarylogfiletodifferentW3Clogfiles,oneforeachIISVirtualSitethatservedarequestloggedinthecentralbinarylog:

C:\>LogParser-c-i:BIN-o:W3Craw1.iblC:\NewLogs\W3SVC*\extend1.log-multiSite:ON

<global_switches>

Globalswitchescontroloverallbehaviorsofthecommand,suchaserrorhandlingandcommandstatisticsverbosity.Formoreinformationonglobalswitches,refertotheGlobalSwitchesReference.

-queryInfo

Displaysdiagnosticinformationabouttheconversioncommand.When"-queryInfo"isspecified,thecommandisnotexecuted,andthefollowingdiagnosticinformationisdisplayedtotheconsolewindow:Thetextoftheconversionquery,afterbeingparsedandinterpretedbytheLogParserSQL-Likeenginecore;Namesoftheinputandoutputformatsselected;Structureofthequeryoutputrecords,includingfieldnamesandfielddatatypes.

Thisinformationcanbeusedtotroubleshootunexpectedconversion

results.

Thefollowingexampleusesthe"-queryInfo"switchtodisplaydiagnosticinformationaboutthespecifiedconversioncommand:

C:\>LogParser-c-i:IISW3C-o:IISextend1.loginetsv1.log-queryInfo

Theoutputofthiscommandis:

Query:SELECT[c-ip],[cs-username],TO_DATE(TO_LOCALTIME(TO_TIMESTAMP([date],[time]))),TO_TIME(TO_LOCALTIME(TO_TIMESTAMP([date],[time]))),[s-sitename],[s-computername],[s-ip],[time-taken],[sc-bytes],[cs-bytes],[sc-status],[sc-win32-status],[cs-method],[cs-uri-stem],[cs-uri-query]INTOinetsv1.logFROMextend1.log

Formatsselected:Inputformat:IISW3C(IISW3CExtendedLogFormat)Outputformat:IIS(IISLogFormat)

Queryfields:c-ip(S)cs-username(S)TO_DATE(TO_LOCALTIME(TO_TIMESTAMP(date,time)))(T)TO_TIME(TO_LOCALTIME(TO_TIMESTAMP(date,time)))(T)s-sitename(S)s-computername(S)s-ip(S)time-taken(I)sc-bytes(I)cs-bytes(I)sc-status(I)sc-win32-status(I)

Seealso:Command-LineOperationReferenceGlobalSwitchesReferenceConvertingFileFormats

cs-method(S)cs-uri-stem(S)cs-uri-query(S)

DefaultsOverrideModeIn"DefaultsOverrideMode"userscanspecifynewdefaultvaluestoreplacethefactorydefaultvaluesofglobalswitches,inputformatparameters,andoutputformatparameters.Valuesareoverriddenonthecomputeronwhichthe"saveDefaults"commandisexecuted,andthenewvaluesareineffectuntiltheyareoverriddenbyanewoverridecommand,oruntilthefactorydefaultsarerestoredwiththe"restoreDefaults"command.ThenewdefaultvaluesalsoaffecttheLogParserscriptableCOMcomponents.

Note:Forsecurityreasons,propertiesthatareusedtospecifyconfidentialorsensitiveinformation,suchasusernamesandpasswords,cannotbeoverridenbythe"DefaultsOverrideMode"feature.

Thegeneralsyntaxofcommandsindefaultsoverridemodeis:

LogParser -saveDefaults[-i:<input_format><input_format_options>][-o:<output_format><output_format_options>][<global_switches>]

LogParser -restoreDefaults

-i:<input_format><input_format_options>

Specifiestheinputformatwhoseparameters'defaultvaluesaretobeoverridden,andthenewdefaultvaluesfortheselectedparameters.The"-i:"switchisfollowedbythenameoftheselectedinputformat,andthenewdefaultvaluesareenteredasswitcheswithnamesmatchingtheinputformat'sparameternames,followedbyacolonandbythevalueforthenewdefault,asinthefollowingexample:

C:\>LogParser-saveDefaults-i:EVT-binaryFormat:ASC-resolveSIDs:ONFormoreinformationoninputformatparameters,refertotheInput

FormatReference.

-o:<output_format><output_format_options>

Specifiestheoutputformatwhoseparameters'defaultvaluesaretobeoverridden,andthenewdefaultvaluesfortheselectedparameters.The"-o:"switchisfollowedbythenameoftheselectedoutputformat,andthenewdefaultvaluesareenteredasswitcheswithnamesmatchingtheoutputformat'sparameternames,followedbyacolonandbythevalueforthenewdefault,asinthefollowingexample:

C:\>LogParser-saveDefaults-o:NAT-rtp:-1

Formoreinformationonoutputformatparameters,refertotheOutputFormatReference.

<global_switches>

Specifynewdefaultvaluesforglobalswitches.

Thefollowingexamplecommandoverridesthedefaultvalueofthe"-stats;"globalswitch,togetherwiththe"rtp"parameteroftheNAToutputformat:

C:\>LogParser-saveDefaults-o:NAT-rtp:-1-stats:OFF

Formoreinformationonglobalswitches,refertotheGlobalSwitchesReference.

-restoreDefaults

Restoresthefactorydefaultsofglobalswitches,inputformatparameters,andoutputformatparameters.Whenspecified,the"-restoreDefaults"switchmustbetheonly

argumentofthecommand,asinthefollowingexample:

C:\>LogParser-restoreDefaults

Seealso:Command-LineOperationReferenceGlobalSwitchesReference

HelpMode"HelpMode",activatedwiththe"-h"switch,offersusersthepossibilitytoaccess"quickreference"helptopicsdisplayedtotheconsoleoutput.Thehelptopics,selectablethroughadditionalcommand-linearguments,are:

GeneralUsageQueryLanguageSyntaxFunctionsSyntaxInputandOutputFormatsConversionModeQueryExamples

GeneralUsageHelp

TheLogParsercommand-lineexecutableusagehelpisaccessedwiththefollowingcommand:

C:\>LogParser-h

QueryLanguageSyntaxHelp

TheLogParserSQL-Likelanguagesyntaxhelpisaccessedwiththefollowingcommand:

C:\>LogParser-hGRAMMAR

FunctionsSyntaxHelp

TheLogParserSQL-Likelanguagefunctionssyntaxhelpisaccessed

withcommandshavingthefollowingsyntax:

LogParser -hFUNC[TIONS][<function>]

TypingthefollowingcommandwilldisplaythesyntaxforallthefunctionsavailableintheLogParserSQL-Likelanguage:

C:\>LogParser-hFUNCTIONS

Typingafunctionnamefollowingthehelpcommanddisplaysthesyntaxoftheselectedfunctiononly:

C:\>LogParser-hFUNCTIONSSUBSTR

Typingthefirstfewlettersofafunctionnamedisplaysthesyntaxofallthefunctionswhosenamestartswiththespecifiedletters:

C:\>LogParser-hFUNCTIONSSTR

InputandOutputFormatsHelp

Inputandoutputformatshelpisdisplayedwithcommandshavingthefollowingsyntax:

LogParser -h-i:<input_format>[<from_entity>][<input_format_options>]

LogParser -h-o:<output_format>

Forexample,thefollowingcommanddisplayshelpontheIISW3Cinputformat:

C:\>LogParser-h-i:IISW3C

TheoutputofthiscommandgivesadetailedoverviewoftheIISW3C

inputformat,includingthesyntaxofthe

<from_entity>,alistofallthesupportedpropertiestogetherwiththeirdefaultvalues,thestructureoftherecordsproducedbytheinputformat(fieldnamesandtypes),andexamplesofqueriesusingtheinputformat.

Whenaninputformatretrievesfieldinformationfromthedatathatneedstobeparsed,thehelpcommandcanincludethefrom-entityfromwhichthefieldinformationistobegathered.Forexample,theCSVinputformatexaminestheinputfilestoretrievethenamesandtypesoftheinputrecordfieldsthatwillbeexported.AhelpcommandaimedatdisplayingtheinputrecordfieldsexportedbytheCSVinputformatwhenparsingaspecificfileshouldincludethefilenamefrom-entity,asshowninthefollowingexample:

C:\>LogParser-h-i:CSVTestLogFile.csv

Inaddition,sincetheparametersofsomeinputformatscanaffectthestructureoftheinputrecords,helpcommandscanincludetheseparameterstodisplaythevaryinginputrecordstructures.Forexample,theNETMONinputformathasa"fMode"parameterthatcanbeusedtospecifyhowtheinputrecordsshouldbestructured.AhelpcommandaimedatdisplayingtheinputrecordfieldsexportedbytheNETMONinputformatwhenthe"fMode"parameterissetto"TCPConn"shouldincludethisparameter,asshowninthefollowingexample:

C:\>LogParser-h-i:NETMON-fMode:TCPConn

ConversionModeHelp

Conversionmodehelpisaccessedwithcommandshavingthefollowingsyntax:

LogParser -h-c[-i:<input_format>-o:<output_format>]

Thefollowingcommanddisplaysgeneralconversionmodehelp,

includingthelistofavailablebuilt-inconversionqueries:

C:\>LogParser-h-c

Thefollowingcommanddisplayshelpontheconversionbetweenthespecifiedlogfileformats,includingthefulltextofthebuilt-inquerythatperformstheconversion:

C:\>LogParser-h-c-i:BIN-o:W3C

QueryExamplesHelp

Examplesofqueriesandcommandscanbedisplayedwiththefollowingcommand:

C:\>LogParser-hEXAMPLES

Seealso:

Command-LineOperationReference

GlobalSwitchesGlobalswitchescontroloverallbehaviorsofacommand,andtheyareusedwithmostoftheLogParsercommand-lineexecutableoperationalmodes.

Theglobalswitchesare:

-e:<max_errors>

-iw[:ON|OFF]

-stats[:ON|OFF]

-q[:ON|OFF]

-e:<max_errors>

Specifiesamaximumnumberofparseerrorstocollectinternallybeforeabortingtheexecutionofthecommand.Thedefaultvalueforthisglobalswitchis-1,whichisaspecialvaluecausingtheSQLenginetoignoreallparseerrorsandreportonlythetotalnumberofparseerrorsencounteredduringtheexecutionofthecommand.Thefollowingexamplecommandsetsthemaximumnumberofparseerrorsto100:

C:\>LogParser"SELECTMessageFROMSystem"-e:100

Formoreinformationonparseerrorsandthe"-e"switch,seeErrors,ParseErrors,andWarnings.

-iw[:ON|OFF]

Specifieswhetherornotwarningsshouldbeignored.

Thedefaultvalueis"OFF",meaningthatruntimewarningswillnotbeignoredandwilltriggeraninteractiveprompttotheuser.Specifying"ON",ontheotherhand,disablestheinteractiveprompt,andruntimewarningswillbeignoredandtheirtotalcountwillbereportedwhenthecommandexecutionhascompleted.Thefollowingexamplecommandexecutesaqueryignoringruntimewarnings:

C:\>LogParser"SELECTMessageFROMSystem"-iw:ON

Formoreinformationonwarningsandthe"-iw"switch,seeErrors,ParseErrors,andWarnings.

-stats[:ON|OFF]

Specifieswhetherornotcommandexecutionstatisticsshouldbedisplayedwhenthecommandexecutionhascompleted.Thedefaultvalueis"ON",causingcommandexecutionstatisticstobealwaysdisplayed.Specifying"OFF"preventsthestatisticsfrombeingdisplayed.Thefollowingexamplecommandexecutesaquerypreventingthestatisticsfrombeingdisplayed:

C:\>LogParser"SELECTCOUNT(*)FROMSystem"-stats:OFF

-q[:ON|OFF]

Enablesordisables"quietmode".When"quietmode"isenabled,theconsoleoutputofacommandcontainsonlytheoutputrecords,suppressinganyadditionalinformation.Forthisreason,theconsoleoutputofacommandexecutedin"quietmode"issuitabletoberedirectedtoatextfile.Enabling"quietmode"disablesthedisplayofparseerrors,warnings,andstatistics.Inaddition,iftheselectedoutputformatistheNAToutputformat,its"rtp"and"headers"parametersareautomaticallysetasfollows:

-rtp:-1-headers:OFF

Asanexample,theoutputoffollowingcommandshowstheextrainformationandtheNAToutputformatheadersthatarenormallydisplayedtotheconsole:

C:\>LogParser"SELECTCOUNT(*)FROMSystem"COUNT(ALL*)------------6913

Inthisexample,enabling"quietmode"suppressestheheadersdisplayedbytheNAToutputformatandthequeryexecutionstatistics,andtheoutputwouldlooklikethefollowing:

C:\>LogParser"SELECTCOUNT(*)FROMSystem"-q:ON6913

Seealso:Command-LineOperationReferenceErrors,ParseErrors,andWarnings

COMAPITheLogParserscriptableCOMcomponentsarchitectureismadeupofthefollowingobjects:

LogQueryobject:thisobjectisthemainCOMobjectintheLogParserscriptableCOMcomponentsarchitecture;itexposesmethodstoexecuteSQL-Likequeriesandprovidesaccesstoglobalparameterscontrollingtheexecutionofaquery.LogRecordSetobject:thisobjectisanenumeratorofLogRecordobjects;itallowsanapplicationtonavigatethroughtheoutputrecordsofaquery.LogRecordobject:thisobjectrepresentsasinglequeryoutputrecord,anditexposesmethodsthatcanbeusedtoretrieveindividualfieldvaluesfromtheoutputrecord.InputFormatobjects:theseobjectsprovideprogrammaticaccesstotheinputformatssupportedbyLogParser;eachinputformatobjectexposespropertieshavingthesamenameastheparametersofthecorrespondingLogParserinputformat.OutputFormatobjects:theseobjectsprovideprogrammaticaccesstotheoutputformatssupportedbyLogParser;eachoutputformatobjectexposespropertieshavingthesamenameastheparametersofthecorrespondingLogParseroutputformat.

Seealso:LogParserCOMAPIOverviewC#Example

LogQueryObjectTheLogQueryobjectexposesthemainAPImethodsthatexecuteaSQL-Likequeryandprovidesaccesstoglobalparameterscontrollingtheexecutionofaquery.

Theobjectisinstantiatedwiththe"MSUtil.LogQuery"ProgId.Theclassnameofthe.NETCOMwrapperforthisobjectis"Interop.MSUtil.LogQueryClassClass".

Methods

Execute ExecutesaqueryandreturnsaLogRecordSetobjectthatcanbeusedtonavigatethroughthequeryoutputrecords.

ExecuteBatch Executesaqueryandwritesthequeryoutputrecordstoanoutputformat.

Properties

errorMessages Returnsacollectionoftheerror,parseerror,andwarningmessagesthatoccurredduringtheexecutionofaquery.

inputUnitsProcessed Returnsthetotalnumberofinputrecordsprocessedduringtheexecutionofaquery.

lastError Returns-1iferrors,parseerrors,orwarningsoccurredduringtheexecution

ofthequery;0otherwise.

maxParseErrors Setsandgetsthemaximumnumberofparseerrorsthatcanoccurduringtheexecutionofaquerybeforeabortingthequeryexecution.

outputUnitsProcessed Returnsthetotalnumberofoutputrecordssenttoanoutputformatduringtheexecutionofaquery.

versionMaj Returnsthe"major"componentoftheversionoftheLogParserscriptableCOMcomponents.

versionMin Returnsthe"minor"componentoftheversionoftheLogParserscriptableCOMcomponents.

Examples

JScriptexample:

VBScriptexample:

DimoLogQuerySetoLogQuery=CreateObject("MSUtil.LogQuery")

Seealso:

LogRecordSetObjectInputFormatObjectsOutputFormatObjectsLogParserCOMAPIOverviewC#Example

ExecuteMethodExecutesaqueryandreturnsaLogRecordSetobjectthatcanbeusedtonavigatethroughthequeryoutputrecords.

ScriptSyntax

objRecordSet=objLogQuery.Execute(strQuery[,objInputFormat]);

Parameters

strQueryAstringcontainingthetextoftheSQL-Likequerytobeexecuted.

objInputFormatEitheranInputFormatobjectoraCustomInputFormatPluginobject.Ifthisparameterisnotspecified,orisnull,LogParserwillattempttoselectautomaticallyaninputformatuponinspectionofthe<from-entity>intheFROMclauseofthespecifiedquery.

ReturnValueALogRecordSetobject,whichcanbeusedtonavigatethroughthequeryoutputrecords.

RemarksIfthequeryexecutionencounterserrors,anexceptionisthrowncontainingtheerrormessageandcode,andthequeryexecutionisaborted.Inthiscase,thelastErrorpropertyoftheLogQueryobjectissetto-1,

andthecollectionofstringsreturnedbytheerrorMessagespropertycontainstheerrormessage.Ifthequeryexecutionencountersparseerrorsorwarnings,thequeryexecutessuccessfully,andthemethodreturnsaLogRecordSetobject.Inthiscase,thelastErrorpropertyoftheLogQueryobjectissetto-1,andthecollectionofstringsreturnedbytheerrorMessagespropertycontainstheparseerrormessagesand/orwarningmessages.AsuccessfulexecutionoftheExecutemethoddoesnotnecessarilymeanthatthequeryexecutionhascompleted.Dependingonthequerystructure,navigatingthequeryoutputrecordswiththeLogRecordSetobjectcancausethequerytofurtherprocessnewinputrecords,whichcouldinturngenerateadditionalerrors,parseerrors,orwarnings.SeetheLogRecordSetObjectReferenceformoreinformation.ThespecifiedquerycannotcontainanINTOclause.

Examples

JScriptexample:

VBScriptexample:

'CreateInputFormatobjectSetoIISW3CInputFormat=CreateObject("MSUtil.LogQuery.IISW3CInp

Seealso:LogQueryObjectExecuteBatchMethodLogRecordSetObjectInputFormatObjectsLogParserCOMAPIOverview

utFormat")

C#Example

ExecuteBatchMethodExecutesaqueryandwritestheoutputrecordstoanoutputformat.

ScriptSyntax

bResult=objLogQuery.ExecuteBatch(strQuery[,objInputFormat[,objOutputFormat]]);

Parameters

strQueryAstringcontainingthetextoftheSQL-Likequerytobeexecuted.

objInputFormatEitheranInputFormatobjectoraCustomInputFormatPluginobject.Ifthisparameterisnotspecified,orisnull,LogParserwillattempttoselectautomaticallyaninputformatuponinspectionofthe<from-entity>intheFROMclauseofthespecifiedquery.

objOutputFormatAnOutputFormatobject.Ifthisparameterisnotspecified,orisnull,LogParserwillattempttoselectautomaticallyanoutputformatuponinspectionofthe<into-entity>intheINTOclauseofthespecifiedquery.

ReturnValueAbooleanvalue.ReturnsTRUEifthequeryexecutedwithparseerrorsorwarnings;FALSEifthequeryexecutedwithoutanyparseerrornorwarning.

RemarksIfthequeryexecutionencounterserrors,anexceptionisthrowncontainingtheerrormessageandcode,andthequeryexecutionisaborted.Inthiscase,thelastErrorpropertyoftheLogQueryobjectissetto-1,andthecollectionofstringsreturnedbytheerrorMessagespropertycontainstheerrormessage.Ifthequeryexecutionencountersparseerrorsorwarnings,thequeryexecutessuccessfully,andthemethodreturnsTRUE.Inthiscase,thelastErrorpropertyoftheLogQueryobjectissetto-1,andthecollectionofstringsreturnedbytheerrorMessagespropertycontainstheparseerrormessagesand/orwarningmessages.

Examples

JScriptexample:

//CreateInputFormatobjectvaroEVTInputFormat=newActiveXObject("MSUtil.LogQuery.EventLogInputFormat");oEVTInputFormat.direction="BW";

//CreateOutputFormatobjectvaroCSVOutputFormat=newActiveXObject("MSUtil.LogQuery.CSVOutputFormat");oCSVOutputFormat.tabs=true;

//Executequery

VBScriptexample:

DimoLogQueryDimoEVTInputFormatDimoCSVOutputFormatDimstrQuery

'CreateInputFormatobjectSetoEVTInputFormat=CreateObject("MSUtil.LogQuery.EventLogInputFormat")oEVTInputFormat.direction="BW"

'CreateOutputFormatobject

Seealso:LogQueryObjectExecuteMethodInputFormatObjectsOutputFormatObjectsLogParserCOMAPIOverviewC#Example

oLogQuery.ExecuteBatch(strQuery,oEVTInputFormat,oCSVOutputFormat);SetoCSVOutputFormat=CreateObject("MSUtil.LogQuery.CSVOutputFormat")oCSVOutputFormat.tabs=TRUE

'ExecutequeryoLogQuery.ExecuteBatchstrQuery,oEVTInputFormat,oCSVOutputFormat

errorMessagesPropertyReturnsacollectionofstringscontainingthemessagesoferrors,parseerrors,orwarningsencounteredwhileexecutingaquerywiththeExecuteorExecuteBatchmethods.

Read-onlyproperty.

ScriptSyntax

value=objLogQuery.errorMessages;

ReturnValueAcollectionofStringscontainingerrormessages.

RemarksTheobjectreturnedbytheerrorMessagespropertyimplementsasingleread-only_NewEnumproperty.The_NewEnumpropertyretrievesanIEnumVARIANTinterfaceonanobjectthatcanbeusedtoenumeratethecollection.The_NewEnumpropertyishiddenwithinscriptinglanguages(JScriptandVBScript).ApplicationswrittenintheJScriptlanguagehandleobjectsimplementingthe_NewEnumpropertyasEnumeratorobjectsorwiththefor...instatement,whileapplicationswrittenintheVBScriptlanguagehandleobjectsimplementingthe_NewEnumpropertywiththeForEach...Nextstatement.Ifyouwanttoretrieveparseerrormessages,makesurethatthemaxParseErrorspropertyoftheLogQueryobjectissettoavaluedifferentthan-1.Ifthevalueofthispropertyis-1(thedefaultvalue),theparseerrormessageswillbediscarded,andtheerrorMessagescollectionwillcontainasinglemessagestatingthetotalnumberofparseerrorsoccurred.

Examples

JScriptexample:

//MakesurethatparseerrormessagesarecollectedoLogQuery.maxParseErrors=100;

//CreatequerytextvarstrQuery="SELECTsc-bytesINTOC:\\output.csvFROMex040528.log";

//ExecutequeryoLogQuery.ExecuteBatch(strQuery);

//Checkiferrorsoccurredif(oLogQuery.lastError!=0){WScript.Echo("Errorsoccurred!");

varoMessages=newEnumerator(oLogQuery.errorMessages);for(;!oMessages.atEnd();oMessages.moveNext()){WScript.Echo("Errormessage:"+oMessages.item());}}else{WScript.Echo("Executedsuccessfully!");}

VBScriptexample:

DimoLogQueryDimstrQuery

'MakesurethatparseerrormessagesarecollectedoLogQuery.maxParseErrors=100

'CreatequerytextstrQuery="SELECTsc-bytesINTOC:\output.csvFROMex040528.log"

'ExecutequeryoLogQuery.ExecuteBatchstrQuery

'CheckiferrorsoccurredIfoLogQuery.lastError<>0Then

WScript.Echo"Errorsoccurred!"

ForEachstrMessageInoLogQuery.errorMessagesWScript.Echo"ErrorMessage:"+strMessageNext

WScript.Echo"Executedsuccesfully!"

Seealso:LogQueryObjectLogParserCOMAPIOverviewC#Example

inputUnitsProcessedPropertyReturnsthetotalnumberofinputrecordsprocessedbyaqueryexecutedwiththeExecuteBatchmethod.

Read-onlyproperty.

ScriptSyntax

value=objLogQuery.inputUnitsProcessed;

ReturnValueAnintegervaluecontainingthetotalnumberofinputrecordsprocessedbythelastqueryexecutedwiththeExecuteBatchmethod.

RemarksWhenaqueryisexecutedwiththeExecutemethod,thispropertyreturnszero.Inthesecases,usetheinputUnitsProcessedpropertyoftheLogRecordSetobject.

Examples

JScriptexample:

//CreatequerytextvarstrQuery="SELECTTimeGenerated,EventIDINTOC:\\output.csvFROMSystem";

VBScriptexample:

DimoLogQuery

strQuery+="WHERESourceName='ApplicationPopup'";

//DisplaytotalnumberofinputrecordsprocessedWScript.Echo("InputRecordsProcessed:"+oLogQuery.inputUnitsProcessed);

DimstrQuery

'DisplaytotalnumberofinputrecordsprocessedWScript.Echo"InputRecordsProcessed:"&oLogQuery.inputUnitsProcessed

Seealso:LogQueryObjectExecuteBatchMethodoutputUnitsProcessedPropertyLogParserCOMAPIOverviewC#Example

lastErrorPropertyReturns-1iftheExecuteorExecuteBatchmethodsencounterederrors,parseerrors,orwarnings;0otherwise.

Read-onlyproperty.

ScriptSyntax

value=objLogQuery.lastError;

ReturnValueAnintegervaluecontaining-1iftheExecuteorExecuteBatchmethodsencounterederrors,parseerrors,orwarnings;0otherwise.

Examples

JScriptexample:

VBScriptexample:

Seealso:LogQueryObjectLogParserCOMAPIOverviewC#Example

}else{WScript.Echo("Executedsuccessfully!");}

'CheckiferrorsoccurredIfoLogQuery.lastError<>0ThenWScript.Echo"Errorsoccurred!"ElseWScript.Echo"Executedsuccesfully!"EndIf

maxParseErrorsPropertySetsorgetsthemaximumnumberofparseerrorsthatcanoccurduringtheexecutionofaquerybeforeabortingthequeryexecution.

Read/writeproperty.

ScriptSyntax

objLogQuery.maxParseErrors=value;

value=objLogQuery.maxParseErrors;

Argument/ReturnValueAnintegervaluespecifyingthemaximumnumberofparseerrorsthatcanoccurduringtheexecutionofaquerybeforeabortingthequeryexecution.Avalueof-1specifiesthatallparseerrorsshouldbeignored.

DefaultValue-1

RemarksThispropertyisanalogoustothe"-e"globalswitchavailablewiththeLogParsercommand-lineexecutable.

Examples

JScriptexample:

oLogQuery.maxParseErrors=10;VBScriptexample:

oLogQuery.maxParseErrors=10Seealso:LogQueryObjectLogParserCOMAPIOverviewC#Example

outputUnitsProcessedPropertyReturnsthetotalnumberofoutputrecordssenttoanoutputformatbyaqueryexecutedwiththeExecuteBatchmethod.

Read-onlyproperty.

ScriptSyntax

value=objLogQuery.outputUnitsProcessed;

ReturnValueAnintegervaluecontainingthetotalnumberofoutputrecordssenttoanoutputformatbythelastqueryexecutedwiththeExecuteBatchmethod.

Examples

JScriptexample:

//DisplaytotalnumberofoutputrecordsgeneratedWScript.Echo("OutputRecordsWritten:"+oLogQuery.outputUnitsProc

VBScriptexample:

'CreatequerytextstrQuery="SELECTTimeGenerated,EventIDINTOC:\output.csvFROMSystem"

Seealso:LogQueryObjectExecuteBatchMethodinputUnitsProcessedProperty

essed);strQuery=strQuery&"WHERESourceName='ApplicationPopup'"

'DisplaytotalnumberofoutputrecordsgeneratedWScript.Echo"OutputRecordsWritten:"&oLogQuery.outputUnitsProcessed

LogParserCOMAPIOverviewC#Example

versionMajPropertyversionMinPropertyReturnthemajorandminorcomponentsoftheversionoftheLogParserscriptableCOMcomponentscurrentlybeingused.

Read-onlyproperties.

ScriptSyntax

value=objLogQuery.versionMaj;

value=objLogQuery.versionMin;

ReturnValuesIntegervaluescontainingthemajorandminorcomponentsoftheversionoftheLogParserscriptableCOMcomponentscurrentlybeingused.

Examples

JScriptexample:

WScript.Echo("LogParserVersion"+oLogQuery.versionMaj+"."+oLogQuery.versionMin);VBScriptexample:

WScript.Echo"LogParserVersion"&oLogQuery.versionMaj&"."&o

LogQuery.versionMinSeealso:LogQueryObjectLogParserCOMAPIOverviewC#Example

LogRecordSetObjectTheLogRecordSetobjectisreturnedbytheExecutemethodoftheLogQueryobject,anditexposesmethodsthatcanbeusedtonavigatethroughtheoutputrecordsofaquery.TheLogRecordSetobjectisanenumeratorofLogRecordobjects.

Theinterfacenameofthe.NETCOMwrapperforthisobjectis"Interop.MSUtil.ILogRecordset".

Methods

atEnd ReturnsaBooleanvalueindicatingiftheenumeratorisattheendofthecollection.

close Releasestheenumerationandalltheassociatedresources.

getColumnCount Returnsthenumberoffieldsinthequeryoutputrecords.

getColumnName Returnsthenameofafieldinthequeryoutputrecords.

getColumnType Returnsthedatatypeofafieldinthequeryoutputrecords.

getRecord ReturnsthecurrentLogRecordobjectintheenumeration.

moveNext AdvancestheenumeratortothenextLogRecordintheenumeration.

Properties

errorMessages Returnsacollectionoftheerror,parseerror,andwarningmessagesthatoccurredduringthelastinvocationofthemoveNextmethod.

inputUnitsProcessed Returnsthetotalnumberofinputrecordsprocessedduringtheexecutionofaquery.

lastError Returns-1iferrors,parseerrors,orwarningsoccurredduringthelastinvocationofthemoveNextmethod;0otherwise.

INTEGER_TYPE ReturnsthevalueoftheconstantrepresentingtheINTEGERdatatype.

NULL_TYPE ReturnsthevalueoftheconstantrepresentingtheNULLdatatype.

REAL_TYPE ReturnsthevalueoftheconstantrepresentingtheREALdatatype.

STRING_TYPE ReturnsthevalueoftheconstantrepresentingtheSTRINGdatatype.

TIMESTAMP_TYPE ReturnsthevalueoftheconstantrepresentingtheTIMESTAMPdatatype.

Examples

JScriptexample:

varoLogQuery=newActiveXObject("MSUtil.LogQuery");varoLogRecordSet=oLogQuery.Execute("SELECT*FROMSystem");VBScriptexample:

DimoLogQueryDimoLogRecordSet

SetoLogQuery=CreateObject("MSUtil.LogQuery")SetoLogRecordSet=oLogQuery.Execute("SELECT*FROMSystem")Seealso:

LogQueryObjectLogRecordObjectLogParserCOMAPIOverviewC#Example

atEndMethodReturnsaBooleanvalueindicatingiftheenumeratorisattheendofthecollection.

ScriptSyntax

value=objRecordSet.atEnd();

ReturnValueABooleanvaluesettoTRUEiftherearenomoreLogRecordobjectstoenumerate;FALSEotherwise.

Examples

JScriptexample:

VBScriptexample:

'CreateInputFormatobjectSetoIISW3CInputFormat=CreateObject("MSUtil.LogQuery.IISW3CInp

Seealso:LogRecordSetObjectLogRecordObjectLogParserCOMAPIOverviewC#Example

utFormat")

closeMethodReleasestheenumerationandalltheassociatedresources.

ScriptSyntax

objRecordSet.close();

ReturnValueNone.

Examples

JScriptexample:

//Visitallrecordswhile(!oRecordSet.atEnd()){ //Getarecord

VBScriptexample:

'CreateInputFormatobjectSetoIISW3CInputFormat=CreateObject("MSUtil.LogQuery.IISW3CInputFormat")

varoRecord=oRecordSet.getRecord();

getColumnCountMethodReturnsthenumberoffieldsinthequeryoutputrecords.

ScriptSyntax

value=objRecordSet.getColumnCount();

ReturnValueAnintegervaluecontainingthenumberoffieldsinthequeryoutputrecords.

Examples

JScriptexample:

//CreatequerytextvarstrQuery="SELECT*FROMSystem";

//ExecutequeryandreceiveaLogRecordSetvaroRecordSet=oLogQuery.Execute(strQuery);

//Displayfieldnamesandtypesfor(varf=0;f<oRecordSet.getColumnCount();f++){//FieldNameWScript.Echo("FieldName:"+oRecordSet.getColumnName(f));

//Fieldtypeswitch(oRecordSet.getColumnType(f))

VBScriptexample:

DimoLogQueryDimoRecordSetDimf

'CreatequerytextstrQuery="SELECT*FROMSystem"

'ExecutequeryandreceiveaLogRecordSetSetoRecordSet=oLogQuery.Execute(strQuery)

Seealso:LogRecordSetObjectLogParserCOMAPIOverviewC#Example

{caseoRecordSet.INTEGER_TYPE:{WScript.Echo("FieldType:INTEGER");break;}

caseoRecordSet.REAL_TYPE:{WScript.Echo("FieldType:REAL");break;}

caseoRecordSet.STRING_TYPE:{WScript.Echo("FieldType:STRING");break;}

caseoRecordSet.TIMESTAMP_TYPE:{WScript.Echo("FieldType:TIMESTAMP");break;}

caseoRecordSet.NULL_TYPE:{WScript.Echo("FieldType:NULL");break;}}}

'DisplayfieldnamesandtypesForf=0TooRecordSet.getColumnCount()-1

'FieldNameWScript.Echo"FieldName:"&oRecordSet.getColumnName(f)

'FieldtypeSelectCaseoRecordSet.getColumnType(f)CaseoRecordSet.INTEGER_TYPEWScript.Echo"FieldType:INTEGER"CaseoRecordSet.REAL_TYPEWScript.Echo"FieldType:REAL"CaseoRecordSet.STRING_TYPEWScript.Echo"FieldType:STRING"CaseoRecordSet.TIMESTAMP_TYPEWScript.Echo"FieldType:TIMESTAMP"CaseoRecordSet.NULL_TYPEWScript.Echo"FieldType:NULL"

EndSelectNext

'CloseLogRecordSetoRecordSet.close()

getColumnNameMethodReturnsthenameofafieldinthequeryoutputrecords.

ScriptSyntax

value=objRecordSet.getColumnName(index);

Parameters

indexThe0-basedindexofthefieldinthequeryoutputrecords.TheindexmustbelessthanthenumberoffieldsreturnedbythegetColumnCountmethod.

ReturnValueAstringvaluecontainingthenameoftheoutputrecordfieldatthespecifiedposition.

Examples

JScriptexample:

VBScriptexample:

//Fieldtypeswitch(oRecordSet.getColumnType(f)){caseoRecordSet.INTEGER_TYPE:{WScript.Echo("FieldType:INTEGER");break;}

EndSelectNext

getColumnTypeMethodReturnsthetypeofafieldinthequeryoutputrecords.

ScriptSyntax

value=objRecordSet.getColumnType(index);

Parameters

indexThe0-basedindexofthefieldinthequeryoutputrecords.TheindexmustbelessthanthenumberoffieldsreturnedbythegetColumnCountmethod.

ReturnValueAnintegervaluecontainingthetypeoftheoutputrecordfieldatthespecifiedposition.ThisvalueisoneoftheconstantsreturnedbytheINTEGER_TYPE,REAL_TYPE,STRING_TYPE,TIMESTAMP_TYPE,andNULL_TYPEproperties.

Examples

JScriptexample:

//CreatequerytextvarstrQuery="SELECT*FROMSystem";VBScriptexample:

caseoRecordSet.NULL_TYPE:{WScript.Echo("FieldType:NULL");break;}

EndSelectNext

'CloseLogRecordSet

oRecordSet.close()

getRecordMethodReturnsthecurrentLogRecordobjectintheenumeration.

ScriptSyntax

objRecord=objRecordSet.getRecord();

ReturnValueThecurrentLogRecordobjectintheenumeration.

Examples

JScriptexample:

//Visitallrecordswhile(!oRecordSet.atEnd()){ //Getarecord

VBScriptexample:

varoRecord=oRecordSet.getRecord();

moveNextMethodAdvancestheenumeratortothenextLogRecordintheenumeration.

ScriptSyntax

objRecordSet.moveNext();

ReturnValueNone.

RemarksDependingonthequerystructure,callingthemoveNextmethodcancausethequerytofurtherprocessnewinputrecords,whichcouldinturngenerateadditionalerrors,parseerrors,orwarnings.IfthemoveNextmethodencounterserrors,anexceptionisthrowncontainingtheerrormessageandcode,andfurtherprocessingisaborted.Inthiscase,thelastErrorpropertyoftheLogRecordSetobjectissetto-1,andthecollectionofstringsreturnedbytheerrorMessagespropertycontainstheerrormessage.IfthemoveNextmethodencountersparseerrorsorwarnings,theenumeratorisadvancedsuccessfully,andthelastErrorpropertyoftheLogRecordSetobjectissetto-1.Inthiscase,thecollectionofstringsreturnedbytheerrorMessagespropertycontainstheparseerrormessagesand/orwarningmessages.

Examples

JScriptexample:

//Visitallrecordswhile(!oRecordSet.atEnd()){ //Getarecord varoRecord=oRecordSet.getRecord();

VBScriptexample:

'AdvanceLogRecordSettonextrecord

oRecordSet.moveNext

errorMessagesPropertyReturnsacollectionofstringscontainingthemessagesoferrors,parseerrors,orwarningsthatoccurredduringthelastinvocationofthemoveNextmethod.

Read-onlyproperty.

ScriptSyntax

value=objLogRecordSet.errorMessages;

ReturnValueAcollectionofStringscontainingerrormessages.

RemarksTheobjectreturnedbytheerrorMessagespropertyimplementsasingleread-only_NewEnumproperty.The_NewEnumpropertyretrievesanIEnumVARIANTinterfaceonanobjectthatcanbeusedtoenumeratethecollection.The_NewEnumpropertyishiddenwithinscriptinglanguages(JScriptandVBScript).ApplicationswrittenintheJScriptlanguagehandleobjectsimplementingthe_NewEnumpropertyasEnumeratorobjectsorwiththefor...instatement,whileapplicationswrittenintheVBScriptlanguagehandleobjectsimplementingthe_NewEnumpropertywiththeForEach...Nextstatement.Ifyouwanttoretrieveparseerrormessages,makesurethatthemaxParseErrorspropertyoftheLogQueryobjectissettoavaluedifferentthan-1.Ifthevalueofthispropertyis-1(thedefaultvalue),theparseerrormessageswillbediscarded,andtheerrorMessagescollectionwillcontainasinglemessagestatingthetotalnumberofparseerrorsoccurred.

Examples

JScriptexample:

varoMessages=newEnumerator(oLogQuery.errorMessages);for(;!oMessages.atEnd();oMessages.moveNext()){WScript.Echo("Errormessage:"+oMessages.item());}}

//Visitallrecordswhile(!oRecordSet.atEnd()){

VBScriptexample:

//Getarecord varoRecord=oRecordSet.getRecord();

//AdvanceLogRecordSettonextrecord oRecordSet.moveNext();

//Checkiferrorsoccurredif(oRecordSet.lastError!=0){WScript.Echo("Errorsoccurred!");

varoMessages=newEnumerator(oRecordSet.errorMessages);for(;!oMessages.atEnd();oMessages.moveNext()){WScript.Echo("Errormessage:"+oMessages.item());}}}

'CheckiferrorsoccurredIfoRecordSet.lastError<>0Then

ForEachstrMessageInoRecordSet.errorMessagesWScript.Echo"ErrorMessage:"+strMessageNext

EndIfLOOP

inputUnitsProcessedPropertyReturnsthetotalnumberofinputrecordsprocessedsofarbyaqueryexecutedwiththeExecutemethod.

Read-onlyproperty.

ScriptSyntax

value=objLogRecordSet.inputUnitsProcessed;

ReturnValueAnintegervaluecontainingthetotalnumberofinputrecordsprocessedsofarbythequerythatreturnedtheLogRecordSetobject.

Examples

JScriptexample:

//Visitallrecordswhile(!oRecordSet.atEnd()){//Displaynumberofinputrecordsprocessedsofar

VBScriptexample:

DimoLogQueryDimoRecordSetDimstrQuery

WScript.Echo("InputRecordsProcessed:"+oRecordSet.inputUnitsProcessed);

//Getarecord varoRecord=oRecordSet.getRecord();

//DisplaytotalnumberofinputrecordsprocessedWScript.Echo("TotalInputRecordsProcessed:"+oRecordSet.inputUnitsProcessed);

'DisplaynumberofinputrecordsprocessedsofarWScript.Echo"InputRecordsProcessed:"&oRecordSet.inputUnitsProcessed

'DisplaytotalnumberofinputrecordsprocessedWScript.Echo"TotalInputRecordsProcessed:"&oRecordSet.inputUnitsProcessed

lastErrorPropertyReturns-1iferrors,parseerrors,orwarningsoccurredduringthelastinvocationofthemoveNextmethod;0otherwise.

Read-onlyproperty.

ScriptSyntax

value=objRecordSet.lastError;

ReturnValueAnintegervaluecontaining-1ifthelastmoveNextmethodinvocationencounterederrors,parseerrors,orwarnings;0otherwise.

Examples

JScriptexample:

//ExecutequeryandreceiveaLogRecordSet

VBScriptexample:

varoRecordSet=oLogQuery.Execute(strQuery,oIISW3CInputFormat);

varoMessages=newEnumerator(oLogQuery.errorMessages);for(;!oMessages.atEnd();oMessages.moveNext()){WScript.Echo("Errormessage:"+oMessages.item());}}

//AdvanceLogRecordSettonextrecord oRecordSet.moveNext();

//Checkiferrorsoccurredif(oRecordSet.lastError!=0){WScript.Echo("Errorsoccurred!");

varoMessages=newEnumerator(oRecordSet.errorMessages);for(;!oMessages.atEnd();oMessages.moveNext()){

WScript.Echo("Errormessage:"+oMessages.item());}}}

'CheckiferrorsoccurredIfoRecordSet.lastError<>0Then

ForEachstrMessageInoRecordSet.errorMessagesWScript.Echo"ErrorMessage:"+strMessageNext

EndIfLOOP

INTEGER_TYPEPropertyTheconstantvaluereturnedbythegetColumnTypemethodtoindicatethatanoutputrecordfieldcontainsvaluesoftheINTEGERdatatype.

Read-onlyproperty.

ScriptSyntax

value=objRecordSet.INTEGER_TYPE;

ReturnValueAnintegervaluecontainingtheconstantthatrepresentstheINTEGERdatatype.

Examples

JScriptexample:

VBScriptexample:

Seealso:NULL_TYPEPropertyREAL_TYPEPropertySTRING_TYPEPropertyTIMESTAMP_TYPEProperty

EndSelectNext

LogRecordSetObjectLogParserCOMAPIOverviewC#Example

NULL_TYPEPropertyTheconstantvaluereturnedbythegetColumnTypemethodtoindicatethatanoutputrecordfieldcontainsvaluesoftheNULLdatatype.

Read-onlyproperty.

ScriptSyntax

value=objRecordSet.NULL_TYPE;

ReturnValueAnintegervaluecontainingtheconstantthatrepresentstheNULLdatatype.

Examples

JScriptexample:

VBScriptexample:

Seealso:INTEGER_TYPEPropertyREAL_TYPEPropertySTRING_TYPEPropertyTIMESTAMP_TYPEProperty

EndSelectNext

REAL_TYPEPropertyTheconstantvaluereturnedbythegetColumnTypemethodtoindicatethatanoutputrecordfieldcontainsvaluesoftheREALdatatype.

Read-onlyproperty.

ScriptSyntax

value=objRecordSet.REAL_TYPE;

ReturnValueAnintegervaluecontainingtheconstantthatrepresentstheREALdatatype.

Examples

JScriptexample:

VBScriptexample:

Seealso:INTEGER_TYPEPropertyNULL_TYPEPropertySTRING_TYPEPropertyTIMESTAMP_TYPEProperty

EndSelectNext

STRING_TYPEPropertyTheconstantvaluereturnedbythegetColumnTypemethodtoindicatethatanoutputrecordfieldcontainsvaluesoftheSTRINGdatatype.

Read-onlyproperty.

ScriptSyntax

value=objRecordSet.STRING_TYPE;

ReturnValueAnintegervaluecontainingtheconstantthatrepresentstheSTRINGdatatype.

Examples

JScriptexample:

VBScriptexample:

Seealso:INTEGER_TYPEPropertyNULL_TYPEPropertyREAL_TYPEPropertyTIMESTAMP_TYPEProperty

EndSelectNext

TIMESTAMP_TYPEPropertyTheconstantvaluereturnedbythegetColumnTypemethodtoindicatethatanoutputrecordfieldcontainsvaluesoftheTIMESTAMPdatatype.

Read-onlyproperty.

ScriptSyntax

value=objRecordSet.TIMESTAMP_TYPE;

ReturnValueAnintegervaluecontainingtheconstantthatrepresentstheTIMESTAMPdatatype.

Examples

JScriptexample:

VBScriptexample:

Seealso:INTEGER_TYPEPropertyNULL_TYPEPropertyREAL_TYPEPropertySTRING_TYPEProperty

EndSelectNext

LogRecordObjectTheLogRecordobjectrepresentsasinglequeryoutputrecord,anditexposesmethodsthatcanbeusedtoretrieveindividualfieldvaluesfromtheoutputrecord.TheLogRecordobjectisreturnedbythegetRecordmethodoftheLogRecordSetobject.

Theinterfacenameofthe.NETCOMwrapperforthisobjectis"Interop.MSUtil.ILogRecord".

Methods

getValue Returnsthevalueofafieldintheoutputrecord.

getValueEx Returnsthevalueofafieldintheoutputrecord.

isNull ReturnsaBooleanvalueindicatingifanoutputrecordfieldisNULL.

toNativeString Returnsafieldorthewholeoutputrecordasastringvalue.

Examples

JScriptexample:

VBScriptexample:

getValueMethodReturnsthevalueofthefieldatthespecifiedpositionintherecord.

ScriptSyntax

value=objRecord.getValue(index);

value=objRecord.getValue(fieldName);

Parameters

indexAnintegercontainingthe0-basedindexofthefieldinthequeryoutputrecords.TheindexmustbelessthanthenumberoffieldsreturnedbythegetColumnCountmethodoftheLogRecordSetobject.

fieldNameAstringcontainingthenameofthefieldinthequeryoutputrecords.

ReturnValueThevalueofthespecifiedfield.ThevalueisreturnedasaVARIANT(i.e.ascriptingvariable)whosetypedependsonthedatatypeofthefield.ThefollowingtableshowstheVARIANTtypereturnedandthecorrespondingscriptingtypesforeachoftheLogParserdatatypes:

FieldTypeVARIANTType JScriptType

VBScriptType

INTEGER VT_I4 number Long

REAL VT_R8 number Double

STRING VT_BSTR string String

TIMESTAMP VT_DATE date(VBdate)

NULL VT_NULL nullobject Null

RemarksSomescriptinglanguagesmightnothandlecorrectlythenullvaluereturnedbythegetValuemethodwhenthefieldatthespecifiedlocationisNULL.Inthesecases,calltheisNullmethodbeforethegetValuemethodtotestthefieldforNULLvalues.AlthoughtheLogParserINTEGERDataTypeisa64-bitvalue,thegetValuemethodreturnsINTEGERvaluesas32-bitintegers,sincescriptinglanguagesdonothandlecorrectly64-bitintegervalues.Thismeansthattruncationmightoccurwhenvaluesarelargerthanthemaximum32-bitvalue.Inthesecases,ifalow-levelprogramminglanguageisbeingused(e.g.C++),applicationscancallthegetValueExmethodtoretrieveINTEGERvaluesas64-bitvalues.

Examples

JScriptexample:

//CreatequerytextvarstrQuery="SELECTTimeGenerated,SourceName,EventID,MessageFROMSystem";

VBScriptexample:

DimoLogQuery

//Visitallrecordswhile(!oRecordSet.atEnd()){//GetarecordvaroRecord=oRecordSet.getRecord();

//DisplayrecordinformationWScript.Echo("TimeGenerated:"+oRecord.getValue("TimeGenerated"));WScript.Echo("SourceName:"+oRecord.getValue(1));WScript.Echo("EventID:"+oRecord.getValue(2));if(!oRecord.isNull(3)){WScript.Echo("Message:"+oRecord.getValue(3));}else{WScript.Echo("Message:<null>");}

//AdvanceLogRecordSettonextrecordoRecordSet.moveNext();}

DimoRecordSetDimstrQueryDimfDimval

'CreatequerytextstrQuery="SELECTTimeGenerated,SourceName,EventID,MessageFROMSystem"

'DisplayrecordinformationWScript.Echo"TimeGenerated:"&oRecord.getValue("TimeGenerated")WScript.Echo"SourceName:"&oRecord.getValue(1)WScript.Echo"EventID:"&oRecord.getValue(2)IfoRecord.isNull(3)=FalseThenWScript.Echo"Message:"&oRecord.getValue(3)ElseWScript.Echo"Message:<null>"EndIf

'CloseRecordSet

Seealso:LogRecordObjectLogParserCOMAPIOverviewC#Example

oRecordSet.close

getValueExMethodReturnsthevalueofthefieldatthespecifiedpositionintherecord.ThevaluereturnedbythegetValueExmethodisintendedforlow-levelprogramminglanguagesandisnotsuitableforconsumptionbyscriptinglanguages.

C++Syntax

HRESULTgetValueEx(INVARIANT*pindexOrName,OUTVARIANT*pVal);

Parameters

pindexOrNameAVT_I4orVT_BSTRVARIANTcontainingeitherthe0-basedindexofthefieldinthequeryoutputrecords,orthenameofthefieldinthequeryoutputrecords.TheindexmustbelessthanthenumberoffieldsreturnedbythegetColumnCountmethodoftheLogRecordSetobject.

ReturnValueThevalueofthespecifiedfield.ThevalueisreturnedasaVARIANTwhosetypedependsonthedatatypeofthefield.ThefollowingtableshowstheVARIANTtypereturnedforeachoftheLogParserdatatypes:

FieldTypeVARIANTType Description

INTEGER VT_I8 64-bitinteger

REAL VT_R8 64-bitfloating-pointnumber

STRING VT_BSTR String

TIMESTAMP VT_I8 64-bitintegerrepresentingthenumberof100-nanosecondintervalssinceJanuary1,year0

NULL VT_NULL VT_NULLVARIANT

RemarksThegetValueExmethodreturns64-bitintegervaluesthatarenothandledcorrectlybyscriptinglanguages,Forthisreason,themethodisintendedforusebylow-level,non-scriptinglanguages,suchasC++.Ifyouaredevelopinganapplicationusingscriptinglanguages,considerusingthegetValuemethodinstead.

Seealso:LogRecordObjectgetValueMethodLogParserCOMAPIOverviewC#Example

isNullMethodReturnsaBooleanvalueindicatingifanoutputrecordfieldisNULL.

ScriptSyntax

value=objRecord.isNull(index);

value=objRecord.isNull(fieldName);

Parameters

indexAnintegercontainingthe0-basedindexofthefieldinthequeryoutputrecords.TheindexmustbelessthanthenumberoffieldsreturnedbythegetColumnCountmethodoftheLogRecordSetobject.

fieldNameAstringcontainingthenameofthefieldinthequeryoutputrecords.

ReturnValueABooleanvalueindicatingifthespecifiedoutputrecordfieldisNULL.

Examples

JScriptexample:

//CreatequerytextVBScriptexample:

varstrQuery="SELECTTimeGenerated,SourceName,EventID,Message,DataFROMSystem";

//DisplayrecordinformationWScript.Echo("TimeGenerated:"+oRecord.getValue("TimeGenerated"));WScript.Echo("SourceName:"+oRecord.getValue(1));WScript.Echo("EventID:"+oRecord.getValue(2));if(!oRecord.isNull(3)){WScript.Echo("Message:"+oRecord.getValue(3));}else{WScript.Echo("Message:<null>");}

if(!oRecord.isNull("Data")){WScript.Echo("Data:"+oRecord.getValue(4));}else{WScript.Echo("Data:<null>");}

//AdvanceLogRecordSettonextrecordoRecordSet.moveNext();

DimoLogQueryDimoRecordSetDimstrQueryDimfDimval

'CreatequerytextstrQuery="SELECTTimeGenerated,SourceName,EventID,Message,DataFROMSystem"

'DisplayrecordinformationWScript.Echo"TimeGenerated:"&oRecord.getValue("TimeGenerated")WScript.Echo"SourceName:"&oRecord.getValue(1)WScript.Echo"EventID:"&oRecord.getValue(2)IfoRecord.isNull(3)=FalseThenWScript.Echo"Message:"&oRecord.getValue(3)ElseWScript.Echo"Message:<null>"EndIf

IfoRecord.isNull("Data")=FalseThenWScript.Echo"Data:"&oRecord.getValue(4)ElseWScript.Echo"Data:<null>"EndIf

toNativeStringMethodReturnsafieldorthewholeoutputrecordasastringvalue.

ScriptSyntax

value=objRecord.toNativeString(index);

value=objRecord.toNativeString(separator);

Parameters

indexAnintegercontainingthe0-basedindexofafieldinthequeryoutputrecords.TheindexmustbelessthanthenumberoffieldsreturnedbythegetColumnCountmethodoftheLogRecordSetobject.

separatorAstringcontainingtheseparatortobeusedbetweenthefieldsoftherecord.

ReturnValueIfafieldindexisusedasargument,themethodreturnsthespecifiedfieldformattedtoastringaccordingtotheinputformatstringrepresentationofthedatatype.Forexample,iftheinputformatusedparsestimestampsformattedas'yyyy-MM-ddhh:mm:ss',thenthemethodformatsTIMESTAMPvaluesusingthesameformat.Ifastringseparatorisusedasargument,themethodreturnstheconcatenationofalltherecordfieldsformattedtoastring,separatedbythespecifiedseparator.

Examples

JScriptexample:

//CreatequerytextvarstrQuery="SELECTTimeGenerated,SourceName,EventID,MessageFROMSystem";

//DisplayrecordinformationWScript.Echo("TimeGenerated:"+oRecord.toNativeString(0));WScript.Echo("WholeRecord:"+oRecord.toNativeString(","));

//AdvanceLogRecordSettonextrecordoRecordSet.moveNext();}

VBScriptexample:

DimoLogQueryDimoRecordSetDimstrQueryDimfDimval

'CreatequerytextstrQuery="SELECTTimeGenerated,SourceName,EventID,MessageFROMSystem"

'DisplayrecordinformationWScript.Echo"TimeGenerated:"&oRecord.toNativeString(0)WScript.Echo"WholeRecord:"&oRecord.toNativeString(",")

InputFormatObjectsInputFormatobjectsprovideprogrammaticaccesstotheinputformatssupportedbyLogParser.

InputFormatobjectsareinstantiatedwiththeProgIdandthe.NETCOMwrapperclassnamesspecifiedinthefollowingtable:

InputFormat ProgId .NETCOMWrapperClassName

ADS MSUtil.LogQuery.ADSInputFormat COMADSInputContextClassClass

BIN MSUtil.LogQuery.IISBINInputFormat COMIISBINInputContextClassClass

CSV MSUtil.LogQuery.CSVInputFormat COMCSVInputContextClassClass

ETW MSUtil.LogQuery.ETWInputFormat COMETWInputContextClassClass

EVT MSUtil.LogQuery.EventLogInputFormat COMEventLogInputContextClassClass

FS MSUtil.LogQuery.FileSystemInputFormat COMFileSystemInputContextClassClass

HTTPERR MSUtil.LogQuery.HttpErrorInputFormat COMHttpErrorInputContextClassClass

IIS MSUtil.LogQuery.IISIISInputFormat COMIISIISInputContextClassClass

IISODBC MSUtil.LogQuery.IISODBCInputFormat COMIISODBCInputContextClassClass

IISW3C MSUtil.LogQuery.IISW3CInputFormat COMIISW3CInputContextClassClass

NCSA MSUtil.LogQuery.IISNCSAInputFormat COMIISNCSAInputContextClassClass

NETMON MSUtil.LogQuery.NetMonInputFormat COMNetMonInputContextClassClass

REG MSUtil.LogQuery.RegistryInputFormat COMRegistryInputContextClassClass

TEXTLINE MSUtil.LogQuery.TextLineInputFormat COMTextLineInputContextClassClass

TEXTWORD MSUtil.LogQuery.TextWordInputFormat COMTextWordInputContextClassClass

TSV MSUtil.LogQuery.TSVInputFormat COMTSVInputContextClassClass

URLSCAN MSUtil.LogQuery.URLScanLogInputFormat COMURLScanLogInputContextClassClass

W3C MSUtil.LogQuery.W3CInputFormat COMW3CInputContextClassClass

XML MSUtil.LogQuery.XMLInputFormat COMXMLInputContextClassClass

Afterinstantiatinganinputformatobject,anapplicationcansettheinputformatparametersandusetheobjectasanargumenttotheExecuteorExecuteBatchmethodsoftheLogQueryobject.

MethodsTheInputFormatobjectsdonotexposemethods.

PropertiesTheInputFormatobjectsexposeread/writepropertieswiththesamenamesandcapitalizationastheparametersacceptedbythecorrespondingLogParserinputformat.Forexample,theMSUtil.LogQuery.EventLogInputFormatinputformatobjectexposesa"resolveSIDs"propertythatcontrolstheresolveSIDsparameteroftheEVTinputformat.Thevaluetypeacceptedandreturnedbyaninputformatobjectpropertydependsonthenatureofthevaluesthatcanbespecifiedfortheinputformatparameter,asdescribedbythefollowingtable:

Parametervalues

Propertyvaluetype JScriptExample

"ON"/"OFF"values Boolean oEVTInputFormat.resolveSIDs=true;

Enumerationvalues(e.g."ASC"/"PRINT"/"HEX")

String oEVTInputFormat.binaryFormat="PRINT";

Stringvalues String oEVTInputFormat.stringsSep=",";

Numericvalues Number oIISW3CInputFormat.recurse=10;

FormoreinformationonInputFormatParameters,seetheInputFormatsReference.

Examples

JScriptexample:

//CreateEVTInputFormatobjectvaroEVTInputFormat=newActiveXObject("MSUtil.LogQuery.EventLogInputFormat");

//SetinputformatparametersoEVTInputFormat.resolveSIDs=true;oEVTInputFormat.binaryFormat="PRINT";oEVTInputFormat.stringsSep=",";oEVTInputFormat.iCheckpoint="MyCheckpoint.lpc";

VBScriptexample:

DimoLogQueryDimoEVTInputFormatDimstrQueryDimoRecordSet

'CreateEVTInputFormatobjectSetoEVTInputFormat=CreateObject("MSUtil.LogQuery.EventLogInputFormat")

Seealso:LogQueryObjectOutputFormatObjectsLogParserCOMAPIOverviewC#Example

//ExecutequeryandreceiveaLogRecordSetvaroRecordSet=oLogQuery.Execute(strQuery,oEVTInputFormat);'SetinputformatparametersoEVTInputFormat.resolveSIDs=TrueoEVTInputFormat.binaryFormat="PRINT"oEVTInputFormat.stringsSep=","oEVTInputFormat.iCheckpoint="MyCheckpoint.lpc"

'ExecutequeryandreceiveaLogRecordSetSetoRecordSet=oLogQuery.Execute(strQuery,oEVTInputFormat)

OutputFormatObjectsOutputFormatobjectsprovideprogrammaticaccesstotheoutputformatssupportedbyLogParser.

OutputFormatobjectsareinstantiatedwiththeProgIdandthe.NETCOMwrapperclassnamesspecifiedinthefollowingtable:

OutputFormat ProgId .NETCOMWrapperClassName

CHART MSUtil.LogQuery.ChartOutputFormat COMChartOutputContextClassClass

CSV MSUtil.LogQuery.CSVOutputFormat COMCSVOutputContextClassClass

DATAGRID MSUtil.LogQuery.DataGridOutputFormat COMDataGridOutputContextClassClass

IIS MSUtil.LogQuery.IISOutputFormat COMIISOutputContextClassClass

NAT MSUtil.LogQuery.NativeOutputFormat COMNativeOutputContextClassClass

SQL MSUtil.LogQuery.SQLOutputFormat COMSQLOutputContextClassClass

SYSLOG MSUtil.LogQuery.SYSLOGOutputFormat COMSYSLOGOutputContextClassClass

TPL MSUtil.LogQuery.TemplateOutputFormat COMTemplateOutputContextClassClass

TSV MSUtil.LogQuery.TSVOutputFormat COMTSVOutputContextClassClass

W3C MSUtil.LogQuery.W3COutputFormat COMW3COutputContextClassClass

XML MSUtil.LogQuery.XMLOutputFormat COMXMLOutputContextClassClass

Afterinstantiatinganoutputformatobject,anapplicationcansettheoutputformatparametersandusetheobjectasanargumenttothe

ExecuteBatchmethodoftheLogQueryobject.

MethodsTheOutputFormatobjectsdonotexposemethods.

PropertiesTheOutputFormatobjectsexposeread/writepropertieswiththesamenamesandcapitalizationastheparametersacceptedbythecorrespondingLogParseroutputformat.Forexample,theMSUtil.LogQuery.CSVOutputFormatoutputformatobjectexposesa"headers"propertythatcontrolstheheadersparameteroftheCSVoutputformat.Thevaluetypeacceptedandreturnedbyanoutputformatobjectpropertydependsonthenatureofthevaluesthatcanbespecifiedfortheoutputformatparameter,asdescribedbythefollowingtable:

Parametervalues

Propertyvaluetype JScriptExample

"ON"/"OFF"values Boolean oCSVOutputFormat.tabs=true;

Enumerationvalues(e.g."ON"/"OFF"/"AUTO")

String oCSVOutputFormat.oDQuotes="OFF";

Stringvalues String oCSVOutputFormat.oTsFormat="yyyy-MM-dd";

Numericvalues Number oCSVOutputFormat.oCodepage=-1;

FormoreinformationonOutputFormatParameters,seetheOutputFormatsReference.

Examples

JScriptexample:

//CreateEVTInputFormatobjectvaroEVTInputFormat=newActiveXObject("MSUtil.LogQuery.EventLogInputFormat");

//CreateCSVOutputFormatobjectvaroCSVOutputFormat=newActiveXObject("MSUtil.LogQuery.CSVOutputFormat");

//SetoutputformatparametersoCSVOutputFormat.tabs=true;oCSVOutputFormat.oDQuotes="OFF";oCSVOutputFormat.oTsFormat="yyyy-MM-dd";oCSVOutputFormat.oCodepage=-1;

//CreatequerytextvarstrQuery="SELECTTimeGenerated,MessageINTOOutput.csvFROMSystem";

//ExecutequeryoLogQuery.ExecuteBatch(strQuery,oEVTInputFormat,oCSVOutputFormat);

VBScriptexample:

DimoLogQueryDimoEVTInputFormatDimoCSVOutputFormatDimstrQueryDimoRecordSet

'CreateEVTInputFormatobjectSetoEVTInputFormat=CreateObject("MSUtil.LogQuery.EventLogInputFormat")

'CreateCSVOutputFormatobjectSetoCSVOutputFormat=CreateObject("MSUtil.LogQuery.CSVOutputFormat")

'SetoutputformatparametersoCSVOutputFormat.tabs=TrueoCSVOutputFormat.oDQuotes="OFF"oCSVOutputFormat.oTsFormat="yyyy-MM-dd"oCSVOutputFormat.oCodepage=-1

'CreatequerytextstrQuery="SELECTTimeGenerated,MessageINTOOutput.csvFROMSystem"

'Executequery

Seealso:LogQueryObjectInputFormatObjectsLogParserCOMAPIOverviewC#Example

oLogQuery.ExecuteBatchstrQuery,oEVTInputFormat,oCSVOutputFormat

COMInputFormatPluginsCOMInputFormatPluginsareuser-developedinputformatsthatcanbeusedwithLogParsertoprovidecustomparsingcapabilities.

CustominputformatsaredevelopedasCOMobjectsimplementingthemethodsoftheILogParserInputContextCOMinterface.

OncedevelopedandregisteredwiththeCOMinfrastructure,custominputformatscanbeusedwitheithertheLogParserscriptableCOMcomponentsthroughtheExecuteandExecuteBatchmethodsoftheLogQueryobject,orwiththeLogParsercommand-lineexecutablethroughtheCOMinputformat.

ILogParserInputContextInterface:describesthemethodsthatmustbeimplementedbycustominputformatCOMobjects.RunTimeInteraction:describeshowLogParserinteractswithcustominputformatCOMobjectsatruntime.

Seealso:CustomPluginsCOMInputFormat

ILogParserInputContextInterfaceCustominputformatsaredevelopedasCOMobjectsimplementingthemethodsoftheILogParserInputContextCOMinterface.AcustominputformatimplementsthemethodsofthisinterfacebyimplementingtheILogParserInputContextinterfacedirectly,orbyimplementingtheIDispatch(Automation)interfaceexposingthemethodsoftheILogParserInputContextinterface.

Interface

////InterfaceGUID//

/*27E78867-48AB-433c-9AFD-9D78D8B1CFC7*/DEFINE_GUID(IID_ILogParserInputContext,0x27E78867,0x48AB,0x433C,0x9A,0xFD,0x9D,0x78,0xD8,0xB1,0xCF,0xC7);

////LogParserInputContextInterfaceimplementedbyLogParserInputpluginsandcalledbyLogParser.

classILogParserInputContext:publicIUnknown{public:

enumFieldType{Integer=1,Real=2,String=3,

Methods

OpenInput Processesthespecifiedfrom-entityandperformsanynecessaryinitialization.

GetFieldCount Returnsthenumberofinputrecordfields.

GetFieldName Returnsthenameofaninputrecordfield.

GetFieldType Returnsthetypeofaninputrecordfield.

ReadRecord Readsthenextinputrecord.

GetValue Returnsthevalueofafieldinthecurrentinputrecord.

CloseInput Releasesalltheresourcesandperformsanynecessarycleanup.

Timestamp=4,Null=5};

virtualHRESULTSTDMETHODCALLTYPEOpenInput(INBSTRbszFromEntity)=0;

virtualHRESULTSTDMETHODCALLTYPEGetFieldCount(OUTDWORD*pnFields)=0;

virtualHRESULTSTDMETHODCALLTYPEGetFieldName(INDWORDfIndex,OUTBSTR*pbszFieldName)=0;

virtualHRESULTSTDMETHODCALLTYPEGetFieldType(INDWORDfIndex,OUTDWORD*pnFieldType)=0;

virtualHRESULTSTDMETHODCALLTYPEReadRecord( OUTVARIANT_BOOL*pbDataAvailable)=0;

virtualHRESULTSTDMETHODCALLTYPEGetValue(INDWORDfIndex,OUTVARIANT*pvarValue)=0;

virtualHRESULTSTDMETHODCALLTYPECloseInput(INVARIANT_BOOLbAbort)=0;};

Properties

CustomProperties CustominputformatsdevelopedasIDispatchCOMobjectscansupportcustompropertiesthatarecontrolledatruntimeasinputformatparameters.

Seealso:RunTimeInteractionCustomPlugins

CloseInputMethodReleasesalltheresourcesandperformsanynecessarycleanup.

C++Syntax

HRESULTSTDMETHODCALLTYPECloseInput(INVARIANT_BOOLbAbort);ScriptSyntax

CloseInput(bAbort);

Parameters

bAbortABooleanvaluesettoTRUEifthequeryexecutionhasbeenaborted,orFALSEifthequeryexecutionhascompletedsuccessfully.

ReturnValueNone.

RemarksThisisthelastmethodinvokedbyLogParserbeforereleasingthecustominputformatCOMobject.

Examples

C++example:

HRESULTCProcessesInputContext::CloseInput(INVARIANT_BOOLbAbort){//Closethesnapshothandleif(m_hSnapshot!=INVALID_HANDLE_VALUE){CloseHandle(m_hSnapshot);m_hSnapshot=INVALID_HANDLE_VALUE;}

returnS_OK;}

VBScriptexample:

FunctionCloseInput(bAbort)

m_objQFEArray=Array()

EndFunctionSeealso:ILogParserInputContextInterfaceOpenInputMethodRunTimeInteractionCustomPlugins

GetFieldCountMethodReturnsthenumberoffieldsintheinputrecords.

C++Syntax

HRESULTSTDMETHODCALLTYPEGetFieldCount(OUTDWORD*pnFields);ScriptSyntax

nFields=GetFieldCount();

ReturnValueAnintegervaluecontainingthenumberoffieldsintheinputrecords.

Examples

C++example:

HRESULTCProcessesInputContext::GetFieldCount(OUTDWORD*pnFields){ //ThisInputContextexports4fields

*pnFields=4;

returnS_OK;}

VBScriptexample:

FunctionGetFieldCount()

'ThisInputFormatreturns4or6fields Ifm_bExtendedFields=TrueThen GetFieldCount=6 Else GetFieldCount=4 EndIf

Seealso:ILogParserInputContextInterfaceRunTimeInteractionCustomPlugins

GetFieldNameMethodReturnsthenameofaninputrecordfield.

C++Syntax

HRESULTSTDMETHODCALLTYPEGetFieldName(INDWORDfIndex,OUTBSTR*pbszFieldName);ScriptSyntax

fieldName=GetFieldName(fIndex);

Parameters

fIndexThe0-basedindexoftheinputrecordfield.TheindexvalueisguaranteedtobesmallerthanthenumberoffieldsreturnedbytheGetFieldCountmethod.

ReturnValueAstringvaluecontainingthenameoftheinputrecordfieldatthespecifiedposition.

Examples

C++example:

HRESULTCProcessesInputContext::GetFieldName(INDWORDfIndex,OUTBSTR*pbszFieldName){VBScriptexample:

switch(fIndex){case0:{*pbszFieldName=SysAllocString(L"ImageName");break;}

case1:{*pbszFieldName=SysAllocString(L"PID");break;}

case2:{*pbszFieldName=SysAllocString(L"ParentPID");break;}

case3:{*pbszFieldName=SysAllocString(L"Threads");break;}}

returnS_OK;}

FunctionGetFieldName(nFieldIndex)

SelectCasenFieldIndex Case0 GetFieldName="QFE" Case1 GetFieldName="Description" Case2 GetFieldName="InstallDate" Case3 GetFieldName="InstalledBy" Case4 GetFieldName="Comments" Case5 GetFieldName="SP" EndSelect

EndFunction

Seealso:ILogParserInputContextInterfaceGetFieldTypeMethodRunTimeInteractionCustomPlugins

GetFieldTypeMethodReturnsthetypeofaninputrecordfield.

C++Syntax

HRESULTSTDMETHODCALLTYPEGetFieldType(INDWORDfIndex,OUTDWORD*pnFieldType);ScriptSyntax

fieldType=GetFieldType(fIndex);

Parameters

ReturnValueAnintegervaluefromtheFieldTypeenumerationcontainingtheLogParserdatatypeoftheinputrecordfieldatthespecifiedposition.

Examples

C++example:

HRESULTCProcessesInputContext::GetFieldType(INDWORDfIndex,OUTDWORD*pnFieldType){VBScriptexample:

switch(fIndex){case0:{//ImageName*pnFieldType=ILogParserInputContext::String;break;}

case1:{//PID*pnFieldType=ILogParserInputContext::Integer;break;}

case2:{//ParentPID*pnFieldType=ILogParserInputContext::Integer;break;}

case3:{//Threads*pnFieldType=ILogParserInputContext::Integer;break;}}

returnS_OK;}

FunctionGetFieldType(nFieldIndex)

SelectCasenFieldIndex Case0 'String GetFieldType=3 Case1 'String GetFieldType=3 Case2 'Timestamp GetFieldType=4 Case3 'String GetFieldType=3 Case4 'String GetFieldType=3 Case5 'String GetFieldType=3

EndSelect

EndFunction

Seealso:ILogParserInputContextInterfaceGetFieldNameMethodRunTimeInteractionCustomPlugins

GetValueMethodReturnsthevalueofaninputrecordfield.

C++Syntax

HRESULTSTDMETHODCALLTYPEGetValue(INDWORDfIndex,OUTVARIANT*pvarValue);ScriptSyntax

value=GetValue(fIndex);

Parameters

ReturnValueAVARIANTcontainingthevalueofthespecifiedfield.TheVARIANTtypemustmatchtheLogParserdatatypedeclaredbytheGetFieldTypemethod,asshowninthefollowingtable:

DeclaredFieldType C++VARIANTType

VBScriptType

INTEGER VT_I8(alsocompatible:VT_I4) Long(VT_I4)

REAL VT_R8 Double

(VT_R8)

STRING VT_BSTR String(VT_BSTR)

TIMESTAMP VT_DATE(alsocompatible:VT_I8,VT_I4containingthenumberof100-nanosecondintervalssinceJanuary1,year0)

Date(VT_DATE)

NULL VT_NULL(alsocompatible:VT_EMPTY)

Null(VT_NULL)

RemarksAnyvaluecanbereturnedasaVT_NULLorVT_EMPTYVARIANT(aNullVBScriptvariable)toindicateaNULLvalue,regardlessofthefieldtypedeclaredbytheGetFieldTypemethod.Duetoqueryexecutionoptimizations,thereisnoguaranteethattheGetValuemethodwillbecalledforallthefieldsofaninputrecord.Infact,theGetValuemethodwillonlybecalledforthosefieldsthatarereferredtobythecurrentlyexecutingquery.Forexample,ifaqueryreferstotwofieldsonlyoutofaninputrecordmadeupoftenfields,thentheGetValuemethodwillbecalledforthosetwofieldsonly.Ifaquerydoesnotrefertoanyinputrecordfield(e.g."SELECTCOUNT(*)"),thentheGetValuemethodwillneverbecalled.

Examples

C++example:

HRESULTCProcessesInputContext::GetValue(INDWORDfIndex,OUTVARIANT*pvarValue){//InitializereturnvalueVariantInit(pvarValue);

switch(fIndex){case0:{//ImageNameV_VT(pvarValue)=VT_BSTR;V_BSTR(pvarValue)=SysAllocString(m_processEntry32.szExeFile);break;}

case1:{//PIDV_VT(pvarValue)=VT_I4;V_I4(pvarValue)=m_processEntry32.th32ProcessID;break;}

case2:{//ParentPIDV_VT(pvarValue)=VT_I4;V_I4(pvarValue)=m_processEntry32.th32ParentProcessID;break;}

case3:{//ThreadsV_VT(pvarValue)=VT_I4;V_I4(pvarValue)=m_processEntry32.cntThreads;break;

VBScriptexample:

FunctionGetValue(nFieldIndex)

SelectCasenFieldIndex

Case0'QFEGetValue=m_objQFEArray(m_nIndex).HotFixIDCase1'DescriptionGetValue=m_objQFEArray(m_nIndex).DescriptionCase2'InstallDateGetValue=m_objQFEArray(m_nIndex).InstallDateCase3'InstalledByGetValue=m_objQFEArray(m_nIndex).InstalledByCase4'CommentsGetValue=m_objQFEArray(m_nIndex).FixCommentsCase5'SPGetValue=m_objQFEArray(m_nIndex).ServicePackInEffect

EndSelect

EndFunction

Seealso:ILogParserInputContextInterfaceReadRecordMethodRunTimeInteractionCustomPlugins

returnS_OK;}

OpenInputMethodProcessesthespecifiedfrom-entityandperformsanynecessaryinitialization.

C++Syntax

HRESULTSTDMETHODCALLTYPEOpenInput(INBSTRbszFromEntity);

ScriptSyntax

OpenInput(bszFromEntity);

Parameters

bszFromEntityThefrom-entityspecifiedintheFROMclauseofthecurrentlyexecutingquery,oranemptystringifLogParserisexecutedinHelpModetodisplaythequick-referencehelponthecustominputformat.

ReturnValueNone.

RemarksTheOpenInputmethodisthefirstmethodcalledbyLogParserafterthecustominputformatCOMobjecthasbeeninstantiated.Animplementationofthismethodwouldusuallyperformanynecessaryobjectinitialization,preparethefrom-entityforinputrecordretrieval(e.g.openinganinputfile),andeventuallypre-processtheinputtogathertheinputrecordfieldsmeta-informationthatwillbereturnedby

theGetFieldCount,GetFieldName,andGetFieldTypemethods.UserscanexecutetheLogParsercommand-lineexecutableinHelpModetodisplayaquick-referencehelponacustominputformat.Thequick-referencehelpdisplaystheinputrecordfieldnamesandtypes,whichareretrievedthroughcallstotheGetFieldCount,GetFieldName,andGetFieldTypemethods.Iftheuser-suppliedhelpmodecommanddoesnotincludeafrom-entity,thebszFromEntityargumentwilbeanemptystring.Inthesecases,acustominputformatCOMobjectcanbehaveintwoways:Iftheinputrecordfieldsdonotdependonthefrom-entityspecifiedinthequery(i.e.iftheinputrecordstructureisfixed),thenthecustominputformatCOMobjectshouldaccepttheemptyfrom-entitywithoutreturninganerror,allowingLogParsertosubsequentlycalltheGetFieldCount,GetFieldName,andGetFieldTypemethodstoretrievetheinputrecordstructure;Iftheinputrecordfieldsdependonthefrom-entityspecifiedinthequery(i.e.iftheinputrecordstructureisextractedfromtheinputdata),thenthecustominputformatCOMobjectshouldrejecttheemptyfrom-entityreturninganerror,whichwillinturncausethehelpcommandtodisplayawarningmessagetotheuserinplaceoftheinputrecordstructure.

Examples

C++example:

HRESULTCProcessesInputContext::OpenInput(INBSTRbszFromEntity){//Initializeobject...

//Thisinputformatdoesnotrequireafrom-entity,so//wewilljustignoretheargument

VBScriptexample:

FunctionOpenInput(strComputerName)

DimobjWMIService DimobjQFEs DimnLengthSeealso:

returnS_OK;} 'Defaultcomputernameislocalmachine IfIsNull(strComputerName)OrLen(strComputerName)=0Then strComputerName="." EndIf

'QueryforalltheQFE'sonthespecifiedmachine SetobjWMIService=GetObject("winmgmts:"&"{impersonationLevel=impersonate}!\\"&strComputerName&"\root\cimv2") SetobjQFEs=objWMIService.ExecQuery("Select*fromWin32_QuickFixEngineering")

'Storeinarray m_objQFEArray=Array() ForEachobjQFEInobjQFEs ReDimPreservem_objQFEArray(UBound(m_objQFEArray)+1) Setm_objQFEArray(UBound(m_objQFEArray))=objQFE Next

m_nIndex=LBound(m_objQFEArray)

EndFunction

ILogParserInputContextInterfaceCloseInputMethodRunTimeInteractionCustomPlugins

ReadRecordMethodReadsthenextinputrecord.

C++Syntax

HRESULTSTDMETHODCALLTYPEReadRecord(OUTVARIANT_BOOL*pbDataAvailable);ScriptSyntax

bDataAvailable=ReadRecord();

ReturnValueABooleanvaluesettoTRUEifanewinputrecordhasbeenreadandisavailableforconsumption,orFALSEiftherearenomoreinputrecordstoreturn.

RemarksAnimplementationoftheReadRecordmethodwouldusuallyreadanewdataitemfromtheinputandstoreitinternally,waitingforLogParsertosubsequentlycalltheGetValuemethodmultipletimestoretrievetheinputrecordfieldvalues.TheBooleanvaluereturnedbytheReadRecordmethodisusedbyLogParsertodeterminewhichcustominputformatmethodswillbecallednext.IfthemethodreturnsTRUE,signalingavailabilityofaninputrecord,LogParserwillcalltheGetValuemethodmultipletimestoretrievetheinputrecordfieldvalues,followedbyanewcalltotheReadRecordmethodtoreadthenextinputrecord.IfthemethodreturnsFALSE,signalingtheendoftheinputdata,LogParserwillcalltheCloseInputmethodandreleasethecustominputformatCOMobject.

Examples

C++example:

HRESULTCProcessesInputContext::ReadRecord(OUTVARIANT_BOOL*pbDataAvailable){if(m_hSnapshot==INVALID_HANDLE_VALUE){//Thisisthefirsttimewehavebeencalled

//Getashapshotofthecurrentprocessesm_hSnapshot=CreateToolhelp32Snapshot(TH32CS_SNAPPROCESS,0);if(m_hSnapshot==INVALID_HANDLE_VALUE){//ErrorreturnHRESULT_FROM_WIN32(GetLastError());}

//Getthefirstentryif(!Process32First(m_hSnapshot,&m;_processEntry32)){DWORDdwLastError=GetLastError();if(dwLastError==ERROR_NO_MORE_FILES){//Noprocesses*pbDataAvailable=VARIANT_FALSE;returnS_OK;}else{//ErrorreturnHRESULT_FROM_WIN32(GetLastError());}

VBScriptexample:

FunctionReadRecord()

Ifm_nIndex>=UBound(m_objQFEArray)Then'EnumerationterminatedReadRecord=FalseElse'Advancem_nIndex=m_nIndex+1ReadRecord=TrueEndIf

EndFunction

Seealso:ILogParserInputContextInterfaceGetValueMethodRunTimeInteractionCustomPlugins

}else{//Thereisdataavailable*pbDataAvailable=VARIANT_TRUE;returnS_OK;}}else{//Wehavealreadybeencalledbefore,andwehavealreadytakenasnapshot

//Getthenextentryif(!Process32Next(m_hSnapshot,&m;_processEntry32)){DWORDdwLastError=GetLastError();if(dwLastError==ERROR_NO_MORE_FILES){//Nomoreprocesses*pbDataAvailable=VARIANT_FALSE;returnS_OK;}else{//ErrorreturnHRESULT_FROM_WIN32(GetLastError());}}else{//Thereisdataavailable*pbDataAvailable=VARIANT_TRUE;returnS_OK;}}}

CustomPropertiesProvideparametersforthecustominputformat.

C++Syntax

HRESULTSTDMETHODCALLTYPEput_propertyName(INVARIANT*value);ScriptSyntax

put_propertyName(value);

Parameters

valueAVT_BSTRVARIANTcontainingthestringparametervaluespecifiedwiththe-iCOMParamsparameteroftheCOMinputformat.

ReturnValueNone.

RemarksCustompropertiescanonlybeexposedbycustominputformatsthatimplementtheIDispatch(Automation)interface.Theseareusuallycustominputformatsdevelopedasscriptlets(.wscfiles)writteninJScriptorVBScript.Custompropertiesexposedbyacustominputformatcanbesetintwodifferentways:WiththeLogParsercommand-lineexecutable,custompropertiescanbesetthroughthe-iCOMParamsparameteroftheCOMinput

format,asshowninthefollowingexample:

C:\>LogParser"SELECT*FROMfile.txt"-i:COM-iProgID:MySample.CustomInputFormat-iCOMParams:property1=value1,property2=value2WiththeLogParserscriptableCOMcomponents,custompropertiescanbesetdirectlyonthecustominputformatobjectbeforespecifyingtheobjectasanargumenttotheExecuteorExecuteBatchmethodsoftheLogQueryobject,asshowninthefollowingJScriptexample:

varobjLogQuery=newActiveXObject("MSUtil.LogQuery");

//CreatecustominputformatobjectvarobjCustomInputFormat=newActiveXObject("MySample.CustomInputFormat");

//SetcustominputformatparametersobjCustomInputFormat.property1="value1";objCustomInputFormat.property2="value2";

//ExecutequeryvarobjRecordSet=objLogQuery.Execute("SELECT*FROMfile.txt",objCustomInputFormat);

Examples

VBScriptexample:

Functionput_extendedFields(strValue)

IfUCase(strValue)="ON"Then m_bExtendedFields=True Else m_bExtendedFields=False EndIf

EndFunction

Seealso:ILogParserInputContextInterfaceRunTimeInteractionCustomPluginsCOMInputFormat

RunTimeInteractionCustominputformatCOMobjectsareusedbyLogParserintwodifferentscenarios:whenexecutingaquery,andwhendisplayingaquick-referencehelponthecustominputformatwhentheLogParsercommand-lineexecutableisusedinHelpMode.

QueryExecutionScenarioInthisscenario,acustominputformatCOMobjectisusedtoretrieveinputrecordsfromthespecifiedfrom-entity.

TomakeanexampleofthesequenceofthemethodcallsinvokedbyLogParseronthecustominputformatCOMobjectinthisscenario,wewillassumethatthecustominputformatgeneratesinputrecordscontainingthefollowingfourfields:

"FirstField",STRINGtype;"SecondField",INTEGERtype;"ThirdField",TIMESTAMPtype;"FourthField",STRINGtype.

Inaddition,wewillassumethatthequerybeingexecutedreferencesonlythreefieldsoutofthefourfieldsexportedbythecustominputformat,asinthefollowingexample:

SELECTFourthField,ThirdFieldFROMInputFile.txtWHEREFirstFieldLIKE'%test%'Thefollowingtableshowsthesequenceofmethodcallsundertheseassumptions:

Methodcall ReturnedvalueReturnedvaluedescription

Objectisinstantiated

OpenInput("InputFile.txt") None

GetFieldCount() 4

GetFieldName(0) "FirstField"

GetFieldType(0) 3 FieldType.String

GetFieldName(1) "SecondField"

GetFieldType(1) 1 FieldType.Integer

GetFieldName(2) "ThirdField"

GetFieldType(2) 4 FieldType.Timestamp

GetFieldName(3) "FourthField"

ReadRecord() TRUE aninputrecordisavailable

GetValue(0) VT_BSTRVARIANT

firstfieldvalue

GetValue(2) VT_DATEVARIANT

thirdfieldvalue

fourthfieldvalue

firstfieldvalue

thirdfieldvalue

fourthfieldvalue

... ... ...

firstfieldvalue

thirdfieldvalue

fourthfieldvalue

ReadRecord() FALSE nomoreinputrecordsavailable

CloseInput(FALSE) None

Objectisreleased

HelpModeScenarioWhentheLogParsercommand-lineexecutableisusedinHelpModetodisplayaquick-referencehelponthecustominputformat,thecustominputformatCOMobjectisonlyusedtoretrievethefieldinformationthatisdisplayedtotheuser.

Theuser-suppliedhelpmodecommandmayormaybenotincludeafrom-entity,asshowninthefollowingexamples:

C:\>LogParser-h-i:COM-iProgID:MySample.CustomInputFormatfile.txt

C:\>LogParser-h-i:COM-iProgID:MySample.CustomInputFormat

Iftheuser-suppliedhelpmodecommanddoesnotincludeafrom-entity,thenthebszFromEntityargumentoftheOpenInputmethodwillbeanemptystring.SeetheRemarkssectionoftheOpenInputMethodReferenceformoreinformationonhowcustominputformatCOMobjectsshouldbehaveinthiscase.

TomakeanexampleofthesequenceofthemethodcallsinvokedbyLogParseronthecustominputformatCOMobjectinthisscenario,wewillassumethatthecustominputformatgeneratesinputrecordscontainingthefollowingfourfields:

"FirstField",STRINGtype;"SecondField",INTEGERtype;"ThirdField",TIMESTAMPtype;"FourthField",STRINGtype.

Inaddition,wewillassumethatthehelpcommanddoesnotincludeafrom-entity.

Thefollowingtableshowsthesequenceofmethodcallsundertheseassumptions:

Methodcall Returnedvalue Returnedvaluedescription

Objectisinstantiated

OpenInput("") None

GetFieldCount() 4

GetFieldName(0) "FirstField"

GetFieldName(1) "SecondField"

GetFieldType(1) 1 FieldType.Integer

GetFieldName(2) "ThirdField"

GetFieldType(2) 4 FieldType.Timestamp

GetFieldName(3) "FourthField"

CloseInput(FALSE) None

Objectisreleased

Seealso:ILogParserInputContextInterfaceCustomPlugins

LegalInformation

MicrosoftDocumentationInformationinthisdocument,includingURLandotherInternetWebsitereferences,issubjecttochangewithoutnotice.Unlessotherwisenoted,theexamplecompanies,organizations,products,domainnames,e-mailaddresses,logos,people,placesandeventsdepictedhereinarefictitious,andnoassociationwithanyrealcompany,organization,product,domainname,e-mailaddress,logo,person,placeoreventisintendedorshouldbeinferred.Complyingwithallapplicablecopyrightlawsistheresponsibilityoftheuser.Withoutlimitingtherightsundercopyright,nopartofthisdocumentmaybereproduced,storedinorintroducedintoaretrievalsystem,ortransmittedinanyformorbyanymeans(electronic,mechanical,photocopying,recording,orotherwise),orforanypurpose,withouttheexpresswrittenpermissionofMicrosoftCorporation.

Microsoftmayhavepatents,patentapplications,trademarks,copyrights,orotherintellectualpropertyrightscoveringsubjectmatterinthisdocument.ExceptasexpresslyprovidedinanywrittenlicenseagreementfromMicrosoft,thefurnishingofthisdocumentdoesnotgiveyouanylicensetothesepatents,trademarks,copyrights,orotherintellectualproperty.

ActiveDirectory,JScript,Microsoft,MSDN,VisualBasic,VisualStudio,Windows,WindowsMedia,andWindowsServerareeitherregisteredtrademarksortrademarksofMicrosoftCorporationintheUnitedStatesand/orothercountries.

Thenamesofactualcompaniesandproductsmentionedhereinmaybethetrademarksoftheirrespectiveowners.