sing Compliance - Role of Analytic Techniques

8/14/2019 sing Compliance - Role of Analytic Techniques

1/39

Page: 1 Date: Thursday, 3 November 2006

__________________

Optimising Compliance

The role of analytic techniques __________________

__________________ Abstract

The use of quantitative analytical techniques has an important and growing role in the optimisation of revenueauthority client compliance. This paper covers some of the issues associated with the application of analytic

techniques in a revenue authority business setting.The views expressed in this paper are those of the author and do not necessarily reflect the considered views ofmy colleagues at the Australian Taxation Office on any matter.

Stuart HamiltonAssistant Commissioner

Corporate Intelligence & RiskAustralian Taxation Office

2 Constitution AvenueCanberra ACT 2600

Marital

0755 cases93.8%

2 Education

Occupation

0322 cases

73.6%

12 Hours < - > 38

021 cases

71.4%

26 Age < - > 33.5

022 cases

63.6%

54

161 cases

73.8%

55

1219 cases

74.4%

7

Decision Tree audit-csv.txt $ Adjusted

Rattle 2006-10-02 16:23:45 Stuart


2/39


Table of Contents

Title & abstract ...................................................................................................................1

Overview .............................................................................................................................3

BackgroundThe ATO intent and business model....................................................................................3Analytics and the personalisation of interactions ...................... ................... ...................... ..5

Scene settingNon-compliance ...................... ...................... ...................... ...................... ...................... .....6Strategic Risk.......................................................................................................................7A consistent measuring framework for Revenue Risk..........................................................9

OptimisationTreatments available (Enhance treatments) .................... ...................... ...................... ......17Assignment of clients to those treatments (Enhance candidate selection process) ...........20Case mix (Enhance case mix) ...................... ...................... ...................... ...................... ...26

Other mattersPrioritising analytic work.....................................................................................................29Measuring strikes...............................................................................................................30The effect of varying strike rates ................... ...................... ...................... ...................... ...30Traditional versus analytic driven case selection an example......................... ................33Optimising capability ..................... ....................... ..................... ................... ......................34

Conclusions .....................................................................................................................34

Annexes ............................................................................................................................35Scoring using Tax ................. ...................... ...................... ...................... ...................... ...35Some additional Rattle analysis output ...................... ...................... ..................... .............36Spreadsheets used in this paper........................................................................................37Summary of some analytic methodologies that might be used in optimisation ..................38


3/39


Optimising Compliance the role of analytic techniques

OverviewThe use of quantitative analytical techniques has an important and growing role in the optimisation of revenueauthority client compliance. This paper covers some of the issues associated with the use of analytic techniques ina revenue authority business setting.

Background The ATO intent and business modelThe ATOs business intent is to optimise voluntary compliance and make payments under the law in a way thatbuilds community confidence 1. It is similar to the underlying intent or mission of many, if not most, other revenueauthorities.

The ATO Business Model is premised on clients self assessing their tax obligations, with the ATO providingeducation and assistance, making it easier to comply, and verifying that compliance is occurring using appropriaterisk management approaches.

The ATO seeks to optimise voluntary compliance by:o The provision of education and advice , including rulings, alerts and self help materials helps clients and

their advisers on understanding the ATO's view of how the law might apply to their facts andcircumstances.

o Making a compliant clients tax experience easier, cheaper and more personalised via the ATO ChangeProgram . (Non-compliant clients may find it becoming harder and more expensive to not comply!)

o Addressing non compliance in an appropriate manner. The Compliance program sets out the ATO viewof compliance risks posed by some clients and how the ATO plans to address them in the coming year.

Addressing non-compliance in an appropriate manner requires use of the ATO the Taxpayers Charter principlesregarding taxpayer rights and obligations; the Compliance Model view of treatment selection and escalation, andthe Brand Navigator on the appropriate ATO persona to present.

These views are brought together in the ATO Community Relationship Model 2:

1 See http://www.ato.gov.au/content/downloads/ARL_77317_Strategic_Statement_booklet.pdf 2 See http://www.ato.gov.au/docs/CommunityRelationshipModel.doc

CommunityRelationship Model,Compliance Model &Taxpayers Charter

Brand Navigator:> Trusted authority

> Professional advisor> Fair administrator> Firm enforcer

Compliance Program

Change Program

Education & Advice


4/39


Effectively, in a manner consistent with the intent of the Taxpayers Charter, the ATO aims to present anappropriate ATO persona to deliver, through the appropriate communication channel, a relatively tailoredcompliance strategy that optimises the long term compliance of the client.

Compliance Model Brand Persona

Willing to do theright thing

Try to, but dontalways succeed

Dont wantto comply

Have decidednot to comply

Use the full forceof the law

Assist tocomply

Deter bydetection

Make it easyAttitude to

ComplianceComplianceStrategy

Createpressure

down

Createpressure

down

The Easier, Cheaper and More Personalised change program sets out eight key principles to guide the designand development of ATO products and services. Two of the guiding principles link to the services that are enabledor facilitated by analytic methods.

These two principles 3 are:

Guiding Principle 04 : You will receive notices and forms that make sense in your terms and that reflectyour personal dealings with the revenue system.

Guiding Principle 08 : You will experience compliance action which takes into account your compliancebehaviour, personal circumstances and level of risk in the system.

3 See http://www.ato.gov.au/content/downloads/Making_it_easier_to_comply_2005_06.pdf

Firm enforcer

Fair Administrator

Firm enforcer

Professional advisor

Trusted authorit

Fair Administrator


5/39


Both of these principles posit that the ATOs interactions with the client will be more personalised that is thenotices, forms and compliance action 4 will better reflect the clients circumstances, behaviours and risk profilesrelative to other clients.

Background - Analytics and the personalisation of interactionsTo provide more personalised interactions, the ATO needs to be able to identify within its client base relevantdifferences between clients. That is those differences that matter.

This is the essence of analytics, being able to examine large data holdings to identify those attributes that correlate

to particular client circumstances, behaviours and risk profiles. Those attributes can then be used to enable theATO to better tailor its client interactions. That is to make the interaction more relevant to the client morepersonalised, more likely to succeed.

This examination of client attributes to identify those that correlate to particular client circumstances, behavioursand risk profiles is an area of analytics known as data mining or knowledge discovery and it is this approach thatheralds an improved way of optimising client compliance. Before delving into these analytic approaches somescene setting is needed to establish a consistent base of terminology and understanding for this paper.

Scene setting - OptimisationMany, if not most, revenue authorities aspire to optimise client compliance with taxation laws, but what doesoptimise really mean and what role might advanced analytic techniques play?

Economic theory (and common sense!) tells us that there is an optimum point in a revenue system at which the netrevenue of the system would be maximised (O s) for given tax rates and rules. Beyond that point the marginal costexceeds marginal revenue it costs more than a dollar to get a dollar in.

In most economies the revenue authority would operate significantly below this point (O 1) reflecting lowerbudgetary needs of the Governments, administration resource constraints and accepted community attitudesregarding the nature and level of intrusion of the revenue authority into the economy.

In these circumstances optimisation takes on a different practical meaning.

The challenge for the revenue authority is to optimise long term net revenue outcomes within its resource and otheroperating constraints ie to maximise long term voluntary compliance given relatively fixed resources.

4 Compliance action in accordance with the ATO Compliance Model and the Taxpayers Charter. The compliance modeldirects that we better understand why people are not complying and that we develop appropriate and proportionate responses. An underlyingobjective is to develop responses that maximise the proportion of the community who are both able to, and choose to, comply. Depending onthe reasons for non-compliance, responses can be aimed at: enabling compliance through education, assistance or making it simpler to comply,or enforcing compliance through via administrative and prosecution action.

Total Revenue

Total Cost

Net Revenue

$

Cost

TheoreticalSystemOptimum

O

O

ResourceConstrainedOptimum

Theoretical Full Compliance bandwidth


6/39


This paper focuses on this view of resource constrained optimisation and how it might be more objectivelyachieved by revenue authorities. The analysis is largely restricted to quantifiable aspects of revenue risk and moresubjective aspects such as reputation risk are not overtly factored in.

In order to optimise long term voluntary compliance it is suggested that number of aspects need to occur:

o The number of clients requiring relatively expensive remedial compliance action should be minimised (ieclients should know how to comply [education], be able to comply [the interaction of client and the revenueauthorities systems needs to 'fit'] and be ready to comply [attitudinal]).

o The right range of remedial compliance treatments should be available. Any remedial compliance actionshould maximise the long term compliance gain within the client base (ie the treatment should be mostappropriate to engender voluntary compliance overall a leverage compliance model viewpoint).

o The right clients should be selected for the appropriate remedial compliance action (ie the strike rate for aparticular remedial compliance treatment should be maximised).

o The right mix of discretionary compliance work needs to be achieved overall so that long term revenues areoptimised within the resource and capability constraints faced by the organisation.

Revenue authorities generally have a range of administrative treatments: education, assistance, review andenforcement products that they can use to address non-compliance, in addition to generally longer term systemand legislative/policy changes.

At present optimisation in tax administration is generally based on the expert views of experienced senior officers.Equally valid, but different views might also be held and hence the outcome is one of subjective optimisation.Colloquially it might be put as informed gut feel of the right balance - a judgment call.

It is suggested in this paper that these judgment based optimisation approaches may be better informed and oftenenhanced by the use of objective business decision support approaches that assist in the determination of theright products, the right mix of products and the right clients to apply those treatments to.

However these objective approaches do require reasonably robust knowledge of the costs and benefits of variouscourses of action and this can be a significant problem. A greater level of consistency of approach andunderstanding is also needed so that the information used in business decision support is valid and verifiable 5.

Scene setting - Non-complianceTo select the right clients for the right treatment we need to have a consistent view of what non-compliance is sothat we can form an objective view on the relativities of various aspects of client non-compliance.

While there are many aspects to compliance with revenue laws and regulations (see the ATO ComplianceProgram) client obligations can generally be thought of in the following broad manner 6:

o Registering in the system (either with the revenue authority or with some other body)

o Lodging or filing the appropriate forms on time

o Providing accurate information on those forms

o Making any transfers or payments due on time

Most revenue systems also require a client to maintain records of appropriate information for some set period. Ie

o Keeping records that allow verification of the information used to satisfy the above obligations.

5 If such information is not available then simulation modelling can still assist in determining key directions and sensitivity aspects.6 See page 7 of the OECD 2004 document: Compliance Risk Management @ http://www.oecd.org/dataoecd/44/19/33818656.pdf


7/39


Scene setting - Strategic RiskAt a macro compliance level we could consider revenue risk to be total tax not collected due to non-compliancewith the above broad obligations. The US IRS for example has produced a 'Tax Gap' analysis based on thisrevenue risk framework. (Note that the IRS have non-registration is included in non-filing.)

The IRS Tax Compliance Measurement Program (now known as the National Research Program -NRP) is primarilybased on a large scale periodic stratified random audit approach and indicates that:

o ~ 10% of non-compliance is related to non registration and non lodgement;

o ~ 80% to under reporting of taxable income; ando ~10% to non payment of debts due.

With an overall non-compliance rate in the USA of ~15% of the theoretical tax believed due.

Simplistically, if the voluntary compliance cost and response was equal between these types of non-compliancethen the compliance resource allocation should broadly match the above splits. This simplistic view does not takeinto account that resource intensity and compliance responsiveness will in practice differ significantly depending onthe nature, causes and extent of the non-compliance and the available treatments for it. (More on this later.) Sucha macro or high level view is of limited practical use in understanding and addressing non-compliance.

Source: http://www.irs.gov/pub/irs-news/tax_gap_figures.pdf 7

Moreover, while superficially compliance may seem a strict matter of fact yes or no - in practice it is often farmore blurred or grey, a question of interpretation and judgement, than may appear to a layman 8. Uncertainties indefinitions, interpretation and measurement compound to render views on what full compliance is as a relativelybroad bandwidth with wide confidence levels. Movements in total compliance are very difficult to ascertain with ahigh degree of confidence and are necessarily dated with this periodic sampling approach.

7 Note - the figures for underreporting are more subjective as they are inflated (by a factor between 2 and 3) to take intoaccount income not detected by audit processes. See for example the US Tax Inspector General for Tax Administrations April 2006 report @http://www.ustreas.gov/tigta/auditreports/2006reports/200650077fr.pdf#search=%22US%20IRS%20Tax%20Gap%20Estimate%20Inspector%20General%22 8 See the OECD 1999 document: Compliance Measurement at page 4 @ http://www.oecd.org/dataoecd/36/1/1908448.pdf


8/39


In these circumstances concepts such as tax gap become significantly less precise and useful in objectivelyguiding compliance activities the what and who to review. The OECD practice note on performancemeasurement in tax administration sums up the tax gap question at paragraph 224:

To sum up, the general position on measuring the tax gap is that it is diff icult if not downright impossible and even if it were possible to get a reliable total figure it would not tell us much of practical value in the struggle against non- compliance .

That does not mean that a random audit program such as that use by the US IRS NRP does not have its place inthe optimising compliance tool kit. It just needs to be used appropriately when knowledge of the risks of a clientgroup is relatively low so that the low strike rate, and hence high rate of intrusion into compliant clients affairs, of asignificant random case selection approach can be justified overall in terms.

Broadly speaking when optimising compliance I would suggest that any large scale random audit program berestricted to those situations where the true informational value and deterrence effects of the random processoutweigh the significant social and opportunity costs it imposes over targeted detection based approaches. Ie Thefew areas where the organisation is flying blind. Eg Where new legislation has been introduced and thediscriminate features of non-compliance are largely unknown and cant be reasonably estimated.

So generally we may conclude that:o Random / coverage based approaches are useful for threat discovery where we cannot effectively target

clients on a known risk. They can also be used to update views on risk relativities.

o Risk based approaches are appropriate where we can target clients by way of known risks. (Note that wecan still discover new risks to the extent they occur, and are detected, in the clients reviewed.)

This discovery/ detection continuum can be represented in the following model:

Most leading OECD revenue authorities have a reasonably robust knowledge of their client base and the broadstrategic risks do not generally change much year to year. Overall most of their clients pay most of their tax mostof the time. The ATO's compliance program http://www.ato.gov.au/content/downloads/ARL_77362_n7769-8-2006_w.pdf details the ATO view of the compliance risks and issues facing the organisation, the treatments beingapplied and the results.

So if we don't use a tax gap analysis to inform the relativities of strategic and operational intelligence (the what andwho to treat) is there a consistent, objective framework that can be applied across products and obligations?

Detection of known risks/issuesRisk Targeted AuditsPredictive Data Mining (Scoring Approaches)

Discovery of new threats/risksStratified Random AuditDescriptive Data Mining (Clustering Approaches)

Low High Knowledge of risk in client population

If you have a robust and effective measure of client risks and how to detect them then discovery processescan be kept relatively smaller than the situation where you have little or no knowledge of client risks.

Discovery processes often have a low strike rate the value is generally in the insight into better detectionapproaches.

Note: The volume of cases needed to discover relative differences in risk (and so tune selectionapproaches) is significantly lower than the volume of cases needed for a statistically accurate estimate ofthe overall tax gap.

Discovery v Detection a risk knowledge continuum


9/39


Scene setting - A consistent measuring framework for Revenue RiskTo objectively optimise compliance across the obligations we need relatively common approach to measuring non-compliance otherwise our prioritisation between types of non-compliance will be necessarily subjective. If onearea prioritises cases on one basis and other area uses a very different measuring stick then the overall view of therelativities of revenue risk will still be based on personal, albeit informed, judgements of the matter.

Without an objective mechanism our legal 'choice of remedy' may be more open to criticism and question overconcerns of bias, subjectivity and inconsistency. (Why did we select case X over case Y or risk A over risk B ?)

Broadly speaking the following conjectures might be stated regarding the requirements of a revenue risk

measurement approach:o For the client obligations of registration, on time lodgment, accurate reporting and correct

accounting/payment, client revenue risk should be consistently viewed where this is practical.

o Client revenue risk ranking should be done as objectively as possible, taking into account aspects such asthe revenue at risk, the severity of non-compliance and our confidence level in the revenue at risk.

o Client revenue risk ranking should be flexible enough to cater for aspects such as losses, recidivism,schemes and associated client linkages.

o Client revenue risk ranking should be highly scalable and stable from lists of a few clients to potentiallyscoring millions of clients.

For the client obligations of registration, lodgment, accurate reporting and correct accounting/payment, I suggestthat revenue risk can generally be distilled into the following four features for the purposes of comparable revenuerisk ranking across and within obligations and products:

o Tax Delta tax - The change in primary tax associated with the non-compliance.[ie Identifies those who may have the most tax wrong. An absolute amount. Client A may have underpaid$5,500 in tax in year y.]

o Tax/( Tax + Tax) Severity - The relative severity of the non-compliance as a percentage of tax paid.[ie Identifies those who may have most of their tax wrong. A relative value. Client A may have underpaid15% of their tax in year y.]

o Cf ( Tax) Confidence - The confidence interval associated with our estimate of Tax. [ieIdentifies how confident we are of the estimate in Tax. We are 90% confident that Client A underpaid$5,500 in tax in year y +/- $550]

o Pf ( Tax) Proportion collectable - The proportion of Tax estimated to be collectable. [Afunction of a clients propensity to pay and their capacity to pay. We estimate that 80% of the $5,500estimated to be underpaid by Client A will be collectable +/- 1,000.]

Let us look at how this revenue risk concept might work for the key client obligations previously identified:

o RegistrationFor registration non-compliance Tax is the estimate of the amount of net revenue that is predicted to bederived by achieving registration (and subsequent lodgement) for those clients not registered in thesystem. (Detection is generally via matching processes, comparing a list of names against registeredclients to detect those operating outside the system.)

o LodgmentFor non Lodgers: Tax is the estimate of the amount of net revenue (ie taking into account withholding andinstalments: PAYGW & PAYGI etc) that is predicted to be derived by achieving lodgment from a treatmentfor those clients who otherwise would not lodge.

Late Lodgers: For those clients who would lodge late Tax is really the time value of money (PV) broughtforward by achieving earlier lodgment following a treatment. Most lodgment clients by number would belate lodgers rather than non lodgers. The distinction between the two classes will overlap and an informeddecision is needed for the transition from one class to the other.. Eg After 3 months a late lodger'transitions' into a non lodger.


10/39


o ReportingTax is the estimated primary tax associated with the incorrect reporting of a value on a lodged form.

Most underpayment of tax is due to incorrect reporting. Unlike the more evident registration, lodgment andpayment compliance aspects (eg Client A did not register/lodge/pay), incorrect reporting is generallyharder to detect without relevant third party data.

This is of concern as in mature tax systems most non-compliance is likely to be associated withunderstatements of taxable income or turnover (or the over claiming of deductions or credits).

o Correct accounting/PaymentNon Payers: Tax is the estimate of the amount of net revenue (ie taking into account PAYGW & PAYGIetc) that is predicted to be derived by achieving payment from a treatment for those clients who otherwisewould not pay.

Late payers: For those clients who would pay late Tax is the time value of money (PV) brought forward byachieving earlier payment following a treatment. Most debt clients by number would be late payers ratherthan non payers. As with the late/non lodgers the two classes will overlap and an informed decision pointis needed for the transition from one class to the other.

Having grouped our non-compliance using a common revenue measuring stick of Tax, what are some of thesalient aspects we would expect to see in the resultant distribution?

Expected Tax distribution where Tax is the estimated primary tax involved in the non-compliance:

o Modal ~ 0. ie most clients comply with their tax obligations (in accordance with the compliance modelview.)

o Smaller negative tail. ie clients, and their advisors, have more of an incentive to detect, or not to make,errors in this direction.

o Longer positive tail following a truncated pareto or power distribution past $Y. ie most errors or omissionsare relatively small while a few are very large. The fall-off is basically in accordance with the well knownpareto distribution (a form of inverse power distribution).

o Our ability to predict Tax from data is limited and has a confidence interval associated with it. Ourconfidence will be greatest where we have significant experience and falls off as we move away from this.

o Risk scores based on deviations from average Tax would be relatively stable for large populations. Ie theAverage overall tax underpaid would not vary significantly from year to year.

$Y

n

Tax

xx

xx

x xx

x

x x

xx

xxx

xx

x

x

x

xx

x x

x

x

x

xx

x

x

xx

xx

xx

xx

xx

x

x

x

x

xx

x

x

xConfidence distributionin Tax Estimate


11/39


A similar distribution would follow for severity:

Theoretical Tax/(Tax+ Tax) DistributionWhere Tax/(Tax+ Tax) is the estimated primary tax involved in the non-compliance divided by the total net taxpayable:

This gives a indication of the relative severity of the non-compliance taken as a proportion of the tax paid:0 implies the amount of the error, avoidance or evasion is relatively insignificant compared to the tax the

client has paid.

1 implies that the tax error, avoidance or evasion affected all or almost all of the tax otherwise payable.

Suggested features, (as with the Tax distribution):

o Modal ~ 0 most clients comply with their tax obligations

o Smaller though longer negative tail clients, and their advisors, have an incentive not to make errors inthis direction, but an overpayment may exceed the total tax that would have been due.

o Longer positive tail ~ truncated pareto or power distribution.

These factors the absolute amount, the relative amount, our confidence in the estimate and our view ofcollectability are important in objectively prioritising our relative concept of risk so that we can optimise the selectionof treatments and clients.

A balance must be struck by the organisation so that case selection prioritisation is defendable and repeatable (sothat our ranking of clients can be evaluated and improved.)

This approach of estimating client errors also raises the aspect of identifying and dealing appropriately withoverpayment or credit situations both in an absolute and relative sense.

To be a fair administrator the appropriate balance needs to be achieved in all things.

n

Tax/( Tax+Tax) 1

xx

xx

xx

x

x

x

x

xx

xx

x

x

xx

x

x

xx

xx

x

x

x

x

x

xx

xx

xx

x

x

xx

x x

x

x

xx x

x

x

x

xConfidence distribution in

Tax/( Tax+Ta x) Estimate

0


12/39


A Tax estimate gives us a view as to who may have evaded or avoided the most tax in absolute terms a criticalfactor for a revenue collection agency:

While a Tax/( Tax + Tax) estimate gives us a view as to who may have evaded or avoided most of their tax inrelative terms a critical factor for a revenue collection agency looking at serious non-compliance and aggressivetax planning.

$Y

n

Tax

xx

xx

xx

xx

x x

xx

xxx

xx

x

x

x

xx

x xx

x

x

x xx

x

xx

xx

xx

x x

xx

x

x

xx

x xx

x

xConfidencedistribution in

Tax Estimate

Tax: Who avoided or evaded the most tax?

n

Tax/( Tax+Tax) 1

x xx

xx

xx x

x x

xx

xxx

xx x

x

xx

xx x

xx

x

x xx

x

xx x x

xx

x x

xx

x

x

xx x

x

x

x

x Confidence distribution inTax/( Tax +Tax) Estimate

0

Severity Tax/( Tax + Tax) : Who avoided or evaded most of their tax?


13/39


We can now use these concepts to bring together a view of the relativities of revenue risks and use this to prioritiseour risks and candidate pool of clients for subsequent treatment:

o RankingRanking scores could be produced for both revenue, severity, confidence and collectability:A revenue score is produced by dividing the client Tax by the average Tax [ie Tax/Ave Tax].A severity score is produced by dividing the client severity [ Tax/( Tax + Tax)] by the average severity. (Ifrequired high, medium and low risk classes could be constructed in regard to the distribution high risk saythe highest 10%, medium risk the next 20% and low risk the next 70%.)

o Weighting revenue and severityA revenue focused score might take a 90/10 weighting of the revenue and severity. A severity focusedscore might take a 10/90 weighting of revenue and severity. Weighting approaches ensure that onedimension doesnt completely dominate the case ranking process. It may be appropriate to allowweightings to be varied to best meet the focus of the case selection run.

o Confidence levels & CollectabilityCf ( Tax) & P f ( Tax) could be used to discount Tax if this was considered necessary.

This approach to revenue risk scoring can provide an objective relative ranking of tax risks and cater for issuessuch as recidivism, losses and the promotion or association of non-compliance by others.

o Tax risks or issueA relative ranking of a tax risk or issue could be derived by the difference in relative of the Tax andseverity of clients affected. (Note that this does not extend into a quasi tax-gap analysis as Tax is notbased on representative samples. Tax will generally be derived from known cases and is thus biased.)

o Recidivism & currencyUse a multiyear score based on the sum ( ) the present value (PV) of Tax over say a standard timehorizon (say three years Y1, Y2 & Y3). Severity scores could be weighted by say (1xY3+0.7xY2 +0.3xY1)/2 to affect a reduced impact of earlier years compared to later years.

o LossesFor multi year scoring losses can be accommodated via the PV of the tax adjustment claw back over astandard time horizon (say three years).

o AgentsAgents could be looked in regard to both the of the Tax of their clients and also from a relative severityapproach. Relatively high scoring agents could be assigned for an appropriate treatment.

o Scheme promoters & schemesScheme promoters and schemes could be looked in regard to both the of the Tax of the participants inthe scheme and from a relative severity approach.

o Industry, occupation and locationIndustry, occupation or location aspects could be looked in regard to both the of the Tax of the clientsand also from a relative severity approach. Relatively high scoring industries, occupations of locationscould be assigned for an appropriate treatment.

o Whole of ClientThe values of estimated Tax can be summed for a client provided they derive from mutually exclusiverisks or they are counted on a first point of error basis. (ie Work related expenditure Tax and rentalproperty Tax and omitted income Tax can be summed whereas Tax for WRE and over-claimedexpenses Tax cannot (unless over-claimed expenses is constructed to exclude WRE claims)).

A whole of client / whole of product risk profile can then be constructed that places the clients and product revenuerisk in relative relationship to other clients / products.

In the ATO these broad client obligation risks are now expanded upon in the case management system down toreturn form/schedule label items.

The rationale for this is that incorrect reporting ultimately relates to aspects not correctly reported at a label item.Our tax return database consists of these label item values by form by client by reporting period and is generally acritical source of discriminate information used in case selection processes.


14/39


It should be noted that the summation of Tax cannot be used to construct a tax gap view like that of the IRS:

It is suggested that a better view of revenue risk is the risk to budgeted revenue flows from non-compliance ratherthan total tax gap concepts. The sum ( ) of Tax from successful cases (strikes) is the direct claw back fromcompliance activities included in the consolidated revenue received by Government and built into budget estimates.

LodgeRegister Report/Advise Account

Income Tax

GST

Excise

Super

(other FBT etc)

All Products(weighted sc ores)

Risk Score

Risk Score

Risk Score

Risk Score

Propensity t o Lodge On-time

Propensity t o Register Correctly

Propensity for Correct information

Propensity to Pay On-time & In full

Risk Score

Risk Score

Risk Score

Risk Score

Risk Score

Risk Score

Risk Score

Risk Score

Risk Score

Risk Score

Risk Score

Risk Score

Fully CompliantPropensity to Meet

All Obligations

Risk Score

Risk Score

Risk Score

Risk Score

Risk Attributes:Assessment History

Label AnalysisRatio AnalysisRefunds/Liabilities

Risk

Attributes:Registration HistoryProof of Identity

RiskAttributes:

Lodgment HistoryTimelinessA in

RiskAttributes:

Payment His toryDebt LevelTimeliness/ A in

Risk Score Risk Score Risk Score Risk Score Risk Score

Whole ofClient Score

Administrative Product

*NOTE: Client Scores can be further aggregated to support Industry, Occupation and Product Risk Scores for the whole cli ent population. This ability is critical to realising a more robust risk assess ment process and linki ng corporate risk rating with c ase-based risk rating.

(weighted score s)

Risk Score Risk Score Risk ScoreRisk Score Risk Score

Obligation ->

Client Risk Scores can be supported at the transaction process level, case level, whole of product level and whole of client level. All scores can be organised in a logical hierarchy of risk with the whole of client score at the highest level of aggregation and complexity*.

A client risk profile can be derived from revenue risk scores

Why Tax cannot be used for Tax Gap estimatesThe Tax predictions are derived from the analysis of successful cases (strikes) and are not drawn from arepresentative sample of the population. At low Tax, predictions the confidence interval would berelatively large compared to Tax. Ie The uncertainty in the tax revenue at risk would exceed the estimateof the revenue involved hence it cannot be used to project overall compliance levels accurately.

Diagrammatically: n

Tax

xx

xx

xx

xx

x x

xx

xxx

x

xx

x

x

xx

x

x

xx

x

x

xx

xx

xx

xx

xx

x

x

xx

xx

x

x

xConfidence distributionin Tax Estimate

0

However Tax predictions could be used to assist budget revenue at risk as they are based on past clawback.


15/39


Tax office revenue risks can be thought of as unanticipated movements in budgeted revenue from changes inunderlying compliance levels whereas forecast errors are a Treasury risk.

Estimates of Tax and changes in Tax could assist in the annual risk analysis process undertaken each year thatis used to guide mitigation activities.

Before turning to the use of analytic techniques for optimisation we should note that it is crucial that the qualitativeinformation from our intelligence gathering activities be combined appropriately with the quantitative intelligenceproduced via the use of analytics. Bring both views together produces a better result than using just one.

Intelligence views:

Revenue

v

Compliance movements =

ATO Risk

Actual claw back

of tax

Tax

Tax BudgetedRevenue

Predicted claw back

of tax

Predicted claw back

of tax

Less revenue

More revenue

Economic movements =

Trea surys Risk

Less revenue

More revenue

Revenue ConsequenceDistinguish betw een compliance behaviour changes and economic changesbeyond our influence of control

Actual Forecast

Descriptor: Aggressive Clients Those at the pointy end of the compliance model whose actions and or influence are such asto require intensive and immediate action or monitoring. People who rort either the tax laws(avoidance) or our administration of them (SNC). Suppression of known promoters and crimefamilies and the detection of new / emerging ones (needle in haystack work.)1-1 relationship once known. Client Profiles/Issue Profiles Tailored treatment> Cuts across all markets/products. Potential for some commonality of staffing & collection excluding technical.

Descriptor: Key Clients Involves the gathering of intelligence for large public groups whose economic importance issuch that the consequence of non compliance is very significant even if the likelihood is low.1-1 relationship. Client Profiles Tailored treatment> LB&I top 200 client groups / GST ILEC top 200 / Top Excise payers / Large Super Potential for common focus on key clients for all products. Common staff excluding technical.

Descriptor: Complex Clients Involves gathering intelligence for reasonably large complex, often private groups oftencontrolled by HWIs. Assembling a complete picture can be difficult and involve Internationalaspects (CFCs). Individual cases can range from high likelihood/high consequence (activemonitoring) to low likelihood/medium consequence (periodic monitoring).1-Few relationship Client Profiles/Issue Profiles Mainly tailored treatment> SB SME / LB&I bottom end (Next 1300 client groups) / GST SME / HWI Potential for common focus on high risk client groups.

Descriptor: Mass Clients Generally involves the analysis of large volumes of data and the use of quantitative techniquesand data matching to identify issues affecting significant numbers of clients. Individual casesare relatively speaking high likelihood/low consequence amenable to large scale mitigationprograms. Industry/Occupation/Region type approaches.1-Many relationship Issue Profiles Large treatment programs targeting segment issues> PTax/ SB Micro / GST Micro / Excise diesel credits Potential for common analytic/data matching staff working with technical experts.


16/39


Optimisation the art / science of making a system as effective as possibleIt is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts." Sherlock Holmes in A Scandal in Bohemia (1891)

Data! Data! Data!" he cried impatiently. "I can't make bricks without clay." Sherlock Holmes in The Adventure of the Copper Beeches (1892)

The primacy of data driven approaches in optimisation

Optimisation is (or should be) an objective data driven approach based on evaluation and analysis of the data.While ideally the scientific methodology should be used (control groups, double blind approaches etc), in abusiness environment this may not always be possible with the degree of rigour normally associated with goodscience. That said the conjunction of computer power, statistical techniques and machine learning algorithms hasnow allowed the development and deployment of robust approaches that can withstand the relatively poor dataquality/ second best data often associated with business systems.

Having set the scene and established a view of what non-compliance is and how we might consistently measurerevenue risk, we now look at how we determine the right treatment for the right client.

We will then examine approaches for case mix optimisation how does the mix of case types and numbers impactupon the revenue and can that be optimised.

Definecandidatepopulation

for risk

Select &rank

candidatepool

Determinetreatmentoptions

Selectrisk to be

treated

Select &allocate

candidatesto treatments

Measure andanalyse

effectiveness

Enhancetreatments

Enhancecandidateselectionprocess

Identifystrategic risks

Treatclients


17/39


Optimisation - Treatments available (Enhance treatments)It is a truism that if the only tool you have in your bag is a hammer then the solution to every problem starts to looklike a nail. All you can optimise in such situations is how you hit the object.

Systems where the only answer is a prosecution will tend to view the solution set to a compliance issue asprosecuting the right clients even though a prosecution might not be the right treatment to engender long termcompliant behaviour. (Indeed a side effect can be that an enforcement culture permeates the organisation ratherthan a client service ethic that realises clients get it wrong for a wide variety of reasons.)

Research on regulatory compliance models indicates that a range of treatments should be available to engender

long term voluntary compliance. An escalatory model is suggested to create an incentive for the client to movetowards a more engaged compliant behaviour set. This should take into account the facts and circumstances ofthe clients situation so as to treat the client in the most appropriate way:

o For example recidivist clients (those who repeatedly offend after treatment) would generally warrant adifferent treatment than a client detected making an error for the first t ime.

o Similarly those who promote non-compliance by others generally warrant different treatment to those whodont.

o Those in special positions of trust and influence in the tax system (eg key intermediaries, revenue authoritystaff, lawyers and accountants) generally warrant different treatment to those who arent in such positions.

o Those involved in avoidance schemes arent all the same. Clients with relatively low knowledge of the tax

system who enter schemes on the advice of their trusted advisor should not be treated the same as thosewho would be reasonably expected to have good knowledge (such as the advisor).

The causal factors involved in the non-compliance also factor into the appropriateness of the choice of remedy.

The causal factors for non-compliance could be as a result of:

o a difference of views a reasonably arguable position that differs from the ATO view,

o not being in a position to comply,

o honest mistake,

o ignorance,

o carelessness,

o negligence or

o deliberate intent.

It is important that our 'choice of remedy' be appropriate and defendable and that the mechanism to get to thedecision on the remedy be evidence based and repeatable.

These aspects can be brought together into an overall model or framework to view non compliant behavioursbased on the clients level of engagement with the regulatory system.

As the NZ IRD version of the compliance model indicates (See diagram on next page):

o Some treatments will apply to many clients at once and be rather general in nature such as educationmaterials available to the public.

o Other treatments will be client group specific, advice aimed at a particular industry or occupation orsegment.

o Finally some treatments are targeted at particular clients.


18/39


Compliance Models: New Zealand Tax Compliance Model

http://www.irs.gov/pub/irs-soi/04moori.pdf - paper by Tony Mossis & Michele Lonsdale NZ IRDTranslating the compliance model into practical reality

The New Zealand IRD model is a useful adaptation on the revenue authority compliance model presented in the1998 ATO Cash Economy Task Force Report:

Improving Tax Compliance in the Cash Economy, Page 58Second Report, ATO Cash Economy Task Force, 1998


19/39


These compliance models, or frameworks, posit an escalating set of remedies to observed client behaviours:

o For those trying and succeeding to do the right thing the majority of clients compliance is made assimple as possible. Information requirements are reduced and interactions are made as cheap and easyas is practical. (See for example the ATO Easier, Cheaper, More Personalised Change Program @http://www.ato.gov.au/content/downloads/Making_it_easier_to_comply_2005_06.pdf )

o For those trying, but not succeeding, in doing the right thing, education and advice is provided. This can begeneral, or aimed at specific client segment an industry or occupation group or some other discernableclient grouping. (See http://atogovau/corporate/content.asp?doc=/content/42628.htm on marketing andtaxation. Here descriptive analytics to identify market segments can assist.)

o Some clients may request assistance and advice and others may be targeted for a review whose outcomeis advice eg a record keeping review. Here predictive analytics can assist.

These interventions are generally relatively low cost mechanisms for enhancing voluntary compliance for clientswho are trying to do the right thing.

o A smaller number of clients will usually exist who for a variety of reasons appear to have carelessly,negligently or deliberately not complied. For these clients a common treatment is to audit the client todetermine the amount of the non-compliance and the reasons and, if considered appropriate, penalise theclient for not complying. The audit may be targeted at a specific issue or may be a more wide rangingexamination of the whole of the clients tax affairs.

o For those few clients that have relatively serious/aggressive non-compliance and other aggravating factors,the treatment may be to investigate with a view to prosecution. Due to the legal evidence gathering natureof these cases they tend to relatively resource intensive and costly.

In order to objectively optimise compliance treatments for clients we need to capture data that reflects relevantclient circumstances, the nature of the treatments used and the clients response to the intervention over time.

Rather than sequentially changing our treatments over time we can evolve our optimum (champion) treatment viathe use of controlled champion / challenger treatment groups where analytically similar clients are assigned todifferent (though still appropriate!) treatments in order to evaluate which treatment works best at engendering longterm voluntary compliance for the relevant client segment.

By creating control groups of clients assigned to different treatment pools, champion / challenger strategies can beevaluated and used to determine the best treatment for a client given their facts and circumstances as revealed inthe data.

To work best this needs to be a deliberate strategy with controlled data capture rather than an after the eventthought when much of the required data is no longer available for analysis.

Standard parametric (or non parametric if the control groups are small) statistical tests can be used to identify theoptimum treatment from the options tried.

Champion / challenger analysis:

Potentialactions

Today

Break even

Current ROItrajectory

Champion treatmentChallenger Treatment 1Challenger Treatment 2

KEY

TIME

R E T U R N O N I N V E S T M E N T Champion

Challenger 1

Challenger 2


20/39


Optimisation - Assignment of clients to those treatments (Enhance candidate selection process)Analytic approaches have a key role to play in the appropriate segmentation and allocation of clients into differenttreatment pools.

Ideally supervised learning approaches (predictive analytics) can be used to identify and assign clients into themost appropriate segment given their particular facts and circumstances as revealed in the data holdings by ananalysis of discriminate features in the data.

Descriptive analytic techniques (eg clustering) can help you to understand the circumstances of groups of clientsthat have difficulty in complying while supervised learning approaches 'mine' data on past interactions to discover

knowledge (associations patterns and trends) that allow us to better predict the response of a client to aninteraction.

Dataminers use statistical and machine learning algorithms squeeze the maximum informational value out of thedataset in question. Often the datasets have to be transformed and cleansed to get the most out of them. Somestatistical techniques for example work best with normally distributed, homoscedastic data.

Much of the 'art' of datamining stems from the knowledge and experience of the dataminer in recognising whattechnique is most appropriate to improve the signal to noise ratio (ie the identification of patterns in the data) in agiven circumstance.

With supervised learning approaches the algorithms try to identify discriminate features in the data that relate to atarget variable. (A discriminate feature is one that is positively or negatively correlated to the target variable in astatistically significant manner.)

For example clients that are considered at high risk of not responding to a letter or telephone call, because of pastrecidivism in relation to a matter as revealed by data on past interactions, may be assigned directly to a team for areview rather than to the lower cost work stream of an automated letter.

Some data mining algorithms can be conceptually quite simple, such as rule induction and decision trees, throughto the more difficult to explain neural network (essentially a black box learned weighting approach), support vector(a machine learning partitioning approach) and random forest approaches (building a large number of decisiontrees from random samples of the data that then vote on the classification outcome).

Decision Tree of Rulesderived from data to assign scores

Treatment Audit

Treatment Review

Call

Letter Y

Letter X

1000950900850800750700650600550500450400350300250200150100500

Score

So we can personalise our treatmentstrategies to the client

Decision Tree

Regression

DM NeuralRule Induction

Neural Net

Decision Tree

Regression

DM NeuralRule Induction

Neural Net

In fact scores are likely to be done via severalmodels voting together Ensembles.

Descriptive and predictive analytics are a relatively recent field of business intelligence, enabled by theconvergence of computing power, machine learning approaches and statistical understanding.

Within the ATO we have a core group of ~16 data miners (most with PhD's in quantitative sciences), who use avariety of software tools such as SAS Enterprise Miner and SAS JMP, NCR Teradata Warehouse Miner and morerecently open source tools like the excellent 'Rattle' to assist them in their work.

I'll provide some examples from the use of Rattle, written by one of our senior data miners to harness a variety ofopen source statistical packages available in R, to give a feel for some of the advanced analytic techniques nowavailable in easy to use software. Rattle is available from http://rattle.togaware.com/


21/39


Rattle open source data mining software:

Exploring data relationships with Rattle to identify possible features for selection modelling

Some more examples of the analysis methodologies supported in Rattle are contained in an Annex to this paper.

20 40 60 80

0 . 0

0 . 4

0 . 8

P r o p o r t

i o n < = x

Adjusted

All01

Cumulative Age

Rattle 2006-10-04 12:49:20 Stuart

0 e+00 2 e+05 4 e+05

0 . 0

0 . 4

0 . 8

P r o p o r t

i o n < = x

Adjusted

All01

Cumulative Income

Rattle 2006-10-04 12:49:20 Stuart

0 500 1000 2000

0 . 0

0 . 4

0 . 8

P r o p o r t

i o n < = x

Adjusted

All01

Cumulative Deductions

Rattle 2006-10-04 12:49:20 Stuart

0 20 40 60 80 100

0 . 0

0 . 4

0 . 8

P r o p o r t

i o n < = x

Adjusted

All01

Cumulative Hours

Rattle 2006-10-04 12:49:20 Stuart

0 40000 80000 120000

0 . 0

0 . 4

0 . 8

P r o p o r t

i o n < = x

Adjusted

All01

Cumulative Adjustment

Rattle 2006-10-04 12:49:20 Stuart

PrivateConsultantPSLocalNA'sPSStateSelf EmpPSFederalUnemployed

0 400 800 1200

Distribution of Employmen

Rattle 2006-10-04 12:49:21 StuarFrequency

Adjusted

All01

HSgradCollegeBachelorMasterVocationalYr11Yr10AssociateYr7t8ProfessionalYr9DoctorateYr12Yr5t6Yr1t4Preschool

0 200 400 600

Distribution of Education


Adjusted

All01

CivilUnmarriedDivorcedWidowedSeparatedAbsentMarried

0 200 400 600 800

Distribution of Marital

Rattle 2006-10-04 12:49:21 StuartFrequency

Adjusted

All01

ProfessionalExecutiveClericalSalesRepairServiceMachinistNA'sCleanerTransportFarmingSupportProtectiveHomeMilitary

0 50 150 250

Distribution of Occupatio


Adjusted

All01


22/39


Some graphic examples from some of the analytic modelling and evaluation techniques available in Rattle:

Descriptive analytics.Segmentation by decision tree against target variable (adjustment).

K-means clustering Find k clusters in the data (3 in this instance)

Marital

0755 cases

93.8%

2 Education

Occupation

0322 cases

73.6%

12 Hours < - > 38

021 cases

71.4%

26 Age < - > 33.5

022 cases

63.6%

54

161 cases

73.8%

55

1219 cases

74.4%

7

Decision Tree audit-csv.txt $ Adjusted

Rattle 2006-10-02 16:23:45 Stuart


23/39


24/39


Predictive analytics.The risk ranking of clients by predictive analytics allows a caseload to be prioritised so that optimal revenue-resourcing decisions might be made. In this example we can see that at 40% of the caseload the algorithm isreturning 80% of the revenue.

Risk lift charts are a way of 'seeing' the improvement that an analytic model can produce and make the trade-offbetween caseload and revenue more obvious to management.

0 20 40 60 80 100

0

2 0

4 0

6 0

8 0

1 0 0

Caseload (%)

P e r f o r m a n c e

( % )

28%

RevenueAdjustmentsStrike Rate

Risk Chart rf audit-csv.txt [test] Adjustment

Rattle 2006-10-02 16:27:39 Stuart


25/39


The area under the ROC (receiver operating characteristic the true positive/false positive ratio as thediscriminating threshold changes) curve gives us a view of the relative performance of different selection models.

In this example we can see that in this instance the Support Vector Model (ksvm) slightly outperforms the RandomForest Model (rf) and the Logistic Regression model (glm) and these outperform the single Decision Tree (rpart)which significantly outperforms the simpler Gradient Boost Model (gbm).

Where different models perform significantly better at different parts of the distribution, ensemble approaches canbe used to harness the right model for the right part of the distribution.

Case selection models need to be evaluated from a number of perspectives. Better selection models select moretrue positives and true negatives and minimise false positives and false negatives.

Note that the strike rate metric [true positives/(true positives+false positives)] is like an iceberg - it is what you cansee and measure but it is only part of the picture as it does not provide details (except by inference) regardingthose not selected (false negatives and true negatives). Consider the following Taylor-Russell diagrams:

Relatively poor selection model: Better selection model:


26/39


Optimisation - Case mixOnce we have our treatments and case selection optimised a final factor for consideration is how the overall casemix impacts upon revenue collections and voluntary compliance.

It would generally be serendipitous if an organisations case mix optimised its revenue yield. Analytic simulationmodelling techniques such as linear programming can provide insights to management regarding the optimum mixof case types and their revenue and resource impacts. Such modelling can provide insights into key parametersand their sensitivity to change.

A simple two case type example will be used to demonstrate how linear programming can be utilised in the

optimisation of case mix by a revenue authority.

Please read the following in conjunction with the graphs on the following two pages

Assume that there are two case types that staff can work on an Income Tax Case Audit and a GST Audit. For thisexample let us say that an Income Tax Audit on average returned $500 in revenue and a GST Audit returned $200.

Assume that: we have three levels of staff involved in the casework EL2's, APS6's and APS4's and that we havefive EL2's, four APS6's and four APS4's all of whom can work for 40 hours per week. An Income Tax Audit takeson average two hours of an EL2's time plus two hours of an APS6 and one hour of an APS4. A GST Audit takeson average an hour of an EL2's time plus an hour of an APS4. In this example we will say an EL2 costs $50 perhour, an APS6 costs 20$ per hour and an APS 4 costs $15 per hour.

The linear optimisation question is what case mix optimises net revenues?

We can see that 2 Income Tax Audits = 5 GST Audits in terms of revenue. This is the slope of our revenue curve.

We can further see that we have five constraints:o The number of Income Tax Audits must be greater than or equal to zero (cannot have negative case

numbers!)o The number of GST Audits must also be greater than or equal to zero.o The EL2 effort on cases must be less than available EL2 effort hours (5 x 40 = 200)o The APS6 effort on cases must be less than available APS6 effort hours (4 x 40 = 160)o The APS4 effort on cases must be less than available APS4 effort hours (4 x 40 = 160)

We can see that:o The EL2 constraint translates into a maximum of 100 Income Tax Audits (@ 2 hrs each) or a maximum of

200 GST Audits (@ 1 hr each)o The APS6 constraint translates into a maximum of 80 Income Tax Audits (@ 2 hrs each); ando The APS4 constraint translates into a maximum of 160 Income Tax Audits (@ 1 hrs each) or 160 GST

Audits (@ 1 hr each).

Graphically this produces a feasibility within which the actual case mix must sit. At the boundary of the feasibilityspace an optimum case mix exists.

When we place our revenue curve onto the diagram and move it to its outer most point on the feasibility space weidentify our optimum case mix: 80 Income Tax Audits and 40 GST Audits. Our 'binding' constraints are the numberof effort hours available of our EL2's and APS4's. We can see at the optimum point we have spare APS6 capacity.

If we introduce a coverage constraint (must do at least 60 GST Audits) we can see that it reduces the feasibilityspace and creates a new optimum point. The binding constraints at this point are the EL2 effort time and the GSTcoverage constraint. We have spare APS6 and APS4 time.

This was of course a very simple example, however the methodology holds for greater numbers of case types andresource types it just cant be shown graphically. (An example spreadsheet for more case types is included inannex that uses the Excel solver add-in.)

This type of analysis can be further enhanced by the modelling distributions rather than fixed amounts. (eg thedistribution of revenue, the distribution of effort time etc by case type. Software such as @Risk can be used for thispurpose. (See @Risk at http://www.palisade.com )


27/39



28/39



29/39


Other matters

Prioritising analytic workThere are a wide range of uses that competent analytic capabilities can be used on within a revenue authority itis an in demand skill and experience set

It is relatively easy for such staff to be deployed on research type projects to discover some interesting insightabout our client base. However this may not be the best method of prioritising tasks for the analytic capability.

When looking at where to apply analytic techniques to optimise revenue, a useful approach may be to consider

either revenue gains or staff savings by the relative strike rate being achieved. (Apriori, an area with a low strikerate has the largest potential to improve. By looking at the revenue being collected or the staff utilised by 1-Sr, arectangle is produced whose relative size indicates a potential room for improvement.) In this example as largebusiness already have a relatively high strike rate and fewer staff there is smaller room for efficiency gains.

In this hypothetical example we can see that from an efficiency viewpoint the cash economy has a relatively highernumber of active compliance staff and a lower strike rate than large business hence analytic enhancements tothe strike rate would provide a greater efficiency gain. From a revenue perspective it is less clear cut.

$

1-Sr

LargeBusiness

Cash Economy

Effectiveness Revenue Focus

FTE

1-Sr

LargeBusiness

Cash Economy

Efficiency Staff Focus


30/39


31/39


It follows that an improvement in the strike rate from 10% to 15% (a 50% improvement) does not have the samerevenue impact as a change from 90% to 95% (a 5.6% improvement).

At higher strike rates the same percentage strike rate change results in fewer additional productive cases than itdoes at lower strike rates.

Productive to Non Productive Caseload by Strike Ratewith a 3:1 Effort Time Differential - Fixed Staffing

30% Base Strike Rate

0

500

1000

1500

2000

S r 9 5

% 8 5

% 7 5

% 6 5

% 5 5

% 4 5

% 3 5

% 2 5

% 1 5

%

Strike Rate

N u m

b e r o

f C a s e s

C o m p

l e t e d

Productive Cases Non Productive

The chart above plots the effects of a 3:1 effort time differential as strike rates change with fixed staffing on casenumbers.

With fixed staffing as the strike rate varies downwards the number of cases able to be completed grows non-linearly in accordance with the effort time differential between a productive and non productive case and thedifference in strike rate: Sr x /(1+[tPC/tNPC-1]*Sr x )

If all cases are productive the number of cases able to be completed = Number of Staff ( nS )/Productive Case EffortTime ( tPC ).

If all cases are non productive then the maximum number of cases able to be completed ( TCmax ) = Number ofStaff ( nS ) /Non Productive Case Effort Time ( tNPC ).

For a particular strike rate ( Sr ) the number of productive cases ( nPC ) is given by:nPC = TCmax*Sr/(1+[tPC/tNPC-1]*Sr)

and the number of non productive cases ( nNPC ) is given by:nNPC=TCmax-(tPC/tNPC)/nPC

The revenue gain from a change in strike rates is given by:R$ = nPC 2 *r$PC-nPC 1*r$PC where nPC x = TCmax*Sr x /(1+[tPC/tNPC-1]*Sr x )

(ie the change in number of productive cases times the revenue per productive case assuming that revenue per productive case does notchange significantly as case numbers or the strike rate changes.)


32/39


Fixed revenue target scenario If the revenue target is considered fixed then the impact of increasing strike rates can generate staff efficiencysavings for the revenue authority. (Fewer staff are needed for the same number of productive cases.)

Productive to Non Productive Caseload by Strike Ratewith a 3:1 Effort Time Differential - Fixed Revenue

30% Base Strike Rate

0

1000

2000

3000

4000

5000

6000

Sr 95% 85% 75% 65% 55% 45% 35% 25% 15%

Strike Rate

N u m

b e r o

f C a s e s

C o m p

l e t e d

Productive Cases Non Productive Cases

The chart above plots the effects of a 3:1 effort time differential as strike rate changes on case numbers with afixed revenue target and variable staffing.

With fixed revenue targets as the strike rate varies downwards the number of cases required to meet the revenuetarget grows non-linearly as we must pump more cases through to maintain the revenue amount.

The number of productive cases required for the revenue is fixed irrespective of the strike rate. It is the totalnumber of cases needed to be completed, and hence the staffing, that is varied to maintain the revenue target.

In this situation if all cases are productive, the number of cases completed at this strike rate equals the revenuetarget ( R$ ) divided by the average productive case result ( $rPC ). (eg So a $3 million revenue target at an averageof $10,000 per case requires the completion of 300 cases.)

The number of productive cases is fixed at this level irrespective of the strike rate so: nPC = R$/$rPC

For a particular strike rate ( Sr ) the number of non productive cases ( nNPC ) is given by:nNPC=(nPC-Sr*nPC)/Sr

and the number of staff need is given by:nS=nPC*tPC+nNPC*tNPC

(ie the time spent on productive cases plus the time spent on non productive cases)

The staff saving from a change in strike rates is given by:

nS = nNPC 2 *tNPC- nNPC 1*tNPC where nNPC x =(nPC-Sr x *nPC)/Sr x\

(ie the change in number of non productive cases times the effort used on those cases - assuming that effort time per non productive case doesnot change significantly as case numbers or strike rates change)


33/39


Traditional versus analytic driven case selection an example

Traditional Case Selection Analytic case selection

Subject matter experts use their experience to createcase selection rules to filter non-compliant clients outfrom the broader population in respect of particularrisks. OLAP type approaches slice n dice.

They focus on client features that they believe assist

in revealing whether a client is compliant or not.

These rules are often refined overtime to enhancethe strike rate.

The rules produced are generally subjectivelyweighted to derive a client risk scores for workprioritisation.

Eg (hypothetical example)o If client has WRE claim > $1,500 and Uniform >

$500 then risk score = 4o If client has motor vehicle claim > $5,000 then

risk score = 2o If client has self education claim > $2,000 then

risk score = 3o Add client risk scores to produce total risk scoreo Select for review clients with total risk score > 8

Rules produced this way (top down) have anadvantage of being known to a subject matterexpert and are more explainable but they aregenerally not optimal.

Past cases of non-compliance are divided intosuccessful cases (where relevant non-compliance wasfound) and unsuccessful cases (where it wasnt). Thedata set is further divided into a training set (used tobuild the analytic case selection model) and a validationset to test that the model works.

Working with subject matter experts the analyticsmodeller identifies client features that appear to beassociated with non-compliance. These are testedstatistically to see if they are associated with non-compliance.

The training data set is then essentially regressedagainst the target set of successful cases to produce arisk scoring algorithm that optimises the probability ofpredicting a successful case from the data set.

The algorithm is then tested against the validation dataset to see how it performs.

The rules produced are weighted by the algorithm toproduce a client risk score for work prioritisation:

Rules produced this way (bottom up) sometimes surprisesubject matter experts and may require effort tounderstand and explain. (Though generally a simpledecision tree can be retro fitted to provide explanatorypower to the output of complex model that produces ahigher strike rate.)

The rules produced from such data driven approacheswill usually outperform rules derived from subjective

subject matter view on discriminate features and theirweighting.


34/39


Optimising capabilityThe analytic methods outlined in this paper are, of course, only part of the picture and will not greatly assist inoptimising compliance if the capabilities behind the treatment strategies are also not operating optimally.

If compliance staff skills, knowledge and experience are not up to the task then the treatment will not be optimal.Staff cannot be overzealous, incompetent or corrupt in the performance of this work and continuousimprovement/quality assurance measures need to be in place to ensure that client treatment is optimally delivered.

For example the use of:o Performance standard & exception monitoringo Peer/team leader quality reviews

o Periodic performance reviews & client perception surveys

o Dual sign-off procedures (eg case officer-team leader)

o Escalation (& de-escalation) procedures

o Case call-over procedures so that all significant/old cases are reviewed by someone outside the team

o Back-log procedures that are triggered when the median age of the case pool reaches a particularthreshold above the standard.

ConclusionsThe development of risk and intelligence approaches is reaching a new level of sophistication.

The convergence of computing power, machine learning approaches and statistical algorithms is providing revenueauthorities with the capability to be much more personalised in the targeting of treatments to clients.

The successful integration of these techniques with robust management approaches to the use of strategicintelligence is enabling revenue authorities to be much more purposeful in how they treat the risks that challengethem.

Harnessing this opportunity requires us to make the transition to a more quantitative evidence based approach to

management.

The challenge is to make it happen.

After all the best way to predict the future is to invent it. 9

Stuart HamiltonNovember 2006

9 Alan Kay. 2003 Turing Prize winner. Inventor of many of the computer GUI interface aspects we take for grantedtoday.


35/39


Annexes

Scoring using TaxLet us look at how a clients risk score might be computed using the Tax concept:

Clients A, B & C have paid the following tax over the last three years:-

Client A Y3: 3,000, Client B Y3: 3,000 Client C Y3: 3,000Y2: 3,000 & Y2: 3,000 Y2: 3,000

Y1: 3,000 Y1: 3,000 Y1: 0

A predictive algorithm estimates that the following tax may have been avoided or evaded (ie at risk):-Client A Y3: 2,700, Client B Y3: 2,700 Client C Y3: 0

Y2: 2,700 & Y2: 0 Y2: 2,700Y1: 2,700 Y1: 0 Y1: 2,700

Values & Scores Multi Year (Y3 -> Y1)Revenue average = (8,100+2,700+5,400)/3 = 5,400, Severity average = (0.474+0.231+0.474)/3 = 0.393

Revenue 8,100 [1.50] Client B 2,700 [0.50] Client C 5,400 [1.00]Severity 0.474 [1.21] 0.231 [0.59] 0.474 [1.21]

where 1.5 = 8,100/5,400 & 0.474 = 8,100/(8,100+9000) & 1.21 = 0.474/0.393 for Client A

A single weighted 90/10 Revenue/Severity Multi Year Risk Score then gives:Client A 1.47 [0.9*1.50+0.1*1.21] Client B 0.51 Client C 1.02

These can be transformed further into a score from 1.000 to 0.000 by setting the highest score equal to 1.000 andscaling the other scores off this:

Client A 1.000 [1.47/1.47] Client B 0.347 [0.51/1.47] Client C 0.694 [1.02/1.47]

High score high risk so we select Client A, then Client C, then Client BNote this example did not use PV, weighting or discounting for clarity


36/39


Some additional Rattle analysis output

It is important that our understanding of our clients move beyond simplistic data analysis approaches based onsummary descriptive statistics such as means, medians, modes and standard deviations.

For example; in a positively skewed distribution, as most are in tax:o A mean estimate of a clients income will overstate income more times than it understates it.o A median estimate will overstate and understate it with equal frequency.o A mode will understate income more times than it overstates it.

In all of these cases a single point estimate based on the mean, median or mode will be significantly more incorrectthan an estimate produced by modelling the income using discriminatory variables. It is the difference betweentrying to sum up a song with a single note versus having an MP3 version of it (compressed to the sounds thatmatter and so still understandable).

We need to reach back into the distributions to understand the variance between client groups and find thevariables that matter (to answer the problem we are looking at.

With the computing power and algorithms now available we should be striving for the highest practical degree ofaccuracy in our estimates. Anything less is almost bordering on professional negligence for an analyst.

All 0 1

2 0

4 0

6 0

8 0

Adjusted

Distribution of Age

Rattle 2006-10-04 12:48:06 Stua

All 0 1 0

e +

0 0

4

e +

0 5

Adjusted

Distribution of Income

Rattle 2006-10-04 12:48:06 Stua

All 0 1

0

1 0 0 0

2 5 0 0

Adjusted

Distribution of De duction

Rattle 2006-10-04 12:48:06 Stua

All 0 1

0

4 0

8 0

Adjusted

Distribution of Hours

Rattle 2006-10-04 12:48:07 Stua

All 0 1

0

6 0 0 0 0

1 4 0

0 0 0

Adjusted

Distribution of Adjustmen

Rattle 2006-10-04 12:48:07 Stua

Private NA's Unemployed

0

5 0 0

1 5 0 0

AdjustedAll01

Distribution of Employmen

Rattle 2006-10-04 12:48:07 Stua

HSgrad Y r10 Y r12

0

2 0 0

5 0 0

AdjustedAll01

Distribution o f Education

Rattle 2006-10-04 12:48:07 Stua

Civil Widowed

0

4 0 0

8 0 0 Adjusted

All01

Distribution of Marita l

Rattle 2006-10-04 12:48:07 Stua

Pr of es s ion al NA 's Home

0

1 0 0

2 5 0 Adjusted

All01

Distribution of Occupatio

Rattle 2006-10-04 12:48:07 Stua


37/39


Spreadsheets used in this paper

Linear programming

Case TypeOptimisationV5.xls (...

Impact of strike rate changes on case numbers

case_work_calcs_v2.xls (46 KB)...

@Risk from Palisade can be used with these spreadsheets to provide distributional input rather than fixed values.This enables simulations such as linear programming to be more realistic in their projections.

See http://www.palisade.com/downloads/pdf/Palisade_RISK_0604PA.pdf

Open source data mining software:Rattle (r based) http://rattle.togaware.com/ Weka (java based) http://www.cs.waikato.ac.nz/ml/weka/ Yale (java based) http://rapid-i.com/content/blogcategory/10/21/lang,en/ (connects to Weka)Knime (eclipse/java based) http://www.knime.org/ (connects to Weka)


38/39


Summary of some analytic methodologies that might be used in optimisation

Another view of the end to end process from risk identification to case outcomes:

Willing to do theright thing

Try to, but dontalways succeed

Dont wantto comply

Have decidednot to comply

Use the full forceof the law

Assist tocomply

Deter bydetection

Make it easyAttitude to

ComplianceComplianceStrategy

Createpressure

down

Createpressure

down

Determine rightexperience

Understand thecustomer

Deliver the rightexperience

Optimise understanding of client> Descriptive analytics>> Distibution mean, median, mode, kurtosis, sk ewness> Exploratory data analysis understand the data>> Cumulative distributions>> Analysis of variance>> Box whisker Interquartile range & outliers

>> Missing values data cleaning>> Transformations / Normalisation>> Principle components>> Benfords analysis>> Dendograms>> Time series analysis> Clustering how do clients naturally group in the data>> k-means>> Hierarchical>> Self organising feature maps

Optimise treatments> Champion / challenger approach> Control groups

Optimise treatment selection> Predictive analytics risk s core modelling>> Decision tree

>> Random forest>> Logistic regression>> Support vector machine>> Neural networks

Optimise delivery > Simulation modelling decision support> Queuing methodologies> Linear programming> Sensitivity modelling of parameters

Research:

> Intelligence (Qualitative)Strategic (What to look at)Operational (Who to look at)Tactical (What is needed to complete c ase)

+> Analytics (Quantitative)

Descriptive analytics

Delivery:

Channel management +Case Management (mult-activities) +Work Management (single activities)

Investigate / prosecute

Audit / Penalise

Review / Advise

Educate / Market

System changesSimplificationPrepopulation


39/39

Taxpayers

CandidatePo ulation

RankedCandidates

Cases

Results

RiskPrioritisation

ResourceAllocation

DemandManagement

RiskIdentification

RiskTreatment

Development

Model &TreatmentStrategy

Coverage& Revenuetargets

Modelling

OperationaliseAnalytics

Seibel Work &Case Mgmt

Optimise treatment & candidate selection

Optimise risk priority & case mix selection

Date post:	30-May-2018
Category:	Documents
Upload:	stuart-hamilton
View:	216 times
Download:	0 times

sing Compliance - Role of Analytic Techniques

Documents