Classifying Attributes with Game-theoretic Rough Sets

Nouman Azam and JingTao Yao

Department of Computer Science University of ReginaCANADA S4S 0A2

azam200n@cs.uregina.ca jtyao@cs.uregina.cahttp://www.cs.uregina.ca/~azam200n http://www.cs.uregina.ca/~jtyao

Rough Sets

• Sets derived from imperfect, imprecise, and incomplete data may not be able to be precisely defined.

• Sets have to be approximated.

• Rough sets introduces a pair of sets for such approximation.– Lower approximation– Upper approximation

Visualizing Rough Sets• Let• Lower approximation.

• Upper approximation.

• Positive Region.

• Boundary.

• Negative region.

Probabilistic Rough Sets• Defines the approximations in terms of conditional

probabilities.– Introduces a pair of threshold denoted as (α, β) to determine

the rough set approximations and regions

– Lower approximation

– Upper approximation

– The three Regions are defined as

A Key Issue in Probabilistic Rough Sets

• Two extreme cases.– Pawlak Model: (α, β) = (1,0)

• Large boundary. Not suitable in practical applications.

– Two-way Decision Model: α = β• No boundary: Forced to make decisions even in cases of

insufficient information.

• Determining Effective Probabilistic thresholds.

• The GTRS model.– Finds effective values of thresholds with a game-

theoretic process among multiple criteria.

Game-theoretic Rough Set ModelUtilities for Criterion C1

0.5

0.7

0.9

0.6

0.3

0.2

(α1, β1)

(α2, β2)

(α6, β6)

(α3, β3)

(α4, β4)

(α5, β5)

Rankings based on C1

1 2 3 4 5 6

Game-theoretic Rough Set ModelUtilities for Criterion C2

0.7

0.1

0.5

0.6

0.8

0.3

(α1, β1)

(α2, β2)

(α6, β6)

(α3, β3)

(α4, β4)

(α5, β5)

Rankings based on C2

1 2 3 4 5 6• Dilemma:

– Ranking of C1 vs C2– Which pair to select

Game Theory for Solving Dilemma

• Game theory is a core subject in decision sciences.– Prisoners Dilemma.

• A classical example in Game Theory.

A Game-theoretic Rough Set Approach

• Obtaining Probabilistic threshold with GTRS.– An (α, β) pair is determined with game-theoretic

equilibrium analysis.

C1

C2

Attribute Types in Rough Sets• Reduct.

– A minimal set of attribute set having the same classification ability as the entire attribute set.

– Generally there may exist multiple reducts.

• Core attribute.– An attribute appearing in every reduct.

• Reduct attribute.– An attribute appearing in at least one reduct.

• Non-reduct attribute.– An attribute that does not appear in any reduct.

Limitations of Existing Methods

• For classifying attributes we need to find most, if not all, reducts.

• Existing methods for finding multiple reducts.– Commonly involve an iterative process.– Each iteration involves a sub-iterative process for

searching a single reduct.– An attribute may be processed multiple times in

different iterations of these methods.

A GTRS Based Approach

• We try to find an additional mechanism for classifying attributes.– Processing each attribute once to avoid extensive

computations.

• A GTRS based solution– Interpreting the classification of a feature as a

decision problem within a game.

Attribute Classification with GTRS

• Formulating problems with GTRS model requires to,– Identify the players.– Identify the strategies of players.– Determine the payoff functions.– Implement a competition.

J T Yao Incorporating Game Theory in Feature Selection for TC 14

• Players were selected as measures of an attribute importance. – Each measure analyzes an attribute for its

importance.– A case of two player game was considered.

• Two strategies were formulated for each player.– Accepting an attribute, denoted as – Rejecting an attribute, denoted as

Players and Strategies

Payoff Functions

• Let represents a particular measure.– The value of corresponding to an attribute A, may

be given as,

• Notation for a payoff function.– Payoff of measure , performing action j, given

action k of his opponent is denoted as,

• The payoff functions of a player in four different situations of a game are calculated as,

Obtaining Attribute Classification• The game may result in three possible

outcomes.– Both players choose to select– One of the players choose to select– None of the players choose to select.

• Attribute classification: An attribute is considered as, – core, when both players choose to select.– reduct, when one of the players choose to select.– Non-reduct, when none of the players select.

Attribute Classification Algorithm

A Demonstrative Example

• Core = {e}• Reduct = {a,c,e}• Non-reduct = {b,d,f}

The Measures in the Game• Conditional Entropy.

• Attribute Dependency.

Payoff Tables

• The bold cell represents Nash equilibrium.– None of the players can achieve a higher payoff

given their opponents chosen action.– The attribute is classified as core, since both

measures choose to select, i.e. core = {e}.

Payoff Tables (Cont.)

• The actions of players classify the above attributes as reduct attributes.

• Equilibrium analysis for attribute b, d, f suggest their classification as non-reduct attributes.

Conclusion

• Limitations of existing approaches.– Extensive computation due to multiple processing of

individual attributes.

• GTRS based method.– Interprets the classification of attributes as a game

among multiple measures of attribute importance.

• Importance of the method.– Each attribute is processed only once in obtaining

the classification of attributes.