Algorithm Selection for PreferredExtensions Enumeration

Federico Cerutti, Massimiliano Giacomin, Mauro Vallati

COMMA-2014 — Wednesday 10th September, 2014

Background on Dung’s AF

Current Approaches for Preferred Extensions Enum.Features

Empirical ResultsConclusions

Background

Definition

Given an AF Γ= ⟨A,R⟩, with R⊆A×A:

a set S⊆A is conflict–free if @ a,b ∈ S s.t. a→ b;

an argument a ∈A is acceptable with respect to a set S⊆A if ∀b ∈A s.t.b→ a, ∃ c ∈ S s.t. c→ b;

a set S⊆A is admissible if S is conflict–free and every element of S isacceptable with respect to S;

a set S⊆A is a complete extension, i.e. S ∈ ECO(Γ), iff S is admissible and∀a ∈A s.t. a is acceptable w.r.t. S, a ∈ S;

a set S⊆A is the grounded extensions, i.e. S ∈ EGR(Γ), iff S is the minimal(w.r.t. set inclusion) complete set;

a set S⊆A is a preferred extension, i.e. S ∈ EPR(Γ), iff S is a maximal (w.r.t. setinclusion) complete set.

Background on Dung’s AF

Current Approaches forPreferred Extensions Enum.

FeaturesEmpirical Results

Conclusions

AspartixM: [Dvorák et al., 2011]

Expresses argumentation semantics in Answer Set Programming(ASP);Tests for subset-maximality exploiting the metasp optimisationfrontend for the ASP-package gringo/claspD;Database of the form:

{arg(a) | a ∈A} ∪ {defeat(a,b) | ⟨a,b⟩ ∈R}

Example of program for checking the conflict–freeness:

πcf = { in(X)← not out(X), arg(X);out(X)← not in(X), arg(X);← in(X), in(Y ), defeat(X,Y )}.

NAD-Alg: [Nofal et al., 2014]

Several improvements in [Nofal et al., 2014]

PrefSAT: [TAFA2013]

ΠΓ is a boolean formula such that eachsatisfying assignment of the formulacorresponds to a complete extension.

SCC-P: [KR2014]

SCC-P: [KR2014]

SCC-P: [KR2014]

AspartixM // NAD-Alg // PrefSAT [DARe2014]

% Solved Average CPU-TimeDensity 25% 50% 75% 25% 50% 75%

AspartixM 98.3 100.0 100.0 56.5 14.7 10.0NAD-Alg 100.0 100.0 100.0 18.9 0.2 0.2PrefSAT 100.0 100.0 100.0 5.1 1.6 2.2

% Solved Average CPU-TimeDensity RAND RAND

AspartixM 98.9 34.0NAD-Alg 93.9 70.6PrefSAT 100.0 4.2

PrefSAT // SCC-P [KR2014]720 AFs varying |SCCΓ| in 5:5:45. Size of SCCs N(µ= 20 : 5 : 40, σ = 5);

attacks among SCCs N(µ= 20 : 5 : 40, σ = 5)

40

50

60

70

80

90

100

0 10 20 30 40 50

|SCCΓ|

IPC value (normalised) for SCC-P and SAT-P when 5 ≤ |SCCΓ| ≤ 45

SCC-P SAT-P

For |SCCΓ|= 35, Md(SCC-P) = 8.81,Md(SAT-P) = 8.53, z=−0.35, p= 0.73;

Background on Dung’s AFCurrent Approaches for Preferred Extensions Enum.

FeaturesEmpirical Results

Conclusions

Features from an Argumentation Graph Γ= ⟨A,R⟩— improvement from [ECAI2014]

Directed Graph (26 features)

Structure:

# vertices ( |A| )# edges ( |R| )# vertices / #edges ( |A|/|R| )# edges / #vertices ( |R|/|A| )densityaverage

Degree: stdevattackers max

min#averagestdevmax

SCCs:

min

Structure:

# self-def# unattackedflow hierarchyEulerianaperiodic

CPU-time: . . .

Undirected Graph (24 features)

Structure:

# edges# vertices / #edges# edges / #verticesdensity

Degree:

averagestdevmaxmin

SCCs:

#averagestdevmaxmin

Structure: Transitivity

3-cycles:

#averagestdevmaxmin

CPU-time: . . .

How Hard is to Get the Features?

Direct Graph Features (DG) Undirect Graph Features (UG)Class CPU-Time # feat Class CPU-Time # feat

Mean stdDev Mean stDevGraph Size 0.001 0.009 5 Graph Size 0.001 0.003 4Degree 0.003 0.009 4 Degree 0.002 0.004 4SCC 0.046 0.036 5 Components 0.011 0.009 5Structure 2.304 2.868 5 Structure 0.799 0.684 1

Triangles 0.787 0.671 5

Average CPU-time, stdev, needed for extracting the features of a given class.

Background on Dung’s AFCurrent Approaches for Preferred Extensions Enum.

Features

Empirical Results

Conclusions

Protocol: Some Numbers

|SCCΓ| in 1 : 100;|A| in 10 : 5,000;|R| in 25 : 270,000 (Erdös-Rényi, p uniformly distributed) ;Overall 10,000 AFs.

Cutoff time of 900 seconds (value also for crashed, timed-out or ranout of memory).

EPMs both for Regression (Random forests) andClassification (M5-Rules) using WEKA;Evaluation using a 10-fold cross-validation approach on a uniformrandom permutation of instances.

Result 1: Best Features for Prediction

Solver B1 B2 B3AspartixM number of arguments density of directed graph size of max. SCCPrefSAT density of directed graph number of SCCs aperiodicityNAD-Alg density of directed graph CPU-time for density CPU-time for EulerianSCC-P density of directed graph number of SCCs size of the max SCC

Determined by a greedy forward search based on the Correlation-based Feature Selection (CFS)attribute evaluator.

AF structure SCCs CPU-time for feature extraction

Result 2: Predicting (log)Runtime

RSME of Regression (Lower is better)B1 B2 B3 DG UG SCC All

AspartixM 0.66 0.49 0.49 0.48 0.49 0.52 0.48PrefSAT 1.39 0.93 0.93 0.89 0.92 0.94 0.89NAD-Alg 1.48 1.47 1.47 0.77 0.57 1.61 0.55SCC-P 1.36 0.80 0.78 0.75 0.75 0.79 0.74

s

∑ni=1

�

log10( bti )− log10( yi )�2

n

AF structure SCCs CPU-time for feature extraction Undirect Graph

Result 3: Best Features for Classification

C-B1 C-B2 C-B3number of arguments density of directed graph min attackers

Determined by a greedy forward search based on the Correlation-based Feature Selection(CFS) attribute evaluator.

AF structure Attackers

Result 4: Classification, i.e. Selecting the BestSolver for a Given Γ= ⟨A,R⟩

Classification (Higher is better)|A| density min attackers DG UG SCC All

Accuracy 48.5% 70.1% 69.9% 78.9% 79.0% 55.3% 79.5%Prec. AspartixM 35.0% 64.6% 63.7% 74.5% 74.9% 42.2% 76.1%Prec. PrefSAT 53.7% 67.8% 68.1% 79.6% 80.5% 60.4% 80.1%Prec. NAD-Alg 26.5% 69.2% 69.0% 81.7% 85.1% 35.3% 86.0%Prec. SCC-P 54.3% 73.0% 72.7% 76.6% 76.8% 57.8% 77.2%

AF structure Attackers Undirect Graph SCCs

Result 5: Algorithm Selection

Metric: Fastest(max. 1007)

AspartixM 106NAD-Alg 170PrefSAT 278SCC-P 453EPMs Regression 755EPMs Classification 788

Metric: IPC(max. 1007)

NAD-Alg 210.1AspartixM 288.3PrefSAT 546.7SCC-P 662.4EPMs Regression 887.7EPMs Classification 928.1

IPC: scale of (log)relative performance

Background on Dung’s AFCurrent Approaches for Preferred Extensions Enum.

FeaturesEmpirical Results

Conclusions

Conclusions

50 featuresBest algorithm selection accuracy: 80%Algorithm selection performance: 2 times better than the bestsingle-solver; 3 times better than the second-best single-solver;Consistency with previous explorations: density is among the mostinformative features.

Undirect Graph Features:AspartixM, often around [800 . . . 899] seconds — predictable?;NAD-Alg, often time-out — predictable?;PrefSAT, SAT-solver performance?;SCC-P, little difference, significant?.

Future Work

Exploiting additional useful features;Relevant semantics-dependent features (comparison with othersemantics);Combining algorithms in portfolios;

Scheduling;Ordering;Solver combination;

Addressing problems other than enumeration (e.g. non-emptiness. . . ).

Acknowledgements

The authors would like to acknowledge the use of the Universityof Huddersfield Queensgate Grid in carrying out this work.

References I

[Dvorák et al., 2011] Dvorák, W., Gaggl, S. A., Wallner, J., and Woltran, S. (2011).Making Use of Advances in Answer-Set Programming for Abstract Argumentation Systems.In Proceedings of the 19th International Conference on Applications of Declarative Programming and Knowledge Management (INAP2011).

[Nofal et al., 2014] Nofal, S., Atkinson, K., and Dunne, P. E. (2014).Algorithms for decision problems in argument systems under preferred semantics.Artificial Intelligence, 207:23–51.