1KJE.n

Summary Statistics

Run ID 1KJE.n

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 915

Summary Statistics

Run ID 1KJE.n

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1193

Recall Level Precision Averages

Recall Precision

0.00 0.6592

0.10 0.5252

0.20 0.3953

0.30 0.3098

0.40 0.2667

0.50 0.2090

0.60 0.1456

0.70 0.1037

0.80 0.0581

0.90 0.0333

1.00 0.0199

Average precision over all

relevant documents

non-interpolated 0.2255

Document Level Averages

Precision

At 5 docs 0.3774

At 10 docs 0.3019

At 15 docs 0.2654

At 20 docs 0.2406

At 30 docs 0.1981

At 100 docs 0.0957

At 200 docs 0.0590

At 500 docs 0.0298

At 1000 docs 0.0173

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2478

Recall Level Precision Averages

Recall Precision

0.00 0.7127

0.10 0.5159

0.20 0.4017

0.30 0.3323

0.40 0.2806

0.50 0.2185

0.60 0.1778

0.70 0.1237

0.80 0.0554

0.90 0.0369

1.00 0.0098

Average precision over all

relevant documents

non-interpolated 0.2360

Document Level Averages

Precision

At 5 docs 0.4302

At 10 docs 0.3434

At 15 docs 0.3057

At 20 docs 0.2811

At 30 docs 0.2396

At 100 docs 0.1221

At 200 docs 0.0773

At 500 docs 0.0388

At 1000 docs 0.0225

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2603

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

1KJE

Summary Statistics

Run ID 1KJE

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 930

Summary Statistics

Run ID 1KJE

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1209

Recall Level Precision Averages

Recall Precision

0.00 0.6356

0.10 0.5108

0.20 0.4256

0.30 0.3269

0.40 0.2755

0.50 0.2318

0.60 0.1771

0.70 0.1147

0.80 0.0702

0.90 0.0408

1.00 0.0270

Average precision over all

relevant documents

non-interpolated 0.2389

Document Level Averages

Precision

At 5 docs 0.3925

At 10 docs 0.3151

At 15 docs 0.2730

At 20 docs 0.2538

At 30 docs 0.2107

At 100 docs 0.0979

At 200 docs 0.0597

At 500 docs 0.0289

At 1000 docs 0.0175

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2588

Recall Level Precision Averages

Recall Precision

0.00 0.6968

0.10 0.5251

0.20 0.4535

0.30 0.3680

0.40 0.3090

0.50 0.2503

0.60 0.1985

0.70 0.1150

0.80 0.0587

0.90 0.0338

1.00 0.0120

Average precision over all

relevant documents

non-interpolated 0.2549

Document Level Averages

Precision

At 5 docs 0.4415

At 10 docs 0.3792

At 15 docs 0.3358

At 20 docs 0.3123

At 30 docs 0.2566

At 100 docs 0.1258

At 200 docs 0.0768

At 500 docs 0.0378

At 1000 docs 0.0228

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2811

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

1KJE3

Summary Statistics

Run ID 1KJE3

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 961

Summary Statistics

Run ID 1KJE3

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1240

Recall Level Precision Averages

Recall Precision

0.00 0.6592

0.10 0.5349

0.20 0.4400

0.30 0.3504

0.40 0.2936

0.50 0.2358

0.60 0.1849

0.70 0.1050

0.80 0.0652

0.90 0.0406

1.00 0.0270

Average precision over all

relevant documents

non-interpolated 0.2466

Document Level Averages

Precision

At 5 docs 0.4075

At 10 docs 0.3151

At 15 docs 0.2767

At 20 docs 0.2632

At 30 docs 0.2088

At 100 docs 0.1013

At 200 docs 0.0616

At 500 docs 0.0305

At 1000 docs 0.0181

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2638

Recall Level Precision Averages

Recall Precision

0.00 0.7192

0.10 0.5499

0.20 0.4713

0.30 0.3862

0.40 0.3030

0.50 0.2471

0.60 0.2005

0.70 0.1003

0.80 0.0577

0.90 0.0343

1.00 0.0126

Average precision over all

relevant documents

non-interpolated 0.2585

Document Level Averages

Precision

At 5 docs 0.4566

At 10 docs 0.3906

At 15 docs 0.3384

At 20 docs 0.3198

At 30 docs 0.2541

At 100 docs 0.1279

At 200 docs 0.0783

At 500 docs 0.0394

At 1000 docs 0.0234

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2838

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

1KJE ij

Summary Statistics

Run ID 1KJE ij

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 52

Total number of documents over all topics

Retrieved: 52000

Relevant: 1878

Rel-ret: 926

Summary Statistics

Run ID 1KJE ij

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 52

Total number of documents over all topics

Retrieved: 52000

Relevant: 2309

Rel-ret: 1204

Recall Level Precision Averages

Recall Precision

0.00 0.6524

0.10 0.5415

0.20 0.4306

0.30 0.3285

0.40 0.2893

0.50 0.2306

0.60 0.1659

0.70 0.0983

0.80 0.0612

0.90 0.0353

1.00 0.0231

Average precision over all

relevant documents

non-interpolated 0.2407

Document Level Averages

Precision

At 5 docs 0.4038

At 10 docs 0.3154

At 15 docs 0.2923

At 20 docs 0.2538

At 30 docs 0.2032

At 100 docs 0.0990

At 200 docs 0.0612

At 500 docs 0.0311

At 1000 docs 0.0178

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2668

Recall Level Precision Averages

Recall Precision

0.00 0.7088

0.10 0.5469

0.20 0.4485

0.30 0.3547

0.40 0.3055

0.50 0.2351

0.60 0.1687

0.70 0.1123

0.80 0.0596

0.90 0.0369

1.00 0.0107

Average precision over all

relevant documents

non-interpolated 0.2507

Document Level Averages

Precision

At 5 docs 0.4615

At 10 docs 0.3731

At 15 docs 0.3487

At 20 docs 0.3038

At 30 docs 0.2468

At 100 docs 0.1254

At 200 docs 0.0792

At 500 docs 0.0404

At 1000 docs 0.0232

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2744

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

ANNO

Summary Statistics

Run ID ANNO

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 38396

Relevant: 1910

Rel-ret: 868

Summary Statistics

Run ID ANNO

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 38396

Relevant: 2345

Rel-ret: 1099

Recall Level Precision Averages

Recall Precision

0.00 0.4867

0.10 0.3901

0.20 0.2536

0.30 0.1939

0.40 0.1654

0.50 0.1442

0.60 0.1019

0.70 0.0652

0.80 0.0376

0.90 0.0194

1.00 0.0145

Average precision over all

relevant documents

non-interpolated 0.1473

Document Level Averages

Precision

At 5 docs 0.2491

At 10 docs 0.1868

At 15 docs 0.1648

At 20 docs 0.1415

At 30 docs 0.1245

At 100 docs 0.0798

At 200 docs 0.0526

At 500 docs 0.0284

At 1000 docs 0.0164

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.1607

Recall Level Precision Averages

Recall Precision

0.00 0.5242

0.10 0.4070

0.20 0.2585

0.30 0.2021

0.40 0.1779

0.50 0.1511

0.60 0.1055

0.70 0.0723

0.80 0.0508

0.90 0.0255

1.00 0.0127

Average precision over all

relevant documents

non-interpolated 0.1497

Document Level Averages

Precision

At 5 docs 0.2717

At 10 docs 0.2113

At 15 docs 0.1962

At 20 docs 0.1745

At 30 docs 0.1572

At 100 docs 0.1068

At 200 docs 0.0684

At 500 docs 0.0366

At 1000 docs 0.0207

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.1673

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

BKJJBIDS

Summary Statistics

Run ID BKJJBIDS

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1050

Summary Statistics

Run ID BKJJBIDS

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1342

Recall Level Precision Averages

Recall Precision

0.00 0.7247

0.10 0.5529

0.20 0.4261

0.30 0.3575

0.40 0.2936

0.50 0.2570

0.60 0.1922

0.70 0.1625

0.80 0.1204

0.90 0.0678

1.00 0.0556

Average precision over all

relevant documents

non-interpolated 0.2715

Document Level Averages

Precision

At 5 docs 0.4642

At 10 docs 0.3623

At 15 docs 0.3170

At 20 docs 0.2689

At 30 docs 0.2239

At 100 docs 0.1077

At 200 docs 0.0683

At 500 docs 0.0341

At 1000 docs 0.0198

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2817

Recall Level Precision Averages

Recall Precision

0.00 0.7751

0.10 0.5800

0.20 0.4623

0.30 0.3896

0.40 0.3207

0.50 0.2722

0.60 0.2150

0.70 0.1809

0.80 0.1372

0.90 0.0777

1.00 0.0541

Average precision over all

relevant documents

non-interpolated 0.2927

Document Level Averages

Precision

At 5 docs 0.5245

At 10 docs 0.4264

At 15 docs 0.3811

At 20 docs 0.3283

At 30 docs 0.2799

At 100 docs 0.1396

At 200 docs 0.0880

At 500 docs 0.0438

At 1000 docs 0.0253

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3147

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

BKJJBIFU

Summary Statistics

Run ID BKJJBIFU

Task adhoc

Relevant Level Relevant

Run Description automatic,long with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1232

Summary Statistics

Run ID BKJJBIFU

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,long with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1628

Recall Level Precision Averages

Recall Precision

0.00 0.7907

0.10 0.7341

0.20 0.6317

0.30 0.5328

0.40 0.4680

0.50 0.4189

0.60 0.3484

0.70 0.2732

0.80 0.1779

0.90 0.0933

1.00 0.0616

Average precision over all

relevant documents

non-interpolated 0.3966

Document Level Averages

Precision

At 5 docs 0.5736

At 10 docs 0.4755

At 15 docs 0.4214

At 20 docs 0.3830

At 30 docs 0.3233

At 100 docs 0.1477

At 200 docs 0.0881

At 500 docs 0.0411

At 1000 docs 0.0232

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.4160

Recall Level Precision Averages

Recall Precision

0.00 0.8848

0.10 0.8020

0.20 0.7020

0.30 0.5882

0.40 0.5323

0.50 0.4557

0.60 0.3653

0.70 0.2625

0.80 0.1990

0.90 0.1219

1.00 0.0552

Average precision over all

relevant documents

non-interpolated 0.4350

Document Level Averages

Precision

At 5 docs 0.6642

At 10 docs 0.5830

At 15 docs 0.5170

At 20 docs 0.4689

At 30 docs 0.4000

At 100 docs 0.1943

At 200 docs 0.1185

At 500 docs 0.0554

At 1000 docs 0.0307

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.4456

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long with concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long with concepts)

BKJJDCFU

Summary Statistics

Run ID BKJJDCFU

Task adhoc

Relevant Level Relevant

Run Description automatic,long with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1093

Summary Statistics

Run ID BKJJDCFU

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,long with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1462

Recall Level Precision Averages

Recall Precision

0.00 0.7478

0.10 0.6352

0.20 0.5373

0.30 0.4575

0.40 0.3924

0.50 0.3220

0.60 0.2656

0.70 0.2016

0.80 0.1274

0.90 0.0571

1.00 0.0441

Average precision over all

relevant documents

non-interpolated 0.3252

Document Level Averages

Precision

At 5 docs 0.5208

At 10 docs 0.4264

At 15 docs 0.3686

At 20 docs 0.3198

At 30 docs 0.2629

At 100 docs 0.1277

At 200 docs 0.0777

At 500 docs 0.0372

At 1000 docs 0.0206

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3656

Recall Level Precision Averages

Recall Precision

0.00 0.8325

0.10 0.7228

0.20 0.5817

0.30 0.4992

0.40 0.4119

0.50 0.3405

0.60 0.2843

0.70 0.2093

0.80 0.1401

0.90 0.0630

1.00 0.0414

Average precision over all

relevant documents

non-interpolated 0.3536

Document Level Averages

Precision

At 5 docs 0.6000

At 10 docs 0.5075

At 15 docs 0.4478

At 20 docs 0.4000

At 30 docs 0.3346

At 100 docs 0.1713

At 200 docs 0.1054

At 500 docs 0.0501

At 1000 docs 0.0276

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3936

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long with concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long with concepts)

CRL10

Summary Statistics

Run ID CRL10

Task adhoc

Relevant Level Relevant

Run Description interactive,long with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 52051

Relevant: 1910

Rel-ret: 1526

Summary Statistics

Run ID CRL10

Task adhoc

Relevant Level Partial Relevant

Run Description interactive,long with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 52051

Relevant: 2345

Rel-ret: 1936

Recall Level Precision Averages

Recall Precision

0.00 0.8178

0.10 0.6953

0.20 0.6060

0.30 0.5351

0.40 0.4579

0.50 0.4132

0.60 0.3415

0.70 0.2683

0.80 0.2000

0.90 0.1166

1.00 0.0614

Average precision over all

relevant documents

non-interpolated 0.3889

Document Level Averages

Precision

At 5 docs 0.5623

At 10 docs 0.4774

At 15 docs 0.4189

At 20 docs 0.3651

At 30 docs 0.3038

At 100 docs 0.1621

At 200 docs 0.1018

At 500 docs 0.0508

At 1000 docs 0.0288

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3956

Recall Level Precision Averages

Recall Precision

0.00 0.9048

0.10 0.7661

0.20 0.6796

0.30 0.6155

0.40 0.5063

0.50 0.4620

0.60 0.3814

0.70 0.3011

0.80 0.2254

0.90 0.1429

1.00 0.0434

Average precision over all

relevant documents

non-interpolated 0.4404

Document Level Averages

Precision

At 5 docs 0.6717

At 10 docs 0.5679

At 15 docs 0.5132

At 20 docs 0.4538

At 30 docs 0.3824

At 100 docs 0.2096

At 200 docs 0.1322

At 500 docs 0.0655

At 1000 docs 0.0365

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.4404

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(interactive) (long with concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(interactive) (long with concepts)

CRL12

Summary Statistics

Run ID CRL12

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1135

Summary Statistics

Run ID CRL12

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1473

Recall Level Precision Averages

Recall Precision

0.00 0.6129

0.10 0.5332

0.20 0.4030

0.30 0.3386

0.40 0.2806

0.50 0.2442

0.60 0.2037

0.70 0.1657

0.80 0.1296

0.90 0.0736

1.00 0.0495

Average precision over all

relevant documents

non-interpolated 0.2575

Document Level Averages

Precision

At 5 docs 0.4075

At 10 docs 0.3245

At 15 docs 0.2906

At 20 docs 0.2585

At 30 docs 0.2057

At 100 docs 0.1087

At 200 docs 0.0710

At 500 docs 0.0364

At 1000 docs 0.0214

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2699

Recall Level Precision Averages

Recall Precision

0.00 0.6678

0.10 0.5410

0.20 0.3994

0.30 0.3413

0.40 0.2910

0.50 0.2581

0.60 0.2189

0.70 0.1708

0.80 0.1339

0.90 0.0794

1.00 0.0279

Average precision over all

relevant documents

non-interpolated 0.2654

Document Level Averages

Precision

At 5 docs 0.4340

At 10 docs 0.3547

At 15 docs 0.3270

At 20 docs 0.2934

At 30 docs 0.2409

At 100 docs 0.1377

At 200 docs 0.0912

At 500 docs 0.0474

At 1000 docs 0.0278

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2753

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

CRL13

Summary Statistics

Run ID CRL13

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1146

Summary Statistics

Run ID CRL13

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1484

Recall Level Precision Averages

Recall Precision

0.00 0.6182

0.10 0.5355

0.20 0.4243

0.30 0.3452

0.40 0.2785

0.50 0.2471

0.60 0.1946

0.70 0.1600

0.80 0.1308

0.90 0.0694

1.00 0.0427

Average precision over all

relevant documents

non-interpolated 0.2584

Document Level Averages

Precision

At 5 docs 0.4113

At 10 docs 0.3340

At 15 docs 0.2843

At 20 docs 0.2557

At 30 docs 0.2107

At 100 docs 0.1102

At 200 docs 0.0708

At 500 docs 0.0368

At 1000 docs 0.0216

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2763

Recall Level Precision Averages

Recall Precision

0.00 0.6630

0.10 0.5360

0.20 0.4275

0.30 0.3508

0.40 0.2922

0.50 0.2618

0.60 0.2096

0.70 0.1667

0.80 0.1352

0.90 0.0819

1.00 0.0270

Average precision over all

relevant documents

non-interpolated 0.2675

Document Level Averages

Precision

At 5 docs 0.4453

At 10 docs 0.3774

At 15 docs 0.3245

At 20 docs 0.2925

At 30 docs 0.2484

At 100 docs 0.1392

At 200 docs 0.0912

At 500 docs 0.0479

At 1000 docs 0.0280

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2910

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

CRL14

Summary Statistics

Run ID CRL14

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1148

Summary Statistics

Run ID CRL14

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1503

Recall Level Precision Averages

Recall Precision

0.00 0.6213

0.10 0.5402

0.20 0.4409

0.30 0.3624

0.40 0.2874

0.50 0.2467

0.60 0.1965

0.70 0.1635

0.80 0.1348

0.90 0.0689

1.00 0.0397

Average precision over all

relevant documents

non-interpolated 0.2627

Document Level Averages

Precision

At 5 docs 0.4075

At 10 docs 0.3434

At 15 docs 0.2943

At 20 docs 0.2585

At 30 docs 0.2176

At 100 docs 0.1119

At 200 docs 0.0708

At 500 docs 0.0371

At 1000 docs 0.0217

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2872

Recall Level Precision Averages

Recall Precision

0.00 0.6664

0.10 0.5415

0.20 0.4439

0.30 0.3679

0.40 0.2991

0.50 0.2613

0.60 0.2127

0.70 0.1737

0.80 0.1413

0.90 0.0793

1.00 0.0288

Average precision over all

relevant documents

non-interpolated 0.2732

Document Level Averages

Precision

At 5 docs 0.4491

At 10 docs 0.3849

At 15 docs 0.3346

At 20 docs 0.2943

At 30 docs 0.2560

At 100 docs 0.1430

At 200 docs 0.0921

At 500 docs 0.0486

At 1000 docs 0.0284

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3040

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

CRL3

Summary Statistics

Run ID CRL3

Task adhoc

Relevant Level Relevant

Run Description automatic,long with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1295

Summary Statistics

Run ID CRL3

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,long with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1677

Recall Level Precision Averages

Recall Precision

0.00 0.7192

0.10 0.6432

0.20 0.5475

0.30 0.4679

0.40 0.3802

0.50 0.3299

0.60 0.2706

0.70 0.2116

0.80 0.1631

0.90 0.0743

1.00 0.0463

Average precision over all

relevant documents

non-interpolated 0.3337

Document Level Averages

Precision

At 5 docs 0.4981

At 10 docs 0.4170

At 15 docs 0.3597

At 20 docs 0.3189

At 30 docs 0.2755

At 100 docs 0.1372

At 200 docs 0.0851

At 500 docs 0.0428

At 1000 docs 0.0244

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3310

Recall Level Precision Averages

Recall Precision

0.00 0.8151

0.10 0.6890

0.20 0.5878

0.30 0.4936

0.40 0.4191

0.50 0.3445

0.60 0.2875

0.70 0.2100

0.80 0.1623

0.90 0.0896

1.00 0.0431

Average precision over all

relevant documents

non-interpolated 0.3575

Document Level Averages

Precision

At 5 docs 0.5623

At 10 docs 0.4849

At 15 docs 0.4239

At 20 docs 0.3802

At 30 docs 0.3321

At 100 docs 0.1726

At 200 docs 0.1095

At 500 docs 0.0557

At 1000 docs 0.0316

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3665

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long with concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long with concepts)

CRL4

Summary Statistics

Run ID CRL4

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 51779

Relevant: 1910

Rel-ret: 1186

Summary Statistics

Run ID CRL4

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 51779

Relevant: 2345

Rel-ret: 1498

Recall Level Precision Averages

Recall Precision

0.00 0.5768

0.10 0.4856

0.20 0.3876

0.30 0.3275

0.40 0.2842

0.50 0.2354

0.60 0.1883

0.70 0.1471

0.80 0.1158

0.90 0.0724

1.00 0.0579

Average precision over all

relevant documents

non-interpolated 0.2404

Document Level Averages

Precision

At 5 docs 0.3660

At 10 docs 0.3019

At 15 docs 0.2616

At 20 docs 0.2406

At 30 docs 0.2025

At 100 docs 0.1062

At 200 docs 0.0680

At 500 docs 0.0368

At 1000 docs 0.0224

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2490

Recall Level Precision Averages

Recall Precision

0.00 0.6314

0.10 0.4983

0.20 0.4144

0.30 0.3460

0.40 0.2800

0.50 0.2468

0.60 0.2034

0.70 0.1576

0.80 0.1176

0.90 0.0743

1.00 0.0388

Average precision over all

relevant documents

non-interpolated 0.2492

Document Level Averages

Precision

At 5 docs 0.4075

At 10 docs 0.3396

At 15 docs 0.3044

At 20 docs 0.2792

At 30 docs 0.2371

At 100 docs 0.1315

At 200 docs 0.0849

At 500 docs 0.0468

At 1000 docs 0.0283

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2602

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

CRL5

Summary Statistics

Run ID CRL5

Task adhoc

Relevant Level Relevant

Run Description interactive,long with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 50665

Relevant: 1910

Rel-ret: 1582

Summary Statistics

Run ID CRL5

Task adhoc

Relevant Level Partial Relevant

Run Description interactive,long with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 50665

Relevant: 2345

Rel-ret: 1984

Recall Level Precision Averages

Recall Precision

0.00 0.7723

0.10 0.6562

0.20 0.5612

0.30 0.4955

0.40 0.4046

0.50 0.3573

0.60 0.2949

0.70 0.2483

0.80 0.1561

0.90 0.0957

1.00 0.0670

Average precision over all

relevant documents

non-interpolated 0.3479

Document Level Averages

Precision

At 5 docs 0.5019

At 10 docs 0.4283

At 15 docs 0.3723

At 20 docs 0.3302

At 30 docs 0.2792

At 100 docs 0.1500

At 200 docs 0.0992

At 500 docs 0.0491

At 1000 docs 0.0298

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3775

Recall Level Precision Averages

Recall Precision

0.00 0.8503

0.10 0.7413

0.20 0.6359

0.30 0.5581

0.40 0.4497

0.50 0.4078

0.60 0.3429

0.70 0.2741

0.80 0.1864

0.90 0.1145

1.00 0.0555

Average precision over all

relevant documents

non-interpolated 0.3922

Document Level Averages

Precision

At 5 docs 0.6151

At 10 docs 0.5170

At 15 docs 0.4579

At 20 docs 0.4132

At 30 docs 0.3547

At 100 docs 0.1923

At 200 docs 0.1280

At 500 docs 0.0633

At 1000 docs 0.0374

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.4160

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(interactive) (long with concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(interactive) (long with concepts)

CRL6

Summary Statistics

Run ID CRL6

Task adhoc

Relevant Level Relevant

Run Description automatic,long with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1334

Summary Statistics

Run ID CRL6

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,long with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1734

Recall Level Precision Averages

Recall Precision

0.00 0.7491

0.10 0.6729

0.20 0.5889

0.30 0.5054

0.40 0.4367

0.50 0.3911

0.60 0.3238

0.70 0.2512

0.80 0.1967

0.90 0.0921

1.00 0.0602

Average precision over all

relevant documents

non-interpolated 0.3716

Document Level Averages

Precision

At 5 docs 0.5358

At 10 docs 0.4736

At 15 docs 0.3962

At 20 docs 0.3575

At 30 docs 0.2981

At 100 docs 0.1509

At 200 docs 0.0938

At 500 docs 0.0451

At 1000 docs 0.0252

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3815

Recall Level Precision Averages

Recall Precision

0.00 0.8310

0.10 0.7323

0.20 0.6512

0.30 0.5505

0.40 0.4894

0.50 0.4159

0.60 0.3538

0.70 0.2729

0.80 0.2150

0.90 0.1080

1.00 0.0576

Average precision over all

relevant documents

non-interpolated 0.4075

Document Level Averages

Precision

At 5 docs 0.6264

At 10 docs 0.5623

At 15 docs 0.4855

At 20 docs 0.4358

At 30 docs 0.3616

At 100 docs 0.1957

At 200 docs 0.1240

At 500 docs 0.0596

At 1000 docs 0.0327

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.4122

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long with concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long with concepts)

CRL7

Summary Statistics

Run ID CRL7

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 50601

Relevant: 1910

Rel-ret: 1113

Summary Statistics

Run ID CRL7

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 50601

Relevant: 2345

Rel-ret: 1425

Recall Level Precision Averages

Recall Precision

0.00 0.5854

0.10 0.4734

0.20 0.3729

0.30 0.3235

0.40 0.2725

0.50 0.2211

0.60 0.1750

0.70 0.1380

0.80 0.1051

0.90 0.0696

1.00 0.0579

Average precision over all

relevant documents

non-interpolated 0.2290

Document Level Averages

Precision

At 5 docs 0.3774

At 10 docs 0.2925

At 15 docs 0.2667

At 20 docs 0.2340

At 30 docs 0.2013

At 100 docs 0.1002

At 200 docs 0.0632

At 500 docs 0.0344

At 1000 docs 0.0210

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2419

Recall Level Precision Averages

Recall Precision

0.00 0.6432

0.10 0.4947

0.20 0.3926

0.30 0.3333

0.40 0.2800

0.50 0.2286

0.60 0.1956

0.70 0.1440

0.80 0.1128

0.90 0.0711

1.00 0.0381

Average precision over all

relevant documents

non-interpolated 0.2402

Document Level Averages

Precision

At 5 docs 0.4151

At 10 docs 0.3340

At 15 docs 0.3031

At 20 docs 0.2689

At 30 docs 0.2346

At 100 docs 0.1251

At 200 docs 0.0799

At 500 docs 0.0445

At 1000 docs 0.0269

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2481

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

CRL8

Summary Statistics

Run ID CRL8

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 40899

Relevant: 1910

Rel-ret: 1115

Summary Statistics

Run ID CRL8

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 40899

Relevant: 2345

Rel-ret: 1438

Recall Level Precision Averages

Recall Precision

0.00 0.6129

0.10 0.5332

0.20 0.4033

0.30 0.3390

0.40 0.2797

0.50 0.2435

0.60 0.2035

0.70 0.1654

0.80 0.1293

0.90 0.0723

1.00 0.0472

Average precision over all

relevant documents

non-interpolated 0.2571

Document Level Averages

Precision

At 5 docs 0.4075

At 10 docs 0.3245

At 15 docs 0.2906

At 20 docs 0.2585

At 30 docs 0.2063

At 100 docs 0.1094

At 200 docs 0.0708

At 500 docs 0.0361

At 1000 docs 0.0210

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2707

Recall Level Precision Averages

Recall Precision

0.00 0.6677

0.10 0.5409

0.20 0.3997

0.30 0.3416

0.40 0.2904

0.50 0.2570

0.60 0.2165

0.70 0.1688

0.80 0.1313

0.90 0.0774

1.00 0.0281

Average precision over all

relevant documents

non-interpolated 0.2646

Document Level Averages

Precision

At 5 docs 0.4340

At 10 docs 0.3547

At 15 docs 0.3270

At 20 docs 0.2934

At 30 docs 0.2415

At 100 docs 0.1387

At 200 docs 0.0910

At 500 docs 0.0469

At 1000 docs 0.0271

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2761

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

CRL9

Summary Statistics

Run ID CRL9

Task adhoc

Relevant Level Relevant

Run Description automatic,long with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1340

Summary Statistics

Run ID CRL9

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,long with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1747

Recall Level Precision Averages

Recall Precision

0.00 0.7928

0.10 0.7013

0.20 0.5936

0.30 0.5324

0.40 0.4619

0.50 0.4003

0.60 0.3389

0.70 0.2714

0.80 0.2163

0.90 0.1317

1.00 0.0753

Average precision over all

relevant documents

non-interpolated 0.3917

Document Level Averages

Precision

At 5 docs 0.5660

At 10 docs 0.4717

At 15 docs 0.4138

At 20 docs 0.3679

At 30 docs 0.3107

At 100 docs 0.1604

At 200 docs 0.0980

At 500 docs 0.0468

At 1000 docs 0.0253

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3905

Recall Level Precision Averages

Recall Precision

0.00 0.8750

0.10 0.7487

0.20 0.6681

0.30 0.5825

0.40 0.5278

0.50 0.4432

0.60 0.3753

0.70 0.2907

0.80 0.2370

0.90 0.1422

1.00 0.0671

Average precision over all

relevant documents

non-interpolated 0.4323

Document Level Averages

Precision

At 5 docs 0.6415

At 10 docs 0.5623

At 15 docs 0.4956

At 20 docs 0.4434

At 30 docs 0.3862

At 100 docs 0.2089

At 200 docs 0.1284

At 500 docs 0.0611

At 1000 docs 0.0330

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.4354

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long with concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long with concepts)

DOVE1

Summary Statistics

Run ID DOVE1

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 977

Summary Statistics

Run ID DOVE1

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1300

Recall Level Precision Averages

Recall Precision

0.00 0.3658

0.10 0.2613

0.20 0.1934

0.30 0.1698

0.40 0.1316

0.50 0.1207

0.60 0.0761

0.70 0.0677

0.80 0.0485

0.90 0.0365

1.00 0.0117

Average precision over all

relevant documents

non-interpolated 0.1170

Document Level Averages

Precision

At 5 docs 0.1736

At 10 docs 0.1340

At 15 docs 0.1182

At 20 docs 0.1179

At 30 docs 0.1013

At 100 docs 0.0625

At 200 docs 0.0491

At 500 docs 0.0295

At 1000 docs 0.0184

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.1334

Recall Level Precision Averages

Recall Precision

0.00 0.4079

0.10 0.2786

0.20 0.2210

0.30 0.1790

0.40 0.1482

0.50 0.1299

0.60 0.0907

0.70 0.0726

0.80 0.0564

0.90 0.0422

1.00 0.0090

Average precision over all

relevant documents

non-interpolated 0.1272

Document Level Averages

Precision

At 5 docs 0.2038

At 10 docs 0.1642

At 15 docs 0.1447

At 20 docs 0.1434

At 30 docs 0.1302

At 100 docs 0.0868

At 200 docs 0.0685

At 500 docs 0.0400

At 1000 docs 0.0245

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.1438

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

DOVE2

Summary Statistics

Run ID DOVE2

Task adhoc

Relevant Level Relevant

Run Description interactive,long without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1194

Summary Statistics

Run ID DOVE2

Task adhoc

Relevant Level Partial Relevant

Run Description interactive,long without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1539

Recall Level Precision Averages

Recall Precision

0.00 0.7189

0.10 0.5388

0.20 0.3793

0.30 0.3202

0.40 0.2713

0.50 0.2192

0.60 0.1717

0.70 0.1183

0.80 0.0910

0.90 0.0420

1.00 0.0167

Average precision over all

relevant documents

non-interpolated 0.2396

Document Level Averages

Precision

At 5 docs 0.4075

At 10 docs 0.2811

At 15 docs 0.2327

At 20 docs 0.2075

At 30 docs 0.1723

At 100 docs 0.1030

At 200 docs 0.0670

At 500 docs 0.0374

At 1000 docs 0.0225

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2491

Recall Level Precision Averages

Recall Precision

0.00 0.7828

0.10 0.5362

0.20 0.4019

0.30 0.3371

0.40 0.2773

0.50 0.2382

0.60 0.1761

0.70 0.1296

0.80 0.0911

0.90 0.0508

1.00 0.0161

Average precision over all

relevant documents

non-interpolated 0.2484

Document Level Averages

Precision

At 5 docs 0.4642

At 10 docs 0.3453

At 15 docs 0.2830

At 20 docs 0.2538

At 30 docs 0.2119

At 100 docs 0.1315

At 200 docs 0.0880

At 500 docs 0.0483

At 1000 docs 0.0290

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2611

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(interactive) (long without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(interactive) (long without concepts)

DOVE3

Summary Statistics

Run ID DOVE3

Task adhoc

Relevant Level Relevant

Run Description interactive,long without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1173

Summary Statistics

Run ID DOVE3

Task adhoc

Relevant Level Partial Relevant

Run Description interactive,long without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1495

Recall Level Precision Averages

Recall Precision

0.00 0.6033

0.10 0.5180

0.20 0.4450

0.30 0.3508

0.40 0.2808

0.50 0.2171

0.60 0.1703

0.70 0.1054

0.80 0.0725

0.90 0.0281

1.00 0.0108

Average precision over all

relevant documents

non-interpolated 0.2317

Document Level Averages

Precision

At 5 docs 0.3736

At 10 docs 0.3132

At 15 docs 0.2591

At 20 docs 0.2368

At 30 docs 0.1956

At 100 docs 0.1017

At 200 docs 0.0704

At 500 docs 0.0375

At 1000 docs 0.0221

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2727

Recall Level Precision Averages

Recall Precision

0.00 0.6777

0.10 0.5723

0.20 0.4909

0.30 0.3684

0.40 0.2999

0.50 0.2407

0.60 0.1838

0.70 0.1224

0.80 0.0879

0.90 0.0383

1.00 0.0144

Average precision over all

relevant documents

non-interpolated 0.2556

Document Level Averages

Precision

At 5 docs 0.4528

At 10 docs 0.3830

At 15 docs 0.3195

At 20 docs 0.2962

At 30 docs 0.2447

At 100 docs 0.1298

At 200 docs 0.0924

At 500 docs 0.0489

At 1000 docs 0.0282

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2951

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(interactive) (long without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(interactive) (long without concepts)

JALAB

Summary Statistics

Run ID JALAB

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 52051

Relevant: 1910

Rel-ret: 84

Summary Statistics

Run ID JALAB

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 52051

Relevant: 2345

Rel-ret: 164

Recall Level Precision Averages

Recall Precision

0.00 0.0075

0.10 0.0064

0.20 0.0049

0.30 0.0025

0.40 0.0014

0.50 0.0006

0.60 0.0002

0.70 0.0000

0.80 0.0000

0.90 0.0000

1.00 0.0000

Average precision over all

relevant documents

non-interpolated 0.0017

Document Level Averages

Precision

At 5 docs 0.0000

At 10 docs 0.0000

At 15 docs 0.0000

At 20 docs 0.0009

At 30 docs 0.0025

At 100 docs 0.0034

At 200 docs 0.0029

At 500 docs 0.0022

At 1000 docs 0.0016

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.0003

Recall Level Precision Averages

Recall Precision

0.00 0.0194

0.10 0.0108

0.20 0.0053

0.30 0.0034

0.40 0.0025

0.50 0.0006

0.60 0.0002

0.70 0.0000

0.80 0.0000

0.90 0.0000

1.00 0.0000

Average precision over all

relevant documents

non-interpolated 0.0028

Document Level Averages

Precision

At 5 docs 0.0075

At 10 docs 0.0075

At 15 docs 0.0050

At 20 docs 0.0057

At 30 docs 0.0069

At 100 docs 0.0072

At 200 docs 0.0062

At 500 docs 0.0042

At 1000 docs 0.0031

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.0066

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

K32001

Summary Statistics

Run ID K32001

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1198

Summary Statistics

Run ID K32001

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1559

Recall Level Precision Averages

Recall Precision

0.00 0.7148

0.10 0.5550

0.20 0.4739

0.30 0.4313

0.40 0.3920

0.50 0.3051

0.60 0.2518

0.70 0.1995

0.80 0.1584

0.90 0.0958

1.00 0.0748

Average precision over all

relevant documents

non-interpolated 0.3159

Document Level Averages

Precision

At 5 docs 0.4566

At 10 docs 0.3811

At 15 docs 0.3296

At 20 docs 0.2981

At 30 docs 0.2434

At 100 docs 0.1262

At 200 docs 0.0798

At 500 docs 0.0392

At 1000 docs 0.0226

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3317

Recall Level Precision Averages

Recall Precision

0.00 0.7837

0.10 0.6284

0.20 0.4996

0.30 0.4651

0.40 0.3968

0.50 0.3383

0.60 0.2890

0.70 0.2267

0.80 0.1764

0.90 0.1111

1.00 0.0721

Average precision over all

relevant documents

non-interpolated 0.3454

Document Level Averages

Precision

At 5 docs 0.5434

At 10 docs 0.4623

At 15 docs 0.4038

At 20 docs 0.3717

At 30 docs 0.3126

At 100 docs 0.1666

At 200 docs 0.1050

At 500 docs 0.0512

At 1000 docs 0.0294

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3573

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

K32002

Summary Statistics

Run ID K32002

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1219

Summary Statistics

Run ID K32002

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1588

Recall Level Precision Averages

Recall Precision

0.00 0.7031

0.10 0.5983

0.20 0.4872

0.30 0.4451

0.40 0.3747

0.50 0.3158

0.60 0.2537

0.70 0.2122

0.80 0.1598

0.90 0.0948

1.00 0.0630

Average precision over all

relevant documents

non-interpolated 0.3181

Document Level Averages

Precision

At 5 docs 0.4717

At 10 docs 0.3981

At 15 docs 0.3484

At 20 docs 0.3142

At 30 docs 0.2535

At 100 docs 0.1298

At 200 docs 0.0808

At 500 docs 0.0399

At 1000 docs 0.0230

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3273

Recall Level Precision Averages

Recall Precision

0.00 0.7650

0.10 0.6441

0.20 0.5140

0.30 0.4763

0.40 0.4077

0.50 0.3501

0.60 0.2875

0.70 0.2360

0.80 0.1825

0.90 0.1147

1.00 0.0628

Average precision over all

relevant documents

non-interpolated 0.3495

Document Level Averages

Precision

At 5 docs 0.5585

At 10 docs 0.4792

At 15 docs 0.4189

At 20 docs 0.3877

At 30 docs 0.3252

At 100 docs 0.1711

At 200 docs 0.1059

At 500 docs 0.0522

At 1000 docs 0.0300

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3657

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

KCTRG1

Summary Statistics

Run ID KCTRG1

Task adhoc

Relevant Level Relevant

Run Description automatic,long without cocepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 989

Summary Statistics

Run ID KCTRG1

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,long without cocepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1270

Recall Level Precision Averages

Recall Precision

0.00 0.5638

0.10 0.3765

0.20 0.2761

0.30 0.2169

0.40 0.1593

0.50 0.1316

0.60 0.0848

0.70 0.0531

0.80 0.0328

0.90 0.0115

1.00 0.0089

Average precision over all

relevant documents

non-interpolated 0.1544

Document Level Averages

Precision

At 5 docs 0.2604

At 10 docs 0.2208

At 15 docs 0.2063

At 20 docs 0.1906

At 30 docs 0.1560

At 100 docs 0.0826

At 200 docs 0.0549

At 500 docs 0.0304

At 1000 docs 0.0187

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.1851

Recall Level Precision Averages

Recall Precision

0.00 0.6397

0.10 0.3988

0.20 0.3070

0.30 0.2517

0.40 0.1938

0.50 0.1638

0.60 0.1096

0.70 0.0713

0.80 0.0513

0.90 0.0203

1.00 0.0066

Average precision over all

relevant documents

non-interpolated 0.1768

Document Level Averages

Precision

At 5 docs 0.3321

At 10 docs 0.2925

At 15 docs 0.2704

At 20 docs 0.2443

At 30 docs 0.2069

At 100 docs 0.1104

At 200 docs 0.0725

At 500 docs 0.0395

At 1000 docs 0.0240

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2142

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long without concepts)

KCTRG2

Summary Statistics

Run ID KCTRG2

Task adhoc

Relevant Level Relevant

Run Description automatic,long without cocepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1088

Summary Statistics

Run ID KCTRG2

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,long without cocepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1443

Recall Level Precision Averages

Recall Precision

0.00 0.6904

0.10 0.5378

0.20 0.4025

0.30 0.3073

0.40 0.2289

0.50 0.1874

0.60 0.1438

0.70 0.1010

0.80 0.0666

0.90 0.0366

1.00 0.0208

Average precision over all

relevant documents

non-interpolated 0.2280

Document Level Averages

Precision

At 5 docs 0.3887

At 10 docs 0.3226

At 15 docs 0.2805

At 20 docs 0.2500

At 30 docs 0.2031

At 100 docs 0.1074

At 200 docs 0.0698

At 500 docs 0.0355

At 1000 docs 0.0205

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2452

Recall Level Precision Averages

Recall Precision

0.00 0.7796

0.10 0.5776

0.20 0.4510

0.30 0.3532

0.40 0.2771

0.50 0.2262

0.60 0.1770

0.70 0.1193

0.80 0.0754

0.90 0.0401

1.00 0.0172

Average precision over all

relevant documents

non-interpolated 0.2584

Document Level Averages

Precision

At 5 docs 0.4679

At 10 docs 0.3981

At 15 docs 0.3484

At 20 docs 0.3160

At 30 docs 0.2654

At 100 docs 0.1460

At 200 docs 0.0939

At 500 docs 0.0475

At 1000 docs 0.0272

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2824

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long without concepts)

KLAB1.dat

Summary Statistics

Run ID KLAB1.dat

Task adhoc

Relevant Level Relevant

Run Description automatic,short with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 33830

Relevant: 1910

Rel-ret: 747

Summary Statistics

Run ID KLAB1.dat

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 33830

Relevant: 2345

Rel-ret: 982

Recall Level Precision Averages

Recall Precision

0.00 0.1619

0.10 0.1267

0.20 0.1120

0.30 0.1078

0.40 0.0848

0.50 0.0536

0.60 0.0472

0.70 0.0400

0.80 0.0300

0.90 0.0205

1.00 0.0151

Average precision over all

relevant documents

non-interpolated 0.0717

Document Level Averages

Precision

At 5 docs 0.0566

At 10 docs 0.0547

At 15 docs 0.0528

At 20 docs 0.0509

At 30 docs 0.0497

At 100 docs 0.0402

At 200 docs 0.0293

At 500 docs 0.0203

At 1000 docs 0.0141

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.0612

Recall Level Precision Averages

Recall Precision

0.00 0.1917

0.10 0.1500

0.20 0.1350

0.30 0.1252

0.40 0.0923

0.50 0.0580

0.60 0.0504

0.70 0.0456

0.80 0.0287

0.90 0.0264

1.00 0.0258

Average precision over all

relevant documents

non-interpolated 0.0808

Document Level Averages

Precision

At 5 docs 0.0642

At 10 docs 0.0736

At 15 docs 0.0755

At 20 docs 0.0698

At 30 docs 0.0660

At 100 docs 0.0528

At 200 docs 0.0395

At 500 docs 0.0274

At 1000 docs 0.0185

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.0738

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short with concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short with concepts)

KLAB2.dat

Summary Statistics

Run ID KLAB2.dat

Task adhoc

Relevant Level Relevant

Run Description automatic,short with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 41979

Relevant: 1910

Rel-ret: 475

Summary Statistics

Run ID KLAB2.dat

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 41979

Relevant: 2345

Rel-ret: 603

Recall Level Precision Averages

Recall Precision

0.00 0.0525

0.10 0.0507

0.20 0.0482

0.30 0.0455

0.40 0.0314

0.50 0.0287

0.60 0.0245

0.70 0.0182

0.80 0.0097

0.90 0.0087

1.00 0.0087

Average precision over all

relevant documents

non-interpolated 0.0327

Document Level Averages

Precision

At 5 docs 0.0038

At 10 docs 0.0132

At 15 docs 0.0201

At 20 docs 0.0255

At 30 docs 0.0226

At 100 docs 0.0194

At 200 docs 0.0165

At 500 docs 0.0129

At 1000 docs 0.0090

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.0220

Recall Level Precision Averages

Recall Precision

0.00 0.0721

0.10 0.0655

0.20 0.0614

0.30 0.0535

0.40 0.0449

0.50 0.0331

0.60 0.0289

0.70 0.0253

0.80 0.0112

0.90 0.0112

1.00 0.0112

Average precision over all

relevant documents

non-interpolated 0.0381

Document Level Averages

Precision

At 5 docs 0.0038

At 10 docs 0.0226

At 15 docs 0.0314

At 20 docs 0.0377

At 30 docs 0.0333

At 100 docs 0.0251

At 200 docs 0.0209

At 500 docs 0.0168

At 1000 docs 0.0114

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.0318

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short with concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short with concepts)

NTE151

Summary Statistics

Run ID NTE151

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 51221

Relevant: 1910

Rel-ret: 966

Summary Statistics

Run ID NTE151

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 51221

Relevant: 2345

Rel-ret: 1203

Recall Level Precision Averages

Recall Precision

0.00 0.6472

0.10 0.5460

0.20 0.4249

0.30 0.3654

0.40 0.2958

0.50 0.2443

0.60 0.1856

0.70 0.1426

0.80 0.1066

0.90 0.0523

1.00 0.0391

Average precision over all

relevant documents

non-interpolated 0.2560

Document Level Averages

Precision

At 5 docs 0.4189

At 10 docs 0.3358

At 15 docs 0.2818

At 20 docs 0.2481

At 30 docs 0.1981

At 100 docs 0.1008

At 200 docs 0.0636

At 500 docs 0.0318

At 1000 docs 0.0182

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2725

Recall Level Precision Averages

Recall Precision

0.00 0.6910

0.10 0.5538

0.20 0.4208

0.30 0.3497

0.40 0.2920

0.50 0.2465

0.60 0.1875

0.70 0.1416

0.80 0.0957

0.90 0.0489

1.00 0.0289

Average precision over all

relevant documents

non-interpolated 0.2586

Document Level Averages

Precision

At 5 docs 0.4717

At 10 docs 0.3811

At 15 docs 0.3270

At 20 docs 0.2906

At 30 docs 0.2403

At 100 docs 0.1253

At 200 docs 0.0804

At 500 docs 0.0402

At 1000 docs 0.0227

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2779

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

NTE152

Summary Statistics

Run ID NTE152

Task adhoc

Relevant Level Relevant

Run Description automatic,short with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1135

Summary Statistics

Run ID NTE152

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1443

Recall Level Precision Averages

Recall Precision

0.00 0.7496

0.10 0.6695

0.20 0.5649

0.30 0.4729

0.40 0.3826

0.50 0.3243

0.60 0.2533

0.70 0.1829

0.80 0.1393

0.90 0.0846

1.00 0.0684

Average precision over all

relevant documents

non-interpolated 0.3316

Document Level Averages

Precision

At 5 docs 0.5132

At 10 docs 0.4151

At 15 docs 0.3522

At 20 docs 0.3038

At 30 docs 0.2497

At 100 docs 0.1196

At 200 docs 0.0743

At 500 docs 0.0371

At 1000 docs 0.0214

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3358

Recall Level Precision Averages

Recall Precision

0.00 0.8080

0.10 0.7049

0.20 0.5862

0.30 0.4810

0.40 0.3962

0.50 0.3321

0.60 0.2672

0.70 0.1955

0.80 0.1384

0.90 0.0788

1.00 0.0534

Average precision over all

relevant documents

non-interpolated 0.3490

Document Level Averages

Precision

At 5 docs 0.5925

At 10 docs 0.4830

At 15 docs 0.4176

At 20 docs 0.3698

At 30 docs 0.3050

At 100 docs 0.1500

At 200 docs 0.0947

At 500 docs 0.0474

At 1000 docs 0.0272

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3485

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short with concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short with concepts)

R2D21

Summary Statistics

Run ID R2D21

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1127

Summary Statistics

Run ID R2D21

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1427

Recall Level Precision Averages

Recall Precision

0.00 0.7033

0.10 0.6064

0.20 0.4943

0.30 0.4114

0.40 0.3415

0.50 0.2914

0.60 0.2329

0.70 0.1688

0.80 0.1342

0.90 0.0455

1.00 0.0352

Average precision over all

relevant documents

non-interpolated 0.2933

Document Level Averages

Precision

At 5 docs 0.4830

At 10 docs 0.3925

At 15 docs 0.3346

At 20 docs 0.2953

At 30 docs 0.2421

At 100 docs 0.1215

At 200 docs 0.0743

At 500 docs 0.0368

At 1000 docs 0.0213

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3017

Recall Level Precision Averages

Recall Precision

0.00 0.7772

0.10 0.6205

0.20 0.5137

0.30 0.4104

0.40 0.3494

0.50 0.2938

0.60 0.2457

0.70 0.1922

0.80 0.1509

0.90 0.0600

1.00 0.0358

Average precision over all

relevant documents

non-interpolated 0.3097

Document Level Averages

Precision

At 5 docs 0.5358

At 10 docs 0.4528

At 15 docs 0.3925

At 20 docs 0.3528

At 30 docs 0.2975

At 100 docs 0.1511

At 200 docs 0.0928

At 500 docs 0.0463

At 1000 docs 0.0269

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3199

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

R2D22

Summary Statistics

Run ID R2D22

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1278

Summary Statistics

Run ID R2D22

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1600

Recall Level Precision Averages

Recall Precision

0.00 0.7181

0.10 0.6057

0.20 0.5370

0.30 0.4631

0.40 0.3865

0.50 0.3295

0.60 0.2524

0.70 0.2063

0.80 0.1692

0.90 0.0791

1.00 0.0550

Average precision over all

relevant documents

non-interpolated 0.3289

Document Level Averages

Precision

At 5 docs 0.4906

At 10 docs 0.4094

At 15 docs 0.3547

At 20 docs 0.3113

At 30 docs 0.2629

At 100 docs 0.1370

At 200 docs 0.0852

At 500 docs 0.0422

At 1000 docs 0.0241

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3323

Recall Level Precision Averages

Recall Precision

0.00 0.7802

0.10 0.6252

0.20 0.5554

0.30 0.4797

0.40 0.3906

0.50 0.3444

0.60 0.2715

0.70 0.2353

0.80 0.1820

0.90 0.0933

1.00 0.0503

Average precision over all

relevant documents

non-interpolated 0.3473

Document Level Averages

Precision

At 5 docs 0.5509

At 10 docs 0.4698

At 15 docs 0.4201

At 20 docs 0.3802

At 30 docs 0.3270

At 100 docs 0.1732

At 200 docs 0.1072

At 500 docs 0.0532

At 1000 docs 0.0302

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3555

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

R2D23

Summary Statistics

Run ID R2D23

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1039

Summary Statistics

Run ID R2D23

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1318

Recall Level Precision Averages

Recall Precision

0.00 0.6958

0.10 0.5858

0.20 0.4735

0.30 0.3957

0.40 0.3234

0.50 0.2776

0.60 0.2177

0.70 0.1493

0.80 0.1171

0.90 0.0443

1.00 0.0336

Average precision over all

relevant documents

non-interpolated 0.2784

Document Level Averages

Precision

At 5 docs 0.4755

At 10 docs 0.3868

At 15 docs 0.3220

At 20 docs 0.2840

At 30 docs 0.2302

At 100 docs 0.1085

At 200 docs 0.0701

At 500 docs 0.0331

At 1000 docs 0.0196

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3010

Recall Level Precision Averages

Recall Precision

0.00 0.7521

0.10 0.5835

0.20 0.4974

0.30 0.3958

0.40 0.3176

0.50 0.2644

0.60 0.2053

0.70 0.1638

0.80 0.1277

0.90 0.0577

1.00 0.0281

Average precision over all

relevant documents

non-interpolated 0.2862

Document Level Averages

Precision

At 5 docs 0.5245

At 10 docs 0.4340

At 15 docs 0.3736

At 20 docs 0.3387

At 30 docs 0.2836

At 100 docs 0.1358

At 200 docs 0.0872

At 500 docs 0.0419

At 1000 docs 0.0249

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3139

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

R2D24

Summary Statistics

Run ID R2D24

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1184

Summary Statistics

Run ID R2D24

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1492

Recall Level Precision Averages

Recall Precision

0.00 0.7022

0.10 0.5985

0.20 0.5294

0.30 0.4294

0.40 0.3542

0.50 0.3014

0.60 0.2375

0.70 0.1899

0.80 0.1540

0.90 0.0598

1.00 0.0452

Average precision over all

relevant documents

non-interpolated 0.3103

Document Level Averages

Precision

At 5 docs 0.4868

At 10 docs 0.4094

At 15 docs 0.3447

At 20 docs 0.2925

At 30 docs 0.2434

At 100 docs 0.1292

At 200 docs 0.0785

At 500 docs 0.0389

At 1000 docs 0.0223

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3225

Recall Level Precision Averages

Recall Precision

0.00 0.7468

0.10 0.6147

0.20 0.5416

0.30 0.4304

0.40 0.3540

0.50 0.3018

0.60 0.2541

0.70 0.2145

0.80 0.1605

0.90 0.0740

1.00 0.0400

Average precision over all

relevant documents

non-interpolated 0.3219

Document Level Averages

Precision

At 5 docs 0.5245

At 10 docs 0.4566

At 15 docs 0.4038

At 20 docs 0.3566

At 30 docs 0.3025

At 100 docs 0.1626

At 200 docs 0.0990

At 500 docs 0.0494

At 1000 docs 0.0282

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3328

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

STIX1

Summary Statistics

Run ID STIX1

Task adhoc

Relevant Level Relevant

Run Description interactive,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 52404

Relevant: 1910

Rel-ret: 350

Summary Statistics

Run ID STIX1

Task adhoc

Relevant Level Partial Relevant

Run Description interactive,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 52404

Relevant: 2345

Rel-ret: 445

Recall Level Precision Averages

Recall Precision

0.00 0.3218

0.10 0.1968

0.20 0.0967

0.30 0.0467

0.40 0.0087

0.50 0.0020

0.60 0.0006

0.70 0.0002

0.80 0.0002

0.90 0.0000

1.00 0.0000

Average precision over all

relevant documents

non-interpolated 0.0460

Document Level Averages

Precision

At 5 docs 0.1434

At 10 docs 0.0981

At 15 docs 0.0742

At 20 docs 0.0623

At 30 docs 0.0465

At 100 docs 0.0249

At 200 docs 0.0155

At 500 docs 0.0096

At 1000 docs 0.0066

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.0723

Recall Level Precision Averages

Recall Precision

0.00 0.3713

0.10 0.2069

0.20 0.0842

0.30 0.0383

0.40 0.0082

0.50 0.0020

0.60 0.0003

0.70 0.0003

0.80 0.0003

0.90 0.0002

1.00 0.0000

Average precision over all

relevant documents

non-interpolated 0.0438

Document Level Averages

Precision

At 5 docs 0.1774

At 10 docs 0.1170

At 15 docs 0.0868

At 20 docs 0.0726

At 30 docs 0.0541

At 100 docs 0.0309

At 200 docs 0.0200

At 500 docs 0.0117

At 1000 docs 0.0084

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.0723

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(interactive) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(interactive) (short without concepts)

STIX2

Summary Statistics

Run ID STIX2

Task adhoc

Relevant Level Relevant

Run Description interactive,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 52404

Relevant: 1910

Rel-ret: 350

Summary Statistics

Run ID STIX2

Task adhoc

Relevant Level Partial Relevant

Run Description interactive,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 52404

Relevant: 2345

Rel-ret: 445

Recall Level Precision Averages

Recall Precision

0.00 0.3110

0.10 0.1877

0.20 0.0934

0.30 0.0470

0.40 0.0090

0.50 0.0020

0.60 0.0006

0.70 0.0002

0.80 0.0002

0.90 0.0000

1.00 0.0000

Average precision over all

relevant documents

non-interpolated 0.0446

Document Level Averages

Precision

At 5 docs 0.1472

At 10 docs 0.0943

At 15 docs 0.0679

At 20 docs 0.0557

At 30 docs 0.0434

At 100 docs 0.0249

At 200 docs 0.0156

At 500 docs 0.0095

At 1000 docs 0.0066

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.0710

Recall Level Precision Averages

Recall Precision

0.00 0.3603

0.10 0.1942

0.20 0.0820

0.30 0.0386

0.40 0.0084

0.50 0.0020

0.60 0.0003

0.70 0.0003

0.80 0.0003

0.90 0.0002

1.00 0.0000

Average precision over all

relevant documents

non-interpolated 0.0429

Document Level Averages

Precision

At 5 docs 0.1774

At 10 docs 0.1113

At 15 docs 0.0805

At 20 docs 0.0660

At 30 docs 0.0509

At 100 docs 0.0315

At 200 docs 0.0201

At 500 docs 0.0117

At 1000 docs 0.0084

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.0683

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(interactive) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(interactive) (short without concepts)

UTS1

Summary Statistics

Run ID UTS1

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 930

Summary Statistics

Run ID UTS1

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1210

Recall Level Precision Averages

Recall Precision

0.00 0.4052

0.10 0.3262

0.20 0.2365

0.30 0.1857

0.40 0.1575

0.50 0.1295

0.60 0.1009

0.70 0.0694

0.80 0.0482

0.90 0.0309

1.00 0.0245

Average precision over all

relevant documents

non-interpolated 0.1412

Document Level Averages

Precision

At 5 docs 0.2151

At 10 docs 0.2019

At 15 docs 0.1899

At 20 docs 0.1698

At 30 docs 0.1390

At 100 docs 0.0791

At 200 docs 0.0540

At 500 docs 0.0294

At 1000 docs 0.0175

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.1645

Recall Level Precision Averages

Recall Precision

0.00 0.4742

0.10 0.3549

0.20 0.2826

0.30 0.2278

0.40 0.1836

0.50 0.1555

0.60 0.1116

0.70 0.0854

0.80 0.0581

0.90 0.0396

1.00 0.0282

Average precision over all

relevant documents

non-interpolated 0.1623

Document Level Averages

Precision

At 5 docs 0.2679

At 10 docs 0.2491

At 15 docs 0.2365

At 20 docs 0.2142

At 30 docs 0.1830

At 100 docs 0.1030

At 200 docs 0.0691

At 500 docs 0.0376

At 1000 docs 0.0228

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.1996

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

UTS2

Summary Statistics

Run ID UTS2

Task adhoc

Relevant Level Relevant

Run Description automatic,long without cocepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1136

Summary Statistics

Run ID UTS2

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,long without cocepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1521

Recall Level Precision Averages

Recall Precision

0.00 0.7009

0.10 0.6219

0.20 0.4960

0.30 0.4380

0.40 0.3737

0.50 0.2984

0.60 0.2593

0.70 0.1913

0.80 0.1307

0.90 0.0664

1.00 0.0429

Average precision over all

relevant documents

non-interpolated 0.3043

Document Level Averages

Precision

At 5 docs 0.4377

At 10 docs 0.3830

At 15 docs 0.3421

At 20 docs 0.3019

At 30 docs 0.2579

At 100 docs 0.1334

At 200 docs 0.0809

At 500 docs 0.0385

At 1000 docs 0.0214

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3243

Recall Level Precision Averages

Recall Precision

0.00 0.7649

0.10 0.6627

0.20 0.5635

0.30 0.4963

0.40 0.4261

0.50 0.3436

0.60 0.2833

0.70 0.2120

0.80 0.1352

0.90 0.0817

1.00 0.0377

Average precision over all

relevant documents

non-interpolated 0.3420

Document Level Averages

Precision

At 5 docs 0.5321

At 10 docs 0.4736

At 15 docs 0.4252

At 20 docs 0.3792

At 30 docs 0.3283

At 100 docs 0.1751

At 200 docs 0.1084

At 500 docs 0.0517

At 1000 docs 0.0287

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3688

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long without concepts)

WSLab1

Summary Statistics

Run ID WSLab1

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 48622

Relevant: 1910

Rel-ret: 595

Summary Statistics

Run ID WSLab1

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 48622

Relevant: 2345

Rel-ret: 825

Recall Level Precision Averages

Recall Precision

0.00 0.1875

0.10 0.1490

0.20 0.1151

0.30 0.0896

0.40 0.0609

0.50 0.0512

0.60 0.0397

0.70 0.0214

0.80 0.0127

0.90 0.0044

1.00 0.0012

Average precision over all

relevant documents

non-interpolated 0.0585

Document Level Averages

Precision

At 5 docs 0.0943

At 10 docs 0.0868

At 15 docs 0.0931

At 20 docs 0.0830

At 30 docs 0.0711

At 100 docs 0.0453

At 200 docs 0.0322

At 500 docs 0.0184

At 1000 docs 0.0112

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.0747

Recall Level Precision Averages

Recall Precision

0.00 0.2272

0.10 0.1680

0.20 0.1333

0.30 0.0899

0.40 0.0697

0.50 0.0605

0.60 0.0439

0.70 0.0200

0.80 0.0125

0.90 0.0042

1.00 0.0012

Average precision over all

relevant documents

non-interpolated 0.0651

Document Level Averages

Precision

At 5 docs 0.1170

At 10 docs 0.1132

At 15 docs 0.1170

At 20 docs 0.1066

At 30 docs 0.0969

At 100 docs 0.0626

At 200 docs 0.0441

At 500 docs 0.0254

At 1000 docs 0.0156

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.0922

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

WSLab2

Summary Statistics

Run ID WSLab2

Task adhoc

Relevant Level Relevant

Run Description automatic,long without cocepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 736

Summary Statistics

Run ID WSLab2

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,long without cocepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1002

Recall Level Precision Averages

Recall Precision

0.00 0.2866

0.10 0.2411

0.20 0.1709

0.30 0.1279

0.40 0.0881

0.50 0.0736

0.60 0.0503

0.70 0.0316

0.80 0.0185

0.90 0.0065

1.00 0.0023

Average precision over all

relevant documents

non-interpolated 0.0882

Document Level Averages

Precision

At 5 docs 0.1623

At 10 docs 0.1283

At 15 docs 0.1233

At 20 docs 0.1123

At 30 docs 0.0975

At 100 docs 0.0589

At 200 docs 0.0417

At 500 docs 0.0227

At 1000 docs 0.0139

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.1084

Recall Level Precision Averages

Recall Precision

0.00 0.3644

0.10 0.2506

0.20 0.1908

0.30 0.1384

0.40 0.1068

0.50 0.0886

0.60 0.0641

0.70 0.0355

0.80 0.0172

0.90 0.0069

1.00 0.0023

Average precision over all

relevant documents

non-interpolated 0.1001

Document Level Averages

Precision

At 5 docs 0.1811

At 10 docs 0.1566

At 15 docs 0.1585

At 20 docs 0.1500

At 30 docs 0.1283

At 100 docs 0.0770

At 200 docs 0.0541

At 500 docs 0.0305

At 1000 docs 0.0189

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.1326

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long without concepts)

jscb1

Summary Statistics

Run ID jscb1

Task adhoc

Relevant Level Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1374

Summary Statistics

Run ID jscb1

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,short without concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1764

Recall Level Precision Averages

Recall Precision

0.00 0.6834

0.10 0.6172

0.20 0.5209

0.30 0.4820

0.40 0.4131

0.50 0.3731

0.60 0.3148

0.70 0.2565

0.80 0.2203

0.90 0.1437

1.00 0.0923

Average precision over all

relevant documents

non-interpolated 0.3596

Document Level Averages

Precision

At 5 docs 0.4528

At 10 docs 0.3962

At 15 docs 0.3572

At 20 docs 0.3245

At 30 docs 0.2843

At 100 docs 0.1455

At 200 docs 0.0919

At 500 docs 0.0462

At 1000 docs 0.0259

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3505

Recall Level Precision Averages

Recall Precision

0.00 0.7163

0.10 0.6351

0.20 0.5678

0.30 0.5209

0.40 0.4367

0.50 0.3987

0.60 0.3393

0.70 0.2801

0.80 0.2275

0.90 0.1311

1.00 0.0616

Average precision over all

relevant documents

non-interpolated 0.3790

Document Level Averages

Precision

At 5 docs 0.5321

At 10 docs 0.4755

At 15 docs 0.4239

At 20 docs 0.3896

At 30 docs 0.3491

At 100 docs 0.1887

At 200 docs 0.1208

At 500 docs 0.0600

At 1000 docs 0.0333

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.3804

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (short without concepts)

jscb2

Summary Statistics

Run ID jscb2

Task adhoc

Relevant Level Relevant

Run Description automatic,long with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1480

Summary Statistics

Run ID jscb2

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,long with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1895

Recall Level Precision Averages

Recall Precision

0.00 0.8398

0.10 0.7408

0.20 0.6687

0.30 0.5896

0.40 0.5231

0.50 0.4660

0.60 0.4004

0.70 0.3350

0.80 0.2609

0.90 0.1603

1.00 0.0965

Average precision over all

relevant documents

non-interpolated 0.4436

Document Level Averages

Precision

At 5 docs 0.5962

At 10 docs 0.5170

At 15 docs 0.4377

At 20 docs 0.3887

At 30 docs 0.3327

At 100 docs 0.1711

At 200 docs 0.1058

At 500 docs 0.0512

At 1000 docs 0.0279

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.4301

Recall Level Precision Averages

Recall Precision

0.00 0.9224

0.10 0.8041

0.20 0.7403

0.30 0.6551

0.40 0.5814

0.50 0.5097

0.60 0.4323

0.70 0.3381

0.80 0.2773

0.90 0.1744

1.00 0.0794

Average precision over all

relevant documents

non-interpolated 0.4852

Document Level Averages

Precision

At 5 docs 0.6981

At 10 docs 0.6170

At 15 docs 0.5358

At 20 docs 0.4783

At 30 docs 0.4101

At 100 docs 0.2221

At 200 docs 0.1388

At 500 docs 0.0660

At 1000 docs 0.0358

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.4673

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long with concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long with concepts)

jscb3

Summary Statistics

Run ID jscb3

Task adhoc

Relevant Level Relevant

Run Description interactive,long with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 1571

Summary Statistics

Run ID jscb3

Task adhoc

Relevant Level Partial Relevant

Run Description interactive,long with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1989

Recall Level Precision Averages

Recall Precision

0.00 0.8626

0.10 0.7769

0.20 0.6925

0.30 0.6363

0.40 0.5606

0.50 0.5044

0.60 0.4481

0.70 0.3945

0.80 0.3329

0.90 0.2073

1.00 0.1128

Average precision over all

relevant documents

non-interpolated 0.4855

Document Level Averages

Precision

At 5 docs 0.6679

At 10 docs 0.5340

At 15 docs 0.4579

At 20 docs 0.4066

At 30 docs 0.3522

At 100 docs 0.1823

At 200 docs 0.1133

At 500 docs 0.0544

At 1000 docs 0.0296

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.4648

Recall Level Precision Averages

Recall Precision

0.00 0.9431

0.10 0.8597

0.20 0.7727

0.30 0.7122

0.40 0.6340

0.50 0.5724

0.60 0.4966

0.70 0.4143

0.80 0.3196

0.90 0.2214

1.00 0.0951

Average precision over all

relevant documents

non-interpolated 0.5358

Document Level Averages

Precision

At 5 docs 0.7811

At 10 docs 0.6434

At 15 docs 0.5610

At 20 docs 0.5000

At 30 docs 0.4358

At 100 docs 0.2364

At 200 docs 0.1471

At 500 docs 0.0695

At 1000 docs 0.0375

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.5007

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(interactive) (long with concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(interactive) (long with concepts)

rmit02

Summary Statistics

Run ID rmit02

Task adhoc

Relevant Level Relevant

Run Description automatic,long with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 963

Summary Statistics

Run ID rmit02

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,long with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1289

Recall Level Precision Averages

Recall Precision

0.00 0.6029

0.10 0.4583

0.20 0.3761

0.30 0.3211

0.40 0.2566

0.50 0.2025

0.60 0.1408

0.70 0.0940

0.80 0.0708

0.90 0.0478

1.00 0.0275

Average precision over all

relevant documents

non-interpolated 0.2166

Document Level Averages

Precision

At 5 docs 0.3811

At 10 docs 0.3094

At 15 docs 0.2604

At 20 docs 0.2330

At 30 docs 0.1943

At 100 docs 0.0970

At 200 docs 0.0592

At 500 docs 0.0310

At 1000 docs 0.0182

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2447

Recall Level Precision Averages

Recall Precision

0.00 0.6990

0.10 0.5242

0.20 0.4414

0.30 0.3657

0.40 0.2922

0.50 0.2351

0.60 0.1569

0.70 0.1085

0.80 0.0706

0.90 0.0552

1.00 0.0337

Average precision over all

relevant documents

non-interpolated 0.2490

Document Level Averages

Precision

At 5 docs 0.4830

At 10 docs 0.3906

At 15 docs 0.3396

At 20 docs 0.3047

At 30 docs 0.2560

At 100 docs 0.1289

At 200 docs 0.0793

At 500 docs 0.0418

At 1000 docs 0.0243

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2800

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long with concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long with concepts)

sstut3

Summary Statistics

Run ID sstut3

Task adhoc

Relevant Level Relevant

Run Description automatic,long without cocepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 52999

Relevant: 1910

Rel-ret: 1028

Summary Statistics

Run ID sstut3

Task adhoc

Relevant Level Partial Relevant

Run Description automatic,long without cocepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 52999

Relevant: 2345

Rel-ret: 1318

Recall Level Precision Averages

Recall Precision

0.00 0.6456

0.10 0.5406

0.20 0.4147

0.30 0.3356

0.40 0.2642

0.50 0.2004

0.60 0.1427

0.70 0.1028

0.80 0.0693

0.90 0.0256

1.00 0.0215

Average precision over all

relevant documents

non-interpolated 0.2303

Document Level Averages

Precision

At 5 docs 0.4075

At 10 docs 0.3340

At 15 docs 0.2943

At 20 docs 0.2651

At 30 docs 0.2151

At 100 docs 0.1091

At 200 docs 0.0673

At 500 docs 0.0334

At 1000 docs 0.0194

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2544

Recall Level Precision Averages

Recall Precision

0.00 0.7586

0.10 0.5823

0.20 0.4822

0.30 0.3624

0.40 0.3029

0.50 0.2172

0.60 0.1615

0.70 0.1155

0.80 0.0783

0.90 0.0356

1.00 0.0186

Average precision over all

relevant documents

non-interpolated 0.2583

Document Level Averages

Precision

At 5 docs 0.4868

At 10 docs 0.4189

At 15 docs 0.3648

At 20 docs 0.3264

At 30 docs 0.2667

At 100 docs 0.1379

At 200 docs 0.0858

At 500 docs 0.0431

At 1000 docs 0.0249

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.2948

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long without concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(automatic) (long without concepts)

tgl1

Summary Statistics

Run ID tgl1

Task adhoc

Relevant Level Relevant

Run Description interactive,long with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 1910

Rel-ret: 857

Summary Statistics

Run ID tgl1

Task adhoc

Relevant Level Partial Relevant

Run Description interactive,long with concepts

Number of Topics 53

Total number of documents over all topics

Retrieved: 53000

Relevant: 2345

Rel-ret: 1139

Recall Level Precision Averages

Recall Precision

0.00 0.4267

0.10 0.2937

0.20 0.1866

0.30 0.1462

0.40 0.1181

0.50 0.1018

0.60 0.0763

0.70 0.0532

0.80 0.0232

0.90 0.0088

1.00 0.0052

Average precision over all

relevant documents

non-interpolated 0.1168

Document Level Averages

Precision

At 5 docs 0.2000

At 10 docs 0.1698

At 15 docs 0.1459

At 20 docs 0.1396

At 30 docs 0.1195

At 100 docs 0.0657

At 200 docs 0.0461

At 500 docs 0.0257

At 1000 docs 0.0162

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.1379

Recall Level Precision Averages

Recall Precision

0.00 0.4755

0.10 0.3113

0.20 0.2059

0.30 0.1708

0.40 0.1365

0.50 0.1173

0.60 0.0794

0.70 0.0351

0.80 0.0184

0.90 0.0087

1.00 0.0034

Average precision over all

relevant documents

non-interpolated 0.1262

Document Level Averages

Precision

At 5 docs 0.2491

At 10 docs 0.2094

At 15 docs 0.1836

At 20 docs 0.1745

At 30 docs 0.1491

At 100 docs 0.0864

At 200 docs 0.0612

At 500 docs 0.0342

At 1000 docs 0.0215

R-Presicion (presicion after

R docs retreived (where R is

the number of relevant

documnets))

Exact 0.1581

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

0

0.2

0.4

0.6

0.8

1

0 0.2 0.4 0.6 0.8 1

Pre

cisi

on

Recall

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Recall-Precision Curve Di�erence from Median in

Average Precision per Topic

(all)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(interactive) (long with concepts)

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

-1-0.8-0.6-0.4-0.2

00.20.40.60.8

1

40 50 60 70 80

Diff

eren

ce

Topic

Di�erence from Median in Di�erence from Median in

Average Precision per Topic Average Precision per Topic

(interactive) (long with concepts)