Development of an Assessment System for Engineering Courses Using Petri-Net

Tung-Nan Institute of Technology, Taiwan*Tamkang University , Taiwan**

Yu-Hur Chou*Hsin-Yih SHYU**

Outline

1. Objectives

2. System Architecture

3. System Analyses and Design

4. System Demonstrations (Test Demo.)

5. Conclusions

Objectives

1. To specify how to design and develop a partial scoring assessment system.

2. This assessment system can make inference diagnosis in order to investigate examinee’s misconceptions and make the reasonable scoring for engineering courses.

3. Reinforced Concrete Design is the objective course.

System Architecture

Test Generator

TestDatabase

Subject (Z)

Step 1. Select Test Subject

TestDatabase

Subject (M)

TestDatabase

Subject (A)

Active Database

Select Test Concepts

Step 2. Select Test Concepts and style.

Test Items

AnsweringEvaluator

CorrectAnswers

Concept-MappingReply sub-system

Examinees’Answers

DiagnosticEvaluator

DiagnosisResults

Step 3. User Reply

Step 4. Assessment results

Database System

: Database

: Internal Information

: Sub-system

System Analyses and Design (1/7)

1. Defining Subject Concepts.

2. Defining Subject Calculations.

3. Evaluation between Concepts and Calculations.

4. Construct the Logic Structure of Correct Answer.

5. Partial Scoring Function.

6. Misconception Evaluation.

RC Beam Analysis (with Single Layer Bar) as example

Procedures:

System Analyses and Design (2/7)

1. Defining Subject Concepts:Item Concept name Item Concept name

1 Required Resisting Strength 12 Parameter of Equivalent Stress Block

2 Stress-Strain Diagram (Reinforcement)

13 Equivalent Stress Block

3 Modulus of Elasticity (Reinforcement)

14 Concept of Under reinforced Case

4 No. Of Reinforcing Steel 15 Concept of Over reinforced Case

5 Area of Reinforcing Steel 16 Concept of Balanced Case

6 Modulus of Elasticity (Concrete) 17 Concept of Proportional Ratio

7 Ultimate Strain of Concrete 18 Equilibrium of Forces

8 Compressive Strength of Concrete

19 Solution of Linear Equations

9 Capacity Reduction Factor 20 Central of Cross-Section

10 Nominal Resisting Strength 21 Solution of second order equation

11 Designed Resisting Strength

System Analyses and Design (3/7)

2. Defining Subject Calculations:Item Calculation Name Item Calculation Name

1 Balanced Depth of Neutral Axis 12 Depth of Equivalent Stress Block

2 Modulus of Elasticity (Concrete) 13 Depth of Neutral Axis (Balance)

3 Area of Bar 14 Stress of Reinforcement (Balance)

4 Depth of Equivalent Stress Block (Balance)

15 Stress of Reinforcement (Under reinforcement)

5 Parameter of Equivalent Stress Block

16 Nominal Resisting Moment (Tension)

6 Under reinforced Case 17 Nominal Resisting Moment (Compression)

7 Area of Bar (Balance) 18 Total Tensile Force

8 Balanced Reinforcement Ratio 19 Depth of Neutral Axis (Under reinforcement)

9 Over reinforced Case 20 Depth of Neutral Axis (Over reinforcement)

10 Balanced Case 21 Stress of Reinforcement (Over reinforcement)

11 Total Compressive Force 22 Ultimate Resisting Moment

System Analyses and Design (4/7)

3. Evaluation between Concepts and Calculations:

Relationship betweenQi and Cj

Concept Cj

C1 C2 C3 ……. Cn

 Calculation

ItemQi

Q1 e11 e12 e13 ……. e1n

Q2 e21 e22 e23 ……. e2n

Q3 e31 e32 e33 ……. e3n

…. …. …. …. ……. ….

Qm em1 em2 em3 ……. emn

eij = 1 (related ) , eij = 0 (irrelated )

System Analyses and Design (5/7)

4. Construct the Logic Structure of Correct Answer:

Total Tensile Force

T

Asfs

ss AfT

Parameter2

Parameter Set

Calculation 1 Calculation 2 Calculation 3 Calculation m

Calculation Set

Input Arcs Output Arcs

Parameter1

Parameter3

Parameter4

ParameterN

An independent unit

The Rule-based Petri-Net

The model of the Rule-based Petri-Net

System Analyses and Design (6/7)

5. Partial Scoring Function:Score = WC x GCI + WS x CRC WhereScore: Final score.GCI: The value of Goldsmith’s Closeness Index (Similarity between correct and examiner's concept map).CRC: The value of Correct Rate of Calculations.WC: Weight of GCI (0~1).WS: Weight of CRC (1-WC).

CRC = Σ( Pi x Di ) / ΣDi ( i = 1 ~ m ) WhereCRC: The value of Correct Rate of Calculations.Pi: The value of each calculation (0 or 1).

Di: The difficulty value of each calculation (1(easy) ~9(hard)).

m: The number of calculations within a test item.

System Analyses and Design (7/7)

6. Misconception Evaluation:

(1) Complete correctness: (2) No reply, but system infers complete correctness: (3) Correct calculation (wrong answer from mistake inheritance): (4) No reply, but system infers correct calculation: (5) Wrong answer: (6) No reply:

Calculation judgment

CRV (i) = WC x C (i) + WS x Pi ( i = 1 ~ m )

WhereCRV (i): The Complete Rating Value of calculation i.C (i): The closeness coefficient of calculation i.

System Demonstrations (1/8)

Entry Screen (Select the Test Subject)

Test Generator

Step 1. Select Test Subject

TestDatabase

Subject (M)Active Database

Select Test Concepts

Step 2. Select Test Concepts and style.

Test Items

AnsweringEvaluator

CorrectAnswers

Concept-MappingReply sub-system

Examinees’Answers

DiagnosticEvaluator

DiagnosisResults

Step 3. User Reply

Step 4. Assessment results

System Demonstrations (2/8)

Select Concepts and Test Style

Test Generator

Step 1. Select Test Subject

TestDatabase

Subject (M)Active Database

Select Test Concepts

Step 2. Select Test Concepts and style.

Test Items

AnsweringEvaluator

CorrectAnswers

Concept-MappingReply sub-system

Examinees’Answers

DiagnosticEvaluator

DiagnosisResults

Step 3. User Reply

Step 4. Assessment results

System Demonstrations (3/8)

Problem’s Description

Test Generator

Step 1. Select Test Subject

TestDatabase

Subject (M)Active Database

Select Test Concepts

Step 2. Select Test Concepts and style.

Test Items

AnsweringEvaluator

CorrectAnswers

Concept-MappingReply sub-system

Examinees’Answers

DiagnosticEvaluator

DiagnosisResults

Step 3. User Reply

Step 4. Assessment results

System Demonstrations (4/8)

Answer Evaluator

Test Generator

Step 1. Select Test Subject

TestDatabase

Subject (M)Active Database

Select Test Concepts

Step 2. Select Test Concepts and style.

Test Items

AnsweringEvaluator

CorrectAnswers

Concept-MappingReply sub-system

Examinees’Answers

DiagnosticEvaluator

DiagnosisResults

Step 3. User Reply

Step 4. Assessment results

System Demonstrations (5/8)

Concept Mapping Reply System

Test Generator

Step 1. Select Test Subject

TestDatabase

Subject (M)Active Database

Select Test Concepts

Step 2. Select Test Concepts and style.

Test Items

AnsweringEvaluator

CorrectAnswers

Concept-MappingReply sub-system

Examinees’Answers

DiagnosticEvaluator

DiagnosisResults

Step 3. User Reply

Step 4. Assessment results

System Demonstrations (6/8)

Assessment Results

Test Generator

Step 1. Select Test Subject

TestDatabase

Subject (M)Active Database

Select Test Concepts

Step 2. Select Test Concepts and style.

Test Items

AnsweringEvaluator

CorrectAnswers

Concept-MappingReply sub-system

Examinees’Answers

DiagnosticEvaluator

DiagnosisResults

Step 3. User Reply

Step 4. Assessment results

System Demonstrations (7/8)

Scoring Instruction

Test Generator

Step 1. Select Test Subject

TestDatabase

Subject (M)Active Database

Select Test Concepts

Step 2. Select Test Concepts and style.

Test Items

AnsweringEvaluator

CorrectAnswers

Concept-MappingReply sub-system

Examinees’Answers

DiagnosticEvaluator

DiagnosisResults

Step 3. User Reply

Step 4. Assessment results

System Demonstrations (8/8)

Misconception Evaluation

Test Demonstration (1/6)

Step 1: Select Test Subject.

Step 2: Select Test Concepts and Style.

Test Demonstration (2/6)

Step 3 : Problem description (generated by system)

Correct Answer Test Demonstration (3/6)

Step 4: User’s Reply Answer

36.43 cm 0.85

30.97 cm

86.99 cm2

21.85 cm

159.18 T

1(True)

36.42 cm 0.8

29.14 cm

81.87 cm2

23.21 cm

159.21 T

Step 5: Assessment Begin

Test Demonstration (4/6)

Step 6: Scoring Instruction

Test Demonstration (5/6)

Step 7: Diagnosis Instruction

Test Demonstration (6/6)

Conclusions

1. This assessment system adopts the dynamic state to make test items. The values of the test variables are generated randomly. Therefore, the same test problem will be shown within different variable values for different tests, to prevent students from memorizing the answers.

2. We also purpose a Rule-based Petri-Net model for building the logic structure of correct answer. This logic structure can match the dynamic pattern and random parameters completely.

3. This system also adopts concept-mapping environment as examiner’s reply system, can acquire enough information to investigate examinee’s misconceptions and measure final score by using partial scoring strategy.

The EndThanks