# Premises = 2, # Vars = 3, # Inference steps = 2,
Inference Rules = {Modus Tollens, Simplification}
Automatically Generating Problems and Solutionsfor Natural Deduction
Natural Deduction• Method for establishing the validity of propositional
type arguments, where the conclusion is derived
from the premises through a series of discrete steps,
by application of some inference/replacement rule.
• Typically taught as part of an introductory course on
logic, which is a central component of college
education.
GoalBuilding an efficient computer-aided education system
for natural deduction that can perform
1. Solution Generation: step-by-step proof generation of
problems.
2. Problem Generation: Generate problems
a) having similar solution to a seed problem.
b) satisfying specified parameters.
Key Insights• Small-sized hypothesis: Propositions
occurring in educational contexts use small
number of variables and have small size.
• Truth-Table Representation: A proposition
can be abstracted using its truth-table,
which can be represented using a bitvector
representation.
• Offline Computation: The symbolic reasoning
required to pattern match propositions for
applying inference rules can be performed
and stored in an offline phase.
Contributions• We propose leveraging the key insights for building efficient
computer-aided education system for natural deduction.
• A novel two-phased methodology for solution generation that
first searches for an abstract solution and then refines it to a
concrete solution.
• A novel methodology for generating problems using a
process that is reverse of solution generation.
• We present detailed experimental results on 279 benchmark
textbook problems. Our tool can solve 84% of these and is
able to generate few thousands of similar / parameterized
problems on average per instance in a few minutes.
Conclusion• Computer-aided instruction can raise the quality of education by making it more interactive & customized.
• Our tool can free instructors from the burden of generating similar difficulty level variants for assignment problems
and creating their sample solutions, which could be useful in both Massive Open Online Courses (MOOCs) and
traditional classroom settings.
Universal Proof GraphThe key data structure used is an offline computed hyper-graph whose
nodes are truth table bit-vectors, and hyper-edges are inference rules.
Simp.
M.P. , M.T.
M.P. , M.T.
Conj.
Add.Add.
204240
15241
51
0243
P Q R ¬P ¬Q P ∧ Q P → Q P → (Q ∧ R)
0 0 0 1 1 0 1 1
0 0 1 1 1 0 1 1
0 1 0 1 0 0 1 1
0 1 1 1 0 0 1 1
1 0 0 0 1 0 0 0
1 0 1 0 1 0 0 0
1 1 0 0 0 1 1 0
1 1 1 0 0 1 1 1
15 51 85 240 204 3 243 241
Truth Table
Umair Z. [email protected]
Sumit [email protected]
Amey Karkare [email protected]
Premise-1 Premise-2 Conclusion
P → (Q ∧ R) ¬Q ¬P
Original Problem
Abstract ProofStep Truth-Table Reason
P1 241 Premise
P2 204 Premise
1 243 P1, Simplification
2 240 1, P2, Modus Tollens
Natural Deduction ProofStep Truth-Table Proposition Reason
P1 241 P → (Q ∧ R) Premise
P2 204 ¬Q Premise
1 241 ¬P ∨ (Q ∧ R) P1, Implication
2 241 (¬P ∨ Q) ∧ (¬P ∨ R) 1, Distribution
3 243 ¬P ∨ Q 2, Simplification
4 243 P → Q 3, Implication
5 240 ¬P 4, P2, Modus Tollens
3Conj.
Premise-1 Premise-2 Conclusion
Q ≡ ¬P P ∧ (Q ∨ R) P ∧ R
P ∧ Q Q → (P ∧ R) P ∧ R
(P → R) → Q (R → Q) → P P
Similar Problem Generation
Parameterized Problem Generation
Premise-1 Premise-2 Conclusion
P ∧ (R ∨ Q) P → (¬Q ∧ R) P ∧ R
R ≡ P (R ∨ P) ∧ (R ≡ ¬ Q) P ∧ R
(Q ≡ P) → (P ∧ R) (R ≡ P) ∧ (Q ≡ P) P ∧ R