January 12 (Tue), 2016 Introduction to the Course: Goals and Topics to Be Covered. Examples of plausible reasoning and causal graphs: icy roads and wet grass. Reasoning in the evidential and causal direction. Intercausal reasoning and explaining away.
January 16 (Thu), 2016 HW1: Do exercises 2.4, 2.5, 2.6, 2.7, and 2.8 [J07], due on Tuesday, 2016-01-19. Another example of plausible reasoning: burglary and earthquake. Definition of d-separation and d-connectedness (def. 2.1[J07]). Examples of d-separation. Definition of Markov blanket (def. 2.2 [J07]). The claim at the end of p.30 [J07] discussed briefly. A more formal treatment will start with explicit axioms, such as the graphoid axioms, from which the claim of p.30 [J07] (based on d-separation) follows. I linked a paper on uncertain evidence on the course website. It is in the "Useful Links" section. This may be a good paper for graduate student presentation.
January 19 (Thu), 2016 Q1. HW1 collected. Historical compositional approaches to uncertain reasoning. Uncertain reasoning is non-compositional. The Chernobyl example. Probability. The three axioms of Kolmogorov [1950]. Outcome space, event space, probability measure. Three historically important interpretations in which the axioms of Kolmogorov are true (i.e., three models of them): classical, frequentist, and subjective. Presentation of the classical approach, including sketch of the proof of the three properties that are the axioms of Kolmogorov. Brief presentation of the frequentist approach. Subjective probability defined from betting behavior: "[De Finetti] defines the probability P(E) of an event E as the fraction of a whole unit value which one would feel is the fair amount to exchange for the promise that one would receive a whole unit of value if E turns out to be true and zero units if E turns out to be false" [Neapolitan, 1990, p.56]. Similarly, subjective probability could be defined using the equivalent urn model. The probability P(E) of an event E is the fraction of red balls in an urn containing red and brown balls such that one would feel indifferent between the statement "E will occur" and "a red ball would be extracted from the urn." (I believe that this definition is due to D.V. Lindley.) Kolmogorov's axioms and the "definition" of conditional probability can be derived from the definition of subjective probability and the assumption of coherence in (betting) behavior, as explained on the slides.
January 21 (Thu), 2016 HW1 collected and discussed. Lauritzen's d-separation algorithm using moralized ancestral graphs. (A proof of correctness has been posted on the course website under "Lecture Notes."
January 26 (Tue), 2016 HW2 assigned: exercises 1.7, 1.11, and 1.12 [the original log entry had the incorrect exercises: 1.11, 1.17, and 1.13---this was an error!], due on Thursday, January 28. The fundamental rule, Bayes theorem. Potentials. Independence for variables. Conditional independence. Table operations are pointwise. By convention, the row of conditional probability tables are indexed by the states of the variable on the left-hand side of a conditioning bar. Bayesian networks are causal networks without directed cycles plus conditional probability tables for each variables. Icy roads example in detail using table operations.
January 28 (Thu), 2016 HW2 due date changed to 2016-02-02 (Tuesday). Wet lawn example in detail using table operations. Section 2.3.3 [J07]: definition of finding; evidence; the probability of evidence. The Athenian Taxi Example. Neapolitan's definition of Bayesian network [Neapolitan, 1990; in ppt Intro slides]. Proof of the Chain Rule for BNs from Neapolitan's definition. The visit to Asia (chest clinic) example.
February 2 (Tue), 2016 HW2 collected. Exercise 1.13 done in class. Ch.2 [J07] completed. Variable elimination for computing conditional probabilities (posterior probabilities). Statistics for reliability of information sources (after Vomlel; see "Useful Links" on course website): true positives, etc.
February 4 (Thu), 2016 HW2 returned and discussed in great detail. Discussion of use of absorption of evidence (P=yes) in the conditional probability table P(T|P) in an possible solution of exercise 1.7. More on variable elimination, including a full symbolic computation of P(t|a,d) for a particular order of summation, done on the board.
February 9 (Tue), 2016 The Belief Update, Most Probable Explanation and Maximum A Posteriori Hypothesis problems. Bucket elimination. The order of elimination does not affect the result, but it affects the complexity of computing a marginal. Better elimination orders lead to H functions that have small domains. Definition of interaction graph. Changes in the interaction graph depend on the order of elimination. Minimum degree heuristic.
February 11 (Thu), 2016 HW3 (renamed PR1) assigned: exercises 2.20 (install Hugin---instructions are in blackboard) and 2.23 [J07], due on 2016-02-16; please turn in a paper with (a) a statement that you installed Hugin, with version and operating system; (b) the solution of exercise 2.23 with at least one screen dump from Hugin. Two exercises from the spring 2014 final: d-separation in a particular BN structure, and an example of variable elimination (using bucket elimination). The coins and bell example: (a) two BNs; (b) the two BNs represent the same joint probability distribution and are therefore indistinguishable from statistical information alone; (c) the first BN network is a dependency map as well as an independency map; (d) the first network uses fewer parameters; (e) the second network can mimic the first, but not viceversal; (f) the first network is causal according to the excision (cutting) semantics, while the second one is not. Claim: causal Bayesian networks are more parsimonious (i.e., require fewer parameters) than non-causal Bayesian networks,
February 16 (Tue), 2016 HW3 (renamed PR1) will be collected on Thursday, February 18 (Thursday). Using Hugin. Checking for independence using Hugin. Detailed example (with visit to Asia BN) of computing the MPE with bucket elimination.
February 18 (Thu), 2016 HW3 (renamed PR1) collected. Outline of the proof that BNs admit d-separation, based on chapter 6 of: Richard E. Neapolitan. _Probabilistic Reasoning in Expert Systems: Theory and Algorithms_. Wiley, 1990. The stratum method for constructing Bayesian networks. Chapter 3 ("Building Models") [J07] started: one-day infected milk test (hypothesis vs. test variables), 7-day infected milk test with Markov assumption, 7-day milk test with two-day incubation period for infection, 7-day milk test with test correctness model.
February 23 (Tue), 2016 PR1 (formerly called HW3) returned. PR2 assigned: exercises 3.5, 3.6, 3.8, 3.9 [J07], due March 1, 2016 Midterm will be on Tuesday, March 1, 2016. Q2. Section 3.1 ("Catching the Structure") [J07] completed.
February 25 (Thu), 2016 Midterm will be on Tuesday, March 1, 2016. The midterm will be closed book. Students are encouraged to review the midterms from previous years that are available on course webpages, accessible from the instructor's webpage (https://cse.sc.edu/~mgv/). We briefly discussed the midterm from fall 2000 (CSCE 590; at the time, CSCE 582 did not exist). Section 3.1.6 [J07], which has an operational test related to Reichenbach's principle. Section 3.2 [J07] ("Determining the Conditional Probabilities") through 3.2.2.
March 1 (Tue), 2016 PR2 collected. On an exceptional basis, PR1 will be accepted through tomorrow at 1700 without penalty. Midterm (MT1). (As this log is written, MT1 grade has already been posted.)
March 3 (Thu), 2016 PR3 assigned: exercises 3.10, 3.12, 3.13, 3.14, 3.16 [J07] due on March 17. Please prepare questions on the assigned exercises to be asked on or before March 15. MT1 returned and corrected. PR2 returned. Section 3.2 [J07] ("Determining the Conditional Probabilities") continued through section 3.2.4.
March 15 (Tue), 2016 PR3 due date changed to March 22, 2016. PRG1 assigned: exercise 3.27 [J07], due on March 24, 2016. This assignment is only for students who are taking the course for graduate credit. Hugin is required for the first two parts. Students are encouraged to review the bibliographical notes at the end of chapter 3 [J07] for a pointer to a paper that will help with parts 3 and 4. Students may be asked to present their solution in class for the benefit of undergraduate students. PR2 returned. Discussion of issues related to exercises 3.8, 3.10, 3.12 (ii and iii), 3.13 (iv), and 3.16. (Note that a Bayesian network in Hugin is required as part of the solution of exercise 3.16, even though this exercise is not marked with the "E" superscript in the textbook). Section 3.2 [J07] ("Determining the Conditional Probabilities") continued through section 3.2; section 3.3.2 ("Noisy Or").
March 17 (Thu), 2016 HW3 (renamed; incorrectly named HW4 originally) assigned: exercise 3.20 [J07] due on Tuesday, March 22, 2016. Noisy-OR continued, Logical constraints ("Undirected Relations", 3.3.1 [J07]), Divorcing, Expert Disagreements, Conflicts.
March 22 (Tue), 2016 HW3 and PR2 collected. Sensitivity to evidence. Sensitivity to parameters (3.4.4 [J07]). Continuous variables (3.3.8 [J07]). Object-oriented Bayesian networks (3.3.6 [J07]). Dynamic Bayesian networks (3.3.7 [J07]). Belief Updating in Bayesian Networks (Chapter 4 [J07]) started: Section 4.1 ("Introductory Examples").
March 24 (Thu), 2016 PRG1 collected. Graph-Theoretic Representation (4.2 [J07]), Triangulated Graphs and Join Trees (4.3 [J07]) started. Slides 1-10 in "Junction tree propagation - BNDG 4-4.6") packet (on course website) used. Note that the correct elimination order for the Figure 4.7 ends with A4, not A2. Note that the formula for computing \phi^'(A1,A2,A5,A6) on slide 8 should include \phi_6(A6,A3) instead of \phi_4(A4,A2).
March 29 (Tue), 2016 PR2 and HW3 returned. PR2 and PRG1 discussed in detail. Join Trees (4.3 [J07]) continued. Slides 1-13 in "Junction tree propagation - BNDG 4-4.6") packet (on course website) used.
March 31 (Thu), 2016 HW4 assigned: exercises 4.19 and 4.20 [J07], due on Tuesday, April 5, 2016. Section 4.1-4.4 [J07] on Junction Tree Propagation completed.
April 5 (Tue), 2016 HW4 collected. Discussion of graduate student presentation. The first presentation will be on Tuesday, April 12. See the dedicated area of the course website for suggested papers. Definition of join graph and proof that join trees are exactly the maximal spanning trees of join graphs (using journal notes of 2009-03-20). Lazy propagation, with an example, and illustration of the fact that the separator mailboxes hold marginal distributions of the separator variables (using journal notes of 2009-03-25 and 2009-04-01). Stochastic simulations (section 4.8 [J07]) started (4.8.1, Probabilistic Logic Sampling), using journal notes of 2009-04-01.
April 7 (Thu), 2016 HW4 returned. Graduate students are encouraged to choose papers for presentation. So far, two students have made a choice. Stochastic simulations (section 4.8 [J07]) started (4.8.1, Probabilistic Logic Sampling; 4.8.2, Likelihood Weighting; 4.8.3, Gibbs Sampling), using journal notes of 2009-04-01 and slides from Thomas Nielsen. 4.7 (Exact Propagation with Bounded Space): 4.7.1, Recursive Conditioning (briefly). 4.9: Loopy Belief Propagation. Ch.9 [J07] (Graphical Languages for Specification of Decision Problems) started. Survey on lotteries A and B on p.289: 5 students prefer A, 8 students prefer B.
April 12 (Tue), 2016 HW5 assigned: exercise 9.8 [J07], due on April 19, 2016. PR4 assigned: exercise 9.11 (ii) [J07], due on April 21, 2016; do only part (ii); use Hugin; recent versions of Hugin only support LIMIDs (LImited Memory Influence Diagrams. First graduate student presentation: Mohamed Ali Javidian: Kasper L. Jensen, Jorn Toftum, Peter Friis-Hansen. "A Bayesian Network Approach to the Evaluation of Building Design and its Consequences for Employee Performance and Operational Costs." Building and Environment, 44 (2009), 456-462. Ch.9 [J07] continued. Survey completed (two lotteries). One shot decision problems (section 9.1), with the poker and mildew examples. Utilities (section 9.2) started: the management of effort example.
April 14 (Thu), 2016 Ch.9 [J07] continued. Axioms of instrumental rationality. Survey results used to describe the Allais paradox. Decision trees. Solving decision trees, i.e., computing the optimal policy at each decision node and the optimal strategy (i.e., the collection of optimal policies for every node). Introduction to influence diagrams. Information links. Difference between (no-forgetting) influence diagrams and LIMIDs (LImited-memory influence diagrams).
April 19 (Tue), 2016 HW5 collected. Presentation by Darren Harton of: Anders L. Madsen and Uffe B. Kjaerulff. "Applications of HUGIN to Diagnosis and Control of Autonomous Vehicles." Peter Lucas, Jose Gamez, and and Antonio Salmeron (eds.). Advances in Probabilistic Graphical Models. Springer, 313-332, 2007. Ch.9 [J07] completed. Description of the oil wildcatter problem.
April 19 (Tue), 2016 HW5 discussed and briefly returned---grades still need to be recorded, so it was only left with students who were asked to improve their solution. PR collected and discussed. Students are asked to turn in all late assignments before the final, which will be on Tuesday, May 3, at 1230 in the classroom. Discussion of final exam. Suggestions include: (1) review the final exam linked to the course website and to websites for previous editions of the course; (2) do exercise 9.11 (i) (oil wildcatter using decision trees). Presentation by Alireza Nasiri of: David N. Barton, Youssouf Cisse, Bocary Kaya, Ibrahima N'Diaye, Harouna Yossi, Abdoulaye Diarra, Souleymane Keita, Amadou Dembele, Daouda Maiga, Graciela M. Rusch, Anders L. Madsen. "Diagnosing agrosilvopastoral practices using Bayesian networks." Agroforestry Systems, March 2016 (online publication). Presentation by Asif J. Chowdhuri and Ahmed Shehab Khan of: Michalis Xenos. "Prediction and assessment of Student Behaviour in Open and Distance Education in Computers Using Bayesian Networks." Computers and Education, 43, 345-369 (2004). End of course survey.