CS417, Spring 2021

CS417 Outline

Spring 2021

General Information

Below are the modules that comprise the course content.

Each module consists of a series of activities.

  • Not every assigned activity requires you to submit something for grading.
    Nonetheless, you are expected to do them all.

  • If no due date is specified, you are supposed to complete the assigned
    activity by the end of the final day allotted for that entire module.

KEYS TO SUCCESS IN THIS COURSE:

  1. READ THE SYLLABUS

    The syllabus lays out the basic course policies. It tells you what you must
    do to earn a passing grade. It tells you how to get in touch with me if you
    run into problems.

  2. HAVE A SCHEDULE

    You have some freedom to schedule your own time in this course, but you
    DO need to set up a schedule. Do not forget that this course exists and that
    you are registered for it. Do not think you can repeatedly set it aside for
    days at a time and make up the time later.

  3. IF YOU DON’T UNDERSTAND SOMETHING, ASK QUESTIONS

    You can ask questions in the course Forums. You can contact me through email.
    You can contact me during office hours. You will find information on these
    options in the syllabus and Course Orientation module.

    Some people are too shy to ask questions. Some are too proud to ask
    questions. Part of being educated is knowing how to exploit your available
    information resources. In this course, I am one of those resources.

Upcoming Events
1 Online Orientation and Course Introductions 01/19/2021 - 01/24/2021

Overview

In this pseudo-module, we will discuss course organization, policies, and mechanics. We will review the CS 417 course website structure, and get set up for the semester to come.

We will take a brief look at the major themes and areas of emphasis that we will discuss during the coming semester. This will include a review of prerequisite mathematics (mainly, Calculus) and programming (namely, principles from CS 250). We will discuss the basics of Python 3 with particular emphasis on:

Objectives

  1. Identify requirements, protocols, policies, and expectations for CS 417.
  2. Discuss and utilize select tools for communication, collaboration, and documentation.
  3. Discuss course themes and their relation to computational methods.

Relevance

You must understand the fundamentals of the tools used any course to be successful. An understanding of expectations for participation, assignment submission, and time management skills is foundational in all academic endeavors.

Activities

Getting Started

  1. lecture Read lecture notes: Syllabus
  2. lecture Read lecture notes: Introduction & Overview
  3. lecture Read lecture notes: Discussion Board Guidance
  4. lecture Read lecture notes: Language Resources (from CS 330)

The Fun Part of Module 1

  1. asst Do assignment: Personal Introduction Post Due: 01/22/2021
  2. video Watch: The Beginning Lecture
  3. lecture Read lecture notes: The Beginning
  4. lecture Read lecture notes: Unfamiliar Notation
  5. lecture Read lecture notes: Unfamiliar Notation - Mountain Story
  6. lecture Read lecture notes: Deriving the Quadratic Formula
  7. lecture Read lecture notes: Converting to Base 2
  8. asst Do assignment: Complete Homework 0 Due: 01/25/2021
  9. text Example Code Repo

Office Hours

  1. asst Do assignment: Complete the office hours survey

Example Code

  1. lab ExampleBuildFiles/C
  2. lab ExampleBuildFiles/CPP
  3. lab SqrtExample
  4. lab TimeDilation

Summary

  1. lecture Read lecture notes: Module Summaries - Introduction
  2. lecture Read lecture notes: Module 1 Summary
  3. asst Do assignment: Review Homework 0 Solution Due: 01/27/2021
2.1 Real Numbers & Machine Numbers & Rounding 01/25/2021 - 01/31/2021

Overview

In mathematics the impact finite precision is oft dismissed as an artifact of rounding. In most–if not all cases–the solution is to use more decimal places or signifigant digits until precision errors are within an “acceptable threshold” (a nebulous phrase). In this module, we will discuss how to measure and analyze finite precision errors arise from representing a single numerical quantity.

Objectives

After this module students will be able to:

  1. Discuss the impact of finite precision on machine representation of floating-point values.
  2. Apply mathematical analysis to examine (i.e., quantify) the magnitudes that can be represented with a finite number of mantissa and exponent bits.
  3. Compare the error that arises in finite representation using the well known absolute error and relative error formulae.
  4. Explain the difference between absolute and relative error, and discuss how absolute and relative error are mathematically linked.

Relevance

When developing software of any size machine precision is a consideration. The impact may be something seemingly inconsequential (e.g., a rounding error of less than one cent). Machine precision can result in interesting impacts (e.g., a solar system simulation may compute planets spiraling into the Sun).

Activities
  1. lecture Read lecture notes: Converting to Base 2
  2. lecture Read lecture notes: TBW - Binary Arithmetic by Hand
  3. video Watch: Precision Intro Lecture
  4. lecture Read lecture notes: Review of Machine Precision
  5. lecture Read lecture notes: Finite Precision Error - Base 2
  6. lecture Read lecture notes: Finite Precision & Error... in General
  7. asst Do assignment: Complete Homework 1 Due: 02/03/2021
  8. asst Do assignment: Complete Machine Assignment 1 Due: 02/03/2021
  9. asst Do assignment: Review Homework 1 Solution Due: 02/07/2021

Example Code

  1. lab FPvsArbitraryPrecision/FP-Error-Estimate-C++
  2. lab FPvsArbitraryPrecision/FP-Error-Estimate-Python
  3. lab FPvsArbitraryPrecision/FP-Error-Estimate-Rust

Summary

  1. text Read Chapter 1 pages 1-4
  2. lecture Read lecture notes: Module Summary
2.2 Rounding & Finite Arithmetic 02/01/2021 - 02/10/2021

Overview

In this module, we will continue our discussion of finite precision and its impact on arithmetic operations. The propagation of error through arithmetic operations will be our primary focus.

Objectives

After this module students will be able to:

  1. Discuss the impact of finite precision on error propagation in arithmetic operations involving two operands.
  2. Examine the impact of finite precision on error propagation in arithmetic operations involving three or more operands and one operation (e.g., addition, subtraction, multiplication, and division).
  3. Examine the impact of finite precision on error propagation in arithmetic operations involving three or more operands and combinations of two or more operations).

Relevance

When developing software of any size rounding is a consideration. The impact may be something seemingly inconsequential (e.g., a rounding error of less than one cent). Machine precision can result in interesting impacts (e.g., a solar system simulation may compute planets spiraling into the Sun).

Activities
  1. lecture Read lecture notes: Review Finite Precision
  2. lecture Read lecture notes: Arithmetic Error
  3. lecture Read lecture notes: Cancellation Error
  4. lecture Read lecture notes: Repeated Arithmetic Operations
  5. lecture Read lecture notes: Bounding Arithmetic Error
  6. asst Do assignment: Complete Homework 2 Due: 02/10/2021
  7. asst Do assignment: Review Homework 2 Solution
  8. asst Do assignment: Complete Machine Assignment 2 Due: 02/10/2021
  9. lecture Read lecture notes: Arithmetic Error Revisited

Summary

  1. text Read Chapter 1 pages 5-9
  2. lecture Read lecture notes: Module Summary
2.3 Condition of a Problem 02/10/2021 - 02/18/2021

Overview

In this module, we will continue our discussion of finite precision and its impact on numerical computations. The propagation of error and the resulting behavior on computations will be our primary focus.

Objectives

After this module students will be able to:

  1. Discuss the sensitivity of computations to errors in input resulting from finite precision.
  2. Quantify (i.e., examine/model) through *Condition Number*s the sensitivity of computations to errors in input resulting from finite precision.
  3. Explain (through mathematical analysis) when a problem is ill-conditioned or well-conditioned.
  4. Leverage the results of Condition Number analysis to develop alternative methods of computation when a problem is ill-conditioned.

Relevance

Error propagation can result in interesting impacts (e.g., a solar system simulation may compute planets spiraling into the Sun). The ability to quantify the impact of a small change in input on a computation is foundational when building (or selecting) numerical methods (and algorithms)

Activities
  1. lecture Read lecture notes: The Condition Number
  2. lecture Read lecture notes: A Quick First Example
  3. lecture Read lecture notes: Condition Number Example - A Line
  4. lecture Read lecture notes: Condition Number Example - Square Root
  5. lecture Read lecture notes: Condition Number Example - Natural Log
  6. lecture Read lecture notes: Condition Number Example - Exponent
  7. lecture Read lecture notes: Condition Number Example - Monomial
  8. lecture Read lecture notes: Condition Number Example - Polynomial
  9. lecture Read lecture notes: Condition Number Example - (Book) Integral
  10. asst Do assignment: Complete Homework 3 Due: 02/22/2021

Summary

  1. text Read Chapter 1 Section 4 pages 10-15
  2. lecture Read lecture notes: Module Summary
2.4 Unit Summary 02/19/2021 - 02/24/2021

Overview

This module serves as a review of Chapter 1. We will revisit finite precision, arithmetic error, and conditioning of problems. This summary module also serves to force another review of requisite principles from Calculus.

Objectives

After this module students will be able to:

  1. Summarize the techniques discussed in this unit.
  2. Combine the techniques discussed in this unit to examine non-trivial problems.
  3. Utilize various principles from Calculus (e.g., differentiation, integration, limits, sequences, and series) in combination without becoming overwhelmed.
Activities
  1. lecture Read lecture notes: Chapter 1 Closing Thoughts
  2. lecture Read lecture notes: Some Trigonometry (Thinking Assignment)
  3. lecture Read lecture notes: Some Trigonometry (Thinking Assignment) - The First Few Steps
  4. asst Do assignment: Review Homework 3 Solution
  5. exam Take the exam: Exam 1: Chapter 1 02/24/2021, 1:00PM EST - 02/26/2021, 11:59PM EST

Matrices & Vector Functions

  1. lecture Read lecture notes: Condition Numbers - Matrices
  2. lecture Read lecture notes: Condition Numbers - Vector Functions

Summary

  1. lecture Read lecture notes: Module Summary

Additional Reading

  1. text Finite Precision in the Real World
3 Chapter 2 Approximation
3.1 Introduction to Approximation 02/24/2021 - 03/05/2021

Overview

Approximation has numerous applications throughout all disciplines. This can be something as simple as plotting data in Excel and computing a line of best fit or something more domain specific (e.g., transforming raw data for use in A.I. applications. In this Module we will explore Least Squares Approximation at an introductory level in preparation for a more rigorous and thorough derivation in later modules.

Objectives

After this module students will be able to:

  1. Locate and review web-based resources covering Least Squares Approximation at an introductory level.
  2. Explain how approximation methods can be employed for trend analysis for simple data (e.g., grade projection and price analysis).
  3. Manually (i.e., by hand on paper) build a solution for discrete data.

Relevance

To apply approximation methods to discrete (and later continuous) data, one must analyze (possibly) unfamiliar mathematical notation. Application of approximation methods requires a solid foundation in linear algebra.

Activities
  1. video Watch: Least Squares Introduction Lecture
  2. lecture Read lecture notes: Least Squares - A Whirlwind Introduction
  3. lecture Read lecture notes: Least Squares - A Quick First Example
  4. lecture Read lecture notes: Least Squares - Quadratic Example
  5. lecture Read lecture notes: Least Squares - More Formal Notation
  6. lecture Read lecture notes: A Tedious Problem?
  7. asst Do assignment: Machine Assignment 3: Implement a Matrix Solver Due: 03/12/2021
  8. asst Do assignment: Start working on CPU Temperature Semester Project

Summary

  1. lecture Read lecture notes: Module Summary
3.2 Approximation Proper 03/06/2021 - 03/30/2021

Overview

The previous module discussed the XTX|XTY method for computing polynomial approximation functions for discrete data. This Module will formalize the process and discuss how to generalize Least Squares approximation to both discrete and continuous data sets.

Objectives

After this module students will be able to:

  1. Explain the differences between the Riemann Integral and Riemann-Stieltjes Integral.
  2. Apply the weighted L2-Norm, properties of inner products, derivatives, and integrals to explain derivations of Least Squares Approximation.
  3. Explain the impact of basis functions.
  4. Manually (i.e., by hand on paper) build a solution for continuous functions/data.
  5. Discuss the motivation for a single Least Squares notation to reconcile the discrete and continuous cases.
  6. Explain how the [XTX|XTY] and [Ac|b] notations are related.
  7. Construct a mathematical proof to show [XTX|XTY] is equivalent to [Ac|b] for discrete data.

Relevance

There are many applications in which one must approximate a combination of discrete and continuous data sets. This includes applications in A.I (e.g., hill climbing), Medical Imaging, and Aerospace applications.

Activities

Questions & the Impact of Point Selection

  1. Discussion - Discrete Case ($X^TX|X^TY$)
  2. Discussion - Discrete Case ($A|\vec{b}$)
  3. Discussion - Discrete Case $||f(x) - \hat{\varphi}||^2$
  4. lecture Read lecture notes: Scanned Notes

Deriving the Formal Method

  1. lecture Read lecture notes: Least Squares Derivation - Overview
  2. lecture Read lecture notes: Least Squares Derivation - Foundations (Discrete vs Continuous)
  3. lecture Read lecture notes: Least Squares Derivation - Foundations (Applying Properties)
  4. lecture Read lecture notes: Least Squares Derivation - The Fun Part

Putting It Together

  1. lecture Read lecture notes: Least Squares Examples
  2. lecture Read lecture notes: Least Squares Closing Thoughts
  3. lecture Read lecture notes: Reconcile the XTX|XTY and Ab Methods
  4. asst Do assignment: Continue working on CPU Temperature Semester Project Due: 04/10/2021
  5. asst Do assignment: Complete Homework 4 Due: 03/09/2021
  6. asst Do assignment: Complete Homework 5 Due: 03/19/2021
  7. asst Do assignment: Complete Homework 6 Due: 03/25/2021
  8. exam Take the exam: Complete Exam 2 03/28/2021 - 04/01/2021

Reference Materials

  1. video Watch: Integration by Parts DI Method

Summary

  1. lecture Read lecture notes: Module Summary
  2. text Read Chapter 2

Homework Review

  1. lecture Read lecture notes: Homework 4 Review
  2. lecture Read lecture notes: Homework 5 Review
  3. lecture Read lecture notes: Homework 6 Review
4 Selected Topics
4.1 Chapter 3: Interpolation 04/02/2021 - 04/16/2021

Overview

Interpolation has applications throughout many (if not all) disciplines. This can be something as simple as resizing an image in GNU Image Manipulation Program (GIMP). Interpolation can also be viewed as a complement to approximation. Just like approximation, we need to find a function of best fit. However, interpolation has a stronger invariant: the computed interpolation function must pass through all input points.

Objectives

After this module students will be able to:

  1. Locate and review web-based resources covering interpolation at an introductory level.
  2. Explain how interpolation methods can be employed for trend analysis for simple data (e.g., price analysis and temperature analysis).
  3. Summarize the differences between approximation and interpolation.
  4. Apply Lagrange Interpolation and Newton Interpolation.
  5. Apply piecewise linear interpolation (e.g., in the Semester Project).
  6. Compute interpolation error.
  7. Compare interpolation error to approximation error.

Relevance

To apply interpolation methods to discrete data, one must analyze (possibly) unfamiliar mathematical notation. Interpolation has applications in various everyday activities (e.g., photo editing, video upscaling, and mesh generation).

Activities

Semester Project

  1. asst Do assignment: Finish working on CPU Temperature Semester Project

Interpolation Methods

  1. lecture Read lecture notes: Interpolation - The Beginning
  2. lecture Read lecture notes: Lagrange Interpolation
  3. lecture Read lecture notes: Newton’s Method
  4. asst Do assignment: Complete Homework 7 Due: 04/12/2021

Interpolation Error

  1. lecture Read lecture notes: Interpolation Error Examples
  2. lecture Read lecture notes: Interpolation Error Derivation (Guest Lecture by Christos)
  3. lecture Read lecture notes: Interpolation Error Derivation

Interpolation Taking Stock

  1. lecture Read lecture notes: Taking Stock
  2. asst Do assignment: Complete Homework 8 Due: 04/19/2021

Piecewise Interpolation

  1. lecture Read lecture notes: (Spring 2020) Piecewise Interpolation & Cubic Splines

Example Code

  1. lab DividedDifferences

Summary

  1. lecture Read lecture notes: Module Summary
4.2 Chapter 4: Non-Linear Solvers 04/17/2021 - 04/28/2021

Overview

There exist many problems that can not be solved analytically. Such problems include finding zeroes (i.e., solutions), computing derivatives, and evaluating integrals. In this module we will discuss for first problem (i.e., computing numerical solutions to equations).

Objectives

After this module students will be able to:

  1. Discuss the requirements for each of the Bisection method, False Position (Regula Falsi) method, Secant method, and Newton’s method.
  2. Manipulate provided pseudocode and refine it into a form suitable for implementation in a selected language (e.g., C, C++, Python 3, or Rust).
  3. Identify the considerations inherent in converting pseudocode (theory) into usable code (application)–with particular emphasis on the impact of finite precision.

Relevance

There exist various numerical methods for solving problems, including finding zeroes of a function, performing numerical integration, and performing numerical differentiation. The ability to select the appropriate method, interpret pseudocode, and design an implementation (using best an appropriate language) is foundational when building numerical software.

Activities
  1. video Watch: Solver Discussion
  2. lecture Read lecture notes: Non-Linear Solvers
  3. lecture Read lecture notes: Polynomial Examples
  4. lecture Read lecture notes: cos(x) Example
  5. lecture Read lecture notes: (Typed) Solver Discussion Notes
  6. asst Do assignment: Complete Homework 9 (TBD)

Example Code

  1. lab NonLinearEquationSolvers/Python
  2. lab NonLinearEquationSolvers/Rust

Summary

  1. lecture Read lecture notes: Module Summary

Final Exam

  1. exam Take the exam: Complete the Final Exam 04/28/2021, 1:00PM EDT - 05/01/2021
5 Future Work

Overview

Computational Methods and Software covers a broad range of topics. In this course, we covered a few selected topics. This section contains resources and references for topics not covered this semester.

6 Special Events and Dates
Activities
  1. Review the academic calendar.

All times in this schedule are given in Eastern Time.

Symbol Key
lecture Read:
slides Slides :
event Event or important date
text Read
lab
asst Assignment:
exam Take the
activity Do:
recitation In your recitation section:
construct Under construction: