Numerical Methods in Scientific Computing I
Math 2070 is our gateway course in computational mathematics.

Syllabus for Fall 2008

Welcome to Math 2070!

Numerical Methods in Scientific Computing

This is the first part of the sequence Math 2070, 2071 and is introduction
_to numerical methods and the fundamentals of the analysis (stability,
consistency, _convergence, efficiency and accuracy) of numerical methods.

The course covers both _theoretical and practical aspects of the methods.
Math 2070 covers _numerical analysis except numerical linear algebra and
numerical Diff Eqs and Math 2071 covers _the remainder,

Math 2070: CRN 13384 _Lecture: MW 3-4:15Ty. 704 ,

Recitation: CRN 16072 MW 4:20-5:35 in GSCC 126
Lab Instructor is Dr. Mike Sussman .
Dr. Mike Sussman .
Labs are available online.

Labs are available online. _ MATLAB will be taught in the lab and used for the labs. _

Office: Ty. 603, (412) 624-8312 _E-mail: W J L at P I T T dot E D U _http://www.math.pitt.edu/~wjl _ _

Course materials:
The main course material will be lecture notes from
the lectures in class. Attend and take good notes!
You might find some material presented simpler than in the text,
some differently than the
text, some small extensions and omissions from the text. The topics are
standard so variations will be minor rather than major. My lecture notes
have evolved through the search for the simplest mathematically rigorous
way to explain WHY and HOW and WHEN and WHEN NOT algorithms that are
important for practical computing work.

Textbooks are only useful when they give the same topics from a different
point of view than the lectures. (This will not be a course where the
teacher puts the book on the board.)

The official text is:
A Ralston and P Rabinowitz, A first course in Numerical Analysis, Dover reprint, 2001,

an excellent and inexpensive book. Many years Atkinson's book is used here,
K. Atkinson, An Introduction to Numerical Analysis, 2nd Ed,
also an excellent book but more expensive.

Course Grades: _Lab grade= 30% _Mid-Term Exam=20% _Final Exam= 20% _Homework= 30%

Misc. General Policies:
Make Up Exams will NOT be given.
HW will be collected and graded.
You are not graded on what you can prove.
in a clear and direct manner to others.
Write in complete and correctly punctuated sentences.
Avoid intoxication with the symbols of symbolic logic.
(Write for people not for automatons!)
Negative partial credit is possible for extraneous and irrelevant nonsensical information.
No quibbling over "points"; Focus on learning!

Matlab Primer
Matlab Tutorial: Postscript; HTML

For useful material related to this course go to Kendall Atkinson's class web page

Accessing MATLAB Remotely

Dr.  Sussman has graciously provided instructions for accessing Matlab remotely.
The first answer is: Yes, Matlab can be accessed remotely using your usual network access. I am not familiar with
"PittNet" -- is it a dialup connection (slow) or some
sort of fast connection? In either case, directions follow.

ASSUMING YOU ARE USING LINUX or another type of Unix:
Use ssh to get to unixs1 or unixs2, type "matlab": example
ssh youraccountname@unixs2.cis.pitt.edu
matlab
Wait a long while (1-2 minutes for a fast connection,
a looooong time for dialup) and Matlab will come up.

WARNING: if you have a (slow) dialup connection, you can
turn off the graphical stuff and run matlab in command mode:
(you get no graphics and no plotting)
ssh YoUrAcCoUnTnAmE@unixs2.cis.pitt.edu
DISPLAY=
matlab

If you have a dialup and you want plotting but no gui, do NOT
reset DISPLAY to nothing. Just do:
matlab -nosplash -nojvm

If you turn off the gui, you will need some sort of editor.
Either open another ssh session with your favorite editor
in it (pico, vi, emacs, etc.) or use the exclamation point
to start an editor from the Matlab command line (!pico ...)

ASSUMING YOU ARE USING WINDOWS
You either have an "X-server" program or you don't. I do
not know whether one is included in the programs they supply
to students.
ASSUMING YOU HAVE A DIALUP CONNECTION OR YOU DO NOT HAVE AN X-server
Use the Start->Run menu and type
telnet unixs2.cis.pitt.edu
(or unixs1 or unixs)
matlab
to get the LINE version of Matlab (no graphics, no plotting). There
is no way to do plotting without an X-server. You also will not have
the Matlab editor, but you can edit a file FROM THE MATLAB COMMAND
LINE with the command "!pico filename.m" where you don't type the
quotation marks and you put your own name where I have "filename.m".

ASSUMING YOU HAVE BOTH AN X-server AND A FAST CONNECTION
Start the X-server first. You will need to know the "display" name
colon zero).
Use the Start->Run menu and type
telnet unixs2.cis.pitt.edu
DISPLAY=192.160.1.1:0
EXPORT DISPLAY
matlab
There might be a telnet or rlogin or ssh command that comes with
the X-server. If so, the DISPLAY variable is automatically set.

REMARK:
There is a free Matlab clone called "octave" that runs on Windows
(If you run linux, install it from your CDs). See
http://octave.sourceforge.net/Octave_Windows.htm
It is not so pretty as Matlab, and it is mostly command-line
driven, but it will do plotting and most simple stuff.

Migration from FORTRAN or C to MATLAB

If you have experience programming in FORTRAN or  C, the
least painful way to migrate to MATLAB is to begin by using MATLAB as a low
level programming language and then learn the higher level features as you need them.
These notes below were prepared by Dr. M. Sussman for exactly that purpose.
For that reason, they parallel "A Synopsis of 5 statement FORTRAN".

1. INTRODUCTION:  Matlab Statements.

You can use "%" anywhere on a line to make Matlab skip
everything you write past the "%".  This is
called "commenting it out."

Ending a line without a semicolon (";") means its result will
be printed.  Ending a line with a semicolon means it will not
be printed.

Types of Variables.
All simple variables in Matlab are actually complex matrices.
Matlab tries to "treat them the way you meant them."  Variable
names are case sensitive, so iT is different from It is
different from IT is different from it.

Arithmetic Operations.
These are denoted by the usual symbols
+, -, , / and *
Exponentiation is denoted by ^

The arithmetic equals (=) has a different meaning than in arithmetic. For
example, the statement
X=X+1.0;
makes perfect sense in Matlab. It means, look at the left hand side, find
the storage location set aside to store the variable called X and find the
value of X. Add one to that value then put the result back in the X
storage location. This requires the computer to do something; hence, it is
called an executable statement. (It's described informally by: "look in
the box marked X. Get the number there; add one to it and put it back in
the X box.")

CONTROL STATEMENTS
Simply typing a variable name on a line by itself without a semicolon after
it will cause it to be printed.  A sequence of variables separated by
commas will all be printed.

Anything put in quotation marks is printed directly as written. Examples
of this are given later.

2. The Heart of the Program.

A computed basically can only do 5 things:

* Input data,
* perform arithmetic calculations,
* decisions based on logical or arithmetic tests,
* output data and
* loop (meaning perform al the above iteratively).
We've seen how to perform basic arithmetic calculations. Let's now review
some of the remainder. The real power of the computer is unleashed with
loops (doing a lot of calculations) and matrices and vectors (handling a
lot of data).

INPUT, OUTPUT and CONTROL
A single variable can be read from the keyboard using the command
r=input('What is the value?')
and the string 'What is the value?' will be printed and then the user
will type a number and the value of that number will become the value
of r.

DECISIONS.
Decisions in Matlab are performed by "IF tests." There are many
kinds. One basic kind takes the form

if (condition is true)
executable statement;
else
executable statement;
end

For example, the following adds 1 to x if x = 0 and subtracts 1 otherwise:
if x>=0
x=x+1.0;
else
x=x-1.0;
end

Equality is denoted "==" to distinguish it from assignment.

LOOPS.
These are "DO-Loops" in FORTRAN and "FOR-Loops" in C and Matlab.

for i = beginning: stepsize : end
executable statements;
end

where "beginning", "end" and "stepsize" are all integers.For example, this
computes the sum
1+1/2+1/4+1/8++ 1/(2**1000).

sum=0.0
for I=0,1000,1
end

Suppose we want to exit a loop early when a condition holds. Use
a "break" statement. For example, to exit the above when SUM>

sum=0.0
for k=0:1:1000
if sum>1.5)
break;
end
end
k
sum

Matlab's BUILT IN FUNCTIONS

Essentially all basic mathematical functitons are built-in.  For example,
exp(x) computes the exponential of x, sin(x) computes its sine, inv(M)
computes the inverse of a matrix, eig(M) computes the eigenvalues of a
matrix, etc.

Inline functions

Simple functions can be put directly in a Matlab statement:
area=inline('pi*r^2','r');
defines a function area(r) that computes the area of a circle given

Here is an example:

area=inline('pi*r^2','r');
x=2.0;
a=area(x);
disp(['radius=', num2str(x), ' area=' , num2str(a)])

note that Matlab knows the value of pi.
If a function uses more calculations or a longer formula then use
a function m-file. Function m-files are very useful!!! Here's an
example:

function a=area(x,y)
a=y*x^2;

If you put those lines in a file named "area.m" then you can call them
from a program easily.

r=2.0
a=area(r,pi)

Here is an example of a simple MATLAB program which does numerical
differentiation:

%           THIS IS IN MATLAB!!!
%
%       This program computes two finite difference approximations
%       to the derivative of a given function.

% this is Matlab syntax to define a function "in line"
% you can also make up a separate m-file
F =  inline('X*exp(X)','X');
FP = inline('X*exp(X)+exp(X)','X');

a=1.0;
h=0.5;
disp('Here we go-hold on!');
for i=0:13
h=1.0/(2.0^i);
d1hf= ( F(a+h)-F(a-h))/(2.0*h);
d2hf=(F(a+2.0*h)-F(a-2.0*h))/(4.0*h);
df=(4.0/3.0)*d1hf-(1.0/3.0)*d2hf;
e1=abs(FP(a)-df);
disp(['h=', num2str(h),' f''~',num2str(df),' error=',num2str(e1)]);
end
disp('Whew-that was fast!')

3. Arrays = Vectors and Matrices.

Vectors and matrices are the way to store and manipulate lots of
data.

For example, this computes the first 20 Fibonacci number:

fibo(1)=1.0;
fibo(2)=2.0;
for k=3:20
fibo(k)=fibo(k-1)+fibo(k-2);
end
fibo(20)

Remark: It's interesting to try running this changing "20" to, e.g., "50".

4.  Programming "Rules of Thumb".
These days one is more often modifying an existing code or linking
pieces of codes than programming a new application from scratch.
Nevertheless, the following "rules of thumb" will, if followed, save you

1. Be careful and correct: every hour you spend in checking, planning and
double checking will save four hours of debugging.

2. Use pseudo code to organize and plan your program.

3. Check it twice before running it! Then run it on several test cases
with known true-solutions.

4. Use modules. Break the program into clear chunks (one way is to use
FUNCTION m-files). Check and debug each module independently of the
whole program before testing the whole program.

5. Use the right amount of generalization!

6. Print intermediate results.

7. Include warning messages.

8. Use variable names that are easy to understand and declare all
variables.

9. Put lots of comments in!

12. Use arrays to store data.

13. Use built in functions and program libraries.

14. Put clarity (for people) above compactness or even speed of execution

FORTRAN INFORMATION

Some people might choose to use FORTRAN (or be forced to because
they deal with legacy programms). I've inclluded some information next for those lucky ones.

HOW TO RUN A FORTRAN PROGRAM

Here are some basic , basic instructions on how to
(a) write a fortran program [ie, create a file].
(b) how to run it.

(a)  To create a file you need to use an editor. In unix, the pico editor
is probably easiest because it's menu driven. Dont forget though that unix
is case sensitive: large and small letters have different meanings , so
get the capitalization exactly right!

At the prompt type:
pico progname.for

(the "progname " you may choose but be sure to make it .for at the
end.)

Next paste in or type in your program. Editing it is easy-read the
commands at
the bottom of the screen. When you are done type
'control'O

then type
'control'X

Now you've created a file progname.for.

(b) Here's how to run progname.for

At the prompt type
setup sunpro

At the next prompt type:
f77 -C progname.for

This compiles your program. Now run it by typing at the next prompt
./ a.out   (i.e., " dot slash a dot out" without any spaces!)
This tells the computer to dump your output to the screen.

Try writing a simple program to add 2 numbers and output it. If it doesnt
work, check carefully the cases [small letters or caps] and the spaces in

Another comment: If you discover something interesting, email me and I'll
send it to the whole class.

More Remarks:
These instructions are for our UNIXS system. There are a lot of free
FORTRAN compilers available [FORTRAN 77] as well as a lot of free C
you are interested in working on a PC. I suggest though that you start on
UNIXS if you've never programmed before.

Have fun!

FORTRAN Compilers-

Here are some FORTRAN resources including free and/or cheap  FORTRAN compilers for a PC.

"The Fortran Library, http://www.fortranlib.com, is a comprehensive guide
to online Fortran resources, including compiler vendors and
resellers, benchmarks, programming tools, books and articles on Fortran
and numerical methods, and commerical and public domain Fortran software.

The Fortran Market, http://www.fortran.com/fortran/ , also provides
many links to Fortran resources and is a reseller for several Fortran 90
and 95 compilers, benchmarking and test suites, and books on Fortran."

One former student installed one. Here below are his instruction on how to do this:

Subject: Fortran Compiler for PC's

It can be gotten via anonymous ftp (using Netscape, for instance) from

ftp://ftp.bu.edu/pub/mirrors/simtelnet/gnu/djgpp/v2gnu/g770523b.zip
and you also need
ftp://ftp.bu.edu/pub/mirrors/simtelnet/gnu/djgpp/unzip386.exe

There are other things like additional documentation, but this provides the
bare essentials (I realized that it needed to be cut down for people to be

Then this file (g770523b.zip) needs to be unzipped.  I recommend making the
directory C:\DJGPP by typing

mkdir \DJGPP

at an MS-DOS prompt.  Then, go to the directory where you downloaded
g770523b.zip to, and type

move g770523b.zip \djgpp
move unzip386.exe \djgpp
cd \djgpp
unzip386 g770523.zip

You'll also need to make sure you can access the compiler, so add
C:\DJGPP\BIN to your PATH statement in your CONFIG.SYS.  If you don't know
how to do this, just type the following from an MS-DOS prompt.

echo PATH=%PATH%;C:\DJGPP\BIN >> \CONFIG.SYS

After that, you'll need to write the FORTRAN programs in something like
Notepad (do not use a Word Processor for this!).  To compile your program,
named (for instance) progname.for, type the following.

g77 -o progname progname.for

and to run the new program, type

progname

A Synopsis of "5-Statement FORTRAN"

ABSTRACT
This is prepared solely for the benefit of my students. ItÕs intended to have 2 purposes.
1. To introduce math students whoÕve never programmed before but are  used to the algorithmic
thought in rigorous proofs to a language quickly and get them working, and
2. To do the same for students who have experience programming in another language  like C, for example.

1. Introduction.

FORTRAN is short for FORmula TRANslation language. It is a low
level language designed for doing scientific calculations and lots of
them. Although it is out of favor in CS departments, it is extremely handy
to know a bit of FORTRAN because:
(estimated) 95% of scientific software is in FORTRAN.
Thus, if you must modify an existing program it will likely be a FORTRAN
one.
It is also useful for us because it is simple ( = easy to learn) and its
logic parallels the logic of mathematical       formulas. Generally, for
these and other reasons, if you program in another language, it is a good
idea to know how to call a FORTRAN subroutine from that language.

2. FORTRAN Statements.

When you type a line of a FORTRAN program, different locations of
characters (i.e., which column it is types in) have different meanings.

Column 1              Column 2-5          Column 6                       Column 7-72
Comment indicator Line number  Blank or continuation Executable FORTRAN statement

Generally, try not to continue a statement from one line to another: Keep
formulas short enough that you can glance at it and see what it is doing.
Thus, try to leave column 6 blank. A character in column 1 tells the
compiler to ignore the line so you use these lines to write notes to
yourself. You can also use "C " in column 1 to de-activate a line. This is
called "commenting it out."

Types of Variables.

Variable names can have no more than 7 characctters.
You should declare variables but in FORTRAN you don't need to
because there are defaults.
INTEGERS:
Integers are stored as such and added exactly. Any variable
beginning with
I, J, K, L, M, N
is by default an integer. Thus, old-style FORTRAN would use the
(undeclared) variable ICOUNT.  Variables can also be declared as integers
initially, meaning before the first executable statement. For example,

_ _ _ _ _ _ _ _ _ INTEGER COUNT

will make COUNT an integer even though the undeclared default of a
variable COUNT is real. In integer arithmetic
I  =1/4
Returns I = 0 while in real arithmetic
X = 1.0/4.0
Returns X = 0.25. Don't use mixed mode arithmetic!

REAL NUMBERS:
These can be declared as real before the first executable
statement by, e.g.,

_ _ _ _ _ _ _ REAL X

By default any variable beginning with
A-H or O-Z
is also real. Reals are numbers with decimal digits such as
PI = 3.141592736
XINDEX = 1.0
They are stored in scientific notation and carry a certain number of
significant digits. Carrying 7 digits base 10 is typical although not
universal.

DOUBLE PRECISION
variables must always be declared n before the first
executable statement, e.g.,

_ _ _ _ _ _ _ DOUBLE PRECISION X

They work like reals but carry more than 2x the number of significant
figures as reals (also called single precision). Double precision is the
normal mode for many scientific calculations. If you add a real to a
double precision variable, you immediately lose al the extra accuracy of
the double precision variable so:
Avoid mixed mode arithmetic.

Arithmetic Operations.
These are denoted by the usual symbols
+, -, , / and *
Exponentiation is denoted by ** with differences between real and integer
exponents:
(Integer) X**2 means X * X
(Real) X**2.0 means EXP(2.0 LOG( X ) ).
The arithmetic equals (=) has a different meaning than in arithmetic. For
example, the statement
X=X+1.0
makes perfect sense in FORTRAN. It means, look at the left hand side, find
the storage location set aside to store the variable called X and find the
value of X. Add one to that value then put the result back in the X
storage location. This requires the computer to do something; hence, it is
called an executable statement. (It's described informally by: "look in
the box marked X. Get the number there; add one to it and put it back in
the X box.")

CONTROL STATEMENTS
Programs always end with two statements that terminate the
program:
STOP
END
Another useful statement is the unformatted print statement which
just dumps the output to your computer screen. This is fine for moderate
amounts of output. (For lots of it, read about formatted print statements
in a FORTRAN book.) An unformatted print takes the form

PRINT *, variables to be printed separated by commas.

Anything put in quotation marks is printed directly as written. Examples
of this are given later. Generally, for moderate amounts of data, an
unformatted print is fine. On your terminal screen, highlight the output.
Next copy and paste it into your favorite text editor and format it and
then print it.

3. The Heart of the Program.

A computed basically can only do 5 things:

¥ Input data,
¥ perform arithmetic calculations,
¥ decisions based on logical or arithmetic tests,
¥ output data and
¥ loop (meaning perform al the above iteratively).

We've seen how to perform basic arithmetic calculations. Let's now review
some of the remainder. The real power of the computer is unleashed with
loops (doing a lot of calculations) and matrices and vectors (handling a
lot of data).

INPUT, OUTPUT and CONTROL
Unformatted input is (usually from your terminal screen)
While unformatted output is
PRINT*, variables.
You can label the variables by putting notes to be printed in quotes as in:

PRINT*, ÒNext we print the value of XÓ, X

This results in a printted line like:

Next we print the value of X      .34567E2

when X has the value 34.567.

DECISIONS.
Decisions in FORTRAN are performed by "IF tests." There are many
kinds. One basic kind takes the form

IF (condition is true) THEN executable statement
ELSE executable statement
ENDIF

For example, the following adds 1 to x if x = 0 and subtracts 1 otherwise:
IF(X.GE.0) THEN X=X+1.0
ELSE X=X-1.0
ENDIF

The conditions are arithmetic inequalities using the following.

Math            FORTRAN                     Example
<                    .LT.                          X<Y   is written as: (X.LT.Y)
>                     .GT.                        X>Y   is written as: (X.GT.Y)
=                     .EQ.                         I=J    is written as:  (I.EQ.J)
> or =              .GE.                            is written as:  (X.GE.Y)
< or =               .LE.                           is written as:    (X.LE.Y)
not equal           .NE.                           is written as:  (I.NE.J)

LOOPS.
These are "DO-Loops" in FORTRAN and "FOR-Loops" in C. If r is a
statement number, they take the form

DO r I = beginning, end, stepsize
executable statements
r       CONTINUE

where "beginning", "end" and "stepsize" are all integers.For example, this
computes the sum
1+1/2+1/4+1/8++ 1/(2**1000).

PROGRAM SUM.for
SUM=0.0
DO 10 I=0,1000,1
10    CONTINUE
STOP
END

Suppose we want to exit a loop early when a condition holds. In
this case, in FORTRAN the only way is to use a "GO TO" statement. (This is
the only valid use of GO TO's.) For example, to exit the above when SUM>
1.5 we add an if-test with a GO TO:

PROGRAM SUM.for
SUM=0.0
DO 10 I=0,1000,1
IF(SUM.GT.1.5) THEN GO TO 20
ENDIF
10    CONTINUE
20    PRINT*, "I=",I, "and sum=", SUM
STOP
END

FORTRAN's BUILT IN FUNCTIONS
There are many built in functions in FORTRAN. If you put a "D" in
front the function returns a double precision answer. Some common ones
include:
ABS(X)=|x| ,
TAN(X) = tangent(x),
ATAN(X)= arctangent(x),
COS(X), SIN(X),TAN(X),... ,
EXP(X) = exponential function of x= e**x ,
LOG(X)= natural logarithm of x,
LOG10(X)= logarithm base 10 of x,
SQRT(X)=square root of x,
FLOAT(I) and REAL(I) convert integer I into a real to avoid mixed mode arithmetic,
DFLOAT(I) makes I a double precision real,
and so on!

FUNCTION STATEMENTS
Function statements are very handy to code a simple formula.
IMPORTANT: MUST BE BEFORE THE FIRST EXECUTABLE STATEMENT!

Here is an example:

PROGRAM AREA.for
AREA( R)=4.0*ATAN(1.0)*R**2
X=2.0
A=AREA(X)
PRINT*, "Radius=", X, "AREA=" , A
STOP
END

Note that 4.0*ATAN(1.0) is a standard way to code PI without typing in all
its digits!
If a function uses more calculations or a longer formula then use
a function subroutine. Function subroutines are very useful!!! Here's an
example:

PROGRAM AREA.for
PI=4.0*ATAN(1.0)
R=2.0
A=AREA(R,PI)
STOP
END
FUNCTION AREA(X,Y)
AREA=Y*X**2
RETURN
END

Notes:
You must have "AREA = " somewhere in function subroutine AREA
And you must always have "RETURN, END" at the end.

4. Arrays = Vectors and Matrices.
Vectors and matrices are the way to store and manipulate lots of
data. They must be set up with various dimensioning statements
BEFORE THE FIRST EXECUTABLE STATEMENT!!!
Examples of these dimensioning statements are the following:

REAL X(50), A(50, 50)
INTEGER INDEX(50)

which means the same as:

REAL X, A
INTEGER INDEX
DIMENSION X(50), A(50, 50), INDEX(50)

For example, this computes the first 20 Fibonacci number:

PROGRAM Fibonacci.for
REAL FIBO(20)
FIBO(1)=1.0
FIBO(2)=2.0
DO 10 I=3,20,1
FIBO(I)=FIBO(I-1)+FIBO(I-2)
10   CONTINUE
PRINT*,FIBO(20)
STOP
END

Remark: It's interesting to try running this changing "20" to, e.g., "50".
You will soon see why FIBO(.) is stored as a real number rather than (at
first and second consideration) the more natural integer.

5. Programming "Rules of Thumb".
These days one is more often modifying an existing code or linking
pieces of codes than programming a new application from scratch.
Nevertheless, the following "rules of thumb" will, if followed, save you

1. Be careful and correct: every hour you spend in checking, planning and
double checking will save four hours of debugging.

2. Use pseudo code to organize and plan your program.

3. Check it twice before running it! Then run it on several test cases
with known true-solutions.

4. Use modules. Break the program into clear chunks (one way is to use
FUNCTION subroutines). Check and debug each module independently of the
whole program before testing the whole program.

5. Use the right amount of generalization!

6. Print intermediate results.

7. Include warning messages.

8. Use variable names that are easy to understand and declare all
variables.

9. Put lots of comments in!