Syllabi Header

Math 691: Stochastic Processes with Applications
Fall 2017 Graduate Course Syllabus

NJIT Academic Integrity Code: All Students should be aware that the Department of Mathematical Sciences takes the University Code on Academic Integrity at NJIT very seriously and enforces it strictly. This means that there must not be any forms of plagiarism, i.e., copying of homework, class projects, or lab assignments, or any form of cheating in quizzes and exams. Under the University Code on Academic Integrity, students are obligated to report any such activities to the Instructor.

Course Information

Course Description: Renewal theory, renewal reward processes and applications. Homogeneous, non-homogeneous, and compound Poisson processes with illustrative applications. Introduction to Markov chains in discrete and continuous time with selected applications.

Number of Credits: 3

Prerequisites: MATH 662.

Course-Section and Instructors

Course-Section Instructor
Math 691-101 Professor S. Subramanian

Office Hours for All Math Instructors: Fall 2017 Office Hours and Emails

Required Textbooks:

Title Introduction to Probability Models
Author Sheldon M. Ross
Edition 11th
Publisher Academic Press
ISBN # 978-0124079489

University-wide Withdrawal Date:The last day to withdraw with a w is Monday, November 6, 2017. It will be strictly enforced.

Course Goals

Course Objectives: This course will teach students concepts of stochastic processes such as Markov chains, Poisson processes, and renewal processes.

Course Outcomes: On successful completion student will be able to demonstrate problem solving skills involving stochastic calculations and show understanding and knowledge of:

Course Assessment: Will be based on regular homework, one midterm exam and one final exam.


DMS Course Policies: All DMS students must familiarize themselves with, and adhere to, the Department of Mathematical Sciences Course Policies, in addition to official university-wide policies. DMS takes these policies very seriously and enforces them strictly.

Grading Policy: The final grade in this course will be determined as follows:

Homework and Quizzes 25%
Midterm Exam 40%
Final Exam 35%

Your final letter grade will be based on the following tentative curve.

A 90 - 100 C 68 - 74
B+ 85 - 89 D 50 - 67
B 80 - 84 F 0 - 49
C+ 75 - 79

Attendance Policy: Attendance at all classes will be recorded and is mandatory. Please make sure you read and fully understand the Math Department’s Attendance Policy. This policy will be strictly enforced.

Homework and Quiz Policy: Homework assignments are due within a week unless announced otherwise by instructor. Late homework will not be accepted.

Exams: One in-class midterm examination and one final examination will be given as shown below. The midterm exam date is tentative and may be subject to change.

Midterm Exam October 19, 2017
Final Exam Period December 15 - 21, 2017

The final exam will test your knowledge of all the course material taught in the entire course. Make sure you read and fully understand the Math Department's Examination Policy. This policy will be strictly enforced.

Makeup Exam Policy: To properly report your absence from a midterm or final exam, please review and follow the required steps under the DMS Examination Policy found here:

Cellular Phones: All cellular phones and other electronic devices must be switched off during all class times.

Additional Resources

Accommodation of Disabilities: Disability Support Services (DSS) offers long term and temporary accommodations for undergraduate, graduate and visiting students at NJIT.

If you are in need of accommodations due to a disability please contact Chantonette Lyles, Associate Director of Disability Support Services at 973-596-5417 or via email at The office is located in Fenster Hall, Room 260.  A Letter of Accommodation Eligibility from the Disability Support Services office authorizing your accommodations will be required.

For further information regarding self identification, the submission of medical documentation and additional support services provided please visit the Disability Support Services (DSS) website at:

Important Dates (See: Fall 2017 Academic Calendar, Registrar)

Date Day Event
September 5, 2017 T First Day of Classes
September 11, 2017 M Last Day to Add/Drop Classes
November 6, 2017 M Last Day to Withdraw
November 21, 2017 T Thursday Classes Meet
November 22, 2017 W Friday Classes Meet
November 23 - 26, 2017 R - Su Thanksgiving Break - University Closed
December 13, 2017 W Last Day of Classes
December 14, 2017 R Reading Day
December 15 - 21, 2017 F - R Final Exam Period

Course Outline

Week Lecture Sections Topic
1 9/7 (Thurs) 1.1-1.3 Discrete Markov Chains - I 
Examples, definitions, Chapman-Kolmogorov, classification of states
2 9/14 (Thurs) 1.4-1.6 Discrete Markov Chains - II 
Stationary distributions, limit behavior, special examples 
3 9/21 (Thurs) 1.8-1.9 Discrete Markov Chains - III 
Exit distributions, exit times
4 9/28 (Thurs) 2.1 Poisson Processes - I 
Exponential distribution, definition of the Poisson process
5 10/5 (Thurs) 2.2-2.3 Poisson Processes - II 
Compound Poisson Processes
6 10/12 (Thurs) 2.4 Poisson Processes - III 
7 10/19 (Thurs) MIDTERM EXAM
8 10/26 (Thurs) 3.1 Renewal Processes – I
Laws of large numbers
9 11/2 (Thurs) 3.2 Renewal Processes - II 
Applications to Queueing Theory
10 11/9 (Thurs) 3.3 Renewal Processes - III  
Age and Residual Life
11 11/16 (Thurs) 4.1-4.2 Continuous time Markov Chains - I 
Definitions and Examples, computing the transition probability
12 11/21 (Tues) 4.3-4.4 Continuous time Markov Chains - II 
Limiting behavior, exit distributions and hitting times
13 11/30 (Thurs) 4.5-4.6 Continuous time Markov Chains – III
Markovian queues, Queuing networks
14 12/7 (Thurs) 5.1-5.2 Review for final exam 
(if afore-specified topics are completed on time)

Updated by Professor S. Subramanian - 9/6/2017
Department of Mathematical Sciences Course Syllabus, Fall 2017