Markov Processes for Stochastic Modeling, 2nd Edition

  • Published By:
  • ISBN-10: 0124078397
  • ISBN-13: 9780124078390
  • DDC: 519.233
  • Grade Level Range: College Freshman - College Senior
  • 514 Pages | eBook
  • Original Copyright 2013 | Published/Released June 2014
  • This publication's content originally published in print form: 2013

  • Price:  Sign in for price



Markov processes are processes that have limited memory. In particular, their dependence on the past is only through the previous state. They are used to model the behavior of many systems including communications systems, transportation networks, image segmentation and analysis, biological systems and DNA sequence analysis, random atomic motion and diffusion in physics, social mobility, population studies, epidemiology, animal and insect migration, queueing systems, resource management, dams, financial engineering, actuarial science, and decision systems. Covering a wide range of  areas of application of Markov processes, this second edition is revised to highlight the most important aspects as well as the most recent trends and applications of Markov processes. The author spent over 16 years in the industry before returning to academia, and he has applied many of the principles covered in this book in multiple research projects. Therefore, this is an applications-oriented book that also includes enough theory to provide a solid ground in the subject for the reader.

Table of Contents

Front Cover.
Half Title Page.
Title Page.
Copyright Page.
Preface to the Second Edition.
Preface to the First Edition.
1: Basic Concepts in Probability.
2: Basic Concepts in Stochastic Processes.
3: Introduction to Markov Processes.
4: Discrete-Time Markov Chains.
5: Continuous-Time Markov Chains.
6: Markov Renewal Processes.
7: Markovian Queueing Systems.
8: Random Walk.
9: Brownian Motion.
10: Diffusion Processes.
11: Levy Processes.
12: Markovian Arrival Processes.
13: Controlled Markov Processes.
14: Hidden Markov Models.
15: Markov Point Processes.