Stability of Markovian Processes Iiifosterlyapunov Criteria for Continuoustime Processes

Ergodicity of Markov Processes via Nonstandard Analysis

About this Title

Haosui Duanmu, Jeffrey S. Rosenthal and William Weiss

Publication: Memoirs of the American Mathematical Society
Publication Year: 2021; Volume 273, Number 1342
ISBNs: 978-1-4704-5002-1 (print); 978-1-4704-6813-2 (online)
DOI: https://doi.org/10.1090/memo/1342
Published electronically: November 4, 2021
Keywords: Nonstandard analysis, nonstandard measure theory, discrete-time Markov processes, continuous-time Markov processes

View full volume as PDF

Table of Contents

Chapters

• 1. Introduction
• 2. Markov Processes and the Main Result
• 3. Preliminaries: Nonstandard Analysis
• 4. Internal Probability Theory
• 5. Measurability of Standard Part Map
• 6. Hyperfinite Representation of a Probability Space
• 7. General Hyperfinite Markov Processes
• 8. Hyperfinite Representation for Discrete-time Markov Processes
• 9. Hyperfinite Representation for Continuous-time Markov Processes
• 10. Markov Chain Ergodic Theorem
• 11. The Feller Condition
• 12. Push-down Results
• 13. Merging of Markov Processes
• 14. Miscellaneous Remarks

Abstract

The Markov chain ergodic theorem is well-understood if either the time-line or the state space is discrete. However, there does not exist a very clear result for general state space continuous-time Markov processes. Using methods from mathematical logic and nonstandard analysis, we introduce a class of hyperfinite Markov processes-namely, general Markov processes which behave like finite state space discrete-time Markov processes. We show that, under moderate conditions, the transition probability of hyperfinite Markov processes align with the transition probability of standard Markov processes. The Markov chain ergodic theorem for hyperfinite Markov processes will then imply the Markov chain ergodic theorem for general state space continuous-time Markov processes.

• Robert M. Anderson, A nonstandard representation for Brownian motion and Itô integration, Bull. Amer. Math. Soc. 82 (1976), no. 1, 99–101. MR 405581, DOI 10.1090/S0002-9904-1976-13976-6
• Robert M. Anderson, Star-finite representations of measure spaces, Trans. Amer. Math. Soc. 271 (1982), no. 2, 667–687. MR 654856, DOI 10.1090/S0002-9947-1982-0654856-1
• Leif O. Arkeryd, Nigel J. Cutland, and C. Ward Henson (eds.), Nonstandard analysis, NATO Advanced Science Institutes Series C: Mathematical and Physical Sciences, vol. 493, Kluwer Academic Publishers Group, Dordrecht, 1997. Theory and applications. MR 1603227, DOI 10.1007/978-94-011-5544-1
• Josef Berger, Horst Osswald, Yeneng Sun, and Jiang-Lun Wu, On nonstandard product measure spaces, Illinois J. Math. 46 (2002), no. 1, 319–330. MR 1936091
• Allen R. Bernstein and Frank Wattenberg, Nonstandard measure theory, Applications of Model Theory to Algebra, Analysis, and Probability (Internat. Sympos., Pasadena, Calif., 1967) Holt, Rinehart and Winston, New York, 1969, pp. 171–185. MR 0247018
• Patrick Billingsley, Probability and measure, 3rd ed., Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons, Inc., New York, 1995. A Wiley-Interscience Publication. MR 1324786
• Nigel J. Cutland, Vítor Neves, Franco Oliveira, and José Sousa-Pinto (eds.), Developments in nonstandard mathematics, Pitman Research Notes in Mathematics Series, vol. 336, Longman, Harlow, 1995. Papers from the International Colloquium (CIMNS94) held in memory of Abraham Robinson at the University of Aveiro, Aveiro, July 18–22, 1994. MR 1394201
• Anthony D'Aristotile, Persi Diaconis, and David Freedman, On merging of probabilities, Sankhyā Ser. A 50 (1988), no. 3, 363–380. MR 1065549
• Roy O. Davies, Measures not approximable or not specifiable by means of balls, Mathematika 18 (1971), 157–160. MR 310162, DOI 10.1112/S0025579300005386
• H. Duanmu and Roy Daniel, Nonstandard complete class theorems, 2016. In preparation.
• Geoffrey R. Grimmett and David R. Stirzaker, Probability and random processes, 3rd ed., Oxford University Press, New York, 2001. MR 2059709
• C. Ward Henson, On the nonstandard representation of measures, Trans. Amer. Math. Soc. 172 (1972), 437–446. MR 315082, DOI 10.1090/S0002-9947-1972-0315082-2
• C. Ward Henson, Analytic sets, Baire sets and the standard part map, Canadian J. Math. 31 (1979), no. 3, 663–672. MR 536371, DOI 10.4153/CJM-1979-066-0
• Olav Kallenberg, Foundations of modern probability, 2nd ed., Probability and its Applications (New York), Springer-Verlag, New York, 2002. MR 1876169, DOI 10.1007/978-1-4757-4015-8
• H. Jerome Keisler, An infinitesimal approach to stochastic analysis, Mem. Amer. Math. Soc. 48 (1984), no. 297, x+184. MR 732752, DOI 10.1090/memo/0297
• H. Jerome Keisler and Yeneng Sun, A metric on probabilities, and products of Loeb spaces, J. London Math. Soc. (2) 69 (2004), no. 1, 258–272. MR 2025340, DOI 10.1112/S0024610703004794
• D. Landers and L. Rogge, Universal Loeb-measurability of sets and of the standard part map with applications, Trans. Amer. Math. Soc. 304 (1987), no. 1, 229–243. MR 906814, DOI 10.1090/S0002-9947-1987-0906814-1
• Peter A. Loeb, A nonstandard representation of Borel measures and $\sigma$-finite measures, Victoria Symposium on Nonstandard Analysis (Univ. Victoria, Victoria, B.C., 1972) Springer, Berlin, 1974, pp. 144–152. Lecture Notes in Math., Vol. 369. MR 0476992
• Peter A. Loeb, Conversion from nonstandard to standard measure spaces and applications in probability theory, Trans. Amer. Math. Soc. 211 (1975), 113–122. MR 390154, DOI 10.1090/S0002-9947-1975-0390154-8
• Peter A. Loeb and Manfred P. H. Wolff (eds.), Nonstandard analysis for the working mathematician, 2nd ed., Springer, Dordrecht, 2015. MR 3381849, DOI 10.1007/978-94-017-7327-0
• W. A. J. Luxemburg, A general theory of monads, Applications of Model Theory to Algebra, Analysis, and Probability (Inte rnat. Sympos., Pasadena, Calif., 1967) Holt, Rinehart and Winston, New York, 1969, pp. 18–86. MR 0244931
• Neal Madras and Deniz Sezer, Quantitative bounds for Markov chain convergence: Wasserstein and total variation distances, Bernoulli 16 (2010), no. 3, 882–908. MR 2730652, DOI 10.3150/09-BEJ238
• Sean Meyn and Richard L. Tweedie, Markov chains and stochastic stability, 2nd ed., Cambridge University Press, Cambridge, 2009. With a prologue by Peter W. Glynn. MR 2509253, DOI 10.1017/CBO9780511626630
• Sean P. Meyn and R. L. Tweedie, Stability of Markovian processes. II. Continuous-time processes and sampled chains, Adv. in Appl. Probab. 25 (1993), no. 3, 487–517. MR 1234294, DOI 10.2307/1427521
• Sean P. Meyn and R. L. Tweedie, Stability of Markovian processes. III. Foster-Lyapunov criteria for continuous-time processes, Adv. in Appl. Probab. 25 (1993), no. 3, 518–548. MR 1234295, DOI 10.2307/1427522
• Ilya Molchanov, Theory of random sets, Probability and its Applications (New York), Springer-Verlag London, Ltd., London, 2005. MR 2132405
• D. Preiss and J. Tišer, Measures in Banach spaces are determined by their values on balls, Mathematika 38 (1991), no. 2, 391–397 (1992). MR 1147839, DOI 10.1112/S0025579300006744
• Gareth O. Roberts and Jeffrey S. Rosenthal, General state space Markov chains and MCMC algorithms, Probab. Surv. 1 (2004), 20–71. MR 2095565, DOI 10.1214/154957804100000024
• Gareth O. Roberts and Jeffrey S. Rosenthal, Harris recurrence of Metropolis-within-Gibbs and trans-dimensional Markov chains, Ann. Appl. Probab. 16 (2006), no. 4, 2123–2139. MR 2288716, DOI 10.1214/105051606000000510
• Abraham Robinson, Non-standard analysis, North-Holland Publishing Co., Amsterdam, 1966. MR 0205854
• Joseph P. Romano and Andrew F. Siegel, Counterexamples in probability and statistics, The Wadsworth & Brooks/Cole Statistics/Probability Series, Wadsworth & Brooks/Cole Advanced Books & Software, Monterey, CA, 1986. MR 831223
• Jeffrey S. Rosenthal, A first look at rigorous probability theory, 2nd ed., World Scientific Publishing Co. Pte. Ltd., Hackensack, NJ, 2006. MR 2279622, DOI 10.1142/6300
• Laurent Saloff-Coste and Jessica Zúñiga, Merging and stability for time inhomogeneous finite Markov chains, Surveys in stochastic processes, EMS Ser. Congr. Rep., Eur. Math. Soc., Zürich, 2011, pp. 127–151. MR 2883857, DOI 10.4171/072-1/7
• Łukasz Stettner, Remarks on ergodic conditions for Markov processes on Polish spaces, Bull. Polish Acad. Sci. Math. 42 (1994), no. 2, 103–114. MR 1810695
• Yeneng Sun, A theory of hyperfinite processes: the complete removal of individual uncertainty via exact LLN, J. Math. Econom. 29 (1998), no. 4, 419–503. MR 1627287, DOI 10.1016/S0304-4068(97)00036-0

• Robert M. Anderson, A nonstandard representation for Brownian motion and Itô integration, Bull. Amer. Math. Soc. 82 (1976), no. 1, 99–101. MR 0405581, DOI 10.1090/S0002-9904-1976-13976-6
• Robert M. Anderson, Star-finite representations of measure spaces, Trans. Amer. Math. Soc. 271 (1982), no. 2, 667–687. MR 654856, DOI 10.2307/1998904
• Leif O. Arkeryd, Nigel J. Cutland, and C. Ward Henson (eds.), Nonstandard analysis, NATO Advanced Science Institutes Series C: Mathematical and Physical Sciences, vol. 493, Kluwer Academic Publishers Group, Dordrecht, 1997. Theory and applications. MR 1603227, DOI 10.1007/978-94-011-5544-1
• Josef Berger, Horst Osswald, Yeneng Sun, and Jiang-Lun Wu, On nonstandard product measure spaces, Illinois J. Math. 46 (2002), no. 1, 319–330. MR 1936091
• Allen R. Bernstein and Frank Wattenberg, Nonstandard measure theory, Applications of Model Theory to Algebra, Analysis, and Probability (Internat. Sympos., Pasadena, Calif., 1967) Holt, Rinehart and Winston, New York, 1969, pp. 171–185. MR 0247018
• Patrick Billingsley, Probability and measure, 3rd ed., Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons, Inc., New York, 1995. A Wiley-Interscience Publication. MR 1324786
• Nigel J. Cutland, Vítor Neves, Franco Oliveira, and José Sousa-Pinto (eds.), Developments in nonstandard mathematics, Pitman Research Notes in Mathematics Series, vol. 336, Longman, Harlow, 1995. Papers from the International Colloquium (CIMNS94) held in memory of Abraham Robinson at the University of Aveiro, Aveiro, July 18–22, 1994. MR 1394201
• Anthony D'Aristotile, Persi Diaconis, and David Freedman, On merging of probabilities, Sankhyā Ser. A 50 (1988), no. 3, 363–380. MR 1065549
• Roy O. Davies, Measures not approximable or not specifiable by means of balls, Mathematika 18 (1971), 157–160. MR 0310162, DOI 10.1112/S0025579300005386
• H. Duanmu and Roy Daniel, Nonstandard complete class theorems, 2016. In preparation.
• Geoffrey R. Grimmett and David R. Stirzaker, Probability and random processes, 3rd ed., Oxford University Press, New York, 2001. MR 2059709
• C. Ward Henson, On the nonstandard representation of measures, Trans. Amer. Math. Soc. 172 (1972), 437–446. MR 0315082, DOI 10.2307/1996361
• C. Ward Henson, Analytic sets, Baire sets and the standard part map, Canad. J. Math. 31 (1979), no. 3, 663–672. MR 536371, DOI 10.4153/CJM-1979-066-0
• Olav Kallenberg, Foundations of modern probability, 2nd ed., Probability and its Applications (New York), Springer-Verlag, New York, 2002. MR 1876169, DOI 10.1007/978-1-4757-4015-8
• H. Jerome Keisler, An infinitesimal approach to stochastic analysis, Mem. Amer. Math. Soc. 48 (1984), no. 297, x+184. MR 732752, DOI 10.1090/memo/0297
• H. Jerome Keisler and Yeneng Sun, A metric on probabilities, and products of Loeb spaces, J. London Math. Soc. (2) 69 (2004), no. 1, 258–272. MR 2025340, DOI 10.1112/S0024610703004794
• D. Landers and L. Rogge, Universal Loeb-measurability of sets and of the standard part map with applications, Trans. Amer. Math. Soc. 304 (1987), no. 1, 229–243. MR 906814
• Peter A. Loeb, A nonstandard representation of Borel measures and $\sigma$-finite measures, Victoria Symposium on Nonstandard Analysis (Univ. Victoria, Victoria, B.C., 1972) Springer, Berlin, 1974, pp. 144–152. Lecture Notes in Math., Vol. 369. MR 0476992
• Peter A. Loeb, Conversion from nonstandard to standard measure spaces and applications in probability theory, Trans. Amer. Math. Soc. 211 (1975), 113–122. MR 0390154, DOI 10.2307/1997222
• Peter A. Loeb and Manfred P. H. Wolff (eds.), Nonstandard analysis for the working mathematician, 2nd ed., Springer, Dordrecht, 2015. MR 3381849, DOI 10.1007/978-94-017-7327-0
• W. A. J. Luxemburg, A general theory of monads, Applications of Model Theory to Algebra, Analysis, and Probability (Inte rnat. Sympos., Pasadena, Calif., 1967) Holt, Rinehart and Winston, New York, 1969, pp. 18–86. MR 0244931
• Neal Madras and Deniz Sezer, Quantitative bounds for Markov chain convergence: Wasserstein and total variation distances, Bernoulli 16 (2010), no. 3, 882–908. MR 2730652
• Sean Meyn and Richard L. Tweedie, Markov chains and stochastic stability, 2nd ed., Cambridge University Press, Cambridge, 2009. With a prologue by Peter W. Glynn. MR 2509253, DOI 10.1017/CBO9780511626630
• Sean P. Meyn and R. L. Tweedie, Stability of Markovian processes. II. Continuous-time processes and sampled chains, Adv. in Appl. Probab. 25 (1993), no. 3, 487–517. MR 1234294, DOI 10.2307/1427521
• Sean P. Meyn and R. L. Tweedie, Stability of Markovian processes. III. Foster-Lyapunov criteria for continuous-time processes, Adv. in Appl. Probab. 25 (1993), no. 3, 518–548. MR 1234295, DOI 10.2307/1427522
• Ilya Molchanov, Theory of random sets, Probability and its Applications (New York), Springer-Verlag London, Ltd., London, 2005. MR 2132405
• D. Preiss and J. Tišer, Measures in Banach spaces are determined by their values on balls, Mathematika 38 (1991), no. 2, 391–397 (1992). MR 1147839, DOI 10.1112/S0025579300006744
• Gareth O. Roberts and Jeffrey S. Rosenthal, General state space Markov chains and MCMC algorithms, Probab. Surv. 1 (2004), 20–71. MR 2095565, DOI 10.1214/154957804100000024
• Gareth O. Roberts and Jeffrey S. Rosenthal, Harris recurrence of Metropolis-within-Gibbs and trans-dimensional Markov chains, Ann. Appl. Probab. 16 (2006), no. 4, 2123–2139. MR 2288716, DOI 10.1214/105051606000000510
• Abraham Robinson, Non-standard analysis, North-Holland Publishing Co., Amsterdam, 1966. MR 0205854
• Joseph P. Romano and Andrew F. Siegel, Counterexamples in probability and statistics, The Wadsworth & Brooks/Cole Statistics/Probability Series, Wadsworth & Brooks/Cole Advanced Books & Software, Monterey, CA, 1986. MR 831223
• Jeffrey S. Rosenthal, A first look at rigorous probability theory, 2nd ed., World Scientific Publishing Co. Pte. Ltd., Hackensack, NJ, 2006. MR 2279622, DOI 10.1142/6300
• Laurent Saloff-Coste and Jessica Zúñiga, Merging and stability for time inhomogeneous finite Markov chains, Surveys in stochastic processes, EMS Ser. Congr. Rep., Eur. Math. Soc., Zürich, 2011, pp. 127–151. MR 2883857, DOI 10.4171/072-1/7
• Łukasz Stettner, Remarks on ergodic conditions for Markov processes on Polish spaces, Bull. Polish Acad. Sci. Math. 42 (1994), no. 2, 103–114. MR 1810695
• Yeneng Sun, A theory of hyperfinite processes: the complete removal of individual uncertainty via exact LLN, J. Math. Econom. 29 (1998), no. 4, 419–503. MR 1627287, DOI 10.1016/S0304-4068(97)00036-0

lascellesodearme.blogspot.com

Source: https://www.ams.org/books/memo/1342/

Share this post