Discrete stochastic processes are essentially probabilistic systems that evolve in time via random changes occurring at discrete fixed or random intervals. This course aims to help students acquire both the mathematical principles and the intuition necessary to create, analyze, and understand insightful models for a broad range of these processes. The range of areas for which discrete stochastic-process models are useful is constantly expanding, and includes many applications in engineering, physics, biology, operations research and finance.

Sequential decision-making via dynamic programming. Unified approach to optimal control of stochastic dynamic systems and Markovian decision problems. Applications in linear-quadratic control, inventory control, and resource allocation models. Optimal decision making under perfect and imperfect state information. Certainty equivalent and open loop-feedback control, and self-tuning controllers. Infinite horizon problems, successive approximation, and policy iteration. Discounted problems, stochastic shortest path problems, and average cost problems. Optimal stopping, scheduling, and control of queues. Approximations and neurodynamic programming.

This is a course on the fundamentals of probability geared towards first- or second-year graduate students who are interested in a rigorous development of the subject. The course covers most of the topics in 6.431 (sample space, random variables, expectations, transforms, Bernoulli and Poisson processes, finite Markov chains, limit theorems) but at a faster pace and in more depth. There are also a number of additional topics, such as language, terminology, and key results from measure theory; interchange of limits and expectations; multivariate Gaussian distributions; deeper understanding of conditional distributions and expectations.

Linear Algebra is both rich in theory and full of interesting applications; in this course the student will try to balance both. This course includes a review of topics learned in Linear Algebra I. Upon successful completion of this course, the student will be able to: Solve systems of linear equations; Define the abstract notions of vector space and inner product space; State examples of vector spaces; Diagonalize a matrix; Formulate what a system of linear equations is in terms of matrices; Give an example of a space that has the Archimedian property; Use the Euclidean algorithm to find the greatest common divisor; Understand polar form and geometric interpretation of the complex numbers; Explain what the fundamental theorem of algebra states; Determine when two matrices are row equivalent; State the Fredholm alternative; Identify matrices that are in row reduced echelon form; Find a LU factorization for a given matrix; Find a PLU factorization for a given matrix; Find a QR factorization for a given matrix; Use the simplex algorithm; Compute eigenvalues and eigenvectors; State Shur's Theorem; Define normal matrices; Explain the composition and the inversion of permutations; Define and compute the determinant; Explain when eigenvalues exist for a given operator; Normal form of a nilpotent operator; Understand the idea of Jordan blocks, Jordan matrices, and the Jordan form of a matrix; Define quadratic forms; State the second derivative test; Define eigenvectors and eigenvalues; Define a vector space and state its properties; State the notions of linear span, linear independence, and the basis of a vector space; Understand the ideas of linear independence, spanning set, basis, and dimension; Define a linear transformation; State the properties of linear transformations; Define the characteristic polynomial of a matrix; Define a Markov matrix; State what it means to have the property of being a stochastic matrix; Define a normed vector space; Apply the Cauchy Schwarz inequality; State the Riesz representation theorem; State what it means for a nxn matrix to be diagonalizable; Define Hermitian operators; Define a Hilbert space; Prove the Cayley Hamilton theorem; Define the adjoint of an operator; Define normal operators; State the spectral theorem; Understand how to find the singular-value decomposition of an operator; Define the notion of length for abstract vectors in abstract vector spaces; Define orthogonal vectors; Define orthogonal and orthonormal subsets of R^n; Use the Gram-Schmidt process; Find the eigenvalues and the eigenvectors of a given matrix numerically; Provide an explicit description of the Power Method. (Mathematics 212)

This course covers the following topics: models of manufacturing systems, including transfer lines and flexible manufacturing systems; calculation of performance measures, including throughput, in-process inventory, and meeting production commitments; real-time control of scheduling; effects of machine failure, set-ups, and other disruptions on system performance.

Welcome to 6.041/6.431, a subject on the modeling and analysis of random phenomena and processes, including the basics of statistical inference. Nowadays, there is broad consensus that the ability to think probabilistically is a fundamental component of scientific literacy. For example: The concept of statistical significance (to be touched upon at the end of this course) is considered by the Financial Times as one of "The Ten Things Everyone Should Know About Science". A recent Scientific American article argues that statistical literacy is crucial in making health-related decisions. Finally, an article in the New York Times identifies statistical data analysis as an upcoming profession, valuable everywhere, from Google and Netflix to the Office of Management and Budget. The aim of this class is to introduce the relevant models, skills, and tools, by combining mathematics with conceptual understanding and intuition.

Introduction to process control in chemical engineering. Application of linearization methods to the analysis of open-loop and closed-loop process dynamics. Stability analysis and gain/phase margins. Controller modes and settings. Applications to the control of level, flow, heat exchangers, reactors, and elementary multivariable systems.