STAT 624: Statistical Computation

Dr. Robert Richardson
Brigham Young University
Fall 2020

  Syllabus

Project
Homework 9 is active.
Homework Link Due Date
Homework 1 September 9
Homework 2 September 16
Homework 3 September 23
Homework 4 September 30
Homework 5 October 7
Homework 6 October 21
Homework 7 October 21
Homework 8 November 4
Homework 9 November 18
Homework 10 December 2


Best Practices Grading (10 points available in every assignment)

Python for R users: Slides
2017 Final
2018 Midterm
2018 Final

Video: Final 2018
2019 Midterm
R solution: Code using this data: Data
Video: Midterm 2019

Video: Final 2019

Video: HW 6
Code: hw6.R and Data: tw.txt

Video: HW 7
Code: hw7.R and acctest.py

Video: HW 8
Code: hw8.R and Data: tw.txt


Class Schedule

Overview of course: Computing in the department and getting set up:
  • Pete Dotson, CSR: Introduction to departmental computing resources and policies
  • Statistics Computing Resources. Login and try again if you get "Access Denied".
  • Introduction to rencher.byu.edu
  • Text-based login (ssh)
    • Linux distributions have a builtin ssh client.
    • Mac OS X has a builtin ssh client.
    • Windows ssh client: PuTTY.
  • File transfer (scp)
    • Linux distributions have a builtin scp client.
    • Mac OS X has a built in scp client.
    • Windows file transfer using WinSCP.
  • Enter your account on rencher and change password
    • ssh net.id@rencher.byu.edu
    • Enter temporary password
    • Change password: enter passwd then follow prompts to enter in old and input new password.
Introduction to vi. Basic UNIX commands: cd; ls; mkdir; rm; rm -r; cp; vi; mv; scp

Setting up account for git access. Use the following terminal commands and just press enter when given any options:
  • ssh-keygen
  • cat ~/.ssh/id_rsa.pub
Email the output starting from ssh-rsa and ending with net.id@rencher
R: An Introduction to R.
Python: An Introduction to Python.
  • Running scripts in BATCH mode.
  • Using tmux for split vi/R session.
More on tmux. Cheat Sheet
  • Download this file and rename it ~/.vimrc
  • Download this file and rename it ~/.vimrc.line-feeder-3
  • Download this file and rename it ~/.vimrc.latex-helper
  • Download this file and rename it ~/.tmux.conf


Clone your personal git repository
  • git clone git624@collings.byu.edu:your.netid
Read-only STAT 624 general git repository:
  • Clone command: git clone git624@collings.byu.edu:general624

Git: A free and open source distributed version control system designed to handle everything from small to very large projects with speed and efficiency.

Submitting Homework
  • git pull
  • git add filename or git add .
  • git commit -m "message"
  • git push
Note: Check git status before each step while you are getting used to it.
Latex: Introduction
  • Article
  • Presentation
Simulation Studies.
  • Represent a real-world process of interest on the computer to evaluate statistical properties associated with the process.
  • Uses include:
    • Evaluating point estimators.
    • Evaluation confidence intervals.
    • Checking the finite-sample statistical properties of estimators and testing procedures that have been motivated through asymptotics. meantest.R and meantest.py
    • Testing hypothesis.
    • Describing distributions.
    • and many others...
  • Especially useful when theoretical derivations are unavailable, difficult, or intractable.
  • "Simulation study" and "Monte Carlo study" are synonymous.

Monte Carlo Integration
???????????
Remember the definition of size, power, and p-value.

Hypothesis Testing and Power Calculation via simulation: hyp.R
MC integration of a generic function.
Hit-and-miss method for finding areas and volumes.
MC Integration Simulation in R
and Python
Hit-and-miss method for finding areas and volumes.
Code for homework urn.R and urn.py

Video lecture: 9/25
Numerical integration.
Root finding methods.


Video Lecture: 10/02
Optimization.
Optimization.
Markov chain Monte Carlo.


Video Lecture: 10/23
More Markov chain Monte Carlo.
More Markov chain Monte Carlo.
Parallel Computing
Intro to C and C++: sum.c and sd.c
Integrating C++ and R

Video Lecture: 11/11
Importance Sampling: Algorithm
  • comp.R and mvnorms.cpp
  • Perform inference for covariance parameters in a spatial model
  • spatial.R and mnlik.cpp

  • Webscraping in python
    Scraping weather data in weather.py in python


    Video Lecture: 11/23

    More Webscraping in python

    Video Lecture: 11/24
    Deep Learning: dl.py using pima.csv
    Keras Models: link
    Activation functions: link

    Video Lecture: 11/30

    Video Lecture: 12/02