Preparing a hashed password with Jupyter Notebook

http://jupyter-notebook.readthedocs.io/en/latest/public_server.html#preparing-a-hashed-password

You can prepare a hashed password manually, using the function notebook.auth.security.passwd():

In [1]: from notebook.auth import passwd
In [2]: passwd()
Enter password:
Verify password:
Out[2]: 'sha1:67c9e60bb8b6:9ffede0825894254b2e042ea597d771089e11aed' 

You can then add the hashed password to your jupyter_notebook_config.py. The default location for this file jupyter_notebook_config.py is in your Jupyter folder in your home directory, ~/.jupyter, e.g.:

c.NotebookApp.password = u'sha1:67c9e60bb8b6:9ffede0825894254b2e042ea597d771089e11aed'

Resolving conflicts on a git branch

You'll need to update your branch with new commits from master, resolve those conflicts and push the updated/resolved branch to GitHub.

Resolve conflicts

I found these instructions much better than the instructions from Bitbucket involving a detached head! Thanks jaw6!

git checkout master
git pull
git checkout <branch>
git merge master
[ ... resolve any conflicts ... ]
git add [files that were conflicted]
git commit
git push

Direct Reference: https://github.com/githubteacher/github-for-developers-sept-2015/issues/648


Merge master

git checkout somebranch # gets you on branch somebranch
git fetch origin        # gets you up to date with origin
git merge origin/master

Reference: https://stackoverflow.com/questions/20101994/git-pull-from-master-into-the-development-branch


If it all goes wrong somehow, reset with the following (but ensure you're previously checked in as you can lose work with this command!!)

git reset –hard
git reset --hard <SOME-COMMIT>

Reference: https://stackoverflow.com/questions/9529078/how-do-i-use-git-reset-hard-head-to-revert-to-a-previous-commit


Bonus: reverting local changes

Revert changes made to your working copy:
git checkout .

Revert changes made to the index (i.e., that you have added), do this. Warning this will reset all of your unpushed commits to master!:
git reset

Revert a change that you have committed:
git revert <commit 1> <commit 2>

Remove untracked files (e.g., new files, generated files):
git clean -f

Remove untracked files directories (e.g., new or automatically generated directories):
git clean -fd

Reference: https://stackoverflow.com/questions/1146973/how-do-i-revert-all-local-changes-in-git-managed-project-to-previous-state

Cleaning up Docker disk space

Docker seems to eat disk space and leave a lot of left overs around. Here are some resources to fix that.

Docker image prune (the new standard fix)

docker image prune

https://docs.docker.com/config/pruning/
https://docs.docker.com/engine/reference/commandline/image_prune/
https://gist.github.com/anildigital/862675ec1b7bccabc311

General Info
https://www.digitalocean.com/community/tutorials/how-to-remove-docker-images-containers-and-volumes

https://stackoverflow.com/questions/17665283/how-does-one-remove-an-image-in-docker


Is the /var/lib/docker/ folder large?
sudo du -sh /var/lib/docker/

If it's large, may need to zap it (careful with this one!):
Ref: https://stackoverflow.com/questions/45798076/how-to-clean-up-docker

$ sudo su
# service docker stop
# cd /var/lib/docker
# rm -rf *
# service docker start

For more info about this folder see: https://stackoverflow.com/questions/19234831/where-are-docker-images-stored-on-the-host-machine

And also......try removing dangling images

https://forums.docker.com/t/how-to-remove-none-images-after-building/7050/3

List dangling:
docker images -f "dangling=true" -q

Delete dangling:
docker rmi $(docker images -f "dangling=true" -q)

Force delete dangling:
docker rmi -f $(docker images -f "dangling=true" -q)

Rsync Broken Pipe Error with SSH

Try adding these settings:

KeepAlive yes
ServerAliveInterval 20
ServerAliveCountMax 6

to your /etc/ssh/ssh_config or ~/.ssh/config file.

To solve Broken Pipe errors.

Reference: https://unix.stackexchange.com/questions/68775/rsync-timed-out

Installing Docker on an Ubuntu AWS EC2 machine

sudo apt-get update

# Install packages to allow apt to use a repository over HTTPS:
sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    software-properties-common

# For "sudo: unable to resolve host ip-x-x-x-x" errors, follow the following link to set "DNS Hostnames" to True:
# http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-dns.html#vpc-dns-updating
sudo true

# Add Docker’s official GPG key
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

# Verify that the key fingerprint is 9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88.
sudo apt-key fingerprint 0EBFCD88

# Use the following command to set up the stable repository.
sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

# Install the latest version of Docker CE
sudo apt-get update
sudo apt-get install docker-ce
sudo docker run hello-world

# Ref: https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/#install-using-the-repository

Set up RVM, Ruby, Gem & Jykll to serve a username.github.io blog

Here are the steps I took to set up RVM, Ruby, Gem & Jykll to serve a username.github.io blog on OSX:

# Install RVM (Ruby version manager) 
# Also read: http://rvm.io/rvm/install and check the script before running
curl -sSL https://get.rvm.io | bash -s stable --ruby

# Install Ruby (replace x.y.z with latest stable version of ruby)
rvm install x.y.z
rvm docs generate-ri
rvm use x.y.z --default

# Check other requirements are available for jekyll and install if missing
gem --version
gcc --version
make --version

# Install jekyll
gem install jekyll

# Install bundler and get all dependencies
gem install bundler
bundle install

# Clone an existing jekyll template, this one from BlackrockDigital is nice
git clone https://github.com/BlackrockDigital/startbootstrap-clean-blog-jekyll.git
cd startbootstrap-clean-blog-jekyll
bundle exec jekyll serve

# Periodically update Jekyll to match the GitHub Pages server
bundle update github-pages

Reference:
https://help.github.com/articles/setting-up-your-github-pages-site-locally-with-jekyll/
http://jekyllrb.com/docs/installation/
https://jekyllrb.com/docs/posts/
https://github.com/planetjekyll/awesome-jekyll-editors

Running PaCAL on OSX with conda

PaCAL is a great library for performing arithmetic on probabilistic random variables just like you do with ordinary program variables. Here is a more detailed paper on PaCAL.

There are a couple of gotchas setting it up on OSX with conda:

• Requires Python 2.7.x (3.x not supported as of March 2017)
• Required dependencies: numpy matplotlib sympy scipy
• Workaround for "RuntimeError: Python is not installed as a framework. The Mac OS X backend will not be able to function correctly if Python is not installed as a framework".

Here are the steps to set up PaCAL under a conda environment on OSX:

$ conda create -n pacal python=2.7
$ source activate pacal

(pacal) $ pip install numpy matplotlib sympy scipy
(pacal) $ pip install pacal

(pacal) $ python
Python 2.7.13 |Continuum Analytics, Inc.| (default, Dec 20 2016, 23:05:08)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
>>>
>>> # This TkAgg bit is to prevent the OSX 'Python is not installed as a framework' error.
>>> import matplotlib as mpl
>>> mpl.use('TkAgg')
>>>
>>> from pacal import *
Compiled interpolation routine not available
Compiled sparse grid routine not available
>>> dL = UniformDistr(1,3)
>>> L0 = UniformDistr(9,11)
>>> dT = NormalDistr(1,1)
>>> K = dL / (L0 * dT)
>>> K.plot()
>>> show()