Sep
23

Install bokeh

  1. Install nodejs
    conda install -c conda-forge nodejs
  2. install jupyter lab
    conda install -c conda-forge jupyterlab
    https://github.com/jupyterlab/jupyterlab
  3. install jupyter lab:
    jupyter labextension install
    jupyter labextension list
  4. install bokeh plugin for jupyter
    jupyter labextension install jupyterlab_bokeh
    https://github.com/bokeh/jupyterlab_bokeh
Sep
21

the useful plugins for new linux installation

  1. install virtualenvironment:
    http://docs.python-guide.org/en/latest/dev/virtualenvs/
  2. Install open terminal here:
    sudo apt-get install nautilus-open-terminal
    nautilus -q
  3. install notebook extenstion
    pip install jupyter_contrib_nbextensions
    jupyter contrib nbextension install –user
    https://github.com/ipython-contrib/jupyter_contrib_nbextensions
Jun
06

Make string to a command in python

https://pypi.python.org/pypi/fmt

Jun
01

Jupyter Magic Memory

  1. %: line magic, %%cell magic
  2. Repeat Command:
    %recall
  3. Write cell content into a file:
    %%writefile file.py
  4. To autoreload your pyfile:
    %load_ext autoreload
    %autoreload 2
    from yourfile import yourFunction
    https://ipython.org/ipython-doc/3/config/extensions/autoreload.html
  5. Use R in jupyter
    %load_ext rpy2.ipythonR magic in cell
    %%Rcode in line
    %R

    Move variable between R and python
    %Rpush, %Rpull
    R magic tutorial: https://blog.dominodatalab.com/lesser-known-ways-of-using-notebooks/
    R magic: https://ipython.org/ipython-doc/2/config/extensions/rmagic.html
    rpy2: http://rpy2.readthedocs.io/en/version_2.8.x/index.html
    Problem in rmagic: https://stackoverflow.com/questions/25829963/r-magic-for-ipython-notebook-runtime-error-turning-off-device-on-new-server-inst

  6. %store
    store variable between different ipython notebook
May
01

Two ways to update docs in cursor objects with pymongo

I have a collection with:

name    weight    amount
—————————————-
apple     2               3
banana  2               5
pear       2               5

I want add a column about the total price, which is equal to weight * amount

name    weight    amount    total
—————————————————-
apple     2               3              6
banana  2               5             10
pear       2               5            10


myFile = [
{"name":"apple", "weight":2, "amount":3}, 
{"name":"banana", "weight":2, "amount":5}, 
{"name":"pear", "weight":2, "amount":5}
]

testDB.db.testCollection.delete_many({})
testDB.db.testCollection.insert_many(myFile)

Method 1: collection.find() + update_one()

fruits = testDB.db.testCollection.find({}, modifiers={"$snapshot": True})

for fruit in fruits:
    total = fruit['weight'] * fruit['amount']
    testDB.db.testCollection.update_one({"_id":fruit['_id']}, {'$set': {'total': total}})

# check the result:
list(testDB.db.testCollection.find())</pre>
<pre>

 

Method 2: collection.aggregate() with $project, $multiply and $out


testDB.db.testCollection.aggregate([{'$project': {
        'name':1,
        'weight':1,
        'total': {"$multiply": ["$weight",'$amount']}
      }}, {
        '$out':'testCollectioncopy'
      }])
# check the result
list(testDB.db.testCollectioncopy.find())
Apr
06

How to reference python variable in jupyter notebook markdown cell

  1. Install Jupyter notebook extension plugin: https://github.com/ipython-contrib/jupyter_contrib_nbextensions
    1. conda install -c conda-forge jupyter_contrib_nbextensions
    2. jupyter contrib nbextension install --user
  2. Go to Jupyter notebook
    1. click Nbextensions
    2. enable Python Markdown
  3. use {{python variable}} in markdown cell. 🙂
Feb
26

pycharm for django and sass

For django:

professional version.

  1. create a new django project: just create a new one
  2. import a django project:
    1. import the project
    2. activate django support

For Sass:

https://www.jetbrains.com/help/pycharm/2016.3/transpiling-sass-less-and-scss-to-css.html

  1. make sure, the sass support is activate: File -> Setting -> Plugin
  2. install node.js: File -> Setting -> Plugin -> install jetbrain plugin
  3. install ruby: during the installation, don’t forget check the option “set path to default”
  4. open teminal to install sass: “gem install sass”
  5. create a file watcher: File -> setting ->tools -> file watcher -> add -> sass
  6. transpiling the code
Nov
25

Cassandra, Python, R, Node.js

It is quite nice, that Cassandra has driver for Python, R and Node.js.

If you want to build a website with Cassandra DB which has also heavily scientific programming, I would suggest the following combination:

  1. Python Flask  or Django + Cassandra
  2. Python(ZeroMQ, zerorpc) + R(opencpu) + Nodejs + Cassandra
    https://ianhinsdale.com/post/communicating-between-nodejs-and-python/
    https://www.opencpu.org/

The first solution is easiest, but the second solution is more elegant since you can separate the Python, Cassandra, NodeJS in different server. Thus the advantages of Node.JS will be kept. Of course, you can also use the childprocess from nodejs to call the Python and R. 
http://www.sohamkamani.com/blog/2015/08/21/python-nodejs-comm/
https://github.com/extrabacon/python-shell

Oct
02

the most useful pycharm hotkey

ctrl + s : save all

ctrl + y : delete row

ctrl + b / ctrl + click: go to the source

 

Jun
07

The useful python libraries

contextlib.py : can include the object in “with” and keep the memory small.

py4j: call java from python