May
24

R important things

R has no block comment, but you can still do the following things:

  • In RStudio: Ctrl+Shift+C
  • if(false){xxx} block

 

python to R

df.tail() -> tail(df)

dropna(df) ->na.omit(df) / df[complete.cases(df),]

df.describe() -> summary(df), library(psych): describe(df)

df.shape -> dim(df)

May
01

Two ways to update docs in cursor objects with pymongo

I have a collection with:

name    weight    amount
—————————————-
apple     2               3
banana  2               5
pear       2               5

I want add a column about the total price, which is equal to weight * amount

name    weight    amount    total
—————————————————-
apple     2               3              6
banana  2               5             10
pear       2               5            10


myFile = [
{"name":"apple", "weight":2, "amount":3}, 
{"name":"banana", "weight":2, "amount":5}, 
{"name":"pear", "weight":2, "amount":5}
]

testDB.db.testCollection.delete_many({})
testDB.db.testCollection.insert_many(myFile)

Method 1: collection.find() + update_one()

fruits = testDB.db.testCollection.find({}, modifiers={"$snapshot": True})

for fruit in fruits:
    total = fruit['weight'] * fruit['amount']
    testDB.db.testCollection.update_one({"_id":fruit['_id']}, {'$set': {'total': total}})

# check the result:
list(testDB.db.testCollection.find())</pre>
<pre>

 

Method 2: collection.aggregate() with $project, $multiply and $out


testDB.db.testCollection.aggregate([{'$project': {
        'name':1,
        'weight':1,
        'total': {"$multiply": ["$weight",'$amount']}
      }}, {
        '$out':'testCollectioncopy'
      }])
# check the result
list(testDB.db.testCollectioncopy.find())
Apr
06

How to reference python variable in jupyter notebook markdown cell

  1. Install Jupyter notebook extension plugin: https://github.com/ipython-contrib/jupyter_contrib_nbextensions
    1. conda install -c conda-forge jupyter_contrib_nbextensions
    2. jupyter contrib nbextension install --user
  2. Go to Jupyter notebook
    1. click Nbextensions
    2. enable Python Markdown
  3. use {{python variable}} in markdown cell. ๐Ÿ™‚
Mar
09

convolutional neural network for 3d object

3D Convolutional Neural Networks for Human Action Recognition

http://machinelearning.wustl.edu/mlpapers/paper_files/icml2010_JiXYY10.pdf

VoxNet: A 3D Convolutional Neural Network for Real-Time Object Recognition

https://www.ri.cmu.edu/pub_files/2015/9/voxnet_maturana_scherer_iros15.pdf

Question Website:

use tensorflow

http://stackoverflow.com/questions/33630840/convolutional-neural-networks-and-3d-images

use keras

https://github.com/fchollet/keras/issues/1359

Mar
08

credit flawed detection kaggle dataset

  1. base line: use all the attributes, tree algorithm, cross-validation parameter 3: 
      true 0 true 1 class precision
    pred. 0 284253 132 99.95%
    pred. 1 62 360 85.31%
    class recall 99.98% 73.17%  
    PerformanceVector:
    accuracy: 99.93% +/- 0.01% (mikro: 99.93%)
    ConfusionMatrix:
    True:	0	1
    0:	284253	132
    1:	62	360
    precision: 85.68% +/- 6.02% (mikro: 85.31%) (positive class: 1)
    ConfusionMatrix:
    True:	0	1
    0:	284253	132
    1:	62	360
    recall: 73.14% +/- 6.06% (mikro: 73.17%) (positive class: 1)
    ConfusionMatrix:
    True:	0	1
    0:	284253	132
    1:	62	360
    AUC (optimistic): 0.965 +/- 0.028 (mikro: 0.965) (positive class: 1)
    AUC: 0.858 +/- 0.035 (mikro: 0.858) (positive class: 1)
    AUC (pessimistic): 0.752 +/- 0.057 (mikro: 0.752) (positive class: 1)
    
    
    Use random forest tree:
    PerformanceVector:
    accuracy: 99.87% +/- 0.01% (mikro: 99.87%)
    ConfusionMatrix:
    True:	0	1
    0:	284298	342
    1:	17	150
    precision: 93.21% +/- 7.43% (mikro: 89.82%) (positive class: 1)
    ConfusionMatrix:
    True:	0	1
    0:	284298	342
    1:	17	150
    recall: 30.47% +/- 12.44% (mikro: 30.49%) (positive class: 1)
    ConfusionMatrix:
    True:	0	1
    0:	284298	342
    1:	17	150
    AUC (optimistic): 0.992 +/- 0.010 (mikro: 0.992) (positive class: 1)
    AUC: 0.852 +/- 0.039 (mikro: 0.852) (positive class: 1)
    AUC (pessimistic): 0.713 +/- 0.081 (mikro: 0.713) (positive class: 1)
  2. use selected attributes: time, v1,v2,v3, amount, class

    PerformanceVector

    PerformanceVector:
    accuracy: 99.84% +/- 0.01% (mikro: 99.84%)
    ConfusionMatrix:
    True:	0	1
    0:	284310	456
    1:	5	36
    precision: 93.00% +/- 11.40% (mikro: 87.80%) (positive class: 1)
    ConfusionMatrix:
    True:	0	1
    0:	284310	456
    1:	5	36
    recall: 7.34% +/- 4.55% (mikro: 7.32%) (positive class: 1)
    ConfusionMatrix:
    True:	0	1
    0:	284310	456
    1:	5	36
    AUC (optimistic): 0.998 +/- 0.006 (mikro: 0.998) (positive class: 1)
    AUC: 0.537 +/- 0.024 (mikro: 0.537) (positive class: 1)
    AUC (pessimistic): 0.075 +/- 0.044 (mikro: 0.075) (positive class: 1)
  3. select v1,v2,v3

    PerformanceVector

    PerformanceVector:
    accuracy: 99.85% +/- 0.01% (mikro: 99.85%)
    ConfusionMatrix:
    True:	0	1
    0:	284295	408
    1:	20	84
    precision: 81.93% +/- 12.69% (mikro: 80.77%) (positive class: 1)
    ConfusionMatrix:
    True:	0	1
    0:	284295	408
    1:	20	84
    recall: 17.08% +/- 5.20% (mikro: 17.07%) (positive class: 1)
    ConfusionMatrix:
    True:	0	1
    0:	284295	408
    1:	20	84
    AUC (optimistic): 0.992 +/- 0.010 (mikro: 0.992) (positive class: 1)
    AUC: 0.586 +/- 0.026 (mikro: 0.586) (positive class: 1)
    AUC (pessimistic): 0.181 +/- 0.055 (mikro: 0.181) (positive class: 1)
Feb
26

pycharm for django and sass

For django:

professional version.

  1. create a new django project: just create a new one
  2. import a django project:
    1. import the project
    2. activate django support

For Sass:

https://www.jetbrains.com/help/pycharm/2016.3/transpiling-sass-less-and-scss-to-css.html

  1. make sure, the sass support is activate: File -> Setting -> Plugin
  2. install node.js: File -> Setting -> Plugin -> install jetbrain plugin
  3. install ruby: during the installation, don’t forget check the option “set path to default”
  4. open teminal to install sass: “gem install sass”
  5. create a file watcher: File -> setting ->tools -> file watcher -> add -> sass
  6. transpiling the code
Jan
03

magic tts code for console

var msg = new SpeechSynthesisUtterance(‘Hello World’);
window.speechSynthesis.speak(msg);

https://developers.google.com/web/updates/2014/01/Web-apps-that-talk-Introduction-to-the-Speech-Synthesis-API

Jan
03

Decode a famous website

Many news medien encode their paid text, but some of them has very simple encode mechanism. ๐Ÿ˜€

I wrote the following code to decode a famous webseite. 

Just copy following codes in the console, then, you will see the text.

JS Bin on jsbin.com

Guess, which medien? ๐Ÿ˜€

Have fun!

Dec
06

vitualbox share folder

  1. device sharefolder -> set your folder path in windows and set folder name
  2. ubuntu terminal:
    1. sudo mount -t vboxsf FOLDERNAME /PATH/OF/FOLDER

For example: 

Windows: share folder: name: share, path: c:???

Ubuntu: sudo mount -t vbox share home/ying/share

 

enable clipboard share:

sudo apt-get install virtualbox-guest-dkms

 


set the right to the share_folder

sudo adduser xxxxxxx vboxsf (XXX: your user name)

Dec
06

virtualbox unbuntu install softwares

install java
sudo apt-add-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java8-installer

install open-terminal-here
$ sudo apt-get update && sudo apt-get upgrade
$ sudo apt-get install nautilus
$ sudo apt-get install nautilus-actions
$ nautilus-actions-config-tool * activate the command

for the case the clipboard sharing in virtualbox doesn’t work

$ sudo apt-get install virtualbox-guest-dkms