Monday, January 15, 2018

Get Open AI (gym) up and running on Windows Subsystem Linux

Quick list of steps to get Open AI running on Windows

1. Activate WSL (Bash on Windows). Refer this for detailed steps.


2. Run the following commands on Bash. These replace the default libzmq3 installation with one having a bug-fix.

pip uninstall pyzmq
sudo add-apt-repository ppa:aseering/wsl
sudo apt-get update
sudo apt-get install libzmq3 libzmq3-dev
export LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu
pip install --no-use-wheel -v pyzmq
pip install jupyter

3. Install dependencies for gym:

sudo apt-get install -y build-essential
sudo apt-get install -y zlib1g-dev liblapack-dev libatlas-base-dev libopenblas-dev libhdf5-dev libedit-dev
pip install gym[all]








Sunday, January 14, 2018

Heuristic Sudoku Solver

A quick google search will provide you with numerous Sudoku  solvers and algorithms including ones using backtracking, exploratory search, etc. Such algorithms can find a solution (if exists) to a Sudoku problem even if it is not humanly possible find deterministic cell values. Of course, a brute force approach by a human being will probably eventually yield the correct solution, but for this post I consider the approach used to selectively use constraints to determine the correct cell values one by one. One can trivially reduce a Sudoku problem to one that doesn't have enough values to apply constraints and find correct values in a deterministic way.

This solver uses the same constraint based approaches to selectively determine the solution cell by cell. In other words, I have tried to make this solver akin to human heuristic solving and any sudoku problem solvable by this is guaranteed to be solvable by a human in the deterministic approach described above. Overall, this solver can double as a classifier of sudoku puzzles into difficulty levels based on certain parameters. Code available here.

Tuesday, March 7, 2017

Remote Debugging Python on Eclipse

Remote debugging is extremely useful, especially when you have limited access to the system used to run the application.

Let us call the system where the application is to be run as rsys and the one used to debug as dsys.

Setup dsys :
1. Install eclipse on dsys.

2. Make sure you have the pydev and debug perspectives enabled in eclipse.

3. Run the debug server on eclipse. The console should give you the port number that the debug server is listening to (default : 5678).

Setup rsys:
1. If you have root privileges on rsys install the python package 'pydevd'.
  pip install pydevd

2. If you do not have root privileges, use the target option in pip :
  pip  install  --target=d:\other\location  package_name

3. If you also do not have pip installed, download the package and install it to a custom directory. (refer https://docs.python.org/2/install/)
For 2. and 3., do not forget to add the custom directory to PYTHONPATH to import the package in your program.

Test the debugger:
1. Create a file hello_world.py and make sure it can be accessed both on rsys and dsys. (It should be the same file and not a copy of it)

2. Type the following into hello_world.py :

import pydevd
pydevd.settrace("10.X.Y.Z", port=5678)
# where 10.X.Y.Z is the IP address of dsys and 5678 is the port of the debug server running on dsys.

print "Hello World!"
# where your actual program goes.

3. In Eclipse (on dsys) create a new PyDev project including hello_world.py in its source. (Make sure the debugger is still running on port specified in the source file.)

4. Run hello_world.py on rsys. Hopefully your running application should be caught on the remote debug server running on Eclipse. (It may prompt you to choose the source file.)
Note: If break points do not seem to be working at this point, relevant paths need to be incorporated in the configuration for Eclipse and pydevd as below.

Configuration for project level debugging and working breakpoints:
(Refer to this pydevd documentation)

Important: Make sure you have the SAME source file structure mounted both on rsys and dsys. Unless there is a one-one correspondence between the files, adding breakpoints in eclipse editor on dsys will not be recognized by the program running on rsys and a warning will be displayed on rsys :
>pydev debugger: warning: trying to add breakpoint to file that does not exist:
/home/pi/python/f:/python projects/server/server.py (will have no effect)

1. Edit  the file pydevd_file_utils.py in the installed packages pydevd on rsys.
A portion of comments on that file explicitly instructs the corresponding project source paths on rsys and dsys be added to the file :
"E.g.:
        If the server (your python process) has the structure
            /user/projects/my_project/src/package/module1.py

        and the client has:
            c:\my_project\src\package\module1.py

        the PATHS_FROM_ECLIPSE_TO_PYTHON would have to be:
            PATHS_FROM_ECLIPSE_TO_PYTHON = [(r'c:\my_project\src', r'/user/projects/my_project/src')]"

4. In Eclipse, go to Window > Preferences > PyDev > Debug > Source Locator and add a similar entry as the one to pydevd_file_utils.py with the Path to translate being the path on rsys and Translated path the one on dsys.

Sunday, April 17, 2016

Get Bash on Windows 10 Build 14316

To get Bash (Ubuntu) as part of Windows 10, you first need to become an 'insider'.
  • Go to Settings > Update & Security > Advanced Options and toggle the Get Insider Preview Builds option with the level to Fast.
  • From Settings > Update & Security > For Developers enable Developer mode.
  • Go to Settings > Update & Security > Advanced Options and in the section Choose how updates are delivered, turn the option for PC's on my local network and the internet on. (This isn't required, but let me download the build immediately after I enabled it).
  • After you get the build, go to Programs and Features from the Control Panel and choose option Turn Windows Features on or off. Enable Windows subsystem for Linux (Beta)
  • Search for bash in the start menu and you're all set!



Saturday, April 16, 2016

IPython freeze on OpenCV destroyAllWindows

If you are running OpenCV on an IPython console, chances are cv2.destroyAllWindows() will freeze your image window and ultimately kill the kernel. To get around this, intersperse your code with the following functions (startWindowThread() and waitKey(1)) :

cv2.namedWindow('image')
cv2.startWindowThread()
.
.
.
cv2.waitKey(1)
cv2.destroyAllWindows()
cv2.waitKey(1)

Should hopefully solve the issue!

OpenCV Error on Anaconda


Error: Unspecified error (The function is not implemented. Rebuild the library with Windows, GTK+ 2.x or Carbon support. If you are on Ubuntu or Debian, install libgtk2.0-dev and pkg-config

Solution:
conda remove opencv
conda update conda
conda install --channel menpo opencv

This installs the opencv from public channel menpo which works fine.

Saturday, March 5, 2016

Convert binaryproto format to numpy (Readable)

To view the values of image-mean data stored in binaryproto format, use the following:

import caffe
import numpy as np
import sys

blob = caffe.proto.caffe_pb2.BlobProto()
data = open( 'YOURBPFILE.binaryproto' , 'rb' ).read()
blob.ParseFromString(data)

arr = np.array( caffe.io.blobproto_to_array(blob) )
print arr


Tuesday, September 15, 2015

Pickled Object Problem

When unpickling an object of some class, if you encounter the error,
AttributeError: 'module' object has no attribute 'classname'
read this.

Insert Code in the 'right format' in Blogger Posts

To insert code into blogger posts, use one of the following :
  1. htmlify
  2. markup
  3. hilite.me

Saturday, February 7, 2015

Building MEX files in MATLAB 2014B on Ubuntu 14.04

Ubuntu 14.04 by default has the compiler GCC 4.8, but Matlab R2014b .mex files require GCC 4.7.x

Error: You are using gcc version '4.8.2'. The version of gcc is not supported.
Solution:

Install GCC 4.7 and G++ 4.7

sudo apt-get install gcc-4.7 g++-4.7

Copy the mex-options file 
cp /usr/local/MATLAB/R2014b/bin/mexopts.sh ~/.matlab/R2014b/mexopts.sh

Provide read permissions
chmod 644 ~/.matlab/R2013b/mexopts.sh

Make edits
vim ~/.matlab/R2013b/mexopts.sh

Change :
CC=’gcc’ to CC=’gcc-4.7′
CCX=’g++’ to CCX=’g++-4.7′

Wednesday, November 5, 2014

How to interrupt Ipython Kernel in Spyder IDE (Anaconda)


In Spyder IDE, the usual interrupt 'Ctrl-C' does not work if you are running your script on an IPython Console. This is because multiple of these consoles are linked to a single kernel. In fact, this message is displayed on the kernel itself :

NOTE: When using the `ipython kernel` entry point, Ctrl-C will not work.

To exit, you will have to explicitly quit this process, by either sending
"quit" from a client, or using Ctrl-\ in UNIX-like environments.

Read more about this, here.


A quick work-around this is to tweak some settings in Spyder:

Go to Run -> Configure. Select Execute in a new dedicated Python console. Check Interact with the Python console after execution.

Now when you run a script, the Interrupt button on the right top of the console should stop the script.

Monday, October 20, 2014

When to use ArrayLists and not LinkedLists in Java

Disclaimer: Posted for my convenience. Referred from here

LinkedList and ArrayList are two different implementations of the List interface. LinkedList implements it with a doubly-linked list. ArrayList implements it with a dynamically resizing array.

As with standard linked list and array operations, the various methods will have different algorithmic runtimes.

For LinkedList<E>
  • get(int index) is O(n)
  • add(E element) is O(1)
  • add(int index, E element) is O(n)
  • remove(int index) is O(n)
  • Iterator.remove() is O(1) <--- main benefit of LinkedList<E>
  • ListIterator.add(E element) is O(1) <--- main benefit of LinkedList<E>

For ArrayList<E>
  • get(int index) is O(1) <--- main benefit of ArrayList<E>
  • add(E element) is O(1) amortized, but O(n) worst-case since the array must be resized and copied
  • add(int index, E element) is O(n - index) amortized, but O(n) worst-case (as above)
  • remove(int index) is O(n - index) (i.e. removing last is O(1))
  • Iterator.remove() is O(n - index)
  • ListIterator.add(E element) is O(n - index)

LinkedList<E> allows for constant-time insertions or removals using iterators, but only sequential access of elements. In other words, you can walk the list forwards or backwards, but finding a position in the list takes time proportional to the size of the list.

ArrayList<E>, on the other hand, allow fast random read access, so you can grab any element in constant time. But adding or removing from anywhere but the end requires shifting all the latter elements over, either to make an opening or fill the gap. Also, if you add more elements than the capacity of the underlying array, a new array (1.5 times the size) is allocated, and the old array is copied to the new one, so adding to an ArrayList is O(n) in the worst case but constant on average.

So depending on the operations you intend to do, you should choose the implementations accordingly. Iterating over either kind of List is practically equally cheap. (Iterating over an ArrayList is technically faster, but unless you're doing something really performance-sensitive, you shouldn't worry about this -- they're both constants.)

The main benefits of using a LinkedList arise when you re-use existing iterators to insert and remove elements. These operations can then be done in O(1) by changing the list locally only. In an array list, the remainder of the array needs to be moved (i.e. copied). On the other side, seeking in a LinkedList means following the links in O(n), whereas in an ArrayList the desired position can be computed mathematically and accessed in O(1).

Also, if you have large lists, keep in mind that memory usage is also different. Each element of a LinkedList has more overhead since pointers to the next and previous elements are also stored. ArrayLists don't have this overhead. However, ArrayLists take up as much memory as is allocated for the capacity, regardless of whether elements have actually been added.

The default initial capacity of an ArrayList is pretty small (10 from Java 1.4 - 1.7). But since the underlying implementation is an array, the array must be resized if you add a lot of elements. To avoid the high cost of resizing when you know you're going to add a lot of elements, construct the ArrayList with a higher initial capacity.

It's worth noting that Vector also implements the List interface and is almost identical to ArrayList. The difference is that Vector is synchronized, so it is thread-safe. Because of this, it is also slightly slower than ArrayList. So as far as I understand, most Java programmers avoid Vector in favor of ArrayList since they will probably synchronize explicitly anyway if they care about that.

Sunday, October 12, 2014

Enable USB Debugging on Moto G

First to turn on the Developer Mode,
Settings => About Phone => Tap "BUILD NUMBER" 7 times

Next, to enable USB Debugging,

Settings => Developer Options => USB Debugging

Thursday, October 9, 2014

Useful IPython Notebook Shortcuts

Command Mode Shortcuts

enter: edit mode
shift+enter: run cell
ctrl+enter: run cell, select below
alt+enter: run cell, insert below
y: to code
m: to markdown
t: to raw
1: to heading 1
2: to heading 2
3: to heading 3
4: to heading 4
5: to heading 5
6: to heading 6
up: select previous cell
down: select next cell
k: select previous cell
j: select next cell
ctrl+k: move cell up

Edit Mode Shortcuts
esc: command mode
ctrl+m: command mode
shift+enter: run cell
ctrl+enter: run cell, select below
alt+enter: run cell, insert below
alt+-: split cell
meta+s: save notebook
ctrl+s: save notebook


ctrl+j: move cell down
a: insert cell above
b: insert cell below
x: cut cell
c: copy cell
v: paste cell below
z: undo last delete
d: delete cell (press twice)
shift+m: merge cell below
s: save notebook
meta+s: save notebook
ctrl+s: save notebook
l: toggle line numbers
o: toggle output
shift+o: toggle output
h: keyboard shortcuts
i: interrupt kernel
.: restart kerne