Thursday 1 May 2008

Matlab on amdx86_64 linux

About two weeks ago, I bumped into the problem with Matlab, refusing to compile mex file under linux AMD64. My desktop machine is opteron and our nodes on cluster AMD as well, so I spend some time figuring out what went wrong.
It turned out that in order to compile this small mex file (only three for-loops through sparse matrix) I had to pass -largeArrayDims option to mex.
But it caused a problem - instant crash on call to the mex function.
According to matlab help here you have to change all you int either to mwSize or to mwIndex. Since both of them will be translated into size_t it wasn't clear for me which one use and where. I followed tutorial, compiled and run my small mex file (created from fulltosparse.c example long time ago), but my calculation returned only sparse matrix full of zeros.
Matlab solution suggest to debug until you will a problem. I can't imagine a ways to debug 4000 X 4000 matrix multiplication and the problem clearly lies within use of mwIndex or mwSize. Also it is worth mentioning that I managed to repeat the fault on my powerbook just compiling with -largeArrayDims.
After spending whole day, trying to fix it, I found a proper solution, posted in comp.soft-sys.matlab. Unfortunately I cannot find this post now, but it is brilliant. I think it should be in matlab help, instead of the current solution. The chap (Tim Davis from CXSparse? Sorry, I didn't find this post again to give you proper credit) wrote:
Use
#define Int mwSignedIndex
and then replace all int with Int. Hurrah. It is now working and producing meaningful results.

Another almost honest way of ripping your off.

Some of you probably have found scratch and match cards, laying around on the train and bus station.
I have found two of those and discovered that I have won a cash prize. It smells fishy since in order to claim the prize you have to phone or text to specific premium number and the costs of both will be near 9 pounds. After 1 month of waiting I decided to give it a go - I text.
What is the bottom line? I won £5 in cash, but I spend £9 claiming it means I just lost £4 (Assuming that they are honest and actually pay prizes). It's usually called scum or plot.
For those of you, who have found this card (I have seen chap sitting next to me yon the train this Monday found a "wining" card as well) I post wining security codes, so you can check you prize without spending £9 as I did.
965391=£500
421132=£1000
123941=£50
331546=£25000
654281=£5 (my number)
543522=£200
811846=£100

Next question, should I try to claim something like nintendo? I have another wining card, but again it's 9 quid.
I hope it will help someone, so unless you are really winner don't try to claim it.

Wednesday 19 March 2008

Matlab replacement

Thank you folks, I didn't expect any comments on my first entry about ocaml and scientists. 
I am also looking for matlab replacement,  although matlab is `de-facto' standard for signal processing.
As far as I know there are several (free) alternatives:
octave
R
Python (scipy+numpy), some people in our university was productive using this language as a glue to fortran routines. Although I tried to use it several times I never progressed far enough. I got frustrated, when python interpreter didn't like my tabs for code alignment. 
And there is even Ruby for scientists.
My first computer language was perl, so I might try it for signal processing.
I would like to hear about the succesful transitions from matlab to anything else. Especially in the areas of signal/image processing, communications.



Thursday 31 January 2008

Why ocaml and haskell are not for scientists?

Why ocaml and haskell is not for scientists.

I am currently postdoc, soon it will be a year. For my new project I decided to try to use ocaml instead of more traditional environment - Matlab.


But after a year reading documentation, books, trying to implement something, I decided to drop it and keep using matlab. If time permits I would keep an eye on the development of this languages, but I would not use it for scientific calculations in the nearest future.

Here are my points and I want someone to convince my that I am wrong - I would like to be convinced.

1) The choice of ocaml for the project was due to the fact that my new project has a lot of recursive functions and functional languages should handle it better than matlab.

2) Native integer precision on amd 64.


3) Speed.


I was really impressed with demos from the book "OCaml for Scientists" by Jon Harrop, however the price tag was too high, so I ended up getting it through inter library loan for the period 3 weeks. The book is worth reading, it gave me few very good ideas. But!! Most examples from the book are not repeatable. That is not scientific for me.


There is nothing in the book about the versions of the modules used or even OS. (It's actually debian linux)


I installed debian, but I still had a problem repeating examples.

Because 2D figures in the book generated using Mathematica rather than ocaml. Why do you think it has been done this way? Because there is no binding to gnuplot or simular lib for plotting. Actually there is, but it is not stable and has not been updated for a while. I was eagerly waiting for the end of Ocaml Summer Project as visualisation toolkit and mathematical framework looked fairly useful for me. It's already summer project 2008 announced, but the projects from osp2007 still in svn even not pre-alpha.


I think this is one of the major problems with ocaml - you can find very interesting projects or modules, but they mostly abandon, never made it ever as far as beta. And with lack of standards and documentation for modules I spend hours trying to install some necessary modules. As a former unix system administrator, I could install server with something like postfix+dovecot+samba+ldap with maia, but I can't install for example fftw module on linux and on mac os x. I was ready to tear my hair off. I think because ocaml community is small you rarely would find real cross platform modules there.


As a newbie I found it frustrating that I could not repeat completely any ocaml tutorial or book. As soon as you try to move from basic calculation, you are hitting  the wall trying to repeat example from the book or tutorial and getting some obscure message.


With haskell I was lucky enough to get hmatrix working, it does have a proper plotting and matrix manipulation. I even got as far as writing my own haar wavelet and started hennon map. Guess what? Hennon map in haskell is slow. Slower than perl and slower then C. But haskell community larger and more dynamic, possible due to the fact that haskell syntax standard, rather then current 3 styles of ocaml.


And because ocaml and haskell have strong types you will have to convert vector matrix to vector double if you want to plot it or to pass from gsl library to fft or another library.


My conclusion - I don't get paid for rewriting common modules and calculations in different languages. I get paid for inventing new thing, new methods.


Although ocaml and haskell interesting languages, they just not ready for scientific use. I didn't mean computer scientist, who actually get paid for writing software, but for scientists in more general sense - physics, signal processing, image processing. Also having good language is not enough to be useful. Creating compiler is a mandatory exercise for computer science undergraduate.


It is a community, modules and common standards that allow easy adoption of new language.


So far neither ocaml no haskell, not even ruby or python reached the level of perl. I am talking about CPAN. Although python and ruby getting close. So it might be worth trying perl for number crunching or draw back to C++ or C. It worth checking new technologies, but in the end it is a result that is required. Repeatable and published.



Update: After initial post Christophe did a lot of work improving fftw3 with new examples.