MPI farm programs on non-dedicated clusters

Abstract

MPI has been extremely successful. In areas like e.g. particle physics most of the available parallel programs are based on MPI. Unfortunately, they must be run in dedicated clusters or parallel machines, being unable to use for long running applications the growing pool of idle time of general-purpose desktop computers. Additionally, MPI offers a quite low level interface, which is hard to use for most scientist programmers. In the research described in this paper, we tried to see how far we could go to solve those two problems, keeping the portability of MPI programs, but drawing upon one restriction - only programs following the FARM paradigm were to be supported. The developed library - MpiFL - did provide us significant insight. It is now being successfully used at the physics department of the University of Coimbra, despite some shortcomings. © Springer-Verlag Berlin Heidelberg 2003.