Dynamically scalable PBS cluster on Amazon EC2 infrastructure

Ács Sándor <>
MTA SZTAKI

Kozlovszky Miklós <>
MTA SZTAKI

Marosi Attila Csaba <>
MTA SZTAKI

Balaton Zoltán <>
MTA SZTAKI

There is a need for a considerable amount of computational performance and storage for solving computational and data intensive problems. The researchers and the ordinary users may confront the fact that the performance of their computer is unsatisfying for them. In this case, the resource intensive applications should be executed on distributed systems. It may occur that the user does not have an opportunity to use (more) cluster(s) or the customer needs a service with SLA (Service Level Agreement). Therefore the scalable and elastic cloud infrastructure may provide a solution. Deployment of clusters to clouds is the simplest way to collect the computational power of the virtual machines. The presentation introduces a system, which can deploy automatically PBS (Portable Batch System) nodes to Amazon EC2 (Elastic Cloud Compute) or EC2 interface compatible clouds. With the provided system, it became possible to use the PBS technology without a local cluster. Other advantage of the designed system is the opportunity to extend local PBS cluster with nodes running in the cloud if the application requests more computational power. Additional benefit of the solution is that legacy PBS application can be used seamlessly with the technology. The presentation introduces the design and implementation of the system, includes the modification of the PBS cluster which makes it dynamically usable with VPN technology and SZTAKI 3G-Bridge. System tests had run successfully on Amazon infrastructure which proves that PBS resources may be used without the need to maintain a local cluster.