The discussion forums in the XSEDE User Portal are for users to share experiences, questions, and comments with other users and XSEDE staff. Visitors are welcome to browse and search, but you must login to contribute to the forums. While XSEDE staff monitor the lists, XSEDE does not guarantee that questions will be answered. Please note that the forums are not a replacement for formal support or bug reporting procedures through the XSEDE Help Desk. You must be logged in to post to the user forums.

« Back to Stampede Forum

ibrun command line inputs / python-managed sequential parallel jobs

Combination View Flat View Tree View
Threads [ Previous | Next ]
toggle
I have a python script (which works well on other systems) that launches a number of parallel jobs sequentially - any recommendations on launching slurm jobs from within python scripts? Especially regarding doing it in the right way for the stampede system?

It is an optimization process, where the optimizer is serial and the function calls are parallel, so it's not feasible to launch each parallel simulation manually.

I have tried 'ibrun -n %i %s' where the inputs are provided later in the script; however this produced an error about not providing a -o command.

Has anyone tried something similar to this, and/or can recommend the correct syntax?
This process previously worked with 'srun -n %i %s' on a different system, so I think it is just the ibrun inputs I need help with.

RE: ibrun command line inputs / python-managed sequential parallel jobs
Answer
1/11/16 11:04 PM as a reply to Heather Louise Kline.
Update:
'ibrun -o 0 -n %i %s'
solves this problem.
where %i and %s are the number of processors and the command string provided later. The -o option specifies the offset for the processors, and in this context the offset is always 0.

I will be updating our open-source code (SU2) so that this is used automatically on TACC machines in case someone else needs to use our code on XSEDE.