Loading README.rst +21 −0 Original line number Diff line number Diff line Loading @@ -101,3 +101,24 @@ The source, `jobtypes.py`, contains 4 basic types of jobs for now, messages if you get the names wrong. Contact me if the error messages are not helpful enough to solve your problems, or if you have new ideas on functionality. Substitutions ------------- Before `make.py` runs jobs, it generates jobscripts from your yaml. The `environment`, `script` and `retry_script` variables of each job go through substitution at creation time for each script. In the case of a `Restartable` jobtype, the `retry_script` and `complete` tests are also substituted. Specifically, the python format() function is invoked. This means ordinary brackets, `{}`, need to be written as double-brackets, `{{}}`. Also, it means you can use the variables defined in the yaml itself in the script. In addition to those variables, you also have access to everything defined the `reqs` dictionary of jobtypes.py. Finally, every project also has: proj = the project ID input to `make.py` jobname = the job name itself (top-level header for the job) jobtypes.py +1 −1 Original line number Diff line number Diff line Loading @@ -43,7 +43,7 @@ class Simple: # Fill-in from kws and type-check class vars for var, cls in self.reqs.items(): if var in kws: # initialize from yaml data val = kws[var] val = kws[var].copy() if isinstance(val, dict) and hasattr(self, var) \ and isinstance(getattr(self,var), dict): # merge together dict_merge( getattr(self,var), val ) # recursively merge parameter hierarchies Loading make.py +2 −0 Original line number Diff line number Diff line Loading @@ -12,6 +12,8 @@ def subst_chain(jobtype, args, types): job0 = types[jobtype] if issubclass(job0, types['Chain']): for i,d in enumerate(args['chain']): if isinstance(d, tuple): # already processed. continue assert len(d) == 1 for type1, args1 in d.items(): pass Loading Loading
README.rst +21 −0 Original line number Diff line number Diff line Loading @@ -101,3 +101,24 @@ The source, `jobtypes.py`, contains 4 basic types of jobs for now, messages if you get the names wrong. Contact me if the error messages are not helpful enough to solve your problems, or if you have new ideas on functionality. Substitutions ------------- Before `make.py` runs jobs, it generates jobscripts from your yaml. The `environment`, `script` and `retry_script` variables of each job go through substitution at creation time for each script. In the case of a `Restartable` jobtype, the `retry_script` and `complete` tests are also substituted. Specifically, the python format() function is invoked. This means ordinary brackets, `{}`, need to be written as double-brackets, `{{}}`. Also, it means you can use the variables defined in the yaml itself in the script. In addition to those variables, you also have access to everything defined the `reqs` dictionary of jobtypes.py. Finally, every project also has: proj = the project ID input to `make.py` jobname = the job name itself (top-level header for the job)
jobtypes.py +1 −1 Original line number Diff line number Diff line Loading @@ -43,7 +43,7 @@ class Simple: # Fill-in from kws and type-check class vars for var, cls in self.reqs.items(): if var in kws: # initialize from yaml data val = kws[var] val = kws[var].copy() if isinstance(val, dict) and hasattr(self, var) \ and isinstance(getattr(self,var), dict): # merge together dict_merge( getattr(self,var), val ) # recursively merge parameter hierarchies Loading
make.py +2 −0 Original line number Diff line number Diff line Loading @@ -12,6 +12,8 @@ def subst_chain(jobtype, args, types): job0 = types[jobtype] if issubclass(job0, types['Chain']): for i,d in enumerate(args['chain']): if isinstance(d, tuple): # already processed. continue assert len(d) == 1 for type1, args1 in d.items(): pass Loading