Skip to content
GitLab
Projects
Groups
Snippets
Help
Loading...
Help
What's new
10
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Open sidebar
Simone Rossi
main
Commits
53013033
Commit
53013033
authored
Dec 21, 2018
by
Simone Rossi
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
[fix] Sleep always after job submission
parent
91679afa
Pipeline
#11295
failed with stages
in 0 seconds
Changes
1
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
17 additions
and
5 deletions
+17
-5
scripts/grid_queue.py
scripts/grid_queue.py
+17
-5
No files found.
scripts/grid_queue.py
View file @
53013033
...
...
@@ -26,6 +26,7 @@ from itertools import product
import
operator
import
time
from
collections
import
Iterable
,
Mapping
from
typing
import
List
,
Dict
import
argparse
import
os
,
sys
...
...
@@ -177,9 +178,18 @@ class ParameterGrid(object):
raise
IndexError
(
'ParameterGrid index out of range'
)
def
get_all_jobs
(
bin
,
executable
,
param_grid
)
->
list
:
def
get_all_jobs
(
bin
:
str
,
executable
:
str
,
param_grid
:
Dict
)
->
list
:
"""
Return a list of jobs for all combination of the parameters
:param bin:
:param executable:
:param param_grid:
:return:
"""
jobs
=
[]
for
combination
in
ParameterGrid
(
param_grid
):
for
combination
in
ParameterGrid
(
param_grid
):
# type: Dict
job
=
bin
+
' '
+
executable
+
' '
for
key
,
value
in
combination
.
items
():
job
+=
'--%s=%s '
%
(
key
,
value
)
...
...
@@ -284,11 +294,12 @@ def main():
number_new_jobs
*
zapp_memory
/
(
1
<<
30
)))
# Number of new jobs to schedule need to satisfy all three constrains (concurrent ex, cores and memory)
count_next_jobs_to_schedule
=
min
(
[
count_next_jobs_to_schedule
=
[
quota_jobs
-
used_jobs
,
int
((
quota_cores
-
used_cores
)
/
zapp_cores
),
int
((
quota_memory
-
used_memory
)
/
zapp_memory
)
])
]
count_next_jobs_to_schedule
=
min
(
count_next_jobs_to_schedule
)
if
count_next_jobs_to_schedule
>
0
:
logger
.
info
(
'Scheduling %d new jobs'
%
count_next_jobs_to_schedule
)
...
...
@@ -308,7 +319,8 @@ def main():
else
:
logger
.
warning
(
'Quota exceeded. No jobs can be scheduled now! Sleeping'
)
time
.
sleep
(
15
)
time
.
sleep
(
15
)
logger
.
info
(
'All jobs have been submitted successfully! Exiting'
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment