Modify

Opened 7 years ago

Last modified 7 years ago

#536 new enhancement

Don't overload machines while building

Reported by: Erik Schnetter Owned by: Erik Schnetter
Priority: optional Milestone:
Component: SimFactory Version:
Keywords: Cc:

Description

Certain machines are easily overloaded by building Cactus; building multiple configurations at the same time is not possible there. For example, LONI does not have sufficient compiler licences, or Orca (Sharcnet) does not allow sufficiently many user processes. On LONI, things will be very slow (also for other users), on Orca building will fail with strange error messages.

I suggest a mechanism that automatically serialises building Cactus configuration on certain sets of machines. Ideally, one would extend the "make -j" mechanism for this; practically, an implementation via locks may be simpler. Note that this has to work for sets of machines, not just single machines.

Attachments (0)

Change History (4)

comment:1 Changed 7 years ago by Ian Hinder

GNU make has a -l option to specify a load threshold above which further tasks will not be spawned. In principle this would solve the problem. In practice I have observed it to severely underutilise the machine, with the load on the machine never approaching what I set it to.

From man make:

-l [load], --load-average[=load]

Specifies that no new jobs (commands) should be started if there are others jobs running and the load average is at
least load (a floating-point number). With no argument, removes a previous load limit.

comment:2 Changed 7 years ago by Erik Schnetter

This works only on a single system; there is no way to ensure that builds on several separate systems are serialised. It also seems to take the whole system load into account, not only the processes started by one user.

comment:3 Changed 7 years ago by Ian Hinder

Why would you want to ensure that builds on several separate systems are serialised?

Why would you want to limit the number of processes started by one user if the whole system can accommodate more?

comment:4 Changed 7 years ago by Erik Schnetter

LONI provides a certain number of compiler licences shared across several systems. Building on several systems simultaneously slows down all builds, and also slows down builds of other users.

Orca does not allow too many processes per user. If one user tries to have more processes, things will fail with strange symptoms.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as new The owner will remain Erik Schnetter.
Next status will be 'review'.
as The resolution will be set.
to The owner will be changed from Erik Schnetter to the specified user.
Next status will be 'confirmed'.
The owner will be changed from Erik Schnetter to anonymous.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.