Robert Vinluan

Hi All,

FYI, starting in tomorrow's Houdini 20.5.452 build, the Deadline Scheduler TOP node will behave as it did pre-Houdini 20.5 and will schedule work items as tasks by default. There will also be a new Schedule Work Items As Jobstoggle parameter on the Deadline Scheduler that when checked on, will switch the scheduling mode back to scheduling work items as separate jobs. The toggle parameter provides a workaround to anyone that experiences dropped tasks on Deadline as a result of high concurrent activity on the farm.

Cheers,
Rob

While I'm on here, I'll provide an update to the Deadline scheduling changes I mentioned earlier in this forum thread. I've added a new toggle parameter to the Deadline Scheduler TOP node to control whether you want pre-Houdini 20.5 scheduling of one-job-of-many-tasks or the new Houdini 20.5 scheduling of one-batch-of-many-jobs. The changes are currently in an in-house development build and are undergoing testing. I'm hoping to roll the changes into a Houdini 20.5 build very soon.

With pre-Houdini 20.5 scheduling, the Concurrent Tasksparameter will work once again.

Cheers,
Rob

lavrenovlad
Hey @rvinluan,

Could I ask you a question regarding deadlinescheduler speeds compared to localscheduler, since you seem to be the right person to ask this type a question.

I am comparing a very fast axiom sim that takes 5 sec to simulate for 240 frames, with localscheduler via TOPs the sim takes around 10-20 seconds while the same type of sim takes 1.5 minutes via deadlinescheduler. The difference is that local scheduler prepares 20-30 tasks on the fly (at least it looks like this in the UI), while deadline scheduler always has 1-2 item overhead and my 5 sec sim turns into 1 minute and 40 second one to be precise, is this supposed to work like this? I don't use "Cook Frames As Single Work Item" for the sim here on purpose as an example of speed between the schedulers. I am just wondering if it's supposed to be like this?
I am on H20.5.410 and Deadline 10.4 btw.


Thanks in advance!

As @HristoVelev mentioned, each task boots up a Houdini process, which adds overhead to the overall time and can be relatively significant for short tasks. You can batch tasks/frames together or use PDG Services (https://www.sidefx.com/docs/houdini/tops/services.html) to help reduce the overhead attributed to starting up processes.

In general, I would expect Deadline scheduling to be slower than Local scheduling. There is overhead with submitting jobs, waiting for Deadline to provision and assign worker nodes to tasks, and then waiting for the workers to pick up the tasks and execute the task commands. I can't really say how much slower, it varies, but it's definitely slower.

Judging by your attached screenshots, it looks like you may only have 1-2 Deadline workers on the farm compared to many "slots" available when performing local scheduling. Note that the number of concurrent tasks is determined by the Total Slotsparameter on the Local Scheduler TOP node (https://www.sidefx.com/docs/houdini/nodes/top/localscheduler.html#maxprocsmenu) for local scheduling and by the number of available Deadline workers for Deadline scheduling. There is a Concurrent Tasksparameter on the Deadline Scheduler TOP node (https://www.sidefx.com/docs/houdini/nodes/top/deadlinescheduler.html#deadline_concurrenttasks) that you can set to control or increase the number of tasks running concurrently on your workers but it's currently broken at the moment (I'm working on a fix).

Cheers,
Rob