Forums PDG/TOPs TOPs Partition with random index

TOPs Partition with random index

   644   4   1
User Avatar
tyler.britton
Member
4 posts
Joined: June 2024
Offline
 
Let's say I have two nodes, one with 5 work items (A B C D E) and another with 3 work items (A B C), how do I partition the work items of my first node with random indexes of work items from the second node? I can do a Partition by Attribute, but how would I find the maximum number of work items in the 2nd node from within the first node's pipe?

Attachments:
tops.png (77.0 KB)

 Report
User Avatar
alexwheezy
Member
296 posts
Joined: Jan. 2013
Offline
 
I think you can get a similar randomization pattern with Partition by Combination.
 Report
User Avatar
tyler.britton
Member
4 posts
Joined: June 2024
Offline
 
alexwheezy
I think you can get a similar randomization pattern with Partition by Combination.

Hmmm I tried that, however it does not always match partitions between the two nodes. Also the partitioning is very linear, and not random.
 Report
User Avatar
AslakKS
Member
185 posts
Joined: Feb. 2016
Offline
 
Partition by comparison can do this, a bit cumbersome to setup if you don't know the work item counts up front. But seems to work well enough

Attachments:
partition_random-select-pair.hip (234.7 KB)
partition_random.png (92.7 KB)

 Report
User Avatar
alexwheezy
Member
296 posts
Joined: Jan. 2013
Offline
 
tyler.britton
alexwheezy
I think you can get a similar randomization pattern with Partition by Combination.

Hmmm I tried that, however it does not always match partitions between the two nodes. Also the partitioning is very linear, and not random.
You could try using a Python Partitioner with a custom index that will have a random value for other items.

Attachments:
partition_by_custom_index.hipnc (156.8 KB)

 Report
  • Quick Links