To manage pipeline workflows, you can turn to Task Operators (TOPs) which are built using a technology called PDG which stands for Procedural Dependency Graph. Workflows created using PDG use TOP nodes that generate work items that distribute tasks to either your local computer or a larger compute farm.
The TOP network lets you determine how different work items depend on other work items and how each of them is contributing to the final output. This information is easy to visualize in the node graph which can be used to define how you want data to flow through your network. TOPs lets you build workflows that can be used to automate, analyze and scale your pipeline.
In this lesson, you will use TOP nodes to take a city map, create buildings for each city block and then grow this system to handle more complex buildings and larger city maps. Houdini artists will probably know how to do this in SOPs, but by using TOPs, you can learn the PDG workflow while creating a system that can be easily scaled to distribute multiple tasks to an external compute farm to process in parallel.
Lesson Goal
- To create a TOP (Task Operator) Network to build up a procedural city and then render it out.
- How to convert a city map image into geometry.
- How to set up a TOP network to save out the city blocks
- How to create Geometry in TOPS to build buildings on each city block.
- How to create a city core to make some buildings higher than others
- How to wedge the cityscape to try different locations for the city core
- How to wedge the use of different map images
- How to use TOPs to render out the city and compare wedges using an image mosaic
NOTE: This lesson uses Image Magick - make sure this app is installed on your computer.
Houdini Apprentice | Alternative Workflow
The setup in this tutorial doesn't work with Houdini Apprentice because the HDA Processor node introduced in Part Eight requires a Houdini Engine license that is not available to Apprentice users. To make this work, follow these alternative steps to avoid this licensing issue:
- Part Eight | Step 02 [pg 12] - Create a ROP Geometry TOP instead of the HDA processor. Double click to dive into it then add the streetgrid_maker.hda asset to the network and wire it into the output node. Don't worry about the incoming node - it isn't needed.
- Part Eight | Step 04 [pg 13] - set the Center parameter on the streetgrid_maker node using the expression shown in step 04. Go back up to to the TOPNET to recook.
CREATED BY
COMMENTS
Cookie428 5 years, 7 months ago |
When I get to the create city grid hda step. It seems to error out. I don't know what I am doing wrong.
rmagee 5 years, 7 months ago |
Can you save your work so far and zip up your project directory and send it to me (rmagee@sidefx.com) and I will take a look.
FranciscoBravo 5 years, 7 months ago |
Hi, nice Tutorial thank you very much. But I am having a problem at the end with the imagemagick node the next error appears every time I try to cook it:
Workitem imagemagick10_5 failed
Workitem imagemagick10_5 failed
Do you know what might be the problem?
rmagee 5 years, 7 months ago |
Do you have Image Magick installed? If so then please send me your project files zipped up and I can look into it for you.
aperry 5 years, 7 months ago |
Is anyone else having the issue where the create_buildings ROP geometry output doesn't write any files? I've fiddled around with the settings a lot but I'm not getting anything from this node in the geo directory. I've checked the output path is correct.
rmagee 5 years, 7 months ago |
Did you TURN OFF the Use External SOP option on that node? I had that issue when I was testing the tutorial - it says to do it in Part 4 step 1 but I missed that step anyway.
JoachimPaul 5 years, 7 months ago |
i guess it's part 01 on page 05
"Set Work Item Generation to Dynamic"
Thats what i missed.
LingLingLing 5 years, 4 months ago |
YES! That's the problem!! But PLEAAASE, anyone can enlighten me a bit why it will cause this problem? I'm sorry if it's a rather stupid question because i'm fresh new to Houdini and I have squeezed all my brain juice out trying to figure out why. Thank a lot in advance...
HoudiniUser2 5 years, 7 months ago |
Hi, thanks for the tutorial. Awesome technology, but ...
I am following this tutorial to the point of cooking the HDA Processor. The work item display turns red with no further error messages. Just saying: "failed 1" in the info box.
I tried the same thing with a new file and an example from the side/fx website ("example_top_hdaprocessor")and got the same result. The node cooked,, the work items display became green as normal and short before finishing changed to red.
Is the Apprentice version of Houdini unable to work with HDA's in the TOP HDA Processor?
rmagee 5 years, 7 months ago |
There are lots of little gotchas when working through PDG. I would need to see your network to see how to fix things. I know that all the required steps are in the tutorial but sometimes it is easy to skip one of them.
HoudiniUser2 5 years, 7 months ago |
The tutorial should be compatibel with Houdini Apprentice but sadly it is not.
Here is the error log:
['/Applications/Houdini/Houdini17.5.173/Frameworks/Houdini.framework/Versions/Current/Resources/bin/HDAProcessor', '/Users/Uli/Desktop/tops_lesson_start/pdgtemp/54028', '/Users/Uli/Desktop/tops_lesson_start', 'hdaprocessor10_3']
Returned error code 1:
hda name: /Users/Uli/Desktop/tops_lesson_start/hda/Uli_test2.hdanc
operatortype: Sop/Uli_test2
Could not load /Users/Uli/Desktop/tops_lesson_start/pdgtemp/54028/Uli_test2.hdanc
HAPI Error: No Valid Licenses Found
- HAPI_LICENSE_MODE unset or is set to 'default' meaning Houdini Engine will first try to acquire a Houdini Engine license and if one is not found it will try to acquire an interactive Houdini license (Houdini Core then Houdini FX).
- No Houdini Engine license found.
- No Houdini license.
- Found a Houdini FX (Houdini Master) license but it is not the required commercial variant.
- No Houdini Engine Indie license found.
- No Houdini Indie license.
Mjag007 5 years, 7 months ago |
I'm also running into an issue. It will cook fine and the small city looks great. But when you go back and adjust the height settings and recook. You end up with one messed up building.
rmagee 5 years, 7 months ago |
Please check your viewer path - if it is pointing to inside the ropgeometry top then things will look strange - go back to the object level and click on a work item - should fix things.
rmagee 5 years, 7 months ago |
It is supposed to work in Apprentice. It should be looking for the Apprentice license. I have forwarded your message to R&D for review.
3clipse 5 years, 7 months ago |
Hi Robert
Tks for the tut very nice so far... clarification required though.
In Part 6 - 03 you say to edit the attributecreate node but the associated image shows the change being made ( + @Cd.g*30) in the building _height node?
Which is it...
Cheers
Greg
rmagee 5 years, 7 months ago |
It could go both places but the attributecreaye node is better for steps down the line. Must be an old screen grab when I tried the other way.
3clipse 5 years, 7 months ago |
k tks... I went the other way so will go back and change that... :)
3clipse 5 years, 7 months ago |
Hi Robert
Another question in Part 8 when converting to hda. I get an error node to convert deleted! On the forums I discovered a post for this error message where Michael Goldfarb advised to shift c (subnet) and then create the HDA from there... I am familiar with creating digital assets this way do you see any problems doing that here?
3clipse 5 years, 7 months ago |
Yeah that didn't work (and neither have another half dozen attempts of various things) I am almost certain that it is a pathing issue... anyway I will keep trying
Luiz_Lobato 5 years, 7 months ago |
It's needed to press shift+C to convert the selected nodes to a subnet first. After this, you can create the HDA.
dothanhphongla 5 years, 1 month ago |
thanks for your help, can you take photo or write how to do. i have the same problem. tx
kenxu 5 years, 7 months ago |
Houdini Engine is not available for Apprentice - that is why HDA Processors will not work in Apprentice. You could use the ROP Geometry node to achieve basically the same thing, however.
Valent 5 years, 7 months ago |
Hi, could you please explain what setting should I use in the ROP Geometry to be able to use it instead of HDA Processor?
rmagee 5 years, 1 month ago |
I have added some notes above for Apprentice users to bypass the issue.
OdFotan 4 years, 3 months ago |
This is not the case anymore,
the node is in Apprentice? Does that mean it works also?
FrancisGagne 5 years, 7 months ago |
Hi Robert,
I cook all the building & i try to figured it out why the height variation doesn't distribute randomly.
All my building have the same height.
Thank you, Francis!
Luiz_Lobato 5 years, 7 months ago |
It's needed to press shift+C to convert the selected nodes to a subnet first. After this, you can create the HDA.
twelveplusplus 5 years, 7 months ago |
Thanks, Robert, for continuing to release quality printed/pdf material! Some of us get way more value from print than video!
I would love more advanced material, too. A Vex wrangle/python cookbook would be extremely useful! Much easier to follow than a youtube video, etc.
Also, a monthly Houdini Zine would be ***super awesome***! Just a couple of pages with tips and tricks. Maybe a bit of coverage for featured artwork, games, animation, etc.
A little zine might be a good way to keep people up to speed with current trends, too...
Development trends change quickly. A regular zine could be a way for literate Houdini users to stay hip.
...I'll see my self out, now. :D
twelveplusplus 5 years, 7 months ago |
oh, and love image magick, too! All the best picture processing algorithms for freeee!! Besides which, it makes a mean animated gif!
kumpa 5 years, 7 months ago |
Thank you for reconsidering making tutorials in PDF as video tutorials are often too stretched and narrator might not be everyone's cup of tea.
nastyatupitsyna 5 years, 7 months ago |
Hi, can you please send the file scene to the nastyatupitsyna@gmail.com?
borbs727 5 years, 7 months ago |
Part 5-2 isn't merging the entire city for me. Sending scene file.
borbs727 5 years, 7 months ago |
I should elaborate that it is creating a merged geo file, its just not displaying in my viewport. I still only see individual buildings.
borbs727 5 years, 7 months ago |
Never mind! I didn't click the work item in the import node.
Mjag007 5 years, 7 months ago |
Its working now, No idea what I did wrong
2b65d3d0a8c04862 5 years, 7 months ago |
Hello , I just want to know where I can download the citymap like this?
2b65d3d0a8c04862 5 years, 7 months ago |
I mean the citygrid
2b65d3d0a8c04862 5 years, 7 months ago |
Hello,where can I download the citygrid,thanks
rmagee 5 years, 7 months ago |
The download scene files button above has the city grid in the Tex folder.
thirdworldman 5 years, 7 months ago |
After cooking the building_merge node, I'm not seeing the entire city -- only single buildings. Any help would be appreciated!
thirdworldman 5 years, 7 months ago |
Nevermind - I'm an idiot!
KieranLatham 4 years, 1 month ago |
What was your solution? I cannot find the option being referred to.
KieranLatham 4 years, 1 month ago |
Gonna have to quote you on this one mate: "Nevermind - I'm an idiot!"
missed the part about clicking the work item...
thirdworldman 5 years, 7 months ago |
Ok, I'm now stuck on city center adjustment based on the sphere. In the attributestransfer node, you wrote to "set primitives to Cd". Is that the correct value? I got an error when I typed this into the primitives field, and then changed it to `@Cd.` Is that correct?
rmagee 5 years, 7 months ago |
Make sure both the color nodes are pointing to primitives. One of them might still be assigning color to points.
thirdworldman 5 years, 7 months ago |
Thank you for your response. Unfortunately, the sphere still isn't affecting the height of buildings within it's radius. Any other thoughts?
rmagee 5 years, 7 months ago |
I'd have to see the scene file - I added my email earlier in this thread - if you wrap it up then I can take a look.
thirdworldman 5 years, 7 months ago |
Got it! Nevermind. I keep forgetting to hit the green circle!
nickryan 5 years, 7 months ago |
I have run into a problem when trying to read the '@pdg_output' on the work_item_viewer node (PC) - I get an error -
Error
Unable to read file "'@pdg_output'".
Error
GeometryIO[hjson]: Unable to open file ''@pdg_output''
GeometryIO[hclassic]: Unable to open file ''@pdg_output''
echoshen 5 years, 4 months ago |
so how do you fix this problem final
echoshen 5 years, 4 months ago |
OK...I Know.the wrong quotes
EmmanuelHoudini 5 years, 6 months ago |
Hi, nice tutorial but i have a problem to cook a "create_buildings" when i cook all the point are red and not green. Another problem, when i see the buildings "attributecreate1" all buildings appear on -Y. I don't know what's wrong. Thank you.
nickryan 5 years, 6 months ago |
Hi,
I am very new to Houdini and probably missing something fundamentally simple in the setup, but have retraced steps meticulously to no avail - stumbling at pointing the file node to '@pdg_output' still - It seems to be a path or Drive letter thing - it is fine if I point it directly to the bgeo files but not by expression....
Any ideas?
Thanks,
rmagee 5 years, 6 months ago |
Be sure to use back ticks not single quotes
nickryan 5 years, 6 months ago |
Thanks Robert....I was indeed missing that fundamental key stroke - I never knew it even existed!!!
cookieclarke 5 years, 6 months ago |
Hi Robert.
Awesome lesson and format. Thanks heaps.
Im hoping you could shed some light on why I lose my city_01_geometryimport file sequence in the file in "work_item_viewer", every time I close and re open my scene file?
Actually it seems that when ever I dive into ROP geometry output I lose my city completely. I managed to finish part 5 successfully but went into change the color of windows, then i lost my city. Is it a matter of cooking again?
cheers
echoshen 5 years, 4 months ago |
I have a same problem with you ,so how do fix it
rmagee 5 years, 4 months ago |
Yes you need to recook after you quit and reload the scene file - it should find the files on disk and load them instead of rebuilding for a fast cook but I am not sure based on the settings we used in the lesson.
Andr 5 years, 4 months ago |
Regarding part 6 and note about "DIRTY & CLEAN WORK ITEMS: As you make changes to the downtown core, you will see that some nodes become dirty and have to be recooked while others are fine the way they are. "
When I shift-G at point number 4, ALL my work items had to be recooked. Is this supposed to happen?
Andr 5 years, 4 months ago |
Similarly in part7, after I create the new 'attribute create' node with @primitiveoffset==62, the topnet recooked all the 73 work items.
Then I changed @primiteveoffset==61, and again all the work items are re-cooked again.
I expected that only the 61 and 62 work items would be reworked.
Hamad 5 years, 4 months ago |
I have issue with part 2_6/Users/hamadalateeqi/Desktop/Screen Shot 2019-06-30 at 7.02.16 AM.png
Paris Hall 5 years, 4 months ago |
Hey there, Robert!
Such a great tutorial. I've learned so much from a short and sweet document.
Truly appreciate the straightforwardness!
Thanks again!
maskn12 5 years, 4 months ago |
I can´t go to the next step cause of the apprentice HDA license, is there anyway to fix this?
chriskopotic 5 years, 2 months ago |
I am also stuck on step 8 and unable to run the HDA processor due to the licence. Is there another way of doing this?
dubstah 5 years, 2 months ago |
same here
rmagee 5 years, 1 month ago |
I have added some notes above for Apprentice users to bypass the issue.
RonanCroyal 5 years, 1 month ago |
With the "building_merge" geometry import node (part05-2) i got an error "empty or missing file path""invalid input geometry". I've redone every steps till their, four times, and always the same issue.
333hiram 5 years ago |
Can you pls make a video Tutorial ? that would be more helpful for us (I m newbee)
thank you sir
King Regards
rmagee 5 years ago |
This video includes steps taken from this tutorial - https://vimeo.com/325768130
Patrick Woo 4 years, 8 months ago |
This is great, Robert! thank you for the link!
blackdogpapa 4 years, 7 months ago |
Is that work on Houdini 18? I got crash everytime on part 5, when i cook the "building_merge" node. I try whole day and do it again and agin, it alway crash. Pls suggest the solution.
rmagee 4 years, 7 months ago |
If Houdini is crashing then the best solution is to report a bug- you can do that from the Support menu above.
LG 4 years, 3 months ago |
An observation on page 6 step 04 where it's stated: ... turn ON the front group and name it building_top. Should this be OutputBack checked instead and Back Group building_top? At least that's the only way I can isolate the roof inset down the line when i get to the roof edge PolyExtrude node.
LG 4 years, 3 months ago |
After seeing the Vimeo version, I probably missed a step . So starting over seems a better move at my side.
rmagee 4 years, 3 months ago |
That is probably a good idea - there are lots of little gotchas in this lesson if an option here or there isn't set up properly. Glad the video helped out.
LG 4 years, 3 months ago |
Starting over and making sure I was precise the second time around helped indeed and completed it without any problems. Video helped out perfectly!
KieranLatham 4 years, 1 month ago |
Hi Robert.
In Part 12, step 5, you go from the 3600x3600 street grid textrue, to a large scaler map, in whichyou advise we up the scale and and "centre widge" attribute.
Was this a custom image you used that we could create our one of, or did you do something in a TOP node to input the four different maps procedurally?
Great tutorial by the way covers a lot of ground and provides a lot of information.
Cheers.
Kieran.
hetareba 3 years, 10 months ago |
Mr. Rob, thank you for this wonderful manual.
However, in my environment, even if I worked according to the material, a bug occurred in Part 8.
Content of the bug
part 8-1
I couldn't make "streetgrid_maker" without packcking nodes into subnet, so I packed those nodes first and then made "streetgrid_maker."
part8-2
I couldn't do "Dirty and Cook" with "streetgrid_maker."
(error message : ['C:/PROGRA~1/SIDEEF~1/HOUDIN~1.460/bin/HDAProcessor', 'C:/Users/HETAREBA/AppData/Local/Temp/houdini_temp/city_01/pdgtemp/39732', 'D:/Documents/Files_Houdini/tops_lesson_start', 'make_citygrid0_4']
Returned error code 0x1:)
My working environment
Houdini: apprentice 17.5.460
Ryoma0z23 3 years, 9 months ago |
Hi all I am super new to Houdini and really appreciate this tutorial but I'm having a bit of trouble with part 05 step 01 & 02. I created the Wait for All TOP node but for some reason i am missing some options from the video and the pdf. There are no options for Work Item Generation to set to Dynamic and no option to turn ON the
Merge Input Geometry option. I'm wondering if this is because I am using the apprentice version? do I need the licensed version to be able to continue this tutorial?
It would be greatly appreciated if someone can point me in the right direction.
Thank you.
0z.
Houdini 18.5.462 apprentice non license.
rmagee 3 years, 8 months ago |
I do not think that this is an Apprentice issue. PDG has evolved since this tutorial was first published - I am planning to review it and publish an update - no timetable at the moment but I will run through it as soon as possible.
Junru_Tao 3 years, 8 months ago |
Hi, ROBERT MAGEE,
I'm currently trying things out with Houdini 18.5. It did not work work step 5.
The geometry import node will show error when the mode is automatic. it works with new option `All upstream Items are Cooked` and somehow it will generate the buildings with the same height as I click on the item. What when wrong with this version?
Cheers, please help
m_vlasov 3 years, 5 months ago |
I was stuck on step 02 in part five, but changing these settings in Geometry Import node worked for me:
- Generate When set to 'All Upstream Items are Cooked'
- Merge Operation set to 'Import and Merge All Geometry'
- Create Merge Attribute set to 'Primitive'
looksoon_register 3 years ago |
Yes, you are right. I stuck here too. The first option you mentioned is necessary, the other two are optional for me.
ancientkult1337 2 years, 11 months ago |
Thank you! This tut was so well made, bravo! ; )
NoCode 2 years, 9 months ago |
Hi,
Thanks for this good tutorial on how to build a procedural city.
I'm wondering if this still works in Houdini 19.x.
Everything was working fine up till I changed from 'WaitForAll' to 'PartitionByAttribute'
-T
rmagee 2 years, 9 months ago |
The tutorial was updated for H19. Please check again - maybe there is a setting that is needed to make this work.
NoCode 2 years, 9 months ago |
Hi,
Found my error :)
Thanks again for the tutorial
agastyafx 2 years, 7 months ago |
How did you fixed it?
agastyafx 2 years, 7 months ago |
Nevermind... Its just it City Merge is displayed in OBJ Level... :)
ywlassassin 2 years, 7 months ago |
Hello Robert, I have a problem with the "Partionby index "top node which seems to not haveing the use "use dynamic parenting" option.Idk if this is a switch in H19.My Houdini version is 18.5!
Dustar 2 years, 6 months ago |
Same Problem here. That "Use Dynamic Parenting" got me confused for a second, but the result was correct anyway so I just ignored it.
MatwGraphics 2 years, 5 months ago |
I seem to be stuck on this one, using Houdini 19.0.589. I am good untill Part Four Step 3, where it states to create a polyextrude nude and put in @base_height + @height_variation into the distnace parameter (the two float attributes created in the attribute create TOP in section 3 steps 1 &2).
When I do this, I get an error which states: Unable to evaluate expression (File "<stdin>", line 1 @base_height + @height_variation Syntax errorL invalid syntax (/obj/topnet1/create_buildings/s/s/building_height/dist).
I've tried encasing the expressing in backticks or quotes or parethesis, tried just tying in the attributes directly without the @, tried to cook the node instead of just clicking generate like it states in the previous step. No luck...
Any help is appreciated!
youyuan7936 2 years, 2 months ago |
Hi, I'm sorry I'm so late, can you upload this scene file? This one doesn't work at the moment.
jimenagrueso 1 year, 11 months ago |
Hello Robert,
Thank you for this tutorial!
I have sent you my file because I had troubles at the end with the ROP mantra node. It gives me a "failed" message, but I couldn't find the reason.
Thank you again
Please log in to leave a comment.