Forums Solaris and Karma mtlx2hda.py | How do I use it?

mtlx2hda.py | How do I use it?

   4699   20   2
User Avatar
Hypershader
Member
37 posts
Joined: March 2017
Offline
 
Hey,

I saw that there is a python script mentioned in the documentation called mtlx2hda.py to automatically convert mtlx node defs into Karma HDAs so I was wondering how do I use this script.

I see it supports a -p (—path) input for a mtlx file but I haven’t had any luck getting the script to work in any form (due to my lack of knowledge).

I would really appreciate it if someone could run me over how to use this script as it would be a very useful tool for me.

Best,
Rich
Edited by Hypershader - March 23, 2022 03:08:52
 Report
User Avatar
rafal
Staff
1455 posts
Joined: July 2005
Offline
 
This script looks for <nodedef> elements in the .mtlx file and creates an HDA based on its inputs and outputs.

This will give you some more info:
 hython mtlx2hda.py --help


Note, the -p is specifying a directory, not a file. So, if you have MySubdir/MyShaders.mtlx file that has MyShader MaterialX shader in it (with one or more 'ND_MyShader_*' definitions), then you can generate an HDA for MyShader running this:
hython mtlx2hda.py -o MyShader.hda -p MySubdir MyShader

Note, mtlx2hda.py is used to generate HDAs for the shaders in MaterialX standard library, so makes some assumptions about naming conventions, etc. TIt should work for most custom shader definitions too, but there is a very slight chance that you might need to tweak the code if your .mtlx does something the script chokes on.
Edited by rafal - March 24, 2022 11:31:52
 Report
User Avatar
Hypershader
Member
37 posts
Joined: March 2017
Offline
 
And do I run this in the Python shell?
Edited by Hypershader - March 24, 2022 11:43:51
 Report
User Avatar
rafal
Staff
1455 posts
Joined: July 2005
Offline
 
This script imports houmodule, so you may as well run Hython.

But if you really need Python shell, here is some more info about it:
https://www.sidefx.com/docs/houdini/hom/commandline.html [www.sidefx.com]
Edited by rafal - March 24, 2022 11:48:47
 Report
User Avatar
Hypershader
Member
37 posts
Joined: March 2017
Offline
 
Thanks! this was really useful! appreciate it!
 Report
User Avatar
spencer_luebbert
Member
9 posts
Joined: April 2021
Offline
 
I have a custom surface shader, defined in myCustomSurface.mtlx, I was able to use mtlx2hda.py to create the vex nodes.

I can drop these nodes into the vex graph, they appear to make valid materialx because I an right-click > Save on the node and export a .mtlx file.

It does not look like they are supported at all in the viewport (karma or GL). I tried putting myCustomSurface.mtlx inside of C:\Program Files\Side Effects Software\Houdini 19.0.589\houdini\materialx\libraries\bxdf thinking that maybe Houdini just need the mtlx source, but that didn't make a difference.

Are custom nodes created from mtlx2hda.py meant to be supported in karma or GL renderer?
 Report
User Avatar
spencer_luebbert
Member
9 posts
Joined: April 2021
Offline
 
I got an answer in the ASWF slack. If you run mtlx2karma.py after mtlx2hda.py it will generate the karma shaders for the new HDAs. This is necessary because Karma does not use MaterialX's code generation, it implements each standard node directly.
 Report
User Avatar
elovikov
Member
130 posts
Joined: June 2019
Offline
 
19.5 supports realtime renderer for mtlx

However, it doesn't seem to read definitions from extensions libraries (ie created via mtlx2hda).
I've tried to add library file to $HH/materialx/librariesbut still no luck (karma on the other hand works fine)

Is there any additional step like mtlx2karma for realtime renderer to register custom definitions?
Edited by elovikov - July 22, 2022 05:41:09
 Report
User Avatar
mbeaud
Member
9 posts
Joined: April 2024
Offline
 
elovikov
19.5 supports realtime renderer for mtlx

However, it doesn't seem to read definitions from extensions libraries (ie created via mtlx2hda).
I've tried to add library file to $HH/materialx/librariesbut still no luck (karma on the other hand works fine)

Is there any additional step like mtlx2karma for realtime renderer to register custom definitions?

Hi @elovikov,

Did you ever find a solution to this ?
In Houdini 20 I can see my custom materialx in KarmaCPU, but not in HoudiniGL or KarmaXPU.

Thanks !
 Report
User Avatar
MR SMITH
Member
32 posts
Joined: Aug. 2011
Offline
 
In Houdini 20 I can see my custom materialx in KarmaCPU, but not in HoudiniGL or KarmaXPU.

Are you sure it's the materialx that doesn't work in gl an xpu?
I also just tested some convertions, when it worked on cpu it also worked on xpu. Gl didnt work at all on my end.

There are still quite a lot of discrepancies between karma cpu xpu gl besides converted mtlx files.
One most obvious example is the default sphere. In cpu it seams to work with uv coords, on the xpu and gl side it doesn't.
 Report
User Avatar
mbeaud
Member
9 posts
Joined: April 2024
Offline
 
Yes, I figured there would still be quite a lot of discrepencies between the 3: even when using the houdini mtlx vop nodes, sometimes I don't get the same results (the mtlx_normalize node does not seem supported in houdiniGL for example, as soon as I add it to a graph, I get a black result in houdiniGL).

MR SMITH
Are you sure it's the materialx that doesn't work in gl an xpu?
I also just tested some convertions, when it worked on cpu it also worked on xpu. Gl didnt work at all on my end.

Would you have a materialx that worked in both CPU and XPU that I could try convert on my end, to see if my set up is correct ?
Edited by mbeaud - April 10, 2024 10:04:50
 Report
User Avatar
MR SMITH
Member
32 posts
Joined: Aug. 2011
Offline
 
mtlx_normalize does work in gl on my end. actually quite a lot of mtlx nodes already work in gl. The space transforms are really the limiting factor from what I found , which is a shame. If those are fixed it really should open the box for procedural shading and texturing in gl. There some other quirks like normalmaps that do work in gl, but you need to provide an tangent attrib.

Would advice to set reset viewport on an hotkey from the labs toolset and hit it often haha. That normalize might be an update issue for you aswel. GL really needs an reload render button like karma does so you don't loose all settings while hitting.


Rafal explained it to me in another thread with some files: https://www.sidefx.com/forum/topic/95316/#post-419183 [www.sidefx.com]
Try if this works for you, otherwise I can search for one I tried.
 Report
User Avatar
mbeaud
Member
9 posts
Joined: April 2024
Offline
 
Hi MR SMITH

Thanks for the details
While I install the labstoolset to figure out the situation with GL, I tried reproducing the steps shown in the other post you linked.
Sadly, for me it only work in KarmaCPU... I still get all black in Karma XPU. See attached.
Do you also have this warning/error log when starting Karma XPU ?

Thanks again

Attachments:
hda_in_CPU.png (1.3 MB)
subnet_in_XPU.png (1.2 MB)
hda_in_XPU.png (1.2 MB)
hda_in_XPU_logs.png (677.0 KB)

 Report
User Avatar
MR SMITH
Member
32 posts
Joined: Aug. 2011
Offline
 
don't use that sphere
Sorry nvm, that was the error i had because my mtlx needed texcoords. Which the sphere only had on cpu.
Edited by MR SMITH - April 11, 2024 11:50:04
 Report
User Avatar
MR SMITH
Member
32 posts
Joined: Aug. 2011
Offline
 
It's difficult to tell where it went wrong on your end. But it should be an digital asset after all the steps. I did't really tested the noise pattern, but should be similair results. Did you put in the nodedef stuff? or did you first made an hda, and than used the hda2mtlx.py ?
 Report
User Avatar
MR SMITH
Member
32 posts
Joined: Aug. 2011
Offline
 
Heres one of the test I did, worked in cpu and xpu but not on gl. This stuff is all new for me aswel, so my ability to provide help is limited haha.

Attachments:
circlepattern.hda (23.5 KB)
circlepattern_converted.hda (1.5 KB)
karmaNodeGraphs.json (15.1 KB)
circlepattern.mtlx (43.5 KB)
circlepattern.hiplc (840.4 KB)

 Report
User Avatar
mbeaud
Member
9 posts
Joined: April 2024
Offline
 
MR SMITH
It's difficult to tell where it went wrong on your end. But it should be an digital asset after all the steps. I did't really tested the noise pattern, but should be similair results. Did you put in the nodedef stuff? or did you first made an hda, and than used the hda2mtlx.py ?
Yes I converted it by hand, and it ended up looking exactly like the one in the archive from the other post.

MR SMITH
Heres one of the test I did, worked in cpu and xpu but not on gl. This stuff is all new for me aswel, so my ability to provide help is limited haha.
Thank you ! I'll try with those. It could just be an issue with my installation.
Do you have any logs when using Karma XPU in the logViewer pane ?

Thanks again for helping out
Edited by mbeaud - April 11, 2024 13:07:12
 Report
User Avatar
mbeaud
Member
9 posts
Joined: April 2024
Offline
 
Using your files, it works for me on both CPU and XPU !

Which of the 2 hda was made using mtlx2hda ?

Edit: I actually can only import the circle_pattern_converted.hda, that works in both CPU and GPU, but the circle_pattern.hda can be imported but not found in the vops nodes afterwards
Edited by mbeaud - April 11, 2024 15:16:43
 Report
User Avatar
MR SMITH
Member
32 posts
Joined: Aug. 2011
Offline
 
No logs, as it just worked.

I first tried that right click convert to mtlx method. But that needed quite a lot of hand adjustment with the nodedef stuff missing, plus all the extra nodes that needed to be removed.

Than I tried to make an .hda first as suggested by jsmack, used that hda2mtlx.py script on that created .hda file.
That way i didn't really need to adjust anything by hand.

So that circlepattern.mtlx file, is created from the circlepattern.hda by using hda2mtlx.py.

Than that created mtlx file is put through mtlx2hda and mtlx2karma to create circlepattern_converted.hda and the json file.


Curious what your reason is to convert, instead of using an hda?

For me it was the mainly the speed, but loosing gl im still not sure if its worth it. Also converting you really loose the ability to quickly dive in and add or change functionality.

I'm also not using it in production yet, really hope next major release will solve these type of issues. Having to convert and create blackboxes isn't the way imo.
 Report
User Avatar
mbeaud
Member
9 posts
Joined: April 2024
Offline
 
Ok, importing the circlepattern_converted.hda was working fine for me too in CPU and XPU.

What is the hda2mtlx.py script you are talking about ? I can't find it with the mtlx2hda.py and mtlx2karma.py. Is it just the right click, save to mtlx on a HDA ?

I'll try reproduce the same steps you did on my end then.

I'm bascially trying to get a working hda from a mtlx, because I'll also use this mtlx in other DCC, so I would want one source, and then convert it to be able to edit the inputs parameters of the mtlx in multiple DCCs.
So the idea would be, from one materialX graph (that could be created anywhere, Houdini, MaterialxGraphEditor, ...), get multiple "black boxes" with only the chosen inputs available for each DCCs. Because, I can't seem to find a way to only expose the inputs of the node graph when importing it as a materialx file. So using a HDA would make more sense.
 Report
  • Quick Links