Just bumping this - pretty disappointed to see that one of the sickest 20.5 features completely skip over the macOS release. Without Vulkan it makes the new cops features really hard to use on macOS.

be careful, or they'll close this thread soon

They might but it would be really great to know what specifically is the technical barrier to Vulkan given that you can now build Vulkan for Mac. It may be as simple as building these libraries on your PC...

via SideFX the Vulkan SwiftShader libraries can be built for Windows, Linux, and macOS. [github.com]

I have been thinking about what makes me react strongly to this and I know as a paying customer it is frustrating to not have keynote features or be slighted in a serious way...

What should really be said to SideFX about focusing on MacOS and unified architectures is that we live in a world of finite resources and the technologies we consume also have other downstream costs. Attention should really be paid to have our machines work smarter, not harder. GPU and memory requirements should be going down not up based on current scientific technology trends.


For example, let's take the MPM solver for instance. Great technology, poor implementation because of the compute needed for the implementation. Physics simulations are some of the most costly computations we do and if we can have neural surrogate models and that is made into tools widely available, then SideFX would save the world a large amount of environmental impact.

I was expecting at least a standard implementation of MPMNet but something with more physics-based because its Houdini, right?

MPMNet: A Data-driven MPM Framework for Dynamic Fluid-solid Interaction [ar5iv.labs.arxiv.org]

The composition of small surrogate networks can out perform the brute MPM approach and does not wastes everyones time.

Simple Chains: Fast CPU Neural Networks | Chris Elrod | JuliaCon 2022
[www.youtube.com]

Automatic Differentiation and SciML: What Can Go Wrong | Chris Rackauckas | JuliaHEP 2023 [www.youtube.com] (starts slow but after predicting epidemics, it is on to digital twins, etc.)

Distributed personal computing using consumer accessible ARM(or future neuromorphic analog accelerators)chips and lower power PCs are necessary for us to sustainably create. I hope there is a shift toward a ecological lens of technology development.

Anthropologist Steven Gonzalez Monserrate draws on five years of research and ethnographic fieldwork in server farms to illustrate some of the diverse environmental impacts of data storage. [computing.mit.edu]

"Why so much energy? Beyond cooling, the energy requirements of data centers are vast. To meet the pledge to customers that their data and cloud services will be available anytime, anywhere, data centers are designed to be hyper-redundant: If one system fails, another is ready to take its place at a moment’s notice, to prevent a disruption in user experiences. Like Tom’s air conditioners idling in a low-power state, ready to rev up when things get too hot, the data center is a Russian doll of redundancies: redundant power systems like diesel generators, redundant servers ready to take over computational processes should others become unexpectedly unavailable, and so forth. In some cases, only 6 to 12 percent of energy consumed is devoted to active computational processes. The remainder is allocated to cooling and maintaining chains upon chains of redundant fail-safes to prevent costly downtime."

"The Cloud may be a carbonivore, but as the example of “The Mouth” shows, the Cloud is also quite thirsty. Like a pasture, server farms are irrigated. In many data centers today, chilled water is piped through the latticework of server racks to more efficiently cool the facility, liquid being a superior convective agent than air."

PHENOMDESIGN
I was expecting at least a standard implementation of MPMNet but something with more physics-based because its Houdini, right?

MPMNet: A Data-driven MPM Framework for Dynamic Fluid-solid Interaction [ar5iv.labs.arxiv.org]

...

These things are quite cutting edge. I'm sorry, but it's very unrealistic to expect Houdini 20.5 to come with an algorithm that was only published in late 2023 in academic context. I'm willing to bet money on that H20.5's roadmap was already settled when H20 was released. A software as complicate as Houdini will never be as "agile" as you expect.

We care greatly about macOS and we have spent, and continue to spend, considerable time optimizing Houdini for the architecture.

macOS does not support Vulkan natively - you instead need to use a translation layer to Metal such as MoltenVK. Adding a layer of indirection is bound to create more bugs, so rather than release something unfinished, we held back so that we can do it right.

SwiftShader is not an option for the viewport as it's a CPU implementation.

johnmather
We care greatly about macOS and we have spent, and continue to spend, considerable time optimizing Houdini for the architecture.

macOS does not support Vulkan natively - you instead need to use a translation layer to Metal such as MoltenVK. Adding a layer of indirection is bound to create more bugs, so rather than release something unfinished, we held back so that we can do it right.

SwiftShader is not an option for the viewport as it's a CPU implementation.

That's great but meanwhile on macOS, we can't use packed primitives, Copernicus, in solaris crowd with textures are not usable ... and so on 🥺

For my job (photographer) I have to switched to Mac, I was for many years an Houdini users, but now I do give up 🫤

johnmather
We care greatly about macOS and we have spent, and continue to spend, considerable time optimizing Houdini for the architecture.

macOS does not support Vulkan natively - you instead need to use a translation layer to Metal such as MoltenVK. Adding a layer of indirection is bound to create more bugs, so rather than release something unfinished, we held back so that we can do it right.

SwiftShader is not an option for the viewport as it's a CPU implementation.

Can you share how soon macOS Vulkan support is planned to be introduced? Can we expect a future 20.5 build to support Vulkan on macOS, or should we expect to wait at least till 21.x?

raincole

PHENOMDESIGN
I was expecting at least a standard implementation of MPMNet but something with more physics-based because its Houdini, right?

MPMNet: A Data-driven MPM Framework for Dynamic Fluid-solid Interaction [ar5iv.labs.arxiv.org]

...

These things are quite cutting edge. I'm sorry, but it's very unrealistic to expect Houdini 20.5 to come with an algorithm that was only published in late 2023 in academic context. I'm willing to bet money on that H20.5's roadmap was already settled when H20 was released. A software as complicate as Houdini will never be as "agile" as you expect.

That is completely fair to point out as I am in academic research.

johnmather
macOS does not support Vulkan natively - you instead need to use a translation layer to Metal such as MoltenVK. Adding a layer of indirection is bound to create more bugs, so rather than release something unfinished, we held back so that we can do it right.

Thank you for the info. It is good to have insights into the technology and the difficulties of multi-platform development. You guys are awesome. Could we just have the Metal Hydra Delegate instead of Vulkan as it runs Material X?

I did see Vulkan in a MacOS project here Making a Vulkan cmake project in macOS
[www.youtube.com] but I trust the process and look forward to a working solution.

Because the 20.5 development cycle was a short one, we opted to postpone Vulkan support on the Mac to the next release. There were simply too many things needing attention, and adding Mac support via MoltenVK wasn't feasible in the timeframe we had available. We'll be beginning that task shortly now that 20.5 is out the door.

malexander
we opted to postpone Vulkan support on the Mac to the next release.

I can only imagine the challenges for the 20.5 release. It would be better to have the Metal Hydra Delegate than MoltenVK. How can I connect to that via a python panel? Vulkan is calling a scene graph, can I just call that scene graph also into a Hydra view in the Python panel like the custom OpenGL python Panel? Any insights?

It is simple to write python-based julia metal code and renders.

Metal Hydra with Material X MSL is already supported in the SideFX code base through USD integrations. You would not need to "add" MoltenVK.

malexander
Because the 20.5 development cycle was a short one, we opted to postpone Vulkan support on the Mac to the next release. There were simply too many things needing attention, and adding Mac support via MoltenVK wasn't feasible in the timeframe we had available. We'll be beginning that task shortly now that 20.5 is out the door.

so this means there won't be a native metal viewport but one with vulkan support based on moltenvk coming with 21?

is there any workaround on the mac right now to be able to use copernicus / the preview material?

PHENOMDESIGN
Metal Hydra with Material X MSL is already supported in the SideFX code base through USD integrations. You would not need to "add" MoltenVK.

We've discussed the possibility of enabling the Metal version of Storm for Mac and Vulkan for Win/Linux, though that would only be useful for LOPs. Storm also has its limitations and isn't maintained by us, so it's not a drop in replacement for the native GL renderer.

chf
so this means there won't be a native metal viewport but one with vulkan support based on moltenvk coming with 21?

It's really too early to say yet, I'm afraid. I know it's not an ideal answer, but it depends on well how MoltenVK fits in with our renderer. It's missing some features, some of which either Metal or the Mac GPU can't implement, and some of which just aren't implemented yet.

is there any workaround on the mac right now to be able to use copernicus / the preview material?

It looks like this is working for OpenGL in the current daily build, at least on my Linux box. If it's not for you, could you post a hip file?

malexander

PHENOMDESIGN
Metal Hydra with Material X MSL is already supported in the SideFX code base through USD integrations. You would not need to "add" MoltenVK.

We've discussed the possibility of enabling the Metal version of Storm for Mac and Vulkan for Win/Linux, though that would only be useful for LOPs. Storm also has its limitations and isn't maintained by us, so it's not a drop in replacement for the native GL renderer.

chf
so this means there won't be a native metal viewport but one with vulkan support based on moltenvk coming with 21?

It's really too early to say yet, I'm afraid. I know it's not an ideal answer, but it depends on well how MoltenVK fits in with our renderer. It's missing some features, some of which either Metal or the Mac GPU can't implement, and some of which just aren't implemented yet.

is there any workaround on the mac right now to be able to use copernicus / the preview material?

It looks like this is working for OpenGL in the current daily build, at least on my Linux box. If it's not for you, could you post a hip file?

It's not only about Copernicus, last year I've made RFE for some issues with no answers, on Mac Studio Ultra M2 :
- The viewportt lags when I use crowd with texture
- The viewport lags when I use packed primitives in RBD for instance
- The visualization of volume (ambient occlusion) is very basic in Mac viewport and so on

And what about GPU rendering for Mac ?

malexander
It looks like this is working for OpenGL in the current daily build, at least on my Linux box. If it's not for you, could you post a hip file?

very basic
create cop network
drop a tile node for example
visualize it - this works

drop a preview material
use tile for displacement - doesn't work
change base color on preview material - doesn't work
and so on

malexander
It looks like this is working for OpenGL in the current daily build, at least on my Linux box. If it's not for you, could you post a hip file?

This should be pretty easy to reproduce on MacOS. I've attached a video and hip file showing the GL viewport not updating unless you copy and paste the preview material in COPs (20.5.298 - M2 Ultra)

It does feel that the viewport on MacOS is creaking somewhat, perhaps unsurprisingly given that the OpenGL version is circa 2013. Really hope that 21 brings an updated viewport, so that there's (hopefully full) cross platform feature parity.

Speaking of outdated graphics technologies, do wonder if the OpenCL support on MacOS might become an issue sometime - that's also stuck in the past, but doesn't feel like OpenCL has evolved particularly.

I'm not too fussed by missing out bells and whistle (volume AO is nice, but I can still work), but more core functionality; COPs being an obvious one, but also things like the Texture Mask Paint and some other bits and bobs.

on my MacBook Pro and Mac Studio with H 20.5.298

always a nightmare

https://www.youtube.com/watch?v=WEl-zkldeR8 [www.youtube.com]

I tried a "workaround" via Karma in Solaris with op: and referencing into the COP
but this is cumbersome, way slower and also very buggy...

malexander
It's really too early to say yet, I'm afraid. I know it's not an ideal answer, but it depends on well how MoltenVK fits in with our renderer. It's missing some features, some of which either Metal or the Mac GPU can't implement, and some of which just aren't implemented yet.

It would be best to use the Whole UMA devices such as the vDSP, vForce, vBigNum, and the emerging Accelerate libraries like BNNGraphs, along with the MPSGraphs. I saw Accelerate is in the Linear Solver SOP. The AX VDB JIT compilation tools might be able to help also.

There is a sample app of a CPP Metal View Hydra Delegate from Apple here: Build a 3D application that integrates with Hydra and USD. [developer.apple.com]

Here are sample Metal OpenGL relevant apps:

Draw with Metal and OpenGL in the same view using an interoperable texture. [developer.apple.com]

Replace your app’s deprecated OpenGL code with Metal. [developer.apple.com]

MacOS has a storm renderer built-in too that renders in the basic file browser at the native system level too.

I'm sorry but wouldn't it be reasonable to expect one of the largest and richest companies in the world to have rock solid support for industry standard graphics APIs? Consider bringing this up with your Apple representative. In the meantime choose your hardware and OS to support your software.

Yeah an industry standard that Blender Foundation, SideFX and most AAA game devs all struggle to build stable products with.

Apple was right to design a next-gen graphic API that isn't super low level, which is Metal. "The industry" was wrong. By the way Metal came first and Vulkan was created as a response against Metal.