Note, this post is mentioned in GitHub issue 34 [github.com]
(…let me first say that your work on this plugin is fantastic. I already enjoyed the quality of the code (it's very useful when learning UE4 plugin development), but now that you're using Github and sharing your plugin roadmap and development process, I'm even more impressed.)
With the compliments out of the way… :wink:
(…to paraphrase Mark Twain: “I didn't have time to write a short post, so I wrote a long one instead.” My apologies.)
I'm wondering if you have any thoughts on C++ access to plugin functionality.
Currently we can manually create and tweak Houdini assets within the editor, with some additonal programmability through Blueprint scripts. But at the C++ level, there's not much available.
The Public folders in the runtime and editor modules only provide a single header with some HAPI related functionality that I suspect can be made private (see GitHub issue 33 [github.com]).
This means that our own custom game or plugin modules can't easily spawn or edit a HoudiniAssetActor instance and attach a HoudiniAssetComponent that's using a HoudiniAsset.
To work around this, I patched our local branch of the plugin and made the Private folders public as well. That's obviously a dirty hack – about on par with doing #define private public in a header file – and I'd love to have a better solution quickly-ish.
There are two obvious responses:
- Don't change anything, keep everything private, only support access through editor and blueprint.
- Move all source files from the Private folder to a Public (or if you prefer older module layouts, a Classes folder) and permit any module to freely use your types.
These two are total opposites, yet they do have one thing in common; they're not much work.
Now obviously I hope you'll consider doing more than just option 1 – having control over the Houdini Engine from C++ is crucial for our project. Should your plugin not allow that, I'd have to drop to the HAPI level in my own code. But that means reimplementing large chunks of code that your plugin already has. Most likely my versions thereof would be much lower quality, not having your experience with Houdini and HAPI (and their subtleties).
Though option 2 helps me in the short-term, I don't recommend you go this route. If past SDK development has taught me anything, it's that success depends on an almost dictatorial control over the surface area of the public API. When you give access to everything, people will depend on everything, and you can change nothing. The smaller the API, the less can go wrong.
But alas, I'm sure you know this.
What then, am I looking for?
Let me describe our use-case, and then give my thoughts on how you might enable this.
Example Use for Houdini from C++ in Unreal
I wrote my own Unreal plugin, it's editor only and provides a custom UFunkyAsset type as well as a specialized editor view that gives knobs and dials tailored to UFunkyAssets.
As the user works on the funky asset in its own editor, the UFunkyAsset instance controls a variable amount of AHoudiniAssetActor instances, in turn spawning AHoudiniAssetComponents and subsequent AStaticMesh assets. When the user closes the editor, all static meshes are baked, static mesh actors are created to own them, and the Houdini objects are removed from the level.
In other words, our plugin uses Houdini to create procedural geometry during the edit session, but makes it disappear thereafter (I guess that makes Houdini an appropriate name, heh), leaving only plain UE4 static meshes and actors.
Today we could set this up by hand, but we can't do a variable number of static meshes. (Note, I'm aware that a single Houdini asset can result in multiple static meshes, but that asset's cooking process is still a single process. Instead, I want to spawn one Houdini asset in one location, another one somewhere else, and only update those that need updating (instead of cooking the whole world every time)).
Furthermore, driving the Houdini parameters and input from our C++ plugin code is somewhat clunky. For parameters, I have to go through HoudinAssetParameter instances, a type that exists largely for Unreal UI purposes. For the input, in the case of geometry, the only way for me to provide it is through a UStaticMesh instance, burdening me with Unreal's object system, whereas all I want to do is give you some geometry.
A First Suggestion
What I'd like to see, is a public API that sits somewhere between HAPI and Unreal's object model. Not needing control over UObject management implies you don't have to expose UHoudiniAsset et.al. Instead, you can give us a more restricted IHoudiniAsset (which UHoudiniAsset implements).
class IHoudiniAsset
{
// The usual C++ boilerplate
private:
IHoudiniAsset(const IHoudiniAsset&) = delete;
IHoudiniAsset& operator = (const IHoudiniAsset&) = delete;
protected:
virtual ~IHoudiniAsset() = 0;
public:
To control parameters we avoid UHoudiniAssetParameter. Instead, We provide methods on the interface (these will be a fairly thin layer over HAPI_SetParm*Values). There's no need for reflection (the C++ code already knows how to control the asset), so we don't need a HAPI_ParmInfo equivalent. Nor do I currently have a need for parameter getters, my data flows in one direction, so I have that information already.
// Set simple one-valued parameters.
void SetBool(FString Name, bool Value);
void SetInt32(FString Name, int32 Value);
void SetFloat(FString Name, float Value);
void SetString(FString Name, const FString& Value);
void SetVector(FString Name, const FVector& Value);
void SetString(FString Name, const FVector2D& Value);
void SetColor(FString Name, const FColor Value);
// Maybe some MultiParm support is useful…
void SetInstanceNum(FString Name, int32 Value);
// …should the functions above get an instance index so
// they work with MultiParm, defaulting to zero for simple
// parameters? Or should the functions do some kind of string
// based destructuring, so MultiParm can be set via
//
// SetInt32("SomeMultiParmName", 666);
//
// I guess that's too Javascript-esque… :-)
// I suppose we can use container based versions as well. I don't need
// them to be iterator based, I'm fine with using plain FArrays.
void SetArrayOfInt32(FString Name, const FArray<int32>& Values);
// …etc…
To provide geometry input, we want something lighter than UStaticMesh. We could make it like FRawMesh, or go with something like the FProceduralMeshComponent interface:
// Using FRawMesh…
void SetGeometryInput(int Idx, const FRawMesh& RawMesh);
// Based on ProceduralMeshComponent…
void CreateGeometryInputSection(int32 SectionIndex,
const TArray<FVector>& Vertices,
const TArray<int32>& Triangles,
const TArray<FVector>& Normals,
const TArray<FVector2D>& UV0,
const TArray<FColor>& VertexColors,
const TArray<FProcMeshTangent>& Tangents);
void UpdateGeometryInputSection((…idem…);
void ClearGeometryInputSection(int32 SectionIndex);
At some point I imagine we'll want curves and Houdini assets as input, perhaps like so:
void SetCurveInput(int Idx, const FRawSpline& RawSpline);
void SetAssetInput(int Idx, const IHoudiniAsset& HoudiniAsset);
We need one more thing on this type to make it useful, and that's the cook step. While the auto-cook on parameter changes looks great in a demo, it doesn't scale, so I'd only provide manual cooking.
Obviously the cooking ougth to be done asynchronously, ideally with some kind of support for cancellation and error-handling ofcourse.
Note, by cancellation I mean that we can tell Houdini we no longer care about the results of a cook process. Whether or not Houdini honours that requests is an implementation detail (I have no idea what Houdini supports, perhaps it always completes regardless of cancellation, or maybe it finishes past a certain point. It doesn't matter, I just want to be able to say; I no longer care about these results.)
Ok, so let's give IHoudiniAsset a method called CookAsync. You can optionally cancel any previous cooking, but to allow for more fine-grained cancellation we don't give it a completion callback, but instead return a promise or future like object. Let's call it FHoudiniAsyncCookResult:
// Starts a cook, returns a shareable promise. Once
// data becomes available, it'll remain available
// as long as somebody holds on to the shared reference.
TSharedRef<FHoudiniAsyncCookResult> CookAsync(
bool bCancelAnyPreviousCooks);
};
class FHoudiniAsyncCookResult
{
public:
// Indicates you won't need this cook's results
// anymore. Note, this may or may not honour
// the request, completion routines could still be called.
void Cancel();
// Event called when cooking failed for some reason.
DECLARE_EVENT_OneParam(FHoudiniAsyncCookResult,
FOnError, cont FString& ErrorInfo);
// Three different events, called on success, for various
// types of output. Note that here too there's no reason
// for reflection, the client code would know what type
// of result to expect.
DECLARE_EVENT_OneParam(
FHoudiniAsyncCookResult, FOnGeometryOutput,
const TSharedRef<const FRawMesh>& Data)
DECLARE_EVENT_OneParam(
FHoudiniAsyncCookResult, FOnCurveOutput,
const TSharedRef<const FRawSpline>& Data)
DECLARE_EVENT_OneParam(
FHoudiniAsyncCookResult, FOnAssetOutput,
const TSharedRef<const IHoudiniAsset>& Data)
// I suppose it's not strictly necessary to use Unreal's
// delegate system. Since there's no blueprint interaction,
// a lambda tailored std::function system could work
// as well.
//
// …but when in Rome, I suppose…
FOnError OnError;
FOnGeometryOutput OnGeometryOutput;
FOnCurveOutput OnGeometryOutput;
FOnAssetOutput OnAssetOutput;
private:
// Non-copyable, etc…
}
Perhaps I should mention; while the asset allows for asynchronous cooking, it's perfectly okay that the asset itself should only be used from a single thread. So no thread-safety for IHoudiniAsset.
Finally, to actually be able to get a hold of IHoudiniAssets, we need some kind of factory or getter. As you design the public API, I'm sure a better place for this will emerge – but for now we might as well stick it in the public module interface, the editor-only one to be precise.
class IHoudiniEngineEditor : public IModuleInterface
{
public:
virtual TSharedPtr<IHoudiniAsset>
CreateHoudiniAsset(
const FString& InFileName,
FString* OptOutError) = 0;
// …etcetera…
And there you have it; most of the functionality I can think of needing, bundled in a very restricted API that isn't easily abused.
The thing is, you already have much of this implemented in today's plugin. But it either lives in HoudiniEngineUtils.cpp (now that's a treasure trove of useful code, thanks!), or it's spread over the many types that Unreal's object model forces upon you (assets, components, actors, parameters, brokers, type actions, visualizers, etcetera).
I've glossed over big pieces of course. For example, I've ignored materials completely – simply because I don't have a need for them today. Another piece that I can probably use quite soon would be support for custom vertex attributes, on both inputs and outputs. That means FRawMesh alone or the existing FProceduralMeshComponent interface wouldn't be sufficient. Worst case we can always go with something ‘void*’-ish, and just pass raw byte array, leaving the client responsible for getting element size right, something like:
struct FVertexAttributeChannel
{
FString Name;
int32 SizeOfElement;
TArray<uint8> Data;
};
using FConstVertexAttribArray =
TArray<const FVertexAttributeChannel>;
void CreateMeshSection(…original parameters…,
const FConstVertexAttribArray& CustomAttribChannels);
That'd need more work to support variable size vertex attributes (say, a string per vertex), but even there my use-cases are adequately covered by doing TCHAR StringAttributePerVertex for example.
Concluding
I'm sure my lack of knowledge on Houdini and Hapi internals led me to make false assumptions. So there's probably plenty of bad ideas in the above design. For example, I've never actually used MultiParms myself, so what do I know. Also, I have no idea to what extent Houdini Engine supports overlapping cooks for a single asset.
Nevertheless, I hope the above conveys some ideas on what I'm after, and that we can start a dialogue from there.
Or, you know, just let me know why this is a terrible idea…
Thanks,
Jaap Suter
- http://www.phoenixlabs.ca [phoenixlabs.ca]
- http://www.jaapsuter.com [jaapsuter.com]
