Houdini Husk Submitter reference¶
Beta
This feature is currently in BETA. We're interested in hearing your feedback and what improvements we can make. Please direct all feedback to support
The Husk Submitter dispatches USD files directly to your render farm using the husk command.
This document is intended to be a comprehensive reference for all Solaris Submitter features and concepts. If you just want to get up and running fast, head over to the Houdini tutorial page.
Introduction¶
The submitter HDA generates a submission payload, which is sent to the cloud when you press Submit. This payload is in the form of a JSON object. You can view the resolved submission at any time in the Preview tab of the submitter UI.
By analyzing this payload, you will gain a good understanding of how Conductor works at a conceptual level and will find troubleshooting submission issues easier.
In the Preview Panel, you'll find:
- All expressions in the HDA are resolved in file paths, in the job title, and other parameters.
- All file dependencies are collected. For optimization reasons, this is done on-demand with the Do asset scan button.
- Environment variables to be active on the render nodes are generated based on your software choices and custom additions.
- Task commands (one per instance) are generated based on your frame range specifications and the task template.
The data in the preview panel is kept up to date as you change configuration settings. For example, when you change the frame range or output image paths in the input ROP, the list of tasks in the preview panel updates.
Actions Row¶
Connect¶
Click to establish a connection with Conductor. This fetches information from your account including the lists of projects, software packages, and instance types available.
Export script¶
Click to save a Python script that can submit this job offline.
Submit¶
Submit the job to Conductor.
General section¶
Job Title¶
The job title appears in the Conductor dashboard. It helps to identify your job by evaluating the hipfile name, the renderer, and other information. You may overwrite the default expression.
Project¶
This refers to a project on the Conductor dashboard. The dropdown menu is populated or updated when the submitter connects to your Conductor account. If the menu contains only the - Not Connected - option, press the Connect button.
If projects are added or removed since connecting to Conductor, you can press the Connect button again to refresh the list.
Instance type family¶
Choose between machines with or without graphics cards. If your job does not require GPUs, then don't select a GPU instance type as they are considerably more expensive.
Instance Type¶
Specify the hardware configuration used to run your tasks. You are encouraged to run tests to find the most cost-efficient combination that meets your deadline. You can read about hardware choices and how they affect costs in this blog post.
Preemptible¶
Preemptible instances are less expensive to run than non-preemptible. The drawback is that they may be stopped at any time by the cloud provider. The probability of a preemption rises with the duration of the task. Conductor does not support checkpointing, so if a preemption occurs, the task starts from scratch on another instance. It is possible to change the preemptible setting in the dashboard for your account.
Preempted retries:¶
Set how many times a preempted task will be retried automatically.
Houdini Version¶
This is the version of Houdini to run on the render nodes. It can be different from your local version, but be aware of feature changes affecting your render.
Render Options¶
Driver software¶
The plugin software (if any) that supports the connected input driver.
Add plugin¶
Add additional plugin software provided and licensed by Conductor that needs to be enabled on the render node.
Frames section¶
Frame spec¶
A frame-spec is a comma-separated list of arithmetic progressions. In most cases, this will be a simple range:
1001-1200
However, any set of frames may be specified efficiently in this way.
1,7,10-20,30-60x3,1001
Negative numbers are also valid.
-7,-20--10
Chunk Size¶
A chunk is the set of frames handled by one task. If your renders are reasonably fast, it may make sense to render many frames per task because the time it takes to spin up instances, and sync can be significant by comparison.
Chunk Size¶
A chunk is the set of frames handled by one task. If your renders are reasonably fast, it may make sense to render many frames per task because the time it takes to spin up instances, and sync can be significant by comparison.
Use scout frames¶
Enable the Scout Frames feature. See below.
Scout Frames¶
Specify a set of frames to render first. We start any tasks that contain these frames. All others are put on hold, which allows you to check a subsample of your sequence before committing to the full render.
You can use a frame spec to specify scout frames, for example: 1-100x30. Alternatively, you can select how many scout frames you want and let the submitter calculate scout frames from the current frame range. To specify three well-spaced scout frames automatically, enter auto:3.
Note
The remote render nodes execute tasks in their entirety, so if you have chunk size set greater than 1, then all frames are rendered in any task containing a scout frame.
ROP Options¶
Image output path¶
Specify the folder where your rendered images will be saved.¶
USD filepath¶
Choose the USD file you want to submit to the render farm.¶
Task template¶
Command that runs on each render node at Conductor. For the Husk submitter, the default command structure is:
husk --verbose 9 -f {first} -n {count} -i {step} -o {image_output} --renderer {render_delegate} {usd_filepath}
Here's a breakdown of each component:
husk: The command-line executable for rendering USD files with Husk.--verbose 9: Sets the verbosity level of Husk's output to9(very detailed logging), which is useful for debugging.-f {first}:-f: Husk's option to specify the first frame to render in a sequence.{first}: This placeholder is populated with the starting frame number of the current render task or chunk.
-n {count}:-n: Husk's option to specify the number of frames to render, starting from the frame specified by-f.{count}: This placeholder is populated with the total number of frames in the current render task or chunk (calculated aslast - first + 1).
-i {step}:-i: Husk's option to specify the frame increment or step.{step}: This placeholder is populated with the increment between frames to be rendered (e.g., if step is 2, frames 1, 3, 5... would be rendered).
-o {image_output}:-o: Husk's option to specify the output image file path and pattern.{image_output}: This placeholder is populated with the base path and filename pattern for the rendered images. Husk typically appends frame numbers and extensions to this.
--renderer {render_delegate}:--renderer: Husk's option to specify which render delegate (renderer plugin) to use.{render_delegate}: This placeholder is populated with the configured render delegate, such as "Karma", "Arnold", "Redshift", etc.
{usd_filepath}:- This is the final argument, representing the path to the USD file that Husk should load and render.
- It's populated with the path specified in the "USD filepath" parameter of the submitter.
Upload daemon¶
Use upload Daemon is off by default, which means that the task of uploading assets happens within Houdini itself. Although this requires no extra setup, if you have many assets, Houdini will block until uploading completes.
A better solution may be to turn on Use Upload Daemon. An upload daemon is a separate background process. It means assets are not uploaded in the application. The submission, including the list of expected assets, is sent to Conductor, and the upload daemon continually asks the server if there are assets to upload. When your job hits the server, the upload daemon will get the list and upload them, which allows you to continue with your work.
You can start the upload daemon either before or after you submit the job. Once started, it will listen to your entire account, and you can submit as many jobs as you like.
Note
You must have Conductor Core installed in order to use the upload daemon and other command line tools. See the installation page for options.
To run an upload daemon, open a terminal or command prompt, and run the following command.
conductor uploader
Once started, the upload daemon runs continuously and uploads files for all jobs submitted to your account.
Location tag¶
Attach a location to this submission for the purpose of matching to an uploader and/or downloader process.
If your organization is distributed in several locations, you can enter a value here, for example, London. Then when you run a downloader daemon you can add the location option to limit downloads to only those that were submitted in London.
Advanced section¶
Asset Scan¶
Asset scan regex¶
In order to scan for assets to be uploaded, all file reference parameters in the scene are evaluated
for one frame. Each evaluated filename is then adjusted to have certain parts replaced by wildcards
to create a globbable pattern. For example, the file reference, $HIP/tex/texture.###.
Asset scan excludes¶
Comma-separated list of Unix-style wildcard patterns to exclude from the asset scan. For more information on Unix-style wildcards, see this page: https://docs.python.org/3/library/fnmatch.html
More info
- Please visit this detailed page for a deeper understanding of the asset scraping mechanism.
Extra assets¶
Browse for files to upload that were not found automatically by the asset scan. The best way to check the results of the asset scan is to look in the preview panel and click the do_asset_scan button.
You may also browse for entire folders.
Extra environment¶
Add variable¶
Add environment variables and values to be set on the render nodes. You may want this for example if you have your own shell script and you need to make an addition to the PATH variable so that the script can be found. In this case you would need to browse for the script in the extra assets section.
Note
- If submitting from Windows, make sure to remove the drive letter from the path to the script while defining the environment variable. Note that environment variables can be defined as exclusive or appendable.
Add existing variable¶
A convenient UI to add extra remote environment variables based on variables that have been defined locally.
Miscellaneous parameters¶
Notifications¶
Add one or more email addresses, separated by commas to receive an email when the job completes.
Preview tab¶
Display Tasks¶
Set the number of tasks to show. This is for display purposes only and does not affect the tasks that are submitted to Conductor.
Do asset scan¶
Click this button to see the results of a full asset scan in the payload. This is an optimization for display purposes only, since the preview is updated frequently and asset scanning may be expensive. On submission, a full asset scan is always run.
Payload The raw submission object. This is automatically updated as you populate the fields in the Configuration tab. You can return to this tab at any point to confirm that your submission includes the correct data.