Deadline plugin¶

Overview¶

Conductor and Deadline are integrated via a toolset that sends jobs to Conductor and ensures they only run on Conductor instances. Upload and download tasks are handled by Conductor's upload and download daemons.

One of Conductor's underlying principles is to only spin-up an instance when a specific job or task needs it.

Just as with any other instance in a Deadline managed farm, Conductor instances run the Deadline Worker.

Installation¶

If you haven't already done so, Download the Deadline Plugin.

Steps 2-4 require SuperUser access in Deadline Monitor

Copy plugins to the correct folders:

deadline_plugin/custom/scripts/Jobs/* -> /custom/scripts/Jobs/
deadline_plugin/custom/events/ConductorWorker -> /custom/events/

Install the pre-requisite Conductor python libraries: pip install ciocore ciopath cioseq
Add the path of the Conductor python libraries (above) and deadline_plugin/src to the python paths in your repo.

Add the Conductor paths to the MayaCmd plugin. You only need to add for the versions that you intend to use

        /opt/autodesk/maya-io/2016.5/maya-io2016.5.SP0/bin/Render
        /opt/autodesk/maya-io/2016/maya-io2016.SP0/bin/Render
        /opt/autodesk/maya-io/2017/maya-io2017.SP0/bin/Render
        /opt/autodesk/maya-io/2017/maya-io2017.SP3/bin/Render
        /opt/autodesk/maya-io/2017/maya-io2017.SP4/bin/Render
        /opt/autodesk/maya-io/2017/maya-io2017.SP5/bin/Render
        /opt/autodesk/maya-io/2018/maya-io2018.SP0/bin/Render
        /opt/autodesk/maya-io/2018/maya-io2018.SP1/bin/Render
        /opt/autodesk/maya-io/2018/maya-io2018.SP2/bin/Render
        /opt/autodesk/maya-io/2018/maya-io2018.SP3/bin/Render
        /opt/autodesk/maya-io/2018/maya-io2018.SP4/bin/Render
        /opt/autodesk/maya-io/2018/maya-io2018.SP5/bin/Render
        /opt/autodesk/maya-io/2018/maya-io2018.SP6/bin/Render
        /opt/autodesk/maya-io/2019/maya-io2019.SP0/bin/Render
        /opt/autodesk/maya-io/2019/maya-io2019.SP1/bin/Render
        /opt/autodesk/maya-io/2019/maya-io2019.SP2/bin/Render
        /opt/autodesk/maya-io/2020/maya-io2020.SP0/bin/Render
        /opt/autodesk/maya-io/2022/maya-io2022.SP0/bin/Render
        /opt/autodesk/maya-io/2022/maya-io2022.SP3/bin/Render
        /opt/autodesk/maya-io/2023/maya-io2023.SP1/bin/Render

Add submit_to_conductor.py as a Job menu script. The path to the icon is deadline_plugin/custom/scripts/General/conductor_logo.ico
The Deadline Workers running on Conductor instances need to connect to your repository via a Remote Connection Server (RCS). See the Deadline documentation for more details. This will often require you to open ports in your firewall to ports 8000 or 4433 (TLS) - however, the exact ports are configurable via the Deadline config.

To connect to your Deadline RCS, you will need the RCS's hostname/IP, port, and the client certificate (.pfx). To provide theses values, you can either set the environment variables CONDUCTOR_DEADLINE_PROXY and CONDUCTOR_DEADLINE_SSL_CERTIFICATE before launching the Monitor or modify the code directly in the submit script. Please note that client certificates with a passphrase cannot be configured through the web interface. Please contact [Conductor's support team](https://support.conductortech.com/hc/en-us/requests/new) for more information.

Usage¶

Prerequisites¶

The Conductor upload daemon must be running
The Conductor download daemon must be running
A dependency sidecar exists for the given job
Your Deadline Remote Connection Server's public IP address was provided to Conductor's support team, to be opened on the render nodes' firewall

Sending Jobs to Conductor¶

From the Deadline Monitor, select one or multiple jobs.
From the context menu, select Scripts->Send to Conductor
A dialog will appear where you can modify the default options for a job
Once you click Submit, the job is shipped to Conductor
The dialog reappears for each selected job. For example, if you select five jobs, the dialog will appear five times.
A valid dependency sidecar file is usually required. For more details, see the section on Dependencies
After a few minutes, new Workers pop-up in the Worker panel with the suffix -conductor. Each Worker will render exactly one Deadline task. After a Worker completes a Deadline task, they will shut down. 8.To delete Conductor Workers that have completed their task, visit the Workers panel. They will have a state of Stalled.

How it works¶

We run a Deadline Worker job on Conductor and configure it to shut down after exactly one Deadline task. There's some additional logic to create a temporary Deadline group (conductorautogroup_<jobid>) to ensure that the submitted Deadline job only runs on the Conductor instances and that the Conductor instances only pick up the given Deadline job.

An essential step is to ensure that a valid sidecar dependency file exists. If it doesn't, the Conductor instances will not have any necessary files to run the task. See the section on dependencies for more details.

Viewing reports and most other functions of a Deadline Worker will work with a Conductor Deadline Worker, except for any operation that involves sending a command directly to the host instance (see below for more details).

Dependencies¶

A task can not render if it does not have all of its required dependencies. We ship all required dependencies to Conductor as part of the submission process. The uploader daemon is responsible for performing the actual upload. Therefore, it's necessary to create this list of dependencies beforehand.

Note

For convenience a sidecar file is not required if the only file needed for the job is the selected scene file

There are several ways to do this:

Generate the sidecar dependency file on job submission. Do this manually or as part of the submission to Deadline. You'll find a modified Deadline Maya submitter included in the repo. (this should be used as a sample only). Below is a python code snippet on how to scan and save dependencies:

This code has been depreciated as it uses our old client_tools library. Please take a look at ciomaya for updated method and/or reach out to Conductor's support team

def get_dependencies():
    from conductor.lib import common, maya_utils
    resources = common.load_resources_file()
    dependency_attrs = resources.get("maya_dependency_attrs") or {}
    return maya_utils.collect_dependencies(dependency_attrs)

dependencies = {'dependencies': get_dependencies()}

ma_path = maya.cmds.file(query=True, sceneName=True)
dependency_sidecar_path = "{}.cdepends".format(ma_path)

with open(dependency_sidecar_path, 'w') as fh:
    json.dump(dependencies, fh)

Submit an upstream job to Deadline that will generate the dependency sidecar file

Generally, if a DCC software is already open with the scene file loaded, scanning for dependencies is a quick task.

Note

There are a few key differences between regular on-site Deadline Workers and Conductor Workers:

Conductor Workers are not directly accessible like other Workers. They run from within the Conductor infrastructure. They don't have an exposed IP/hostname, and therefore, remote commands do not work.
Conductor Workers can only render the jobs they're assigned. If you try to render a different job, it will fail (as none of the files are available).
Conductor Workers will manage themselves. They'll start-up and shut down as needed

Package Mapping¶

Conductor also needs to know what software is required for the job. This can be obtained from the Deadline job. The DeadlineToConductorPackageMapper class is used for this purpose and will likely need to be modified by the customer to support the customer's specific DCC's and plugins. More details on extending DeadlineToConductorPackageMapper are available in the code-level documentation. Please contact Conductor support with any questions.

Licensing¶

Deadline 10.1.23 and greater no longer require a license. Earlier versions of 10.1 don't require a license if running on AWS. It is the customer's responsibility to choose the appropriate licensing model. Licensing of DCC's and all other software is managed by Conductor and included in the cost.

Licenses can be assigned to Conductor Deadline Workers by using the Auto Configure feature in the Repository options.

Supported software¶

Currently, the Deadline integration is set up to support Maya with Arnold, VRay and Renderman renderers using the MayaCmd plugin. Arnold standalone and generic Cmd plugins are also supported. Additional software can be supported by extending conductor_deadline.plugin_mapper.

It is possible to upload the files simultaneously as submitting the job without using the uploader daemon. This can be configured in the Conductor config. However, this method is not recommended when submitting jobs via Deadline due to the potentially long wait times.* ↩