-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor Deadline Submission #49
base: develop
Are you sure you want to change the base?
Refactor Deadline Submission #49
Conversation
These settings should be base of generic JobInfo values for Deadline submission. They should contain variables previously contained in Submit* Settings. Some of them should be exposed to Publisher UI as artist overrides.
It was decided that dataclasses should be used instead of attrs. This moves DeadlineJobInfo which is full mapping of JobInfo from abstract submitters to collectors to limit need of new class and necessary remapping later.
Added new argument for old get_job_info (which should be probabaly renamed) to pass base of prepared object to be enhanced with DCC specific fields
Empty strings overrides None defaults which might cause issue (it definitely does for job_delay).
'deadline' dictionary wasnt used at all, it contained large DeadlineJobInfo which just enlarged metadata json unnecessary.
It is handling EnvironmentKey* type of fields
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like where this is going! :)
chunk_size: int = SettingsField(999, title="Frames per Task") | ||
priority: int = SettingsField(50, title="Priority") | ||
group: str = SettingsField("", title="Group") | ||
limit_groups: list[LimitGroupsSubmodel] = SettingsField( | ||
default_factory=list, | ||
title="Limit Groups", | ||
) | ||
concurrent_tasks: int = SettingsField( | ||
1, title="Number of concurrent tasks") | ||
department: str = SettingsField("", title="Department") | ||
use_gpu: bool = SettingsField("", title="Use GPU") | ||
job_delay: str = SettingsField( | ||
"", title="Delay job", | ||
placeholder="dd:hh:mm:ss" | ||
) | ||
use_published: bool = SettingsField(True, title="Use Published scene") | ||
asset_dependencies: bool = SettingsField(True, title="Use Asset dependencies") | ||
workfile_dependency: bool = SettingsField(True, title="Workfile Dependency") | ||
multiprocess: bool = SettingsField(False, title="Multiprocess") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be very nice to get description
argument set for each of these.
enabled: bool = SettingsField(True) | ||
optional: bool = SettingsField(title="Optional") | ||
active: bool = SettingsField(title="Active") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With all the other attributes removed from these models we can now make one EnableStateModel
class instead that we use for all of these making this file much smaller.
dln_job_info.add_render_job_env_var() | ||
|
||
# already collected explicit values for rendered Frames | ||
if not dln_job_info.Frames: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I actually think this is a place where we should not be setting this in a Deadline specific way but have it accessible on the instance even outside of the job info. So Frames
would be build up from instance.data["frames"]
instead so that it could turn out to be non-Deadline specific. However, not critical for now and we can do that later. Might be worth a TODO in code or an issue after this PR.
client/ayon_deadline/plugins/publish/global/submit_publish_job.py
Outdated
Show resolved
Hide resolved
Not yet working on AOVs.
Deadline portion must be there for ValidateExpectedFiles
|
||
def to_json(self) -> str: | ||
"""Serialize the dataclass instance to a JSON string.""" | ||
return json.dumps(asdict(self)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
return json.dumps(asdict(self)) | |
return json.dumps(asdict(self)) | |
|
||
# will be reworked when CreateContext contains settings and task types | ||
project_name = create_context.project_name | ||
project_settings = get_project_settings(project_name) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
project_settings = get_project_settings(project_name) | |
project_settings = ( | |
create_context.get_current_project_settings() | |
) |
if not instance["active"]: # TODO origin_data seem not right | ||
return [] | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think it is necessary to hide attributes on not enabled instance?
if not instance["active"]: # TODO origin_data seem not right | |
return [] |
task_name = instance["task"] | ||
folder_path = instance["folderPath"] | ||
folder_entity = ayon_api.get_folder_by_path(project_name,folder_path) | ||
task_entity = ayon_api.get_task_by_name( | ||
project_name, folder_entity["id"], task_name) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note: This happens only to get task type. Fetching this, for each instance, is very slow. We should have option to get task type of an instance from create context -> @kalisp could you create issue?
task_name = instance["task"] | |
folder_path = instance["folderPath"] | |
folder_entity = ayon_api.get_folder_by_path(project_name,folder_path) | |
task_entity = ayon_api.get_task_by_name( | |
project_name, folder_entity["id"], task_name) | |
task_name = instance["task"] | |
folder_path = instance["folderPath"] | |
folder_entity = ayon_api.get_folder_by_path(project_name,folder_path) | |
task_entity = ayon_api.get_task_by_name( | |
project_name, folder_entity["id"], task_name) |
defs = [] | ||
|
||
defs.extend([ | ||
UISeparatorDef("deadline_defs_starts"), | ||
]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
defs = [] | |
defs.extend([ | |
UISeparatorDef("deadline_defs_starts"), | |
]) | |
defs = [ | |
UISeparatorDef("deadline_defs_starts"), | |
] |
override_defs = OrderedDict({ | ||
"chunkSize": NumberDef( | ||
"chunkSize", | ||
label="Frames Per Task", | ||
default=1, | ||
decimals=0, | ||
minimum=1, | ||
maximum=1000 | ||
), | ||
"priority": NumberDef( | ||
"priority", | ||
label="Priority", | ||
decimals=0 | ||
), | ||
"department": TextDef( | ||
"department", | ||
label="Department", | ||
default="", | ||
), | ||
"limit_groups": TextDef( | ||
"limit_groups", | ||
label="Limit Groups", | ||
default="", | ||
placeholder="machine1,machine2" | ||
), | ||
"job_delay": TextDef( | ||
"job_delay", | ||
label="Delay job (timecode dd:hh:mm:ss)", | ||
default="" | ||
), | ||
}) | ||
defs = [] | ||
# The Arguments that can be modified by the Publisher | ||
for key, value in override_defs.items(): | ||
if key not in overrides: | ||
continue | ||
|
||
default_value = profile[key] | ||
value.default = default_value | ||
defs.append(value) | ||
|
||
return defs |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
override_defs = OrderedDict({ | |
"chunkSize": NumberDef( | |
"chunkSize", | |
label="Frames Per Task", | |
default=1, | |
decimals=0, | |
minimum=1, | |
maximum=1000 | |
), | |
"priority": NumberDef( | |
"priority", | |
label="Priority", | |
decimals=0 | |
), | |
"department": TextDef( | |
"department", | |
label="Department", | |
default="", | |
), | |
"limit_groups": TextDef( | |
"limit_groups", | |
label="Limit Groups", | |
default="", | |
placeholder="machine1,machine2" | |
), | |
"job_delay": TextDef( | |
"job_delay", | |
label="Delay job (timecode dd:hh:mm:ss)", | |
default="" | |
), | |
}) | |
defs = [] | |
# The Arguments that can be modified by the Publisher | |
for key, value in override_defs.items(): | |
if key not in overrides: | |
continue | |
default_value = profile[key] | |
value.default = default_value | |
defs.append(value) | |
return defs | |
override_defs = [ | |
NumberDef( | |
"chunkSize", | |
label="Frames Per Task", | |
default=1, | |
decimals=0, | |
minimum=1, | |
maximum=1000 | |
), | |
NumberDef( | |
"priority", | |
label="Priority", | |
decimals=0 | |
), | |
TextDef( | |
"department", | |
label="Department", | |
default="", | |
), | |
TextDef( | |
"limit_groups", | |
label="Limit Groups", | |
default="", | |
placeholder="machine1,machine2" | |
), | |
TextDef( | |
"job_delay", | |
label="Delay job (timecode dd:hh:mm:ss)", | |
default="" | |
), | |
] | |
defs = [] | |
# The Arguments that can be modified by the Publisher | |
for attr_def in override_defs: | |
if attr_def.key not in overrides: | |
continue | |
attr_def.default = profile[attr_def.key] | |
defs.append(attr_def) | |
return defs |
@classmethod | ||
def on_value_change(cls, event): | ||
for instance_change in event["changes"]: | ||
if not cls.instance_matches_plugin_families(instance): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
instance
is not defined variable.
task_name = task_entity["name"] | ||
task_type = task_entity["taskType"] | ||
profiles = ( | ||
project_settings["deadline"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
project_settings["deadline"] | |
project_settings | |
["deadline"] |
project_settings = context_data["project_settings"] | ||
task_entity = context_data["taskEntity"] | ||
|
||
task_name = task_type = "" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
task_name = task_type = "" | |
task_name = task_type = None |
if profiles: | ||
profile = filter_profiles( | ||
profiles, | ||
{ | ||
"host_names": host_name, | ||
"task_types": task_type, | ||
"task_names": task_name, | ||
# "product_type": product_type | ||
} | ||
) | ||
if profile: | ||
attr_values = profile |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if profiles: | |
profile = filter_profiles( | |
profiles, | |
{ | |
"host_names": host_name, | |
"task_types": task_type, | |
"task_names": task_name, | |
# "product_type": product_type | |
} | |
) | |
if profile: | |
attr_values = profile | |
profile = filter_profiles( | |
profiles, | |
{ | |
"host_names": host_name, | |
"task_types": task_type, | |
"task_names": task_name, | |
# "product_type": product_type | |
} | |
) | |
if profile: | |
attr_values = profile |
def get_attr_defs_for_instance(cls, create_context, instance): | ||
defs = super().get_attr_defs_for_instance(create_context, instance) | ||
|
||
defs.extend([ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will show definitions for all instances in maya.
Changelog Description
This PR tries to make DL submissions easier, mostly handling of
JobInfo
segment. Collection of values is moved out of submission phase much earlier where values could be filled from default values in Settings or overriden by selected list of field by artist in Publisher UI.Move to universal collector should help with potential validation of filled values.
Profiles in Settings should allow more control of values (like
priority
,limit_groups
) based ontask_types
orhosts
etc.Additional info
Very WIP, currently working only in AE.
Should be used with ynput/ayon-core#958
Testing notes: