Fix deadline serialization, repr, and prune edge cases - #70421
Fix deadline serialization, repr, and prune edge cases#70421seanghaeli wants to merge 8 commits into
Conversation
Split out of apache#68919 per review: decoder __class_path routing, clear error for missing __class_path, repr guards for severed dagrun and dict-shaped interval, and prune guard for missed deadlines.
…/deadline-robustness-followup-draft
09a9f50 to
385914d
Compare
| def setup_method(): | ||
| _clean_db() | ||
|
|
There was a problem hiding this comment.
If we need to clean the db before our tests, then something else is leaving artifacts that should not be there, this will hide issues.
There was a problem hiding this comment.
Good point, just removed the _clean_db calls on setup. But something I noticed is that this pattern already exists in the repo:
Perhaps it's worth it to address the other instances too in another PR?
| if isinstance(data, (int, float)): | ||
| interval_seconds = int(data) |
There was a problem hiding this comment.
It might be safer to check if __classname__ == "datetime.timedelta" instead, otherwise I think this could catch a broader scope than intended. I may be overthinking this one though.
There was a problem hiding this comment.
I think this would just be a style preference. Whether data's interval is of type timedelta or variableinterval, the dict would be handled correctly.
I've just added a VariableInterval case in test test_deadline_alert_repr_does_not_raise_on_json_dict_interval to cover this exact situation
Split out of #68919 per review. Edge-case fixes in the deadline model and serialization paths:
__class_pathahead ofreference_type(custom references can share a builtin's class name), and a reference with no importable__class_pathraises a clear error instead of aKeyError.__repr__onDeadlineandDeadlineAlertnever raises (guards thedagrunrelationship post-cascade-delete and the dict-shaped interval).prune_deadlinesexcludes missed deadlines so a missed deadline's queued callback is not cascade-deleted.