-
Notifications
You must be signed in to change notification settings - Fork 464
feat(idempotency): Allow durable functions to replay #7764
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Conversation
Durable Functions introduces the concept of function replays. Previously, the idempotency utility would throw an "IdempotencyItemAlreadyExistsError", as the replay has the same payload of the initial execution. It appears as a duplicate, so is rejected. Now, a replay is allowed
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #7764 +/- ##
========================================
Coverage 96.52% 96.52%
========================================
Files 275 275
Lines 13123 13139 +16
Branches 990 992 +2
========================================
+ Hits 12667 12683 +16
Misses 353 353
Partials 103 103 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
leandrodamascena
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey @ConnorKirk! Overall the PR looks good! I'd suggest considering these changes to use a Protocol instead of a concrete class - this avoids type conflicts with the real DurableContext from the Durable SDK and enables proper duck typing. I haven't fully tested with mypy, but we shouldn't have any issues.
If you think that make sense, you need to change the class name in all the other files.
Thanks a lot for working on this.
| class DurableContext: | ||
| _lambda_context: LambdaContext | ||
| _state: object | ||
|
|
||
| @property | ||
| def lambda_context(self) -> LambdaContext: | ||
| return self._lambda_context | ||
|
|
||
| @property | ||
| def state(self) -> object: | ||
| return self._state |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changed from a concrete class to a Protocol with @runtime_checkable. This avoids type conflicts with the real DurableContext from the Durable SDK - users can use the SDK's class directly without import collisions and we avoid creating a type only for this.
| class DurableContext: | |
| _lambda_context: LambdaContext | |
| _state: object | |
| @property | |
| def lambda_context(self) -> LambdaContext: | |
| return self._lambda_context | |
| @property | |
| def state(self) -> object: | |
| return self._state | |
| # Protocol enables duck typing without competing with the real DurableContext from the SDK | |
| @runtime_checkable | |
| class DurableContextProtocol(Protocol): | |
| """Protocol for durable execution context compatibility.""" | |
| @property | |
| def lambda_context(self) -> LambdaContext: | |
| """The underlying Lambda context.""" | |
| ... | |
| @property | |
| def state(self) -> object: | |
| """The durable execution state.""" | |
| ... |
| if hasattr(context, "state"): | ||
| # Extract lambda_context from DurableContext | ||
| durable_context = cast("DurableContext", context) | ||
| config.register_lambda_context(durable_context.lambda_context) | ||
| # Note: state.operations is accessed via duck typing at runtime | ||
| is_replay = len(durable_context.state.operations) > 1 # type: ignore[attr-defined] | ||
| else: | ||
| # Standard LambdaContext | ||
| config.register_lambda_context(context) | ||
| is_replay = False |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think if we change hasattr(context, "state") to hasattr(context, "state") and hasattr(context, "lambda_context"). We have more explicit check that ensures we only treat it as a durable context when both required attributes exist.
| if hasattr(context, "state"): | |
| # Extract lambda_context from DurableContext | |
| durable_context = cast("DurableContext", context) | |
| config.register_lambda_context(durable_context.lambda_context) | |
| # Note: state.operations is accessed via duck typing at runtime | |
| is_replay = len(durable_context.state.operations) > 1 # type: ignore[attr-defined] | |
| else: | |
| # Standard LambdaContext | |
| config.register_lambda_context(context) | |
| is_replay = False | |
| # Check for durable context using duck typing | |
| if hasattr(context, "state") and hasattr(context, "lambda_context"): | |
| durable_context = cast("DurableContextProtocol", context) | |
| config.register_lambda_context(durable_context.lambda_context) | |
| # Note: state.operations is accessed via duck typing at runtime | |
| is_replay = len(durable_context.state.operations) > 1 # type: ignore[attr-defined] | |
| else: | |
| # Standard LambdaContext | |
| config.register_lambda_context(context) | |
| is_replay = False |
|



Issue number: fixes #7762
Summary
This PR adds support for durable functions in the Idempotency utility.
is_replayvariable. We use this to determine if a function is being replayed.Changes
is_replayparameter to thehandlemethodDurableContextclassUser experience
Idempotency utility now allows durable functions to execute after suspension.
Notes
This leaves one issue remaining in the idempotency utility when used with durable functions.
We use
context.getTimeInMillis()to calculate the expiry time of theINPROGRESSstatus. Duplicate payloads received after that expiry time will be allowed. In a durable execution, the length of the workflow can be up to 1 year, across multiple lambda invocations. This is longer than the maximum 15 minutes that anINPROGRESSstatus can currently have. Idempotency records for long running workflows will not be protected from duplicate invokes beyond the first 15 minutes of their life.In future, we could
DurableContext.getExecutionTimeoutInMillis()method. Once this is added, we can calculate the workflow timeout in the same manner we do for a single invocation.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.