Maintenance: Change parameters appconfig utility API due to GetConfiguration deprecation

[ran-isenberg]

Summary

The current API will be deprecated in the near future according to AWS AppConfig team.
See this
See https://github.com/awslabs/aws-lambda-powertools-python/blob/develop/aws_lambda_powertools/utilities/parameters/appconfig.py

Why is this needed?

GetConfiguration - this API action has been deprecated. Calls to receive configuration data should use the StartConfigurationSession and GetLatestConfiguration APIs instead.

Which area does this relate to?

Parameters

Solution

Use new APIs, hopefully in a non breaking manner.

Acknowledgment

• This request meets Lambda Powertools Tenets
• Should this be considered in other Lambda Powertools languages? i.e. Java, TypeScript

[ran-isenberg]

@leandrodamascena FYI, opened as you requested.

[heitorlessa]

Thank you @ran-isenberg. @leandrodamascena @rubenfonseca this should be noted in V2 because we will have to warn customers that they'll have to bring their own AWS SDK for this to work --- these APIs fail today when using AWS Lambda provided SDKs.

Let's give a ping to the Lambda team in the meantime

[ran-isenberg]

@heitorlessa this APIs will fail only on news accounts that didnt use the old API before. For existing accounts that already use it, they will continue to work (told me by AppConfig team).

[ran-isenberg]

@heitorlessa All we need to do is fix the API call here, which should be a transparent fix. Since the AppConfig said it's going to affect new accounts very soon, i think we should solve this ASAP

[leandrodamascena]

Hi @ran-isenberg! We've done some research and created some example code to simulate various scenarios and we have a few points to discuss before we start coding this.

Can confirm that the old accounts still work with the GetConfiguration API, but the new accounts do not. When a new account tries to run this API, it gets an error like this:

Response
{
  "errorMessage": "An error occurred (BadRequestException) when calling the GetConfiguration operation: Feature flag configurations must be accessed via AWS AppConfig Data's GetLatestConfiguration API.",
  "errorType": "BadRequestException",
  "stackTrace": [
    "  File \"/var/task/lambda_function.py\", line 18, in lambda_handler\n    response = cliente.get_configuration(**sdk_options)\n",
    "  File \"/var/runtime/botocore/client.py\", line 391, in _api_call\n    return self._make_api_call(operation_name, kwargs)\n",
    "  File \"/var/runtime/botocore/client.py\", line 719, in _make_api_call\n    raise error_class(parsed_response, operation_name)\n"
  ]
}

We think a good way do this and without breaking any execution is to replace the get_configuration API call with the two new APIs indicated on the AppConfig page. We checked and this new API works on old and new accounts and on all Python runtimes (3.7, 3.8 and 3.9). But it's bring an additional problem and we need to deal with that before deciding which way to go.

Duration using current api call:

Duration using new api call:

Both Lambda are using Python 3.9 and 128 MB of memory and we see that the second run takes at least 100 ms longer. I think we need to investigate further if there is a way to cache the execution and reduce this time.

We appreciate if you have any insight into this.

Thank you.

[ran-isenberg]

@leandrodamascena thank you for advancing the research. Did you guys ask the appconfig team? I'll try to contact them too and point them to this thread.

[ran-isenberg]

Did you try a second run? in the second run, you are not supposed to call the start configuration API but use the token from the get latest configuration. Did you notice the performance hit in that instance too?

i think you need to store the token in the cache, if it's None, use the start config API, if not use the get latest conf API, set the new token from the response and return the config back.

[ran-isenberg]

@leandrodamascena I think the top priority here should be to get this new API working for new accounts and research the 100ms later on with the AppConfig team.

[leandrodamascena]

Hi @ran-isenberg! Thanks for the feedback. We’re gonna discuss this internally and come back with an update as soon as we can.
We want to talk to the Lambda team on the foreseeable impact and communication to existing customers - anyone relying on runtime SDK will be impacted too.