Make copilot integration tests pass on the replace-litellm branch (#1109)

* Fix copilot secret unredaction

The copilot provider always sends `cleanup_sensitive` set to `False` as it
manages the context itself. On streams where `finish_stream` was set to
`False` as well, we would have yielded the rest of the context buffer though
which would break secret unredaction.

To reproduce, ask Copilot to make a simple modification in a file
containing secrets so that it's forced to print the secrets back to you.

* Port recent copilot change to the new chunk API

Author: jhrozek

src/codegate/providers/copilot/provider.py

@@ -874,16 +874,15 @@ async def stream_iterator():
                 while not self.stream_queue.empty():
                     incoming_record = await self.stream_queue.get()
                     for choice in incoming_record.choices:
-                        if choice.finish_reason and \
-                                choice.finish_reason in ["stop", "length", "content_filter", "tool_calls"]:
+                        if choice.finish_reason and choice.finish_reason is not None:
                             self.finish_stream = True
                     yield incoming_record
 
             # needs to be set as the flag gets reset on finish_data
             finish_stream_flag = any(
-                choice.get("finish_reason") == "stop"
+                choice.finish_reason is not None
                 for record in list(self.stream_queue._queue)
-                for choice in record.get("content", {}).get("choices", [])
+                for choice in record.choices
             )
             async for record in self.output_pipeline_instance.process_stream(
                 stream_iterator(),