[Feature]: Support for Snowflake LLMs and Embedding models

[jruokola]

The Feature

Please add native support for Snowflake’s LLMs and embedding models to streamline integration for AI and data-driven applications. This feature would enable seamless querying and embedding generation within Snowflake, reducing the need for external tools. It aligns with the growing demand for in-database machine learning and natural language processing capabilities. Supporting this would enhance workflows for users leveraging Snowflake for advanced analytics and AI.

Thank you for considering this feature!

Motivation, pitch

Snowflake’s addition of LLM and embedding model support would unlock powerful AI capabilities directly within its platform, enabling users to leverage advanced natural language understanding and vector search without leaving the data warehouse. This would streamline workflows, reducing the need for external integration and lowering operational complexity for organizations using Snowflake for analytics and AI. As the demand for AI-driven insights grows, native support for these models would give Snowflake users a competitive edge, enabling faster development and deployment of intelligent applications. This feature aligns with Snowflake’s vision of being a unified data platform, empowering businesses to combine data processing, analytics, and AI under one roof. Supporting LLMs and embeddings would attract forward-thinking organizations seeking scalable, AI-ready solutions.

Are you a ML Ops Team?

Yes

Twitter / LinkedIn details

www.linkedin.com/in/saijani

[krrishdholakia]

could you point me to their api docs?

If it's openai compatible, this should already be possible @jruokola - https://docs.litellm.ai/docs/providers/openai_compatible

[jruokola]

https://docs.snowflake.com/en/user-guide/snowflake-cortex/cortex-llm-rest-api)
https://github.com/b-art-b/langchain-snowpoc/tree/main/langchain_snowpoc

I think it's not compatible and connecting to Snowflake LLMS/Embeddings requires some work

[sumitdas66]

@krrishdholakia This is a needed feature. There is a PR for this raised last year. Do you think that is ready to merge?
#5974

[krrishdholakia]

Hi @sumitdas66 are you referring to embeddings or chat completions? If chat completions this should already work via openai compatible route

[jruokola]

Embeddings and I dont think they have a openAI compatible route

[sfc-gh-alherrera]

@krrishdholakia let me know if you need me to update #5974 to merge. thanks

[krrishdholakia]

cc: @vibhavbhat

[SunnyWan59]

Has support for Snowflake LLMs been added yet? If not, I'd like to work on it!

[krrishdholakia]

Yes please! @SunnyWan59