Add Foundry Local as a chat and embedding provider in the AI Chat Web template #7586
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -92,6 +92,34 @@ ollama pull all-minilm | |
| ### 3. Learn more about Ollama | ||
| Once the models are installed, you can start using them in your application. Refer to the [Ollama documentation](https://github.com/ollama/ollama/blob/main/docs/README.md) for detailed instructions on how to explore models locally. | ||
|
|
||
| #### ---#endif | ||
| #### ---#if (IsFoundryLocal) | ||
| ## Setting up a local environment using Foundry Local | ||
| This project is configured to use Foundry Local, which runs models on your workstation through a local OpenAI-compatible endpoint. It does not need an API key. | ||
|
|
||
| ### 1. Install Foundry Local | ||
| Install Foundry Local for your operating system by following the [Foundry Local documentation](https://learn.microsoft.com/azure/ai-foundry/foundry-local/). | ||
|
Member
There was a problem hiding this comment. I think this is not needed, is also mentioned in the SDK reference docs:
|
||
|
|
||
| ### 2. Run the app | ||
| The app starts the Foundry Local service for you. On first run, it downloads the configured models, then loads them into the local service. | ||
|
|
||
| The default chat model alias is `qwen3-4b`. The default embedding model alias is `qwen3-embedding-0.6b`. | ||
|
|
||
| ### 3. Override model aliases or the service URL | ||
| You can change the defaults in `appsettings.json`, `appsettings.Development.json`, or user secrets: | ||
|
|
||
| ```json | ||
| { | ||
| "FoundryLocal": { | ||
| "ChatModel": "qwen3-4b", | ||
| "EmbeddingModel": "qwen3-embedding-0.6b", | ||
| "ServiceUrl": "http://127.0.0.1:5273" | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| Use `FoundryLocal:ServiceUrl` if another local process already uses the default port. | ||
|
|
||
| #### ---#endif | ||
| #### ---#if (IsAzureOpenAI) | ||
| ## Using Azure OpenAI | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -7,6 +7,8 @@ public class IngestedChunk | |
| { | ||
| #if (IsOllama) | ||
| public const int VectorDimensions = 384; // 384 is the default vector size for the all-minilm embedding model | ||
| #elif (IsFoundryLocal) | ||
| public const int VectorDimensions = 1024; // 1024 is the default vector size for the qwen3-embedding-0.6b embedding model | ||
| #else | ||
| public const int VectorDimensions = 1536; // 1536 is the default vector size for the OpenAI text-embedding-3-small model | ||
| #endif | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -6,4 +6,11 @@ | |
| "Microsoft.EntityFrameworkCore": "Warning" | ||
| } | ||
| } | ||
| //#if (IsFoundryLocal) | ||
| ,"FoundryLocal": { | ||
| "ChatModel": "qwen3-4b", | ||
| "EmbeddingModel": "qwen3-embedding-0.6b", | ||
| "ServiceUrl": "http://127.0.0.1:5273" | ||
| } | ||
| //#endif | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -6,5 +6,12 @@ | |
| "Microsoft.EntityFrameworkCore": "Warning" | ||
| } | ||
| }, | ||
| //#if (IsFoundryLocal) | ||
| "FoundryLocal": { | ||
| "ChatModel": "qwen3-4b", | ||
| "EmbeddingModel": "qwen3-embedding-0.6b", | ||
| "ServiceUrl": "http://127.0.0.1:5273" | ||
| }, | ||
| //#endif | ||
| "AllowedHosts": "*" | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -32,6 +32,7 @@ public AIChatWebSnapshotTests(ITestOutputHelper log) | |
| [InlineData /* Defaults: --provider=githubmodels --vector-store=local */] | ||
| [InlineData("--provider=ollama", "--vector-store=qdrant")] | ||
| [InlineData("--provider=openai", "--vector-store=azureaisearch")] | ||
| [InlineData("--provider=foundrylocal", "--vector-store=local", "--webHttpPort", "9996", "--webHttpsPort", "9995")] | ||
| [InlineData("--aspire")] | ||
| [InlineData("--aspire", "--provider=azureopenai", "--vector-store=azureaisearch")] | ||
| public async Task RunSnapshotTests(params string[] templateArgs) | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| <!DOCTYPE html> | ||
| <html lang="en"> | ||
|
|
||
| <head> | ||
| <meta charset="utf-8" /> | ||
| <meta name="viewport" content="width=device-width, initial-scale=1.0" /> | ||
| <base href="/" /> | ||
| <link rel="stylesheet" href="@Assets["app.css"]" /> | ||
| <link rel="stylesheet" href="@Assets["aichatweb.styles.css"]" /> | ||
| <ImportMap /> | ||
| <HeadOutlet @rendermode="@renderMode" /> | ||
| </head> | ||
|
|
||
| <body> | ||
| <Routes @rendermode="@renderMode" /> | ||
| <script src="@Assets["app.js"]" type="module"></script> | ||
| <script src="@Assets["_framework/blazor.web.js"]"></script> | ||
| </body> | ||
|
|
||
| </html> | ||
|
|
||
| @code { | ||
| private readonly IComponentRenderMode renderMode = new InteractiveServerRenderMode(prerender: false); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| <div class="lds-ellipsis"><div></div><div></div><div></div><div></div></div> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,89 @@ | ||
| /* Used under CC0 license */ | ||
|
|
||
| .lds-ellipsis { | ||
| color: #666; | ||
| animation: fade-in 1s; | ||
| } | ||
|
|
||
| @keyframes fade-in { | ||
| 0% { | ||
| opacity: 0; | ||
| } | ||
|
|
||
| 100% { | ||
| opacity: 1; | ||
| } | ||
| } | ||
|
|
||
| .lds-ellipsis, | ||
| .lds-ellipsis div { | ||
| box-sizing: border-box; | ||
| } | ||
|
|
||
| .lds-ellipsis { | ||
| margin: auto; | ||
| display: block; | ||
| position: relative; | ||
| width: 80px; | ||
| height: 80px; | ||
| } | ||
|
|
||
| .lds-ellipsis div { | ||
| position: absolute; | ||
| top: 33.33333px; | ||
| width: 10px; | ||
| height: 10px; | ||
| border-radius: 50%; | ||
| background: currentColor; | ||
| animation-timing-function: cubic-bezier(0, 1, 1, 0); | ||
| } | ||
|
|
||
| .lds-ellipsis div:nth-child(1) { | ||
| left: 8px; | ||
| animation: lds-ellipsis1 0.6s infinite; | ||
| } | ||
|
|
||
| .lds-ellipsis div:nth-child(2) { | ||
| left: 8px; | ||
| animation: lds-ellipsis2 0.6s infinite; | ||
| } | ||
|
|
||
| .lds-ellipsis div:nth-child(3) { | ||
| left: 32px; | ||
| animation: lds-ellipsis2 0.6s infinite; | ||
| } | ||
|
|
||
| .lds-ellipsis div:nth-child(4) { | ||
| left: 56px; | ||
| animation: lds-ellipsis3 0.6s infinite; | ||
| } | ||
|
|
||
| @keyframes lds-ellipsis1 { | ||
| 0% { | ||
| transform: scale(0); | ||
| } | ||
|
|
||
| 100% { | ||
| transform: scale(1); | ||
| } | ||
| } | ||
|
|
||
| @keyframes lds-ellipsis3 { | ||
| 0% { | ||
| transform: scale(1); | ||
| } | ||
|
|
||
| 100% { | ||
| transform: scale(0); | ||
| } | ||
| } | ||
|
|
||
| @keyframes lds-ellipsis2 { | ||
| 0% { | ||
| transform: translate(0, 0); | ||
| } | ||
|
|
||
| 100% { | ||
| transform: translate(24px, 0); | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| @inherits LayoutComponentBase | ||
|
|
||
| @Body | ||
|
|
||
| <div id="blazor-error-ui" data-nosnippet> | ||
| An unhandled error has occurred. | ||
| <a href="." class="reload">Reload</a> | ||
| <span class="dismiss">🗙</span> | ||
| </div> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| #blazor-error-ui { | ||
| color-scheme: light only; | ||
| background: lightyellow; | ||
| bottom: 0; | ||
| box-shadow: 0 -1px 2px rgba(0, 0, 0, 0.2); | ||
| box-sizing: border-box; | ||
| display: none; | ||
| left: 0; | ||
| padding: 0.6rem 1.25rem 0.7rem 1.25rem; | ||
| position: fixed; | ||
| width: 100%; | ||
| z-index: 1000; | ||
| } | ||
|
|
||
| #blazor-error-ui .dismiss { | ||
| cursor: pointer; | ||
| position: absolute; | ||
| right: 0.75rem; | ||
| top: 0.5rem; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| <div class="surveyContainer"> | ||
| <div class="tool-icon" aria-hidden="true"> | ||
| <svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" class="size-5"> | ||
| <path stroke-linecap="round" stroke-linejoin="round" d="M16.862 4.487l1.687-1.688a1.875 1.875 0 112.652 2.652L10.582 16.07a4.5 4.5 0 01-1.897 1.13L6 18l.8-2.685a4.5 4.5 0 011.13-1.897l8.932-8.931zm0 0L19.5 7.125" /> | ||
| </svg> | ||
| </div> | ||
|
|
||
| <div> | ||
| How well is this template working for you? Please take a | ||
| <a target="_blank" href="https://aka.ms/dotnet-chat-templatePreview2-survey">brief survey</a> | ||
| and tell us what you think. | ||
| </div> | ||
| </div> |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Making the options mutually exclusive is good. We can follow a different approach though, similar to how we handle qdrant.