Semantic Kernel | 使用 Phi-3 SLM

前言

在連接 LM Studio Local Phi-2 Server中透過LM Studio下載 Phi-2 來運行。
之前透過AddAzureOpenAIChatCompletion還需要實作HttpClientHandler。
看到Run Phi-3 SLM on your machine with C# Semantic Kernel and Ollama後，
所以改用AddOpenAIChatCompletion就不需要自行去實作HttpClientHandler。

實作

最近可以下載Phi-3，所以透過LM Studio下載Phi-3來測試。

從上圖可以找到，對話的 Endpoint 是 http://localhost:1234/v1/chat/completions，
apikey 是lm-studio，modelId 或是 deploymentName 可以自由給，
所以程式改寫如下，

using Microsoft.SemanticKernel;
using System.Text;
using Microsoft.SemanticKernel.ChatCompletion;

Console.OutputEncoding = Encoding.UTF8;
var deploymentName = "phi-3";
var endpoint = "http://localhost:1234/v1/chat/completions";
var apikey = "lm-studio";
var kernelBuilder = Kernel.CreateBuilder();
//kernelBuilder.AddAzureOpenAIChatCompletion(deploymentName, endpoint, apikey, httpClient: httpClient);
kernelBuilder.AddOpenAIChatCompletion(modelId: deploymentName, endpoint: new Uri(endpoint), apiKey:apikey);
var kernel = kernelBuilder.Build();
var ai = kernel.GetRequiredService<IChatCompletionService>();
ChatHistory chat = new("請以繁體中文並簡要地回答問題，不用說明。");
StringBuilder builder = new();
while (true)
{
    Console.Write("Question: ");
    chat.AddUserMessage(Console.ReadLine()!);
    builder.Clear();
    await foreach (var message in ai.GetStreamingChatMessageContentsAsync(chat, kernel: kernel))
    {
        Console.Write(message);
        builder.Append(message.Content);
    }
    Console.WriteLine();
    chat.AddAssistantMessage(builder.ToString());
    Console.WriteLine();
}

執行結果如下，

或是加入Microsoft.SemanticKernel.Connectors.HuggingFace套件，
使用AddHuggingFaceChatCompletion Method 也可以哦，如下，

using Microsoft.SemanticKernel;
using System.Text;
using Microsoft.SemanticKernel.ChatCompletion;

Console.OutputEncoding = Encoding.UTF8;
var deploymentName = "phi-3";
var endpoint = "http://localhost:1234";
var apikey = "lm-studio";
var kernelBuilder = Kernel.CreateBuilder();
//kernelBuilder.AddAzureOpenAIChatCompletion(deploymentName, endpoint, apikey, httpClient: httpClient);
//kernelBuilder.AddOpenAIChatCompletion(modelId: deploymentName, endpoint: new Uri(endpoint), apiKey: apikey);
kernelBuilder.AddHuggingFaceChatCompletion(model: deploymentName, endpoint: new Uri(endpoint), apiKey: apikey);
var kernel = kernelBuilder.Build();
var ai = kernel.GetRequiredService<IChatCompletionService>();
ChatHistory chat = new("請以繁體中文並簡要地回答問題，不用說明。");
StringBuilder builder = new();
while (true)
{
    Console.Write("Question: ");
    chat.AddUserMessage(Console.ReadLine()!);
    builder.Clear();
    await foreach (var message in ai.GetStreamingChatMessageContentsAsync(chat, kernel: kernel))
    {
        Console.Write(message);
        builder.Append(message.Content);
    }
    Console.WriteLine();
    chat.AddAssistantMessage(builder.ToString());
    Console.WriteLine();
}

• 註: endpoint 各 Method 都有所不同，所以使用上要注意。

參考資源

Run Phi-3 SLM on your machine with C# Semantic Kernel and Ollama
連接 LM Studio Local Phi-2 Server

一門生意兩門做 - 手法不同: 一樣的買賣，兩種手段。個人有各人的手法。