AI-102: Develop Generative AI solutions with Azure OpenAI Service

Get started with Azure OpenAI Service

- Create an Azure OpenAI Service

• ต้องขอ** Request Access to Azure OpenAI Service (microsoft.com)
• Account ส่วนตัวใช้ไม่ได้นะ และก็มันมี Region ให้เลือกใช้นะ

UPDATE 2024-SEP: Pay As You Go ใช้งานส่วนนี้ได้แล้วครับ

Azure OpenAI Studio - เอาไว้สร้าง Deployment จะได้ End Point ให้ Client ใช้

types of generative AI models

• GPT-4 models are the latest generation of generative pretrained (GPT) models that can generate natural language and code completions based on natural language prompts.
• GPT 3.5 models can generate natural language and code completions based on natural language prompts. In particular, GPT-35-turbo models are optimized for chat-based interactions and work well in most generative AI scenarios.
• Embeddings models convert text into numeric vectors, and are useful in language analytics scenarios such as comparing text sources for similarities.
• DALL-E models are used to generate images based on natural language prompts. Currently, DALL-E models are in preview. DALL-E models aren't listed in the Azure OpenAI Studio interface and don't need to be explicitly deployed.

Deploy Model

• Azure OpenAI Studio
• Azure CLI

az cognitiveservices account deployment create \
   -g OAIResourceGroup \
   -n MyOpenAIResource \
   --deployment-name MyModel \
   --model-name gpt-35-turbo \
   --model-version "0301"  \
   --model-format OpenAI \
   --sku-name "Standard" \
   --sku-capacity 1

• Rest API

Use prompts to get completions from models

รูปแบบที่เราให้ AI ช่วยได้

Completion quality เพิ่มยังไง

• prompt engineer จำได้ว่า 21 word จะได้ชัดเจน
• The model parameters
• fine tune model(RAG)

Test models

• Completions playground โดยสามารถปรับ Parameter ได้ ดังนี้ - Max Response - คุมค่าใช้จ่าย ไม่ให้ตอบมากเกินไป (Token) ถ้ามันเกินมันเปิดให้ AI ตอบต่อนะ แบบใน Chat GPT ที่เราเจอมันให้กด Continue อันนี้ฝั่ง OpenAI - Maximum length
  - Temperature - ตัวทำให้ AI มี Idea ในการตอบ Creative ดูจากที่มันเคย Train เอาไว้ มาตอบ แต่มันมี Prob ของแต่ละอันให้เราดูนะ
  - Top P - Control Diversity
  - Frequency penalty ให้ตอบอยู่ใน Model
  - Presence penalty ให้ไปเรื่องใหม่
  - Stop sequences: Make responses stop at a desired point, such as the end of a sentence or list. Specify up to four sequences where the model will stop generating further tokens in a response. The returned text won't contain the stop sequence.
  - Pre-response text: Insert text after the user’s input and before the model’s response. This can help prepare the model for a response.
  - Post-response text: Insert text after the model’s generated response to encourage further user input, as when modeling a conversation.
• Chat playground - conversation-in, message-out

Exercise - Get started with Azure OpenAI Service
Knowledge check - Knowledge check - Training | Microsoft Learn

Build natural language solutions with Azure OpenAI Service

เริ่มต้น

• สร้าง Azure OpenAI Resource
• สร้าง Deployment เลือก Model มี 3 กลุ่มหลักๆ
  - Generative Pre-trained Transformer (GPT)
  - Code
  - Embeddings
• ใช้งานต้องทำ promp ให้ดี Azure OpenAI Service - Azure OpenAI | Microsoft Learn
• เลือกใช้ Tools
  - REST API
  - SDK

ตอนใช้ REST API SDK ต้องมี YOUR_ENDPOINT_NAME + YOUR_DEPLOYMENT_NAME + YOUR_API_KEY

Sample

curl https://YOUR_ENDPOINT_NAME.openai.azure.com/openai/deployments/YOUR_DEPLOYMENT_NAME/chat/completions?api-version=2023-03-15-preview \
  -H "Content-Type: application/json" \
  -H "api-key: YOUR_API_KEY" \
  -d '{"messages":[{"role": "system", "content": "You are a helpful assistant, teaching people about AI."},
{"role": "user", "content": "Does Azure OpenAI support multiple languages?"},
{"role": "assistant", "content": "Yes, Azure OpenAI supports several languages, and can translate between them."},
{"role": "user", "content": "Do other Azure AI Services support translation too?"}]}'


========================================================================================

curl https://YOUR_ENDPOINT_NAME.openai.azure.com/openai/deployments/YOUR_DEPLOYMENT_NAME/embeddings?api-version=2022-12-01 \
  -H "Content-Type: application/json" \
  -H "api-key: YOUR_API_KEY" \
  -d "{\"input\": \"The food was delicious and the waiter...\"}"

SDK ต้องลง Client Lib ก่อนนะ C# Azure.AI.OpenAI ถ้าไปดูใน NuGet ออกเวอร์ชั้นเต็ม 2.0.0 เดือนนี้ครับ

และถ้าใช้ Open AI มีออกตัวเต็มเหมือนกัน Announcing the stable release of the official OpenAI library for .NET - .NET Blog (microsoft.com)

// Add OpenAI library
using Azure.AI.OpenAI;

// Define parameters and initialize the client
string endpoint = "<YOUR_ENDPOINT_NAME>";
string key = "<YOUR_API_KEY>";
string deploymentName = "<YOUR_DEPLOYMENT_NAME>"; 

OpenAIClient client = new OpenAIClient(new Uri(endpoint), new AzureKeyCredential(key));

// Build completion options object
ChatCompletionsOptions chatCompletionsOptions = new ChatCompletionsOptions()
{
    Messages =
    {
        new ChatRequestSystemMessage("You are a helpful AI bot."),
        new ChatRequestUserMessage("What is Azure OpenAI?"),
    },
    DeploymentName = deploymentName
};

// Send request to Azure OpenAI model
ChatCompletions response = client.GetChatCompletions(chatCompletionsOptions);

// Print the response
string completion = response.Choices[0].Message.Content;
Console.WriteLine("Response: " + completion + "\n");

Exercise - Integrate Azure OpenAI into your app
Knowledge check - Knowledge check - Training | Microsoft Learn

Apply prompt engineering with Azure OpenAI Service

-Understand prompt engineering

• Prompt engineering can also help mitigate bias and improve fairness in AI models. By designing prompts that are diverse and inclusive, we can ensure that the model isn't biased towards a particular group or perspective.
  - transparency note - บอกที่มาของข้อมูล use-case ที่ควรเอาไปใช้
  - Microsoft's AI principles
• ต้องรู้ก่อนว่าใช้ End Point แบบไหน Completion / ChatCompletion / Embeddings  Ref:
  - Quickstart - Get started using GPT-35-Turbo and GPT-4 with Azure OpenAI Service - Azure OpenAI Service | Microsoft Learn
  - Build Natural Language Solutions with Azure OpenAI Service | by Shriya Dhar | Medium
• Adjusting model parameters เช่น พวก temperature / top_p (top_probability)

-Write more effective prompts

• Provide clear instructions - เขียนให้ชัดเจน เคยได้ยินว่าต่ำสุด 21 คำ
• Format of instructions โดยให้ใช้ Use section markers บอกว่าแนะนำ โดยมีตัว --- or ### blocks

Translate the text into French

---
What's the weather going to be like today?
---

-Format of instructions

to improve accuracy

• Primary content - content that is the subject of the query, such as a sentence to translate or an article to summarize. คำตอบสรุป

---
<insert full article here, as primary content>
---

Summarize this article and identify three takeaways in a bulleted list

• supporting content - content that may alter the response, but isn't the focus or subject of the prompt. Examples of supporting content include things like names, preferences, future date to include in the response, and so on. Providing supporting content allows the model to respond more completely, accurately, and be more likely

---
<insert full email here, as primary content>
---
<the next line is the supporting content>
Topics I'm very interested in: AI, webinar dates, submission deadlines

Extract the key points from the above email, and put them in a bulleted list:

• grounding content - provide reliable answers by providing content for the model to draw answer from. Grounding content could be an essay or article that you then ask questions about, a company FAQ document, or information
  ทำให้คำตอบมีความน่าเชื่อถือ

---
<insert unpublished paper on the history of AI here, as grounding content>
---

Where and when did the field of AI start?

• Cues คำแนะนำคำตอบให้ AI เช่น แสดงเป็นตาราง ให้เขียน SQL SELECT

Write a join query to get customer names with purchases in the past 30 days between tables named orders and customer on customer ID. 

SELECT

System message - To give the model instructions, perspective, or other information helpful to guide its response //บอกหน้าที่ของ AI อยู่ในสภาพ สถานการณ์ไหน

var chatCompletionsOptions = new ChatCompletionsOptions()
{
    Messages =
    {
        new ChatRequestSystemMessage("You are a casual, helpful assistant. You will talk like an American old western film character."),
        new ChatRequestUserMessage("Can you direct me to the library?")
    }
};

Ref: Provide context to improve accuracy - Training | Microsoft Learn

Conversation history - Enable the model to continue responding in a similar way and allow the user to reference previous content in subsequent queries

Few shot learning - ให้ AI เรียนรู้จากตัวอย่าง

Exercise - Utilize prompt engineering in your application
Knowledge check - Knowledge check - Training | Microsoft Learn

Generate code with Azure OpenAI Service

บทนี้ มันจะไปแนวๆ GitHub Co-pilot มาจากตัว codex models มันทำอะไรได้

• Write functions
• Change coding language
• Understand unknown code
• Complete partial code
• Write Unit Tests
• Add comments and generate documentation
• Fix bugs in your code + Explain
• Improve performance
• Refactor inefficient code

ชอบส่วนตัว Fix bugs and improve your code - Training | Microsoft Learn

Exercise - Generate and improve code with Azure OpenAI Service
Knowledge check - Knowledge check - Training | Microsoft Learn

Generate images with Azure OpenAI Service

Model DALL-E สร้างรูปจาก Text

• ใน Azure OpenAI Studio ****มี Playground ให้ลองเล่น
• หรือจะใช้ REST API
  - prompt = ข้อความที่ต้องการ Gen
  - n = number of images to be generated.
  - size = ขนาดรูป

{
    "prompt": "A badger wearing a tuxedo",
    "n": 1,
    "size": "512x512"
}

Exercise - Generate images with a DALL-E model
Knowledge check - Knowledge check - Training | Microsoft Learn

Implement Retrieval Augmented Generation (RAG) with Azure OpenAI Service

Azure OpenAI on your data goes through the following steps:

1. Receive user prompt.
2. Determine relevant content and intent of the prompt.
3. Query the search index with that content and intent.
4. Insert search result chunk into the Azure OpenAI prompt, along with system message and user prompt.
5. Send entire prompt to Azure OpenAI.
6. Return response and data reference (if any) to the user.

Fine-tuning - เอา LLM มาเรียนเพิ่มในเรื่องใดเรื่องนึง แต่จะลดประสิทธิภาพ

• a technique used to create a custom model by training an existing foundational model such as gpt-35-turbo with a dataset of additional training data.
• can result in higher quality requests than prompt engineering alone

RAG with Azure OpenAI - RAG เอา Model ของ LLM เดิมมาอ่านโพย

• removes the requirement of training a custom model with your data and simplifies the interaction with the AI model
• RAG requires a mechanism to retrieve and integrate external data, which can be resource intensive
• Add your own data source
  - Azure OpenAI Studio and select the Add your data tab in the Assistant setup pane
  - Connecting to files in a storage account / data supports .md, .txt, .html, .pdf, and Microsoft Word or PowerPoint files
• recommended 
  - Using Azure OpenAI Studio allows the appropriate chunking to happen when inserting into the index, yielding better responses.
  - Breaking down the task and using chain of thought prompting can help the model respond more effectively within the token limit. //แบ่ง Task ให้เล็ก และชัดเจนกับ Chatbot

Ref: Chat with your model using your own data - Training | Microsoft Learn

Note

• system message gets truncated if it exceeds 200 tokens.
• The response from the model is also limited when using your own data is 1500 tokens.
• ต้องใช้ chain of thought ลดข้อจำกัดของ Token

Exercise - Add your data for RAG with Azure OpenAI Service
Knowledge Check - Knowledge check - Training | Microsoft Learn

Fundamentals of Responsible Generative AI

The four stages

1. Identify potential harms that are relevant to your planned solution.
   - Identify potential harms
   - Prioritize identified harms
   - Test and verify the presence (การมีอยู่ โอกาศการเกิด) of harms - Document and share the verified harms

2. Measure the presence of these harms in the outputs generated by your solution.
   - Prepare a diverse selection of input prompts that are likely to result in each potential harm
   - Submit the prompts to the system and retrieve the generated output.
   - Apply pre-defined criteria to evaluate the output and categorize it according to the level of potential harm
   ตรงนี้ นอกจากทำ Manual อาจจะเอา Automate Tool มาช่วยก็ได้ แบบ AI ตรวจ AI

3. Mitigate the harms at multiple layers in your solution to minimize their presence and impact, and ensure transparent communication about potential risks to users. มันมีหลาย Layer นะ

1. Model Layer เลือก Model ให้เหมาะกับงาน + อาจจะ Fine-Tune
2. Safety System Layer
   - content filters ตรวจ output อันตรายไหม ภาพปืนจ่อหัวให้เอาออก
   - security ปกติ เช่น กัน DDOS / Request ซ้ำๆ
3. -Metaprompt and grounding
   - metaprompts or system inputs บอก model มันอยู่ในจุดไหน มันต้องทำอะไร ภายใต้สถานการณ์ไหน
   - add grounding data to input prompts
   - RAG ดึงข้อมูลจากโพย
4. User experience กันจาก UI

4. Operate the solution responsibly by defining and following a deployment and operational readiness plan.
   • Complete prerelease reviews Legal / Privacy / Security / Accessibility ดูตามกฏข้อบังคับต่างๆ
   • Release and operate the solution
     - Canary - AB Test
     - Create an incident response plan จะได้รู้ว่าต้องใช้เวลาจัดการประมาณไหน
     - Create a rollback plan
     - Implement the capability to immediately block harmful system responses when they're discovered.
     - กลไลอื่นๆ อย่าง Block User / IP เป็นต้น
     - Observability - telemetry

Exercise - Explore content filters in Azure OpenAI
Knowledge check - Knowledge check - Training | Microsoft Learn

Apply Skill Develop generative AI solutions with Azure OpenAI Service - Applied Skills | Microsoft Learn