반응형
블로그 이미지
개발자로서 현장에서 일하면서 새로 접하는 기술들이나 알게된 정보 등을 정리하기 위한 블로그입니다. 운 좋게 미국에서 큰 회사들의 프로젝트에서 컬설턴트로 일하고 있어서 새로운 기술들을 접할 기회가 많이 있습니다. 미국의 IT 프로젝트에서 사용되는 툴들에 대해 많은 분들과 정보를 공유하고 싶습니다.
솔웅

최근에 올라온 글

최근에 달린 댓글

최근에 받은 트랙백

글 보관함

카테고리

LC-Modules-Agents Quickstart

2024. 2. 14. 12:41 | Posted by 솔웅


반응형

https://python.langchain.com/docs/modules/agents/quick_start

 

Quickstart | 🦜️🔗 Langchain

quickstart}

python.langchain.com

 

 

 

Quickstart

To best understand the agent framework, let’s build an agent that has two tools: one to look things up online, and one to look up specific data that we’ve loaded into a index.

 

에이전트 프레임워크를 가장 잘 이해하기 위해 두 가지 도구가 있는 에이전트를 구축해 보겠습니다. 하나는 온라인으로 항목을 조회하는 도구이고 다른 하나는 인덱스에 로드한 특정 데이터를 조회하는 도구입니다.

 

This will assume knowledge of LLMs and retrieval so if you haven’t already explored those sections, it is recommended you do so.

 

이는 LLM 및 검색에 대한 지식을 가정하므로 해당 섹션을 아직 탐색하지 않았다면 탐색하는 것이 좋습니다.

 

Setup: LangSmith

By definition, agents take a self-determined, input-dependent sequence of steps before returning a user-facing output. This makes debugging these systems particularly tricky, and observability particularly important. LangSmith is especially useful for such cases.

 

정의에 따르면 에이전트는 사용자에게 표시되는 출력을 반환하기 전에 입력에 따라 자체 결정된 일련의 단계를 수행합니다. 이로 인해 이러한 시스템을 디버깅하는 것이 특히 까다로워지고 관찰 가능성이 특히 중요해집니다. LangSmith는 이러한 경우에 특히 유용합니다.

 

When building with LangChain, all steps will automatically be traced in LangSmith. To set up LangSmith we just need set the following environment variables:

 

LangChain으로 구축할 때 모든 단계는 LangSmith에서 자동으로 추적됩니다. LangSmith를 설정하려면 다음 환경 변수만 설정하면 됩니다.

 

export LANGCHAIN_TRACING_V2="true"
export LANGCHAIN_API_KEY="<your-api-key>"

 

 

Define tools

We first need to create the tools we want to use. We will use two tools: Tavily (to search online) and then a retriever over a local index we will create

 

먼저 사용하려는 도구를 만들어야 합니다. 우리는 두 가지 도구를 사용할 것입니다: Tavily(온라인 검색용)와 우리가 생성할 로컬 인덱스에 대한 검색기

 

Tavily

We have a built-in tool in LangChain to easily use Tavily search engine as tool. Note that this requires an API key - they have a free tier, but if you don’t have one or don’t want to create one, you can always ignore this step.

 

우리는 Tavily 검색 엔진을 도구로 쉽게 사용할 수 있도록 LangChain에 내장된 도구를 가지고 있습니다. 이를 위해서는 API 키가 필요합니다. 무료 등급이 있지만 API 키가 없거나 생성하고 싶지 않은 경우 언제든지 이 단계를 무시할 수 있습니다.

 

Once you create your API key, you will need to export that as:

 

API 키를 생성한 후에는 이를 다음과 같이 내보내야 합니다.

 

export TAVILY_API_KEY="..."
from langchain_community.tools.tavily_search import TavilySearchResults
search = TavilySearchResults()
search.invoke("what is the weather in SF")
[{'url': 'https://www.metoffice.gov.uk/weather/forecast/9q8yym8kr',
  'content': 'Thu 11 Jan Thu 11 Jan Seven day forecast for San Francisco  San Francisco (United States of America) weather Find a forecast  Sat 6 Jan Sat 6 Jan Sun 7 Jan Sun 7 Jan Mon 8 Jan Mon 8 Jan Tue 9 Jan Tue 9 Jan Wed 10 Jan Wed 10 Jan Thu 11 Jan  Find a forecast Please choose your location from the nearest places to : Forecast days Today Today Sat 6 Jan Sat 6 JanSan Francisco 7 day weather forecast including weather warnings, temperature, rain, wind, visibility, humidity and UV ... (11 January 2024) Time 00:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00 08:00 09:00 10:00 11:00 12:00 ... Oakland Int. 11.5 miles; San Francisco International 11.5 miles; Corte Madera 12.3 miles; Redwood City 23.4 miles;'},
 {'url': 'https://www.latimes.com/travel/story/2024-01-11/east-brother-light-station-lighthouse-california',
  'content': "May 18, 2023  Jan. 4, 2024 Subscribe for unlimited accessSite Map Follow Us MORE FROM THE L.A. TIMES  Jan. 8, 2024 Travel & Experiences This must be Elysian Valley (a.k.a. Frogtown) Jan. 5, 2024 Food  June 30, 2023The East Brother Light Station in the San Francisco Bay is not a destination for everyone. ... Jan. 11, 2024 3 AM PT ... Champagne and hors d'oeuvres are served in late afternoon — outdoors if ..."}]

 

Retriever

We will also create a retriever over some data of our own. For a deeper explanation of each step here, see this section.

 

우리는 또한 우리 자신의 일부 데이터에 대한 검색기를 만들 것입니다. 각 단계에 대한 자세한 설명은 이 섹션을 참조하세요.

 

from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.document_loaders import WebBaseLoader
from langchain_community.vectorstores import FAISS
from langchain_openai import OpenAIEmbeddings

loader = WebBaseLoader("https://docs.smith.langchain.com/overview")
docs = loader.load()
documents = RecursiveCharacterTextSplitter(
    chunk_size=1000, chunk_overlap=200
).split_documents(docs)
vector = FAISS.from_documents(documents, OpenAIEmbeddings())
retriever = vector.as_retriever()
retriever.get_relevant_documents("how to upload a dataset")[0]
Document(page_content="dataset uploading.Once we have a dataset, how can we use it to test changes to a prompt or chain? The most basic approach is to run the chain over the data points and visualize the outputs. Despite technological advancements, there still is no substitute for looking at outputs by eye. Currently, running the chain over the data points needs to be done client-side. The LangSmith client makes it easy to pull down a dataset and then run a chain over them, logging the results to a new project associated with the dataset. From there, you can review them. We've made it easy to assign feedback to runs and mark them as correct or incorrect directly in the web app, displaying aggregate statistics for each test project.We also make it easier to evaluate these runs. To that end, we've added a set of evaluators to the open-source LangChain library. These evaluators can be specified when initiating a test run and will evaluate the results once the test run completes. If we’re being honest, most of", metadata={'source': 'https://docs.smith.langchain.com/overview', 'title': 'LangSmith Overview and User Guide | 🦜️🛠️ LangSmith', 'description': 'Building reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.', 'language': 'en'})

 

Now that we have populated our index that we will do doing retrieval over, we can easily turn it into a tool (the format needed for an agent to properly use it)

 

이제 검색을 수행할 인덱스를 채웠으므로 이를 도구(에이전트가 올바르게 사용하는 데 필요한 형식)로 쉽게 전환할 수 있습니다.

 

from langchain.tools.retriever import create_retriever_tool
retriever_tool = create_retriever_tool(
    retriever,
    "langsmith_search",
    "Search for information about LangSmith. For any questions about LangSmith, you must use this tool!",
)

 

 

Tools

Now that we have created both, we can create a list of tools that we will use downstream.

 

이제 두 가지를 모두 만들었으므로 다운스트림에서 사용할 도구 목록을 만들 수 있습니다.

 

tools = [search, retriever_tool]

 

 

Create the agent

Now that we have defined the tools, we can create the agent. We will be using an OpenAI Functions agent - for more information on this type of agent, as well as other options, see this guide

 

Agent Types | 🦜️🔗 Langchain

This categorizes all the available agents along a few dimensions.

python.langchain.com

 

이제 도구를 정의했으므로 에이전트를 만들 수 있습니다. OpenAI Functions 에이전트를 사용할 것입니다. 이러한 유형의 에이전트와 기타 옵션에 대한 자세한 내용은 이 가이드를 참조하세요.

 

First, we choose the LLM we want to be guiding the agent.

 

먼저 에이전트를 안내할 LLM을 선택합니다.

 

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)

 

Next, we choose the prompt we want to use to guide the agent.

 

다음으로 상담원을 안내하는 데 사용할 프롬프트를 선택합니다.

 

 

If you want to see the contents of this prompt and have access to LangSmith, you can go to:

 

이 프롬프트의 내용을 보고 LangSmith에 액세스하려면 다음으로 이동하세요.

 

https://smith.langchain.com/hub/hwchase17/openai-functions-agent

 

LangSmith

 

smith.langchain.com

 

from langchain import hub

# Get the prompt to use - you can modify this!
prompt = hub.pull("hwchase17/openai-functions-agent")
prompt.messages

 

[SystemMessagePromptTemplate(prompt=PromptTemplate(input_variables=[], template='You are a helpful assistant')),
 MessagesPlaceholder(variable_name='chat_history', optional=True),
 HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['input'], template='{input}')),
 MessagesPlaceholder(variable_name='agent_scratchpad')]

 

Now, we can initalize the agent with the LLM, the prompt, and the tools. The agent is responsible for taking in input and deciding what actions to take. Crucially, the Agent does not execute those actions - that is done by the AgentExecutor (next step). For more information about how to think about these components, see our conceptual guide

 

Concepts | 🦜️🔗 Langchain

The core idea of agents is to use a language model to choose a sequence of actions to take.

python.langchain.com

 

 

이제 LLM, 프롬프트 및 도구를 사용하여 에이전트를 초기화할 수 있습니다. 에이전트는 입력을 받아들이고 어떤 조치를 취할지 결정하는 역할을 담당합니다. 결정적으로 에이전트는 이러한 작업을 실행하지 않습니다. 이는 AgentExecutor(다음 단계)에 의해 수행됩니다. 이러한 구성 요소에 대해 생각하는 방법에 대한 자세한 내용은 개념 가이드를 참조하세요.

 

from langchain.agents import create_openai_functions_agent

agent = create_openai_functions_agent(llm, tools, prompt)

 

Finally, we combine the agent (the brains) with the tools inside the AgentExecutor (which will repeatedly call the agent and execute tools). For more information about how to think about these components, see our conceptual guide

 

Concepts | 🦜️🔗 Langchain

The core idea of agents is to use a language model to choose a sequence of actions to take.

python.langchain.com

 

마지막으로 에이전트(브레인)를 AgentExecutor 내부 도구(반복적으로 에이전트를 호출하고 도구를 실행함)와 결합합니다. 이러한 구성 요소에 대해 생각하는 방법에 대한 자세한 내용은 개념 가이드를 참조하세요.

 

from langchain.agents import AgentExecutor

agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

 

Run the agent

We can now run the agent on a few queries! Note that for now, these are all stateless queries (it won’t remember previous interactions).

 

이제 몇 가지 쿼리에 대해 에이전트를 실행할 수 있습니다! 현재로서는 이는 모두 상태 비저장 쿼리입니다(이전 상호 작용을 기억하지 않음).

 

agent_executor.invoke({"input": "hi!"})

 


> Entering new AgentExecutor chain...
Hello! How can I assist you today?

> Finished chain.

 

{'input': 'hi!', 'output': 'Hello! How can I assist you today?'}

 

agent_executor.invoke({"input": "how can langsmith help with testing?"})

 


> Entering new AgentExecutor chain...

Invoking: `langsmith_search` with `{'query': 'LangSmith testing'}`


[Document(page_content='LangSmith Overview and User Guide | 🦜️🛠️ LangSmith', metadata={'source': 'https://docs.smith.langchain.com/overview', 'title': 'LangSmith Overview and User Guide | 🦜️🛠️ LangSmith', 'description': 'Building reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.', 'language': 'en'}), Document(page_content='Skip to main content🦜️🛠️ LangSmith DocsPython DocsJS/TS DocsSearchGo to AppLangSmithOverviewTracingTesting & EvaluationOrganizationsHubLangSmith CookbookOverviewOn this pageLangSmith Overview and User GuideBuilding reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.Over the past two months, we at LangChain have been building and using LangSmith with the goal of bridging this gap. This is our tactical user guide to outline effective ways to use LangSmith and maximize its benefits.On by default\u200bAt LangChain, all of us have LangSmith’s tracing running in the background by default. On the Python side, this is achieved by setting environment variables, which we establish whenever we launch a virtual environment or open our bash shell and leave them set. The same principle applies to most JavaScript', metadata={'source': 'https://docs.smith.langchain.com/overview', 'title': 'LangSmith Overview and User Guide | 🦜️🛠️ LangSmith', 'description': 'Building reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.', 'language': 'en'}), Document(page_content='You can also quickly edit examples and add them to datasets to expand the surface area of your evaluation sets or to fine-tune a model for improved quality or reduced costs.Monitoring\u200bAfter all this, your app might finally ready to go in production. LangSmith can also be used to monitor your application in much the same way that you used for debugging. You can log all traces, visualize latency and token usage statistics, and troubleshoot specific issues as they arise. Each run can also be assigned string tags or key-value metadata, allowing you to attach correlation ids or AB test variants, and filter runs accordingly.We’ve also made it possible to associate feedback programmatically with runs. This means that if your application has a thumbs up/down button on it, you can use that to log feedback back to LangSmith. This can be used to track performance over time and pinpoint under performing data points, which you can subsequently add to a dataset for future testing — mirroring the', metadata={'source': 'https://docs.smith.langchain.com/overview', 'title': 'LangSmith Overview and User Guide | 🦜️🛠️ LangSmith', 'description': 'Building reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.', 'language': 'en'}), Document(page_content='inputs, and see what happens. At some point though, our application is performing\nwell and we want to be more rigorous about testing changes. We can use a dataset\nthat we’ve constructed along the way (see above). Alternatively, we could spend some\ntime constructing a small dataset by hand. For these situations, LangSmith simplifies', metadata={'source': 'https://docs.smith.langchain.com/overview', 'title': 'LangSmith Overview and User Guide | 🦜️🛠️ LangSmith', 'description': 'Building reliable LLM applications can be challenging. LangChain simplifies the initial setup, but there is still work needed to bring the performance of prompts, chains and agents up the level where they are reliable enough to be used in production.', 'language': 'en'})]LangSmith can help with testing in several ways. Here are some ways LangSmith can assist with testing:

1. Tracing: LangSmith provides tracing capabilities that can be used to monitor and debug your application during testing. You can log all traces, visualize latency and token usage statistics, and troubleshoot specific issues as they arise.

2. Evaluation: LangSmith allows you to quickly edit examples and add them to datasets to expand the surface area of your evaluation sets. This can help you test and fine-tune your models for improved quality or reduced costs.

3. Monitoring: Once your application is ready for production, LangSmith can be used to monitor your application. You can log feedback programmatically with runs, track performance over time, and pinpoint underperforming data points. This information can be used to improve your application and add to datasets for future testing.

4. Rigorous Testing: When your application is performing well and you want to be more rigorous about testing changes, LangSmith can simplify the process. You can use existing datasets or construct small datasets by hand to test different scenarios and evaluate the performance of your application.

For more detailed information on how to use LangSmith for testing, you can refer to the [LangSmith Overview and User Guide](https://docs.smith.langchain.com/overview).

> Finished chain.

 

{'input': 'how can langsmith help with testing?',
 'output': 'LangSmith can help with testing in several ways. Here are some ways LangSmith can assist with testing:\n\n1. Tracing: LangSmith provides tracing capabilities that can be used to monitor and debug your application during testing. You can log all traces, visualize latency and token usage statistics, and troubleshoot specific issues as they arise.\n\n2. Evaluation: LangSmith allows you to quickly edit examples and add them to datasets to expand the surface area of your evaluation sets. This can help you test and fine-tune your models for improved quality or reduced costs.\n\n3. Monitoring: Once your application is ready for production, LangSmith can be used to monitor your application. You can log feedback programmatically with runs, track performance over time, and pinpoint underperforming data points. This information can be used to improve your application and add to datasets for future testing.\n\n4. Rigorous Testing: When your application is performing well and you want to be more rigorous about testing changes, LangSmith can simplify the process. You can use existing datasets or construct small datasets by hand to test different scenarios and evaluate the performance of your application.\n\nFor more detailed information on how to use LangSmith for testing, you can refer to the [LangSmith Overview and User Guide](https://docs.smith.langchain.com/overview).'}

 

 

Adding in memory

 

As mentioned earlier, this agent is stateless. This means it does not remember previous interactions. To give it memory we need to pass in previous chat_history. Note: it needs to be called chat_history because of the prompt we are using. If we use a different prompt, we could change the variable name

 

앞서 언급했듯이 이 에이전트는 Stateless입니다. 이는 이전 상호작용을 기억하지 못한다는 것을 의미합니다. 메모리를 제공하려면 이전 chat_history를 전달해야 합니다. 참고: 우리가 사용하는 프롬프트 때문에 chat_history라고 불러야 합니다. 다른 프롬프트를 사용하면 변수 이름을 변경할 수 있습니다.

 

# Here we pass in an empty list of messages for chat_history because it is the first message in the chat
agent_executor.invoke({"input": "hi! my name is bob", "chat_history": []})

> Entering new AgentExecutor chain...
Hello Bob! How can I assist you today?

> Finished chain.
{'input': 'hi! my name is bob',
 'chat_history': [],
 'output': 'Hello Bob! How can I assist you today?'}
from langchain_core.messages import AIMessage, HumanMessage
agent_executor.invoke(
    {
        "chat_history": [
            HumanMessage(content="hi! my name is bob"),
            AIMessage(content="Hello Bob! How can I assist you today?"),
        ],
        "input": "what's my name?",
    }
)

 


> Entering new AgentExecutor chain...
Your name is Bob.

> Finished chain.
{'chat_history': [HumanMessage(content='hi! my name is bob'),
  AIMessage(content='Hello Bob! How can I assist you today?')],
 'input': "what's my name?",
 'output': 'Your name is Bob.'}

 

 

If we want to keep track of these messages automatically, we can wrap this in a RunnableWithMessageHistory. For more information on how to use this, see this guide

 

Add message history (memory) | 🦜️🔗 Langchain

The RunnableWithMessageHistory let us add message history to certain

python.langchain.com

 

이러한 메시지를 자동으로 추적하려면 이를 RunnableWithMessageHistory로 래핑할 수 있습니다. 이를 사용하는 방법에 대한 자세한 내용은 이 가이드를 참조하세요.

 

from langchain_community.chat_message_histories import ChatMessageHistory
from langchain_core.runnables.history import RunnableWithMessageHistory
message_history = ChatMessageHistory()
agent_with_chat_history = RunnableWithMessageHistory(
    agent_executor,
    # This is needed because in most real world scenarios, a session id is needed
    # It isn't really used here because we are using a simple in memory ChatMessageHistory
    lambda session_id: message_history,
    input_messages_key="input",
    history_messages_key="chat_history",
)
agent_with_chat_history.invoke(
    {"input": "hi! I'm bob"},
    # This is needed because in most real world scenarios, a session id is needed
    # It isn't really used here because we are using a simple in memory ChatMessageHistory
    config={"configurable": {"session_id": "<foo>"}},
)

> Entering new AgentExecutor chain...
Hello Bob! How can I assist you today?

> Finished chain.
{'input': "hi! I'm bob",
 'chat_history': [],
 'output': 'Hello Bob! How can I assist you today?'}
agent_with_chat_history.invoke(
    {"input": "what's my name?"},
    # This is needed because in most real world scenarios, a session id is needed
    # It isn't really used here because we are using a simple in memory ChatMessageHistory
    config={"configurable": {"session_id": "<foo>"}},
)

> Entering new AgentExecutor chain...
Your name is Bob.

> Finished chain.
{'input': "what's my name?",
 'chat_history': [HumanMessage(content="hi! I'm bob"),
  AIMessage(content='Hello Bob! How can I assist you today?')],
 'output': 'Your name is Bob.'}

 

Conclusion

That’s a wrap! In this quick start we covered how to create a simple agent. Agents are a complex topic, and there’s lot to learn! Head back to the main agent page to find more resources on conceptual guides, different types of agents, how to create custom tools, and more!

 

마무리입니다! 이 빠른 시작에서는 간단한 에이전트를 만드는 방법을 다루었습니다. 에이전트는 복잡한 주제이며 배울 것이 많습니다! 기본 에이전트 페이지로 돌아가서 개념 가이드, 다양한 유형의 에이전트, 사용자 정의 도구를 만드는 방법 등에 대한 추가 리소스를 찾아보세요!

반응형


반응형

시애틀에서 100키로미터 쯤 떨어진 곳에 Olympia 라는 곳이 있습니다.

그곳에서 1년동안 은둔 생활을 하면서 틈틈이 AI를 공부했습니다.

 

그 공부한 내용을 정리하는 의미에서 유투브 채널을 만들 결심을 했습니다.

드디어 첫번째 클립이 업로드 됐어요.

 

채널 이름은 CatchupAI (https://www.youtube.com/@catchupai).

AI 따라잡기 입니다.

10년전 미국에 와서 IT 기술 따라잡기라는 이름으로 이 티스토리 블로그 활동을 처음 시작했는데요.

이제 AI 따라잡기가 다시 시작 됐습니다.

 

https://youtu.be/G_X-NtEZ0us?si=heJ5Vz1pa2S9qa7T

 

 

작년 중순에 시애틀 지역 한인 IT 엔지니어 모임인 창발이라는 곳에서 AI Study Group에 가입해서 3개월간 Deep Learning Fundamental을 공부 했었어요.

교재는 Dive into Deep Learning 이었구요.

 

그 내용을 기반으로 Deep Learning Fundamental 을 주제로 PPT를 만들었었습니다.

 

한두번 이 PPT를 가지고 발표 할 기회가 있었는데요.

 

그 내용을 유투브로 만들었습니다.

 

그 중 첫번째 클립이 업로드 된 겁니다.

많은 시청 바라구요. 구독과 좋아요 꼭 부탁 드립니다.

 

 

이 클립은 첫번째 Introduce (Target) 과 추가로 인간과 AI 가 이 세상을 인식하는 방법을 비교하는 내용도 있습니다.

 

 

AI는 이 세상을 어떻게 이해하고 그것이 인간과 무엇이 다르고 무엇이 같은지 알려 드립니다.

대부분 같지만 약간 다른점이 있는데 그것이 무엇인지 위 유투브 보시고 확인해 보세요.

 

그 다른점 때문에 생성형 AI의 단점인 Hallucination이 발생하게 되는 겁니다.

 

 

그리고 AI 는 어떻게 학습을 하는지 아주 간단한 예제로 AI가 학습하는 과정을 그대로 따라해 봤습니다.

Back propagation이 무엇인지 그리고 그 방법 때문에 ChatGPT 같은 아주 놀라운 성능의 AI 제품이 나올 수 있었지만 동시에 Freshness problem이라는 한계점도 가지고 있다는 점.

 

위 유투브 내용을 보시면 들어 보지 못했던 흥미로운 점을 이해하실 수 있으실 거예요.

 

시간과 열정을 가지고 열심히 공부해서 나름대로 이해하고 깨달은 내용을 정리했습니다.

많은 관심과 구독 좋아요 부탁드립니다.

 

Note: Deep Learning Fundamental 강좌가 끝나면 지금 공부하고 있는 AI Web App 개발과 관련된 내용도 이어질 예정입니다.

 

제가 지금 개발하고 있는 저만의 AI Web App Portfolio는 아래 웹사이트로 가면 보실 수 있습니다.

 

https://catchupai.streamlit.app/

 

Catchup AI Portfolio

This app was built in Streamlit! Check it out and visit https://streamlit.io for more awesome community apps. 🎈

catchupai.streamlit.app

 

반응형


반응형

https://openai.com/blog/new-embedding-models-and-api-updates

 

New embedding models and API updates

We are launching a new generation of embedding models, new GPT-4 Turbo and moderation models, new API usage management tools, and soon, lower pricing on GPT-3.5 Turbo.

openai.com

 

 

Open AI 에서는 2024년 1월 25일 new generation of embedding models, new GPT-4 Turbo, moderation models, new API usage management tools 를 발표 했습니다.

 

그리고 조만간 더 저렴한 GPT-3.5 Turbo 모델을 발표할 예정입니다.

 

이들 새로운 모델에는 다음과 같은 것 들이 포함 됩니다.

 

  • Two new embedding models
  • An updated GPT-4 Turbo preview model 
  • An updated GPT-3.5 Turbo model
  • An updated text moderation model

 

새로운 임베딩 모델은 더 낮은 가격으로 이용할 수 있습니다.

새로운 임베딩 모델은 아래와 같습니다.

 

* Small embedding model

 

- text-embedding-3-small : 고효율 임베딩 모델, 이전 모델인 text-embedding-ada-002 보다 업그레이드 됨

 

더욱 강력한 성능. text-embedding-ada-002와 text-embedding-3-small을 비교하면 일반적으로 사용되는 다중 언어 검색 벤치마크(MIRACL)의 평균 점수가 31.4%에서 44.0%로 증가했습니다. 영어 과제 벤치마크(MTEB)가 61.0%에서 62.3%로 증가했습니다.

 

인하 된 가격. text-embedding-3-small은 이전 세대 text-embedding-ada-002 모델보다 훨씬 더 효율적입니다. text-embedding-3-small의 가격은 text-embedding-ada-002에 비해 1,000개 토큰당 가격이 $0.0001에서 $0.00002로 5배 인하되었습니다.

 

* Large text embedding model

 

- text-embedding-3-large : 최고 성능 모델. 

text-embedding-3-large는 새로운 차세대 대형 임베딩 모델이며 최대 3072차원의 임베딩을 생성합니다.

 

더욱 강력한 성능. text-embedding-3-large는 새로운 최고 성능 모델입니다. text-embedding-ada-002와 text-embedding-3-large 비교: MIRACL에서는 평균 점수가 31.4%에서 54.9%로 증가한 반면, MTEB에서는 평균 점수가 61.0%에서 64.6%로 증가했습니다.

 

 

text-embedding-3-large의 가격은 $0.00013/1,000토큰입니다.
Embeddings guide 에서 새로운 임베딩 모델 사용에 대해 자세히 알아볼 수 있습니다.

 

Native support for shortening embeddings

Using larger embeddings, for example storing them in a vector store for retrieval, generally costs more and consumes more compute, memory and storage than using smaller embeddings.

 

예를 들어 검색을 위해 벡터 저장소에 저장하는 등 더 큰 임베딩을 사용하면 더 작은 임베딩을 사용하는 것보다 일반적으로 더 많은 비용이 들고 더 많은 컴퓨팅, 메모리 및 스토리지를 사용합니다.

 

Both of our new embeddings models were trained with a technique that allows developers to trade-off performance and cost of using embeddings. Specifically, developers can shorten embeddings (i.e. remove some numbers from the end of the sequence) without the embedding losing its concept-representing properties by passing in the dimensions API parameter. For example, on the MTEB benchmark, a text-embedding-3-large embedding can be shortened to a size of 256 while still outperforming an unshortened text-embedding-ada-002 embedding with a size of 1536.

 

두 가지 새로운 임베딩 모델은 모두 개발자가 임베딩 사용 비용과 성능을 절충할 수 있는 기술로 교육되었습니다. 특히 개발자는 차원 API 매개변수를 전달하여 임베딩이 개념을 나타내는 속성을 잃지 않고 임베딩을 단축할 수 있습니다(예: 시퀀스 끝에서 일부 숫자 제거). 예를 들어, MTEB 벤치마크에서 text-embedding-3-large 임베딩은 크기 256으로 단축되면서도 크기 1536의 단축되지 않은 text-embedding-ada-002 임베딩보다 성능이 뛰어납니다.

 

 

 

This enables very flexible usage. For example, when using a vector data store that only supports embeddings up to 1024 dimensions long, developers can now still use our best embeddings model text-embedding-3-large and specify a value of 1024 for the dimensions API parameter, which will shorten the embedding down from 3072 dimensions, trading off some accuracy in exchange for the smaller vector size.

 

이를 통해 매우 유연한 사용이 가능합니다. 예를 들어, 최대 1024차원 길이의 임베딩만 지원하는 벡터 데이터 저장소를 사용할 때 개발자는 이제 최고의 임베딩 모델 text-embedding-3-large를 사용하고 차원 API 매개변수에 값 1024를 지정할 수 있습니다. 3072 차원에서 임베딩을 줄여 벡터 크기가 더 작아지는 대가로 어느 정도 정확도를 희생했습니다.

 

Other new models and lower pricing

Updated GPT-3.5 Turbo model and lower pricing

 

Next week we are introducing a new GPT-3.5 Turbo model, gpt-3.5-turbo-0125, and for the third time in the past year, we will be decreasing prices on GPT-3.5 Turbo to help our customers scale. Input prices for the new model are reduced by 50% to $0.0005 /1K tokens and output prices are reduced by 25% to $0.0015 /1K tokens. This model will also have various improvements including higher accuracy at responding in requested formats and a fix for a bug which caused a text encoding issue for non-English language function calls.

 

다음 주에 우리는 새로운 GPT-3.5 Turbo 모델인 gpt-3.5-turbo-0125를 선보일 예정이며, 고객의 확장을 돕기 위해 작년에 세 번째로 GPT-3.5 Turbo의 가격을 인하할 예정입니다. 새 모델의 입력 가격은 $0.0005/1K 토큰으로 50% 인하되고, 출력 가격은 $0.0015/1K 토큰으로 25% 인하됩니다. 또한 이 모델에는 요청된 형식으로 응답할 때 더 높은 정확도와 영어가 아닌 언어 함수 호출에 대한 텍스트 인코딩 문제를 일으키는 버그 수정 등 다양한 개선 사항이 포함됩니다.

 

Customers using the pinned gpt-3.5-turbo model alias will be automatically upgraded from gpt-3.5-turbo-0613 to gpt-3.5-turbo-0125 two weeks after this model launches.

 

고정된 gpt-3.5-turbo 모델 별칭을 사용하는 고객은 이 모델 출시 2주 후 gpt-3.5-turbo-0613에서 gpt-3.5-turbo-0125로 자동 업그레이드됩니다.

 

 

Updated GPT-4 Turbo preview

Over 70% of requests from GPT-4 API customers have transitioned to GPT-4 Turbo since its release, as developers take advantage of its updated knowledge cutoff, larger 128k context windows, and lower prices. 

 

개발자가 업데이트된 지식 컷오프, 더 커진 128k 컨텍스트 창 및 저렴한 가격을 활용함에 따라 GPT-4 API 고객의 요청 중 70% 이상이 출시 이후 GPT-4 Turbo로 전환되었습니다.

 

Today, we are releasing an updated GPT-4 Turbo preview model, gpt-4-0125-preview. This model completes tasks like code generation more thoroughly than the previous preview model and is intended to reduce cases of “laziness” where the model doesn’t complete a task. The new model also includes the fix for the bug impacting non-English UTF-8 generations.

 

오늘 우리는 업데이트된 GPT-4 Turbo 미리보기 모델인 gpt-4-0125-preview를 출시합니다. 이 모델은 이전 미리 보기 모델보다 코드 생성과 같은 작업을 더 철저하게 완료하며 모델이 작업을 완료하지 못하는 "게으름"의 경우를 줄이기 위한 것입니다. 새 모델에는 영어가 아닌 UTF-8 세대에 영향을 미치는 버그에 대한 수정 사항도 포함되어 있습니다.

 

For those who want to be automatically upgraded to new GPT-4 Turbo preview versions, we are also introducing a new gpt-4-turbo-preview model name alias, which will always point to our latest GPT-4 Turbo preview model. 

 

새로운 GPT-4 Turbo 미리보기 버전으로 자동 업그레이드하려는 사람들을 위해 항상 최신 GPT-4 Turbo 미리보기 모델을 가리키는 새로운 gpt-4-turbo-preview 모델 이름 별칭도 도입합니다.

 

We plan to launch GPT-4 Turbo with vision in general availability in the coming months.

 

우리는 앞으로 몇 달 안에 비전이 포함된 GPT-4 Turbo를 일반 출시할 계획입니다.

 

 

Updated moderation model

 

The free Moderation API allows developers to identify potentially harmful text. As part of our ongoing safety work, we are releasing text-moderation-007, our most robust moderation model to-date.

 

무료 Moderation API를 통해 개발자는 잠재적으로 유해한 텍스트를 식별할 수 있습니다. 지속적인 안전 작업의 일환으로 현재까지 가장 강력한 조정 모델인 text-moderation-007을 출시합니다.

 

The text-moderation-latest and text-moderation-stable aliases have been updated to point to it. You can learn more about building safe AI systems through our safety best practices guide.

 

text-moderation-latest 및 text-moderation-stable 별칭이 이를 가리키도록 업데이트되었습니다. 안전 모범 사례 가이드를 통해 안전한 AI 시스템 구축에 대해 자세히 알아볼 수 있습니다.

 

New ways to understand API usage and manage API keys

 

We are launching two platform improvements to give developers both more visibility into their usage and control over API keys.

 

우리는 개발자에게 API 키 사용에 대한 더 많은 가시성과 제어권을 제공하기 위해 두 가지 플랫폼 개선 사항을 출시할 예정입니다.

 

First, developers can now assign permissions to API keys from the API keys page. For example, a key could be assigned read-only access to power an internal tracking dashboard, or restricted to only access certain endpoints.

 

첫째, 이제 개발자는 API 키 페이지에서 API 키에 권한을 할당할 수 있습니다. 예를 들어 내부 추적 대시보드를 구동하기 위해 키에 읽기 전용 액세스 권한을 할당하거나 특정 엔드포인트에만 액세스하도록 제한할 수 있습니다.

 

Second, the usage dashboard and usage export function now expose metrics on an API key level after turning on tracking. This makes it simple to view usage on a per feature, team, product, or project level, simply by having separate API keys for each.

 

둘째, 이제 추적을 활성화한 후 사용량 대시보드 및 사용량 내보내기 기능이 API 키 수준에 지표를 노출합니다. 이를 통해 각각에 대해 별도의 API 키를 보유함으로써 기능, 팀, 제품 또는 프로젝트 수준별 사용량을 간단하게 확인할 수 있습니다.

 

 

In the coming months, we plan to further improve the ability for developers to view their API usage and manage API keys, especially in larger organizations.

 

앞으로 몇 달 안에 우리는 특히 대규모 조직에서 개발자가 자신의 API 사용량을 확인하고 API 키를 관리할 수 있는 기능을 더욱 향상시킬 계획입니다.

 

For the latest updates on OpenAI's APIs, follow us on X at @OpenAIDevs.

 

OpenAI API에 대한 최신 업데이트를 보려면 @OpenAIDevs에서 X를 팔로우하세요.

반응형