前言

為了保證系統(tǒng)的可靠性和穩(wěn)定性，ChatGPT設(shè)置了速率限制，限制每個用戶在特定時間段內(nèi)可以發(fā)送的消息數(shù)量。這樣可以防止某些用戶對系統(tǒng)進行濫用，并且減少資源占用。ChatGPT 的速率限制比較靈活，會根據(jù)用戶的行為以及服務(wù)器的負載情況動態(tài)調(diào)整。例如，在繁忙的時段，我們可能會采取更加嚴格的限制策略，以確保服務(wù)器的穩(wěn)定性?？梢哉fChatGPT 的速率限制是確保系統(tǒng)運行穩(wěn)定、避免惡意濫用的重要措施。

Introduction 導(dǎo)言

What are rate limits? 什么是速率限制？

A rate limit is a restriction that an API imposes on the number of times a user or client can access the server within a specified period of time.
速率限制是API對用戶或客戶端在指定時間段內(nèi)可以訪問服務(wù)器的次數(shù)施加的限制。

Why do we have rate limits? 為什么我們有速率限制？

Rate limits are a common practice for APIs, and they’re put in place for a few different reasons:
速率限制是API的常見做法，它們的實施有幾個不同的原因：

• They help protect against abuse or misuse of the API. For example, a malicious actor could flood the API with requests in an attempt to overload it or cause disruptions in service. By setting rate limits, OpenAI can prevent this kind of activity.
  **它們有助于防止濫用或誤用API。**例如，惡意行為者可能會向API發(fā)送大量請求，試圖使其過載或?qū)е路?wù)中斷。通過設(shè)置速率限制，OpenAI可以防止此類活動。
• Rate limits help ensure that everyone has fair access to the API. If one person or organization makes an excessive number of requests, it could bog down the API for everyone else. By throttling the number of requests that a single user can make, OpenAI ensures that the most number of people have an opportunity to use the API without experiencing slowdowns.
  **速率限制有助于確保每個人都可以公平地訪問API。**如果一個人或組織發(fā)出過多的請求，它可能會使其他人的API陷入困境。通過限制單個用戶可以發(fā)出的請求數(shù)量，OpenAI確保大多數(shù)人有機會使用API而不會遇到速度減慢。
• Rate limits can help OpenAI manage the aggregate load on its infrastructure. If requests to the API increase dramatically, it could tax the servers and cause performance issues. By setting rate limits, OpenAI can help maintain a smooth and consistent experience for all users.
  **速率限制可以幫助OpenAI管理其基礎(chǔ)設(shè)施上的總負載。**如果對API的請求急劇增加，可能會加重服務(wù)器的負擔并導(dǎo)致性能問題。通過設(shè)置速率限制，OpenAI可以幫助所有用戶保持流暢一致的體驗。

Please work through this document in its entirety to better understand how OpenAI’s rate limit system works. We include code examples and possible solutions to handle common issues. It is recommended to follow this guidance before filling out the Rate Limit Increase Request form with details regarding how to fill it out in the last section.
請完整閱讀本文檔，以更好地了解OpenAI的速率限制系統(tǒng)是如何工作的。我們包括代碼示例和處理常見問題的可能解決方案。建議您在填寫費率限額增加申請表之前遵循本指南，并在最后一節(jié)中詳細說明如何填寫。

What are the rate limits for our API? 我們API的速率限制是什么？

We enforce rate limits at the organization level, not user level, based on the specific endpoint used as well as the type of account you have. Rate limits are measured in two ways: RPM (requests per minute) and TPM (tokens per minute). The table below highlights the default rate limits for our API but these limits can be increased depending on your use case after filling out the Rate Limit increase request form.
我們根據(jù)所使用的特定端點以及您擁有的帳戶類型，在組織級別（而非用戶級別）實施速率限制。速率限制以兩種方式測量：RPM（每分鐘請求數(shù)）和TPM（每分鐘標記數(shù)）。下表突出顯示了我們API的默認速率限制，但在填寫速率限制增加請求表單后，這些限制可以根據(jù)您的用例進行增加。

The TPM (tokens per minute) unit is different depending on the model:
TPM（每分鐘標記數(shù)）單位因模型而異：

In practical terms, this means you can send approximately 200x more tokens per minute to an ada model versus a davinci model.
實際上，這意味著您每分鐘可以向 ada 模型發(fā)送大約200倍的標記，而不是 davinci 模型。

It is important to note that the rate limit can be hit by either option depending on what occurs first. For example, you might send 20 requests with only 100 tokens to the Codex endpoint and that would fill your limit, even if you did not send 40k tokens within those 20 requests.
重要的是要注意，根據(jù)首先發(fā)生的情況，任何一種選擇都可能達到速率限制。例如，您可以向Codex端點發(fā)送20個僅包含100個標記的請求，這將滿足您的限制，即使您在這20個請求中沒有發(fā)送40K標記。

GPT-4 rate limits GPT-4速率限制

During the rollout of GPT-4, the model will have more aggressive rate limits to keep up with demand. Default rate limits for gpt-4/gpt-4-0314 are 40k TPM and 200 RPM. Default rate limits for gpt-4-32k/gpt-4-32k-0314 are 80k TPM and 400 RPM. Please note that during the limited beta phase of GPT-4 we will be unable to accommodate requests for rate limit increases. In its current state, the model is intended for experimentation and prototyping, not high volume production use cases.
在GPT-4的推出期間，該模型將具有更積極的速率限制，以跟上需求。 gpt-4 / gpt-4-0314 的默認速率限制為40k TPM和200 RPM。 gpt-4-32k / gpt-4-32k-0314 的默認速率限制為80k TPM和400 RPM。請注意，在GPT-4的有限測試階段，我們將無法滿足費率限制增加的請求。在目前的狀態(tài)下，該模型旨在用于實驗和原型設(shè)計，而不是大批量生產(chǎn)用例。

How do rate limits work? 速率限制是如何工作的？

If your rate limit is 60 requests per minute and 150k davinci tokens per minute, you’ll be limited either by reaching the requests/min cap or running out of tokens—whichever happens first. For example, if your max requests/min is 60, you should be able to send 1 request per second. If you send 1 request every 800ms, once you hit your rate limit, you’d only need to make your program sleep 200ms in order to send one more request otherwise subsequent requests would fail. With the default of 3,000 requests/min, customers can effectively send 1 request every 20ms, or every .02 seconds.
如果您的速率限制是每分鐘60個請求和每分鐘150k個 davinci 標記，您將受到限制，要么達到請求/分鐘上限，要么用完標記-以先發(fā)生的為準。例如，如果您的max requests/min是60，那么您應(yīng)該能夠每秒發(fā)送1個請求。如果你每800 ms發(fā)送一個請求，一旦你達到了你的速率限制，你只需要讓你的程序休眠200 ms就可以再發(fā)送一個請求，否則后續(xù)的請求就會失敗。在默認值為3，000個請求/分鐘的情況下，客戶實際上可以每20 ms或每0.02秒發(fā)送一個請求。

What happens if I hit a rate limit error? 如果我遇到速率限制錯誤會發(fā)生什么？

Rate limit errors look like this:
速率限制錯誤如下所示：

Rate limit reached for default-text-davinci-002 in organization org-{id} on requests per min. Limit: 20.000000 / min. Current: 24.000000 / min.
組織org-{id}中的default-text-davinci-002達到每分鐘請求數(shù)的速率限制。限制：20.000000 /分鐘 現(xiàn)在：24.000000 /分鐘

If you hit a rate limit, it means you’ve made too many requests in a short period of time, and the API is refusing to fulfill further requests until a specified amount of time has passed.
如果你達到了速率限制，這意味著你在短時間內(nèi)發(fā)出了太多的請求，API將拒絕滿足更多的請求，直到指定的時間過去。

Rate limits vs max_tokens 速率限制與最大標記數(shù)

Each model we offer has a limited number of tokens that can be passed in as input when making a request. You cannot increase the maximum number of tokens a model takes in. For example, if you are using text-ada-001, the maximum number of tokens you can send to this model is 2,048 tokens per request.
我們提供的每個模型都有有限數(shù)量的標記，可以在發(fā)出請求時作為輸入傳入。不能增加模型接受的最大標記數(shù)。例如，如果您使用 text-ada-001 ，則您可以向此模型發(fā)送的標記的最大數(shù)量為每個請求2，048個標記。

Error Mitigation 錯誤消除

What are some steps I can take to mitigate this? 我可以采取哪些措施來緩解這種情況？

The OpenAI Cookbook has a python notebook that explains details on how to avoid rate limit errors.
OpenAI Cookbook有一個Python筆記本，解釋了如何避免速率限制錯誤的細節(jié)。

You should also exercise caution when providing programmatic access, bulk processing features, and automated social media posting - consider only enabling these for trusted customers.
在提供程序化訪問、批量處理功能和自動社交媒體發(fā)布功能時，您還應(yīng)謹慎行事-請考慮僅為受信任的客戶啟用這些功能。

To protect against automated and high-volume misuse, set a usage limit for individual users within a specified time frame (daily, weekly, or monthly). Consider implementing a hard cap or a manual review process for users who exceed the limit.
為了防止自動化和大量濫用，請在指定的時間范圍內(nèi)（每天、每周或每月）為單個用戶設(shè)置使用限制。考慮對超出限制的用戶實施硬上限或手動審查流程。

Retrying with exponential backoff 使用指數(shù)回退重試

One easy way to avoid rate limit errors is to automatically retry requests with a random exponential backoff. Retrying with exponential backoff means performing a short sleep when a rate limit error is hit, then retrying the unsuccessful request. If the request is still unsuccessful, the sleep length is increased and the process is repeated. This continues until the request is successful or until a maximum number of retries is reached. This approach has many benefits:
避免速率限制錯誤的一種簡單方法是使用隨機指數(shù)退避自動重試請求。使用指數(shù)回退重試意味著當達到速率限制錯誤時執(zhí)行短暫休眠，然后重試不成功的請求。如果請求仍然不成功，則增加休眠長度并重復(fù)該過程。這將持續(xù)到請求成功或達到最大重試次數(shù)。這種方法有很多好處：

• Automatic retries means you can recover from rate limit errors without crashes or missing data
  自動重試意味著您可以從速率限制錯誤中恢復(fù)，而不會崩潰或丟失數(shù)據(jù)
• Exponential backoff means that your first retries can be tried quickly, while still benefiting from longer delays if your first few retries fail
  指數(shù)回退意味著您的第一次重試可以快速嘗試，同時如果您的前幾次重試失敗，仍然可以從更長的延遲中受益
• Adding random jitter to the delay helps retries from all hitting at the same time.
  向延遲添加隨機抖動有助于同時重試所有命中。

Note that unsuccessful requests contribute to your per-minute limit, so continuously resending a request won’t work.
請注意，不成功的請求會影響您的每分鐘限制，因此連續(xù)重新發(fā)送請求將不起作用。

Below are a few example solutions for Python that use exponential backoff.
下面是一些使用指數(shù)回退的Python示例解決方案。

Example #1: Using the Tenacity library 示例#1：使用Tenacity庫

Tenacity is an Apache 2.0 licensed general-purpose retrying library, written in Python, to simplify the task of adding retry behavior to just about anything. To add exponential backoff to your requests, you can use the tenacity.retry decorator. The below example uses the tenacity.wait_random_exponential function to add random exponential backoff to a request.
Tenacity是一個Apache 2.0許可的通用重試庫，用Python編寫，用于簡化將重試行為添加到任何內(nèi)容的任務(wù)。要在請求中添加指數(shù)回退，可以使用 tenacity.retry 裝飾器。下面的示例使用 tenacity.wait_random_exponential 函數(shù)向請求添加隨機指數(shù)退避。

Using the Tenacity library 使用Tenacity庫 Python 代碼示例：

import openai
from tenacity import (
    retry,
    stop_after_attempt,
    wait_random_exponential,
)  # for exponential backoff
 
@retry(wait=wait_random_exponential(min=1, max=60), stop=stop_after_attempt(6))
def completion_with_backoff(**kwargs):
    return openai.Completion.create(**kwargs)
 
completion_with_backoff(model="text-davinci-003", prompt="Once upon a time,")

Note that the Tenacity library is a third-party tool, and OpenAI makes no guarantees about its reliability or security.
請注意，Tenacity庫是第三方工具，OpenAI不保證其可靠性或安全性。

Example #2: Using the backoff library 示例2：使用backoff 庫

Another python library that provides function decorators for backoff and retry is backoff:
另一個為backoff和retry提供函數(shù)裝飾器的python庫是backoff：

import backoff 
import openai 
@backoff.on_exception(backoff.expo, openai.error.RateLimitError)
def completions_with_backoff(**kwargs):
    return openai.Completion.create(**kwargs)
 
completions_with_backoff(model="text-davinci-003", prompt="Once upon a time,")

Like Tenacity, the backoff library is a third-party tool, and OpenAI makes no guarantees about its reliability or security.
與Tenacity一樣，backoff 庫是第三方工具，OpenAI不保證其可靠性或安全性。

Example 3: Manual backoff implementation 示例3：手動backoff 實現(xiàn)

If you don’t want to use third-party libraries, you can implement your own backoff logic following this example:
如果你不想使用第三方庫，你可以按照這個例子實現(xiàn)你自己的退避邏輯：

# imports
import random
import time
 
import openai
 
# define a retry decorator
def retry_with_exponential_backoff(
    func,
    initial_delay: float = 1,
    exponential_base: float = 2,
    jitter: bool = True,
    max_retries: int = 10,
    errors: tuple = (openai.error.RateLimitError,),
):
    """Retry a function with exponential backoff."""
 
    def wrapper(*args, **kwargs):
        # Initialize variables
        num_retries = 0
        delay = initial_delay
 
        # Loop until a successful response or max_retries is hit or an exception is raised
        while True:
            try:
                return func(*args, **kwargs)
 
            # Retry on specific errors
            except errors as e:
                # Increment retries
                num_retries += 1
 
                # Check if max retries has been reached
                if num_retries > max_retries:
                    raise Exception(
                        f"Maximum number of retries ({max_retries}) exceeded."
                    )
 
                # Increment the delay
                delay *= exponential_base * (1 + jitter * random.random())
 
                # Sleep for the delay
                time.sleep(delay)
 
            # Raise exceptions for any errors not specified
            except Exception as e:
                raise e
 
    return wrapper
    
@retry_with_exponential_backoff
def completions_with_backoff(**kwargs):
    return openai.Completion.create(**kwargs)

Again, OpenAI makes no guarantees on the security or efficiency of this solution but it can be a good starting place for your own solution.
同樣，OpenAI不保證此解決方案的安全性或效率，但它可以成為您自己解決方案的良好起點。

Batching requests 批處理請求

The OpenAI API has separate limits for requests per minute and tokens per minute.
OpenAI API對每分鐘請求和每分鐘標記有單獨的限制。

If you’re hitting the limit on requests per minute, but have available capacity on tokens per minute, you can increase your throughput by batching multiple tasks into each request. This will allow you to process more tokens per minute, especially with our smaller models.
如果您達到了每分鐘請求數(shù)的限制，但每分鐘標記有可用容量，則可以通過將多個任務(wù)批處理到每個請求中來提高吞吐量。這將允許您每分鐘處理更多的標記，特別是對于我們較小的模型。

Sending in a batch of prompts works exactly the same as a normal API call, except you pass in a list of strings to the prompt parameter instead of a single string.
批量發(fā)送提示的工作方式與普通API調(diào)用完全相同，不同之處在于您向prompt參數(shù)傳遞的是字符串列表而不是單個字符串。

Example without batching 無批處理示例 Python代碼示例

import openai
 
num_stories = 10
prompt = "Once upon a time,"
 
# serial example, with one story completion per request
for _ in range(num_stories):
    response = openai.Completion.create(
        model="curie",
        prompt=prompt,
        max_tokens=20,
    )
    # print story
    print(prompt + response.choices[0].text)

Example with batching 批處理示例 Python代碼示例

import openai  # for making OpenAI API requests
 
 
num_stories = 10
prompts = ["Once upon a time,"] * num_stories
 
# batched example, with 10 story completions per request
response = openai.Completion.create(
    model="curie",
    prompt=prompts,
    max_tokens=20,
)
 
# match completions to prompts by index
stories = [""] * len(prompts)
for choice in response.choices:
    stories[choice.index] = prompts[choice.index] + choice.text
 
# print stories
for story in stories:
    print(story)

Warning: the response object may not return completions in the order of the prompts, so always remember to match responses back to prompts using the index field.
警告：響應(yīng)對象可能不會按照提示的順序返回完成，因此請始終記住使用索引字段將響應(yīng)匹配回提示。

Request Increase 增加請求

When should I consider applying for a rate limit increase?我應(yīng)該在什么時候考慮申請?zhí)岣呃氏拗疲?/h3>

Our default rate limits help us maximize stability and prevent abuse of our API. We increase limits to enable high-traffic applications, so the best time to apply for a rate limit increase is when you feel that you have the necessary traffic data to support a strong case for increasing the rate limit. Large rate limit increase requests without supporting data are not likely to be approved. If you’re gearing up for a product launch, please obtain the relevant data through a phased release over 10 days.
我們的速率限制幫助我們最大限度地提高穩(wěn)定性，防止濫用API。我們會提高限制以支持高流量應(yīng)用程序，因此，申請?zhí)岣咚俾氏拗频淖罴褧r機是當您認為有必要的流量數(shù)據(jù)來支持提高速率限制的有力理由時。沒有支持數(shù)據(jù)的大幅度速率限額增加請求不太可能獲得批準。如果您正在為產(chǎn)品發(fā)布做準備，請在10天內(nèi)通過分階段發(fā)布獲取相關(guān)數(shù)據(jù)。

Keep in mind that rate limit increases can sometimes take 7-10 days so it makes sense to try and plan ahead and submit early if there is data to support you will reach your rate limit given your current growth numbers.
請記住，速率限制的增加有時可能需要7-10天，因此如果有數(shù)據(jù)支持您將達到當前增長數(shù)字的速率限制，則嘗試提前計劃并盡早提交是有意義的。

Will my rate limit increase request be rejected? 我的速率上限增加請求會被拒絕嗎？

A rate limit increase request is most often rejected because it lacks the data needed to justify the increase. We have provided numerical examples below that show how to best support a rate limit increase request and try our best to approve all requests that align with our safety policy and show supporting data. We are committed to enabling developers to scale and be successful with our API.
提高速率限制的請求最常被拒絕，因為它缺乏證明提高合理性所需的數(shù)據(jù)。我們在下面提供了數(shù)字示例，說明如何最好地支持速率限制增加請求，并盡最大努力批準符合我們安全政策的所有請求，并顯示支持數(shù)據(jù)。我們致力于使開發(fā)人員能夠擴展并成功使用我們的API。

I’ve implemented exponential backoff for my text/code APIs, but I’m still hitting this error. How do I increase my rate limit? 我已經(jīng)為我的文本/代碼API實現(xiàn)了指數(shù)回退，但我仍然遇到這個錯誤。我如何提高我的速率上限？

We understand the frustration that limited rate limits can cause, and we would love to raise the defaults for everyone. However, due to shared capacity constraints, we can only approve rate limit increases for paid customers who have demonstrated a need through our Rate Limit Increase Request form. To help us evaluate your needs properly, we ask that you please provide statistics on your current usage or projections based on historic user activity in the ‘Share evidence of need’ section of the form. If this information is not available, we recommend a phased release approach. Start by releasing the service to a subset of users at your current rate limits, gather usage data for 10 business days, and then submit a formal rate limit increase request based on that data for our review and approval.
我們理解有限的速率限制可能導(dǎo)致的挫折感，我們希望提高每個人的速率限制。但是，由于共享容量的限制，我們只能批準通過我們的速率限額增加請求表證明需要的付費客戶的速率限額增加。為了幫助我們正確評估您的需求，我們要求您在表格的“分享需求證據(jù)”部分提供有關(guān)您當前使用情況的統(tǒng)計數(shù)據(jù)或基于歷史用戶活動的預(yù)測。如果沒有這些信息，我們建議采用分階段發(fā)布的方法。首先，以您當前的速率限制向一部分用戶發(fā)布服務(wù)，收集10個工作日的使用數(shù)據(jù)，然后根據(jù)該數(shù)據(jù)提交正式的速率限制增加請求，供我們審核和批準。

We will review your request and if it is approved, we will notify you of the approval within a period of 7-10 business days.
我們將審核您的請求，如果獲得批準，我們將在7-10個工作日內(nèi)通知您。

Here are some examples of how you might fill out this form:
以下是一些如何填寫此表單的示例：

DALL-E API examples DALL-E API示例

Language model examples 語言模型示例

Code model examples 代碼模型示例

Please note that these examples are just general use case scenarios, the actual usage rate will vary depending on the specific implementation and usage.
請注意，這些示例只是一般的用例場景，實際使用速率會根據(jù)具體的實現(xiàn)和使用情況而有所不同。

其它資料下載

如果大家想繼續(xù)了解人工智能相關(guān)學(xué)習路線和知識體系，歡迎大家翻閱我的另外一篇博客《重磅 | 完備的人工智能AI 學(xué)習——基礎(chǔ)知識學(xué)習路線，所有資料免關(guān)注免套路直接網(wǎng)盤下載》
這篇博客參考了Github知名開源平臺，AI技術(shù)平臺以及相關(guān)領(lǐng)域?qū)＜遥篋atawhale，ApacheCN，AI有道和黃海廣博士等約有近100G相關(guān)資料，希望能幫助到所有小伙伴們。文章來源地址http://www.zghlxwxcb.cn/news/detail-437025.html

到了這里，關(guān)于OpenAI-ChatGPT最新官方接口《速率并發(fā)限制》全網(wǎng)最詳細中英文實用指南和教程，助你零基礎(chǔ)快速輕松掌握全新技術(shù)（八）（附源碼）的文章就介紹完了。如果您還想了解更多內(nèi)容，請在右上角搜索TOY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！