OpenAI Stream response, not working as expected

发布于:2024-09-18 ⋅ 阅读:(152) ⋅ 点赞:(0)

题意:OpenAI流式响应未按预期工作。

问题背景:

On my NodeJS app, i have set-up a POST method that looks like this:

在我的NodeJS应用程序中,我设置了一个看起来像这样的POST方法:

exports.complete = async (req, res, next) => {\
    const {prompt} = req.body
    res.writeHead(200, {
      'Content-Type': 'text/plain',
      'Transfer-Encoding': 'chunked'
    })
    const result = await openai.createCompletion(
    {
      model: 'text-davinci-003',
      prompt: prompt,
      temperature: 0.6,
      max_tokens: 700,
      stream: true
    },
    { responseType: 'stream' }
  )

  result.data.on('data', data => {
    const lines = data
      .toString()
      .split('\n')
      .filter(line => line.trim() !== '')
    for (const line of lines) {
      const message = line.replace(/^data: /, '')
      if (message === '[DONE]') {
          res.end()
      }
      try {
        const parsed = JSON.parse(message)
        res.write(parsed.choices[0].text)
      } catch (error) {
        // console.error('Could not JSON parse stream message', message, error)
      }
    }
  })
}

As the OpenAI Node SDK doesn't natively support streaming responses, i scrapped this code from a few sources.

由于OpenAI的Node SDK原生不支持流式响应,我从几个来源收集了这段代码。

To some extent, it is working i guess. But when i make a call to this endpoint from Postman and also from command line (using curl), instead of actually getting the response in chunks, i am getting a final response when the entire completion call finishes.

在某种程度上,我猜它是有效的。但当我从Postman或命令行(使用curl)调用这个端点时,本应该逐块接收的响应,实际上是在整个completion调用完成后才收到最终的完整响应。

I am not sure what i am doing wrong here. Note: This code above is a part of a Firebase Function call where i have set-up express. I don't think it affects anything, but still mentioning.

我不确定我哪里做错了。注意:上面的代码是Firebase Function调用的一部分,我在其中设置了express。我认为这不会影响什么,但还是提一下。

Edit 1: The curl request

Here's my curl request:        这是我的curl请求:

curl --location 'http://localhost:5001/testapp-7aca9/us-central1/api/completion/complete' \
--header 'Content-Type: application/json' \
--data '{
    "message": "Say two random lines"
}'

问题解决:

You can refer to the solution posted here: What is the correct way to send a long string by an HTTP stream in ExpressJS/NestJS?

你可以参考这里发布的解决方案:如何在ExpressJS/NestJS中通过HTTP流正确发送一个长字符串?

It might be caused by 2 reasons:

这可能由两个原因引起:

  • In curl, you will need to add -N flag to buffer the response to receive the response directly

在curl中,你需要添加`-N`标志来缓冲响应,以直接接收响应。

  • In browser, the data are not returned until at least 1024 bytes (for Chrome) is returned, unless you add a X-Content-Type-Options of nosniff

在浏览器中,数据不会返回,除非至少返回1024字节(对于Chrome来说),除非你添加`X-Content-Type-Options: nosniff`。


网站公告

今日签到

点亮在社区的每一天
去签到