题意:OpenAI流式响应未按预期工作。
问题背景:
On my NodeJS app, i have set-up a POST
method that looks like this:
在我的NodeJS应用程序中,我设置了一个看起来像这样的POST方法:
exports.complete = async (req, res, next) => {\
const {prompt} = req.body
res.writeHead(200, {
'Content-Type': 'text/plain',
'Transfer-Encoding': 'chunked'
})
const result = await openai.createCompletion(
{
model: 'text-davinci-003',
prompt: prompt,
temperature: 0.6,
max_tokens: 700,
stream: true
},
{ responseType: 'stream' }
)
result.data.on('data', data => {
const lines = data
.toString()
.split('\n')
.filter(line => line.trim() !== '')
for (const line of lines) {
const message = line.replace(/^data: /, '')
if (message === '[DONE]') {
res.end()
}
try {
const parsed = JSON.parse(message)
res.write(parsed.choices[0].text)
} catch (error) {
// console.error('Could not JSON parse stream message', message, error)
}
}
})
}
As the OpenAI Node SDK doesn't natively support streaming responses, i scrapped this code from a few sources.
由于OpenAI的Node SDK原生不支持流式响应,我从几个来源收集了这段代码。
To some extent, it is working i guess. But when i make a call to this endpoint from Postman and also from command line (using curl
), instead of actually getting the response in chunks, i am getting a final response when the entire completion
call finishes.
在某种程度上,我猜它是有效的。但当我从Postman或命令行(使用curl)调用这个端点时,本应该逐块接收的响应,实际上是在整个completion调用完成后才收到最终的完整响应。
I am not sure what i am doing wrong here. Note: This code above is a part of a Firebase Function call where i have set-up express
. I don't think it affects anything, but still mentioning.
我不确定我哪里做错了。注意:上面的代码是Firebase Function调用的一部分,我在其中设置了express。我认为这不会影响什么,但还是提一下。
Edit 1: The curl
request
Here's my curl
request: 这是我的curl请求:
curl --location 'http://localhost:5001/testapp-7aca9/us-central1/api/completion/complete' \
--header 'Content-Type: application/json' \
--data '{
"message": "Say two random lines"
}'
问题解决:
You can refer to the solution posted here: What is the correct way to send a long string by an HTTP stream in ExpressJS/NestJS?
你可以参考这里发布的解决方案:如何在ExpressJS/NestJS中通过HTTP流正确发送一个长字符串?
It might be caused by 2 reasons:
这可能由两个原因引起:
- In
curl
, you will need to add-N
flag to buffer the response to receive the response directly
在curl中,你需要添加`-N`标志来缓冲响应,以直接接收响应。
- In browser, the data are not returned until at least 1024 bytes (for Chrome) is returned, unless you add a
X-Content-Type-Options
ofnosniff
在浏览器中,数据不会返回,除非至少返回1024字节(对于Chrome来说),除非你添加`X-Content-Type-Options: nosniff`。