" errors" when processing output stream

[teamblubee]

Every so often I'll get something like this in my output stream when doing async processing

{"event":{"event_type":"progress","delta":" errors","logprobs":null,"stop_reason":null}}

here's the code reading and printing the response, it's a modified client:

        # Process the response from the server
        async for log in client.chat_completion(request):
            # print(f"Log structure: {log}\n{type(log)}\n")
            # print(f"\nLog structure: {log}\n{type(log)}\n{log.event}\n{type(log.event)}\n")
            if isinstance(log, ChatCompletionResponseStreamChunk):
                # print(f"RESPONSE STREAM CHUNK")
                if isinstance(log.event, ChatCompletionResponseEvent):
                    # print("\tRESPONSE EVENT")
                    if isinstance(log.event.event_type, ChatCompletionResponseEventType):
                        if log.event.event_type == ChatCompletionResponseEventType.start:
                            cprint(f"\n", "white", flush=True)
                        elif log.event.event_type == ChatCompletionResponseEventType.complete:
                            _stop_reason = log.event.stop_reason
                            cprint(f"\nCOMPLETED::Stop Reason::[{_stop_reason}]\n", "red", flush=True)
                            break
                        elif log.event.event_type == ChatCompletionResponseEventType.progress:
                            if ("errors" in log.event.delta and  log.event.logprobs == None and log.event.stop_reason == None):
                                break
                            else:
                                cprint(log.event.delta, "cyan", end='', flush=True)
                        else:
                            print(f"""
                            Log structure: {log}
                            {type(log)}
                            {log.event}
                            {type(log.event)}
                            ---------------------------------------------------------------------
                            print(f"\t\tEVENT TYPE             ::[{log.event.event_type}]")
                            print(f"\t\tEVENT [type] TYPE      ::[{type(log.event.event_type)}]")
                            print(f"\t\tEVENT TYPE DELTA       ::[{log.event.delta}]")
                            print(f"\t\tEVENT TYPE LOGPROBS    ::[{log.event.logprobs}]")
                            print(f"\t\tEVENT TYPE STOP_REASON ::[{log.event.stop_reason}]")
                            ---------------------------------------------------------------------
                            UNHANDLED ChatCompletionResponseEventType::[{log.event.event_type}]
                                """)

why do I get the errors every so often and my attempts to catch them and skip seems to fail.

Any suggestions?

[teamblubee]

I edited the default client.py file to add a userprompt and I still get the same error in the output stream. Meaning it seems like an issue with the way tokens are being parsed?

        *{"event":{"event_type":"progress","delta":" Terrorism","logprobs":null,"stop_reason":null}}

[teamblubee]

Another update, it seems those messages are just sent by the model and needs to be parsed.

Can I get pointed to a reference on how to parse those out and how to better process them?

Any documentation on what are the purpose of those values when parsing the response?

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

Cutting Knowledge Date: December 2023
Today Date: 23 July 2024

You are a helpful assistant<|eot_id|><|start_header_id|>user<|end_header_id|>
lets discuss Ni Electrochemical synthesis and redetermination of Ni(acac)x(H20)2.H20
<|start_header_id|>assistant<|end_header_id|>

Ni Electrochemical synthesis and redetermination of Ni(acac)x(H[2]
O)[2]
.H[2]
O is a topic in inorganic chemistry, specifically focusing on the electrochemical synthesis and structural analysis of a nickel complex.

To provide a better understanding, let's break down the key components:

[1]
. **Ni(acac)x(H[2]
O)[2]
.H[2]
O**: This is the chemical formula for the nickel complex being studied. 'acac' stands for acetylacetonate, a common ligand in coordination chemistry. The 'x' in the formula indicates that the complex has a variable number of acetylacetonate ligands. The 'H[2]
O' at the end represents two water molecules.

[2]
. **Electrochemical synthesis**: This is a method of synthesizing compounds using an electric current. In this case, the synthesis of Ni(acac)x(H[2]
O)[2]
.H[2]
O likely involves the reduction of nickel ions at an electrode, resulting in the formation of the complex.

[3]
. **Redetermination**: This refers to the process of re-determining the structure of a compound using advanced analytical techniques, such as X-ray crystallography or NMR spectroscopy. In this case, the researchers likely used these techniques to re-determine the structure of Ni(acac)x(H[2]
O)[2]
.H[2]
O.

Some possible questions or topics related to this subject include:

- What are the conditions for the electrochemical synthesis of Ni(acac)x(H[2]
O)[2]
.H[2]
O?
- What is the structure of Ni(acac)x(H[2]
O)[2]
.H[2]
O, and how does it differ from previously reported structures?
- What are the properties of Ni(acac)x(H[2]
O)[2]
.H[2]
O, such as its magnetic properties or reactivity?

If you have any specific questions or topics you'd like to discuss, I'd be happy to help.
COMPLETED::Stop Reason::[StopReason.end_of_turn]