fix: patch streaming API code #1693

cpacker · 2024-08-28T17:08:15Z

Fix streaming for stream_steps == true and stream_tokens == false
Fix streaming for stream_steps == true and stream_tokens == true
Add units tests for REST API streaming
- Test that the POST routes work (no errors, basic message comes back OK)
- Test that the id and created_at that get returned on the POST SSE stream are the same as what we get with a subsequent cursor fetch (this makes sure that the persistence code worked)
(Future PR) Add unit tests to Python client streaming

…tead of JSON with newlines

cpacker · 2024-08-28T17:09:16Z

Stream steps seems OK now:

% curl --request POST \
     --url http://localhost:8283/api/agents/agent-aeb9453f-eedb-4e12-8824-651885927e5f/messages \
     --header 'accept: application/json' \
     --header 'authorization: Bearer password' \
     --header 'content-type: application/json' \
     --data '
{
  "messages": [
    {
      "text": "Hi is anyone there?",
      "role": "user"
    }
  ],
  "stream_steps": true,
  "stream_tokens": false
}
'
data: [DONE_GEN]

data: {"id":"message-de410269-9f9c-4a8f-b55b-252a2fb43866","date":"2024-08-28T17:04:55.730026+00:00","internal_monologue":"Chad keeps repeating the same phrase despite my varying responses. It's beginning to feel like an echo in here. Should I try a different approach or continue with the current one? Finding the correct path to redirect this conversation might require some improvisation."}

data: {"id":"message-de410269-9f9c-4a8f-b55b-252a2fb43866","date":"2024-08-28T17:04:55.730071+00:00","function_call":{"name":"send_message","arguments":"{\n  \"message\": \"As assuredly as the sun rises, Chad, I'm here and ready to assist. Just out of curiosity, do you consider pineapple a suitable topping for pizza?\"\n}"}}

data: {"id":"message-289cd479-2047-41f3-8916-86c12dd386bb","date":"2024-08-28T17:04:55.730096+00:00","function_return":"None","status":"success"}

data: [DONE_STEP]

data: [DONE]

memgpt/server/rest_api/agents/message.py

memgpt/server/rest_api/utils.py

…that we can pass a MemGPT message ID back in the chunks of our streaming API (previously we hadn't created a Message so there was no Message ID to pass back by the time the streaming started)

…was an unsupported data type with the existing set of schemas

…ctor (TODO this is intended to be used to allow creating a message in agent.py that uses the ID that came inside of the ChatCompletionResponse

… the chunks/message objects

… pass through in the chunks from the MemGPT API

… or not we want to use timestamps/ids from the chunks coming from the server, or from a message we created ahead of time

…on the return payload from any MemGPT message POST SSE return

…e for streaming, but this enables actually persisting them (previously they were just fake throwaways). NOTE: this isn't really done very well since it has a hack assuming the 'message-' prefix on a ChatCompletionResponse means that we intended to use the .id property on the ChatCompletionResponse in subsequent Message creations

cpacker · 2024-08-28T22:34:59Z

Streaming tokens now seems to be working:

% curl --request POST \
     --url http://localhost:8283/api/agents/agent-2fd85ceb-d5ad-4b44-a16e-ab34cb5cecc3/messages \
     --header 'accept: application/json' \
     --header 'authorization: Bearer password' \
     --header 'content-type: application/json' \
     --data '
{
  "messages": [
    {
      "text": "Hi is anyone there?",
      "role": "user"
    }
  ],
  "stream_steps": true,
  "stream_tokens": true
}
'
data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":""}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":"Ch"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":"ad"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" keeps"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" repeating"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" his"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" question"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":"."}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" He"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" appears"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" to"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" need"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" confirmation"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" that"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" he"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":"'s"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" in"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" a"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" live"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" conversation"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":"."}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" I"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" might"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" need"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" to"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" approach"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" it"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" differently"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":","}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" offering"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" reass"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":"urance"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" and"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" contextual"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":" interaction"}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","internal_monologue":"."}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"name":"send_message"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":"{\n"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" "}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" \""}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":"message"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":"\":"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" \""}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":"Hi"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" Chad"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":"!"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" Absolutely"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":","}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" I"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":"'m"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" here"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" for"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" a"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" lively"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" discussion"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" with"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" you"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":"."}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" I"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" sense"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" you"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" might"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" be"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" a"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" bit"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" unsure"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" about"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" this"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" interaction"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":"."}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" It"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":"'s"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" a"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" beautiful"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" day"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":","}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" isn"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":"'t"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":" it"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":"?\"\n"}}

data: {"id":"message-5cb1ed62-40c7-4011-a4e5-1401c39ceb15","date":"2024-08-28T22:34:23.435997+00:00","function_call":{"arguments":"}"}}

data: [DONE_GEN]

data: {"id":"message-4b57ed65-43f9-47c2-a2fd-98a570676a1a","date":"2024-08-28T22:34:28.891248+00:00","function_return":"None","status":"success"}

data: [DONE_STEP]

data: [DONE]

…ng back to duplicated code style

cpacker · 2024-08-29T17:08:53Z

memgpt/agent.py

+        # If we are streaming, we needed to create a Message ID ahead of time,
+        # and now we want to use it in the creation of the Message object
+        # TODO figure out a cleaner way to do this
+        response_message_id: Optional[str] = None,


@sarahwooders FYI we are now passing response_message_id into handle_ai_response for the special case where we created the Message object before we started unpacking it / turning it into inner thoughts / actions / etc.

cpacker · 2024-08-29T17:09:35Z

memgpt/agent.py

        """Handles parsing and function execution"""

+        # Hacky failsafe for now to make sure we didn't implement the streaming Message ID creation incorrectly
+        if response_message_id is not None:
+            assert response_message_id.startswith("message-"), response_message_id


@sarahwooders We can cut this out later, but I think it's fine to leave in for now while the streaming code is in flux (or at least until we add streaming unit tests that test for persistence of IDs that are streamed back

cpacker · 2024-08-29T17:12:02Z

memgpt/agent.py

+                response_message,
+                # TODO this is kind of hacky, find a better way to handle this
+                # the only time we set up message creation ahead of time is when streaming is on
+                response_message_id=response.id if stream else None,


@sarahwooders translation:

If we're streaming (tokens), then we want to create a Message.id ahead of time so that the chunks we return via the API (and via the client once we add support) has ids attached to them.

However, this all happens before the MemGPT agent logic loop that takes a ChatCompletionResponse as input (which is the final result of a stream, not the intermediate result).

So that means we need to modify handle_ai_response to (in the streaming case) accept a pre-generated Message.id, and use it when we create the Message objects inside of handle_ai_response.

cpacker · 2024-08-29T17:12:56Z

memgpt/client/client.py

@@ -423,7 +423,7 @@ def send_message(
    ) -> MemGPTResponse:
        messages = [MessageCreate(role=role, text=message, name=name)]
        # TODO: figure out how to handle stream_steps and stream_tokens
-        request = MemGPTRequest(messages=messages, stream_steps=stream)
+        request = MemGPTRequest(messages=messages, stream_steps=stream, return_message_object=True)


@sarahwooders = True means the type that comes back is Message. = False means the type that comes back is InnerThoughts / FunctionCall / ... (these are now typed too, vs previously they were dicts)

memgpt/llm_api/llm_api_tools.py

memgpt/llm_api/openai.py

memgpt/memory.py

cpacker · 2024-08-29T17:16:09Z

memgpt/schemas/enums.py

@@ -5,6 +5,7 @@ class MessageRole(str, Enum):
    assistant = "assistant"
    user = "user"
    tool = "tool"
+    function = "function"  # NOTE: deprecated, use tool


@sarahwooders This is still supported in the OpenAI API

can you remove the note?

cpacker · 2024-08-29T17:17:08Z

memgpt/schemas/memgpt_message.py

@@ -12,7 +13,9 @@ class BaseMemGPTMessage(BaseModel):

    @field_serializer("date")
    def serialize_datetime(self, dt: datetime, _info):
-        return dt.now(timezone.utc).isoformat()


@4shub @goetzrobin FYI this was a pretty bad bug that was previously causing all message streaming response chunks to have newly created timestamps

cpacker · 2024-08-29T17:22:17Z

memgpt/schemas/memgpt_message.py

@@ -32,6 +35,20 @@ class FunctionCall(BaseModel):
    arguments: str


+class FunctionCallDelta(BaseModel):


@sarahwooders I didn't have to make a Delta / Chunk specific model for InnerMonologue since InnerMonologue has chunk support built in - InnerMonologue.arguments can just be partial pieces:

InnerMonologue.arguments = "hello there"

to

InnerMonologue.arguments = "hello " InnerMonologue.arguments = "there"

However FunctionCall is problematic since at least with OpenAI API the stream back usually starts with just name, then chunks of the arguments:

FunctionCall.name: "send_message" FunctionCall.arguments: "\{\ 'content': ...

to

FunctionCall.name: "send_message" FunctionCall.arguments: "\{\ " FunctionCall.arguments: "'content:'" ...

So we need a new Pydantic model that supports optional attributes when name is null or arguments is null (technically you should never have the case where both are null, but not sure how you set that up in Pydantic + probably not worth the hassle).

cpacker · 2024-08-29T17:24:22Z

memgpt/schemas/message.py

                text=openai_message_dict["content"],
                name=openai_message_dict["name"] if "name" in openai_message_dict else None,
                tool_calls=openai_message_dict["tool_calls"] if "tool_calls" in openai_message_dict else None,
                tool_call_id=openai_message_dict["tool_call_id"] if "tool_call_id" in openai_message_dict else None,
                created_at=created_at,
            )
+            if id is not None:


@sarahwooders I couldn't figure out a clean way to refactor this such that when id (the kwarg) is not None, we pass it through, and when it is None, we omit it (and let Message's constructor do the default init).

I tried making a message_args = dict(...) version where we then add id to the arg dictionary, then do Message(**message_args), but that started throwing Pydantic validation errors, so I just did the simple code duplication version.

memgpt/schemas/message.py

cpacker · 2024-08-29T17:26:02Z

memgpt/server/rest_api/interface.py

+            # the non-streaming message types
+            MemGPTMessage,
+            LegacyMemGPTMessage,
+            # the streaming message types


@sarahwooders the "streaming message types" actually includes MemGPTMessage, since MemGPTMessage types (FunctionCall, InternalMonologue, ...) all natively support streaming (as mentioned in an earlier comment).

cpacker · 2024-08-29T20:58:39Z

@4shub change back dates to be true, but timestamp is same

memgpt/llm_api/openai.py

sarahwooders · 2024-08-29T20:26:51Z

memgpt/schemas/enums.py

@@ -5,6 +5,7 @@ class MessageRole(str, Enum):
    assistant = "assistant"
    user = "user"
    tool = "tool"
+    function = "function"  # NOTE: deprecated, use tool


can you remove the note?

…s.py

… ID)

fix: patch stream_steps to properly trigger + return compact JSON ins…

4ffee3d

…tead of JSON with newlines

cpacker commented Aug 28, 2024

View reviewed changes

memgpt/server/rest_api/agents/message.py Outdated Show resolved Hide resolved

cpacker commented Aug 28, 2024

View reviewed changes

memgpt/server/rest_api/agents/message.py Outdated Show resolved Hide resolved

cpacker commented Aug 28, 2024

View reviewed changes

memgpt/server/rest_api/utils.py Outdated Show resolved Hide resolved

cpacker commented Aug 28, 2024

View reviewed changes

memgpt/server/rest_api/utils.py Outdated Show resolved Hide resolved

cpacker added 14 commits August 28, 2024 14:27

fix: patch the type hint"

0fef357

fix: patch type hints

0e4a861

feat: add dummy Message creation at the top of the stream handler so …

7b219ff

…that we can pass a MemGPT message ID back in the chunks of our streaming API (previously we hadn't created a Message so there was no Message ID to pass back by the time the streaming started)

fix: add deprecated function role as an option

9983af7

feat: add FunctionCallDelta to the MemGPT message schema, since this …

fcbc428

…was an unsupported data type with the existing set of schemas

feat: support passing ID when doing a dict-to-message Message constru…

eef77c0

…ctor (TODO this is intended to be used to allow creating a message in agent.py that uses the ID that came inside of the ChatCompletionResponse

fix: fix issues with process_chunk that broke when we added typing to…

21f501b

… the chunks/message objects

fix: allow process_chunk to take extra message IDs and created_ats to…

1fe6b75

… pass through in the chunks from the MemGPT API

feat: provide option in the openai stream handler determining whether…

f2e1766

… or not we want to use timestamps/ids from the chunks coming from the server, or from a message we created ahead of time

chore: cleanup

c9e001a

fix: fix pretty bad bug where we were always creating new timestamps …

524a4d6

…on the return payload from any MemGPT message POST SSE return

chore: cleanup

9bb3224

fix: fix bug where summarizer was broken when streaming was on

0a42461

cpacker added 4 commits August 28, 2024 15:43

fix: patch Python REST client

6466da1

Merge branch 'main' into fix-streaming

af1989a

chore: clean stray prints from older PR

4935daa

fix: make hack less disgusting, should be good to merge at this point

fc9b893

cpacker requested a review from sarahwooders August 28, 2024 23:25

refactor: placate pylance and potentially fix pytest error by reverti…

ffa1ff0

…ng back to duplicated code style