Python: DevUI Fix Serialization, Timestamp and Other Issues (microsoft#1584)

* refactor(devui): adopt standard OpenAI lifecycle events for agents and workflows

- Replace custom workflow events with OpenAI Responses API standard lifecycle events
- Add AgentStartedEvent, AgentCompletedEvent, AgentFailedEvent for clean separation
- Implement ExecutorActionItem for workflow executor tracking
- Convert informational events to trace events to reduce noise
- Update README mapper table with comprehensive event mappings
- Maintain full backward compatibility with legacy events

* fix(devui): resolve timestamp overwriting and Content serialization errors

- Fix tool call timestamps being overwritten on each render (microsoft#1483)
- Add recursive Content serialization to handle ChatMessage and nested objects (microsoft#1548)
- Implement proper MCP tool cleanup on server shutdown
- Add timestamp field to function_result.complete events
- Enhance credential and client resource cleanup

Fixes microsoft#1483, microsoft#1548
Partial improvements for microsoft#1476

Author: victordibia

python/packages/devui/README.md

@@ -49,6 +49,19 @@ devui ./agents --port 8080
 
 When DevUI starts with no discovered entities, it displays a **sample entity gallery** with curated examples from the Agent Framework repository. You can download these samples, review them, and run them locally to get started quickly.
 
+## Using MCP Tools
+
+**Important:** Don't use `async with` context managers when creating agents with MCP tools for DevUI - connections will close before execution.
+
+```python
+# ✅ Correct - DevUI handles cleanup automatically
+mcp_tool = MCPStreamableHTTPTool(url="http://localhost:8011/mcp", chat_client=chat_client)
+agent = ChatAgent(tools=mcp_tool)
+serve(entities=[agent])
+```
+
+MCP tools use lazy initialization and connect automatically on first use. DevUI attempts to clean up connections on shutdown
+
 ## Directory Structure
 
 For your agents to be discovered by the DevUI, they must be organized in a directory structure like below. Each agent/workflow must have an `__init__.py` that exports the required variable (`agent` or `workflow`).
@@ -157,42 +170,62 @@ Options:
 
 Given that DevUI offers an OpenAI Responses API, it internally maps messages and events from Agent Framework to OpenAI Responses API events (in `_mapper.py`). For transparency, this mapping is shown below:
 
-| Agent Framework Content         | OpenAI Event/Type                        | Status   |
-| ------------------------------- | ---------------------------------------- | -------- |
-| `TextContent`                   | `response.output_text.delta`             | Standard |
-| `TextReasoningContent`          | `response.reasoning_text.delta`          | Standard |
-| `FunctionCallContent` (initial) | `response.output_item.added`             | Standard |
-| `FunctionCallContent` (args)    | `response.function_call_arguments.delta` | Standard |
-| `FunctionResultContent`         | `response.function_result.complete`      | DevUI    |
-| `FunctionApprovalRequestContent`| `response.function_approval.requested`   | DevUI    |
-| `FunctionApprovalResponseContent`| `response.function_approval.responded`  | DevUI    |
-| `ErrorContent`                  | `error`                                  | Standard |
-| `UsageContent`                  | Final `Response.usage` field (not streamed) | Standard |
-| `WorkflowEvent`                 | `response.workflow_event.complete`       | DevUI    |
-| `DataContent`                   | `response.trace.complete`                | DevUI    |
-| `UriContent`                    | `response.trace.complete`                | DevUI    |
-| `HostedFileContent`             | `response.trace.complete`                | DevUI    |
-| `HostedVectorStoreContent`      | `response.trace.complete`                | DevUI    |
-
-- **Standard** = OpenAI Responses API spec
-- **DevUI** = Custom extensions for Agent Framework features (workflows, traces, function approvals)
+| OpenAI Event/Type                                            | Agent Framework Content           | Status   |
+| ------------------------------------------------------------ | --------------------------------- | -------- |
+|                                                              | **Lifecycle Events**              |          |
+| `response.created` + `response.in_progress`                  | `AgentStartedEvent`               | OpenAI   |
+| `response.completed`                                         | `AgentCompletedEvent`             | OpenAI   |
+| `response.failed`                                            | `AgentFailedEvent`                | OpenAI   |
+| `response.created` + `response.in_progress`                  | `WorkflowStartedEvent`            | OpenAI   |
+| `response.completed`                                         | `WorkflowCompletedEvent`          | OpenAI   |
+| `response.failed`                                            | `WorkflowFailedEvent`             | OpenAI   |
+|                                                              | **Content Types**                 |          |
+| `response.content_part.added` + `response.output_text.delta` | `TextContent`                     | OpenAI   |
+| `response.reasoning_text.delta`                              | `TextReasoningContent`            | OpenAI   |
+| `response.output_item.added`                                 | `FunctionCallContent` (initial)   | OpenAI   |
+| `response.function_call_arguments.delta`                     | `FunctionCallContent` (args)      | OpenAI   |
+| `response.function_result.complete`                          | `FunctionResultContent`           | DevUI    |
+| `response.function_approval.requested`                       | `FunctionApprovalRequestContent`  | DevUI    |
+| `response.function_approval.responded`                       | `FunctionApprovalResponseContent` | DevUI    |
+| `error`                                                      | `ErrorContent`                    | OpenAI   |
+| Final `Response.usage` field (not streamed)                  | `UsageContent`                    | OpenAI   |
+|                                                              | **Workflow Events**               |          |
+| `response.output_item.added` (ExecutorActionItem)*           | `ExecutorInvokedEvent`            | OpenAI   |
+| `response.output_item.done` (ExecutorActionItem)*            | `ExecutorCompletedEvent`          | OpenAI   |
+| `response.output_item.done` (ExecutorActionItem with error)* | `ExecutorFailedEvent`             | OpenAI   |
+| `response.workflow_event.complete`                           | `WorkflowEvent` (other)           | DevUI    |
+| `response.trace.complete`                                    | `WorkflowStatusEvent`             | DevUI    |
+| `response.trace.complete`                                    | `WorkflowWarningEvent`            | DevUI    |
+|                                                              | **Trace Content**                 |          |
+| `response.trace.complete`                                    | `DataContent`                     | DevUI    |
+| `response.trace.complete`                                    | `UriContent`                      | DevUI    |
+| `response.trace.complete`                                    | `HostedFileContent`               | DevUI    |
+| `response.trace.complete`                                    | `HostedVectorStoreContent`        | DevUI    |
+
+\*Uses standard OpenAI event structure but carries DevUI-specific `ExecutorActionItem` payload
+
+- **OpenAI** = Standard OpenAI Responses API event types
+- **DevUI** = Custom event types specific to Agent Framework (e.g., workflows, traces, function approvals)
 
 ### OpenAI Responses API Compliance
 
 DevUI follows the OpenAI Responses API specification for maximum compatibility:
 
-**Standard OpenAI Types Used:**
+**OpenAI Standard Event Types Used:**
+
 - `ResponseOutputItemAddedEvent` - Output item notifications (function calls and results)
+- `ResponseOutputItemDoneEvent` - Output item completion notifications
 - `Response.usage` - Token usage (in final response, not streamed)
 - All standard text, reasoning, and function call events
 
 **Custom DevUI Extensions:**
+
 - `response.function_approval.requested` - Function approval requests (for interactive approval workflows)
 - `response.function_approval.responded` - Function approval responses (user approval/rejection)
 - `response.workflow_event.complete` - Agent Framework workflow events
 - `response.trace.complete` - Execution traces and internal content (DataContent, UriContent, hosted files/stores)
 
-These custom extensions are clearly namespaced and can be safely ignored by standard OpenAI clients.
+These custom extensions are clearly namespaced and can be safely ignored by standard OpenAI clients. Note that DevUI also uses standard OpenAI events with custom payloads (e.g., `ExecutorActionItem` within `response.output_item.added`).
 
 ### Entity Management
 
@@ -224,12 +257,14 @@ These custom extensions are clearly namespaced and can be safely ignored by stan
 DevUI is designed as a **sample application for local development** and should not be exposed to untrusted networks or used in production environments.
 
 **Security features:**
+
 - Only loads entities from local directories or in-memory registration
 - No remote code execution capabilities
 - Binds to localhost (127.0.0.1) by default
 - All samples must be manually downloaded and reviewed before running
 
 **Best practices:**
+
 - Never expose DevUI to the internet
 - Review all agent/workflow code before running
 - Only load entities from trusted sources

python/packages/devui/agent_framework_devui/_discovery.py

@@ -127,7 +127,7 @@ async def load_entity(self, entity_id: str) -> Any:
 
         # Cache the loaded object
         self._loaded_objects[entity_id] = entity_obj
-        logger.info(f"✅ Successfully loaded entity: {entity_id} (type: {enriched_info.type})")
+        logger.info(f"Successfully loaded entity: {entity_id} (type: {enriched_info.type})")
 
         return entity_obj
 
@@ -217,7 +217,7 @@ def invalidate_entity(self, entity_id: str) -> None:
         if entity_info and "lazy_loaded" in entity_info.metadata:
             entity_info.metadata["lazy_loaded"] = False
 
-        logger.info(f"♻️  Entity invalidated: {entity_id} (will reload on next access)")
+        logger.info(f"Entity invalidated: {entity_id} (will reload on next access)")
 
     def invalidate_all(self) -> None:
         """Invalidate all cached entities.

python/packages/devui/agent_framework_devui/_executor.py

@@ -217,6 +217,11 @@ async def _execute_agent(
             Agent update events and trace events
         """
         try:
+            # Emit agent lifecycle start event
+            from .models._openai_custom import AgentStartedEvent
+
+            yield AgentStartedEvent()
+
             # Convert input to proper ChatMessage or string
             user_message = self._convert_input_to_chat_message(request.input)
 
@@ -266,8 +271,19 @@ async def _execute_agent(
             else:
                 raise ValueError("Agent must implement either run() or run_stream() method")
 
+            # Emit agent lifecycle completion event
+            from .models._openai_custom import AgentCompletedEvent
+
+            yield AgentCompletedEvent()
+
         except Exception as e:
             logger.error(f"Error in agent execution: {e}")
+            # Emit agent lifecycle failure event
+            from .models._openai_custom import AgentFailedEvent
+
+            yield AgentFailedEvent(error=e)
+
+            # Still yield the error for backward compatibility
             yield {"type": "error", "message": f"Agent execution error: {e!s}"}
 
     async def _execute_workflow(
@@ -284,14 +300,9 @@ async def _execute_workflow(
             Workflow events and trace events
         """
         try:
-            # Get input data - prefer structured data from extra_body
-            input_data: str | list[Any] | dict[str, Any]
-            if request.extra_body and isinstance(request.extra_body, dict) and request.extra_body.get("input_data"):
-                input_data = request.extra_body.get("input_data")  # type: ignore
-                logger.debug(f"Using structured input_data from extra_body: {type(input_data)}")
-            else:
-                input_data = request.input
-                logger.debug(f"Using input field as fallback: {type(input_data)}")
+            # Get input data directly from request.input field
+            input_data = request.input
+            logger.debug(f"Using input field: {type(input_data)}")
 
             # Parse input based on workflow's expected input type
             parsed_input = await self._parse_workflow_input(workflow, input_data)