Yufei Wang

Enterprise Agent Memory, Retrieval & Agent Harness

Mar 2026 – Present

Built a scalable memory service that turns massive, heterogeneous enterprise histories (chats, documents, email, meetings, app events) into a searchable memory store, and answers questions with cited evidence through agentic retrieval. The system reached #1 on the EnterpriseRAG-Bench leaderboard.

• Built a concurrent LLM-based memory extraction pipeline supporting a 512k-document corpus, generating dual-resolution memories with embeddings, cue anchors and entity facets stored in PostgreSQL/pgvector; achieved 3x higher throughput through buffered bulk writes, embedding pre-warm and deferred deletes.
• Implemented an offline memory lifecycle with union-find clustering, cross-batch reconciliation, temporal fact supersession and access-decay scoring; optimized similarity computation with blocked incremental scans, reducing memory usage by 82% (60 GB → 11 GB) and the merge-matrix footprint by 99% (25 GB → 256 MB), while improving retrieval accuracy from 67% to 70% and cutting median latency by 19%.
• Integrated LLM-generated predictive cues and a hybrid cognitive retriever, improving LoCoMo-Plus accuracy from 46% to 73% and session recall from 30% to 99%.
• Rearchitected a policy-driven retriever into a plan-based, multi-channel pipeline with LLM query planning, HyDE, hybrid BM25/dense/cue retrieval fused via RRF, and parallel sharded LLM reranking to mitigate position bias; improved LoCoMo accuracy from 83.8% to 93.3% and EnterpriseRAG-Bench from 70.1% to 83.2%.
• Replaced in-memory BM25 with a disk-based Tantivy search engine featuring crash-safe synchronization, leader election and incremental indexing; reduced index storage by 98% (79 GB → 1.5 GB) and accelerated index builds by 2.7x (4,146 s → 1,557 s).
• Built a tool-use retrieval agent with candidate adjudication, verification and per-document enrichment, improving EnterpriseRAG-Bench from 83.2% to 90.91% and BEAM-10M accuracy from 55% to 71%.
• Built an evaluation stack with four configurable quality–latency tiers, multi-provider LLM routing via LiteLLM, and token/cost tracking using Python and FastAPI.
• Developed an asynchronous Python/FastAPI and PydanticAI agent harness for an Electron desktop app, integrating MCP tools, dynamic skills, human-in-the-loop approvals and real-time streaming for auditable agent workflows.
• Developed "My Day", integrating Gmail, Slack, Google Workspace, Jira and Calendar for LLM-based task triage and reply drafting; built an ontology-driven knowledge system with typed relationships, temporal reconciliation and incremental updates for personalized agent context.

Technologies: Python, FastAPI, PydanticAI, LLM agents, RAG, PostgreSQL/pgvector, Tantivy, LiteLLM, MCP, Electron.

• Next Project

AI Security Assistants & Cloud Infrastructure Intelligence

Feb 2025 – Sep 2025

Developed cloud infrastructure AI assistants powered by digital-twin knowledge graphs to support security analysis, compliance verification and infrastructure impact simulation.

• Developed multi-turn AI assistants using Go, OpenAI, WebSocket, FalkorDB and Milvus, implementing persistent conversation memory, context-window management, streaming responses and graph-grounded retrieval for cloud infrastructure analysis.
• Built a Text-to-Cypher retrieval pipeline combining LLM-generated graph queries, vector-based query caching, schema-aware prompting and read-only query validation to enable targeted infrastructure knowledge retrieval.
• Implemented a compliance policy management system using Go and PostgreSQL, supporting custom policies, configurable severity levels, and PCI DSS, NIST 800-53 and SOC 2 mappings to dynamically guide LLM-based Terraform impact analysis.
• Developed an automated LLM evaluation platform using Python, RAGAS, Arize Phoenix and FastAPI, implementing semantic similarity and hallucination detection; integrated scheduled regression evaluations into GitHub Actions CI/CD workflows to monitor agent quality across releases.

Technologies: Go, Python, OpenAI, FalkorDB, Cypher, Milvus, PostgreSQL, FastAPI, RAGAS, Arize Phoenix, Terraform, Docker, GitHub Actions.

• Earlier Experience

Software Engineer Internship at Feel

Earlier experience

Feel is a new type of social media application geared towards supporting its users’ emotional wellbeing. On Feel, users can log and track their emotions to keep a pulse on how they are feeling. I involed in the chatbot development where users can talk with for additional support. Specifically, the chatbot can act as an empathetic listening ear, ask meaningful questions, recommend resources, comfort users, etc.

I firstly started by designing rigid conditions to decide wether the bot would act as a friend or a therapist using intent analysis and sentiment analysis powered by LangChain. Then I optimzed the by function calling chatbot using tools parameter to specify additional tools or features that should be enabled or utilized during the completion (or response generation) process.

The bot can also memorize what users said a long time ago, by vectorizing the conversations and storing in the PGVector concurrently while the bot is answering the user's questions.

Suggested by my manager, I further optimized the bot using Semantic Kernel developed by Microsoft, which involved developments of plugins to lead to more efficient and responsive applications.

I also created a Python script to evaluate the chatbot's performance on the test set using the BLEU metric and GPT-4o scoring to for better hyperparameter fine-tuning and prompt engineering.

• Know More

AI-driven recommendation assistant

The internship focused on developing a recommendation assistant for agents of Ping’an of China, one of the largest insurance companies in China, utilizing the open-source Large Language Model, ChatGLM. The recommendation process involved a four-step reasoning procedure designed to ensure a personalized and effective user experience. Additionally, the system can also provide recommendation rationales, helping users comprehend the reasons behind the recommendations it offered. For instance,

If a user requested recommendations for medical insurance for a 61-year-old individual, the system would respond as follows: (1/4) Recognizing the need for products recommendation and I am proceeding to offer suggestions. (2/4) Identifying various product options, such as "Hospital Insurance" and "Worry-Free Medical Care."." (3/4) Based on your description, "Worry-Free Medical Care" is more suitable for you. (4/4) Here are the reasons: The insurance age for this insurance is 18-65 years old, and the coverage covers 120 diseases, etc..

My responsibilities included data preprocessing, where I analyzed and cleaned the raw JSON data, and extracted keywords related to product characteristics using an IDF file. Due to lack of user queries for testing, I collaborated with the product manager team to build 10 test set consisting of 10 queries across 3 real-word scenarios: elderly individuals, frequent business travelers, and children. Furthermore, we opted to evaluate the model using two key metrics: "Helpfulness" to ensure that the model provides useful information, and "Honestness" to guarantee that the model refrains from fabricating information or misleading users.

I also engaged in prompt engineering by experimenting with different prompts, such as defined roles (e.g., assuming you are an excellent and professional account manager), specified scenes (e.g., explaining recommended products to users), word limits (e.g., 200 words), delimiters, incorporating non-linguistic context (e.g., marked with colons), and style induction(e.g., introduce friendly).

Using proper prompts, the system can achieve a 10/10 score in terms of Honestness.

• Other Projects

Online Dating Web Application

• Detail

HDFS Emulation

• Detail
• Demo

Click Through Rate Prediction

• Detail

LA Crime Analysis

• Detail