Proposal: Notebook guide for RAG failure analysis using WFGY ProblemMap (MIT-licensed) #316

Hi, and thanks for Deepnote – collaborative notebooks are becoming the default workspace for many ML and data teams.

I maintain an open-source project called WFGY (MIT-licensed, ~1.5k GitHub stars). One of its pillars is a 16-problem “ProblemMap” that enumerates common RAG / LLM failure modes end to end:

• WFGY ProblemMap (16 failure modes for RAG & LLM pipelines):
  https://github.com/onestardao/WFGY/blob/main/ProblemMap/README.md

This checklist has been integrated or referenced by:

• ToolUniverse from Harvard MIMS Lab
• Multimodal RAG Survey from QCRI LLM Lab

They use it as a practical taxonomy when debugging RAG systems.

Proposal

Since many people prototype RAG pipelines inside Deepnote, I would like to propose an official notebook guide that:

1. Walks through building a minimal RAG pipeline in Deepnote
2. Shows how to log and tag failures using the 16 ProblemMap categories (retriever issues, chunking problems, etc.)
3. Encourages teams to treat ProblemMap as a shared language when commenting and collaborating inside notebooks

The notebook would be Deepnote-centric and could live in an official examples gallery or template set. WFGY ProblemMap would simply be linked as an external, MIT-licensed reference taxonomy.

Question

Would you be open to:

• Hosting such a notebook as an official example / template, and
• Including a short reference to the WFGY ProblemMap for the failure-mode labels?

If that sounds interesting, I can draft the notebook in your preferred format and get feedback before any formal PR.

If not, completely understood – thanks again for providing Deepnote.