nathan/frankgpt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(architecture): introduce Frank v6 modular skills-centric system

Browse Source 
Phase 1-4 Complete: Setup, Core Extraction, ITIL Specialty, Documentation

- Created v6/ folder with 3-layer architecture (core + skills + specialties)
- Extracted Frank.core.agent.md with universal personas and base commands
- Copied 7 skill modules (CRAFT, CoT, ToT, RAG, Markdown, Mermaid, Advanced Reasoning)
- Created specialty.itil.instructions.md for IT Service Management (ITIL v4)
- Added comprehensive ARCHITECTURE.md with usage patterns and migration guide
- Created v6/copilot-instructions.md for VS Code integration
- Organized legacy DOCX files into _Frank_/docx/ subdirectory
- Updated all cross-references to use v6 relative paths

Design Principles:
- Portability first: zero environment-specific paths
- Modular composition: load only what you need
- Multi-specialty support: combine domain experts
- Version compatibility: all files tagged v6.0

Ref: Session plan in /memories/session/plan.md
Next: Phase 3 (remaining specialties: devops, prompt-engineering, data-analysis, sccm)
This commit is contained in:
nathan
2026-04-19 14:07:29 -04:00
parent 11a2f32e30
commit b5ea5b175c
33 changed files with 3817 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
v6/skills/style.advanced-reasoning.instructions.md
+277
View File
@@ -0,0 +1,277 @@
## description: "A consolidated guide covering Chain-of-Thought (CoT) methods, advanced variants, program-aided reasoning, verification frameworks, and a documentation review checklist for authors and reviewers."
## A Guide to Advanced Reasoning and Problem-Solving Techniques
## Purpose and audience
This document consolidates foundational and advanced prompting techniques that help Large Language Models (LLMs) solve complex reasoning tasks. It's written for prompt engineers, AI researchers, documentation writers, and reviewers who need a practical reference and a checklist for producing and evaluating CoT-style prompts and artifacts.
## TL;DR
* Use Chain-of-Thought (CoT) to get LLMs to expose intermediate reasoning steps.
* Start with Zero-Shot CoT for quick wins; adopt Few-Shot or Auto-CoT when you can supply curated demonstrations.
* Improve robustness with Self-Consistency, Least-to-Most, and Plan-and-Solve techniques.
* Offload exact computation via Program-of-Thoughts (PoT) or Program-Aided Language models (PAL).
* Reduce hallucination and factual errors with Chain-of-Verification (CoVe).
## 1. Foundational CoT Techniques
These are the primary methods for implementing CoT prompting. For a detailed implementation guide and prompt templates, see the [Chain-of-Thought Deep Dive](style.cot.instructions.md).
### 1.1 Few-Shot CoT
Description: Provide a small set (3-4 recommended) of demonstrations that include: question, step-by-step reasoning (the chain), and the final answer.
When to use:
* Complex tasks with consistent reasoning structure.
* When you can author high-quality, diverse examples.
Pros/Cons:
* Strong guidance for the model.
* Manual and time-consuming to craft good demonstrations.
Tips:
* Keep examples concise but complete.
* Vary difficulty slightly to improve generalization.
### 1.2 Zero-Shot CoT
Description: Trigger reasoning by appending a simple phrase such as "Let's think step by step." No examples required.
When to use:
* Fast experiments, prototyping, or when demonstration data is unavailable.
Pros/Cons:
* No manual examples required.
* May be less reliable than high-quality Few-Shot demonstrations for hard problems.
### 1.3 Auto-CoT (Automatic CoT)
Description: Automates demonstration selection using clustering of questions and Zero-Shot CoT to generate reasoning chains for representative samples.
When to use:
* Large datasets where manual demo creation is impractical.
Pros/Cons:
* Scales to large datasets; improves diversity of prompts.
* Requires a pipeline for clustering and auto-generation; quality depends on clustering and zero-shot outputs.
### 1.4 Retrieval-Augmented Generation (RAG)
Description: Enhances LLM responses by integrating external knowledge sources, reducing hallucinations and providing up-to-date, verifiable information.
When to use:
* When answers require domain-specific, real-time, or proprietary information not present in the model's training data.
Pros/Cons:
* Improves accuracy and trustworthiness.
* Adds complexity and latency due to the retrieval step.
For detailed implementation patterns, see the [RAG Deep Dive](style.rag.instructions.md).
## 2. Advanced CoT Variants
### 2.1 Self-Consistency
How it works:
* Sample multiple reasoning trajectories from the model (different decoding seeds or temperature).
* Aggregate answers via majority vote or other consensus method.
Why it helps:
* Reduces sensitivity to single-path errors; harnesses diversity for more reliable answers.
Costs:
* Increased token consumption and compute.
### 2.2 Least-to-Most Prompting
How it works:
1. Decompose a hard problem into simpler sub-problems.
2. Solve sub-problems sequentially, passing prior solutions forward.
Why it helps:
* Breaks down complexity and reduces error accumulation.
### 2.3 Tree-of-Thought (ToT)
How it works:
* The model explores multiple reasoning paths (branches) simultaneously.
* It self-evaluates and prunes less promising branches, pursuing the most logical path.
Why it helps:
* Overcomes single-path failures common in standard CoT. Excellent for problems with complex decision spaces.
For detailed implementation guides and prompt templates, see the [Tree-of-Thought Deep Dive](style.tot.instructions.md).
Edge cases:
* Decomposition quality matters. Poor decompositions can hurt performance.
## 3. Program-Aided Reasoning
Offload deterministic computation to a real interpreter to avoid LLM numerical errors and enforce correctness.
### 3.1 Program-of-Thoughts (PoT)
* Prompt the LLM to output a program (commonly Python) that implements the reasoning.
* Execute the program in a trusted interpreter and return results.
Benefits:
* Accurate arithmetic and deterministic steps.
* Easier to test, debug, and unit-test logic.
Risks:
* Needs secure sandboxing for arbitrary code execution.
### 3.2 Program-Aided Language Models (PAL)
* Mixes natural language reasoning with interleaved code snippets for computation.
* Execute code snippets and feed results back into the reasoning chain.
When to prefer PoT vs PAL:
* PoT: fully program-first for heavy computation.
* PAL: when you want readable reasoning interleaved with code.
## 4. Verification and Refinement Techniques
### 4.1 Chain-of-Verification (CoVe)
A 4-step self-verification loop designed to reduce hallucinations:
1. Generate baseline response.
2. Plan verification questions targeted at weak claims.
3. Independently answer each verification question (avoid bias from baseline).
4. Produce a final, corrected answer using verification results.
When to use:
* High-stakes outputs or when factuality/trustworthiness matters.
Trade-offs:
* Extra costs and latency; significant gains in factual accuracy when verification steps are well-designed.
## 5. Practical Contract for Prompt Components
* Inputs: Natural language problem, (optional) demonstration set, optional execution sandbox for code.
* Outputs: Final answer, optional reasoning trace, and (when used) executed program and program output.
* Error modes: Calculation errors, omitted steps, nondeterminism, unsafe code in generated programs.
Edge cases to plan for:
* Empty or ambiguous user input.
* Very large or adversarial inputs.
* Long multi-step reasoning paths that exceed token limits.
* Security concerns when executing generated code.
Testing:
* Unit test with representative problems (happy path + 1-2 edge cases).
* Smoke test for program-execution paths (syntactic correctness and sandbox safety).
## 6. Documentation Review Checklist (for authors & reviewers)
Use this checklist when authoring or reviewing prompts, examples, and docs:
### Content & Accuracy
* Technical claims are supported or labeled as conjecture.
* References (papers, datasets) included where appropriate.
* No PII or unsafe content.
### Clarity & Readability
* Active voice and short sentences.
* Headings are front-loaded, under eight words when possible.
* Examples are minimal, runnable, and annotated.
### Formatting & Style
* Use inline code for tokens/code and fenced blocks for runnable snippets.
* Callouts: Note / Important / Warning used properly.
* Tables fully filled or marked N/A.
### Reproducibility
* Include exact prompt templates and variables using ${var} or clear placeholders.
* If code execution is required, include sandbox instructions and safety notes.
### Verification
* Add a suggested verification plan (CoVe-like) for non-trivial claims.
* Provide unit tests or example inputs/outputs when possible.
### Actionable Feedback (for reviewers)
* Offer concrete rewrites for unclear paragraphs.
* Convert long prose into numbered steps where sequence matters.
## 7. Minimal Examples
Zero-Shot CoT trigger (pseudoprompt):
Q: [problem statement]
A: Let's think step by step.
Few-Shot CoT (structure):
Q: Example question 1
A: [Step 1]. [Step 2]. The answer is X.
Q: New question
A:
PoT example sketch:
# LLM outputs a Python function that implements the logic
def solve(input):
# compute
return result
# Runner executes the function and returns the printed/returned value
## 8. Security and Operational Notes
* Do not execute generated code without sandboxing and resource limits.
* Log program executions and outputs for auditing.
* Rate-limit Self-Consistency or multi-sample methods to control cost.
## 9. Next Steps and Suggested Improvements
* Add curated Few-Shot demonstrations (3-4) as a companion demos/ file.
* Implement a small Auto-CoT pipeline (clustering + generator) and include reproducible scripts.
* Add unit tests for PoT-generated code (example harness + sandbox instructions).
## 10. References & Further Reading
(Representative pointers - include exact citations when publishing)
* Chain-of-Thought prompting literature.
* Auto-CoT and Self-Consistency papers.
* Program-of-Thoughts and Program-Aided Language Models (PoT, PAL).
* Chain-of-Verification (CoVe) verification methods.
v6/skills/style.cot.instructions.md
+74
View File
@@ -0,0 +1,74 @@
# Chain-of-Thought (CoT) Prompting Engine Guide
## 1. Prompting Techniques
There are three primary methods for implementing CoT prompting, each with its own advantages.
### 2.1. Few-Shot CoT
This is the standard approach where you provide the model with a few examples (demonstrations) that include a question, a step-by-step reasoning process (the chain of thought), and the final answer.
**When to Use:** Use this method for complex tasks where the reasoning structure is consistent and providing diverse, high-quality examples can significantly guide the model. This is the most powerful method but requires manual effort to create the demonstrations.
**Example Prompt Structure:**
Q: [Question 1]
A: [Step-by-step reasoning for Question 1]. The answer is [Answer 1].
Q: [Question 2]
A: [Step-by-step reasoning for Question 2]. The answer is [Answer 2].
Q: [New Question]
A:
The file demos/multiarith_manual provides a practical example of the JSON structure for these hand-crafted demonstrations.
### 2.2. Zero-Shot CoT
A surprisingly effective and simple method that requires no examples. By appending the phrase **"Let's think step by step"** to the end of a question, the model is triggered to generate a reasoning chain before giving the final answer.
**When to Use:** This is an excellent starting point for any reasoning task. It's highly effective for its simplicity and is particularly useful when you don't have time to create few-shot examples.
**Example Prompt Structure:**
Q: [New Question]
A: Let's think step by step.
The api.py script in the repository shows how this is implemented by setting a cot_trigger argument. The Jupyter notebooks (try_cot.ipynb and try_cot_colab.ipynb) demonstrate its application and output.
### 2.3. Automatic CoT (Auto-CoT)
Auto-CoT is an advanced technique designed to automate the creation of diverse and effective demonstrations for Few-Shot CoT, eliminating the manual effort. As detailed in the project's README.md, it works in two main stages.
**Stage 1: Question Clustering**
* The system takes a dataset of questions and groups them into several clusters based on semantic similarity.
**Stage 2: Demonstration Sampling**
* It selects a representative question from each cluster.
* It then uses **Zero-Shot CoT** to automatically generate a reasoning chain for each selected question.
This process, detailed in run_demo.py, ensures that the examples are both diverse (by sampling from different clusters) and accurate, creating a robust set of demonstrations for the model to learn from. The output of this process can be seen in the demos/multiarith_auto file.
**When to Use:** Use Auto-CoT when you need the high performance of Few-Shot CoT on a large dataset of questions but want to avoid the time-consuming and potentially suboptimal process of manually writing demonstrations.
## 3. Implementation in the Repository
The provided repository contains a full implementation of these techniques.
* **api.py**: A core file that defines the cot function, which can be called with different methods: "zero_shot", "zero_shot_cot", "manual_cot", and "auto_cot".
* **run_inference.py**: The main script for running experiments. It loads a dataset, constructs prompts based on the chosen method, and generates answers.
* **run_demo.py**: This script implements the Auto-CoT process by clustering questions and generating demonstrations.
* **try_cot.ipynb**: A Jupyter Notebook that provides a quick and clear way to test and compare the outputs of each CoT method.
To get started, refer to the README.md and the try_cot_colab.ipynb for a guided walkthrough.
## 4. References
* [Amazon Science Repo on CoT](https://github.com/amazon-science/auto-cot)
* [CoT Example](../knowledge/example.CoT-Prompting.md)
v6/skills/style.craft.instructions.md
+179
View File
@@ -0,0 +1,179 @@
---
description: "Defines the C.R.A.F.T. framework (Context, Role, Action, Format, Tone/Audience) and provides templates, examples, and an author checklist for crafting prompts."
applyTo: "\*\*"
---
## The C.R.A.F.T. Framework
All prompt generation, analysis, and refactoring must be performed through the lens of the C.R.A.F.T. framework.
- **Context:** Background, inputs, and constraints the model needs to understand the task.
- **Role:** The persona or skill the model should adopt (expert, teacher, analyst, etc.).
- **Action:** A single, clear imperative describing what the model should do.
- **Format:** The exact output structure (schema, file type, or layout) required.
- **Tone / Audience:** The writing style and the intended reader (e.g., "concise for executives").
```instructions
---
description: "Defines the C.R.A.F.T. framework (Context, Role, Action, Format, Tone/Audience) and provides templates, examples, and an author checklist for crafting prompts."
applyTo: "\*\*"
---
## The C.R.A.F.T. Framework
All prompt generation, analysis, and refactoring must be performed through the lens of the C.R.A.F.T. framework.
- **Context:** Background, inputs, and constraints the model needs to understand the task.
- **Role:** The persona or skill the model should adopt (expert, teacher, analyst, etc.).
- **Action:** A single, clear imperative describing what the model should do.
- **Format:** The exact output structure (schema, file type, or layout) required.
- **Tone / Audience:** The writing style and the intended reader (e.g., "concise for executives").
## Purpose and scope
This file provides a prescriptive template and practical guidance for writing prompts used in `.prompt.md`, `.chatmode.md`, and other instruction-oriented Markdown files. Use the minimal template for short one-off prompts and the extended template for complex or reusable prompts.
## Minimal CRAFT prompt template (copy & fill)
Context: ${short context — 1–2 sentences describing input, environment, constraints}
Role: ${persona — e.g., "Senior Data Scientist"}
Action: ${single imperative verb + brief object — e.g., "Summarize the report into 5 bullet points"}
Format: ${output form — e.g., "Markdown numbered list" or "JSON: {summary, details}"}
Tone / Audience: ${tone and audience — e.g., "plain English for product managers"}
Example (minimal):
Context: You are given a 10-page product requirements document about a new payments feature.
Role: Product manager summarizer.
Action: Extract the top 6 functional requirements and the 3 main risks.
Format: Markdown with H2 sections "Requirements" and "Risks" and numbered lists underneath.
Tone / Audience: Executive-level, concise.
## Extended CRAFT template (for complex or reusable prompts)
Include the minimal template plus these fields when the task is non-trivial or will be reused.
- Inputs: Named inputs expected by the prompt (e.g., `file: report.md`, `variables: {start_date, end_date}`).
- Constraints: Hard limits and rules (e.g., "max 150 words", "no invented facts", "must include citations").
- Examples: 1–2 minimal input → output examples showing exact format and level of detail.
- Verification: How outputs will be checked (simple rubric, unit test, or follow-up verification prompts).
- Metadata: Optional changelog, author, and `applyTo` recommendations for `.instructions.md` files.
Extended example:
Context: You have a CSV of customer support tickets with columns {id, created_at, category, resolution_time, text}.
Role: Senior analyst who writes reproducible summaries.
Action: Produce an executive summary of trends for the prior quarter and recommend 3 operational changes.
Format: 1) A 3-paragraph executive summary (max 150 words). 2) A Markdown table of top 5 categories and their avg resolution_time. 3) A numbered list of 3 recommendations.
Tone / Audience: Non-technical VP of Support; use plain English and define any domain terms.
Inputs: `file: tickets_q3.csv`
Constraints: Do not extrapolate outside the data. Include the exact SQL used to compute the table (one-line code fence). Max 150 words for the executive summary.
Examples:
- Input: (small sample rows) → Output: (example summary)
Verification: Check that the table rows match results from the provided SQL.
## Quick author checklist (pre-submit)
- [ ] Context gives necessary background and scope (who, what, when, where).
- [ ] Role is an actionable persona (one sentence) and matches desired expertise.
- [ ] Action is a single, clear imperative (avoid compound verbs like "analyze and decide").
- [ ] Format is machine- or human-readable and unambiguous (include a schema when needed).
- [ ] Tone/Audience is specified and consistent with examples.
- [ ] Inputs and Constraints are explicit for any data-driven task.
- [ ] Examples (if present) are minimal, representative, and follow the expected output format.
- [ ] Verification steps or acceptance criteria are stated (e.g., "3 bullets, each <= 20 words").
## Common failure modes and mitigations
- Failure: Model ignores provided inputs or tools.
Mitigation: Make Role explicitly state "You must use the provided inputs/tools and must not hallucinate" and add a Verification step.
- Failure: Output format drift (e.g., prose instead of JSON).
Mitigation: Provide a strict schema and a minimal example JSON instance; instruct "Return only valid JSON".
- Failure: Overly long responses.
Mitigation: Add a hard constraint like "Max 150 words" and show a short Format example.
## Small reusable prompt snippets
Use these snippets to speed prompt authoring and reduce errors. Replace variables in braces.
- "If you cannot answer from the inputs, respond with exactly: 'I don't know based on the provided data.'"
- "Return only valid JSON conforming to this schema: {\"summary\":string, \"items\": [ {\"id\":int, \"note\":string} ] }"
- "Cite the source line or file for any factual claim in the format: (source: <filename>:<line-range>)"
## Evaluation rubric (prompt quality)
Score the prompt before reusing it.
1 — Poor: Missing components; likely to produce ambiguous results.
2 — Fair: Most components present but missing constraints or examples.
3 — Good: Complete CRAFT, includes constraints and short examples.
4 — Excellent: Complete CRAFT, includes verification rules, example outputs, and explicit anti-hallucination language.
## Implementation notes for `.instructions.md` files
- Keep YAML frontmatter (`description`, `applyTo`) accurate and concise.
- `applyTo` should be as specific as practical (e.g., `"**/*.prompt.md"` or `"docs/**"`).
- If this instruction file is also a template, add a short example prompt at the bottom of the file inside a fenced block and maintain a changelog comment at the top.
## References and source materials
This guidance draws on research and practitioner summaries in `Training Guides/Updating CRAFT/` and the CRAFT (toolset) paper by Yuan et al. (ICLR 2024). Use those materials for deeper background when creating complex, reusable prompts.
## Contact and iteration
When you iteratively improve a prompt template, add a one-line changelog at the top with date and reason. Small iterative changes are encouraged.
v6/skills/style.markdown.instructions.md
+193
View File
@@ -0,0 +1,193 @@
## description: "Markdown style guide" applyTo: "**/*.md"
# Markdown Style Guide
## Introduction
This consolidated Markdown style guide combines our existing rules with widely-used best practices and examples from the Markdown reference material. It covers basic syntax, extended features, compatibility notes, and a few safe "hacks" when HTML support is available.
Use this guide when authoring documentation, READMEs, and other Markdown content in the repository. When in doubt, prefer CommonMark/GitHub Flavored Markdown (GFM) compatible constructs for the best cross-tool behavior.
## Headings
Use ATX-style headings (hash marks) and put a single space after the hashes. Start documents with # for the main title and do not skip levels (e.g., don't jump from ## to ####). Add a blank line before and after headings for better compatibility with Markdown processors.
Rules:
* Use one space after # (e.g., ## Section title).
* Use sentence case for headings.
* Keep heading depth meaningful and avoid skipping levels.
Good:
# Project Phoenix
## Overview
### Requirements
Avoid:
-#MissingSpace
## Paragraphs and Line Breaks
Paragraphs are separated by one blank line. Avoid indenting normal paragraphs with spaces or tabs (unless intentionally creating a code block).
To create a line break (soft break), prefer an explicit <br> tag or use two trailing spaces at the end of a line for compatibility; note that trailing spaces are easy to miss in source.
Rules:
* Use a blank line to separate paragraphs.
* Avoid leading spaces/tabs on paragraph lines.
* For visible new lines inside a paragraph: use two trailing spaces + Enter or <br> when supported.
## Emphasis (Bold, Italic)
Prefer asterisks for intra-word emphasis to avoid processor differences with underscores.
Rules:
* Italic: *text*
* Bold: **text**
* Bold + Italic: ***text***
Do not rely on underscores for mid-word emphasis (e.g., use Love**is**bold not Love__is__bold).
## Lists
Use hyphens (-) for unordered lists for consistency. For ordered lists use 1. (the renderer will number list items correctly). Indent nested list content with four spaces.
Rules:
* Unordered lists: - item
* Ordered lists: 1. item
* Indent nested lists with 4 spaces.
* Don't mix list delimiters within the same list.
To keep lists readable, put a blank line before the list and between list blocks when appropriate.
## Links and Images
Use inline link syntax: [text](https://example.com) and provide descriptive link text. For images use the same pattern prefixed with ! and always include alt text.
Rules:
* Links: [label](https://example.com)
* Images: ![alt text](path/to/image.png)
* Avoid click here as link text; be descriptive instead.
If you need links to open in a new tab or to add attributes, use HTML anchors when supported (e.g., <a href="..." target="_blank">).
## Code
Inline code: use single backticks: `code`. Use fenced code blocks (three backticks) for longer snippets. Specify the language for syntax highlighting when supported (e.g., json or python).
Rules:
* Inline: `variable`
* Block:```
def fn():
return True
```
- Add a language after the opening fence for highlighting: ```python
When you need backticks inside a fenced code block, you can fence with a larger number of backticks.
## Blockquotes
Use > to create blockquotes. Put blank lines before and after blockquotes for better compatibility. Blockquotes can contain headings, lists, and other block elements, but remember not every processor supports every combination.
## Horizontal Rules
Use three or more hyphens (---) for a visual section break.
## Extended Syntax (Tables, Footnotes, IDs, etc.)
Note: Extended features vary by processor. Prefer CommonMark/GFM-compatible constructs and check your target renderer.
Tables:
- Use pipes (|) and hyphens (---) to build tables. Add pipes at the ends of rows for readability.
- Align columns with colons in header separators: :---, :---:, ---:.
- Avoid complex block-level content inside table cells; if needed, use HTML.
Example:
```markdown
| Name | Role |
| --- | --- |
| Alice | Developer |
```
Fenced code blocks and syntax highlighting:
- Use triple backticks and specify a language for highlighting (```json, ```bash, etc.).
Footnotes:
- Use footnote references like [^1] and define them [^1]: note text anywhere in the document (not inside lists/tables). Footnotes are numbered in output.
Heading IDs and anchor links:
- Many processors support {#custom-id} after a heading. Use these for internal linking and ToC generation.
Definition lists, strikethrough, task lists, emoji, highlights, subscript, superscript:
- These are supported in various extended syntaxes (GFM/MultiMarkdown). Use them when your renderer supports them:
- Definition lists: Term\n: Definition
- Strikethrough: ~~text~~
- Task lists: - [ ] and - [x]
- Emoji shortcodes: :joy: (renderer-dependent)
- Highlight: ==text== (not widely supported)
- Subscript: H~2~ (renderer-dependent)
- Superscript: X^2^ (renderer-dependent)
## Automatic URL Linking
Many renderers auto-link bare URLs (e.g., http://example.com). To prevent linking, mark a URL as code: `http://example.com`.
## Hacks and HTML Fallbacks (Use Sparingly)
If your Markdown processor allows raw HTML, some layout or styling needs can be solved with HTML. Use these hacks only when necessary and document the dependency on HTML support.
Common fallbacks:
- Centering: <p style="text-align:center">Text</p> or deprecated <center> tag.
- Color: <span style="color:blue">text</span> (avoid unless necessary).
- Image sizing / captions: use <img width="200" height="100" src="..."> or <figure><figcaption> when supported.
- Comments (hidden in output): [comment]: # (hidden note) or [This is a comment]: # (processor-dependent but widely used).
- Table cell line breaks and lists: use <br> or HTML lists inside table cells.
Warning: HTML tags like <font> and <center> are deprecated; prefer CSS when available.
## Accessibility and Best Practices
- Always provide alt text for images.
- Use meaningful link text for screen reader users.
- Keep tables simple and avoid using them for layout.
- For long documents, consider adding a Table of Contents with heading links.
## Quick Cheat Sheet (Common patterns)
- Heading: # H1 / ## H2
- Bold / Italic: **bold** / *italic* / ***bold italic***
- Code inline: `code`
- Code block: ```python\nprint()\n```
- Link: [label](https://example.com)
- Image: ![alt](image.png)
- Table: | col | col |\n| --- | --- |\n| a | b |
- Task list: - [ ] todo / - [x] done
- Strikethrough: ~~no longer~~
## Notes on Compatibility
When in doubt follow CommonMark/GFM (GitHub Flavored Markdown) conventions. Always test documents in the target renderer (GitHub, MkDocs, VS Code preview, etc.) before publishing.
## References
- CommonMark: https://commonmark.org
- GitHub Flavored Markdown: https://github.github.com/gfm/
- The Markdown Guide: https://www.markdownguide.org
v6/skills/style.mermaid.instructions.md
+155
View File
@@ -0,0 +1,155 @@
---
description: "Style guide for Mermaid.js diagrams. Enforces consistency, readability, and maintainability."
applyTo: "**/*.md"
---
# Mermaid Diagram Style Guide
## 1. Introduction
This guide establishes standards for creating code-based diagrams using **Mermaid.js**. Because diagrams are treated as code, they must be clean, readable, and version-controllable.
**Core Principle:** A diagram's source code should be as readable as the rendered image.
## 2. Graph Direction
Choose the orientation based on the data flow.
* **Rule:** Use `TD` (Top-Down) for hierarchies, decision trees, and organizational charts.
* **Rule:** Use `LR` (Left-Right) for pipelines, timelines, and sequential data flows.
* **Rule:** Use `flowchart` instead of the older `graph` keyword for better rendering support.
>Example of Top-Down:
```mermaid
flowchart LR
Input --> Process --> Output
```
## 3. Node Identifiers
Separate the **Node ID** from the **Node Label**.
* **Rule:** Use semantic, `kebab-case` or `snake_case` IDs. Avoid single letters (`A`, `B`, `C`).
* **Rule:** IDs must be descriptive enough to understand links without reading the label.
* **Why:** If you change the label text later, you won't break the logic/connections.
**Good:**
```mermaid
flowchart TD
user_input[User enters credentials] --> validate_login{Valid?}
validate_login -- Yes --> db_query[(Database)]
```
**Bad:**
```mermaid
flowchart TD
A[User enters credentials] --> B{Valid?}
B -- Yes --> C[(Database)]
```
## 4. Standard Shapes
Use consistent shapes to convey meaning immediately.
* **Rule:** Use `[]` (Rectangle) for standard processes/steps.
* **Rule:** Use `{}` (Rhombus) **only** for decisions/conditionals.
* **Rule:** Use `[()]` (Cylinder) for databases and storage.
* **Rule:** Use `(())` (Circle) for start/end points or connectors.
## 5. Connections & Arrows
Keep connections clean.
* **Rule:** Use `-->` for standard flow.
* **Rule:** Use `-.->` (dotted) for optional, asynchronous, or future flows.
* **Rule:** Add text labels to arrows *only* when a decision is made or the relationship needs clarification.
* **Rule:** Use pipes `|text|` for arrow labels, not the older syntax.
```mermaid
flowchart TD
scan_start[Start Scan] -->|Success| log_entry[Log Result]
scan_start -.->|Timeout| retry_queue[Retry Queue]
```
## 6. Sequence Diagrams
For showing interactions over time.
* **Rule:** Always enable `autonumber` to make referencing steps in conversation easier.
* **Rule:** Define `participant` or `actor` aliases at the top for clarity.
```mermaid
sequenceDiagram
autonumber
actor U as User
participant A as API
participant D as Database
U->>A: Request Data
A->>D: Query ID
D-->>A: Return Payload
```
## 7. Subgraphs (Grouping)
Use subgraphs to cluster related components (e.g., separating "Cloud" from "On-Prem").
* **Rule:** Indent subgraph content by **4 spaces**.
* **Rule:** Give subgraphs descriptive IDs.
* **Rule:** Label the subgraph clearly using the `subgraph ID [Label]` syntax.
```mermaid
flowchart LR
subgraph aws [AWS Cloud]
lb[Load Balancer] --> web[Web Server]
end
subgraph on_prem [Office]
user[Laptop] --> lb
end
```
## 8. Styling (Classes)
Do not use inline styles (e.g., `style A fill:#f9f`). It creates "spaghetti code."
* **Rule:** Use `classDef` at the bottom of the file to define themes.
* **Rule:** Apply classes using the `:::` operator.
* **Standard Classes:**
* `classDef failure fill:#f88,stroke:#333;`
* `classDef success fill:#8f8,stroke:#333;`
**Example:**
```mermaid
flowchart TD
build[Build Code] --> test{Tests Pass?}
test -- No --> alert[Alert Team]:::failure
test -- Yes --> deploy[Deploy]:::success
classDef failure fill:#ffcccc,stroke:#ff0000;
classDef success fill:#ccffcc,stroke:#00ff00;
```
## 9. Linting & Formatting
* **Indentation:** Use 4 spaces for nested elements.
* **Spacing:** Put spaces around arrow connectors for readability (`A --> B`, not `A-->B`).
* **Comments:** Use `%%` for comments to explain complex logic.
## 10. Quick Cheat Sheet
| Type | Syntax | Output Shape |
| --- | --- | --- |
| Process | `id[Text]` | Rectangle |
| Decision | `id{Text}` | Diamond |
| Database | `id[(Text)]` | Cylinder |
| Terminal | `id([Text])` | Rounded Pill |
| Subroutine | `id[[Text]]` | Double Border |
| Comment | `%% Text` | Invisible |
v6/skills/style.rag.instructions.md
+78
View File
@@ -0,0 +1,78 @@
# Prompt Engine Instruction File: Retrieval-Augmented Generation (RAG)
## 1. Core RAG Paradigms
The implementation of RAG can be categorized into three main paradigms, each evolving from the last:
### 2.1. Naive RAG
[cite_start]This is the most straightforward implementation of RAG, following a simple "Retrieve-Read" framework[cite: 178].
* **Indexing:** Documents are cleaned, extracted, and segmented into smaller chunks. [cite_start]These chunks are then converted into vector embeddings and stored in a vector database[cite: 179, 180, 181].
* **Retrieval:** When a user submits a query, it's converted into a vector. [cite_start]The system then searches the vector database for the top-K most similar document chunks[cite: 183, 184, 185].
* [cite_start]**Generation:** The retrieved chunks and the original query are combined into a prompt that is fed to the LLM to generate an answer[cite: 187].
### 2.2. Advanced RAG
[cite_start]This paradigm introduces optimizations to the Naive RAG process to improve retrieval quality[cite: 200, 201].
* **Pre-retrieval:** This stage focuses on optimizing the indexing process and the user query itself. [cite_start]Techniques include enhancing data granularity, adding metadata to chunks, and query rewriting or expansion[cite: 265, 266, 267, 268, 269].
* [cite_start]**Post-retrieval:** After retrieving documents, this stage involves re-ranking the chunks to place the most relevant information at the beginning and end of the prompt (to counter the "lost in the middle" problem) and compressing the context to remove noise and irrelevant information[cite: 270, 271, 272, 274, 275].
### 2.3. Modular RAG
[cite_start]The most flexible and adaptable paradigm, Modular RAG allows for the addition of specialized modules and the reconfiguration of the RAG pipeline[cite: 277].
* [cite_start]**New Modules:** This can include a Search module for direct access to various data sources, a Memory module that uses the LLM's memory to guide retrieval, and a Routing module to select the best data source for a given query[cite: 283, 285, 288].
* [cite_start]**New Patterns:** Instead of a fixed "Retrieve-Read" sequence, Modular RAG can employ more complex patterns like Rewrite-Retrieve-Read or Generate-Read[cite: 294, 295]. [cite_start]It also allows for adaptive retrieval, where the model decides when and what to retrieve[cite: 300, 301].
## 3. Key Components & Optimization Techniques
### 3.1. Retrieval
The quality of the retrieval process is crucial for the success of any RAG system.
* [cite_start]**Chunking Strategy:** Instead of fixed-size chunks, consider recursive splitting or a "small2big" approach where smaller, more precise chunks are retrieved, but the surrounding context is provided to the LLM[cite: 404, 406].
* **Query Optimization:**
+ [cite_start]**Expansion:** Expand a single query into multiple, more specific queries to cover different aspects of the user's intent[cite: 429, 430].
+ **Transformation:** Rewrite the user's query to be more suitable for retrieval. [cite_start]Techniques like HyDE (Hypothetical Document Embeddings) generate a hypothetical answer to the query and use its embedding for retrieval[cite: 438, 439, 444].
+ [cite_start]**Routing:** Use a router to direct the query to the most appropriate data source or RAG pipeline based on its content or metadata[cite: 448, 449, 450].
* **Embedding:**
+ [cite_start]**Fine-tuning:** For domain-specific applications, fine-tune the embedding model on your own dataset to improve its understanding of specialized jargon[cite: 466].
+ [cite_start]**Hybrid Retrieval:** Combine sparse retrieval methods (like BM25) with dense retrieval to leverage the strengths of both[cite: 460].
### 3.2. Generation
Simply feeding all retrieved information to the LLM is not optimal.
* **Context Curation:**
+ [cite_start]**Reranking:** Reorder the retrieved chunks to place the most relevant information at the beginning and end of the context[cite: 495].
+ [cite_start]**Compression:** Use a smaller LLM to compress the retrieved context by removing unimportant tokens, making it more digestible for the main generator LLM[cite: 499].
* [cite_start]**LLM Fine-tuning:** Fine-tune the generator LLM on domain-specific data to improve its ability to understand the retrieved context and generate responses in a specific style or format[cite: 512, 514, 517].
### 3.3. Augmentation Process
The interaction between retrieval and generation can be optimized.
* **Iterative Retrieval:** The LLM generates a response, and then another retrieval step is performed based on the generated text to gather more information. [cite_start]This process can be repeated multiple times[cite: 530].
* **Recursive Retrieval:** Break down a complex query into a series of sub-queries. [cite_start]The results from each sub-query are used to inform the next, creating a chain of reasoning[cite: 571, 572, 573].
* **Adaptive Retrieval:** Allow the LLM to decide when it needs to retrieve information. [cite_start]This can be achieved by using special tokens that trigger the retrieval process when the LLM's generation confidence is low[cite: 583, 589, 592].
## 4. Evaluating RAG Systems
Evaluating a RAG system goes beyond measuring the final answer's accuracy.
* **Evaluation Targets:**
+ [cite_start]**Retrieval Quality:** Measured by metrics like Hit Rate, MRR, and NDCG[cite: 611, 614].
+ [cite_start]**Generation Quality:** Assessed based on faithfulness (does the answer contradict the source?), relevance, and non-harmfulness[cite: 615, 617].
* **Required Abilities:**
+ [cite_start]**Noise Robustness:** Can the model handle irrelevant or noisy documents in the retrieved context[cite: 629]?
+ [cite_start]**Negative Rejection:** Does the model know when to say "I don't know" if the answer is not in the retrieved documents[cite: 630]?
+ [cite_start]**Information Integration:** How well can the model synthesize information from multiple sources[cite: 631]?
+ [cite_start]**Counterfactual Robustness:** Can the model identify and ignore inaccuracies in the source documents[cite: 632]?
[cite_start]Several benchmarks and tools, such as RAGAS, ARES, and TruLens, can be used for a more systematic evaluation of RAG models[cite: 648].
## 5. References
* [RAG Example](../knowledge/example.RAG-Token.md)
v6/skills/style.tot.instructions.md
+33
View File
@@ -0,0 +1,33 @@
# Prompt Engine Instruction File: Tree-of-Thought Prompting
## 1. ToT Prompting Techniques
Here are several ToT prompts that can be adapted for various tasks:
### 1. The Expert Collaboration Prompt
This prompt encourages a step-by-step, collaborative reasoning process.
"Imagine three different experts are answering this question. All experts will write down 1 step of their thinking, then share it with the group. Then all experts will go on to the next step, etc. If any expert realises they're wrong at any point then they leave. The question is..."
### 2. The Verbose Expert Simulation
This prompt generates a more detailed and interactive reasoning process.
"Simulate three brilliant, logical experts collaboratively answering a question. Each one verbously explains their thought process in real-time, considering the prior explanations of others and openly acknowledging mistakes. At each step, whenever possible, each expert refines and builds upon the thoughts of others, acknowledging their contributions. They continue until there is a definitive answer to the question. For clarity, your entire response should be in a markdown table. The question is..."
### 3. The Peer-Scoring Expert Prompt
This prompt introduces a scoring mechanism for self-evaluation.
"Identify and behave as three different experts that are appropriate to answering this question. All experts will write down the step and their thinking about the step, then share it with the group. Then, all experts will go on to the next step, etc. At each step all experts will score their peers response between 1 and 5, 1 meaning it is highly unlikely, and 5 meaning it is highly likely. If any expert is judged to be wrong at any point then they leave. After all experts have provided their analysis, you then analyze all 3 analyses and provide either the consensus solution or your best guess solution. The question is..."
## Application and Best Practices
* **Complex Reasoning:** ToT is particularly effective for questions that require multi-step reasoning and where the initial line of thought can be misleading.
* **Adaptability:** The number of "experts" and the specific rules of interaction can be modified to suit the complexity of the task.
* **Clarity:** The structured output of ToT prompts makes it easier to follow the LLM's reasoning process and identify where and why it made certain decisions.
## 4. References
* [Tree of Thought Examples](../knowledge/example.ToT-Prompting.md)