LLM Architect Skill

This skill provides a comprehensive workflow and resources for building unique Large Language Model (LLM) architectures and showcasing these skills on GitHub.

Core Workflows

When asked to help with LLM architecture or portfolio building, follow these workflows based on the user's specific goal:

1. Building an LLM from Scratch

When the user wants to understand core architecture by building from scratch:

1. Read /home/ubuntu/skills/llm-architect/references/architecture_guide.md for foundational concepts (Transformers, Attention, Tokenization).
2. Use the starter script /home/ubuntu/skills/llm-architect/scripts/transformer_base.py to set up the basic PyTorch architecture.
3. Guide the user through implementing Encoder-Decoder blocks and Self-Attention mechanisms.

2. Fine-Tuning Open Source Models

When the user wants to adapt existing models (e.g., Llama, Falcon) for specific tasks:

1. Read /home/ubuntu/skills/llm-architect/references/fine_tuning_guide.md for PEFT, LoRA, and QLoRA techniques.
2. Provide the configuration template /home/ubuntu/skills/llm-architect/templates/lora_config.yaml for setting up the fine-tuning environment.
3. Guide the user through the instruction tuning or RLHF process.

3. Implementing End-to-End RAG Systems

When the user wants to build practical applications with custom context:

1. Read /home/ubuntu/skills/llm-architect/references/rag_guide.md for application architecture and vector database integration.
2. Use the starter script /home/ubuntu/skills/llm-architect/scripts/rag_pipeline.py to set up the basic LangChain and vector DB connection.
3. Guide the user through document ingestion, embedding, and retrieval strategies.

4. Setting Up Development Environments

When the user needs to configure their workspace for LLM development:

1. Provide the setup script /home/ubuntu/skills/llm-architect/scripts/setup_env.sh to install necessary dependencies (PyTorch, Hugging Face, etc.).
2. Guide the user through configuring Git, GitHub, and Jupyter Notebooks for experimentation.

Portfolio Development

To help the user showcase their skills on GitHub:

1. Encourage creating a dedicated repository for each major project (From Scratch, Fine-Tuning, RAG).
2. Provide the template /home/ubuntu/skills/llm-architect/templates/README_template.md for documenting the projects effectively.
3. Suggest including rigorous evaluation metrics (offline and online benchmarks) in the documentation.

Bundled Resources

• Scripts (scripts/): Executable code for setting up architectures, pipelines, and environments.
• References (references/): Detailed guides on specific LLM topics (Architecture, Fine-Tuning, RAG).
• Templates (templates/): Configuration files and documentation templates for GitHub repositories.