How I use LLMs
Andrej Karpathy
Summary
To effectively leverage Large Language Models (LLMs) like ChatGPT, users must understand their core probabilistic nature, intelligently select models and features (such as reasoning, internet search, or code interpretation), and employ multimodal interactions and personalization for diverse tasks while always verifying outputs.
Key Takeaways
- Diverse LLM Ecosystem: Beyond OpenAI's ChatGPT, a rich ecosystem of LLM providers exists, including Google's Gemini, Meta's Llama, Microsoft's Co-pilot, Anthropic's Claude, and xAI's Grok, with performance tracked on leaderboards like Chatbot Arena. 1:02
- LLM Core Functionality: Think of an LLM as a "1TB zip file" (neural network parameters) containing a lossy, probabilistic compression of internet knowledge from pre-training, which determines its factual basis (with a knowledge cut-off), while post-training shapes its assistant persona. 8:16
- Context Window Management: Always start a new chat when switching topics to reset the token context window; this is crucial because an overloaded context can distract the model, decrease accuracy, and increase computation cost and response time. 16:26
- Model Tiers and Selection: Be mindful of the specific LLM model and pricing tier you are using (e.g., GPT-4o, Mini, Sonnet) as larger, paid models generally offer superior creativity, knowledge, and reduced hallucination, especially for professional use. 18:02
- Thinking Models for Complex Tasks: Leverage "thinking models" (trained with reinforcement learning) for difficult math, code, or reasoning problems, as they perform an internal monologue, exploring ideas and backtracking, which significantly boosts accuracy, though responses may take longer. 23:12
- Tool Use for Dynamic Information: Employ LLMs with integrated internet search tools (e.g., ChatGPT's web search, Perplexity) when querying recent, niche, or rapidly changing information, as the LLM's core knowledge has a cutoff date. 31:11
- Deep Research for Comprehensive Reports: Utilize "Deep Research" features (e.g., ChatGPT Pro, Perplexity's Deep Research) for custom, in-depth reports on complex topics, which involve multiple internet searches and extended reasoning over several minutes, though verification of sources remains essential. 42:08
- Code Interpreters and Data Analysis: Use LLMs with Python interpreters (e.g., ChatGPT's Advanced Data Analysis) for precise calculations, data plotting, and analysis; however, scrutinize the generated code and output for implicit assumptions or hallucinations. 59:02
- Artifacts and Code Development: Leverage tools like Claude's "Artifacts" for on-the-fly custom app generation (e.g., flashcard apps, diagrams) or dedicated coding assistants like Cursor for professional code development with full project context and autonomous agent capabilities. 1:09:14
- Multimodal Interaction (Audio): Utilize voice input (speech-to-text) for faster queries (50-80% of queries for the speaker) and explore "true audio" voice modes (e.g., ChatGPT's advanced voice, Grok's voice mode) for native voice understanding and generation, offering unique interactive experiences and even custom podcast generation (Google's NotebookLM). 1:22:37
- Multimodal Interaction (Images/Video): Employ image input for tasks like OCR from nutrition labels, analyzing blood test results, or explaining memes, and use video input (e.g., ChatGPT's mobile app) to point the camera at objects and ask real-time questions about the visual context. 1:41:44
- Quality of Life Features: Personalize your LLM experience with "Memory" features (allowing the model to remember your preferences across chats), "Custom Instructions" for global behavior modification, and "Custom GPTs" to save frequently used prompts and create specialized tools (e.g., detailed language translators, vocabulary extractors). 1:53:34
