Ai Engineer
Lộ trình phát triển toàn diện Ai Engineer theo tiêu chuẩn quốc tế nilbuild/developer-roadmap
Hướng dẫn từng bước từ nền tảng đến chuyên sâu giúp bạn làm chủ Ai Engineer. Tích hợp tài liệu lý thuyết, bài viết thực chiến, video tham khảo và bài tập lập trình trực tiếp trên IDE.
Nền Tảng & Khái Niệm Cốt Lõi
Giai đoạn 1 tập trung hoàn thiện 39 chủ đề then chốt.
Adding end-user IDs in prompts
Sending end-user IDs in your requests can be a useful tool to help OpenAI monitor and detect abuse. This allows OpenAI to provide your team with more actionable feedback in the event that we detect any policy violations in your application.
Agents Use Cases
AI Agents have a variety of use cases ranging from customer support, workflow automation, cybersecurity, finance, marketing, and sales, and more.
AI Agents
In AI engineering, "agents" refer to autonomous systems or components that can perceive their environment, make decisions, and take actions to achieve specific goals. Agents often interact with external systems, users, or other agents to carry out complex tasks. They can vary in complexity, from simple rule-based bots to sophisticated AI-powered agents that leverage machine learning models, natural language processing, and reinforcement learning.
AI Agents
In AI engineering, "agents" refer to autonomous systems or components that can perceive their environment, make decisions, and take actions to achieve specific goals. Agents often interact with external systems, users, or other agents to carry out complex tasks. They can vary in complexity, from simple rule-based bots to sophisticated AI-powered agents that leverage machine learning models, natural language processing, and reinforcement learning.
AI Engineer vs ML Engineer
An AI Engineer uses pre-trained models and existing AI tools to improve user experiences. They focus on applying AI in practical ways, without building models from scratch. This is different from AI Researchers and ML Engineers, who focus more on creating new models or developing AI theory.
AI Safety and Ethics
AI safety and ethics involve establishing guidelines and best practices to ensure that artificial intelligence systems are developed, deployed, and used in a manner that prioritizes human well-being, fairness, and transparency. This includes addressing risks such as bias, privacy violations, unintended consequences, and ensuring that AI operates reliably and predictably, even in complex environments. Ethical considerations focus on promoting accountability, avoiding discrimination, and aligning AI systems with human values and societal norms. Frameworks like explainability, human-in-the-loop design, and robust monitoring are often used to build systems that not only achieve technical objectives but also uphold ethical standards and mitigate potential harms.
AI vs AGI
AI (Artificial Intelligence) refers to systems designed to perform specific tasks by mimicking aspects of human intelligence, such as pattern recognition, decision-making, and language processing. These systems, known as "narrow AI," are highly specialized, excelling in specific areas such as image classification or recommender algorithms but lacking broader cognitive abilities. In contrast, AGI (Artificial General Intelligence) is a theoretical form of intelligence that can understand, learn, and apply knowledge across a wide range of tasks at a human-like level. AGI would have the capacity for abstract thinking, reasoning, and adaptability similar to human cognitive abilities, making it far more versatile than today’s AI systems. While current AI technology is powerful, AGI remains a distant goal and presents complex challenges in safety, ethics, and technical feasibility.
Anomaly Detection
Anomaly detection with embeddings works by transforming data, such as text, images, or time-series data, into vector representations that capture their patterns and relationships. In this high-dimensional space, similar data points are positioned close together, while anomalies stand out as those that deviate significantly from the typical distribution. This approach is highly effective for detecting outliers in tasks like fraud detection, network security, and quality control.
Anthropic's Claude
Anthropic's Claude is an AI language model designed to facilitate safe and scalable AI systems. Named after Claude Shannon, the father of information theory, Claude focuses on responsible AI use, emphasizing safety, alignment with human intentions, and minimizing harmful outputs.
Arize AI
Arize is an ML observability platform that supports both traditional ML models and LLM applications. For LLMs, it provides tracing, drift detection, and evaluation tooling. It is often used in enterprise settings where you need to monitor models already deployed in production and need deep integration with existing ML infrastructure.
Atlan
Atlan is a data catalog and governance platform that helps organizations discover, document, and manage their data assets across warehouses, pipelines, and BI tools. It provides a searchable inventory of datasets along with lineage, ownership, and quality information, so teams know where data comes from and whether it can be trusted. For AI systems, this kind of catalog can act as a source of vetted context, helping ensure that agents pull from data that is current and properly governed rather than from unreliable or duplicate sources. Atlan is offered as a managed, cloud based product rather than a self hosted tool.
Audio Processing
Audio processing in multimodal AI enables a wide range of use cases by combining sound with other data types, such as text, images, or video, to create more context-aware systems. Use cases include speech recognition paired with real-time transcription and visual analysis in meetings or video conferencing tools, voice-controlled virtual assistants that can interpret commands in conjunction with on-screen visuals, and multimedia content analysis where audio and visual elements are analyzed together for tasks like content moderation or video indexing.
Bias and Fairness
Bias and fairness in AI refer to the challenges of ensuring that machine learning models do not produce discriminatory or skewed outcomes. Bias can arise from imbalanced training data, flawed assumptions, or biased algorithms, leading to unfair treatment of certain groups based on race, gender, or other factors. Fairness aims to address these issues by developing techniques to detect, mitigate, and prevent biases in AI systems. Ensuring fairness involves improving data diversity, applying fairness constraints during model training, and continuously monitoring models in production to avoid unintended consequences, promoting ethical and equitable AI use.
Building an MCP Client
Building an MCP (Model Context Protocol) client involves creating software that can interact with AI models using a standardized protocol. This client acts as an intermediary, formatting requests for the model, sending them, and then processing the model's responses into a usable format for other applications or systems. Essentially, it's the piece of software that allows you to communicate with and leverage the capabilities of an AI model in a structured and consistent way.
Building an MCP Server
An MCP (Model Context Protocol) server acts as an intermediary between AI agents and various data sources or tools. It provides a standardized way for agents to access and interact with external information, enabling them to perform tasks that require context beyond their internal knowledge. Building an MCP server involves defining the API endpoints, handling requests from agents, retrieving data from relevant sources, and formatting the responses in a way that the agents can understand.
Choosing the Right Model
Selecting the appropriate large language model (LLM) involves considering factors like task complexity, budget, and desired performance. Simpler tasks may only require smaller, more efficient models, while more complex problems benefit from larger models with greater capacity. Cost is also a crucial factor, as larger models generally require more computational resources. You'll also need to assess the model's accuracy, speed, and ability to generalize to new, unseen data. Consider fine-tuning existing models on your specific dataset if you need specialized performance.
Chroma
Chroma is an open-source vector database and AI-native embedding database designed to handle and store large-scale embeddings and semantic vectors. It is used in applications that require fast, efficient similarity searches, such as natural language processing (NLP), machine learning (ML), and AI systems dealing with text, images, and other high-dimensional data.
Chunking
The chunking step in Retrieval-Augmented Generation (RAG) involves breaking down large documents or data sources into smaller, manageable chunks. This is done to ensure that the retriever can efficiently search through large volumes of data while staying within the token or input limits of the model. Each chunk, typically a paragraph or section, is converted into an embedding, and these embeddings are stored in a vector database. When a query is made, the retriever searches for the most relevant chunks rather than the entire document, enabling faster and more accurate retrieval.
Claude Agent SDK
The Claude Agent SDK provides tools and libraries to build autonomous AI agents powered by Anthropic's Claude models. It simplifies the development process by offering pre-built components and abstractions for tasks such as planning, tool usage, memory management, and human interaction. This allows developers to focus on defining the agent's behavior and capabilities rather than the underlying infrastructure.
Claude Code
Claude Code refers to the code-generation capabilities of Anthropic's Claude AI model. It's designed to assist developers by understanding natural language prompts and translating them into functional code across various programming languages. This allows developers to automate repetitive coding tasks, generate code snippets, and even create entire functions or modules based on descriptive instructions.
Claude Messages API
The Claude Messages API provides a structured way to interact with the Claude AI model. It allows developers to send a series of messages to Claude, mimicking a conversation. These messages can include text, images, and even structured data, enabling you to build complex interactions and extract specific insights from the model's responses through structured inputs and outputs.
Open vs Closed Source Models
Open-source models are freely available for customization and collaboration, promoting transparency and flexibility, while closed-source models are proprietary, offering ease of use but limiting modification and transparency.
Codex
Codex is an AI model created by OpenAI that translates natural language into code. It's designed to understand and generate code in a variety of programming languages, including Python, JavaScript, and more. Codex is particularly adept at interpreting comments and instructions to produce functional code snippets, making it a powerful tool for automating and accelerating the software development process.
Cohere
Cohere offers a suite of closed large language models (LLMs) designed for various natural language processing tasks. These models are accessible through an API and are trained and maintained by Cohere, meaning users don't have to worry about the complexities of training and fine-tuning their own models from scratch. Cohere focuses on providing enterprise-grade NLP solutions emphasizing safety, reliability, and ease of integration.
Cohere Embeddings
Cohere offers powerful text embeddings, which are numerical representations of text data. These embeddings capture the semantic meaning of words, sentences, and documents, allowing AI models to understand relationships and perform tasks like similarity search, clustering, and information retrieval. Cohere's embedding models are designed for high accuracy and performance across various natural language processing applications.
Conducting adversarial testing
Adversarial testing involves intentionally exposing machine learning models to deceptive, perturbed, or carefully crafted inputs to evaluate their robustness and identify vulnerabilities. The goal is to simulate potential attacks or edge cases where the model might fail, such as subtle manipulations in images, text, or data that cause the model to misclassify or produce incorrect outputs. This type of testing helps to improve model resilience, particularly in sensitive applications like cybersecurity, autonomous systems, and finance.
Connect to Local Server
A Local Desktop deployment means running the MCP server directly on your own computer instead of a remote cloud or server. You install the MCP software, needed runtimes, and model files onto your desktop or laptop. The server then listens on a local address like `127.0.0.1:8000`, accessible only from the same machine unless you open ports manually. This setup is great for fast tests, personal demos, or private experiments since you keep full control and avoid cloud costs. However, it's limited by your hardware's speed and memory, and others cannot access it without tunneling tools like ngrok or local port forwarding.
Connect to Remote Server
Remote or cloud deployment places the MCP server on a cloud provider instead of a local machine. You package the server as a container or virtual machine, choose a service like AWS, Azure, or GCP, and give it compute, storage, and a public HTTPS address. A load balancer spreads traffic, while auto-scaling adds or removes copies of the server as demand changes. You secure the endpoint with TLS, API keys, and firewalls, and you send logs and metrics to the provider’s monitoring tools. This setup lets the server handle many users, updates are easier, and you avoid local hardware limits, though you must watch costs and protect sensitive data.
Constraining outputs and inputs
Constraining outputs and inputs in AI models refers to implementing limits or rules that guide both the data the model processes (inputs) and the results it generates (outputs). Input constraints ensure that only valid, clean, and well-formed data enters the model, which helps to reduce errors and improve performance. This can include setting data type restrictions, value ranges, or specific formats. Output constraints, on the other hand, ensure that the model produces appropriate, safe, and relevant results, often by limiting output length, specifying answer formats, or applying filters to avoid harmful or biased responses. These constraints are crucial for improving model safety, alignment, and utility in practical applications.
Constraining Prompts
Constraining system prompts involves explicitly defining boundaries and limitations within the instructions given to a large language model (LLM). This technique focuses on guiding the LLM to operate within a specific scope, preventing it from generating irrelevant, harmful, or factually incorrect responses by setting rules and restrictions on its behavior and output format.
Constraining Prompts
Constraining system prompts involves explicitly defining boundaries and limitations within the instructions given to a large language model (LLM). This technique focuses on guiding the LLM to operate within a specific scope, preventing it from generating irrelevant, harmful, or factually incorrect responses by setting rules and restrictions on its behavior and output format.
Content Moderation APIs
Content Moderation APIs are tools that automatically analyze text, images, video, and audio to detect potentially harmful or inappropriate content. These APIs use machine learning models to identify violations of predefined policies related to areas like hate speech, violence, self-harm, and sexually suggestive material. The results allow developers to filter or take action against problematic user-generated content.
Context Compaction
Context compaction is a technique used to reduce the length of the context provided to a large language model (LLM) without sacrificing relevant information. This process aims to remove redundant, irrelevant, or less important information from the context window to make room for more data or improve the efficiency and effectiveness of the LLM's processing. Compaction can involve techniques like summarization, filtering, or re-ranking of context information.
Context Engineering
Context engineering is the practice of carefully designing and organizing the information you give to an AI model so it can do its job better. Think of it like preparing everything an AI needs before it starts working (including providing the right instructions, examples, background knowledge, and conversation history), all put together smartly so the model gives you the best possible answer. Instead of just asking a question and hoping for the best, you are building the perfect "environment" of information around the AI to guide it toward success.
Context Engineering
Context engineering is the process of strategically curating, formatting, and refining the information provided to a Large Language Model to guide its reasoning and improve the accuracy of its output. It involves techniques such as retrieval-augmented generation (RAG), prompt structuring, and selecting relevant metadata to ensure the model has the necessary background knowledge to handle specific tasks effectively.
Context Evaluation
Context evaluation measures whether the information delivered to a model actually helps it perform a task well. This includes checking whether retrieved documents are relevant, whether important details are missing, and whether irrelevant or outdated content is crowding out what matters. Teams use a mix of automated metrics, such as retrieval precision and recall, and manual review of model outputs to catch problems. Without this evaluation, a context pipeline can look complete while still giving the model poor material to work with.
Context Failure Modes
Context failure modes are the common ways a context pipeline can go wrong and degrade an agent's performance. These include context poisoning, where incorrect information gets included and treated as fact, context distraction, where too much irrelevant content pulls the model's attention away from what matters, and context rot, where accuracy drops as the amount of content grows even within the model's stated limits. Other failures include stale data that no longer reflects reality and conflicting information from different sources that the model cannot reconcile.
Context Isolation
Context isolation is about keeping different tasks or areas of knowledge separate when working with large language models (LLMs). Think of it like giving each task its own dedicated space. Instead of one big LLM trying to handle everything at once, you use multiple, smaller "agents" that are each focused on a specific job and trained on their own specific data. This prevents unrelated information from interfering with each other, leading to more accurate and reliable results.
Context Security
Context security covers the risks that come from feeding external or untrusted content into an AI system. A malicious document, email, or web page can contain hidden instructions designed to manipulate the model, a technique known as prompt injection. Poor access controls can also let a model surface data to a user who should not see it, or let sensitive information leak through tool calls and logs. Building context pipelines securely means validating sources, applying permission checks before data reaches the model, and treating retrieved content as data rather than as trusted instructions.
Kỹ Năng Trọng Tâm & Thực Hành
Giai đoạn 2 tập trung hoàn thiện 39 chủ đề then chốt.
Context Sources
Context sources are the places an AI system pulls information from to build what a model sees at inference time. Common sources include documents and knowledge bases, databases and data warehouses, code repositories, chat and support tools, and the outputs of other tool calls or agents. Each source has its own update frequency, access rules, and format, so combining them well requires normalizing and filtering before the data reaches the model. Choosing the right sources for a task matters as much as how much data is included.
Context vs Prompt Eng.
Prompt engineering focuses on how a single instruction is worded to get a better response from a model. Context engineering is broader: it deals with what information reaches the model at all, from which sources, in what order, and in what format, across an entire application or agent workflow. A well written prompt can still fail if the model lacks the right data, tools, or history to act on it. Context engineering treats the prompt as just one part of a larger system that also includes retrieval, memory, and tool outputs.
Context Window
The context window is the amount of text an LLM can process in a single request, measured in tokens. It includes the system prompt, conversation history, retrieved documents, and the model's own output as it generates a response. Once the total content exceeds this limit, older or lower priority information has to be dropped, summarized, or moved out of the active context. Model providers have expanded context windows significantly over the past few years, but a larger window does not guarantee the model uses all of it well.
Contextual prompting
Contextual prompting provides specific background information or situational details relevant to the current task, helping LLMs understand nuances and tailor responses accordingly. Unlike system or role prompts, contextual prompts supply immediate, task-specific information that's dynamic and changes based on the situation. For example: "Context: You are writing for a blog about retro 80's arcade video games. Suggest 3 topics to write articles about." This technique ensures responses are relevant, accurate, and appropriately framed for the specific context provided.
Context
Context, in the realm of Large Language Models (LLMs), refers to the information provided to the model alongside the prompt, which it uses to generate a relevant and coherent response. It encompasses the user's query, any supporting text, previous turns in a conversation, or any other data that helps the LLM understand the desired output. Essentially, it's the background knowledge and instructions that guide the LLM's reasoning and generation process.
Cost & Latency Monitoring
Cost and latency monitoring tracks token usage, the resulting financial cost, and response times across your AI system. Without this visibility, production costs can compound quickly and silently, especially when using large reasoning models that charge significantly per token. Tracking these alongside quality metrics lets you make informed tradeoffs, such as routing simpler queries to cheaper models or caching common responses to reduce redundant API calls.
Chain-of-Thought Prompting
Chain of Thought (CoT) is a way for an AI agent to think out loud. Before giving its final answer, the agent writes short notes that show each step it takes. These notes can list facts, name sub-tasks, or do small bits of math. By seeing the steps, the agent stays organized and is less likely to make a mistake. People who read the answer can also check the logic and spot any weak points. The same written steps can be fed back into the agent so it can plan, reflect, or fix itself. Because it is easy to use and boosts trust, CoT is one of the most common designs for language-based agents today.
Cursor
Cursor is an AI-powered code editor designed to enhance developer productivity. It leverages large language models to provide features such as code generation, intelligent autocompletion, and code refactoring suggestions, all within a familiar editor environment. Cursor aims to streamline the coding process and accelerate software development.
DALL-E API
The DALL-E API is a tool provided by OpenAI that allows developers to integrate the DALL-E image generation model into applications. DALL-E is an AI model designed to generate images from textual descriptions, capable of producing highly detailed and creative visuals. The API enables users to provide a descriptive prompt, and the model generates corresponding images, opening up possibilities in fields like design, advertising, content creation, and art.
Data Classification
Once data is embedded, a classification algorithm, such as a neural network or a logistic regression model, can be trained on these embeddings to classify the data into different categories. The advantage of using embeddings is that they capture underlying relationships and similarities between data points, even if the raw data is complex or high-dimensional, improving classification accuracy in tasks like text classification, image categorization, and recommendation systems.
Data Layer in Model Context Protocol
The Data Layer within the Model Context Protocol (MCP) is responsible for managing and providing access to the data that AI agents use to reason, learn, and make decisions. It acts as an intermediary between the agent and various data sources, ensuring data is readily available, properly formatted, and securely accessed. This layer handles data storage, retrieval, caching, and transformation, enabling agents to efficiently utilize relevant information from diverse sources.
DataHub
DataHub is an open source metadata platform originally built at LinkedIn for cataloging and managing data across an organization. It tracks where data lives, how it flows between systems, who owns it, and how it is used, giving teams a searchable map of their data assets. In the context of AI systems, this kind of metadata helps determine which datasets are trustworthy and relevant enough to feed into a model. Because it is open source, teams can self host and extend it rather than relying on a managed vendor.
DeepEval
DeepEval is an open-source LLM evaluation framework that lets you write automated tests for your AI application in a way that feels similar to unit testing with pytest. It comes with over 50 built-in metrics covering accuracy, faithfulness, relevance, toxicity, and agent behavior, and it works across RAG pipelines, chatbots, and multi-step agents. Because it integrates directly into CI/CD pipelines, teams can run evals automatically on every code change rather than treating evaluation as a manual, one-off task.
DeepSeek
DeepSeek refers to a family of large language models (LLMs) developed by DeepSeek AI. These models are designed to be powerful and efficient, capable of handling various natural language processing tasks such as text generation, translation, and question answering. They are made available with open weights, allowing developers and researchers to use, study, and modify the model architecture and parameters.
Deterministic Evals
Deterministic evals use fixed, rule-based checks to score model outputs. Examples include checking whether a response contains a required keyword, matches an expected format, or equals a known correct answer exactly. They are fast, cheap, and fully reproducible, making them a good first layer of evaluation for any system. Their limitation is that they cannot assess open-ended or subjective output quality.
Development Tools
AI has given rise to a collection of AI-powered development tools of various varieties. We have IDEs like Cursor that have AI baked into it, live context capturing tools such as Pieces, and a variety of browser-based tools like V0, Claude, and more.
Embedding Models
Embedding models transform data, like text or images, into numerical representations called embeddings. These embeddings capture the semantic meaning and relationships within the data in a vector space. By representing data as vectors, we can perform mathematical operations to determine similarity, cluster related items, and feed the data into machine learning models.
Embedding
In Retrieval-Augmented Generation (RAG), embeddings are essential for linking information retrieval with natural language generation. Embeddings represent both the user query and documents as dense vectors in a shared space, enabling the system to retrieve relevant information based on similarity. This retrieved information is then fed into a generative model, such as GPT, to produce contextually informed and accurate responses. By using embeddings, RAG enhances the model's ability to generate content grounded in external knowledge, making it effective for tasks like question answering and summarization.
Embeddings
Embeddings are dense, continuous vector representations of data, such as words, sentences, or images, in a lower-dimensional space. They capture the semantic relationships and patterns in the data, where similar items are placed closer together in the vector space. In machine learning, embeddings are used to convert complex data into a numerical form that models can process more easily. For example, word embeddings represent words based on their meanings and contexts, allowing models to understand relationships like synonyms or analogies. Embeddings are widely used in tasks like natural language processing, recommendation systems, and image recognition to improve model performance and efficiency.
Evaluation MetricsEvaluation Metrics
Evaluation metrics are the specific measurements used to score LLM outputs against defined quality criteria. Each metric targets a different dimension of quality, such as whether the response is factually grounded, relevant to the question, free of harmful content, or genuinely useful to the user. Choosing the right set of metrics for your use case is one of the most important decisions in building an eval system, since metrics determine what your system optimizes for and what failure modes it can actually detect.
External Memory
External memory, in the context of large language models (LLMs), refers to mechanisms that allow these models to access and utilize information stored outside of their internal parameters. This can involve retrieving relevant data from databases, knowledge graphs, or other external sources during the prompt processing or generation phases to augment the model's knowledge and improve its performance on specific tasks. This enhances the LLM's ability to handle complex queries and generate more accurate and contextually relevant responses.
FAISS
FAISS (Facebook AI Similarity Search) is a library developed by Facebook AI for efficient similarity search and clustering of dense vectors, particularly useful for large-scale datasets. It is optimized to handle embeddings (vector representations) and enables fast nearest neighbor search, allowing you to retrieve similar items from a large collection of vectors based on distance or similarity metrics like cosine similarity or Euclidean distance. FAISS is widely used in applications such as image and text retrieval, recommendation systems, and large-scale search systems where embeddings are used to represent items. It offers several indexing methods and can scale to billions of vectors, making it a powerful tool for handling real-time, large-scale similarity search problems efficiently.
Few-Shot Prompting
Few-shot prompting is a technique used with large language models (LLMs) where you provide a small number of example input-output pairs in the prompt itself. These examples demonstrate the desired behavior of the LLM for a specific task. By seeing these "few shots" of correct answers, the LLM can better understand the pattern and generate accurate and relevant responses for new, unseen inputs.
Fine-tuning
Fine-tuning involves taking a pre-trained large language model (LLM) and further training it on a smaller, task-specific dataset. This adapts the LLM to perform better on a particular task or domain. However, fine-tuning can be resource-intensive and may not always be the most efficient approach. Prompt engineering, retrieval-augmented generation (RAG), or using smaller, specialized models can sometimes achieve comparable or even better results with less computational overhead and data requirements.
Function Calling
Function calling allows Large Language Models (LLMs) to interact with external tools and APIs. Instead of just generating text, the LLM can be instructed to recognize when a specific function should be called based on the user's input, and then output the arguments needed to execute that function. This enables the LLM to perform actions beyond its training data, such as retrieving real-time information or automating tasks.
Gemini Embedding
The Gemini API offers methods to convert text, images, or other data types into numerical vector representations called embeddings. These embeddings capture the semantic meaning and relationships between different pieces of information, allowing for efficient similarity search, clustering, and other machine learning tasks. These proprietary models are offered as a service, often through an API, and require payment or subscription.
Gemini
Gemini is a multimodal AI model developed by Google. It's designed to understand and reason across different types of information, including text, code, audio, images, and video. This allows Gemini to solve complex problems and potentially generate new types of content, offering a more holistic approach compared to models focused on a single modality.
Gemma
Gemma is a family of open-source large language models (LLMs) developed by Google. These models are designed to be lightweight and high-performing, making them suitable for a variety of tasks, including text generation, question answering, and code completion. Gemma models are available in different sizes, allowing developers to select the best model for their specific resource constraints and performance requirements.
Generation
Generation refers to the process where a generative language model, such as GPT, creates a response based on the information retrieved during the retrieval phase. After relevant documents or data snippets are identified using embeddings, they are passed to the generative model, which uses this information to produce coherent, context-aware, and informative responses. The retrieved content helps the model stay grounded and factual, enhancing its ability to answer questions, provide summaries, or engage in dialogue by combining retrieved knowledge with its natural language generation capabilities. This synergy between retrieval and generation makes RAG systems effective for tasks that require detailed, accurate, and contextually relevant outputs.
Google ADK
The Google Agent Development Kit (ADK) is a framework designed to help developers build, test, and deploy AI agents. It provides tools and libraries that streamline the agent development process, offering features like agent orchestration, tool integration, and evaluation metrics. ADK aims to simplify the complexities of creating sophisticated AI agents capable of interacting with the real world.
Google Gemini API
The Google Gemini API provides developers with programmatic access to Google's Gemini family of multimodal AI models. It allows applications to understand and generate content across text, images, audio, and video. Developers can use this API to build features like conversational interfaces, content creation tools, and systems that can analyze and respond to complex multi-sensory data.
Google's Gemini
Google Gemini is an advanced AI model by Google DeepMind, designed to integrate natural language processing with multimodal capabilities, enabling it to understand and generate not just text but also images, videos, and other data types. It combines generative AI with reasoning skills, making it effective for complex tasks requiring logical analysis and contextual understanding.
Haystack
Langchain ========= Haystack is an open-source Python framework that helps you build search and question-answering agents fast. You connect your data sources, pick a language model, and set up pipelines that find the best answer to a user’s query. Haystack handles tasks such as indexing documents, retrieving passages, running the model, and ranking results. It works with many back-ends like Elasticsearch, OpenSearch, FAISS, and Pinecone, so you can scale from a laptop to a cluster. You can add features like summarization, translation, and document chat by dropping extra nodes into the pipeline. The framework also offers REST APIs, a web UI, and clear tutorials, making it easy to test and deploy your agent in production.
Helicone
Helicone is a logging and observability proxy for LLM APIs. Instead of calling OpenAI or Anthropic directly, you route requests through Helicone, which captures every request and response with zero code changes. It provides dashboards for cost tracking, latency, error rates, and user-level analytics.
How LLMs Work
Large Language Models (LLMs) are sophisticated AI systems trained on vast amounts of text data to understand, generate, and manipulate human language. They operate by learning statistical relationships between words and phrases, enabling them to predict the next word in a sequence or generate coherent text based on a given prompt. This is achieved through deep neural networks, primarily using a transformer architecture, which allows them to capture long-range dependencies in text and produce contextually relevant outputs.
Hugging Face Hub
The Hugging Face Hub is a central platform where users can discover, share, and collaborate on pre-trained models, datasets, and demos for machine learning. It hosts a vast repository of resources contributed by the community, allowing users to easily access and integrate these assets into their AI projects. The Hub provides tools for version control, model card documentation, and even web-based deployment, simplifying the process of leveraging existing AI models and contributing back to the open-source community.
Inference SDK
The Hugging Face Inference SDK is a powerful tool that allows developers to easily integrate and run inference on large language models hosted on the Hugging Face Hub. By using the `InferenceClient`, users can make API calls to various models for tasks such as text generation, image creation, and more. The SDK supports both synchronous and asynchronous operations, thus compatible with existing workflows.
Hugging Face Models
Hugging Face models are a collection of pre-trained machine learning models available through the Hugging Face platform, covering a wide range of tasks like natural language processing, computer vision, and audio processing. The platform includes models for tasks such as text classification, translation, summarization, question answering, and more, with popular models like BERT, GPT, T5, and CLIP. Hugging Face provides easy-to-use tools and APIs that allow developers to access, fine-tune, and deploy these models, fostering a collaborative community where users can share, modify, and contribute models to improve AI research and application development.
Hugging Face Tasks
Hugging Face supports text classification, named entity recognition, question answering, summarization, and translation. It also extends to multimodal tasks that involve both text and images, such as visual question answering (VQA) and image-text matching. Each task is done by various pre-trained models that can be easily accessed and fine-tuned through the Hugging Face library.
Kiến Trúc Nâng Cao & Tối Ưu
Giai đoạn 3 tập trung hoàn thiện 39 chủ đề then chốt.
Hugging Face
Hugging Face is a leading AI company and open-source platform that provides tools, models, and libraries for natural language processing (NLP), computer vision, and other machine learning tasks. It is best known for its "Transformers" library, which simplifies the use of pre-trained models like BERT, GPT, T5, and CLIP, making them accessible for tasks such as text classification, translation, summarization, and image recognition.
Human Evals
Human evals involve people directly reviewing and scoring model outputs against defined criteria. They are the most accurate form of evaluation for nuanced or subjective quality dimensions, and serve as the ground truth that other eval methods are validated against. Human evals are slower and more expensive than automated approaches, so they are typically used for high-stakes decisions, calibrating automated evals, or reviewing edge cases flagged by other methods.
Image Generation
Image generation is a process in artificial intelligence where models create new images based on input prompts or existing data. It involves using generative models like GANs (Generative Adversarial Networks), VAEs (Variational Autoencoders), or more recently, transformer-based models like DALL-E and Stable Diffusion.
Image Understanding
Multimodal AI enhances image understanding by integrating visual data with other types of information, such as text or audio. By combining these inputs, AI models can interpret images more comprehensively, recognizing objects, scenes, and actions, while also understanding context and related concepts. For example, an AI system could analyze an image and generate descriptive captions, or provide explanations based on both visual content and accompanying text.
Impact on Product Development
AI engineering transforms product development by automating tasks, enhancing data-driven decision-making, and enabling the creation of smarter, more personalized products. It speeds up design cycles, optimizes processes, and allows for predictive maintenance, quality control, and efficient resource management. By integrating AI, companies can innovate faster, reduce costs, and improve user experiences, giving them a competitive edge in the market.
Indexing Embeddings
Embeddings are stored in a vector database by first converting data, such as text, images, or audio, into high-dimensional vectors using machine learning models. These vectors, also called embeddings, capture the semantic relationships and patterns within the data. Once generated, each embedding is indexed in the vector database along with its associated metadata, such as the original data (e.g., text or image) or an identifier. The vector database then organizes these embeddings to support efficient similarity searches, typically using techniques like approximate nearest neighbor (ANN) search.
Inference
In artificial intelligence (AI), inference refers to the process by which a trained machine learning model makes predictions or draws conclusions from new, unseen data. Unlike training, inference involves the model applying what it has learned to make decisions without needing examples of the exact result. In essence, inference is the AI model actively functioning. For example, a self-driving car recognizing a stop sign on a road it has never encountered before demonstrates inference. The model identifies the stop sign in a new setting, using its learned knowledge to make a decision in real-time.
Input Format
Input format in prompt engineering deals with how you structure and present your prompt to a large language model (LLM). This includes the specific words used, the overall layout, and the inclusion of any examples or instructions. A well-defined input format guides the LLM towards generating the desired output, improving the relevance, accuracy, and consistency of its responses. Effective input formatting is key to unlocking the full potential of LLMs for specific tasks.
Introduction
AI Engineering is the process of designing and implementing AI systems using pre-trained models and existing AI tools to solve practical problems. AI Engineers focus on applying AI in real-world scenarios, improving user experiences, and automating tasks, without developing new models from scratch. They work to ensure AI systems are efficient, scalable, and can be seamlessly integrated into business applications, distinguishing their role from AI Researchers and ML Engineers, who concentrate more on creating new models or advancing AI theory.
Jina AI
Jina Embeddings are a family of high-performance, open-source, multilingual, and multimodal embedding models developed by Jina AI. They transform text and visual data (like images and charts) into dense numerical vectors for semantic search, RAG, and AI applications. Key features include supporting long contexts (up to 32k tokens), multi-task capabilities, and specialized versions for text-to-image and code retrieval.
Know your Customers / Usecases
To know your customer means deeply understanding the needs, behaviors, and expectations of your target users. This ensures the tools you create are tailored precisely for their intended purpose, while also being designed to prevent misuse or unintended applications. By clearly defining the tool’s functionality and boundaries, you can align its features with the users’ goals while incorporating safeguards that limit its use in contexts it wasn’t designed for. This approach enhances both the tool’s effectiveness and safety, reducing the risk of improper use.
LanceDB
LanceDB is a vector database designed for efficient storage, retrieval, and management of embeddings. It enables users to perform fast similarity searches, particularly useful in applications like recommendation systems, semantic search, and AI-driven content retrieval. LanceDB focuses on scalability and speed, allowing large-scale datasets of embeddings to be indexed and queried quickly, which is essential for real-time AI applications. It integrates well with machine learning workflows, making it easier to deploy models that rely on vector-based data processing, and helps manage the complexities of handling high-dimensional vector data efficiently.
LangChain for Multimodal Apps
LangChain is a framework designed to build applications that integrate multiple AI models, especially those focusing on language understanding, generation, and multimodal capabilities. For multimodal apps, LangChain facilitates seamless interaction between text, image, and even audio models, enabling developers to create complex workflows that can process and analyze different types of data.
Langchain
LangChain is a development framework that simplifies building applications powered by language models, enabling seamless integration of multiple AI models and data sources. It focuses on creating chains, or sequences, of operations where language models can interact with databases, APIs, and other models to perform complex tasks. LangChain offers tools for prompt management, data retrieval, and workflow orchestration, making it easier to develop robust, scalable applications like chatbots, automated data analysis, and multi-step reasoning systems.
LangFuse
LangFuse is an open-source LLM observability platform that provides tracing, prompt management, and evaluation tooling. You can self-host it or use the cloud version. It integrates with most popular frameworks and SDKs through a simple API, and lets you score traces manually or automatically to build up a quality signal over time.
LangSmith
LangSmith is an observability and evaluation platform built by the LangChain team, designed specifically for LLM applications. It automatically captures traces of LangChain runs and also works with non-LangChain code. It lets you inspect inputs and outputs at every step of a chain or agent, compare prompt versions, and run evals directly on logged traces.
LLMs
An LLM (Large Language Model) is an AI trained on vast text datasets to understand and generate human-like language via next-token prediction. While it natively handles tasks like translation, summarization, and writing, capabilities like web browsing or code execution come from software layers wrapped around the model. Though highly versatile, LLMs require massive computing power and can mirror biases present in their training data.
Llama Index
LlamaIndex, formerly known as GPT Index, is a tool designed to facilitate the integration of large language models (LLMs) with structured and unstructured data sources. It acts as a data framework that helps developers build retrieval-augmented generation (RAG) applications by indexing various types of data, such as documents, databases, and APIs, enabling LLMs to query and retrieve relevant information efficiently.
LlamaIndex for Multi-modal Apps
LlamaIndex enables multi-modal apps by linking language models (LLMs) to diverse data sources, including text and images. It indexes and retrieves information across formats, allowing LLMs to process and integrate data from multiple modalities. This supports applications like visual question answering, content summarization, and interactive systems by providing structured, context-aware inputs from various content types.
LLM Evaluations
LLM evaluations are structured tests that measure how well a model or AI system performs across a set of defined criteria. Rather than judging outputs informally, evals give you a repeatable, quantitative way to compare prompt versions, model updates, and system changes. They are the primary tool for making confident, evidence-based decisions about your AI system.
LLM Observability
LLM observability is the practice of monitoring and understanding what happens inside your AI application at runtime, tracking things like which prompts were sent, what responses came back, how long each call took, and how many tokens were used. Without visibility into these details, debugging failures or understanding why outputs changed is extremely difficult. Good observability gives you a continuous record of your system's behavior in development and production.
LM Studio
LM Studio is a desktop application designed for discovering, downloading, and running large language models (LLMs) locally on your computer. It provides a user-friendly interface to experiment with different open-source LLMs without needing extensive technical knowledge or cloud infrastructure. Essentially, it helps you manage and interact with LLMs directly on your machine.
Long-Context Processing
Long-context processing deals with feeding an AI model large amounts of text, such as entire codebases, long documents, or extended conversation histories, in a way that keeps the model accurate. Even when a model's context window is technically large enough, performance can degrade as the amount of content grows, a problem often called context rot. Techniques to manage this include chunking content, retrieving only the most relevant sections, and summarizing older material instead of passing it in full.
Manual Implementation of AI Agents
Manually implementing AI agents involves crafting their logic and behavior from the ground up, without relying heavily on pre-built frameworks or automated tools. This approach requires writing code to define how the agent perceives its environment, makes decisions, and takes actions, often involving explicit programming of algorithms for tasks like perception, planning, and learning. It offers maximum control and customization but demands a deep understanding of AI principles and software engineering.
MCP Client
The MCP Client is a software component that allows AI agents to interact with a Model Context Protocol (MCP) server. It handles the communication, serialization, and deserialization of data exchanged between the agent and the server, enabling the agent to access and manage contextual information relevant to its tasks. This client simplifies the process of integrating agents with the MCP ecosystem.
MCP Host
The MCP Host is a central component within the Model Context Protocol (MCP) framework, responsible for managing and coordinating interactions between AI agents and the environment. It acts as a bridge, providing a standardized interface for agents to access and utilize contextual information, tools, and resources. The host handles requests from agents, ensures proper authorization and security, and facilitates communication with external systems or data sources.
MCP Server
The MCP Server acts as a central hub for managing and serving contextual information to AI agents. It's responsible for receiving requests from agents, retrieving relevant context from various data sources, and delivering that context in a standardized format. This allows agents to make more informed decisions by leveraging external knowledge and data.
MCP
MCP, or Model Context Protocol, is an open standard that lets AI applications connect to external tools, data sources, and services in a consistent way. Instead of writing a custom integration for every tool a model might need, developers expose that tool through an MCP server, and any MCP compatible client can then call it. This reduces the number of one-off integrations needed as agents connect to more systems. Anthropic introduced MCP, and it has since been adopted by other model providers and tool builders.
Memory Systems
Memory systems let an AI agent retain information across interactions instead of starting fresh every time. Short-term memory usually covers the current session, while long-term memory stores facts, preferences, or past decisions that persist across sessions. Common implementations combine a vector store for semantic recall with structured storage for facts that need to be retrieved exactly. Designing memory well involves deciding what to keep, how long to keep it, and how to summarize or discard information that is no longer useful.
Meta Llama
Meta Llama is a family of large language models (LLMs) developed by Meta AI. These models are designed to be open-source and accessible, allowing researchers and developers to build and experiment with LLMs without the restrictions of proprietary models. Llama models have different versions and sizes, and they aim to provide a capable and efficient base for various natural language processing tasks.
Mistral AI
Mistral AI is a company focused on developing open-weight, large language models (LLMs) to provide high-performance AI solutions. Mistral aims to create models that are both efficient and versatile, making them suitable for a wide range of natural language processing tasks, including text generation, translation, and summarization. By releasing open-weight models, Mistral promotes transparency and accessibility, allowing developers to customize and deploy AI solutions more flexibly compared to proprietary models.
Model-Based Evals
Model-based evals use a separate AI model to automatically score or assess the outputs of your LLM application. Instead of writing manual rules or relying on human reviewers, you delegate the judgment to another model, a technique commonly known as LLM-as-a-Judge. You write a prompt describing the evaluation criteria, and the judge model rates the response. This approach handles subjective, open-ended quality dimensions that rules cannot capture, while scaling far more cheaply than human review, though it requires careful prompt design to avoid bias and inconsistency in the judges themselves.
Model Context Protocol (MCP)
Model Context Protocol (MCP) provides a standardized way for AI agents to manage and share contextual information. It defines a structure for representing the agent's current understanding of the environment, user, and goals, enabling more effective communication and collaboration between different components of an AI system or across multiple agents. This protocol facilitates the seamless transfer of relevant data, ensuring that each agent has the necessary information to make informed decisions and perform its tasks efficiently.
Models on Hugging Face
Embedding models are used to convert raw data like text, code, or images into high-dimensional vectors that capture semantic meaning. These vector representations allow AI systems to compare, cluster, and retrieve information based on similarity rather than exact matches. Hugging Face provides a wide range of pretrained embedding models, which are commonly used for tasks like semantic search, recommendation systems, duplicate detection, and retrieval-augmented generation (RAG). These models can be accessed through libraries like `transformers` or `sentence-transformers`, making it easy to generate high-quality embeddings for both general-purpose and task-specific applications.
modus
modus is a Context Warehouse, an infrastructure layer that sits between a company's internal data stack and its AI agents. Instead of asking teams to hand-model semantics, it mines context from how the organization already uses its data (query logs, dbt models, dashboards, pipelines, code, docs) and keeps that understanding current as the business changes. At query time it composes only the context relevant to the task, so agents work on signal instead of noise. It runs independently of any specific data warehouse, model, or app platform, and serves context to existing agents over MCP. The typical use is internal, giving any team in the company reliable AI answers about its own business data.
MongoDB Atlas
MongoDB Atlas, traditionally known for its document database capabilities, now includes vector search functionality, making it a strong option as a vector database. This feature allows developers to store and query high-dimensional vector data alongside regular document data. With Atlas’s vector search, users can perform similarity searches on embeddings of text, images, or other complex data, making it ideal for AI and machine learning applications like recommendation systems, image similarity search, and natural language processing tasks. The seamless integration of vector search within the MongoDB ecosystem allows developers to leverage familiar tools and interfaces while benefiting from advanced vector-based operations for sophisticated data analysis and retrieval.
Multi-agent Context Sharing
Multi-agent context sharing covers how separate AI agents working on related tasks exchange the information each one needs. When one agent hands off a task to another, it has to pass along enough history and data for the second agent to continue without repeating work or losing important details. Poorly designed sharing can lead to agents working with outdated or incomplete pictures of the task, while passing too much context can waste tokens and slow the system down. Common approaches include shared memory stores, structured message passing between agents, and a coordinating agent that filters what each sub-agent receives.
Multi-Agent Systems
Multi-agent systems involve the coordinated behavior of multiple intelligent agents within an environment. These agents interact with each other, potentially cooperating or competing, to achieve individual or collective goals. The complexity arises from the need for agents to reason about the actions, intentions, and knowledge of other agents to make informed decisions and effectively navigate the environment.
Multimodal AI Usecases
Multimodal AI powers applications like visual question answering, content moderation, and enhanced search engines. It drives smarter virtual assistants and interactive AR apps, combining text, images, and audio for richer, more intuitive user experiences across e-commerce, accessibility, and entertainment.
Hệ Sinh Thái & Triển Khai Thực Tế
Giai đoạn 4 tập trung hoàn thiện 39 chủ đề then chốt.
Multimodal AI
Multimodal AI is an approach that combines and processes data from multiple sources, such as text, images, audio, and video, to understand and generate responses. By integrating different data types, it enables more comprehensive and accurate AI systems, allowing for tasks like visual question answering, interactive virtual assistants, and enhanced content understanding. This capability helps create richer, more context-aware applications that can analyze and respond to complex, real-world scenarios.
NanoBanana API
The NanoBanana API is a tool designed to facilitate the integration and processing of data from multiple modalities, such as text, images, and audio. It provides a unified interface for managing diverse data types, enabling developers to build AI systems that can understand and reason about information from various sources simultaneously. This API typically includes functionalities for data ingestion, feature extraction, fusion techniques, and output generation tailored for multimodal applications.
Ollama
Ollama is a platform that offers large language models (LLMs) designed to run locally on personal devices, enabling AI functionality without relying on cloud services. It focuses on privacy, performance, and ease of use by allowing users to deploy models directly on laptops, desktops, or edge devices, providing fast, offline AI capabilities. With tools like the Ollama SDK, developers can integrate these models into their applications for tasks such as text generation, summarization, and more, benefiting from reduced latency, greater data control, and seamless local processing.
OpenAI Embeddings API
The OpenAI Embeddings API provides a straightforward way to transform text into numerical vector representations called embeddings. These embeddings capture the semantic meaning of text, allowing you to perform tasks like semantic search, clustering, and similarity comparison by analyzing the relationships between these vectors. The API simplifies the process of generating these embeddings, abstracting away the complexities of training and managing embedding models.
OpenAI AgentKit
OpenAI AgentKit is a framework designed to simplify the process of creating AI agents that can perform specific tasks by leveraging large language models (LLMs). It provides tools and abstractions for managing agent state, defining agent behavior, and connecting agents to various tools and resources, allowing developers to build more complex and capable AI systems without needing to manage all the low-level details.
OpenAI-compatible APIs
OpenAI-compatible APIs are interfaces that mimic the structure and functionality of OpenAI's API. These APIs allow developers to switch between different language models or providers with minimal code changes. This is particularly useful for testing different models, mitigating vendor lock-in, or building fallback mechanisms in case of service disruptions.
OpenAI Models (GPT, o-series)
OpenAI models, including the GPT and "o-series" (like Whisper), are pre-trained AI models offered by OpenAI, accessible via their API. GPT models excel at natural language tasks like text generation, translation, and question answering. The "o-series" refers to OpenAI's other specialized models; for example, Whisper is a speech-to-text transcription model.
OpenAI Response API
The OpenAI Response API allows developers to interact with various AI models offered by OpenAI. It provides a structured way to send requests to these models and receive generated text, code, or other outputs. You can control aspects like the prompt, temperature, maximum length, and other parameters to tailor the response to your specific needs. It is intended to eventually replace the Assistants API and the Chat Completions API for new projects.
OpenAI Vision API
The OpenAI Vision API enables models to analyze and understand images, allowing them to identify objects, recognize text, and interpret visual content. It integrates image processing with natural language capabilities, enabling tasks like visual question answering, image captioning, and extracting information from photos. This API can be used for applications in accessibility, content moderation, and automation, providing a seamless way to combine visual understanding with text-based interactions.
OpenRouter
OpenRouter acts as a unified API, providing access to various large language models (LLMs) from different providers like OpenAI, Cohere, and Anthropic through a single platform. This allows developers to easily switch between models, compare their performance, and optimize their applications without needing to manage multiple API keys or integration processes. It essentially streamlines the process of experimenting with and deploying different AI models.
Performing Similarity Search
In a similarity search, the process begins by converting the user’s query (such as a piece of text or an image) into an embedding—a vector representation that captures the query’s semantic meaning. This embedding is generated using a pre-trained model, such as BERT for text or a neural network for images. Once the query is converted into a vector, it is compared to the embeddings stored in the vector database.
Pinecone
Pinecone is a managed vector database designed for efficient similarity search and real-time retrieval of high-dimensional data, such as embeddings. It allows developers to store, index, and query vector representations, making it easy to build applications like recommendation systems, semantic search, and AI-driven content discovery. Pinecone is scalable, handles large datasets, and provides fast, low-latency searches using optimized indexing techniques.
PostHog
PostHog is a product analytics platform that also offers what it calls a context warehouse, a combined store of product event data, session replays, and business context from tools like Slack and support tickets. Rather than requiring a separate ETL pipeline to move data between systems, PostHog exposes this combined data directly to AI agents through its MCP server, letting an agent query product behavior and business context using the same interface a person would use.
Pre-trained Models
Pre-trained models are Machine Learning (ML) models that have been previously trained on a large dataset to solve a specific task or set of tasks. These models learn patterns, features, and representations from the training data, which can then be fine-tuned or adapted for other related tasks. Pre-training provides a good starting point, reducing the amount of data and computation required to train a new model from scratch.
Production Monitoring
Production monitoring is the continuous observation of your AI system once it is live and handling real user traffic. Unlike testing in a controlled environment, production surfaces edge cases, unexpected inputs, and failure modes that never appeared during development. Monitoring in production means tracking quality metrics, error rates, and behavioral changes over time so that regressions and anomalies are caught before they affect a large number of users.
Prompt Caching
Prompt caching is a technique that stores the results of previous LLM prompts, allowing you to quickly retrieve and reuse them instead of re-running the prompt every time. This can significantly improve efficiency and reduce costs when dealing with frequently used or computationally expensive prompts.
Prompt vs. Context Engineering
Prompt engineering focuses on crafting effective instructions (prompts) to elicit desired outputs from Large Language Models (LLMs). Context engineering, on the other hand, involves structuring and providing relevant background information (the context) to the LLM, enabling it to generate more accurate and informed responses based on a richer understanding of the subject matter.
Prompt Engineering
Prompt engineering is the art and science of crafting effective prompts, which are the instructions or inputs given to a large language model (LLM). The goal is to design prompts that elicit the desired response from the LLM, guiding it to generate accurate, relevant, and creative outputs. This involves understanding the LLM's capabilities and limitations, and experimenting with different prompt structures, keywords, and contextual cues to achieve optimal results.
Prompt Injection Attacks
Prompt injection attacks are a type of security vulnerability where malicious inputs are crafted to manipulate or exploit AI models, like language models, to produce unintended or harmful outputs. These attacks involve injecting deceptive or adversarial content into the prompt to bypass filters, extract confidential information, or make the model respond in ways it shouldn't. For instance, a prompt injection could trick a model into revealing sensitive data or generating inappropriate responses by altering its expected behavior.
Prompt vs. Context Engineering
Prompt engineering focuses on crafting effective instructions (prompts) to elicit desired outputs from Large Language Models (LLMs). Context engineering, on the other hand, involves structuring and providing relevant background information (the context) to the LLM, enabling it to generate more accurate and informed responses based on a richer understanding of the subject matter.
Purpose and Functionality
A vector database is designed to store, manage, and retrieve high-dimensional vectors (embeddings) generated by AI models. Its primary purpose is to perform fast and efficient similarity searches, enabling applications to find data points that are semantically or visually similar to a given query. Unlike traditional databases, which handle structured data, vector databases excel at managing unstructured data like text, images, and audio by converting them into dense vector representations. They use indexing techniques, such as approximate nearest neighbor (ANN) algorithms, to quickly search large datasets and return relevant results. Vector databases are essential for applications like recommendation systems, semantic search, and content discovery, where understanding and retrieving similar items is crucial.
Qdrant
Qdrant is an open-source vector database designed for efficient similarity search and real-time data retrieval. It specializes in storing and indexing high-dimensional vectors (embeddings) to enable fast and accurate searches across large datasets. Qdrant is particularly suited for applications like recommendation systems, semantic search, and AI-driven content discovery, where finding similar items quickly is essential. It supports advanced filtering, scalable indexing, and real-time updates, making it easy to integrate into machine learning workflows.
Qwen
Qwen is a series of large language models (LLMs) developed by Alibaba Group. These models are open-source, meaning their code and architecture are publicly available, allowing researchers and developers to use, study, and modify them for their own purposes. Qwen models are designed to perform various natural language processing tasks, such as text generation, translation, and question answering, and come in different sizes, allowing for flexibility based on computational resources and specific application requirements.
RAG and Dynamic Filters
Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by providing them with relevant, up-to-date information from external sources. Dynamic filters are techniques that selectively filter the information retrieved for RAG, ensuring that the LLM receives only the most pertinent context based on the specific query and user. This results in more accurate, focused, and contextually appropriate LLM responses.
RAG and Dynamic Filters
Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by providing them with relevant, up-to-date information from external sources. Dynamic filters are techniques that selectively filter the information retrieved for RAG, ensuring that the LLM receives only the most pertinent context based on the specific query and user. This results in more accurate, focused, and contextually appropriate LLM responses.
RAG Usecases
Retrieval-Augmented Generation (RAG) enhances applications like chatbots, customer support, and content summarization by combining information retrieval with language generation. It retrieves relevant data from a knowledge base and uses it to generate accurate, context-aware responses, making it ideal for tasks such as question answering, document generation, and semantic search. RAG’s ability to ground outputs in real-world information leads to more reliable and informative results, improving user experience across various domains.
RAG vs Fine-tuning
RAG (Retrieval-Augmented Generation) and fine-tuning are two approaches to enhancing language models, but they differ in methodology and use cases. Fine-tuning involves training a pre-trained model on a specific dataset to adapt it to a particular task, making it more accurate for that context but limited to the knowledge present in the training data. RAG, on the other hand, combines real-time information retrieval with generation, enabling the model to access up-to-date external data and produce contextually relevant responses. While fine-tuning is ideal for specialized, static tasks, RAG is better suited for dynamic tasks that require real-time, fact-based responses.
RAG
Retrieval-Augmented Generation (RAG) is an AI approach that combines information retrieval with language generation to create more accurate, contextually relevant outputs. It works by first retrieving relevant data from a knowledge base or external source, then using a language model to generate a response based on that information. This method enhances the accuracy of generative models by grounding their outputs in real-world data, making RAG ideal for tasks like question answering, summarization, and chatbots that require reliable, up-to-date information.
RAGAS
RAGAS is an open-source framework specifically designed for evaluating retrieval-augmented generation pipelines. It provides a suite of reference-free metrics, meaning it does not require hand-labeled ground truth answers, which makes it practical to run on real production data. Its core metrics cover the main ways a RAG system can fail: retrieving the wrong context, generating answers that are not supported by the retrieved context, and producing responses that do not actually address the user's question.
RAGFlow
RAGFlow is a framework designed to streamline the creation, evaluation, and deployment of Retrieval-Augmented Generation (RAG) pipelines. It provides tools and abstractions for building modular RAG systems, allowing developers to easily experiment with different components like data loaders, retrievers, and generators, and then assess their performance.
RAG
Retrieval-Augmented Generation (RAG) is an AI approach that combines information retrieval with language generation to create more accurate, contextually relevant outputs. It works by first retrieving relevant data from a knowledge base or external source, then using a language model to generate a response based on that information. This method enhances the accuracy of generative models by grounding their outputs in real-world data, making RAG ideal for tasks like question answering, summarization, and chatbots that require reliable, up-to-date information.
ReAct Prompting
ReAct prompting is a technique that combines reasoning and action by guiding language models to think through a problem step-by-step and then take specific actions based on the reasoning. It encourages the model to break down tasks into logical steps (reasoning) and perform operations, such as calling APIs or retrieving information (actions), to reach a solution. This approach helps in scenarios where the model needs to process complex queries, interact with external systems, or handle tasks requiring a sequence of actions, improving the model's ability to provide accurate and context-aware responses.
ReAct
ReAct (Reason and Act) prompting enables LLMs to solve complex tasks by combining reasoning with external tool interactions. It follows a thought-action-observation loop: analyze the problem, perform actions using external APIs, review results, and iterate until solved. Useful for research, multi-step problems, and tasks requiring current data.
Recommendation Systems
In the context of embeddings, recommendation systems use vector representations to capture similarities between items, such as products or content. By converting items and user preferences into embeddings, these systems can measure how closely related different items are based on vector proximity, allowing them to recommend similar products or content based on a user's past interactions. This approach improves recommendation accuracy and efficiency by enabling meaningful, scalable comparisons of complex data.
Regression Testing
Regression testing for AI systems means verifying that a change to a prompt, model version, retrieval strategy, or any other component does not degrade performance on cases that previously worked correctly. Because LLM behavior can shift in subtle ways with small changes, having a regression suite lets you catch quality regressions before they reach users.
Repetition Penalties
Repetition penalties discourage LLMs from repeating words or phrases by reducing the probability of selecting previously used tokens. This includes frequency penalty (scales with usage count) and presence penalty (applies equally to any used token). These parameters improve output quality by promoting vocabulary diversity and preventing redundant phrasing.
Replit
Replit is an online integrated development environment (IDE) that allows users to write and run code in various programming languages directly in their web browser. It provides a collaborative coding environment with features like real-time collaboration, version control, and package management, making it easy to build and deploy projects without needing to install software locally. Replit also incorporates AI features like code completion and generation to help streamline the coding process.
Retrieval Process
The retrieval process in Retrieval-Augmented Generation (RAG) involves finding relevant information from a large dataset or knowledge base to support the generation of accurate, context-aware responses. When a query is received, the system first converts it into a vector (embedding) and uses this vector to search a database of pre-indexed embeddings, identifying the most similar or relevant data points. Techniques like approximate nearest neighbor (ANN) search are often used to speed up this process.
Robust prompt engineering
Robust prompt engineering involves carefully crafting inputs to guide AI models toward producing accurate, relevant, and reliable outputs. It focuses on minimizing ambiguity and maximizing clarity by providing specific instructions, examples, or structured formats. Effective prompts anticipate potential issues, such as misinterpretation or inappropriate responses, and address them through testing and refinement. This approach enhances the consistency and quality of the model's behavior, making it especially useful for complex tasks like multi-step reasoning, content generation, and interactive systems.
Chuyên Gia & Mở Rộng Hệ Thống
Giai đoạn 5 tập trung hoàn thiện 38 chủ đề then chốt.
Role & Behavior
System prompting involves crafting instructions that define the AI model's role, personality, and overall behavior when interacting with users. This allows you to shape the AI's responses, ensuring they are consistent with desired guidelines, such as adopting a specific persona (e.g., a helpful assistant, an expert) or adhering to constraints on tone and style. By carefully defining these aspects, you can significantly influence the AI's output and guide it towards more relevant and effective interactions.
Roles and Responsibilities
AI Engineers are responsible for designing, developing, and deploying AI systems that solve real-world problems. Their roles include building machine learning models, implementing data processing pipelines, and integrating AI solutions into existing software or platforms. They work on tasks like data collection, cleaning, and labeling, as well as model training, testing, and optimization to ensure high performance and accuracy. AI Engineers also focus on scaling models for production use, monitoring their performance, and troubleshooting issues. Additionally, they collaborate with data scientists, software developers, and other stakeholders to align AI projects with business goals, ensuring that solutions are reliable, efficient, and ethically sound.
Sampling Parameters
Sampling parameters are settings that control the randomness and creativity of the text generated by the LLM. They influence how the model chooses the next word in a sequence, impacting the overall coherence, diversity, and relevance of the output. Adjusting these parameters allows users to fine-tune the LLM's responses for specific applications, balancing between predictable, safe outputs and more imaginative, exploratory ones.
Security and Privacy Concerns
Security and privacy concerns in AI revolve around the protection of data and the responsible use of models. Key issues include ensuring that sensitive data, such as personal information, is handled securely during collection, processing, and storage, to prevent unauthorized access and breaches. AI models can also inadvertently expose sensitive data if not properly designed, leading to privacy risks through data leakage or misuse. Additionally, there are concerns about model bias, data misuse, and ensuring transparency in how AI decisions are made.
Self-Hosted AI Models
Self-hosted AI models are machine learning models that are deployed and run on your own infrastructure, rather than relying on a third-party service or cloud provider. This means you have complete control over the hardware, software, and data used to run the model, allowing for greater customization, security, and potentially lower long-term costs, depending on your specific needs and scale.
Semantic Search
Embeddings are used for semantic search by converting text, such as queries and documents, into high-dimensional vectors that capture the underlying meaning and context, rather than just exact words. These embeddings represent the semantic relationships between words or phrases, allowing the system to understand the query’s intent and retrieve relevant information, even if the exact terms don’t match.
Sentence Transformers
Sentence Transformers are a type of model designed to generate high-quality embeddings for sentences, allowing them to capture the semantic meaning of text. Unlike traditional word embeddings, which represent individual words, Sentence Transformers understand the context of entire sentences, making them ideal for tasks that require semantic similarity, such as sentence clustering, semantic search, and paraphrase detection. Built on top of transformer models like BERT and RoBERTa, they convert sentences into dense vectors, where similar sentences are placed closer together in vector space.
Speech-to-Text
In the context of multimodal AI, speech-to-text technology converts spoken language into written text, enabling seamless integration with other data types like images and text. This allows AI systems to process audio input and combine it with visual or textual information, enhancing applications such as virtual assistants, interactive chatbots, and multimedia content analysis. For example, a multimodal AI can transcribe a video’s audio while simultaneously analyzing on-screen visuals and text, providing richer and more context-aware insights.
State & Historical Context
State and historical context refers to tracking what has already happened in a conversation or workflow so an agent can act consistently over time. This includes the sequence of user messages, past tool calls and their results, and any decisions the agent has already made. Without this tracking, an agent can repeat steps, contradict earlier answers, or lose track of a multi-step task. Systems typically store this as a structured log or state object that gets updated after each step and referenced before the next one.
Streaming Responses
Streamed responses is one of the techniques an AI agent can use to send its answer to the user. With a streamed response, the agent starts sending words as soon as it generates them. The user sees the text grow on the screen in real time. This feels fast and lets the user stop or change the request early. It is useful for long answers and chat-like apps. By contrast, an unstreamed response waits until the whole answer is ready, then sends it all at once. This makes the code on the client side simpler and is easier to cache or log, but the user must wait longer, especially for big outputs.
Structured Outputs
Structured output in system prompting refers to designing prompts that guide a Large Language Model (LLM) to generate responses in a predefined format, such as JSON, XML, or a specific text-based structure. This approach focuses on crafting system prompts to elicit predictable and parseable outputs, making it easier to integrate LLM responses into downstream applications and workflows. By providing clear instructions and examples of the desired output structure, we can reliably extract information and automate processes.
Supabase
Supabase Vector is an extension of the Supabase platform, specifically designed for AI and machine learning applications that require vector operations. It leverages PostgreSQL's pgvector extension to provide efficient vector storage and similarity search capabilities. This makes Supabase Vector particularly useful for applications involving embeddings, semantic search, and recommendation systems. With Supabase Vector, developers can store and query high-dimensional vector data alongside regular relational data, all within the same PostgreSQL database.
System Prompting
System prompting sets the overall context, purpose, and operational guidelines for LLMs. It defines the model's role, behavioral constraints, output format requirements, and safety guardrails. System prompts provide foundational parameters that influence all subsequent interactions, ensuring consistent, controlled, and structured AI responses throughout the session.
Temperature
Temperature is a parameter used in language models that controls the randomness of the generated text. A higher temperature value (e.g., 1.0) leads to more diverse and unpredictable outputs, as the model is more likely to sample less probable words. Conversely, a lower temperature value (e.g., 0.2) yields more deterministic, conservative outputs, favoring the most likely words according to the model's training data. Essentially, it influences the probability distribution from which the next word is selected.
Text-to-Speech
In the context of multimodal AI, text-to-speech (TTS) technology converts written text into natural-sounding spoken language, allowing AI systems to communicate verbally. When integrated with other modalities, such as visual or interactive elements, TTS can enhance user experiences in applications like virtual assistants, educational tools, and accessibility features. For example, a multimodal AI could read aloud text from an on-screen document while highlighting relevant sections, or narrate information about objects recognized in an image. By combining TTS with other forms of data processing, multimodal AI creates more engaging, accessible, and interactive systems for users.
Tokens in Large Language Models
Tokens are fundamental units of text that LLMs process, created by breaking text into smaller components such as words, subwords, or characters. Understanding tokens is crucial because models predict the next token in sequences, API costs are based on token count, and models have maximum token limits for input and output.
Tools & Function Calling in AI Agents
Tools and function calling equip AI agents with the ability to interact with the real world and access external information. Instead of just providing text-based responses, the agent can use predefined functions, tools, or APIs to perform specific actions like searching the web, sending emails, or controlling devices. This extends the agent's capabilities and allows it to solve more complex tasks that require actions beyond simple language generation.
Top-K Sampling
Top-K sampling is a method used by Large Language Models (LLMs) during text generation to select the next word. Instead of considering the entire vocabulary, it narrows down the choices to the K most probable words predicted by the model. Low values (1-10) produce conservative, factual outputs. Medium values (20-50) balance creativity and quality. High values (50+) enable diverse, creative outputs. Use low K for technical tasks, high K for creative writing.
Top-P Sampling
Top-P sampling, also known as nucleus sampling, is a technique used in language models to generate text. Instead of considering all possible next words, it focuses on the smallest set of words whose cumulative probability exceeds a threshold 'P'. Unlike Top-K's fixed number, Top-P dynamically adjusts based on the probability distribution. Low values (0.1-0.5) produce focused outputs, medium (0.6-0.9) balance creativity and coherence, and high (0.9-0.99) enable creative diversity.
Tracing & Logging
Tracing records the full lifecycle of a request through your AI system, from the initial user input through any intermediate LLM calls, tool uses, or retrieval steps, all the way to the final response. Logging captures individual events like errors, latency spikes, or unexpected outputs. Together, they let you reconstruct exactly what happened during any given interaction, which is essential for debugging agents and multi-step pipelines.
Training
Training refers to the process of teaching a machine learning model to recognize patterns and make predictions by exposing it to a dataset. During training, the model learns from the data by adjusting its internal parameters to minimize errors between its predictions and the actual outcomes. This process involves iteratively feeding the model with input data, comparing its outputs to the correct answers, and refining its predictions through techniques like gradient descent. The goal is to enable the model to generalize well so that it can make accurate predictions on new, unseen data.
Transformers.js
Transformers.js is a JavaScript library that enables transformer models, like those from Hugging Face, to run directly in the browser or Node.js, without needing cloud services. It supports tasks such as text generation, sentiment analysis, and translation within web apps or server-side scripts. Using WebAssembly (Wasm) and efficient JavaScript, Transformers.js offers powerful NLP capabilities with low latency, enhanced privacy, and offline functionality, making it ideal for real-time, interactive applications where local processing is essential for performance and security.
Transport Layer in Model Context Protocol (MCP)
The Transport Layer in the Model Context Protocol (MCP) is responsible for reliably and efficiently moving data between different components of an AI agent system. It defines how messages are packaged, addressed, and transmitted across a network or within a single machine, ensuring that information reaches its intended destination without errors or loss. This layer handles the underlying communication mechanisms, abstracting away the complexities of network protocols and hardware.
Types of AI Models
AI models come in various forms. Open models provide transparent access to their architecture and training data, fostering collaboration and customization, while closed models keep these details proprietary. Pre-trained models are trained on massive datasets and can be fine-tuned for specific tasks, saving time and resources. Self-hosted models, on the other hand, offer greater control and privacy as they are deployed and managed on your own infrastructure.
Using SDKs Directly
While tools like Langchain and LlamaIndex make it easy to implement RAG, you don't have to necessarily learn and use them. If you know about the different steps of implementing RAG, you can simply do it all yourself e.g., do the chunking using `@langchain/textsplitters` package, create embeddings using any LLM e.g., use OpenAI Embedding API through their SDK, save the embeddings to any vector database e.g. if you are using Supabase Vector DB, you can use their SDK, and similarly, you can use the relevant SDKs for the rest of the steps as well.
Vector Database
When implementing Retrieval-Augmented Generation (RAG), a vector database is used to store and efficiently retrieve embeddings, which are vector representations of data like documents, images, or other knowledge sources. During the RAG process, when a query is made, the system converts it into an embedding and searches the vector database for the most relevant, similar embeddings (e.g., related documents or snippets). These retrieved pieces of information are then fed to a generative model, which uses them to produce a more accurate, context-aware response.
Vector Databases
Vector databases are systems specialized in storing, indexing, and retrieving high-dimensional vectors, often used as embeddings for data like text, images, or audio. Unlike traditional databases, they excel at managing unstructured data by enabling fast similarity searches, where vectors are compared to find the closest matches. This makes them essential for tasks like semantic search, recommendation systems, and content discovery. Using techniques like approximate nearest neighbor (ANN) search, vector databases handle large datasets efficiently, ensuring quick and accurate retrieval even at scale.
Vector Databases
Vector databases are specialized systems designed to store, index, and retrieve high-dimensional vectors, often used as embeddings that represent data like text, images, or audio. Unlike traditional databases that handle structured data, vector databases excel at managing unstructured data by enabling fast similarity searches, where vectors are compared to find those that are most similar to a query. This makes them essential for tasks like semantic search, recommendation systems, and content discovery, where understanding relationships between items is crucial. Vector databases use indexing techniques such as approximate nearest neighbor (ANN) search to efficiently handle large datasets, ensuring quick and accurate retrieval even at scale.
Vertex AI
Vertex AI is Google Cloud's fully-managed, unified development platform for building, training, deploying, and managing machine learning (ML) models at scale. It provides tools for the entire ML lifecycle, from data preparation and custom training with AutoML to model monitoring and deployment. Vertex AI offers access to Google's foundation models, such as Gemini, along with custom training options and tools for building AI agents. It streamlines the ML workflow into a single interface, supporting both low-code and custom development on managed infrastructure
Video Understanding
Video understanding with multimodal AI involves analyzing and interpreting both visual and audio content to provide a more comprehensive understanding of videos. Common use cases include video summarization, where AI extracts key scenes and generates summaries; content moderation, where the system detects inappropriate visuals or audio; and video indexing for easier search and retrieval of specific moments within a video. Other applications include enhancing video-based recommendations, security surveillance, and interactive entertainment, where video and audio are processed together for real-time user interaction.
Weaviate
Weaviate is an open-source vector database that allows users to store, search, and manage high-dimensional vectors, often used for tasks like semantic search and recommendation systems. It enables efficient similarity searches by converting data (like text, images, or audio) into embeddings and indexing them for fast retrieval. Weaviate also supports integrating external data sources and schemas, making it easy to combine structured and unstructured data.
What are Embeddings
Embeddings are dense, numerical vector representations of data, such as words, sentences, images, or audio, that capture their semantic meaning and relationships. By converting data into fixed-length vectors, embeddings allow machine learning models to process and understand the data more effectively. For example, word embeddings represent similar words with similar vectors, enabling tasks like semantic search, recommendation systems, and clustering. Embeddings make it easier to compare, search, and analyze complex, unstructured data by mapping similar items close together in a high-dimensional space.
RAG & Implementation
Retrieval-Augmented Generation (RAG) combines information retrieval with language generation to produce more accurate, context-aware responses. It uses two components: a retriever, which searches a database to find relevant information, and a generator, which crafts a response based on the retrieved data. Implementing RAG involves using a retrieval model (e.g., embeddings and vector search) alongside a generative language model (like GPT). The process starts by converting a query into embeddings, retrieving relevant documents from a vector database, and feeding them to the language model, which then generates a coherent, informed response. This approach grounds outputs in real-world data, resulting in more reliable and detailed answers.
What is a Context Layer?
A context layer is the part of an AI system responsible for gathering, organizing, and delivering the right information to a model before it generates a response. It sits between raw data sources, such as databases, documents, and APIs, and the model itself, deciding what gets included and what gets left out. Some teams build this layer themselves using retrieval pipelines and memory stores, while others rely on dedicated context platforms. The goal is to give the model relevant, trustworthy information without overwhelming it with everything available.
What is an AI Engineer?
AI engineers are professionals who specialize in designing, developing, and implementing artificial intelligence (AI) systems. Their work is essential in various industries, as they create applications that enable machines to perform tasks that typically require human intelligence, such as problem-solving, learning, and decision-making.
Whisper API
The Whisper API by OpenAI enables developers to integrate speech-to-text capabilities into their applications. It uses OpenAI's Whisper model, a powerful speech recognition system, to convert spoken language into accurate, readable text. The API supports multiple languages and can handle various accents, making it ideal for tasks like transcription, voice commands, and automated captions. With the ability to process audio in real time or from pre-recorded files, the Whisper API simplifies adding robust speech recognition features to applications, enhancing accessibility and enabling new interactive experiences.
Devin
Devin is an autonomous AI software engineer capable of handling entire development projects from start to finish. It functions by planning, writing, debugging, and deploying code through a simulated environment that mirrors a human developer’s workflow. Originally developed by Cognition, this tool—formerly known as Windsurf—integrates directly into the development process to execute complex tasks, manage terminal commands, and navigate browser-based documentation independently.
Zero Shot Prompting
Zero-shot prompting is a prompt engineering method that relies on the pretraining of a large language model (LLM) to infer an appropriate response. In contrast to other prompt engineering methods, such as few-shot prompting, models aren’t provided with examples of output when prompting with the zero-shot technique.