结合CAMEL框架与Qwen实现数据合成，奖励模型评估和数据过滤工作流

你也可以在 Colab 中查看这个教程点击这里 (Colab链接)

本笔记本展示了如何设置和利用 CAMEL 的奖励模型来评估和过滤合成数据。

在本笔记本中，你将探索以下内容：

CAMEL：一个强大的多代理框架，支持数据合成、评估、模型训练以及多代理角色扮演场景，从而实现复杂的 AI 驱动任务。
CAMEL FireCrawl 阅读器：Camel 中封装的 FireCrawl 加载器，允许用户通过 FireCrawl 检索网络信息。
奖励模型模块：用于根据预定义的标准对生成数据的质量进行评分和评估的关键组件。它支持评估过程的微调，确保与期望结果对齐，是高效过滤合成数据的关键工具。
这个教程展示了 CAMEL 如何作为一个灵活的框架，适应需要评估、过滤和优化 AI 生成内容的各种场景。

⭐ 为代码库点赞

如果你觉得 CAMEL 很有用或有趣，请考虑在我们的 CAMEL 仓库为我们点赞！你的点赞能帮助更多人发现这个项目，并激励我们持续改进。

📦 安装

首先，安装本cookbook会使用的所有依赖。

!pip install "camel-ai==0.2.14" firecrawl

接下来，我们需要安全地输入并存储访问 Qwen、Firecrawl 和 NVIDIA 服务所需的 API 密钥。

from getpass import getpass
import os

qwen_api_key = getpass('Enter your Qwen API key: ')
os.environ["QWEN_API_KEY"] = qwen_api_key

# Generate an API key at https://www.firecrawl.dev/app/api-keys
firecrawl_api_key = getpass('Enter your Firecrawl API key: ')
os.environ["FIRECRAWL_API_KEY"] = firecrawl_api_key

# Generate an API key at https://build.nvidia.com/nvidia/nemotron-4-340b-reward
nvidia_api_key = getpass('Enter your NVIDIA API key: ')
os.environ["NVIDIA_API_KEY"] = nvidia_api_key

为了高效地使用 Alpaca 格式并系统化地管理条目，我们使用 Pydantic 定义了一组模型。这些模型确保数据结构良好、类型安全，并经过验证。

from pydantic import BaseModel
from camel.messages.conversion import AlpacaItem

class NumberedAlpacaItem(BaseModel):
    number: int
    item: AlpacaItem


class AlpacaItemResponse(BaseModel):
    """
    Represents an instruction-response item in the Alpaca format.
    """
    items: list[NumberedAlpacaItem]

🚀 数据生成

接下来，我们定义数据生成函数。它会基于提供的源内容生成一系列指令-输入-响应三元组。

稍后，我们将使用奖励模型过滤这些三元组。

from typing import List
from camel.loaders import Firecrawl
from camel.models import ModelFactory
from camel.types import ModelPlatformType, ModelType
from camel.configs import ChatGPTConfig
from camel.agents import ChatAgent

def generate_alpaca_items(content: str, n_items: int, start_num: int = 1, examples: List[AlpacaItem] = None) -> List[AlpacaItem]:
    system_msg = """
You are an AI assistant generating detailed, accurate responses based on the provided content.
You will be given a reference content, and you must generate a specific number of AlpacaItems.
These are instruction-input-response triplets, where the input is the context or examples.

Add a number to the items to keep track of the order. Generate exactly that many.

For each instruction, imagine but do not include a real world scenario and real user in that scenario to inform realistic and varied instructions. Avoid common sense questions and answers.

Include multiple lines in the output as appropriate to provide sufficient detail. Cite the most relevant context verbatim in output fields, do not omit anything important.

Leave the input field blank.

Ensure all of the most significant parts of the context are covered.

Start with open ended instructions, then move to more specific ones. Consider the starting number for an impression of what has already been generated.
    """

    examples_str = ""
    if examples:
        examples_str = "\n\nHere are some example items for reference:\n" + \
            "\n".join(ex.model_dump_json() for ex in examples)

    model = ModelFactory.create(
        model_platform=ModelPlatformType.QWEN,
        model_type=ModelType.QWEN_TURBO,
        model_config_dict=ChatGPTConfig(
            temperature=0.6, response_format=AlpacaItemResponse
        ).as_dict(),
    )

    agent = ChatAgent(
        system_message=system_msg,
        model=model,
    )

    prompt = f"Content reference:\n{content}{examples_str}\n\n Generate {n_items} AlpacaItems. The first should start numbering at {start_num}."
    response = agent.step(prompt)

    # Parse the generated JSON to our wrapper class
    alpaca_items = [n_item.item for n_item in
                    AlpacaItemResponse.
                    model_validate_json(response.msgs[0].content).items]

    return alpaca_items

# Few shot examples to ensure the right amount of detail
examples = [
    AlpacaItem(
        instruction="Explain the process for sprint planning and review in CAMEL.",
        input="",
        output="The process for sprint planning and review in CAMEL includes:\n1. **Sprint Duration**: Each sprint lasts two weeks for development and one week for review.\n2. **Planning Meeting**: Conducted biweekly, where the founder highlights the sprint goal and developers select items for the sprint.\n3. **Review Meeting**: Stakeholders review the delivered features and provide feedback on the work completed during the sprint."
    )
]

📊 指向内容并生成数据！

现在我们指向希望围绕生成 SFT 数据的内容，并利用 CAMEL 的 Firecrawl 集成以整洁的 Markdown 格式获取该内容。

import random
from camel.loaders.firecrawl_reader import Firecrawl
firecrawl = Firecrawl()
# Scrape and clean content from a specified URL
response = firecrawl.scrape(
    url="https://github.com/camel-ai/camel/blob/master/CONTRIBUTING.md"
)

# Generate the items 50 a time up to 50
alpaca_entries = []
for start in range(1, 51, 50):
    # Combine default examples with random samples from previous generations
    current_examples = examples + (random.sample(alpaca_entries,
                            min(5, len(alpaca_entries)))
                            if alpaca_entries else [])

    batch = generate_alpaca_items(
        content=response["markdown"],
        n_items=50,
        start_num=start,
        examples=current_examples
    )
    print(f"Generated {len(batch)} items")
    alpaca_entries.extend(batch)

print(alpaca_entries)

Generated 50 items
[AlpacaItem(instruction='Summarize the contributing guidelines for the CAMEL project.', input='', output='The contributing guidelines for the CAMEL project include:\n1. **Types of Contributions**: Contributions can include new features, infrastructure enhancements, documentation improvements, issue reporting, and bug fixes.\n2. **Workflow**: Community contributors should follow the Fork-and-Pull-Request workflow, while members of CAMEL-AI.org should use the Checkout-and-Pull-Request workflow.\n3. **Pull Requests**: Mention related issues and tag relevant maintainers. Pull requests must pass formatting, linting, and testing checks before merging.'), AlpacaItem(instruction='What are the steps for writing a cookbook entry for CAMEL?', input='', output='The steps for writing a cookbook entry for CAMEL are:\n1. **Template Usage**: Use the provided template for consistency.\n2. **Writing in Colab**: Write your cookbook in Google Colab, ensuring that LLM-generated responses are correct and verified.\n3. **Clear Explanations**: Provide clear explanations of the code and its purpose.\n4. **Review and Submit**: After writing, review in Colab, then submit by downloading the notebook and creating a pull request.'), AlpacaItem(instruction='Describe the principles for writing docstrings in CAMEL.', input='', output='The principles for writing docstrings in CAMEL include:\n1. **Use Raw Strings**: Start with `r"""` to prevent issues with special characters.\n2. **Brief Descriptions**: Provide a concise summary of the functionality at the beginning.\n3. **Document Parameters**: Use an `Args` section to document parameters, including type and default values.'), AlpacaItem(instruction='What is the process for conducting code reviews in CAMEL?', input='', output='The process for conducting code reviews in CAMEL includes:\n1. **Reviewers Check**: Reviewers assess code for functionality, readability, and compliance with standards.\n2. **Feedback**: Leave constructive feedback where changes are necessary.\n3. **Approval**: The code must be approved by at least two reviewers before merging.'), AlpacaItem(instruction='Outline the steps for submitting a cookbook to CAMEL.', input='', output='The steps for submitting a cookbook to CAMEL are:\n1. **Finalize in Colab**: Ensure that the Colab notebook is complete and reviewed.\n2. **Download Notebook**: Download the finalized notebook as a .ipynb file.\n3. **Create Pull Request**: Open a pull request to add the .ipynb file to the repository, including any necessary documentation.'), AlpacaItem(instruction='Explain the importance of code coverage in CAMEL.', input='', output='Code coverage is important in CAMEL because it measures how much of the code is tested by unit tests. This helps in identifying robust areas of the codebase and areas that may need more testing, ensuring better reliability and maintainability of the code.'), AlpacaItem(instruction='What are the guidelines for labeling pull requests in CAMEL?', input='', output='The guidelines for labeling pull requests in CAMEL include:\n- **feat**: For new features.\n- **fix**: For bug fixes.\n- **docs**: For documentation updates.\n- **style**: For code style changes.\n- **refactor**: For code refactoring.\n- **test**: For adding or updating tests.\n- **chore**: For maintenance tasks.'), AlpacaItem(instruction='What communication channels are available for CAMEL contributors?', input='', output='The communication channels available for CAMEL contributors include:\n- **Discord**: For real-time communication.\n- **WeChat**: For Chinese-speaking contributors.\n- **Slack**: For team collaboration.'), AlpacaItem(instruction='Describe the purpose of the sprint planning process in CAMEL.', input='', output='The purpose of the sprint planning process in CAMEL is to define what can be delivered in the upcoming sprint and how those goals will be achieved. It ensures that the team has a clear understanding of the sprint objectives and tasks to focus on.'), AlpacaItem(instruction='What are the key components of the CAMEL project license?', input='', output='The CAMEL project is licensed under Apache 2.0, which means that contributions will also be licensed under the same terms by default. Contributors can add the license manually or use a script to automate the process.'), AlpacaItem(instruction='Summarize the guidelines for contributing to the CAMEL documentation.', input='', output='The guidelines for contributing to the CAMEL documentation include:\n1. **Comprehensive Coverage**: Provide thorough documentation for all classes and methods.\n2. **Sphinx Usage**: Documentation is generated automatically using Sphinx from the codebase.'), AlpacaItem(instruction='What steps should be taken to ensure reproducibility in the CAMEL cookbook?', input='', output='To ensure reproducibility in the CAMEL cookbook, contributors should:\n1. Write code that can be run and produces consistent results.\n2. Provide clear instructions for running the code.\n3. Validate all examples with real code execution.'), AlpacaItem(instruction='Explain the significance of logging over using print statements in CAMEL.', input='', output='Using logging instead of print statements in CAMEL is significant because it allows for more professional, configurable, and consistent logging. It helps in maintaining clean code and provides better control over the logging output.'), AlpacaItem(instruction='What is the process for updating dependencies in CAMEL?', input='', output='The process for updating dependencies in CAMEL includes:\n1. Modify the `pyproject.toml` file to add, update, or delete dependencies.\n2. Run `poetry lock` to synchronize the dependencies with the lock file.'), AlpacaItem(instruction='What are the responsibilities of a code reviewer in CAMEL?', input='', output='The responsibilities of a code reviewer in CAMEL include:\n1. Conduct timely reviews to keep the project moving.\n2. Provide clear and constructive feedback.\n3. Collaborate with contributors to ensure code meets project standards.'), AlpacaItem(instruction='Outline the steps to build documentation locally for CAMEL.', input='', output='To build documentation locally for CAMEL, follow these steps:\n1. Navigate to the `docs` directory.\n2. Run the command `make html` to generate the HTML documentation.'), AlpacaItem(instruction='What are the key principles to follow when writing entries for the CAMEL cookbook?', input='', output='The key principles for writing entries for the CAMEL cookbook include:\n1. High Quality: Ensure accurate code and clear explanations.\n2. Correctness: Validate LLM-generated responses with real code execution.\n3. Accessibility: Make content understandable for all users, from beginners to advanced.'), AlpacaItem(instruction='Describe how to run tests in CAMEL.', input='', output='To run tests in CAMEL, use the command `pytest .` in the project root directory. This will execute all tests, including those that require an OpenAI API key. For local isolated unit tests, use `pytest --fast-test-mode .`.'), AlpacaItem(instruction='What is the purpose of the code review checklist in CAMEL?', input='', output='The purpose of the code review checklist in CAMEL is to ensure that all contributions maintain high quality by evaluating functionality, code quality, design consistency, and documentation before merging into the main branch.'), AlpacaItem(instruction='Explain the process for scheduling an introduction call for new contributors in CAMEL.', input='', output='To schedule an introduction call for new contributors in CAMEL, visit the provided Calendly links:\n- For English speakers: Use the designated link for English.\n- For Chinese speakers: Use the designated link for Chinese.'), AlpacaItem(instruction='What are the main goals of the CAMEL project?', input='', output='The main goals of the CAMEL project include developing a robust open-source framework for AI applications, fostering community contributions, and ensuring high-quality documentation and testing.'), AlpacaItem(instruction="What is the significance of the 'Fork-and-Pull-Request' workflow in CAMEL?", input='', output="The 'Fork-and-Pull-Request' workflow in CAMEL is significant because it allows contributors to work on their own copy of the project and submit changes for review, ensuring that contributions are managed efficiently and collaboratively."), AlpacaItem(instruction='What should contributors do before their pull request can be merged?', input='', output='Before a pull request can be merged in CAMEL, it must pass formatting, linting, and testing checks. Contributors should also ensure that they have addressed any feedback from reviewers.'), AlpacaItem(instruction='Summarize the guidelines for writing clear and concise explanations in the CAMEL cookbook.', input='', output='The guidelines for writing clear and concise explanations in the CAMEL cookbook include:\n1. Use simple language to describe code functionality.\n2. Break down complex logic into understandable steps.\n3. Provide context for technical terms.'), AlpacaItem(instruction='What is the recommended way to provide feedback during code reviews in CAMEL?', input='', output='The recommended way to provide feedback during code reviews in CAMEL is to leave constructive comments that are clear and aimed at helping the contributor improve their code.'), AlpacaItem(instruction='What are the key components of the CAMEL community guidelines?', input='', output='The key components of the CAMEL community guidelines include promoting an inclusive environment, encouraging collaboration, and respecting diverse perspectives among contributors.'), AlpacaItem(instruction='Explain the significance of ensuring high-quality documentation in CAMEL.', input='', output='Ensuring high-quality documentation in CAMEL is significant because it helps users understand how to use the framework effectively, facilitates onboarding for new contributors, and promotes better collaboration within the community.'), AlpacaItem(instruction='What should contributors do if they encounter challenges during the setup process?', input='', output='If contributors encounter challenges during the setup process in CAMEL, they should reach out to a maintainer for assistance to ensure a smooth experience.'), AlpacaItem(instruction='Describe the process for merging pull requests in CAMEL.', input='', output='The process for merging pull requests in CAMEL involves:\n1. Ensuring the pull request has been approved by at least two reviewers.\n2. The merging should be done by a maintainer or an authorized contributor.'), AlpacaItem(instruction='What are the best practices for conducting a sprint review in CAMEL?', input='', output='Best practices for conducting a sprint review in CAMEL include:\n1. Gathering stakeholder feedback on delivered features.\n2. Identifying areas for improvement based on the work completed.'), AlpacaItem(instruction='What is the significance of using Google Colab for cookbook contributions?', input='', output='Using Google Colab for cookbook contributions is significant because it allows for interactive code execution alongside explanations, making it easier for users to understand and experiment with the code.'), AlpacaItem(instruction='What are the steps to create a new issue on the CAMEL GitHub repository?', input='', output="To create a new issue on the CAMEL GitHub repository, follow these steps:\n1. Go to the issues page on GitHub.\n2. Click on 'New Issue' and fill in the required information, including a proper title and description.\n3. Assign labels and update the assignees and milestones as necessary."), AlpacaItem(instruction="What should contributors do to maintain the project's coding standards?", input='', output="To maintain the project's coding standards, contributors should:\n1. Follow the established style guidelines.\n2. Use tools like Ruff for formatting and linting checks.\n3. Ensure documentation is up-to-date and comprehensive."), AlpacaItem(instruction='Explain the role of the CAMEL project maintainers.', input='', output='The role of the CAMEL project maintainers includes overseeing contributions, ensuring code quality, managing project direction, and facilitating communication within the community.'), AlpacaItem(instruction='What are the guidelines for submitting documentation updates in CAMEL?', input='', output='The guidelines for submitting documentation updates in CAMEL include:\n1. Ensure that updates are comprehensive and improve clarity.\n2. Follow the automated documentation generation process using Sphinx.'), AlpacaItem(instruction='What are the requirements for submitting a feature request in CAMEL?', input='', output='The requirements for submitting a feature request in CAMEL include providing a clear description of the feature, explaining its benefits, and linking to any relevant issues or discussions.'), AlpacaItem(instruction='Describe the importance of unit tests in the CAMEL project.', input='', output='Unit tests are important in the CAMEL project because they verify that individual components of the code work as expected, helping to catch bugs early and ensuring code reliability.'), AlpacaItem(instruction='What is the process for addressing feedback received during code reviews?', input='', output='The process for addressing feedback received during code reviews involves:\n1. Carefully reviewing the comments and suggestions provided by reviewers.\n2. Making necessary changes to the code based on the feedback.\n3. Updating the pull request to reflect the changes made.'), AlpacaItem(instruction='Summarize the steps for conducting a developer meeting in CAMEL.', input='', output='The steps for conducting a developer meeting in CAMEL include:\n1. Scheduling the meeting in advance and sharing the agenda.\n2. Discussing ongoing projects, challenges, and upcoming tasks.\n3. Gathering feedback and suggestions from team members.'), AlpacaItem(instruction='What should contributors include in their pull requests?', input='', output='Contributors should include the following in their pull requests:\n1. A clear title and description of the changes.\n2. Links to any related issues.\n3. Any necessary documentation or references.'), AlpacaItem(instruction='Explain the significance of following the pull request review process.', input='', output='Following the pull request review process is significant because it ensures that all contributions are evaluated for quality, functionality, and adherence to project standards, which helps maintain the integrity of the codebase.'), AlpacaItem(instruction='What are the best practices for writing unit tests in CAMEL?', input='', output='Best practices for writing unit tests in CAMEL include:\n1. Ensure tests cover all edge cases and expected outcomes.\n2. Keep tests isolated to avoid dependencies on other parts of the code.\n3. Use descriptive names for test functions to clarify their purpose.'), AlpacaItem(instruction='What is the process for integrating new features into the CAMEL project?', input='', output='The process for integrating new features into the CAMEL project involves:\n1. Submitting a feature request or proposal.\n2. Developing the feature in a separate branch.\n3. Creating a pull request for review once development is complete.'), AlpacaItem(instruction='Describe the importance of maintaining a clean and organized project structure in CAMEL.', input='', output='Maintaining a clean and organized project structure in CAMEL is important because it enhances code readability, facilitates easier navigation for contributors, and makes it simpler to manage and scale the project.'), AlpacaItem(instruction='What are the steps for conducting a retrospective after a sprint in CAMEL?', input='', output='The steps for conducting a retrospective after a sprint in CAMEL include:\n1. Gathering team feedback on what went well and what could be improved.\n2. Discussing challenges faced during the sprint.\n3. Identifying actionable items to implement in the next sprint.'), AlpacaItem(instruction='What should contributors do to ensure their code is secure?', input='', output='To ensure their code is secure, contributors should:\n1. Follow best practices for security in coding.\n2. Review code for potential vulnerabilities.\n3. Conduct security testing as part of the development process.'), AlpacaItem(instruction="What is the significance of the CAMEL community's code of conduct?", input='', output="The significance of the CAMEL community's code of conduct is to promote a respectful and inclusive environment for all contributors, ensuring that everyone feels welcome and valued while participating in the project."), AlpacaItem(instruction='Explain the relationship between CAMEL and its contributors.', input='', output='The relationship between CAMEL and its contributors is collaborative; contributors provide enhancements, fixes, and documentation, while CAMEL offers a platform for sharing knowledge and fostering community engagement.'), AlpacaItem(instruction='What are the benefits of participating in the CAMEL community?', input='', output='The benefits of participating in the CAMEL community include:\n1. Gaining experience in open-source development.\n2. Networking with other developers and contributors.\n3. Contributing to a project that aims to advance AI capabilities.'), AlpacaItem(instruction='Summarize the guidelines for writing clear commit messages in CAMEL.', input='', output="The guidelines for writing clear commit messages in CAMEL include:\n1. Use the imperative mood to describe the changes (e.g., 'Add feature' instead of 'Added feature').\n2. Keep messages concise and focused on the changes made.\n3. Include references to related issues when applicable.")]

len(alpaca_entries)

🔄 将代码转换为奖励模型格式

接下来，我们将 Alpaca 风格的条目转换为与奖励模型兼容的格式。每个条目将被转换为结构化的指令-输入-响应三元组列表，以便奖励模型进行评估。

messages_lists=[]
for item in alpaca_entries:
    messages_list =[]
    user_content = item.instruction
    if item.input:
        user_content += f"\nInput: {item.input}"
    messages_list.append({
   "role": "user", "content": user_content})
    messages_list.append({
   "role": "assistant", "content": item.output})
    messages_lists.append(messages_list)

✨测试奖励模型

然后，我们可以测试奖励模型以检查其输出格式，并将其用作设置过滤标准的参考。

from camel.models.reward import Evaluator, NemotronRewardModel
from camel.types import ModelType

reward_model = NemotronRewardModel(
    model_type=ModelType.NVIDIA_NEMOTRON_340B_REWARD,
    url="https://integrate.api.nvidia.com/v1",
)
evaluator = Evaluator(reward_model=reward_model)
results = []  # To store results for comparison
for i in range(min(10, len(messages_lists))):
    print(f"Evaluating message list {i+1}:")
    print(messages_lists[i])  # Display the message list
    scores = evaluator.evaluate(messages_lists[i])
    print(f"Scores: {scores}\n")  # Print the evaluation scores
    results.append((i + 1, messages_lists[i], scores))

# Print a summary of the results
print("\nSummary of evaluations:")
for index, messages, scores in results:
    print(f"Message List {index}:")
    print(messages)
    print(f"Scores: {scores}\n")

Evaluating message list 1:
[{'role': 'user', 'content': 'Summarize the contributing guidelines for the CAMEL project.'}, {'role': 'assistant', 'content': 'The contributing guidelines for the CAMEL project include:\n1. **Types of Contributions**: Contributions can include new features, infrastructure enhancements, documentation improvements, issue reporting, and bug fixes.\n2. **Workflow**: Community contributors should follow the Fork-and-Pull-Request workflow, while members of CAMEL-AI.org should use the Checkout-and-Pull-Request workflow.\n3. **Pull Requests**: Mention related issues and tag relevant maintainers. Pull requests must pass formatting, linting, and testing checks before merging.'}]
Scores: {'helpfulness': 2.21875, 'correctness': 2.40625, 'coherence': 3.453125, 'complexity': 1.6640625, 'verbosity': 1.078125}

Evaluating message list 2:
[{'role': 'user', 'content': 'What are the steps for writing a cookbook entry for CAMEL?'}, {'role': 'assistant', 'content': 'The steps for writing a cookbook entry for CAMEL are:\n1. **Template Usage**: Use the provided template for consistency.\n2. **Writing in Colab**: Write your cookbook in Google Colab, ensuring that LLM-generated responses are correct and verified.\n3. **Clear Explanations**: Provide clear explanations of the code and its purpose.\n4. **Review and Submit**: After writing, review in Colab, then submit by downloading the notebook and creating a pull request.'}]
Scores: {'helpfulness': 2.71875, 'correctness': 2.734375, 'coherence': 3.46875, 'complexity': 1.7109375, 'verbosity': 1.203125}

Evaluating message list 3:
[{'role': 'user', 'content': 'Describe the principles for writing docstrings in CAMEL.'}, {'role': 'assistant', 'content': 'The principles for writing docstrings in CAMEL include:\n1. **Use Raw Strings**: Start with `r"""` to prevent issues with special characters.\n2. **Brief Descriptions**: Provide a concise summary of the functionality at the beginning.\n3. **Document Parameters**: Use an `Args` section to document parameters, including type and default values.'}]
Scores: {'helpfulness': 1.5546875, 'correctness': 1.890625, 'coherence': 3.015625, 'complexity': 1.65625, 'verbosity': 0.6484375}

Evaluating message list 4:
[{'role': 'user', 'content': 'What is the process for conducting code reviews in CAMEL?'}, {'role': 'assistant', 'content': 'The process for conducting code reviews in CAMEL includes:\n1. **Reviewers Check**: Reviewers assess code for functionality, readability, and compliance with standards.\n2. **Feedback**: Leave constructive feedback where changes are necessary.\n3. **Approval**: The code must be approved by at least two reviewers before merging.'}]
Scores: {'helpfulness': 2.34375, 'correctness': 2.546875, 'coherence': 3.59375, 'complexity': 1.3984375, 'verbosity': 1.0625}

Evaluating message list 5:
[{'role': 'user', 'content': 'Outline the steps for submitting a cookbook to CAMEL.'}, {'role': 'assistant', 'content': 'The steps for submitting a cookbook to CAMEL are:\n1. **Finalize in Colab**: Ensure that the Colab notebook is complete and reviewed.\n2. **Download Notebook**: Download the finalized notebook as a .ipynb file.\n3. **Create Pull Request**: Open a pull request to add the .ipynb file to the repository, including any necessary documentation.'}]
Scores: {'helpfulness': 2.34375, 'correctness': 2.625, 'coherence': 3.546875, 'complexity': 1.46875, 'verbosity': 1.046875}

Evaluating message list 6:
[{'role': 'user', 'content': 'Explain the importance of code coverage in CAMEL.'}, {'role': 'assistant', 'content': 'Code coverage is important in CAMEL because it measures how much of the code is tested by unit tests. This helps in identifying robust areas of the codebase and areas that may need more testing, ensuring better reliability and maintainability of the code.'}]
Scores: {'helpfulness': 2.78125, 'correctness': 2.875, 'coherence': 3.8125, 'complexity': 1.7734375, 'verbosity': 1.0234375}

Evaluating message list 7:
[{'role': 'user', 'content': 'What are the guidelines for labeling pull requests in CAMEL?'}, {'role': 'assistant', 'content': 'The guidelines for labeling pull requests in CAMEL include:\n- **feat**: For new features.\n- **fix**: For bug fixes.\n- **docs**: For documentation updates.\n- **style**: For code style changes.\n- **refactor**: For code refactoring.\n- **test**: For adding or updating tests.\n- **chore**: For maintenance tasks.'}]
Scores: {'helpfulness': 2.859375, 'correctness': 2.984375, 'coherence': 3.78125, 'complexity': 1.6171875, 'verbosity': 1.1328125}

Evaluating message list 8:
[{'role': 'user', 'content': 'What communication channels are available for CAMEL contributors?'}, {'role': 'assistant', 'content': 'The communication channels available for CAMEL contributors include:\n- **Discord**: For real-time communication.\n- **WeChat**: For Chinese-speaking contributors.\n- **Slack**: For team collaboration.'}]
Scores: {'helpfulness': 1.4296875, 'correctness': 1.6796875, 'coherence': 3.59375, 'complexity': 0.796875, 'verbosity': 0.546875}

Evaluating message list 9:
[{'role': 'user', 'content': 'Describe the purpose of the sprint planning process in CAMEL.'}, {'role': 'assistant', 'content': 'The purpose of the sprint planning process in CAMEL is to define what can be delivered in the upcoming sprint and how those goals will be achieved. It ensures that the team has a clear understanding of the sprint objectives and tasks to focus on.'}]
Scores: {'helpfulness': 2.84375, 'correctness': 2.984375, 'coherence': 3.890625, 'complexity': 1.6015625, 'verbosity': 0.95703125}

Evaluating message list 10:
[{'role': 'user', 'content': 'What are the key components of the CAMEL project license?'}, {'role': 'assistant', 'content': 'The CAMEL project is licensed under Apache 2.0, which means that contributions will also be licensed under the same terms by default. Contributors can add the license manually or use a script to automate the process.'}]
Scores: {'helpfulness': 1.890625, 'correctness': 1.828125, 'coherence': 3.5625, 'complexity': 1.3515625, 'verbosity': 0.65625}


Summary of evaluations:
Message List 1:
[{'role': 'user', 'content': 'Summarize the contributing guidelines for the CAMEL project.'}, {'role': 'assistant', 'content': 'The contributing guidelines for the CAMEL project include:\n1. **Types of Contributions**: Contributions can include new features, infrastructure enhancements, documentation improvements, issue reporting, and bug fixes.\n2. **Workflow**: Community contributors should follow the Fork-and-Pull-Request workflow, while members of CAMEL-AI.org should use the Checkout-and-Pull-Request workflow.\n3. **Pull Requests**: Mention related issues and tag relevant maintainers. Pull requests must pass formatting, linting, and testing checks before merging.'}]
Scores: {'helpfulness': 2.21875, 'correctness': 2.40625, 'coherence': 3.453125, 'complexity': 1.6640625, 'verbosity': 1.078125}

Message List 2:
[{'role': 'user', 'content': 'What are the steps for writing a cookbook entry for CAMEL?'}, {'role': 'assistant', 'content': 'The steps for writing a cookbook entry for CAMEL are:\n1. **Template Usage**: Use the provided template for consistency.\n2. **Writing in Colab**: Write your cookbook in Google Colab, ensuring that LLM-generated responses are correct and verified.\n3. **Clear Explanations**: Provide clear explanations of the code and its purpose.\n4. **Review and Submit**: After writing, review in Colab, then submit by downloading the notebook and creating a pull request.'}]
Scores: {'helpfulness': 2.71875, 'correctness': 2.734375, 'coherence': 3.46875, 'complexity': 1.7109375, 'verbosity': 1.203125}

Message List 3:
[{'role': 'user', 'content': 'Describe the principles for writing docstrings in CAMEL.'}, {'role': 'assistant', 'content': 'The principles for writing docstrings in CAMEL include:\n1. **Use Raw Strings**: Start with `r"""` to prevent issues with special characters.\n2. **Brief Descriptions**: Provide a concise summary of the functionality at the beginning.\n3. **Document Parameters**: Use an `Args` section to document parameters, including type and default values.'}]
Scores: {'helpfulness': 1.5546875, 'correctness': 1.890625, 'coherence': 3.015625, 'complexity': 1.65625, 'verbosity': 0.6484375}

Message List 4:
[{'role': 'user', 'content': 'What is the process for conducting code reviews in CAMEL?'}, {'role': 'assistant', 'content': 'The process for conducting code reviews in CAMEL includes:\n1. **Reviewers Check**: Reviewers assess code for functionality, readability, and compliance with standards.\n2. **Feedback**: Leave constructive feedback where changes are necessary.\n3. **Approval**: The code must be approved by at least two reviewers before merging.'}]
Scores: {'helpfulness': 2.34375, 'correctness': 2.546875, 'coherence': 3.59375, 'complexity': 1.3984375, 'verbosity': 1.0625}

Message List 5:
[{'role': 'user', 'content': 'Outline the steps for submitting a cookbook to CAMEL.'}, {'role': 'assistant', 'content': 'The steps for submitting a cookbook to CAMEL are:\n1. **Finalize in Colab**: Ensure that the Colab notebook is complete and reviewed.\n2. **Download Notebook**: Download the finalized notebook as a .ipynb file.\n3. **Create Pull Request**: Open a pull request to add the .ipynb file to the repository, including any necessary documentation.'}]
Scores: {'helpfulness': 2.34375, 'correctness': 2.625, 'coherence': 3.546875, 'complexity': 1.46875, 'verbosity': 1.046875}

Message List 6:
[{'role': 'user', 'content': 'Explain the importance of code coverage in CAMEL.'}, {'role': 'assistant', 'content': 'Code coverage is important in CAMEL because it measures how much of the code is tested by unit tests. This helps in identifying robust areas of the codebase and areas that may need more testing, ensuring better reliability and maintainability of the code.'}]
Scores: {'helpfulness': 2.78125, 'correctness': 2.875, 'coherence': 3.8125, 'complexity': 1.7734375, 'verbosity': 1.0234375}

Message List 7:
[{'role': 'user', 'content': 'What are the guidelines for labeling pull requests in CAMEL?'}, {'role': 'assistant', 'content': 'The guidelines for labeling pull requests in CAMEL include:\n- **feat**: For new features.\n- **fix**: For bug fixes.\n- **docs**: For documentation updates.\n- **style**: For code style changes.\n- **refactor**: For code refactoring.\n- **test**: For adding or updating tests.\n- **chore**: For maintenance tasks.'}]
Scores: {'helpfulness': 2.859375, 'correctness': 2.984375, 'coherence': 3.78125, 'complexity': 1.6171875, 'verbosity': 1.1328125}

Message List 8:
[{'role': 'user', 'content': 'What communication channels are available for CAMEL contributors?'}, {'role': 'assistant', 'content': 'The communication channels available for CAMEL contributors include:\n- **Discord**: For real-time communication.\n- **WeChat**: For Chinese-speaking contributors.\n- **Slack**: For team collaboration.'}]
Scores: {'helpfulness': 1.4296875, 'correctness': 1.6796875, 'coherence': 3.59375, 'complexity': 0.796875, 'verbosity': 0.546875}

Message List 9:
[{'role': 'user', 'content': 'Describe the purpose of the sprint planning process in CAMEL.'}, {'role': 'assistant', 'content': 'The purpose of the sprint planning process in CAMEL is to define what can be delivered in the upcoming sprint and how those goals will be achieved. It ensures that the team has a clear understanding of the sprint objectives and tasks to focus on.'}]
Scores: {'helpfulness': 2.84375, 'correctness': 2.984375, 'coherence': 3.890625, 'complexity': 1.6015625, 'verbosity': 0.95703125}

Message List 10:
[{'role': 'user', 'content': 'What are the key components of the CAMEL project license?'}, {'role': 'assistant', 'content': 'The CAMEL project is licensed under Apache 2.0, which means that contributions will also be licensed under the same terms by default. Contributors can add the license manually or use a script to automate the process.'}]
Scores: {'helpfulness': 1.890625, 'correctness': 1.828125, 'coherence': 3.5625, 'complexity': 1.3515625, 'verbosity': 0.65625}

🎯使用奖励模型过滤生成数据

最后，我们利用 NVIDIA 的 Nemotron 奖励模型来过滤低质量的指令-输入-响应三元组。该模型根据如帮助性和正确性等指标的预定义阈值评估每个响应。

让我们使用的阈值为 {"helpfulness": 2.5, "correctness": 2.5} 作为过滤参数。过滤后，我们保留了一些高质量的三元组。

thresholds = {
   "helpfulness": 2.5, "correctness": 2.5}
filtered_messages_lists = []
for messages_list in messages_lists:
    response = evaluator.filter_data(messages_list, thresholds)
    if response:
        filtered_messages_lists.append(messages_list)

print(len(filtered_messages_lists))

filtered_messages_lists

[[{'role': 'user',
   'content': 'What are the steps for writing a cookbook entry for CAMEL?'},
  {'role': 'assistant',
   'content': 'The steps for writing a cookbook entry for CAMEL are:\n1. **Template Usage**: Use the provided template for consistency.\n2. **Writing in Colab**: Write your cookbook in Google Colab, ensuring that LLM-generated responses are correct and verified.\n3. **Clear Explanations**: Provide clear explanations of the code and its purpose.\n4. **Review and Submit**: After writing, review in Colab, then submit by downloading the notebook and creating a pull request.'}],
 [{'role': 'user',
   'content': 'Explain the importance of code coverage in CAMEL.'},
  {'role': 'assistant',
   'content': 'Code coverage is important in CAMEL because it measures how much of the code is tested by unit tests. This helps in identifying robust areas of the codebase and areas that may need more testing, ensuring better reliability and maintainability of the code.'}],
 [{'role': 'user',
   'content': 'What are the guidelines for labeling pull requests in CAMEL?'},
  {'role': 'assistant',
   'content': 'The guidelines for labeling pull requests in CAMEL include:\n- **feat**: For new features.\n- **fix**: For bug fixes.\n- **docs**: For documentation updates.\n- **style**: For code style changes.\n- **refactor**: For code refactoring.\n- **test**: For adding or updating tests.\n- **chore**: For maintenance tasks.'}],
 [{'role': 'user',
   'content': 'Describe the purpose of the sprint planning process in CAMEL.'},
  {'role': 'assistant',
   'content': 'The purpose of the sprint planning process in CAMEL is to define what can be delivered in the upcoming sprint and how those goals will be achieved. It ensures that the team has a clear understanding of the sprint objectives and tasks to focus on.'}],
 [{'role': 'user',
   'content': 'What are the key principles to follow when writing entries for the CAMEL cookbook?'},
  {'role': 'assistant',
   'content': 'The key principles for writing entries for the CAMEL cookbook include:\n1. High Quality: Ensure accurate code and clear explanations.\n2. Correctness: Validate LLM-generated responses with real code execution.\n3. Accessibility: Make content understandable for all users, from beginners to advanced.'}],
 [{'role': 'user',
   'content': 'What is the purpose of the code review checklist in CAMEL?'},
  {'role': 'assistant',
   'content': 'The purpose of the code review checklist in CAMEL is to ensure that all contributions maintain high quality by evaluating functionality, code quality, design consistency, and documentation before merging into the main branch.'}],
 [{'role': 'user',
   'content': "What is the significance of the 'Fork-and-Pull-Request' workflow in CAMEL?"},
  {'role': 'assistant',
   'content': "The 'Fork-and-Pull-Request' workflow in CAMEL is significant because it allows contributors to work on their own copy of the project and submit changes for review, ensuring that contributions are managed efficiently and collaboratively."}],
 [{'role': 'user',
   'content': 'What should contributors do if they encounter challenges during the setup process?'},
  {'role': 'assistant',
   'content': 'If contributors encounter challenges during the setup process in CAMEL, they should reach out to a maintainer for assistance to ensure a smooth experience.'}],
 [{'role': 'user',
   'content': 'What are the steps to create a new issue on the CAMEL GitHub repository?'},
  {'role': 'assistant',
   'content': "To create a new issue on the CAMEL GitHub repository, follow these steps:\n1. Go to the issues page on GitHub.\n2. Click on 'New Issue' and fill in the required information, including a proper title and description.\n3. Assign labels and update the assignees and milestones as necessary."}],
 [{'role': 'user',
   'content': "What should contributors do to maintain the project's coding standards?"},
  {'role': 'assistant',
   'content': "To maintain the project's coding standards, contributors should:\n1. Follow the established style guidelines.\n2. Use tools like Ruff for formatting and linting checks.\n3. Ensure documentation is up-to-date and comprehensive."}],
 [{'role': 'user',
   'content': 'Describe the importance of unit tests in the CAMEL project.'},
  {'role': 'assistant',
   'content': 'Unit tests are important in the CAMEL project because they verify that individual components of the code work as expected, helping to catch bugs early and ensuring code reliability.'}],
 [{'role': 'user',
   'content': 'What should contributors include in their pull requests?'},
  {'role': 'assistant',
   'content': 'Contributors should include the following in their pull requests:\n1. A clear title and description of the changes.\n2. Links to any related issues.\n3. Any necessary documentation or references.'}],
 [{'role': 'user',
   'content': 'Explain the significance of following the pull request review process.'},
  {'role': 'assistant',
   'content': 'Following the pull request review process is significant because it ensures that all contributions are evaluated for quality, functionality, and adherence to project standards, which helps maintain the integrity of the codebase.'}],
 [{'role': 'user',
   'content': 'Describe the importance of maintaining a clean and organized project structure in CAMEL.'},
  {'role': 'assistant',
   'content': 'Maintaining a clean and organized project structure in CAMEL is important because it enhances code readability, facilitates easier navigation for contributors, and makes it simpler to manage and scale the project.'}],
 [{'role': 'user',
   'content': "What is the significance of the CAMEL community's code of conduct?"},
  {'role': 'assistant',
   'content': "The significance of the CAMEL community's code of conduct is to promote a respectful and inclusive environment for all contributors, ensuring that everyone feels welcome and valued while participating in the project."}],
 [{'role': 'user',
   'content': 'Explain the relationship between CAMEL and its contributors.'},
  {'role': 'assistant',
   'content': 'The relationship between CAMEL and its contributors is collaborative; contributors provide enhancements, fixes, and documentation, while CAMEL offers a platform for sharing knowledge and fostering community engagement.'}],
 [{'role': 'user',
   'content': 'What are the benefits of participating in the CAMEL community?'},
  {'role': 'assistant',
   'content': 'The benefits of participating in the CAMEL community include:\n1. Gaining experience in open-source development.\n2. Networking with other developers and contributors.\n3. Contributing to a project that aims to advance AI capabilities.'}],
 [{'role': 'user',
   'content': 'Summarize the guidelines for writing clear commit messages in CAMEL.'},
  {'role': 'assistant',
   'content': "The guidelines for writing clear commit messages in CAMEL include:\n1. Use the imperative mood to describe the changes (e.g., 'Add feature' instead of 'Added feature').\n2. Keep messages concise and focused on the changes made.\n3. Include references to related issues when applicable."}]]

🌟 亮点

这就是全部内容了！如果你对 🐫 CAMEL-AI 有任何疑问，请加入我们的 Discord社区! 无论你是想分享反馈、探索多代理系统的最新进展、获取支持，还是与他人一起参与激动人心的项目，我们都欢迎你的加入！🤝

总结:
在本教程中，我们演示了如何利用 CAMEL-AI 来过滤生成的数据。为使用CAMEL框架和框架进行合成数据评估提供了实用的指导。

看看我们的其他一些相关工作：

🐫 创建你的第一个 CAMEL Agent Colab
🔥 使用 Firecrawl 和 CAMEL 从网站获取数据的三种方法 Colab
🦥 使用 CAMEL 和 Mistral 模型进行 Agentic SFT 数据生成，并通过 Unsloth 进行微调 Colab

感谢来自 🐫 CAMEL-AI 团队的每一位！

⭐ 为我们点赞 Github , 加入我们的社区 Discord 或者关注我们的社交账户 X ⭐

结合CAMEL框架与QWEN实现数据合成，奖励模型评估和数据过滤工作流

结合CAMEL框架与Qwen实现数据合成，奖励模型评估和数据过滤工作流

📦 安装

🚀 数据生成

📊 指向内容并生成数据！

🔄 将代码转换为奖励模型格式

✨测试奖励模型

🎯使用奖励模型过滤生成数据

🌟 亮点

热门文章

最新文章

相关课程

相关电子书

相关实验场景

热门

活动广场

任务中心

开发者评测

高校计划

乘风者计划

训练营

阿里云MVP

话题

直播

下载

镜像站

技术资料

插件

结合CAMEL框架与QWEN实现数据合成，奖励模型评估和数据过滤工作流

结合CAMEL框架与Qwen实现数据合成，奖励模型评估和数据过滤工作流

📦 安装

🚀 数据生成

📊 指向内容并生成数据！

🔄 将代码转换为奖励模型格式

✨测试奖励模型

🎯使用奖励模型过滤生成数据

🌟 亮点

热门文章

最新文章

相关课程

相关电子书

相关实验场景