AI 辅助测试(MEAP)(一)(4)https://developer.aliyun.com/article/1516295
4.4.5 重构代码
与他人合作的一个好处是合作伙伴能够看到我们可能没有看到的潜在问题和风险。然而,在没有合作伙伴的情况下,我选择使用本章早些时候探讨过的第二个提示——具体地分析代码并返回建议的提示。将之前的提示与我的代码和质量特征的准确性和一致性相结合,我将这个提示发送给 ChatGPT:
#A You are a software testing expert. Analyze the Java code delimited by triple hashes and identify risks that might impact the code. If no code is provided, respond with "No risks could be found" #B * Risks must be related to the quality characteristics: Accuracy and Consistency * Multiple risks can be identified for each quality characteristic * Output identified risks in the format of Risk - Quality Characteristic - Explanation #C Example: Overflow - Security - Adding in large integers as parameters might cause the method to trigger a buffer overflow. #D ### public class Timesheet { private HashMap<String, Integer> timesheets = new HashMap<>(); public boolean submitTimesheet(String projectName, int duration) { if (projectName != null && duration > 0) { // Check to see if project already exists if (timesheets.containsKey(projectName)) { // If it does, add the new duration to the existing total duration += timesheets.get(projectName); } timesheets.put(projectName, duration); return true; } else { return false; } } public Collection<Integer> getTimesheets() { return timesheets.values(); } public int getTotalTimesheetHours(String projectName) { return timesheets.get(projectName); } } ###
发送提示返回了以下我可能想考虑的风险:
Risk 1 - Accuracy - submitTimesheet method can overwrite existing duration: The submitTimesheet method allows adding a new duration to an existing project's total if the project already exists in the timesheets. However, this approach may not always ensure accuracy. If the user intends to submit a correction or update the duration for a specific project, this method will not provide accurate results. It merely adds the new duration to the existing one without considering the possibility of editing or deleting a previous entry. Risk 2 - Consistency - Null key in timesheets HashMap: The timesheets HashMap allows null keys. This might lead to inconsistency issues when querying for timesheet information by project name. If a project with a null name is mistakenly added to the timesheets, it may cause issues in subsequent operations, leading to unexpected behavior. Risk 3 - Consistency - Lack of validation for project names: The submitTimesheet method does not validate the project name before adding it to the timesheets HashMap. This lack of validation might result in inconsistent data, such as duplicate or improperly formatted project names. For example, if two different variations of the same project name (e.g., "ProjectA" and "projecta") are added, the system might treat them as separate projects, leading to inconsistencies in reporting or querying. Risk 4 - Accuracy - getTotalTimesheetHours method may throw NullPointerException: The getTotalTimesheetHours method directly calls timesheets.get(projectName), which may throw a NullPointerException if the given projectName is not present in the timesheets HashMap. This can lead to inaccurate results if the method is used without checking for the existence of the project name in the timesheets before calling this method. Risk 5 - Accuracy - Return type of getTotalTimesheetHours method: The getTotalTimesheetHours method returns an int data type, which might lead to accuracy issues if the actual total timesheet hours for a project exceed the maximum value that an int can hold. This can result in data loss due to overflow if the sum of durations for a project exceeds Integer.MAX_VALUE. Risk 6 - Consistency - Lack of synchronization in a multi-threaded environment: If this Timesheet class is used in a multi-threaded environment, there is a risk of inconsistent behavior since the methods of this class are not synchronized. This can lead to data corruption or incorrect calculations if multiple threads concurrently access and modify the timesheets HashMap.
与问题提示不同,这个回复似乎指出了我的代码的具体项目,并说明了事情可能出错的原因。有了这个回复,我有以下几个选择:
- 逐个检查每个风险,然后减轻我认为重要的风险并忽略其他风险。
- 完全忽略所提出的风险,或者也许发送另一个提示给 ChatGPT,看看是否还有更多的风险。
- 将 ChatGPT 的回复本身作为提示,帮助我重构代码。
为了演示的目的,我选择了第三个选项,并向 ChatGPT 发送了一个额外的提示:
Refactor the code to mitigate the risks suggested. Check to see that each risk has been mitigated before outputting the solution.
注意我是如何运用思考时间原则来让 LLM 检查每个风险是否确实被它建议的重构代码所减轻的。这有助于增加返回的代码实际上会有用的可能性。将此提示发送给 ChatGPT 返回了以下代码:
import java.util.Collection; import java.util.concurrent.ConcurrentHashMap; public class Timesheet { #A private ConcurrentHashMap<String, Long> timesheets = new ConcurrentHashMap<>(); public boolean submitTimesheet(String projectName, long duration) { if (projectName != null && duration > 0) { // Perform validation on project name by converting it to lowercase #B projectName = projectName.toLowerCase(); // Check to see if project already exists timesheets.put(projectName, timesheets.getOrDefault(projectName, 0L) + duration); return true; } else { return false; } } #C public Collection<Long> getTimesheets() { return timesheets.values(); } public long getTotalTimesheetHours(String projectName) { return timesheets.getOrDefault(projectName.toLowerCase(), 0L); } }
为了检查这些修改是否有效,我将建议的代码复制到Timesheet
中,修改TimesheetTest
中的testCalculateTotalHoursWorked
以接受一个long
而不是int
,并发现我的检查仍然通过了,我已经重构了我的代码。
然而,这实际上改进了我的代码吗?为了确认,我再次用我的更新后的代码运行原始代码分析提示,这一次收到了新的风险:
Risk - Accuracy - Case Sensitivity Issue: Explanation: The method submitTimesheet converts the projectName to lowercase before using it as a key in the timesheets map. This conversion might lead to inaccurate data retrieval if there are projects with the same name but different letter cases (e.g., "ProjectA" and "projecta" will be considered the same project). Risk - Consistency - Inconsistent Data Entry: Explanation: The submitTimesheet method does not enforce consistent data entry for projectName values. If different users enter project names in various formats (e.g., "Project A", "project-A", "project_A"), it could lead to multiple entries for what should be the same project in the timesheets map. Risk - Accuracy - Missing Validation for Duration: Explanation: Although the method submitTimesheet checks for a positive duration, it does not handle cases where the duration is 0 or negative. This could lead to incorrect data being recorded in the timesheets map. Risk - Consistency - Lack of Input Sanitization: Explanation: The submitTimesheet method does not sanitize the projectName input, which could lead to inconsistent data due to invalid characters or unexpected inputs. Risk - Accuracy - Potential Data Loss: Explanation: The method submitTimesheet returns a boolean value, but it does not provide any feedback or error handling mechanism if the submission fails. This lack of feedback could lead to data loss if the submission fails silently, and the user is not aware of the issue.
这个回应中有趣的是,我可以确认一些风险已经得到了减轻,比如:
Risk 5 - Accuracy - Return type of getTotalTimesheetHours method: The getTotalTimesheetHours method returns an int data type, which might lead to accuracy issues if the actual total timesheet hours for a project exceed the maximum value that an int can hold. This can result in data loss due to overflow if the sum of durations for a project exceeds Integer.MAX_VALUE.
但似乎仍然有一些风险还没有被减轻。例如,在我收到的第一个风险清单中,我收到了以下风险:
Risk 3 - Consistency - Lack of validation for project names: The submitTimesheet method does not validate the project name before adding it to the timesheets HashMap. This lack of validation might result in inconsistent data, such as duplicate or improperly formatted project names. For example, if two different variations of the same project name (e.g., "ProjectA" and "projecta") are added, the system might treat them as separate projects, leading to inconsistencies in reporting or querying.
ChatGPT 通过实现一个lowerCase
方法来帮助清理项目名称来处理这个风险。然而,在第二次分析中,我收到了以下内容:
Risk - Consistency - Inconsistent Data Entry: Explanation: The submitTimesheet method does not enforce consistent data entry for projectName values. If different users enter project names in various formats (e.g., "Project A", "project-A", "project_A"), it could lead to multiple entries for what should be the same project in the timesheets map.
这个风险与原始的、据称已经减轻的风险非常相似。感觉这个关于不一致数据输入的额外风险应该在我的代码重构时得到妥善处理。我可以再次要求 LLM 为我重构代码,但考虑到与 LLM 打转的潜力,对我来说更明智的做法是自己主导并修复问题。这是一个重要的技能要发展,何时依赖于 LLM,何时主导。
选择重要的原因可以通过第二轮分析提出的其他建议风险之一来突出显示。具体地说:
Risk - Accuracy - Missing Validation for Duration: Explanation: Although the method submitTimesheet checks for a positive duration, it does not handle cases where the duration is 0 or negative. This could lead to incorrect data being recorded in the timesheets map.
这听起来像是一个令人信服的风险,但这只是一种幻觉的示范。就目前的代码而言,如果持续时间为 0 或更少,则该方法只需返回 false 并退出时间表存储:
if (projectName != null && duration > 0)
有时,LLMs 可能会存在偏见,优先考虑提供答案,而不考虑其质量。这意味着我们越多次要求 LLM 分析我们的代码,它就越有可能开始产生幻觉,以给出它正在产生有用结果的印象,而不是返回一个没有有用信息可分享的响应。这就是为什么我们必须仔细掌握何时使用 LLM 何时不使用的原因。
在这一点上,我选择停止使用案例,因为我们所涵盖的内容展示了不同类型的 LLMs 如何在不同方面帮助我。Copilot 提供了快速生成代码的能力,但它需要基于代码的提示来帮助它提出建议。项目越新,我们越有可能在 Copilot 中看到命中和未命中,但我们可以通过单元检查来帮助这个过程。这不仅有助于指导 Copilot 构建我们的代码,而且给我们带来了 TDD 的好处,包括良好设计的可测试代码。
通过 ChatGPT,我们已经证明了它可以是一个有用的分析工具,当正确提示时。构建可以分析思想和代码并提出风险和改进建议的提示可以迅速为我们提供不同的考虑角度,然后我们可以采取行动或拒绝。利用 LLM 作为倡导质量角色的模拟可以帮助我们改进工作。
4.5 改进与 LLMs 的文档和沟通
尽管看起来可能不是那么重要,但通过代码注释和发布说明来传达我们所做工作的方式对产品质量的贡献可能很大。通过分享新的发展和对代码库的更改,我们可以帮助其他开发人员了解我们的工作如何影响他们的工作,指导测试人员在测试我们的工作时应该关注什么,甚至帮助用户了解我们的产品(例如,Slack 早期的发布说明通过其清晰的沟通和幽默帮助市场推广他们的工具)。
尽管存在这些好处,文档和发布说明有时会被放置在开发周期的最后阶段,或者完全被忽略。考虑到编写和维护有用和相关的代码注释和发布说明所需的时间,尤其是当有时间压力不断交付新功能时,这是有道理的。然而,通过使用 LLMs,我们可以减少这种时间开销,同时确保我们创建有用的文档,为未来的读者创造价值。所以让我们看一些可以迅速为我们生成文档的有用提示。
4.5.1 生成代码注释
尽管我们应该始终努力创建以流畅易读的方式排列的代码,无论对于代码库的经验如何,代码注释可以提供额外的细节,以防止代码的误用并加快开发速度。如果我们发布的 API 将被他人使用,这一点尤为重要。(我曾多次因为 API 文档记录不足而浪费时间,试图理解一个库的工作原理。)挑战在于在代码注释中取得正确的平衡:太少了,读你代码的人将不得不自行解决;太多了,我们将为自己增加更多的工作量,以确保代码和注释保持一致。
因此,让我们看一个可以为我们处理注释工作的提示:
#A You are a Java Developer. Update the code delimited by three hashes and add in code comments to explain how the code works. Check that no parts of the code have been changed before outputting the completed code commented code. #B ### import java.util.*; import java.util.concurrent.ConcurrentHashMap; public class Timesheet { private ConcurrentHashMap<String, Long> timesheets = new ConcurrentHashMap<>(); public boolean submitTimesheet(String projectName, long duration) { if (projectName != null && duration > 0) { projectName = projectName.toLowerCase(); timesheets.put(projectName, timesheets.getOrDefault(projectName, 0L) + duration); return true; } else { return false; } } public Collection<Long> getTimesheets() { return timesheets.values(); } public long getTotalTimesheetHours(String projectName) { return timesheets.getOrDefault(projectName.toLowerCase(), 0L); } } ###
在这个提示中,我们使用通常的分隔符策略提供我们希望看到注释的代码,但我们也希望确保 LLM 不会改变我们的代码,以使其与它创建的注释保持一致。所以我们在提示中也明确表示,在输出完成的注释代码之前,检查代码的任何部分是否已更改。将此提示与我们之前的示例代码发送到 ChatGPT 返回了以下注释代码:
import java.util.*; import java.util.concurrent.ConcurrentHashMap; public class Timesheet { // ConcurrentHashMap to store project timesheets where the key is the project name (in lowercase) and the value is the total duration. private ConcurrentHashMap<String, Long> timesheets = new ConcurrentHashMap<>(); // Method to submit a timesheet entry for a project. public boolean submitTimesheet(String projectName, long duration) { if (projectName != null && duration > 0) { // Normalize the project name to lowercase to ensure case-insensitive matching. projectName = projectName.toLowerCase(); // Update the timesheet entry for the project by adding the submitted duration. timesheets.put(projectName, timesheets.getOrDefault(projectName, 0L) + duration); return true; // Submission successful } else { return false; // Invalid input, submission failed } } // Method to retrieve all timesheet durations. public Collection<Long> getTimesheets() { return timesheets.values(); } // Method to get the total hours for a specific project. public long getTotalTimesheetHours(String projectName) { // Retrieve the total duration for the specified project (case-insensitive) or return 0 if not found. return timesheets.getOrDefault(projectName.toLowerCase(), 0L); } }
正如我们所见,现在方法和其中的代码段都已经被注释以提供更多细节。最引人注目的是诸如:
// ConcurrentHashMap to store project timesheets where the key is the project name (in lowercase) and the value is the total duration. private ConcurrentHashMap<String, Long> timesheets = new ConcurrentHashMap<>();
提供的注释不仅提到了ConcurrentHashMap
的用途,还强调了所有项目名称将以小写形式存储,这是对submitTimesheet
内部代码的一个参考,与此相比,像这样的注释感觉有点多余:
// Method to submit a timesheet entry for a project. public boolean submitTimesheet(String projectName, long duration)
再次,我们可以自行选择使用什么,不使用什么。虽然我们使用的提示已经为我们提供了注释,但我们观察到有些可能比其他更有用。因此,我们的角色从注释的作者变为编辑,选择并调整看起来最有用的注释。
使用提示维护代码注释
这个过程如此有用的原因在于,当我们不可避免地遇到代码变化和我们的注释需要更新的时候,我们只需简单地运行带有我们更新代码的提示,以重新生成我们的注释,以整合我们的变化。
最后的观察是,尽管注释对于直接阅读我们的代码的人可能很有用,但如果我们想以结构化文档的形式(例如 Javadoc)提供更多的文档怎么办?在那种情况下,我们可以修改提示,并使用结构化数据策略来创建一个如下所示的提示:
#A You are a Java Developer. Update the code delimited by three hashes and add in code comments in a JavaDoc format to explain how the code works. Check that no parts of the code have been changed before outputting the completed code commented code. #B ### import java.util.*; import java.util.concurrent.ConcurrentHashMap; public class Timesheet { private ConcurrentHashMap<String, Long> timesheets = new ConcurrentHashMap<>(); public boolean submitTimesheet(String projectName, long duration) { if (projectName != null && duration > 0) { projectName = projectName.toLowerCase(); timesheets.put(projectName, timesheets.getOrDefault(projectName, 0L) + duration); return true; } else { return false; } } public Collection<Long> getTimesheets() { return timesheets.values(); } public long getTotalTimesheetHours(String projectName) { return timesheets.getOrDefault(projectName.toLowerCase(), 0L); } } ###
将此提示与我们的示例代码一起发送给 ChatGPT 产生以下结果:
import java.util.*; import java.util.concurrent.ConcurrentHashMap; /** * Represents a timesheet management system for tracking hours spent on different projects. */ public class Timesheet { // A concurrent hash map to store project names and their corresponding total hours private ConcurrentHashMap<String, Long> timesheets = new ConcurrentHashMap<>(); /** * Submits a timesheet entry for a project. * * @param projectName The name of the project. * @param duration The duration in hours to be submitted. * @return true if the submission was successful, false otherwise. */ public boolean submitTimesheet(String projectName, long duration) { if (projectName != null && duration > 0) { // Normalize the project name to lowercase for consistency projectName = projectName.toLowerCase(); // Update the total hours for the project or add a new entry if it doesn't exist timesheets.put(projectName, timesheets.getOrDefault(projectName, 0L) + duration); return true; } else { return false; } } /** * Retrieves a collection of all total timesheet hours for all projects. * * @return A collection of total hours for all projects. */ public Collection<Long> getTimesheets() { return timesheets.values(); } /** * Retrieves the total timesheet hours for a specific project. * * @param projectName The name of the project. * @return The total hours spent on the specified project, or 0 if the project is not found. */ public long getTotalTimesheetHours(String projectName) { // Normalize the project name to lowercase for consistency and retrieve the total hours return timesheets.getOrDefault(projectName.toLowerCase(), 0L); } }
使用这个提示,我们改进了一些注释的质量,并创建了可以用于为外部用户文档化我们的代码的 JavaDoc 友好的注释。我们需要做的就是在适当的地方进行微调和编辑。
4.5.2 生成发布说明
正如我们可以使用 LLM 修改我们的代码以添加注释一样,我们也可以使用 LLM 将我们的代码库转换为可以由团队和更广泛的人员消费的发布说明。为此,我们创建一个具有以下详细信息的新提示:
#A You are a Java Developer. Convert the code delimited by three hashes into a bullet-pointed set of release notes that describes how the code works. Check that the release notes can be read by someone who has no experience with the code created before outputting it. #B * Only output the release notes. #C ### import java.util.*; import java.util.concurrent.ConcurrentHashMap; public class Timesheet { private ConcurrentHashMap<String, Long> timesheets = new ConcurrentHashMap<>(); public boolean submitTimesheet(String projectName, long duration) { if (projectName != null && duration > 0) { projectName = projectName.toLowerCase(); timesheets.put(projectName, timesheets.getOrDefault(projectName, 0L) + duration); return true; } else { return false; } } public Collection<Long> getTimesheets() { return timesheets.values(); } public long getTotalTimesheetHours(String projectName) { return timesheets.getOrDefault(projectName.toLowerCase(), 0L); } } ###
这个提示遵循了与以前的代码注释为重点的提示类似的模式,但这次我们不是要求它将注释注入我们的代码,而是要求将代码转换为自然语言发布说明。例如,将带有示例代码的提示发送给 ChatGPT 返回了以下结果:
Release Notes: * The code defines a Java class named Timesheet that is used to manage timesheet data. * It uses a ConcurrentHashMap to store timesheet entries, where project names (in lowercase) are associated with their respective durations (in hours). * The submitTimesheet method allows users to submit timesheet entries. It requires a project name and a positive duration as parameters. If these conditions are met, it stores the duration associated with the project name in the map. If not, it returns false. * The getTimesheets method returns a collection of all recorded timesheet durations. * The getTotalTimesheetHours method allows users to retrieve the total duration for a specific project name (case-insensitive). If the project name is not found, it returns 0. Note: Ensure that project names are provided in lowercase when using the submitTimesheet method to avoid case sensitivity issues when retrieving total timesheet hours.
我们得到的结果是完全不同于我们原始代码的格式,试图详细说明我们的代码库做了什么。回顾输出,它确实感觉非常技术化,这是有道理的,因为我们要求对少量代码进行详细说明。然而,即使是这些发布说明也可以派上用场,因为我们可以修改先前的提示以建议分析发布说明的风险:
#A You are a software testing expert. Analyze the release notes delimited by triple hashes and identify risks that might impact the code. If no code is provided, respond with "No risks could be found" #B * Risks must be related to the quality characteristics: Performance, security and interoperability * Multiple risks can be identified for each quality characteristic * Output identified risks in the format of Risk - Quality Characteristic - Explanation #C Example: Overflow - Security - Adding in large integers as parameters might cause the method to trigger a buffer overflow. #D ### Release Notes: * The code defines a Java class named Timesheet that is used to manage timesheet data. * It uses a ConcurrentHashMap to store timesheet entries, where project names (in lowercase) are associated with their respective durations (in hours). * The submitTimesheet method allows users to submit timesheet entries. It requires a project name and a positive duration as parameters. If these conditions are met, it stores the duration associated with the project name in the map. If not, it returns false. * The getTimesheets method returns a collection of all recorded timesheet durations. * The getTotalTimesheetHours method allows users to retrieve the total duration for a specific project name (case-insensitive). If the project name is not found, it returns 0. Note: Ensure that project names are provided in lowercase when using the submitTimesheet method to avoid case sensitivity issues when retrieving total timesheet hours. ###
使用这个提示在 ChatGPT 中返回了以下建议的风险:
Concurrent Modification - Performance - The use of ConcurrentHashMap may not be suitable for high concurrent access, leading to performance bottlenecks if there are frequent modifications to the timesheet entries by multiple users simultaneously. Lack of Input Validation - Security - The submitTimesheet method lacks proper input validation. It only checks for positive durations but does not validate the project name, making it vulnerable to injection attacks or other malicious input. Case Sensitivity - Interoperability - The requirement to provide project names in lowercase may lead to interoperability issues when integrating with systems that expect case-insensitive input or have different naming conventions. No Authentication/Authorization - Security - The release notes do not mention any authentication or authorization mechanisms, which can pose security risks if unauthorized users can submit or access timesheet data. Data Exposure - Security - The getTimesheets method provides access to all recorded timesheet durations, which might expose sensitive information if not properly restricted or authenticated.
当然,如果我们可以直接要求 LLM 根据代码直接提出风险,这可能是一个不必要的步骤,但它确实展示了 LLM 将数据从一种格式转换为另一种格式的能力。
4.6 与代码助手保持平衡
对于像 Copilot 和 ChatGPT 这样的工具所提供的价值观取决于个人。对于一些人来说,这意味着开发的自动化和许多角色的流失,对于其他人来说,它只是一个随机选择建议代码的先进工具。本章展示的持续主题是它们的价值和用处在这两个极端之间。
这些模型是在大量来自 Stack Overflow 和 GitHub 存储库的数据上进行训练的,它们对于生产和测试代码提出了非常复杂的建议。但是它们仍然需要我们人类的指导 - 这种指导是由我们与利益相关者沟通的能力、分析需求和设计实施所引导的。我们能够多好地使用 AI 工具来进行开发取决于我们如何完善我们的辅助技能,这可以通过使用图 4.2 所示的区域效应模型进行总结。
图 4.2 一个区域效应模型,更新以展示人类技能和代码助手工具的能力
平衡的方法可以帮助我们更快地交付功能,但仍然确保我们建立质量。因此,我们的目标是在需要依赖我们自己的能力或代码助手工具的情况下保持平衡。有时,代码助手无法提供正确的实现建议,我们需要掌控。这可以给我们更多的控制,但会牺牲速度。在其他时候,我们可以依赖代码助手工具参考大量数据来通过单元测试或对话来提出新的设计理念。但我们希望确保我们的 TDD 循环专注于设计,而不是测试覆盖率。过多的单元测试会使我们失去设计的视野,最终陷入一个钩钩检查的活动中。
4.7 总结
- 目前市场上大多数的人工智能工具都依赖于从互联网上采集的大量数据进行训练的大型语言模型。
- LLM 是一种先进的算法,通过对我们的请求应用统计分析来确定应该回应的输出内容。
- Copilot 是一个使用 OpenAI GPT-4 并训练于存储在 GitHub 上的代码的编码助手工具。
- Copilot 在 IDE 中工作,通过读取您的代码作为提示来建议添加到测试代码和生产代码中的内容。
- 像 Copilot 这样的工具可以很好地与 TDD 红/绿/重构循环配合使用,帮助我们快速创建单元测试和生产代码。
- 为了帮助 Copilot 返回有价值的代码,我们需要通过提示对其进行引导。
- 成功地使用人工智能代码助手取决于我们对自己的能力以及代码助手工具的功能的了解。
- 我们的主导程度与工具主导设计之间存在一种推拉关系。
- 当平衡从人类主导转变为工具主导时,我们必须意识到其中的权衡。