Innovation Mindset – Prompt Engineering in Accounting (Notes)

Overview

• Prompt engineering involves crafting clear, concise instructions (prompts) for AI Large Language Models (LLMs) (e.g., ChatGPT, Bard, Llama) to elicit accurate, useful, and task-specific outputs.

• In accounting, it is crucial for leveraging AI to automate data entry, analyze financial data to derive insights, improve stakeholder communication, and enhance efficiency and accuracy.

• The goal is to move from routine tasks to more value-added, strategic work.

Principles of Effective Prompting

• Clarity: Provide context, format, and end-user details to avoid ambiguity in your prompt.

• Brevity: Balance sufficient context with concise instruction, keeping instructions focused.

• Iterative Refinement: Improve prompts through repeated testing and trial and error; prompts are often 80–90% complete, with humans finalizing the rest.

• Feedback Loops: Use ongoing refinement and feedback to improve the accuracy of model responses for domain-specific tasks.

Types of Prompting Techniques

• Zero-shot prompting: Ask the model to perform a task without providing any examples, relying on its pre-existing knowledge.

• Few-shot prompting: Provide a few examples within the prompt to guide the model toward the desired output format and content.

• Chain-of-thought prompting: Request the AI model to show its intermediate reasoning steps, which can be applied with or without examples.

Examples in Accounting

• Data Entry: Prompt models to extract and categorize financial data from invoices into debit and credit columns.

  • Example Prompt: $ext{"Extract and categorize the financial data from the attached invoice into debit and credit columns."}$

• Financial Analysis: Prompt models to examine financial reports and highlight anomalies or significant year-over-year variations in revenue or expense accounts.

  • Example Prompt: $ext{"Examine the quarterly financial report and highlight any anomalies in the revenue and expense accounts compared to the previous quarter."}$

• Client Communication: Prompt models to draft professional emails or summaries for clients, detailing audit findings or clarifying discrepancies.

  • Example Prompt: $ext{"Draft a professional email to my client, Sam Wilson, at ABC Widgets, summarizing the key findings from their annual financial statement audit and suggesting a meeting to discuss further."}$

• Assessing Internal Controls: Identify suspicious or repeated transactions from logs that might indicate control weaknesses or fraud.

  • Example Prompt: $ext{Based on the provided transaction logs, identify any suspicious or repeated transactions that might indicate a potential control weakness or fraudulent activity. Compare the frequency and volume of such transactions to the historical average.}$

Risks and Avoiding Hallucinations

• Hallucinations: AI models may produce incorrect, fabricated, or nonsensical outputs, which is a significant risk, especially in accounting.

• Common Causes of Hallucinations:

  • Ambiguous or unclear prompts.

  • Lack of domain-specific training for the model.

  • Long or overly complex responses requested.

  • Uncommon or rare scenarios.

  • Biased or outdated training data.

  • Overreliance on the model's perceived confidence.

  • Iterative prompting that may unintentionally mislead the model.

• Mitigation Strategies to Avoid Hallucinations:

  • Clarity and Specificity: Design clear, concise, and domain-focused prompts, avoiding ambiguity.

  • Break Down Complexity: Break complex questions or tasks into smaller, manageable parts.

  • Verify Outputs: Always include a human review step to validate AI outputs against reliable source documents and calculations before dissemination. Do not rely solely on model confidence.

  • Contextual Awareness: Be cautious with long or complex responses and verify their accuracy.

  • Domain Expertise: Leverage domain-specific training and knowledge in prompt design.

  • Avoid Uncommon Scenarios: Exercise caution when dealing with rare or unfamiliar accounting scenarios.

  • Example of Hallucinated Calculation:

    • Prompt: $ext{Calculate the net profit given: Total revenue of } 1000000, ext{ COGS } = 500000, ext{ Operating expenses } = 300000.$

    • Hallucinated (Incorrect) Response: "The net profit is $250,000."

    • Correct Calculation: $ext{Net profit} = ext{Total revenue} - ( ext{COGS} + ext{Operating expenses})$

      • $1000000 - (500000 + 300000) = 200000$

Conclusion

• Prompt engineering is a powerful and evolving skill essential for leveraging AI in accounting.

• It is best learned through experimentation, iterative refinement of prompts, and continuous feedback.

• Always include a human review and validation step for AI outputs to ensure accuracy, consistency, and adherence to professional standards before finalizing deliverables.