I have a document that I am extracting data from using a qbo3 ScoreTemplate using the AiScoreEngine. Additionally, I need to set the Document's Template. I have defined the following items in my score:
- Account Number
- Document is a Mortgage?
- Document is a Note?
- Document is a Payoff Request?
- ...
Then I have a matrix that defined a template based on whether my score questions are true or false:
| Mortgage | Note | Payoff Request | Template |
|---|---|---|---|
| yes | (not yes) | (not yes) | Mortgage |
| (not yes) | yes | (not yes) | Note |
| (not yes) | (not yes) | yes | Payoff |
Sometimes, my classification based on my matrix fails. What am I doing wrong?
Answer
The AiScoreEngine will prompt the LLM along these lines:
Extract the following values as key value pairs:
- AccountNumber
- Document is a Mortgage
- Document is a Note
- Document is a Payoff Request
from the following text:
{your document text here}
Depending on the text of your document, it's possible that the LLM will answer Yes to more than one of your questions. E.g. LLM may think that the document is both a Note and a Payoff request.
The presence of (not yes) in your Matrix would prohibit accepting any rows whether AI answered yes to more than one of your document template questions.
You might address this by removing the (not yes) from each row, and weighing your Mortgage, Note, and Payoff Request columns based on which is more important.
However, you would probably be better served using Document/AiClassify, which would prompt the LLM like this:
Given the following clasifications:
- Mortgage
- Note
- Payoff Request
Classify the following text:
{your document text here}
This suggests to the LLM that it should pick the best match, rather than evaluating each question independently of the others.