Verify if a statement about the text is true or not

R/llm-verify.R

llm_verify

Description

Use a Large Language Model (LLM) to see if something is true or not based the provided text

Usage

 
llm_verify( 
  .data, 
  col, 
  what, 
  yes_no = factor(c(1, 0)), 
  pred_name = ".verify", 
  additional_prompt = "" 
) 
 
llm_vec_verify( 
  x, 
  what, 
  yes_no = factor(c(1, 0)), 
  additional_prompt = "", 
  preview = FALSE 
) 

Arguments

Arguments Description
.data A data.frame or tbl object that contains the text to be analyzed
col The name of the field to analyze, supports tidy-eval
what The statement or question that needs to be verified against the provided text
yes_no A size 2 vector that specifies the expected output. It is positional. The first item is expected to be value to return if the statement about the provided text is true, and the second if it is not. Defaults to: factor(c(1, 0))
pred_name A character vector with the name of the new column where the prediction will be placed
additional_prompt Inserts this text into the prompt sent to the LLM
x A vector that contains the text to be analyzed
preview It returns the R call that would have been used to run the prediction. It only returns the first record in x. Defaults to FALSE Applies to vector function only.

Value

llm_verify returns a data.frame or tbl object. llm_vec_verify returns a vector that is the same length as x.

Examples

 
 
library(mall) 
 
data("reviews") 
 
llm_use("ollama", "llama3.2", seed = 100, .silent = TRUE) 
 
# By default it will return 1 for 'true', and 0 for 'false', 
# the new column will be a factor type 
llm_verify(reviews, review, "is the customer happy") 
#> # A tibble: 3 × 2
#>   review                                                                 .verify
#>   <chr>                                                                  <fct>  
#> 1 This has been the best TV I've ever used. Great screen, and sound.     1      
#> 2 I regret buying this laptop. It is too slow and the keyboard is too n… 0      
#> 3 Not sure how to feel about my new washing machine. Great color, but h… 0
 
# The yes_no argument can be modified to return a different response 
# than 1 or 0. First position will be 'true' and second, 'false' 
llm_verify(reviews, review, "is the customer happy", c("y", "n")) 
#> # A tibble: 3 × 2
#>   review                                                                 .verify
#>   <chr>                                                                  <chr>  
#> 1 This has been the best TV I've ever used. Great screen, and sound.     y      
#> 2 I regret buying this laptop. It is too slow and the keyboard is too n… n      
#> 3 Not sure how to feel about my new washing machine. Great color, but h… n
 
# Number can also be used, this would be in the case that you wish to match 
# the output values of existing predictions 
llm_verify(reviews, review, "is the customer happy", c(2, 1)) 
#> # A tibble: 3 × 2
#>   review                                                                 .verify
#>   <chr>                                                                    <dbl>
#> 1 This has been the best TV I've ever used. Great screen, and sound.           2
#> 2 I regret buying this laptop. It is too slow and the keyboard is too n…       1
#> 3 Not sure how to feel about my new washing machine. Great color, but h…       1