Benchmark Run Details

System Prompt

You are a linguistic expert specializing in phonetics. 
Your task is to provide the IPA (International Phonetic Alphabet) pronunciation for English words.
Use American English pronunciation as your default standard.
Provide only the IPA transcription with no additional text or explanation.
Include stress markers and all appropriate IPA symbols.

Run Summary

Model llama3.2:3b:Q4_K_M
Benchmark 0061_english_to_ipa
Normed Score 12
Run Timestamp 2025-04-04 16:34:10

Question-Level Details

Question ID Score Evaluation Time (ms) Debug Info
0061_english_to_ipa:0 0 479 { "response": "/rɛd/", "expected": "ɹid", "is_correct": false }
[+]
0061_english_to_ipa:1 0 453 { "response": "/riːd/", "expected": "ɹɛd", "is_correct": false }
[+]
0061_english_to_ipa:10 0 727 { "response": "/ˈθruː/ˈθruːə/", "expected": "θɹu", "is_correct": false }
[+]
0061_english_to_ipa:11 100 425 { "response": "/naɪt/", "expected": "naɪt", "is_correct": true }
[+]
0061_english_to_ipa:12 0 701 { "response": "/psɪˈkɒlədʒi/", "expected": "saɪˈkɑlədʒi", "is_correct": false }
[+]
0061_english_to_ipa:13 0 519 { "response": "/kər\nənɛl/", "expected": "ˈkɜɹnəl", "is_correct": false }
[+]
0061_english_to_ipa:14 0 524 { "response": "/sʌb.təl/", "expected": "ˈsʌtl̩", "is_correct": false }
[+]
0061_english_to_ipa:15 0 679 { "response": "/ɛpɪˈtoʊmi/", "expected": "əˈpɪtəmi", "is_correct": false }
[+]
0061_english_to_ipa:16 0 495 { "response": "/ˈkoʊz/", "expected": "ˈkeɪɑs", "is_correct": false }
[+]
0061_english_to_ipa:17 0 425 { "response": "/kjuː/", "expected": "kju", "is_correct": false }
[+]
0061_english_to_ipa:18 0 453 { "response": "/ˈkeɪ/", "expected": "ki", "is_correct": false }
[+]
0061_english_to_ipa:19 0 487 { "response": "/aɪlənd/", "expected": "ˈaɪlənd", "is_correct": false }
[+]
0061_english_to_ipa:2 0 577 { "response": "/maɪˈnuːt/", "expected": "ˈmɪnɪt", "is_correct": false }
[+]
0061_english_to_ipa:20 0 538 { "response": "/skedʒuːl/", "expected": "ˈskɛdʒul", "is_correct": false }
[+]
0061_english_to_ipa:21 0 567 { "response": "/lɛzˈjuər/", "expected": "ˈliʒɚ", "is_correct": false }
[+]
0061_english_to_ipa:22 100 425 { "response": "/dɛt/", "expected": "dɛt", "is_correct": true }
[+]
0061_english_to_ipa:23 0 517 { "response": "/sælˈmɒn/", "expected": "ˈsæmən", "is_correct": false }
[+]
0061_english_to_ipa:24 0 329 { "response": "/ajt/", "expected": "jɑt", "is_correct": false }
[+]
0061_english_to_ipa:25 0 461 { "response": "/ɔɪlz/", "expected": "aɪl", "is_correct": false }
[+]
0061_english_to_ipa:26 0 435 { "response": "/hɛr/", "expected": "ɛɹ", "is_correct": false }
[+]
0061_english_to_ipa:27 0 521 { "response": "/ˈɒnəst/", "expected": "ˈɑnɪst", "is_correct": false }
[+]
0061_english_to_ipa:28 0 425 { "response": "/ɡnoʊm/", "expected": "noʊm", "is_correct": false }
[+]
0061_english_to_ipa:29 0 620 { "response": "/njuːˈmɔniə/", "expected": "nuˈmoʊniə", "is_correct": false }
[+]
0061_english_to_ipa:3 0 579 { "response": "/maɪˈnuːt/", "expected": "maɪˈnut", "is_correct": false }
[+]
0061_english_to_ipa:30 0 706 { "response": "/ˈzɪloʊfoʊn/", "expected": "ˈzaɪləfoʊn", "is_correct": false }
[+]
0061_english_to_ipa:31 0 542 { "response": "/ˈflɛɡm/", "expected": "flɛm", "is_correct": false }
[+]
0061_english_to_ipa:32 0 460 { "response": "/ˈɛðər/", "expected": "ˈiðɚ", "is_correct": false }
[+]
0061_english_to_ipa:33 0 619 { "response": "/kærˈɑmɛl/", "expected": "ˈkɛɹəməl", "is_correct": false }
[+]
0061_english_to_ipa:34 0 437 { "response": "/dætə/", "expected": "ˈdeɪtə", "is_correct": false }
[+]
0061_english_to_ipa:35 0 447 { "response": "/raʊt/", "expected": "ɹut", "is_correct": false }
[+]
0061_english_to_ipa:36 0 458 { "response": "/ɔfən/", "expected": "ˈɔfən", "is_correct": false }
[+]
0061_english_to_ipa:37 0 566 { "response": "/ˈsɜrʌp/", "expected": "ˈsɪɹəp", "is_correct": false }
[+]
0061_english_to_ipa:38 0 636 { "response": "/praɪˈvæsɪti/", "expected": "ˈpɹaɪvəsi", "is_correct": false }
[+]
0061_english_to_ipa:39 0 695 { "response": "/dʒɪnˈdʒɾə/", "expected": "ˈʒɑnɹə", "is_correct": false }
[+]
0061_english_to_ipa:4 100 447 { "response": "/beɪs/", "expected": "beɪs", "is_correct": true }
[+]
0061_english_to_ipa:5 100 439 { "response": "/bæs/", "expected": "bæs", "is_correct": true }
[+]
0061_english_to_ipa:6 100 426 { "response": "/wɪnd/", "expected": "wɪnd", "is_correct": true }
[+]
0061_english_to_ipa:7 0 420 { "response": "/wɪnd/", "expected": "waɪnd", "is_correct": false }
[+]
0061_english_to_ipa:8 0 445 { "response": "/tɛr/", "expected": "tɪɹ", "is_correct": false }
[+]
0061_english_to_ipa:9 0 424 { "response": "/tɪr/", "expected": "tɛɹ", "is_correct": false }
[+]