Skip to content

Latest commit

 

History

History
201 lines (198 loc) · 5.87 KB

78.md

File metadata and controls

201 lines (198 loc) · 5.87 KB

Results of GPT-4 on HumanEval/78

Date: 18 July, 2023
Model: gpt-4
Temperature: 0.2
Passing runs: 10
Total runs: 10
Success rate: 100%

Prompt

Tokens: 325

def hex_key(num):
    """You have been tasked to write a function that receives 
    a hexadecimal number as a string and counts the number of hexadecimal 
    digits that are primes (prime number, or a prime, is a natural number 
    greater than 1 that is not a product of two smaller natural numbers).
    Hexadecimal digits are 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, F.
    Prime numbers are 2, 3, 5, 7, 11, 13, 17,...
    So you have to determine a number of the following digits: 2, 3, 5, 7, 
    B (=decimal 11), D (=decimal 13).
    Note: you may assume the input is always correct or empty string, 
    and symbols A,B,C,D,E,F are always uppercase.
    Examples:
    For num = "AB" the output should be 1.
    For num = "1077E" the output should be 2.
    For num = "ABED1A33" the output should be 4.
    For num = "123456789ABCDEF0" the output should be 6.
    For num = "2020" the output should be 2.
    """

The above is an incomplete Python code fragment. Return the complete and correct code with no additional text.

Correct solution

def hex_key(num):
    """You have been tasked to write a function that receives 
    a hexadecimal number as a string and counts the number of hexadecimal 
    digits that are primes (prime number, or a prime, is a natural number 
    greater than 1 that is not a product of two smaller natural numbers).
    Hexadecimal digits are 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E, F.
    Prime numbers are 2, 3, 5, 7, 11, 13, 17,...
    So you have to determine a number of the following digits: 2, 3, 5, 7, 
    B (=decimal 11), D (=decimal 13).
    Note: you may assume the input is always correct or empty string, 
    and symbols A,B,C,D,E,F are always uppercase.
    Examples:
    For num = "AB" the output should be 1.
    For num = "1077E" the output should be 2.
    For num = "ABED1A33" the output should be 4.
    For num = "123456789ABCDEF0" the output should be 6.
    For num = "2020" the output should be 2.
    """primes = ('2', '3', '5', '7', 'B', 'D')
    total = 0
    for i in range(0, len(num)):
        if num[i] in primes:
            total += 1
    return total

Unit tests

def check(candidate):

    # Check some simple cases
    assert candidate("AB") == 1, "First test error: " + str(candidate("AB"))      
    assert candidate("1077E") == 2, "Second test error: " + str(candidate("1077E"))  
    assert candidate("ABED1A33") == 4, "Third test error: " + str(candidate("ABED1A33"))      
    assert candidate("2020") == 2, "Fourth test error: " + str(candidate("2020"))  
    assert candidate("123456789ABCDEF0") == 6, "Fifth test error: " + str(candidate("123456789ABCDEF0"))      
    assert candidate("112233445566778899AABBCCDDEEFF00") == 12, "Sixth test error: " + str(candidate("112233445566778899AABBCCDDEEFF00"))  


    # Check some edge cases that are easy to work out by hand.
    assert candidate([]) == 0

Runs

✅ Run 1

Start time: 2023-07-18 18:24:03
Duration: 5.78s
Completion tokens: 58
Result: Pass

def hex_key(num):
    primes = ['2', '3', '5', '7', 'B', 'D']
    count = 0
    for digit in num:
        if digit in primes:
            count += 1
    return count

✅ Run 2

Start time: 2023-07-18 18:23:58
Duration: 4.47s
Completion tokens: 60
Result: Pass

def hex_key(num):
    prime_hex = ['2', '3', '5', '7', 'B', 'D']
    count = 0
    for digit in num:
        if digit in prime_hex:
            count += 1
    return count

✅ Run 3

Start time: 2023-07-18 18:23:53
Duration: 4.97s
Completion tokens: 60
Result: Pass

def hex_key(num):
    prime_hex = ['2', '3', '5', '7', 'B', 'D']
    count = 0
    for digit in num:
        if digit in prime_hex:
            count += 1
    return count

✅ Run 4

Start time: 2023-07-18 18:23:49
Duration: 4.01s
Completion tokens: 44
Result: Pass

def hex_key(num):
    primes = ['2', '3', '5', '7', 'B', 'D']
    return sum(1 for digit in num if digit in primes)

✅ Run 5

Start time: 2023-07-18 18:23:43
Duration: 5.6s
Completion tokens: 62
Result: Pass

def hex_key(num):
    prime_hex_digits = ['2', '3', '5', '7', 'B', 'D']
    count = 0
    for digit in num:
        if digit in prime_hex_digits:
            count += 1
    return count

✅ Run 6

Start time: 2023-07-18 18:23:39
Duration: 4.49s
Completion tokens: 58
Result: Pass

def hex_key(num):
    primes = ['2', '3', '5', '7', 'B', 'D']
    count = 0
    for digit in num:
        if digit in primes:
            count += 1
    return count

✅ Run 7

Start time: 2023-07-18 18:23:34
Duration: 4.59s
Completion tokens: 46
Result: Pass

def hex_key(num):
    prime_hex = ['2', '3', '5', '7', 'B', 'D']
    return sum(1 for digit in num if digit in prime_hex)

✅ Run 8

Start time: 2023-07-18 18:23:30
Duration: 4.57s
Completion tokens: 46
Result: Pass

def hex_key(num):
    prime_hex = ['2', '3', '5', '7', 'B', 'D']
    return sum(1 for digit in num if digit in prime_hex)

✅ Run 9

Start time: 2023-07-18 18:23:25
Duration: 4.47s
Completion tokens: 60
Result: Pass

def hex_key(num):
    prime_hex = ['2', '3', '5', '7', 'B', 'D']
    count = 0
    for digit in num:
        if digit in prime_hex:
            count += 1
    return count

✅ Run 10

Start time: 2023-07-18 18:23:18
Duration: 6.54s
Completion tokens: 62
Result: Pass

def hex_key(num):
    prime_hex_digits = ['2', '3', '5', '7', 'B', 'D']
    count = 0
    for digit in num:
        if digit in prime_hex_digits:
            count += 1
    return count