Skip to content

Latest commit

 

History

History
241 lines (224 loc) · 8.16 KB

101.md

File metadata and controls

241 lines (224 loc) · 8.16 KB

Results of GPT-4 on HumanEval/101

Date: 18 July, 2023
Model: gpt-4
Temperature: 0.2
Passing runs: 10
Total runs: 10
Success rate: 100%

Prompt

Tokens: 142

def words_string(s):
    """
    You will be given a string of words separated by commas or spaces. Your task is
    to split the string into words and return an array of the words.
    
    For example:
    words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    """

The above is an incomplete Python code fragment. Return the complete and correct code with no additional text.

Correct solution

def words_string(s):
    """
    You will be given a string of words separated by commas or spaces. Your task is
    to split the string into words and return an array of the words.
    
    For example:
    words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    """if not s:
        return []

    s_list = []

    for letter in s:
        if letter == ',':
            s_list.append(' ')
        else:
            s_list.append(letter)

    s_list = "".join(s_list)
    return s_list.split()

Unit tests

def check(candidate):

    # Check some simple cases
    assert True, "This prints if this assert fails 1 (good for debugging!)"
    assert candidate("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    assert candidate("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    assert candidate("Hi, my name") == ["Hi", "my", "name"]
    assert candidate("One,, two, three, four, five, six,") == ["One", "two", "three", "four", "five", "six"]

    # Check some edge cases that are easy to work out by hand.
    assert True, "This prints if this assert fails 2 (also good for debugging!)"
    assert candidate("") == []
    assert candidate("ahmed     , gamal") == ["ahmed", "gamal"]

Runs

✅ Run 1

Start time: 2023-07-18 18:15:04
Duration: 10.3s
Completion tokens: 130
Result: Pass

def words_string(s):
    """
    You will be given a string of words separated by commas or spaces. Your task is
    to split the string into words and return an array of the words.
    
    For example:
    words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    """
    return [word.strip() for word in s.replace(',', ' ').split()]

✅ Run 2

Start time: 2023-07-18 18:14:55
Duration: 9.88s
Completion tokens: 140
Result: Pass

def words_string(s):
    """
    You will be given a string of words separated by commas or spaces. Your task is
    to split the string into words and return an array of the words.
    
    For example:
    words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    """
    s = s.replace(',', '') # remove commas
    return s.split() # split string into words and return as list

✅ Run 3

Start time: 2023-07-18 18:14:42
Duration: 12.32s
Completion tokens: 126
Result: Pass

def words_string(s):
    """
    You will be given a string of words separated by commas or spaces. Your task is
    to split the string into words and return an array of the words.
    
    For example:
    words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    """
    s = s.replace(",", "")
    return s.split()

✅ Run 4

Start time: 2023-07-18 18:14:32
Duration: 9.93s
Completion tokens: 140
Result: Pass

def words_string(s):
    """
    You will be given a string of words separated by commas or spaces. Your task is
    to split the string into words and return an array of the words.
    
    For example:
    words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    """
    s = s.replace(',', '') # remove commas
    return s.split() # split string into words and return as list

✅ Run 5

Start time: 2023-07-18 18:14:18
Duration: 14.1s
Completion tokens: 142
Result: Pass

def words_string(s):
    """
    You will be given a string of words separated by commas or spaces. Your task is
    to split the string into words and return an array of the words.
    
    For example:
    words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    """
    s = s.replace(',', '')  # remove commas
    return s.split()  # split string into words and return as list

✅ Run 6

Start time: 2023-07-18 18:14:05
Duration: 12.96s
Completion tokens: 140
Result: Pass

def words_string(s):
    """
    You will be given a string of words separated by commas or spaces. Your task is
    to split the string into words and return an array of the words.
    
    For example:
    words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    """
    s = s.replace(',', '') # remove commas
    return s.split() # split string into words and return as list

✅ Run 7

Start time: 2023-07-18 18:13:53
Duration: 11.56s
Completion tokens: 142
Result: Pass

def words_string(s):
    """
    You will be given a string of words separated by commas or spaces. Your task is
    to split the string into words and return an array of the words.
    
    For example:
    words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    """
    s = s.replace(',', '') # remove commas
    return s.split() # split the string into words and return as a list

✅ Run 8

Start time: 2023-07-18 18:13:41
Duration: 11.94s
Completion tokens: 140
Result: Pass

def words_string(s):
    """
    You will be given a string of words separated by commas or spaces. Your task is
    to split the string into words and return an array of the words.
    
    For example:
    words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    """
    s = s.replace(',', '') # remove commas
    return s.split() # split string into words and return as list

✅ Run 9

Start time: 2023-07-18 18:13:30
Duration: 10.73s
Completion tokens: 140
Result: Pass

def words_string(s):
    """
    You will be given a string of words separated by commas or spaces. Your task is
    to split the string into words and return an array of the words.
    
    For example:
    words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    """
    s = s.replace(',', '') # remove commas
    return s.split() # split string into words and return as list

✅ Run 10

Start time: 2023-07-18 18:13:13
Duration: 17.19s
Completion tokens: 140
Result: Pass

def words_string(s):
    """
    You will be given a string of words separated by commas or spaces. Your task is
    to split the string into words and return an array of the words.
    
    For example:
    words_string("Hi, my name is John") == ["Hi", "my", "name", "is", "John"]
    words_string("One, two, three, four, five, six") == ["One", "two", "three", "four", "five", "six"]
    """
    s = s.replace(',', '') # remove commas
    return s.split() # split string into words and return as list