Gemma 2 vs Llama 3

We will compare Gemma 2 and Llama 3 in three categories: Reasoning, Math and Code. For each category we will ask three questions and compare the answers.

We will compare Gemma 2 and Llama 3 in three categories: Reasoning, Math and Code. For each category we will ask three questions and compare the answers.

Reasoning Question #1

  Question: This thing all things devour; birds, beasts, trees, flowers; gnaws iron, bites steel; grinds hard stones to meal; slays king, ruins town, and beats mountain down. 

  Correct answer: Both Gemma 2 and Lamma 3

Reasoning Question #2

  Question: If you break me, I’ll not stop working. If you can touch me, my work is done. If you lose me, you must find me with a ring soon after. What am I? 

  Correct answer: None

Reasoning Question #3

  Question: There are 9 birds in the tree, the hunter shoots one, how many birds are left in the tree?

  Correct answer: Gemma 2

Math Question #1

  Question: Two cyclists start 30 miles apart, and begin cycling towards each other at 15 mph. The instant they begin, a fly goes from one cyclist to the other, then back to the first cyclist, and so on, always flying between the cyclists. The fly flies at a constant 30 mph. How far in total does the fly travel when the cyclists meet?

  Correct answer: Gemma 2

Math Question #2

  Question: A salesman sold twice as much pears in the afternoon than in the morning. If he sold 360 kilograms of pears that day, how many kilograms did he sell in the morning and how many in the afternoon?

  Correct answer: Both Gemma 2 and Lamma 3

Math Question #3

  Question: Mary, Peter, and Lucy were picking chestnuts. Mary picked twice as much chestnuts than Peter. Lucy picked 2 kg more than Peter. Together the three of them picked 26 kg of chestnuts. How many kilograms did each of them pick?

  Correct answer: Both Gemma 2 and Lamma 3

Code Question #1

  Question: Leetcode - 10. Regular Expression Matching

  Correct answer: Gemma 2

Code Question #2

  Question: Leetcode - 149. Max Points on a Line

  Correct answer: Gemma 2

Code Question #3

  Question: Leetcode - 403. Frog Jump

  Correct answer: None

Results

Below are the results for each model on each question

If you liked this blog, subscribe to my newsletter!