Scores "Averaged" : How ? Geometrically or Arithmetically ??

jj.gud's Avatar

jj.gud

18 Aug, 2013 08:59 AM

I understand the Geekbench scores are derived from measured execution times for 26 (or 39) programs, referenced to a baseline made of a 2003 entry-level Power Mac G5.
What I could not find in the knowledge base is the way this "average" is exactly computed.

If there is any logic, and since all the 26 programs are unrelated, that should be the geometric mean of all 26 execution times ratios vs the 2003 entry-level Power Mac G5 baseline, giving a score of 1000. Only the geometric mean of those execution times ratios would guarantee that a 10% speedup for any of the 26 programs would give the same exact aggregated score improvement (0.4% assuming equal geometric weight and 26 programs).
But since I did not find the execution times of the baseline, I cannot confirm this geometric mean hypothesis.

Am I right (about the geometric mean) ?

PS : I attempted to recompute the "averages" in Excel from the Geekbench 2.2.7 scores I got.
The Integer Score, Floating Point Score, Memory Score and Stream Score all match an arithmetic average.
The aggregated Geekbench Score does not match either arithmetic or geometric averages (weighting ?).

  1. 1 Posted by Anthony on 19 Aug, 2013 02:56 PM

    Anthony's Avatar

    Hello,

    You are correct that in Geekbench 2 each section score is computed by arithmetic mean of the workload scores in that section. The Geekbench score is a then a weighted arithmetic mean of the section scores with weights:
    *integer: 0.35 *floating point: 0.35 *memory: 0.2 *stream: 0.1

    In Geekbench 3, the baseline is 2500 (so a score of 2500 means identical performacne to the baseleine) and the section scores are computed by geometric mean. The Geekbench score is a weighted arithmetic mean of the section scores using the weights:
    *integer: 0.4 *floating point: 0.4 *memory: 0.2

    Sincerely,

    Tony

  2. 2 Posted by jj.gud on 19 Aug, 2013 04:04 PM

    jj.gud's Avatar

    Hi Tony,

    Thanks for the quick answer. This final weighting does explain all my discrepancies.

    I am pleased you moved to geometric mean for Geekbench 3, since it has much better properties when you have to "add" carrots and turtles.
    However I am not sure why you did not also computed the aggregated Geekbench 3 Score using also a weighted geometric mean, which would have kept the nice property that any 10% increase of the Integer Score would give the exact same aggregated Geekbench 3 Score increase as a 10% increase of the Floating Point Score, or a 21% increase of the Memory Score (=1,1^2-1).
    The formula would be very simple (using Excel notation with ^ for exponent) :
    Geekbench_3_Score = (Integer_Score^2 * Floating_Point^2 * Memory_Score)^(1/5)
    = (Integer_Score^4 * Floating_Point^4 * Memory_Score^2)^(1/10)

    Mixing Geometric and Arithmetic means is usually a bad idea since at the end you get the good properties of neither.
    Since you have just introduced Geekbench 3 and have no massive database of Scores yet, may be you could still go all the way geometric ? :-)

    Regards
    JJ

  3. Support Staff 3 Posted by John on 20 Aug, 2013 04:42 AM

    John's Avatar

    Hi JJ,

    Thank you for your feedback. While moving "all geometric" does have some nice properties, we're reluctant to change something so fundamental even when we only have 15,000 results in the Geekbench Browser. It's certainly something we will consider doing for Geekbench 4, though.

    Thanks again,
    John

  4. 4 Posted by jj.gud on 20 Aug, 2013 05:00 AM

    jj.gud's Avatar

    Hi John,

    I was not aware you already had 15 000 results with Geekbench 3.
    You definitely don't want to throw them away.
    Note however that since you archive the detailed results for Integer / Floating Point and Memory Scores for each of those measurements, it would be quite easy to recompute the aggregated score in a geometrical way for all 15 000 of them.
    At worst, it would modify the top 10.

    Keep the good work
    JJ

  5. Support Staff 5 Posted by John on 20 Aug, 2013 05:06 AM

    John's Avatar

    Hi JJ,

    I thought about recalculating the scores in the Browser, but I'm concerned that will confuse users (e.g., "Why did my scores change?"). Plus any scores that are stored "offline" will not be affected by the recomputing.

    Again, I think it's too late for v3, but it's something we will consider for v4.

    Thanks,
    John

  6. 6 Posted by Fabián Frommel on 28 Jul, 2017 12:54 PM

    Fabián Frommel's Avatar

    Hello!

    I would like to know the weights of different categories in order to calculates Geekbench score from categories scores.

    Thank you. Regards.

  7. Support Staff 7 Posted by Colin on 24 Aug, 2017 05:15 PM

    Colin's Avatar

    Hi,

    I apologize for the delayed response. Geekbench 4 uses the following weights to compute the weighted arithmetic mean of subsection scores:.

    Cryptography: 5%
    Integer: 45%
    Floating Point: 30%
    Memory: 20%

    These values are listed in the Geekbench 4 CPU Workloads document on our website here.

    Let me know if you have any other questions and I would be happy to help out.

    Regards,
    Colin
    Primate Labs Inc.

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac