Discoveries from Exploring: Innovation
Industry: State and Territory government administration
Dataset: Aus Innovation 2009-2016.csv
Explored on: 2024-03-21
Selected Goal Column: Highly Innovative
Help me understand this page
Overview
There is a Strong Signal to predict Highly Innovative - the best model on this data had +48.52

What is an F1 Score?

The F1 score is useful because it tells you how well you balance accuracy and completeness.

Imagine you are playing a game where you have to guess which animals are cats and which are not. You have a bunch of pictures of different animals, and you have to say "cat" or "not cat" for each one.

Now, there are two things you want to do well in this game:

  • You want to be accurate, which means you don't say "cat" when the animal is not a cat, and you don't say "not cat" when the animal is a cat.
  • You also want to be complete, which means you don't miss any cats that are in the pictures.
The F1 score is a way of measuring how well you do both of these things. It is usually a number between 0 and 1 (and in this case we multiply it by 100 to be more readable), where 100 means you are perfect, and 0 means you are very bad.

How is it calculated?

To calculate the F1 score, you need to know four things:
  • How many times you said "cat" and the animal was actually a cat. This is called a true positive (TP).
  • How many times you said "cat" and the animal was not a cat. This is called a false positive (FP).
  • How many times you said "not cat" and the animal was actually a cat. This is called a false negative (FN).
  • How many times you said "not cat" and the animal was not a cat. This is called a true negative (TN).

Then, you can use these numbers to calculate two other things:

  • Your precision, which is the fraction of times you were right when you said "cat". You can find it by dividing TP by TP + FP.
  • Your recall, which is the fraction of times you found all the cats in the pictures. You can find it by dividing TP by TP + FN.

Finally, you can use your precision and recall to calculate the F1 score. The F1 score is the harmonic mean of precision and recall, which means it is a kind of average that gives more weight to lower values. You can find it by multiplying precision and recall by 2, and then dividing by precision + recall.

Here is the formula for the F1 score:

If you only care about accuracy, you might say "not cat" for every picture, and never make a mistake. But then your recall would be zero, because you would miss all the cats. If you only care about completeness, you might say "cat" for every picture, and find all the cats. But then your precision would be very low, because you would also include many animals that are not cats. The F1 score helps you find a good middle ground, where you are both accurate and complete.

 compared to guessing.
Most Likely Yes:
 New business entries (per 10,000 residents)>120.4 and
Latitude<=-26.44 and
Agriculture, Forestry and Fishing (count of businesses)<=1672.0 and
Remoteness is 'Major Cities of Australia'
Most Likely No:
 New business entries (per 10,000 residents)<=98.4 and
Count of Research Institutes (2015)<=7.0
Explored 40 factors over 2640 records, in which 20.53% had Highly Innovative is Yes (542 of 2640).

Of these factors:
  • 7 were relevant for predicting Highly Innovative.
  • 33 were found to be irrelevant or redundant, so were ignored.
    • Key Factors

    We analyzed each factor (column) in the data, looking for which factors matter and which are irrelevant in determining Highly Innovative.

    Strongly Correlated Factors - In this dataset, we found some factors to be so strongly correlated with other factors that we could essentially swap them out and keep only one of them.

    Key Factors - The most important factors and their values more likely to result in 'Yes' for Highly Innovative are:
    Factor

    How do we determine a Single Value Group?

    For numeric factors, we calculate whether values above or below
    the midpoint are more likely to result in 'Yes' for Highly Innovative.
    This is not necessarily the optimal value for this factor, but is a
    good indication whether higher or lower values
    are preferred.

    For text factors, we show which value is most or least correlated
    with 'Yes' for Highly Innovative.

    		 Group Breakdown
    1
    New business entries (per 10,000 residents)
    New business entries (per 10,000 residents)>=110.15
    2
    Agriculture, Forestry and Fishing (count of businesses)
    Agriculture, Forestry and Fishing (count of businesses)<=212.0
    3
    Remoteness
    Remoteness is 'Major Cities of Australia'
    4
    SA3_Name
    SA3_Name is 'Pittwater'
    5
    Latitude
    Latitude<=-33.77
    6
    Count of Research Institutes (2015)

    Strongly Correlated With: Count of University Campuses (2015)
    Count of Research Institutes (2015)>=0.0
    7
    Top industry (by count of businesses)
    Top industry (by count of businesses) is 'Professional, Scientific and Technical Services'
    Irrelevant Factors - The following factors in your data were not very important in determining Highly Innovative:
    Factor
    8 State
    9 Count of TAFEs (2015)
    10 Construction (count of businesses)
    11 Health Care and Social Assistance (count of businesses)
    12 Longitude
    13 Education and Training (count of businesses)
    14 Accommodation and Food Services (count of businesses)
    15 Advanced Diploma and Diploma Level (total, 2016)
    16 Bachelor Degree Level (total, 2016)
    17 Manufacturing (count of businesses)
    18 Graduate Diploma and Graduate Certificate Level (total, 2016)
    19 Labour force (total)
    20 Wholesale Trade (count of businesses)
    21 Did not go to school (total, 2016)
    22 Estimated Resident Population
    23 Certificate Level (total, 2016)
    24 Financial and Insurance Services (count of businesses)
    25 New business entries
    26 Professional, Scientific and Technical Services (count of businesses)
    27 Information Media and Telecommunications (count of businesses)
    28 Public Administration and Safety (count of businesses)
    29 Administrative and Support Services (count of businesses)
    30 Year 12 or equivalent (total, 2016)
    31 Arts and Recreation Services (count of businesses)
    32 Rental, Hiring and Real Estate Services (count of businesses)
    33 Transport, Postal and Warehousing (count of businesses)
    34 Electricity, Gas, Water and Waste Services (count of businesses)
    35 Unemployed (total)
    36 Retail Trade (count of businesses)
    37 Mining (count of businesses)
    38 Business Research and Development Expenditure
    39 Postgraduate Degree Level (total, 2016)
    Expand to learn more about Factors

    Certain factors (or columns) are more important when determining 'Yes' or 'No' for your Goal than others. You can optimize your time and resources by prioritizing key factors across data gathering, data management, and decision-making.


    For example, if data gathering is time-consuming or expensive, you might consider de-prioritizing irrelevant factors. Just be mindful that they may be important for other problems.


    If you need help, set up a call with our Berrijam AI Coaches to brainstorm how you can adopt these insights from factors.

    Key Segments

    A combination of factors can be used to define a segment, giving deeper insights and making them easier to act upon.

    We have automatically identified top segments for 'Yes' and 'No', by determining their importance based on the size of the segment and % Yes or % No within the segment. Alternatively, you can explore the segments below.

    More Likely Yes
    Segment Segment Breakdown
    New business entries (per 10,000 residents)>120.4 and
    Latitude<=-26.44 and
    Agriculture, Forestry and Fishing (count of businesses)<=1672.0 and
    Remoteness is 'Major Cities of Australia'
    New business entries (per 10,000 residents)>120.4 and
    Latitude<=-26.44 and
    Agriculture, Forestry and Fishing (count of businesses)<=1672.0 and
    Remoteness is 'Outer Regional Australia'
    New business entries (per 10,000 residents)<=120.4 and
    Count of Research Institutes (2015)>7.0
    More Likely No
    Segment Segment Breakdown
    New business entries (per 10,000 residents)<=98.4 and
    Count of Research Institutes (2015)<=7.0
    New business entries (per 10,000 residents)>98.4, New business entries (per 10,000 residents)<=120.4 and
    Count of Research Institutes (2015)<=7.0 and
    Agriculture, Forestry and Fishing (count of businesses)>216.07
    New business entries (per 10,000 residents)>98.4, New business entries (per 10,000 residents)<=120.4 and
    Count of Research Institutes (2015)<=7.0 and
    Agriculture, Forestry and Fishing (count of businesses)<=216.07
    New business entries (per 10,000 residents)>120.4 and
    Latitude<=-26.44 and
    Agriculture, Forestry and Fishing (count of businesses)<=1672.0 and
    Remoteness is one of: [ 'Inner Regional Australia','Remote Australia','Very Remote Australia']
    New business entries (per 10,000 residents)>120.4 and
    Latitude>-26.44
    Explore More Segments
    Click on the rings to drill down into sub-segments and click the center circle to navigate back up.

    As you hover over a segment, you’ll see the factors that define the segment, the segment size as a percentage of all records, and the percentage of 'Yes' within that segment.
    Expand to learn more about Segments

    Segments help identify important groups or populations where to invest time and effort. For example, a teacher could use segments to focus on students who are least likely to pass the course. They also identify which combination of the factors and values to focus on if you want to increase or decrease the percentage of ‘Yes’.


    For example, a maintenance team might identify that machines operating above a certain temperature and load are more likely to break down. So they might set up temperature alerts and reduce the load at certain temperatures to keep the machine working.


    If you need help, set up a call with our Berrijam AI Coaches to brainstorm some ideas on how to adopt the insights from segments to design strategies or policies that are relevant to you.

    Model Results

    We explored 546 different models to find which produced the best results.

    Since there were no rows where Highly Innovative was empty, we did not make any predictions with these models.

    The Best Explainable Model performed very well on this dataset:

    +44.30

    What is an F1 Score?

    The F1 score is useful because it tells you how well you balance accuracy and completeness.

    Imagine you are playing a game where you have to guess which animals are cats and which are not. You have a bunch of pictures of different animals, and you have to say "cat" or "not cat" for each one.

    Now, there are two things you want to do well in this game:

    • You want to be accurate, which means you don't say "cat" when the animal is not a cat, and you don't say "not cat" when the animal is a cat.
    • You also want to be complete, which means you don't miss any cats that are in the pictures.
    The F1 score is a way of measuring how well you do both of these things. It is usually a number between 0 and 1 (and in this case we multiply it by 100 to be more readable), where 100 means you are perfect, and 0 means you are very bad.

    How is it calculated?

    To calculate the F1 score, you need to know four things:
    • How many times you said "cat" and the animal was actually a cat. This is called a true positive (TP).
    • How many times you said "cat" and the animal was not a cat. This is called a false positive (FP).
    • How many times you said "not cat" and the animal was actually a cat. This is called a false negative (FN).
    • How many times you said "not cat" and the animal was not a cat. This is called a true negative (TN).

    Then, you can use these numbers to calculate two other things:

    • Your precision, which is the fraction of times you were right when you said "cat". You can find it by dividing TP by TP + FP.
    • Your recall, which is the fraction of times you found all the cats in the pictures. You can find it by dividing TP by TP + FN.

    Finally, you can use your precision and recall to calculate the F1 score. The F1 score is the harmonic mean of precision and recall, which means it is a kind of average that gives more weight to lower values. You can find it by multiplying precision and recall by 2, and then dividing by precision + recall.

    Here is the formula for the F1 score:

    If you only care about accuracy, you might say "not cat" for every picture, and never make a mistake. But then your recall would be zero, because you would miss all the cats. If you only care about completeness, you might say "cat" for every picture, and find all the cats. But then your precision would be very low, because you would also include many animals that are not cats. The F1 score helps you find a good middle ground, where you are both accurate and complete.
    compared to guessing

    The Best Non-Explainable Model performed very well on this dataset:

    +48.52

    What is an F1 Score?

    The F1 score is useful because it tells you how well you balance accuracy and completeness.

    Imagine you are playing a game where you have to guess which animals are cats and which are not. You have a bunch of pictures of different animals, and you have to say "cat" or "not cat" for each one.

    Now, there are two things you want to do well in this game:

    • You want to be accurate, which means you don't say "cat" when the animal is not a cat, and you don't say "not cat" when the animal is a cat.
    • You also want to be complete, which means you don't miss any cats that are in the pictures.
    The F1 score is a way of measuring how well you do both of these things. It is usually a number between 0 and 1 (and in this case we multiply it by 100 to be more readable), where 100 means you are perfect, and 0 means you are very bad.

    How is it calculated?

    To calculate the F1 score, you need to know four things:
    • How many times you said "cat" and the animal was actually a cat. This is called a true positive (TP).
    • How many times you said "cat" and the animal was not a cat. This is called a false positive (FP).
    • How many times you said "not cat" and the animal was actually a cat. This is called a false negative (FN).
    • How many times you said "not cat" and the animal was not a cat. This is called a true negative (TN).

    Then, you can use these numbers to calculate two other things:

    • Your precision, which is the fraction of times you were right when you said "cat". You can find it by dividing TP by TP + FP.
    • Your recall, which is the fraction of times you found all the cats in the pictures. You can find it by dividing TP by TP + FN.

    Finally, you can use your precision and recall to calculate the F1 score. The F1 score is the harmonic mean of precision and recall, which means it is a kind of average that gives more weight to lower values. You can find it by multiplying precision and recall by 2, and then dividing by precision + recall.

    Here is the formula for the F1 score:

    If you only care about accuracy, you might say "not cat" for every picture, and never make a mistake. But then your recall would be zero, because you would miss all the cats. If you only care about completeness, you might say "cat" for every picture, and find all the cats. But then your precision would be very low, because you would also include many animals that are not cats. The F1 score helps you find a good middle ground, where you are both accurate and complete.
    compared to guessing
    All Model Results
    Credit

    Explored by avishkar@gmail.com using Aus Innovation 2009-2016.csv on 2024-03-21 with Berrijam. Try it yourself at Berrijam.com