Discoveries from Exploring: Customer Churn Prediction
Industry: Telecommunications services
Dataset: Telecom - Churn Predict.csv
Explored on: 2024-03-15
Selected Goal Column: Churn
Help me understand this page
Overview
There is a Strong Signal to predict Churn - the best model on this data had +34.72

What is an F1 Score?

The F1 score is useful because it tells you how well you balance accuracy and completeness.

Imagine you are playing a game where you have to guess which animals are cats and which are not. You have a bunch of pictures of different animals, and you have to say "cat" or "not cat" for each one.

Now, there are two things you want to do well in this game:

  • You want to be accurate, which means you don't say "cat" when the animal is not a cat, and you don't say "not cat" when the animal is a cat.
  • You also want to be complete, which means you don't miss any cats that are in the pictures.
The F1 score is a way of measuring how well you do both of these things. It is usually a number between 0 and 1 (and in this case we multiply it by 100 to be more readable), where 100 means you are perfect, and 0 means you are very bad.

How is it calculated?

To calculate the F1 score, you need to know four things:
  • How many times you said "cat" and the animal was actually a cat. This is called a true positive (TP).
  • How many times you said "cat" and the animal was not a cat. This is called a false positive (FP).
  • How many times you said "not cat" and the animal was actually a cat. This is called a false negative (FN).
  • How many times you said "not cat" and the animal was not a cat. This is called a true negative (TN).

Then, you can use these numbers to calculate two other things:

  • Your precision, which is the fraction of times you were right when you said "cat". You can find it by dividing TP by TP + FP.
  • Your recall, which is the fraction of times you found all the cats in the pictures. You can find it by dividing TP by TP + FN.

Finally, you can use your precision and recall to calculate the F1 score. The F1 score is the harmonic mean of precision and recall, which means it is a kind of average that gives more weight to lower values. You can find it by multiplying precision and recall by 2, and then dividing by precision + recall.

Here is the formula for the F1 score:

If you only care about accuracy, you might say "not cat" for every picture, and never make a mistake. But then your recall would be zero, because you would miss all the cats. If you only care about completeness, you might say "cat" for every picture, and find all the cats. But then your precision would be very low, because you would also include many animals that are not cats. The F1 score helps you find a good middle ground, where you are both accurate and complete.

 compared to guessing.
Most Likely Yes:
 Contract is 'Month-to-Month' and
Number of Referrals<=2.0 and
Online Security is 'No' and
Premium Tech Support is 'No'
Most Likely No:
 Contract is one of: ['One Year', 'Two Year']
Explored 37 factors over 6043 records, in which 26.77% had Churn is Yes (1618 of 6043).

Of these factors:
  • 1 was strongly correlated with Churn, so was excluded.
  • 10 were relevant for predicting Churn.
  • 26 were found to be irrelevant or redundant, so were ignored.

    • Also generated predictions for the 1000 rows where the Churn column was empty.
    Key Factors

    We analyzed each factor (column) in the data, looking for which factors matter and which are irrelevant in determining Churn.

    Strongly Correlated Factors - In this dataset, we found some factors to be so strongly correlated with other factors that we could essentially swap them out and keep only one of them.

    The Churn column was Strongly Correlated With: Customer Status, so this factor was excluded.

    Key Factors - The most important factors and their values more likely to result in 'Yes' for Churn are:
    Factor

    How do we determine a Single Value Group?

    For numeric factors, we calculate whether values above or below
    the midpoint are more likely to result in 'Yes' for Churn.
    This is not necessarily the optimal value for this factor, but is a
    good indication whether higher or lower values
    are preferred.

    For text factors, we show which value is most or least correlated
    with 'Yes' for Churn.

    		 Group Breakdown
    1
    Contract
    Contract is 'Month-to-Month'
    2
    Number of Referrals
    Number of Referrals<=0.0
    3
    Online Security
    Online Security is 'No'
    4
    Internet Type
    Internet Type is 'Fiber Optic'
    5
    Premium Tech Support
    Premium Tech Support is 'No'
    6
    Number of Dependents
    Number of Dependents<=0.0
    7
    Married
    Married is 'No'
    8
    Age
    Age>=46.0
    9
    Payment Method
    Payment Method is not 'Credit Card'
    10
    Tenure in Months

    Strongly Correlated With:
    - Total Charges
    - Total Long Distance Charges
    - Total Revenue
    Tenure in Months<=29.0
    Irrelevant Factors - The following factors in your data were not very important in determining Churn:
    Factor
    11 Latitude

    Strongly Correlated With:
    - Longitude
    - Zip Code
    12 Streaming Music

    Strongly Correlated With: Streaming Movies
    13 Online Backup
    14 Total Refunds
    15 Streaming TV
    16 Population
    17 Paperless Billing
    18 Avg Monthly GB Download
    19 Multiple Lines
    20 Gender
    21 Device Protection Plan
    22 City
    23 Avg Monthly Long Distance Charges
    24 Offer
    25 Unlimited Data
    26 Total Extra Data Charges
    27 Internet Service
    28 Monthly Charge
    29 Customer ID
    30 Phone Service
    Expand to learn more about Factors

    Certain factors (or columns) are more important when determining 'Yes' or 'No' for your Goal than others. You can optimize your time and resources by prioritizing key factors across data gathering, data management, and decision-making.


    For example, if data gathering is time-consuming or expensive, you might consider de-prioritizing irrelevant factors. Just be mindful that they may be important for other problems.


    If you need help, set up a call with our Berrijam AI Coaches to brainstorm how you can adopt these insights from factors.

    Key Segments

    A combination of factors can be used to define a segment, giving deeper insights and making them easier to act upon.

    We have automatically identified top segments for 'Yes' and 'No', by determining their importance based on the size of the segment and % Yes or % No within the segment. Alternatively, you can explore the segments below.

    More Likely Yes
    Segment Segment Breakdown
    Contract is 'Month-to-Month' and
    Number of Referrals<=2.0 and
    Online Security is 'No' and
    Premium Tech Support is 'No'
    Contract is 'Month-to-Month' and
    Number of Referrals<=2.0 and
    Online Security is 'Yes' and
    Age<=64.0
    Contract is 'Month-to-Month' and
    Number of Referrals<=2.0 and
    Online Security is 'No' and
    Premium Tech Support is 'Yes'
    Contract is 'Month-to-Month' and
    Number of Referrals<=2.0 and
    Online Security is 'Yes' and
    Age>64.0
    More Likely No
    Segment Segment Breakdown
    Contract is one of: ['One Year', 'Two Year']
    Contract is 'Month-to-Month' and
    Number of Referrals<=2.0 and
    Online Security is 'Blank'
    Contract is 'Month-to-Month' and
    Number of Referrals>2.0 and
    Married is 'Yes' and
    Number of Dependents<=0.0
    Contract is 'Month-to-Month' and
    Number of Referrals>2.0 and
    Married is 'Yes' and
    Number of Dependents>0.0
    Explore More Segments
    Click on the rings to drill down into sub-segments and click the center circle to navigate back up.

    As you hover over a segment, you’ll see the factors that define the segment, the segment size as a percentage of all records, and the percentage of 'Yes' within that segment.
    Expand to learn more about Segments

    Segments help identify important groups or populations where to invest time and effort. For example, a teacher could use segments to focus on students who are least likely to pass the course. They also identify which combination of the factors and values to focus on if you want to increase or decrease the percentage of ‘Yes’.


    For example, a maintenance team might identify that machines operating above a certain temperature and load are more likely to break down. So they might set up temperature alerts and reduce the load at certain temperatures to keep the machine working.


    If you need help, set up a call with our Berrijam AI Coaches to brainstorm some ideas on how to adopt the insights from segments to design strategies or policies that are relevant to you.

    Model Results

    We explored 626 different models to find which produced the best results.

    For the rows without a value for Churn, we generated predictions using the best models.
    We will auto-delete these on 2024-03-22 so please download them before then, otherwise you will need to re-run the exploration.

    The Best Explainable Model performed very well on this dataset:

    +30.04

    What is an F1 Score?

    The F1 score is useful because it tells you how well you balance accuracy and completeness.

    Imagine you are playing a game where you have to guess which animals are cats and which are not. You have a bunch of pictures of different animals, and you have to say "cat" or "not cat" for each one.

    Now, there are two things you want to do well in this game:

    • You want to be accurate, which means you don't say "cat" when the animal is not a cat, and you don't say "not cat" when the animal is a cat.
    • You also want to be complete, which means you don't miss any cats that are in the pictures.
    The F1 score is a way of measuring how well you do both of these things. It is usually a number between 0 and 1 (and in this case we multiply it by 100 to be more readable), where 100 means you are perfect, and 0 means you are very bad.

    How is it calculated?

    To calculate the F1 score, you need to know four things:
    • How many times you said "cat" and the animal was actually a cat. This is called a true positive (TP).
    • How many times you said "cat" and the animal was not a cat. This is called a false positive (FP).
    • How many times you said "not cat" and the animal was actually a cat. This is called a false negative (FN).
    • How many times you said "not cat" and the animal was not a cat. This is called a true negative (TN).

    Then, you can use these numbers to calculate two other things:

    • Your precision, which is the fraction of times you were right when you said "cat". You can find it by dividing TP by TP + FP.
    • Your recall, which is the fraction of times you found all the cats in the pictures. You can find it by dividing TP by TP + FN.

    Finally, you can use your precision and recall to calculate the F1 score. The F1 score is the harmonic mean of precision and recall, which means it is a kind of average that gives more weight to lower values. You can find it by multiplying precision and recall by 2, and then dividing by precision + recall.

    Here is the formula for the F1 score:

    If you only care about accuracy, you might say "not cat" for every picture, and never make a mistake. But then your recall would be zero, because you would miss all the cats. If you only care about completeness, you might say "cat" for every picture, and find all the cats. But then your precision would be very low, because you would also include many animals that are not cats. The F1 score helps you find a good middle ground, where you are both accurate and complete.
    compared to guessing
    Download Predictions

    The Best Non-Explainable Model performed very well on this dataset:

    +34.72

    What is an F1 Score?

    The F1 score is useful because it tells you how well you balance accuracy and completeness.

    Imagine you are playing a game where you have to guess which animals are cats and which are not. You have a bunch of pictures of different animals, and you have to say "cat" or "not cat" for each one.

    Now, there are two things you want to do well in this game:

    • You want to be accurate, which means you don't say "cat" when the animal is not a cat, and you don't say "not cat" when the animal is a cat.
    • You also want to be complete, which means you don't miss any cats that are in the pictures.
    The F1 score is a way of measuring how well you do both of these things. It is usually a number between 0 and 1 (and in this case we multiply it by 100 to be more readable), where 100 means you are perfect, and 0 means you are very bad.

    How is it calculated?

    To calculate the F1 score, you need to know four things:
    • How many times you said "cat" and the animal was actually a cat. This is called a true positive (TP).
    • How many times you said "cat" and the animal was not a cat. This is called a false positive (FP).
    • How many times you said "not cat" and the animal was actually a cat. This is called a false negative (FN).
    • How many times you said "not cat" and the animal was not a cat. This is called a true negative (TN).

    Then, you can use these numbers to calculate two other things:

    • Your precision, which is the fraction of times you were right when you said "cat". You can find it by dividing TP by TP + FP.
    • Your recall, which is the fraction of times you found all the cats in the pictures. You can find it by dividing TP by TP + FN.

    Finally, you can use your precision and recall to calculate the F1 score. The F1 score is the harmonic mean of precision and recall, which means it is a kind of average that gives more weight to lower values. You can find it by multiplying precision and recall by 2, and then dividing by precision + recall.

    Here is the formula for the F1 score:

    If you only care about accuracy, you might say "not cat" for every picture, and never make a mistake. But then your recall would be zero, because you would miss all the cats. If you only care about completeness, you might say "cat" for every picture, and find all the cats. But then your precision would be very low, because you would also include many animals that are not cats. The F1 score helps you find a good middle ground, where you are both accurate and complete.
    compared to guessing
    Download Predictions
    Recommended
    The top Non-Explainable model performed well, but none of the Explainable models achieved a similar performance. If you only care about the Predictions, we recommend using the Non-Explainable predictions. However, if you want to understand how the models arrived at the predictions, we recommend revisiting your data or setting up a consultation with one of our data scientists for better Explainable Model results.
    All Model Results
    Credit

    Explored by avishkar@gmail.com using Telecom - Churn Predict.csv on 2024-03-15 with Berrijam. Try it yourself at Berrijam.com