How do I interpret improvement in AUC ROC from the business perspective?Classification probability threshold2D binary classificationDownsampling vs upsampling on the significance of the predictors in logistic regressionHow to report average error rate/performance/error metrics from train function in R package caret regarding the hold-out samples from cross-validationAUC vs.Class imbalance in both training data and test dataEffectiveness of Standardization and Normalization in Machine LearningRandom Forest has almost perfect training AUC compared to other modelsAUPRC vs. AUC-ROC?How much higher accuracy of train than test is enough to consider the model overfitted?Two different approaches of oversampling data with GridSearchCV leads to similar test resultsModel Selection with Oversampling/ Cross-Validation leads to similar test results in 2 approaches
On studying Computer Science vs. Software Engineering to become a proficient coder
Can I use my laptop, which says 100-240V, in the USA?
Ex-manager wants to stay in touch, I don't want to
How can a layman easily get the consensus view of what academia *thinks* about a subject?
What to do if SUS scores contradict qualitative feedback?
Find hamming distance between two Strings of equal length in Java
What is the largest number of identical satellites launched together?
Extracting X and Y coordinates from .gpx file using QGIS?
Does SQL Server allow (make visible) DDL inside a transaction to the transaction prior to commit?
What kind of SATA connector is this?
How can I answer high-school writing prompts without sounding weird and fake?
Developers demotivated due to working on same project for more than 2 years
On what legal basis did the UK remove the 'European Union' from its passport?
Earliest use of "rookie"?
Is there any good reason to write "it is easy to see"?
Is Germany still exporting arms to countries involved in Yemen?
How to cope with regret and shame about not fully utilizing opportunities during PhD?
using `is` operator with value type tuples gives error
In books, how many dragons are there in present time?
How can dragons propel their breath attacks to a long distance
what does a native speaker say when he wanted to leave his work?
How does emacs `shell-mode` know to prompt for sudo?
How did Thanos not realise this had happened at the end of Endgame?
Why was Thor doubtful about his worthiness to Mjolnir?
How do I interpret improvement in AUC ROC from the business perspective?
Classification probability threshold2D binary classificationDownsampling vs upsampling on the significance of the predictors in logistic regressionHow to report average error rate/performance/error metrics from train function in R package caret regarding the hold-out samples from cross-validationAUC vs.Class imbalance in both training data and test dataEffectiveness of Standardization and Normalization in Machine LearningRandom Forest has almost perfect training AUC compared to other modelsAUPRC vs. AUC-ROC?How much higher accuracy of train than test is enough to consider the model overfitted?Two different approaches of oversampling data with GridSearchCV leads to similar test resultsModel Selection with Oversampling/ Cross-Validation leads to similar test results in 2 approaches
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
$begingroup$
Let's say I'm training a logistic regression model to predict the click-through-rate (CTR) on online display ads. The training dataset consists of positive examples (ads were clicked) and negative ones (ads were not clicked). The binary classifier has achieved 0.7 AUC ROC on test dataset.
Now, instead of using logistic regression, I found that gradient boosting trees performs better, yielding 0.75 AUC ROC on test dataset.
How does the improvement in AUC ROC translate to potential improvement in CTR?
machine-learning classification
asked 5 hours ago
cwlcwl
31719
$endgroup$
add a comment |
$begingroup$
Let's say I'm training a logistic regression model to predict the click-through-rate (CTR) on online display ads. The training dataset consists of positive examples (ads were clicked) and negative ones (ads were not clicked). The binary classifier has achieved 0.7 AUC ROC on test dataset.
Now, instead of using logistic regression, I found that gradient boosting trees performs better, yielding 0.75 AUC ROC on test dataset.
How does the improvement in AUC ROC translate to potential improvement in CTR?
machine-learning classification
asked 5 hours ago
cwlcwl
31719
$endgroup$
add a comment |
$begingroup$
Let's say I'm training a logistic regression model to predict the click-through-rate (CTR) on online display ads. The training dataset consists of positive examples (ads were clicked) and negative ones (ads were not clicked). The binary classifier has achieved 0.7 AUC ROC on test dataset.
Now, instead of using logistic regression, I found that gradient boosting trees performs better, yielding 0.75 AUC ROC on test dataset.
How does the improvement in AUC ROC translate to potential improvement in CTR?
machine-learning classification
asked 5 hours ago
cwlcwl
31719
$endgroup$
Let's say I'm training a logistic regression model to predict the click-through-rate (CTR) on online display ads. The training dataset consists of positive examples (ads were clicked) and negative ones (ads were not clicked). The binary classifier has achieved 0.7 AUC ROC on test dataset.
Now, instead of using logistic regression, I found that gradient boosting trees performs better, yielding 0.75 AUC ROC on test dataset.
How does the improvement in AUC ROC translate to potential improvement in CTR?
machine-learning classification
machine-learning classification
asked 5 hours ago
cwlcwl
31719
asked 5 hours ago
cwlcwl
31719
asked 5 hours ago
cwlcwl
31719
asked 5 hours ago
cwlcwl
31719
asked 5 hours ago
cwlcwl
31719
31719
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
We have no idea. And the AUROC won't help you.
A statistical model alone does not generate value. (Therefore, you cannot quantify how much more value an improvement in your model yields.)
What generates value is decisions. Better decisions generate more value. If your improved model leads to better decisions, then it generates more value. But you might also be able to generate more value using the very same model, by changing the way you base decisions on the model.
Therefore, you will need to think about and possibly simulate how your model is translated into decisions, and you will need to assign a value to each pair (decision, actual outcome). Note that maximizing AUROC may not be the output from your model you need to maximize value. More info at my answer to the earlier "Classification probability threshold".
answered 4 hours ago
Stephan KolassaStephan Kolassa
50.4k8103189
$endgroup$
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "65"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f407990%2fhow-do-i-interpret-improvement-in-auc-roc-from-the-business-perspective%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
We have no idea. And the AUROC won't help you.
A statistical model alone does not generate value. (Therefore, you cannot quantify how much more value an improvement in your model yields.)
What generates value is decisions. Better decisions generate more value. If your improved model leads to better decisions, then it generates more value. But you might also be able to generate more value using the very same model, by changing the way you base decisions on the model.
Therefore, you will need to think about and possibly simulate how your model is translated into decisions, and you will need to assign a value to each pair (decision, actual outcome). Note that maximizing AUROC may not be the output from your model you need to maximize value. More info at my answer to the earlier "Classification probability threshold".
answered 4 hours ago
Stephan KolassaStephan Kolassa
50.4k8103189
$endgroup$
add a comment |
$begingroup$
We have no idea. And the AUROC won't help you.
A statistical model alone does not generate value. (Therefore, you cannot quantify how much more value an improvement in your model yields.)
What generates value is decisions. Better decisions generate more value. If your improved model leads to better decisions, then it generates more value. But you might also be able to generate more value using the very same model, by changing the way you base decisions on the model.
Therefore, you will need to think about and possibly simulate how your model is translated into decisions, and you will need to assign a value to each pair (decision, actual outcome). Note that maximizing AUROC may not be the output from your model you need to maximize value. More info at my answer to the earlier "Classification probability threshold".
answered 4 hours ago
Stephan KolassaStephan Kolassa
50.4k8103189
$endgroup$
add a comment |
$begingroup$
We have no idea. And the AUROC won't help you.
A statistical model alone does not generate value. (Therefore, you cannot quantify how much more value an improvement in your model yields.)
What generates value is decisions. Better decisions generate more value. If your improved model leads to better decisions, then it generates more value. But you might also be able to generate more value using the very same model, by changing the way you base decisions on the model.
Therefore, you will need to think about and possibly simulate how your model is translated into decisions, and you will need to assign a value to each pair (decision, actual outcome). Note that maximizing AUROC may not be the output from your model you need to maximize value. More info at my answer to the earlier "Classification probability threshold".
answered 4 hours ago
Stephan KolassaStephan Kolassa
50.4k8103189
$endgroup$
We have no idea. And the AUROC won't help you.
A statistical model alone does not generate value. (Therefore, you cannot quantify how much more value an improvement in your model yields.)
What generates value is decisions. Better decisions generate more value. If your improved model leads to better decisions, then it generates more value. But you might also be able to generate more value using the very same model, by changing the way you base decisions on the model.
Therefore, you will need to think about and possibly simulate how your model is translated into decisions, and you will need to assign a value to each pair (decision, actual outcome). Note that maximizing AUROC may not be the output from your model you need to maximize value. More info at my answer to the earlier "Classification probability threshold".
answered 4 hours ago
Stephan KolassaStephan Kolassa
50.4k8103189
answered 4 hours ago
Stephan KolassaStephan Kolassa
50.4k8103189
answered 4 hours ago
Stephan KolassaStephan Kolassa
50.4k8103189
answered 4 hours ago
Stephan KolassaStephan Kolassa
50.4k8103189
50.4k8103189
add a comment |
add a comment |
Thanks for contributing an answer to Cross Validated!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f407990%2fhow-do-i-interpret-improvement-in-auc-roc-from-the-business-perspective%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
