Type II error bias is not limited to just the FDA. In the April 2006 journal Public Choice Russel Sobell and Peter Leeson explain that type II error bias played a role in the delayed response of the federal government after hurricane Katrina.
Let’s say our government leader finds themself in the following predicament. After the destruction and the levees have broken someone has to make the decision to send in relief workers. However there are risks. Disease infested water, collapsing buildings and roads, roaming bandits, toxic chemical exposure etc. If our leader sends in workers and they meet a terrible fate, the consequences are on his shoulders. Later the media would crucify him over sending our brave heroes in harms way.
Given limited information, here is the model in the context of hypothesis testing:
Ho: X= Xo ‘it is too harmful to send in relief ASAP’
Ha: X= Xa: ‘the harm is trivial, and justified to send in relief ASAP’
Type I Error: sending relief workers into unnecessary danger
If on the other hand, our government official decides to wait, until he has better information he could avoid this. The consequences may be the lives of hurricane victims, but the blame can be shared with nature.
Type II Error: Over cautiousness that prevents a quick response by relief authorities
In general this is one of the explanations given in the Public Choice article explaining the federal government’s response to Hurricane Katrina. It is also one of the reasons in general that many of our leaders at the federal, state, local, levels fail to make timely decisions or provide leadership when needed.
See: Russell Sobel & Peter Leeson, 2006.
"Government's response to Hurricane Katrina: A public choice analysis," Public Choice, Springer, vol. 127(1), pages 55-73, April.
link
Monday, May 23, 2011
Type II Error Bias and the FDA
Let’s assume that FDA researchers are reviewing a drug for possible approval. They are reviewing results and data from clinical trials etc. Of course there are some side effects , but only from a small percentage of the samples. Let’s look at a model of their decision making process in the context of hypothesis testing. Let’s assume:
Ho: X = Xo ‘drug is harmful’
Ha: X = Xa ‘drug is safe’
If the decision makers make the mistake of releasing a harmful drug, the consequences would be easily identifiable, and could be visibly traced to their decision. They would want to avoid this at all cost. In essence, they want to avoid making a type I error.
Type I Error: = releasing a harmful drug
From my previous discussion on hypothesis testing, we know that to decrease the probability of committing a type I error, we choose an alpha level that is lower. In a t-test, if we are really afraid of making a type I error, we might set alpha ( the significance level) at .05, .01, or to go overboard.001 etc.
As previously discussed, setting alpha lower and lower increases beta, or the probability of committing a type II error. Recall, a type II error is accepting a false Ho. In this case that would be equivalent to falsely concluding that the ‘drug is harmful’ when it could actually be released and improve the lives of millions.
Type II error bias = over precaution; setting alpha so low, or setting the standard of proof so high as to almost always reject Ho, and biasing the decision in such a way that the probability of a type two error (beta) is greatly increased.
As a result of type II error bias, many life improving drugs never make it to the market.
Ho: X = Xo ‘drug is harmful’
Ha: X = Xa ‘drug is safe’
If the decision makers make the mistake of releasing a harmful drug, the consequences would be easily identifiable, and could be visibly traced to their decision. They would want to avoid this at all cost. In essence, they want to avoid making a type I error.
Type I Error: = releasing a harmful drug
From my previous discussion on hypothesis testing, we know that to decrease the probability of committing a type I error, we choose an alpha level that is lower. In a t-test, if we are really afraid of making a type I error, we might set alpha ( the significance level) at .05, .01, or to go overboard.001 etc.
As previously discussed, setting alpha lower and lower increases beta, or the probability of committing a type II error. Recall, a type II error is accepting a false Ho. In this case that would be equivalent to falsely concluding that the ‘drug is harmful’ when it could actually be released and improve the lives of millions.
Type II error bias = over precaution; setting alpha so low, or setting the standard of proof so high as to almost always reject Ho, and biasing the decision in such a way that the probability of a type two error (beta) is greatly increased.
As a result of type II error bias, many life improving drugs never make it to the market.
Type I and Type II Errors
In an experimental setting where a statistical test of a hypothesis is conducted one may either reject the null hypothesis ‘Ho’, or fail to reject Ho. Let’s assume that the true population parameter we are testing is X. Our null hypothesis may be Ho: X=Xo.
The probability of rejecting the null hypothesis when it is true can be defined by:
Alpha = P (reject Ho X = Xo)
Rejecting the null hypothesis when it is actually true is referred to in statistics as a type I error. Therefore alpha is the probability of ‘committing’ a type 1 error. In a basic statistics, when you conduct a basic t- test ( i.e. reject Ho if t > t-critical) at the 5% level of significance, you are establishing a 5% chance of committing a type one error.
Of course, you may fail to reject Ho, or loosely speaking ‘accept’ Ho.
The probability of ‘accepting’ Ho when it is false can be defined by:
Beta = P (accept Ho X = Xa)
Speaking heuristically, accepting a false Ho is referred to as a type II error. Beta is therefore the probability of committing a type 2 error.
It turns out, that as you increase the significance level of a test ( by making alpha lower and decreasing the probability of a type I error), the probability of a type II error (beta) increases. This is the theoretical basis for ‘type II error bias.’
The probability of rejecting the null hypothesis when it is true can be defined by:
Alpha = P (reject Ho X = Xo)
Rejecting the null hypothesis when it is actually true is referred to in statistics as a type I error. Therefore alpha is the probability of ‘committing’ a type 1 error. In a basic statistics, when you conduct a basic t- test ( i.e. reject Ho if t > t-critical) at the 5% level of significance, you are establishing a 5% chance of committing a type one error.
Of course, you may fail to reject Ho, or loosely speaking ‘accept’ Ho.
The probability of ‘accepting’ Ho when it is false can be defined by:
Beta = P (accept Ho X = Xa)
Speaking heuristically, accepting a false Ho is referred to as a type II error. Beta is therefore the probability of committing a type 2 error.
It turns out, that as you increase the significance level of a test ( by making alpha lower and decreasing the probability of a type I error), the probability of a type II error (beta) increases. This is the theoretical basis for ‘type II error bias.’
Public Choice Theory
There are fundamental differences between the ways that markets allocate resources (using prices which reflect trade-offs based on the knowledge and preferences of millions of individuals) vs government (which allocates resources using command and control based on the more limited knowledge and preferences of a few voters, elected officials, or appointed bureaucrats).
The economic analysis of political institutions represents a sub field of economics referred to as 'Public Choice Economics.' The following link will take you to an article 'The Public Choice Revolution', Regulation Fall 2004.
This article summarizes some of the major findings from the field of public choice. The article discusses among many things, why do we need government, and what are the tradeoffs between tyranny (or Leviathan ) and anarchy.
"Once it is admitted that the state is necessary, positive public choice analyzes how it assumes its missions of allocative efficiency and redistribution. Normative public choice tries to identify institutions conducive to individuals getting from the state what they want without being exploited by it."
Another section looks at the analysis of voting. It discusses problems with using voting to allocate resources, such as cycling and the median voter theorem. It also looks at issues related to special interests, bureaucracies, and the role of representative government in democratic systems.
"In our democracies, voters do not decide most issues directly. In some instances, they vote for representatives who reach decisions in parliamentary assemblies or committees. In other instances, they elect representatives who hire bureaucrats to make decisions. The complexity of the system and the incentives of its actors do not necessarily make collective choices more representative of the citizens’ preferences."
Particularly the article looks at how more government with a larger budget leads to more power for special interests:
"Interest groups will engage in what public choice theorists call “rent seeking,” i.e., the search for redistributive benefits at the expense of others. The larger the state and the more benefits it can confer, the more rent-seeking will occur. “The entire federal budget,” writes Mueller, “can be viewed as a gigantic rent up for grabs for those who can exert the most political muscle.”
The moral of the public choice story is that democracy and governments are not perfect. When we have issues with market outcomes and we are thinking about new regulations or government spending to correct those problems, public choice analysis offers a guide. Public choice analysis implies that we have to question which system will provide the best information and incentives to act on that information to achieve the results that we want. The democratic process itself offers no guarantee that the decision will be the best solution to our problems.
Concepts related to public choice theory include the following:
RENT SEEKING - the act of seeking special privileges or protections form the government.
TYPE TWO ERROR BIAS - overcautious behavior, ex: FDA drug approval, response to Hurricane Katrina
VOTING PARADOXES- randomness of election outcomes
MEDIAN VOTER THEOREM- leads to exploitation of minority by majority
TRAGEDY OF THE COMMONS – lack of property rights and pollution
COASE THEOREM – symmetry of environmental pollution, internalizing effect of property rights and markets
TRAGEDY OF THE ANTICOMMONS – underutilized resources due to excessive checks on power, bureaucracy. Ex: response to hurricane Katrina
KNOWLEDGE PROBLEM- government relies on a ‘shrunken’ pool of knowledge vs. markets
By clicking the ‘public choice’ link below, or under the ‘labels’ sidebar you can find more detailed discussions of each of these concepts
The economic analysis of political institutions represents a sub field of economics referred to as 'Public Choice Economics.' The following link will take you to an article 'The Public Choice Revolution', Regulation Fall 2004.
This article summarizes some of the major findings from the field of public choice. The article discusses among many things, why do we need government, and what are the tradeoffs between tyranny (or Leviathan ) and anarchy.
"Once it is admitted that the state is necessary, positive public choice analyzes how it assumes its missions of allocative efficiency and redistribution. Normative public choice tries to identify institutions conducive to individuals getting from the state what they want without being exploited by it."
Another section looks at the analysis of voting. It discusses problems with using voting to allocate resources, such as cycling and the median voter theorem. It also looks at issues related to special interests, bureaucracies, and the role of representative government in democratic systems.
"In our democracies, voters do not decide most issues directly. In some instances, they vote for representatives who reach decisions in parliamentary assemblies or committees. In other instances, they elect representatives who hire bureaucrats to make decisions. The complexity of the system and the incentives of its actors do not necessarily make collective choices more representative of the citizens’ preferences."
Particularly the article looks at how more government with a larger budget leads to more power for special interests:
"Interest groups will engage in what public choice theorists call “rent seeking,” i.e., the search for redistributive benefits at the expense of others. The larger the state and the more benefits it can confer, the more rent-seeking will occur. “The entire federal budget,” writes Mueller, “can be viewed as a gigantic rent up for grabs for those who can exert the most political muscle.”
The moral of the public choice story is that democracy and governments are not perfect. When we have issues with market outcomes and we are thinking about new regulations or government spending to correct those problems, public choice analysis offers a guide. Public choice analysis implies that we have to question which system will provide the best information and incentives to act on that information to achieve the results that we want. The democratic process itself offers no guarantee that the decision will be the best solution to our problems.
Concepts related to public choice theory include the following:
RENT SEEKING - the act of seeking special privileges or protections form the government.
TYPE TWO ERROR BIAS - overcautious behavior, ex: FDA drug approval, response to Hurricane Katrina
VOTING PARADOXES- randomness of election outcomes
MEDIAN VOTER THEOREM- leads to exploitation of minority by majority
TRAGEDY OF THE COMMONS – lack of property rights and pollution
COASE THEOREM – symmetry of environmental pollution, internalizing effect of property rights and markets
TRAGEDY OF THE ANTICOMMONS – underutilized resources due to excessive checks on power, bureaucracy. Ex: response to hurricane Katrina
KNOWLEDGE PROBLEM- government relies on a ‘shrunken’ pool of knowledge vs. markets
By clicking the ‘public choice’ link below, or under the ‘labels’ sidebar you can find more detailed discussions of each of these concepts
Biotech Alfalfa: Who May Harm Who- An application of the Coase Theorem
From Drovers/Cattle Network "Using Property Rights Is A Trick Against Biotech Crops"
"Some politicians wrap themselves in the flag to justify their positions, and then there is Secretary of Agriculture Tom Vilsack appealing to farmers and ranchers' belief in "private property rights" to justify limiting biotech crop production"
Great article with a lot of great points. For the sake of this discussion, lets view biotech contamination of organic crops as 'pollution.' (despite the evidence that the risks are slight)
Traditionally when it comes to environmental pollution, the general philosophy was that 'the polluter pays'. A factory polluting the air or water should pay for the damages that are caused. In a much simpler case, if you build a house next to me and you don't like the smell of livestock waste coming from my property, the traditional philosophy would hold that you could have the government stop my operation. (or in this case, the biotech alfalfa grower pays for genetic contamination of organic alfalfa)
The economist Ronald Coase brought additional insight to this issue.
1) yes it is true that my operation is harming you via air pollution. (odor)
2) however, in stopping me via government or legal intervention ( or taxing my waste production) you are harming me.
Coase says that the issue is that nonone owns the air that surrounds my livestock operation and your home. There then follows a dispute over how the air should be used- to absorb livestock odor, or to provide a scent free atmosphere in your back yard. Whenever the cost of one's behavior is not factored into a price at which a choice can be valued, I can harm you without compensating you for it. ( i.e. an externality exists)
However, if I own rights to the air, then I can choose to pollute the air. If you own rights to the air, then you can prevent me from polluting it. If noone owns the air, then it is first come first served or winner takes all.
That is not the end of the story though. What Coase emphasizes is that if I own the rights to pollute, you can pay me to limit my pollution i.e. buy those rights from me. I can then use the proceeds to alter my livestock nutrition, genetics, and management to reduce the odor my operation is causing. On the other hand, if you own the rights to pollute I can purchase those rights from you, or invest in technology that will allow me to continue my operation without violating your rights. I will do which ever is most optimal. This can be accomplished without major government regulation, or the arbitrary imposition of a tax.
The assignment of property rights and the potential for bargaining results in behavior that is changed or altered to account for the negative impact our choices have on others. This is the essence of what is known as the 'Coase Theorem"
However, if transaction costs are high, then bargaining may not take place. In that case, Coase emphasizes that any assignmnet of property rights should be based on which party can bear the externality at the lowest cost. Transaction costs can change based on changes in technology, which can also change how we define property rights. (for example, the technology that allows us to monitor CO2 emissions is what makes the concept of cap and trade possible).
How might this apply in the context of biotech alfalfa? According to the Coase Theorem, it shouldn't matter who is assigned the rights in this case (giving the biotech producer the right to pollute, or giving the organic producer the right to stop neighbors from planting biotech). Both parties could bargain ahead of time to determine the optimal mix of biotech/organic production. Transaction costs should not be any higher than any normal land rental agreement. Alternatively, one producer or the other could purchase insurance that would pay an indemnity in the event of contamination. (who would have to pay the premiums would depend on who has the right to pollute etc.) However, monitoring and enforcement costs could be high in terms of determining genetic contamination.
Another option would be a regulatory approach, limiting planting options for biotech producers. This is what the Drovers article is critical of Tom Vilsack for. You could say it is enforcing property rights, but only in a very arbitrary way, and unnecessary.
The agriculture industry offers some of the greatest examples of how technological advances and market forces lead to self correcting or internalization of externalities. The adoption of biotechnology has led to reduced groundwater pollution, increased biodiversity, and reduced greenhouse gas emissions. All of which has occured in absence of taxes or government regulations. In the case of biotech alfalfa, a technological advancement that would trump legal or regulatory remedies would be use of 'terminator' gene technology. Of course, that takes the power and prestige away from regulators, and empowers property owners and market forces. In any case, what the Coase Theorem tells us is that there is no case for arbitrarily giving organic growers a trump card over those that want to use biotech alfalfa. The principle of polluter pays is not always optimal.
"Some politicians wrap themselves in the flag to justify their positions, and then there is Secretary of Agriculture Tom Vilsack appealing to farmers and ranchers' belief in "private property rights" to justify limiting biotech crop production"
Great article with a lot of great points. For the sake of this discussion, lets view biotech contamination of organic crops as 'pollution.' (despite the evidence that the risks are slight)
Traditionally when it comes to environmental pollution, the general philosophy was that 'the polluter pays'. A factory polluting the air or water should pay for the damages that are caused. In a much simpler case, if you build a house next to me and you don't like the smell of livestock waste coming from my property, the traditional philosophy would hold that you could have the government stop my operation. (or in this case, the biotech alfalfa grower pays for genetic contamination of organic alfalfa)
The economist Ronald Coase brought additional insight to this issue.
1) yes it is true that my operation is harming you via air pollution. (odor)
2) however, in stopping me via government or legal intervention ( or taxing my waste production) you are harming me.
Coase says that the issue is that nonone owns the air that surrounds my livestock operation and your home. There then follows a dispute over how the air should be used- to absorb livestock odor, or to provide a scent free atmosphere in your back yard. Whenever the cost of one's behavior is not factored into a price at which a choice can be valued, I can harm you without compensating you for it. ( i.e. an externality exists)
However, if I own rights to the air, then I can choose to pollute the air. If you own rights to the air, then you can prevent me from polluting it. If noone owns the air, then it is first come first served or winner takes all.
That is not the end of the story though. What Coase emphasizes is that if I own the rights to pollute, you can pay me to limit my pollution i.e. buy those rights from me. I can then use the proceeds to alter my livestock nutrition, genetics, and management to reduce the odor my operation is causing. On the other hand, if you own the rights to pollute I can purchase those rights from you, or invest in technology that will allow me to continue my operation without violating your rights. I will do which ever is most optimal. This can be accomplished without major government regulation, or the arbitrary imposition of a tax.
The assignment of property rights and the potential for bargaining results in behavior that is changed or altered to account for the negative impact our choices have on others. This is the essence of what is known as the 'Coase Theorem"
However, if transaction costs are high, then bargaining may not take place. In that case, Coase emphasizes that any assignmnet of property rights should be based on which party can bear the externality at the lowest cost. Transaction costs can change based on changes in technology, which can also change how we define property rights. (for example, the technology that allows us to monitor CO2 emissions is what makes the concept of cap and trade possible).
How might this apply in the context of biotech alfalfa? According to the Coase Theorem, it shouldn't matter who is assigned the rights in this case (giving the biotech producer the right to pollute, or giving the organic producer the right to stop neighbors from planting biotech). Both parties could bargain ahead of time to determine the optimal mix of biotech/organic production. Transaction costs should not be any higher than any normal land rental agreement. Alternatively, one producer or the other could purchase insurance that would pay an indemnity in the event of contamination. (who would have to pay the premiums would depend on who has the right to pollute etc.) However, monitoring and enforcement costs could be high in terms of determining genetic contamination.
Another option would be a regulatory approach, limiting planting options for biotech producers. This is what the Drovers article is critical of Tom Vilsack for. You could say it is enforcing property rights, but only in a very arbitrary way, and unnecessary.
The agriculture industry offers some of the greatest examples of how technological advances and market forces lead to self correcting or internalization of externalities. The adoption of biotechnology has led to reduced groundwater pollution, increased biodiversity, and reduced greenhouse gas emissions. All of which has occured in absence of taxes or government regulations. In the case of biotech alfalfa, a technological advancement that would trump legal or regulatory remedies would be use of 'terminator' gene technology. Of course, that takes the power and prestige away from regulators, and empowers property owners and market forces. In any case, what the Coase Theorem tells us is that there is no case for arbitrarily giving organic growers a trump card over those that want to use biotech alfalfa. The principle of polluter pays is not always optimal.
The Impact of Farm Subsidies on the Federal Budget and Small Farms
"This is good news. Agricultural subsidies cost taxpayers more than $15 billion each year, and until those subsidies are eliminated, farming in America will never be sustainable." - Baltimore Sun, May 18, 2011.
I've seen similar numbers reported recently in the New York Times as well. Bear in mind, $15 billion sounds like a lot, but it amounts to less than 1/2 of 1% of total federal spending. The data also show that small farms depend more heavily on subsidies than larger farms (often misunderstood as 'big agribusiness' or 'industrial farms' ). Eliminating subsidies then, if anything would lead to more concentration in the industry and larger farms.
So in terms of financial sustainability, then yes the article would be correct on that point, as the smaller, less financially sustainable farms may go away. But what about environmental sustainability?
Many of the green technologies (herbicide and pest resistant GMO crops, pharmaceuticals) used by modern farmers dwarf the impact of other consumer green technologies like hybrid cars. Many of these are 'scale neutral' (for instance, the single largest growing demographic among GMO adoption is small farmers in developing countries) so eliminating small farms that use these technologies won't help with sustainability, given they have adopted these technologies. Other green technologies in agriculture include GPS and auto steer technology. Larger scale production is likely necessary to get the most (financially and environmentally) from these technologies.
Among those most lagging in green technology adoption are organic producers, which have zero tolerance for GMOs, (although fully embracing more volatile methods utilizing nuclear radiation to breed better plants)
However, even as the market for these products is greatly expanding, it makes up a very small proportion of the food we consume (largely supplementals like fruits and vegetables vs. the staple commodities that feed the world) making organic largely irrelevant to the overall conversation about sustainability in agriculture and subsidies. I'm not sure why the article even goes down this path.
Some are arguing for a compromise, capping subsidies based on income level. That may be a way to preserve smaller farms, but doesn't really make much difference in terms of government spending and likely won't matter much in terms of sustainability without knowing more about green technology adoption rates and productivity of smaller farms. Many of the arguments for ending farm subsidies based on spending, sustainability, nutrition etc. lack empirical support. Ultimately the argument about farm subsidies comes down to your view on the role of government.
Some are arguing for a compromise, capping subsidies based on income level. That may be a way to preserve smaller farms, but doesn't really make much difference in terms of government spending and likely won't matter much in terms of sustainability without knowing more about green technology adoption rates and productivity of smaller farms. Many of the arguments for ending farm subsidies based on spending, sustainability, nutrition etc. lack empirical support. Ultimately the argument about farm subsidies comes down to your view on the role of government.
As far as the article's comments on farming like our great grandparents, I'm not sure 26 bushels an acre, no matter what the method, is going to cut it these days.
Sunday, May 1, 2011
R Code Example for Linear Regression and Analysis of Variance
# EXAMPLE CODE FOR REGRESSION & AOV # read in data x<-c(20,16,34,23,27,32,18,22) y<-c(64,61,84,70,88,92,72,77) mymodel <- lm(y~x) # run regression model summary(mymodel) # output dat1 <- data.frame(x,y) # combine x and y into an R data frame ss <- aov(y~x, data = dat1) # conduct analysis of variance & save results as 'ss' summary(ss) # print results # to read the output you must realize that the row 'x' gives you the SSR & MSR # the row 'Residuals' gives tyou SSE & MSE # SST is not given but you know the formula to get it SST = SSR + SSE # based on output, calculate SST = SSR + SSE SST <- 658.5 + 227.5 print(SST) # this gives you SST # use this info to calculate R-square = SSR / SST 658.5/SST # gives .743 which was given in the original regression output # verify that the F- stat is MSR / MSE based on the output from summary(ss) F<- 658.5/37.92 print(F) # this gives the same results as summary(ss)
Subscribe to:
Posts (Atom)
