Analysis and SynthesisThis unit begins the bulk of the critical work we will do in this course. We have read many articles covering many or most of the major strands of research on literacy in the U.S. at this time, and thus are in a position to begin evaluating these articles' arguments not only on their own merits, but also with respect to one another.
The unit is, as the name suggests, composed of two sub-units: analysis and synthesis. The first section, analysis, will entail critiquing the articles, which is to say, looking for the strengths and weaknesses of those articles' arguments. In the second section, synthesis, we will be looking for common themes that tie the articles together: assumptions, methods, data, programmatic concerns, etc. (Note that this necessarily involves noting differences among the articles, as well.) Finally, we will return to analysis again, and will discuss the approaches we've identified in terms of strengths and weaknesses.
The best way to learn these skills is to apply them, and so over the next several days we'll spend class time analyzing various authors' arguments. The next few pages will give you specifics on several aspects of this unit, beginning with general principles of critique and proceeding to the specific heuristics we will use in this unit.
General introduction to critique
"Critique" refers to the assessment of an argument. More specifically, it denotes the activity of generating reasons for adhering to or not adhering to another's position on an issue.
Note that our assessment of another's argument will always be in relation to its ability to address a particular issue. We will not be conducting a general critique of each authors' argument; rather, we will assess each argument as it responds to a specific compelling question or problem.
Note further that critique involves both positive and negative evaluations of an argument. You will rarely find an argument that has nothing whatever to recommend it, and equally rarely will you find an argument that has nothing whatever wrong with it. The process of critique is meant to hone arguments, to look for those useful things in others' arguments that may be taken away for further use, as well as for those less-useful sections or problems that we wish to eliminate or minimize.
This section will discuss two major classes of arguments: good (or at least adequate) arguments and bad arguments, which are also called fallacies.
Good arguments
These arguments are good grounds for evaluating others' arguments. Again, note that each of these permits of both positive and negative evaluations. These are not presented in any particular order (i.e., the ones at the beginning are not the strongest). Furthermore, sometimes different kinds of evaluation will overlap, so you need not feel that these are rigid categories so much as suggestions to help your inquiry.
Discussion of the problem
Has the author clearly described the situation s/he is responding to? Whom does s/he wish to help? What is the nature of the problem? Is it really a problem? (I.e., is anyone really being harmed by the situation as it stands?) Is the problem accurately described, or has the author misrepresented events?
An argument can be positively evaluated in this way if it seems that the author has really explained what it is going on and why it is a problem. On the other hand, if these links are missing, if it's not clear that the problem really exists or that it is a significant problem, you might evaluate it negatively.
Explanatory power
This has two major divisions: causal explanations and correlative explanations.
For the former: Does the author's description of the problem work internallyÑin other words, if you look at the problem through an agent/action/goal/result framework, does the author's description of who is doing what to whom and with what results really explain the problem as it stands? Does s/he have evidence to support his/her contentions about the agent, action, goal, and result? Does the scope of the author's claim hold waterÑdoes s/he make good claims, or does s/he try to make claims that are too strong or too broad (or both) based on the evidence?
As for correlative explanations, the same questions basically apply: is there sufficient evidence to support the hypothesis that there is some reliable connection between events A and B? Is the author using too little evidence to make claims that are too broad? Or does the author make good connections between events?
This is very important. Authors should provide a good explanation for the causes of a problem and/or solution (as Ogbu attempts to do), or provide you with enough evidence for you to agree that, though there may not be an immediately evident cause, that nonetheless events A and B are tied together (as Hirsch tries to do when he suggests that declining SAT scores can be tied to a decline in cultural literacy). Ideally, both kinds of arguments will be present.
Evaluation of evidence
This is obviously tied to the above considerations, which often ask you to evaluate the evidence in order to decide whether the author is resting too strong an argument on too slender a basis.
General questions: Where does the author pull his or her evidence from? Is it from his or her own research, or from others'? If s/he uses others' evidence, is it properly documented?
If the author is using statistical evidence: Does statistical evidence seem accurate? Are sample sizes, errors, etc., presented and accounted for? Can you tell where the statistics were taken? Does the sample seem representative of the larger group for which claims are being made? Are any statistical correlations sound, or does the author seem to be using numbers for the sake of appearances?
If the author is using anecdotes: Do anecdotes, if used, seem truly representative to you, or do they seem unusual or absurd? Where are the anecdotes taken from? Are they from someone's experience, or are they hypothetical?
If the author is making explicit references to someone else's work: Does the author provide enough context for you to feel secure that s/he is accurately representing that work? Is the source reliable, or have you heard or read anything that leads you to doubt the reliability of that source? Is the author using a source to bolster a point, or merely to name-drop?
Breadth of argument
Does the author neglect any important cases? If you can think of any important counterexamples to the author's argument, any significant instance that the author's theory ought to account for but cannot, this might be a serious flaw in his or her argument. This is, of course, adjunct to other discussions of the "fit" between argument and evidence.
If, on the other hand, the author's argument not only fully explains the situation s/he is describing, but also seems to account for a range of other events, this can be a strength of the argument. Or, less directly, an author's entire argument may not be useful for many other situations, but the general thesis or methods might be useful for other situations. A certain theory, in other words, may be fruitful for further inquiry.
Make sure not to fault the author too heavily for failing to account for events and problems outside his or her stated field of research. While you can perhaps suggest that Ogbu's or Resnick and Resnick's is too narrow and esoteric to be of much general interest, nonetheless it would be unfair to critique Ogbu for, say, failing to account for the experiences of Ukrainians in Russian schools or Resnick and Resnick for failing to account for fourteenth-century Flemish schooling when they never discussed that subject nor intended it to be part of their argument.
Consequences and usefulness
Generally speaking, we must ask how useful a theory is before we can apply it. What consequences will arise from applying a certain program or theory? Will it be helpful? For instance, Ogbu critiques the "mismatch hypothesis" on this ground: "And third, although data and insights from studies based on the mismatch model can be used for remedial efforts (Simons 1976; Erickson 1978), they cannot lead to social change that would eventually eliminate the need for remedial efforts" (Ogbu 138). Conversely, one can suggest that a theory would lead to very useful outcomes.
Again, be sure to be fair in your critique. If you critique the usefulness of a theory, make sure you do not critique it for failing to solve problems the author did not address. For instance, you cannot critique Ogbu for failing to address the problems of illiteracy in China, simply because Ogbu did not write his article with the intent of discussing illiteracy in countries outside the U.S.
Implications and results
You can also discuss the outcomes of a theory, either intended or unintended. You may wish to argue that a certain plan for action might produce harmful side-effects, or conversely might have unintended benefits.
Ease of application
There is value to a theory that is both straightforward and easy to use. However, this is hardly the most complimentary thing one can say about a method or theory, so be careful not to stress this point too heavily. Furthermore, useless or even harmful theories can be readily and easily applied, so evaluating the ease of application of a theory is really best done if the theory has many other features to recommend it. Conversely, however, if a theory is particularly difficult to implement, it can be used as a negative critique of that theory. After all, given limited time and money, usually some trade-offs between the general worth of a theory and the ease of application must be made.
Necessity
This too is not the most obviously complimentary thing one can say about a theory, but nonetheless it can be important. One can suggest that a theory, method, or program lays important groundwork, either in theory or in practice. For instance, many basic literacy programs are not sufficient in and of themselves, but of course students must have basic literacy skills before they can reap the benefit of advanced literacy skills, so basic literacy programs can lay needed groundwork for later efforts. Conversely, one can argue that a given theory is unnecessary, that it does not do anything we need or want.
Assumptions
You can also critique an author's assumptions, either positively or negatively. This usually takes the form of critiquing the author's values or goals (where the two obviously cannot be wholly disentangled). For the former, you might argue that an author rests his or her theory on certain values (self-reliance, charity, compassion, etc.) that are of the kind we wish to either encourage or discourage. For the latter, you can argue, as Knoblauch does, that certain goals are implicit (or even explicit) in a given program, and ask whether those are desirable or not.
Contradictions
Are there any contradictions in the author's argument, or does it hold water? This has largely been covered above, but it bears repeating: check to make sure that the author's evidence supports his or her claims, that s/he does not contradict him- or herself on any points, etc. You can make arguments of values in this wayÑyou might suggest, for instance, that a certain supposition in an argument entails holding certain values that contradict explicit claims elsewhere in the article.
Using other authors' arguments
Finally, don't forget that these authors have been subjected to criticism from other writers, too. For instance, Gee discusses Hirsch's research, and Knoblauch critiques various of the authors we've read. So long as you give proper attribution to the authors of a critique, there's nothing wrong with employing others' expertise to make a point.
Fallacies
Fallacies are errors in reasoning, and are not valid grounds on which to critique an argument. Here are three of the most common fallacies:
1.) The argument ad hominem, or "against the man."
This fallacy occurs when a critic attacks an author personally (for his or her character, motives, actions, affiliations, or questionable sincerity) instead of engaging with his or her argument on its own merits. It is a fallacy because it prematurely shuts down the evaluative process.
Examples:
Hirsch's argument must be rejected for its rampant conservative bias.
Gee clearly has a social agenda; therefore, we need not consider his politically offensive argument.
Note that there is a fine line between critiquing assumptions and values and mounting ad hominem arguments. Look at what Knoblauch does to see the difference: while Knoblauch argues that each theory of literacy masks certain agendas, she then goes on to suggest what's unhelpful about each agenda (i.e., that it doesn't help enough people, that it doesn't do everything it claims it will, that it protects certain interests, etc.). At certain points, you will have to come down to arguments of value: you might want to suggest in the end that Hirsch's conservatism leads to inescapable problems. But it you don't suggest why this is the case, if your readers don't immediately agree with you that conservative theories of literacy are by nature wrong, then you will not have convinced them. This limits the effectiveness and reach of your argument.
2.) The "slippery slope" fallacy.
This error in reasoning occurs when a proposal is criticized on the ground that it will lead, by an inevitable sequence of linked consequences, to an end result that is catastrophic. This strategy is often called "alarmist."
Examples:
If we were to take action based on Gee's argument, we would eliminate the study of grammar, and soon any variety of English would be considered acceptable in any situation. This would lead to chaos.
If we take Hashimoto's lead, then no reliable techniques of writing would be taught, and our students would come away from school with no skills whatsoever.
The fallacy occurs because the writer makes no argument for the causal connection between the initial event and its alarming consequence. Note also that such an argument, again, is unlikely to persuade anyone but the gullible and the already sympathetic.
3. ) The "straw man" argument
This error in reasoning occurs when the critique of an argument is based on a weak, incomplete, or inaccurate version of that argument.
Example:
If we adopt Scribner's critique of definitions of literacy and fail to agree on a definition of literacy, then we will have no solid foundation to stand on, for we will not be able to agree on what kinds of skills to teach in any given situation.
This critique ignores Scribner's frequent notes that each of the metaphors for literacy that she examines has its uses in various times and places, even though no one of them is sufficient in and of itself. It is an unfair reduction of Scribner's argument, in other words, and ignores the provisions she makes to counteract the above arguments.
Analysis grid
This heuristic is intended to help you prepare for your analysis paper. We have talked in class about discussing a subject with reference to many authors' arguments. The analysis grid will give you a format in which to test these idea before you start writing the paper.
What you need to do:
First, generate an interesting question or issue that you think can be applied to at least five of the authors we've studied (six or more if you're inspired). We have discussed many specific topics concerning literacy (education, social factors, cognitive factors, the purposes and roles of literacy, etc.), and anything pertaining to those discussions would be applicable. Here are a few things to keep in mind about this question:
1.) It should be broad enough to apply fairly well to at least five authors. While you can extrapolate from an author's stated position to an hypothetical position (i.e., Ogbu does not mention anything that resembles Gee's Discourses, but we were still able to discuss the Ogbu article with some reference to Gee), you may find that some questions simply can't be applied to many authors. Note that while your question must be broad enough to apply to five or more authors, somewhat specific questions generally produce more interesting, informative answersÑfor instance, you might think about something as specific as, "What role does the home environment play in students' literacy levels?" This is both a specific question and yet is something that is relevant to each author's article. A question as broad as Òwhat is literacy?Ó will, on the other hand, produce vague, dull answers.
2.) It should be a question or an issue that sparks some discussion. One way to think of this would be that you are to come up with "what" or "how" or "why" questions, rather than "yes/no" questions. For instance, if you were to try to answer the question, "Do the authors think illiteracy is bad?" you would probably get very short, very uninformative answers (i.e., "Yes, illiteracy is bad" five or six times). If you instead asked the question, "Why does each author think illiteracy is bad? What are the specific ill effects?" you would find more complicated answers. If you asked a more complicated question, say "How is illiteracy determined in real life? How is a person's 'level' of literacy determined at any given moment?", you might come up with more interesting answers still.
State briefly some issues of existence and significance. In other words, describe in a couple of lines when the situation you describe actually happens and why it's important to talk about it. In the example of standardized tests, you might argue for the existence of this problem by pointing out that standardized tests are used broadly as proof of students' literacy. As an argument for significance, you might note that these tests have a lot to do with how literacy is judged and taught or that they determine access to higher education, and thus have real effects on students' livesÑyou could even use the Applebee article as an example. (An example of something that does not fulfill these criteria is, "Would aliens from outer space be considered literate if they came to Earth?") Finally, fill out a grid like the one below:
This will allow you to decide whether you're posing a good, relevant question. It will also
give you a way to see what the analysis assignment is all about: evaluating authors' positions with respect to a specific research question.
Synthesis tree
For this heuristic, take a research question, such as you have used above, and construct a ÒtreeÓ of the authors' positionsÑwhich is to say, look for the similarities between the authors' positions on the question you put forward. You should begin with a major division, something that differentiates the authors into two or three ÒcampsÓ of two or more authors each. Do not construct a group for each authorÑin other words, do not make each author a representative of his or her own idiosyncratic position, but rather try to view each author as being in some agreement with some other author(s). This first split is called the major split, and should reflect some major difference of opinion, some way in which you can roughly view certain authors as being in accord with one another.
The only exception is that you may place one author within his or her own group if you think there is a preliminary split that separates him or her from the other authors, which is to say that there is some issue upon which that author disagrees the other authors at a fundamental levelÑas you might if you asked the question, ÒWhat does each author see as the defining characteristic of literacy?Ó Obviously, Scribner would fall outside almost any grouping of authors you made in response to this question. (And yes, if you do this, you need to divide the remaining authors into at least two groups, each of two or more authors.)
After this, differentiate among the authors in each group. You may make sub-groups within the major groups, or you may find that you see the authors as diverging completely from one another at this point. You should end, however, with something characterizing the positions of each author.
Include in your tree the question and the basis upon which you make each split. You need not draw a tree, but be orderly and make it clear where the splits come, what the points of difference are, and which authors fall on which side of the question. Remember to take your tree down to the individual authors, so that you differentiate among the authors in a given group, even if only on minor points. You should, in other words, begin with a general discussion of the major similarities and differences among the authors' positions, and end with a specific discussion of each author's position.