[CTcard]

Here's a version of this story that's not behind a paywall.

Thanks for the link. I wasn't able to read this before.
Apparently that's a copy of the WSJ article, though unfortunately the links seem to be deactivated, so there is no link in the parentheses that say "See full results".

A few thoughts.

Many of the comments in the thread refer to original thinking, which isn't really what the test was trying to measure. Apparently the critical thinking that the test claims to measure is whether students can "make a cohesive argument, assess the quality of evidence in a document or interpret data in a table".

While I am also suspicious of the method of testing different groups of frosh and seniors at approximately the same time, it's not really the changing abilities of the student body overall over the course of three years. Rather, unless every student in the class is tested (which apparently is not the case), that would leave open opportunities for student selection bias - either consciously (which presumably would tend to inflate improvements) or unconsciously (which might go either way).

Trying to use my own powers of critical thinking, I would say this quality of evidence in this particular document is not very good. The overall tone of the article is that colleges overall are not doing a good job improving critical thinking skills, but it never gives any overall scores, score changes frosh to seniors, or any reference as to what kind of scores and score improvement ought to be expected. However, what bothers me the most is how the article treats objections to the test. The article suggests that anyobody's comments on the test are merely self-interested spin, "Colleges where students perform poorly say it is unfair to draw sweeping conclusions from a single test. ... Colleges where students perform well on the test say it is an accurate gauge of their academic programs." There are a few quotations from people who fall into the second category, mostly from Plymouth State College, though those are mainly from particular professors bragging about their own courses rather than any analysis of the test itself. There is nothing from those in the first category, with comments like "Kentucky and UT Austin officials criticized the test and said they no longer use it." Well, what exactly were the criticisms of the test and is there any evidence those are valid or not?

I am personally biased that many majors at many colleges don't actually teach their students very much but this article doesn't really give me further information on the matter. Perhaps the poor critical writing skills displayed in this article by (presumably) college educated writers at the WSJ is evidence itself. 

[CTcard]

Here's a version of this story that's not behind a paywall.

Thanks for the link. I wasn't able to read this before.
Apparently that's a copy of the WSJ article, though unfortunately the links seem to be deactivated, so there is no link in the parentheses that say "See full results".

A few thoughts.

Many of the comments in the thread refer to original thinking, which isn't really what the test was trying to measure. Apparently the critical thinking that the test claims to measure is whether students can "make a cohesive argument, assess the quality of evidence in a document or interpret data in a table".

While I am also suspicious of the method of testing different groups of frosh and seniors at approximately the same time, it's not really the changing abilities of the student body overall over the course of three years. Rather, unless every student in the class is tested (which apparently is not the case), that would leave open opportunities for student selection bias - either consciously (which presumably would tend to inflate improvements) or unconsciously (which might go either way).

Trying to use my own powers of critical thinking, I would say this quality of evidence in this particular document is not very good. The overall tone of the article is that colleges overall are not doing a good job improving critical thinking skills, but it never gives any overall scores, score changes frosh to seniors, or any reference as to what kind of scores and score improvement ought to be expected. However, what bothers me the most is how the article treats objections to the test. The article suggests that anyobody's comments on the test are merely self-interested spin, "Colleges where students perform poorly say it is unfair to draw sweeping conclusions from a single test. ... Colleges where students perform well on the test say it is an accurate gauge of their academic programs." There are a few quotations from people who fall into the second category, mostly from Plymouth State College, though those are mainly from particular professors bragging about their own courses rather than any analysis of the test itself. There is nothing from those in the first category, with comments like "Kentucky and UT Austin officials criticized the test and said they no longer use it." Well, what exactly were the criticisms of the test and is there any evidence those are valid or not?

I am personally biased that many majors at many colleges don't actually teach their students very much but this article doesn't really give me further information on the matter. Perhaps the poor critical writing skills displayed in this article by (presumably) college educated writers at the WSJ is evidence itself.