Sunday, May 22, 2005

What would we want in a really good CMS?

A little bit more than a year ago, Chris Vento, CTO at WebCT wrote a column in Campus-Technology magazine where he argued for Open Standards in CMS development done by commercial ventures (such as WebCT) as an alternative to Open Source. I wrote a response to that (partly at the urging of my colleague Steve Acker who edits the column) arguing that the business discipline commercial companies bring is probably good in many respects, but one consequence is that we get incrementalism in terms of improvements in the CMS rather than dramatic pedagogic change. The argument is that dramatic change is inherently risky and the commercial vendors can't afford to fail.

But we in academe can. So we should be the ones to develop the more interesting pedagogic applications. And we should be the ones to pilot them. Many will fail, if not technically then because they don't generate sufficient interest. The ones that succeed can then be brought into the commercial environments. This is the juncture that departs from open source thinking. Consequently, this might sound like sacrilege to many of my colleagues. Why should we give away to the vendors the intellectual property that we've developed? (I've heard that refrain over and over again.) The answer is simple. We move on. When we do, what we developed earlier that at the time was an asset becomes a liability. Now we've got to maintain it, but our interest is elsewhere. Let the vendors maintain it instead. And let them earn a decent return on their software so they have incentive to do just that.

Let's begin by noting that most faculty currently use the CMS to put up their content. But this content is mostly documents; PowerPoint is the quintessential example. In the main, when we talk about instructors improving the online part of their course pedagogically, we talk about them using other tools, such as the discussion board, that add human to human interaction. What about improving the presentation content itself?

Way back when, about ten years ago Alfred Hubler, a faculty member in the Physics department, was working on a software system he developed called CyberProf. His goal was for his system to enable really good content. I got friendly with Alfred over the next few years and I know from discussions with him that Plato was a source of inspiration and that he looked at Mathematica rather than other course management systems as the main rival. The idea that the CMS can enable really good content is why we started to support Mallard in SCALE and why our Biology units on campus now use LON-CAPA.

What does really good content look like? Here I'm going to take an extreme position. It's not the graphical design or the Flash animation that matters. (Elegance and simplicity in presentation do matter, but bear with me for a bit.) What matters is that the content promote critical thinking. The content must help the student visualize the idea and reason through the material. Furthermore, the content should take advantage of the computer and display.

I'm going to be unfair and pick on a product that some of my staff like, StudyMate by Respondus. This is an easy to use tool that makes Flash objects one can use in the CMS. These objects remind me of HyperCard, the old Apple tool. Indeed among the objects one can make with StudyMate are Fact Cards, Flash Cards, and simply quiz questions, all nicely designed. These would be really great for my kids (5th and 7th grade) for studying social studies or language arts. But is that what we want to be promoting in college? I can see it being used in a foreign language class or a class that has a huge amount of terminology. But in the main, I don't see it.

A lot of the better online instructional content is done via Java applets or some other software that enables animation. For example, here is a nice page on the Monty Hall Paradox. I like that page a lot in terms of what it delivers both to the student and to other statistics instructors. But it leaves as a mystery how to make the applet. So those that can do. Most instructors can't and therefore don't.

Now I'm going to talk about my own stuff made with Excel, because I know what I did in designing the content. Here Excel serves as an alternative to a CMS, both for the simulation and for automatically graded questions based on the simulation. I think a good CMS should be able to allow this type of content and I'll explain how as I go along and also what Excel contributes in this regard.

This is an exercise from my principles of economics course. You must Enable Macros to have this function properly. Assuming that, open the file and fill in the info on the login sheet. (This file is sitting on your own computer. You are not giving away any personal information. Lie about your age and say you were born after 1980.) Then go to the ReservationPrice worksheet.

The upper pane is "the experiment area" and you should leave it fixed. The lower pane is "the quiz area" and you should scroll down as you progress through the exercise. Push the buttons in the upper pane to change the values in the graph and the table. The price buttons affect the table and the pink point. The scenario button only affects the graph.

After you've finished experimenting reset both prices to zero and then proceed to answer the questions in the bottom frame. See if you can get to part B of the quiz, if not all the way through. (I'm betting that if you get to part B you will finish the quiz.) The design is so that you can't make progress unless you get the previous question right.

Now let's consider what is going on here. The buttons in the upper pane affect the data value in some cell in the spreadsheet. Change the value in the cell and change the data table or the graph or both. The quiz questions are of the following general form: If the answer is correct more content is revealed which includes a response in bold green and then another question. If the answer is incorrect, some other content is revealed in red italic. The assessment of whether correct or incorrect involves satisfying an equality or not (the first two questions in part A) or satisfying an inequality or not (the questions in part A that ask for a price pair).

One other point I'd like to mention, which is something I learned from some of the Plato programmers who came over to the IT organization after Plato was licensed off as NovaNet. The changes render quickly. The entire page does not refresh. Only the content that has changed refreshes. Since the Excel file sits on your local computer, not on the server, this happens very quickly. This quick response is necessary to keep student interest.

Most current course management systems can do some of these functions ---- in their grade book. They can render data graphically via a histogram of the scores. WebCT Vista has a very nice query tool that can query data columns on whether an equality or inequality holds and return all values for which the statement is true. The grade book is much more sophisticated than the quiz engine in this respect. Why not allow data tables that are grade book like, but that are used to generate content a la my economics exercise? There are quite a few courses that would utilize numeric content of this sort, if they saw the pedagogic benefit.

Would instructors create this type of content? (I probably spent a month to make that workbook, but a fair amount of that was on figuring out what to show. You won't find these type of diagrams in any econ text.) My view is this. If they think they have to make it all themselves, they won't do it. (And then it would be rational for the CMS vendors to completely ignore the ideas in this post.) But they might make one or two to show what they have in mind and then either (a) develop a community with colleagues at other schools who do likewise and trade them or (b) have their students do these in lieu of some other course project and then re-use the good ones.

No comments: