Skip to content

Flip Fixation Flaw

Tuesday, April 28, 2009

I’ve decided to coin a new term to describe an issue I see in many flashcard applications and web sites. I have argued with developers over email and in comments about this issue but it is one I feel strongly about:

A flashcard application is said to be guilty of the Flip Fixation Flaw when the developer’s attempt to emulate the experience of studying and flipping a physical flashcard comes at the cost to the student of time or software functionality.

In principle, there is nothing wrong with creating a digital flashcard experience within educational software which is modeled upon the original physical paper based flashcard. This only becomes a problem when the marginal benefit of this approach is outweighed by the cost in terms of speed, space, and visual experience.

It’s Not Just about the Flipping

The three most common ways developers commit the flip fixation flaw are:

Lost Speed - When there is a non-trivial amount of time wasted upon creating the visual experience of “flipping” a digital flashcard to show another side of the card or sweep the card off the screen to show the next card. Because reviewing a well-studied flashcard can take less than a second, anything but the fastest flip animations can easily double the time it takes for a student to study a medium to high volume of flashcards.

If developers really want to flip the cards, then please: flip’em really fast!

Lost Space - In order to further give the user the impression they are looking at a physical card, developers often draw an image of a rounded or regular rectangle on the screen in which the content of the card is displayed and upon which the flip action is performed. This is a terrible waste of screen real estate because both the card and sometimes dozens of precious pixels are being wasted that might have been better put to slave labor portraying bigger fonts or more text.

Even if you want to “flip” the digital flashcard, we don’t need to waste screen space on a cute little flashcard, just flip the whole screen or canvas on the window.

Lost Memories - Some developers, who apparently really miss the real thing, like to create flashcards which reproduce the red vertical line and blue horizontal lines on ancient paper flaschards and elementary school notebooks.

Get over it, move on! Are you worried the users are going to type out of line or off the left margin? It may spark moments of nostalgia for people like myself who have actually used the brand of physical flashcards that looked like this, but no, we really don’t want to stare at the blue and red lines when we practice our flashcards.

Filed in Commentary | Comments (0)

FlashcardDb Review

Tuesday, April 28, 2009

FlashcardDb is one of the growing number of online flashcard review sites that allow you to review online and share flashcards with others. The site offers full data portability and support for both a static time-to-forget “Leitner” form of interval study as well as an interval study approach similar to that provided by Supermemo, Anki, and Mnemosyne. The developer posts some interesting comments on interval study (spaced repetition) on the site blog and also has a twitter feed one can follow.
(Continued)

Filed in Reviews, Web | Comments (1)

Cramberry Review

Saturday, April 25, 2009

If Flashcard Exchange (see my review here) is ugly but offers (at least to its $20 one-time fee premium users) access to interval study and export features as well as a huge database of cards, Cramberry seems to be the attractive new kid in town. Several people have contacted me asking me to post a review. Well, here it is, but to be honest, there isn’t much there to review. I hope to return to the site in a few months time and hope to see some progress.

Cramberry

Cramberry is like a beautiful little statue of a young cupid with cute little wings and a bow. It is pleasant to look at but it can’t do anything. The website has a beautiful clean design but almost no functionality so far.

However, Cramberry developers would be quick to point out they are still in the early stages of getting the project off the ground. Cramberry has its own twitter feed, its own blog, and a still very basic but promising $4 iPhone application, Flash-Me that connects directly with Cramberry.

Cramberry has a rating system built in for publicly available sets, which is probably better than the favorites count of Flashcard Exchange, and it offers a simple way to share sets between friends or make them publicly available.

Cards can be added quickly and easily with the use of the keyboard alone (pressing return after filling the back of the card saves the card and presents a new blank card). More advanced users can make use of its supported metalanguage textile to add formatting and images to the cards. This is a nice feature, but really not necessary now that there are great libraries for creating rich text input within webpages. Why force the user to memorize a metalanguage when you can provide buttons directly on the page for things like “bold” and “italic” etc.? Adding such buttons does clutter the interface but this is a case where functionality boost outweighs the decrease of a clean look (one could also simply add a link to “show/hide advanced editing tools”).

The website is clearly in its infancy and still has very far to go though before it can really be worthy of a full review but I hope to see it address some of these issues in future:

-Cramberry supports only two fields, which is very disappointing for students of Asian languages.

-Cramberry has No Support for Interval Study of any kind. The home page claims that “Cramberry records your progress on each card, and shows you cards you’re having trouble with more often, letting you study more effectively, faster.” which suggests that there is some kind of spaced repetition at work but users are left at the mercy of a “black box” approach without any inkling of how this process works. I hope the developers will describe there system in greater detail and offer ways of viewing the progress of individual words. It might be worth looking into some of the most powerful interval study implementations out there, such as those used in Anki, SuperMemo, or VTrain or simpler but clearly described ones such as those found in Flaschard Exchange with a simple static TTF schedule (See terms page).

-Cramberry has No Data Portability, cards cannot be downloaded in any format.

-Cramberry has almost no options for flashcard study and I can’t find any information on keyboard shortcuts.

-Cramberry seems to be connecting to Google Analytics between every card. Whatever else it might be doing, moving between cards is way too slow. There is a nice color coding of the sides of the card.

-There is no cycle elimination. The cards just continue to flip indefinitely. Basically this means it is guilty of the insatiability flaw.

-There are no statistics whatsoever.

-The website is beautiful but overly minimalist. Eliminating distractions when doing flashcard study is important, as I suggested when criticizing Flashcard Exchange, but surely someone on the design team could spare 15 minutes to put something on their help page other than “Contact us with your questions by twitter or email.” You can search for sets and show “more sets” but you have no way of going back through earlier pages in the search results, know how many total sets were found, etc.

-Many links are not universally available. The help link is not visible on the home page, the home page (which is the only page with any explanation of the site) is completely inaccessible to users who are logged in, etc.

I look forward to seeing Cramberry grow, but for now, I would recommend users consider other online and offline alternatives until it develops a basic feature set.

See also:

-My flashcard application Basics and explanation of Terms
-My introduction to online flashcard websites.

UPDATE: Cramberry posted an entry on their blog about the review. It is promising to see them acknowledging some of the issues and announcing some upcoming features on the site. Among them are a commitment to incorporating full interval study features, data portability, and statistics. I hope they will also strongly consider adding support for three fields, which is useful for students of many languages, especially Chinese and Japanese. Again, as I mentioned in the review, it is still an early stage for them so I look forward to revisiting the site and reviewing them when they follow through on some of their development goals.

Filed in Uncategorized | Comments (1)

Flashcard Exchange Review

Saturday, April 25, 2009

Now, let us take a look at some of the major offerings out there beginning with the most famous of flashcard websites:

Flashcard Exchange

Data Portability: None for free users. Export of flashcards allowed with one-time fee of $20 or for owners of the Mental Case iPhone application.
Interval Study: None for free users. Spaced repetition for premium users.
Fields: Normally 2. Three possible by using the hint field and a special option.

Flashcard Exchange is perhaps the most well known online library of flashcards and web site allowing the online review of such websites. I have watched this website grow through the years. It now dwarfs most of its competitors with the huge quantity of cards it offers in all languages.1

This website started ugly and despite years of growth and change, it is still ain’t pretty. Interface elements like tabs float strangely out of place in my Firefox browser when clicked and the whole design of the site is full of color inconsistencies and poorly thought out placements. Searching for flashcards is handled via google and the interface feels a little like the web from the late 1990s. The huge size of its database, however, does keep people from dismissing it entirely, however. It also prevents a mass exodus of its users by offering No Data Portability for its free users. It also provides No Interval Study for its free users.

For premium users cards found on the site can be exported in a wide variety of formats and can be studied using interval study. Interval study provided by flashcard exchange is a basic static TTF (Time to Forget schedule, see my Terms page) which advances from a spacing of 4 days for words at stage 1 to 11 years for level 14. Incorrect words have their TTF reset. My feeling is that the intervals increase too quickly for only 14 stages and the site should at least offer users the option of tailoring the interval schedule to their own memories (iFlash for OS X offers this ability, and Mental Case and Anki offer similar). However the site is to be commended for being one of the earliest online sites to appreciate the power of spaced repetition.

Flashcard Exchange does support three sided cards via a special option, but in a bizarre way: the third side must be included in the “hint” field which is then included in the rotation of each card.

The flashcard study itself is fairly smooth and allows you to continue studying incorrect cards (cycle elimination) but the flash screen is distracting with all the content included the window. There are keyboard shortcuts for studying but they are chosen without any thought to convenience of location (i for correct, x for incorrect - these two keys should be next to eachother. Same for p for previous card and n for next card). I also found that clicking is only accepted on the words of the card itself, not everywhere on the card which led to a lot of missed clicks. Flashing was also somewhat slow and there is no differentiation between the sides so it can sometimes be unclear what side is being viewed without looking at the top right (color coded or shaded sides is a better method). Overall the flashcard interface is way too cluttered with options. Most of the page should be stripped away, or a full screen option be permitted.

Finally, although the use of the site is tempting given the millions of cards it is host to, I have found that the quality of these flashcard collection is often incredibly poor. This is inevitable, given the huge number of users contributing, but one should choose a web site based primarily on functionality, and only judge the number of flashcards the site hosts if one finds quality flashcards for the textbook or language one is studying, not based on the total aggregate number of cards a site hosts. The only way to get some indication of the quality of the cards without looking closely is to to compare the “favorite count” which is the number of people who added the set to their favorites. It might be more useful to offer a more traditional rating system instead.

Mental Case users on the iPhone get free download access to the flashcards without a premium account and can study their cards on their iPhone or iPod touch so one is no longer tied to the online web version. This was a fantastic move for Mental Case and a boon to its users since it gave it immediate access to a large database of cards.

Overall, however, the design of the website leaves much to be desired, it has a very basic and inflexible interval study feature provided only to premium users, and also provides export only to premium users so many students will want to look elsewhere for their online study home.

  1. Flashcard Exchange is nearing 20 million cards, Quizlet.com claims over 24 million terms and close to half a million users [↩]
Filed in Web | Comments (0)

Online Flashcard Websites - Introduction

Saturday, April 25, 2009

There are a growing number of ways to practice flashcards online. In the next posting and possible more posts in the future, I will give very short reviews of some of the online solutions out there. First, however, let is list a few of the things I suggest students look for when they consider various online flashcard solutions:

Data Portability

Many of these websites are either advertising or subscription driven, or are at least contemplating these sources of income in the future. The more content they come to host, and the more traffic they attract, the more costly it becomes to manage such sites in terms of bandwidth, hosting costs, and labor. If monetization becomes a potential goal then these websites usually come to realize that the flashcards that their users upload the website, or which they provide for their users themselves, are their biggest asset. There is often, thus, a clash between the needs and desires of those running the site on the one hand, and those who use it on the other.

Sites will be very tempted to prevent users from downloading flashcards in a format that can be easily migrated to an offline solution or another website. If users can download flaschards, especially without paying for these flashcards that, in many cases, were typed up and uploaded by other users of the site, then they are essentially giving away “their” assets for free. Sometimes they will use excuses like copyright, which is a ridiculous argument since most such websites allow you to share your uploaded sets (often typed up from copyrighted language textbooks) with other users and some allow it only if you have paid for special “premium” features.

As users it is in our interests to avoid such “closed” web sites in favor of “open” websites which allow you to easily download any flashcards you have access to through the site in a format convenient to you. In reviewing the websites, therefore, I will lay heavy emphasis on data portability.

Other Features

The other major things I look for in an online flashcard solutions beyond the above key issue of data portability are:

1. Interval Study - Does the site provide a solid spaced repetition study system?

2. Fields and Unicode - Does the site provide the ability to review cards with 3 sides useful for studying Asian languages? Does it use Unicode and support non-roman characters?

3. Does it provide a good range of statistics on your study.

4. Does the site make good use of Javascript and or Ajax technologies so that flashcards are loaded quickly and cleanly without the page repeatedly reloading.

5. Does the site provide an easy way to share your flashcards with everyone who visits the site and a way to share with only a few people or optionally, with no one?

I also am looking for other things that I have listed on my Basics page.

See also my Terms page and Issues page.

Filed in Reviews, Web | Comments (0)

Facts and Cards in Flashcard Study

Tuesday, April 21, 2009

The vast majority of flashcard applications use cards as the fundamental unit of knowledge. If they record the user’s performance with a given flashcard in, say, the graded slideshow method, it is usually recorded for the card irrespective of the direction of study for that card. If the flashcard application uses a form of spaced repetition or interval study

Most flashcard developers realize however, that this approach has a certain disadvantage to it based upon a necessary assumption the developer makes about the user. Either the developer assumes the user is only interested in studying a card in one given direction (e.g. Russian to English) or they assume that one’s performance in either direction is roughly equivalent (if I know it in one direction Russian to English, then I probably know it in the other direction, English to Russian, roughly as well). This is, of course, rarely the case.1 While some language learners are primarily interested in studying in one direction, most will want to have both passive recognition and active recall of a vocabulary word, being both able to immediately recognize its meaning when read, but also recall the word when thinking of the word in one’s own native language.

Most flashcard applications allow you to control the direction of study or reverse it. iFlash (review upcoming) allows you to reorder the two or three or more sides of a card shown or even randomize the direction of cards but in interval study it records performance for all directions together. Mental Case (see my review) allows you to designate the “note reversibility” for a card, and can thus prompt you in multiple directions for a card, but again this information is not recorded independently.

There are a number of methods for confronting this problem but the most elegant approach I have seen so far is taken by the most advanced software applications such as Anki (see my earlier review) and I believe that other flashcard developers would do well to consider emulating the approach.

Anki has chosen for its the fundamental unit of knowledge a ‘fact’ which consists of, for example, a word and its definition, and in the case of three sided Asian language cards, its pronunciation. The difference between this and the card approach is that a fact that a fact can be a mother to two or three cards each with their own intervals and directions. Anki stores the fundamental ‘fact’ information independently of the users performance in the application and when you enter information for the ‘fact’ the application can be configured to automatically add two or three cards associated with that card in the various directions that the user wants to study.

For example, if I’m studying Japanese or Chinese, I may want to be able to record, independently, my performance across time in interval study in all three of the following directions: 1) be shown the English and then guess the Kanji/Hanzi characters as well as their pronunciation (and tones in the case of Chinese) 2) be shown the pronunciation (and tones) and be able to guess the characters and meaning in English and 3) be shown the Kanji/Hanzi character and be able to guess the pronunciation and English meaning. In most cases I think learners of Chinese and Japanese will at least want to review in directions (1) and (3) so advanced flashcard applications like Anki allow you to automatically create cards to that effect when the facts have been inputted. These cards then progress along their own interval study schedule as you review them.

Because, however, Anki maintains a relationship between cards and facts, when the content of the original fact is changed, so too is the content of any cards associated with the fact. This is a big advantage over other applications which might be tempted to simply let the user create duplicate cards in other directions. In another advanced interval study focused application Memosyne (see my review), for example, you can create “Vice versa” cards which then tracks the performance on the reversed card separately. However, if you then go and correct or edit a card, it will not correct the corresponding reversed card. Anki does not suffer from this problem.

Given this fact to card relationship, however, it means there is a significant separation between applications which acknowledge the relationship and those which don’t which impacts the full exchangeability of data between them. If you export data from Anki you get cards, without the preserved link between cards that are born of the same fact.

I hope that flashcard application developers reflect on the benefits of this approach and the power it gives to users in maintaining large collections of flashcards that need to be memorized in multiple directions. Implementation of this kind of feature likely requires adding a layer of complexity to the way that the data is maintained, but I have become convinced that the advantages are truly significant, especially for those who want to engage in long-term study.

  1. Though I do find that active mastery of a word, going from, say English to Russian, makes it far more likely that you know the reverse. [↩]
Filed in Commentary | Comments (3)

Chinese Flashcards iPhone/iPod Review

Sunday, February 15, 2009

cf1.jpg

Chinese Flashcards is an iPhone/iPod application created by the developers at ChineseLearnOnline.com. The interface is extremely straightforward and simple, allowing students of Chinese to review some 2700 most frequently used Chinese characters in either their simplified or traditional form going from the character to its English approximate meaning and pinyin with tone marks. The application provides a graded slideshow broken into fairly compact rounds of study called “sessions” which are components of the whole study environment called a “test.”

Performance in each session is fed into a form of interval study that prompts the user at a later point to review words mistaken. Users can begin any given session again or erase all interval study data and start the “test” again from scratch.

Application Name: Chinese Flashcards
iTunes Application Link: Chinese Flashcards
Version Reviewed: 1.00
Software License: Commercial (about $5)
Review Date: 2009.02.15
OS Tested: iPod Touch 2.2

Note: This review is from the perspective of language learners, and especially those who will be engaged in high-volume and long-term study of vocabulary. See the Terms page for an explanation of the technical terms used in these reviews. See the Basics page for a list of basic features found in flashcard applications useful to language learners.

The website for the application has a very nicely written, complete with images, description of the features of the application which I recommend looking at. Beginning students of Chinese may also want to see the newest application they are offering which focuses on vocabulary rather than individual characters.

The application has one extremely frustrating aspect that I hope will be rectified in future releases: it has extremely slow transitions between cards and card sides. The time taken to produce animation for removing the visual tab covering up the answer and flipping the card is much longer than it takes for users to recognize and offer feedback on already known characters. Some visitors to this site have said they disagree with my complaints about this frequent problem in Mac and iPhone/iPod applications but I think it is a very serious problem for high-volume students and the feature should at least be optional. In my estimates with this application, the time taken to animate the removal of the tab and flip the Chinese character card triples the total study time for 25 already confidently known cards. Given 10-20 minutes of study per day, over a month or several months, this translates into very severe waste of study time lost to viewing animations. This is worse than the lost time I have seen in any other flashcard application on the iPhone/iPod so far. Given most students of language engaging in flashcard study will want to maximize the efficiency of their daily review, this UI problem is crippling and needs to be addressed before it can be recommended for serious students of Chinese.

On the good side, this is one of few applications which recognizes the needs of many students who wish to learn the traditional forms of characters who are studying in Taiwan, or who, for other reasons, wish to master the traditional forms. It also allows students to skip characters and leave them out completely from the loop (but it would be nice to offer the ability to selectively reintroduce them if necessary without restarting the “test” completely).

Although this language targeted flashcard application is priced reasonably for one that includes both content and a form of interval study, it can benefit from some improvements in future versions, in addition to resolving the above critical UI problem:

-Users are left completely in the dark about their interval study performance or about how interval study is carried out on the application, let alone giving them some control over the process. They have no way to tell how far any given card has progressed in the interval study process, or an overview of their study except for a basic summary of a session. This application really could benefit from adopting at least some of my recommendations related to statistics in interval study applications. If you are a user of this application, consider requesting some of these features from the developers.

-While this application has interval study to a certain degree, it suffers from a classic case of The Cookie Monster Flaw. It considers all characters that have been correctly reviewed 4 times as “memorized.” Hopefully this will be addressed in future updates and a more thorough interval study spaced interval system will be implemented that keeps in mind that nothing lives in memory forever.

-The 2700 most frequent characters said to make up about 95% of the characters used in newspapers. But 95% is not as good as it sounds. We think of this as an “A” or even an “A+” but it translates into, by my estimate (confirmed by my own experience over the years), around 10-20 Chinese characters per average page of a Chinese book that are not recognized. Perhaps half of these can probably be inferred from context, that still often results in more than half a dozen characters that need to be looked up, or simply ignored. I would recommend serious students of Chinese to use applications that provide a set of 3500 (or more) Chinese characters.

-Since the application already nicely provides smaller sessions of cards, it would be best if a form of cycle elimination was provided before sending these cards into the distant interval study future. This can dramatically help student performance in the future and prevents them from having to manually start a session over.

If the slow visuals are made optional, the cookie monster flaw is resolved, cycle elimination introduced, and some useful statistics for users are included, this could be a major contender in the flashcard market for students of Chinese wanting to master 2700 characters.

Filed in Commercial, iPhone, iPod | Comments (0)

iFlipr Review

Saturday, January 31, 2009

iflipr.png iFlipr is one of the leading general purpose flashcard applications for iPhone/iPod which offers interval study and the recommended graded slideshow approach. I see great potential for this application. The clean and powerful web counterpart, in particular, is impressive, and the web centered approach may indicated a general direction for applications in the future.

My review below points out many strengths of this application but also points out some issues with the flashcard interface which will frustrate high-volume students, as well as some of the limits of interval study which will concern long-term students. (Continued)

Filed in Commercial, Web, iPhone, iPod | Comments (10)

The Insatiability Flaw

Friday, January 30, 2009

I have started to compile a list of common issues and problems I see in the various flashcard programs I have been reviewing. I have created a page which will gather this information for developer reference: The Issues Page

The insatiability flaw is exhibited by flashcard applications which provide no pause or “completion” during the course of interval study in such a way that recognizes that some cards are not currently in need of review. It is problematic because it provides users with no way to efficiently manage their time by endlessly prompting them to review cards which the student is not likely to be on the verge of forgetting. Obviously, they can and should be continually prompted if there are indeed words that are untested or which are due for review, but otherwise, the application shouldn’t drag out cards that need not be reviewed for weeks or months (Though they should provide a way for users truly eager to continue reviewing, or offer a comprehensive study on demand feature).

Students turn to flashcards because they believe their study is a useful and efficient form of study. Most of them recognize that, when it comes to vocabulary acquisition and maintenance for example, it is an inferior method when compared to the frequent and sustained production and practice of a language in an organic communication setting. However, given that we do not always find ourself in such a setting, or find the range of our communication more limited than that needed for reaching and preserving our desired level of proficiency, flashcard study is an imperfect but helpful alternative.

However, given that our time is a limited resource and we may be engaged in flashcard study of multiple languages or sets of knowledge units generally defined, flashcard applications which are “insatiable” in their appetite to prompt us to review words give us no indication of when we have completely reviewed all words we are on the verge of forgetting.

The best way for a developer to think about this when designing their application is to remember that while it is crucial that the developer avoid the more serious Cookie Monster Flaw they should also keep this principle in mind:

An interval study system nears perfection the further it approaches an environment which only prompts a student to review those units of knowledge they are on the verge of forgetting.

If there is some advanced algorithm included which takes into account the user’s truancy it may want to provide early prompting of words that are not quite yet on the verge, but only based on some statistically guided expectation of future truancy of the user.

Filed in Issues, Uncategorized | Comments (0)

The Cookie Monster Flaw

Friday, January 30, 2009

I have started to compile a list of common issues and problems I see in the various flashcard programs I have been reviewing. I have created a page which will gather this information for developer reference: The Issues Page

The first problem I write about on that page, I have called the “Cookie Monster Flaw” and so far I found it in both Mental Case and the “burn wheels” of Mindburn.

In the memorable words of a failed Sesame Student, “Me not continually review cookies at increasingly spaced intervals, me eat the cookie.” This flaw is, of course, exhibited by any flashcard software which offers no interval study but it is included here because it is also a problem with some approaches of applications which actually do support interval study: If an interval study system offers a finite number of interval stages, at which point the interval study for a given unit of information is said to be completed, it is guilty of the Cookie Monster Flaw.

The idea for this comes from the practice, prevalent among some students in places such as China, Korea and Japan, of learning vocabulary directly from a dictionary. In an ritual act of symbolism, once all the words, or chosen words, from a given page of a dictionary have been “memorized” the page is ripped out of the dictionary and consumed. My experience, admittedly not backed by evidence from a full empirical study, suggests that thus physically digesting the ink and highlighter fluid of memorized words does not in fact have much of an impact on long-term retention of vocabulary.

Developers should keep this principle in mind:

Even native speakers forget their own languages during extended periods of disuse; no interval study system should be designed in such a way that implies the complete and final memorization of any unit of information.

Filed in Uncategorized | Comments (0)