AI-generated Anki cards are not to be trusted
Learning incorrect things is bad
Recently I got very excited about using Anki to help me memorize things about biology. In my previous post I wrote that I had asked Claude to generate 100 Anki cards, and only 2 had problems:
Out of the first 100 cards, I caught two errors (one was using yeast gene names for human meiosis, the other was because the model didn’t know what the TTM complex was). So I do have to be vigilant, and I wouldn’t use AI-generated cards on topics where I couldn’t tell if something seemed wrong. This method is most useful for solidifying information that I already know vaguely, rather than learning completely new things.
Well, there was actually a third Anki card that had seemed correct on first glance but was actually subtly yet catastrophically flawed.1 This led me to make an error in designing an experiment, which was thankfully pointed out by one of my team members at Ovelle before we actually did the experiment. But if this hadn’t been caught, it would have caused a lot of wasted time and money.
I did a detailed check of all my other Anki cards and they look OK for now, but at this point I don’t think using LLMs to generate Anki cards is worth the risk. I still like Anki, but the issue is that learning incorrect things is really bad.
Addendum: after a commenter suggested I use Gemini instead of Claude 4 Opus, I gave the deck to Gemini 2.5 Pro and it did not find Claude's signaling gradient error. Also it said a correct card was erroneous. The prompt was:
Please check the following developmental biology Anki deck for errors. I know it contains at least two errors and possibly more. For each error that you find, output the incorrect card and a corrected version.
And even when I specifically pointed Gemini 2.5 Pro to the defective card it still failed to correct the issue.
The card was about a signaling molecule gradient involved in a certain developmental process. It named the correct molecule but got the direction of the gradient backwards.
for sure. i've found the same. however they can be useful for putting already verified information into mnemonically optimal configurations. see dwarkesh's build here:
https://www.generateflash.cards/
Honestly, I wouldn't update super strongly in either direction on this.
"Professionally designed" flash cards sometimes contain "errors" in them; I find a particularly common form of errors in human-generated "professional" (as opposed to "passionate autodidact") cards is oversimplifying a nuanced belief into a definitive assertion that fits within a card.
Even if you manually make your own flash cards, if you make thousands of them over your life time, you'll probably make a typo at some point, and "reversing the direction of the gradient" sounds like a plausible typo one might make when designing one's own flash cards.