Spaced Repetition Research + Knowt Algo Planning
Knowt Implementation of Algorithm
Flow:
User goes to the note editor screen and clicks “Study Button”
If they haven’t made flashcards → create flashcards and show them on the note
They can then add more flashcards or continue the study process
If they have made flashcards → let user pick how they want to study
Study Flashcards
When do you want to know the material?
Ask users to place their answers into one of the boxes and use our algo to show the cards
Too Soon
I forgot
Not sure
Too easy
Keep track of terms studied
Track progress independently from quizzes
Show progress bar at the top
Learn Concepts
How well do you know the notes already?
When do you want to know the material?
Quiz:
Instant Feedback only
Create a large quiz and have the user go through the questions
Spaced out with their choices
Give “breaks” every 10 question to recap information (maybe show this in the UI)
Ask users to place their answers into one of the boxes and use our algo to show the questions
Too Soon
I forgot
Not sure
Too easy
Bad question box? (Only for quizzes)
Track progress independently from flashcards
Show progress bar at the top
You should be able to resume either from where you left off
How do we handle switching between the 2?
Show 2 buttons?
Let them switch in the settings if they’ve started
Just ask them at the end
Algorithm:
We use the same three phases: Learning, Exponential, Relearning
We adjust the values of each variable based on the following factors:
The date of their exam
Intervals should change based on the time we have
How well they know the note
Should adjust the quiz difficulty (nothing we can do for flashcards)
Quiz
Should follow showing the 4 or 5 buttons
Buttons should override the user input
Got it wrong → Clicking easy overrides (mark it as easy)
Got it right → Clicking hard overrides (mark it as hard)
Look into defining or adjusting the algo based on incorrect/correct and response to the buttons
Repeat questions as we are repeating flashcards
Notifications
Notify the user to come back (nudge) and continue studying
See how this adjusts based on the timeline that the user gives us
2 days study period → multiple notifications during the day (find a good balance)
7 day period → one notification per day
Questions
How do we handle the mix of flashcards and quiz questions
Value is assigned to each term
Proposed → Make them independent for quiz/flashcards[a]
Anki’s System
Video: https://www.youtube.com/watch?v=cUW10l4ZZjg&ab_channel=RevisingRubies
Follows the forgetting curve
Learning starts off steep so you are forgetting a lot constantly
Curve starts to get shallower as you add learn it
Spaced Repetition helps to get this curve shallower
Cards have 4 options: Again, Hard, Good, Easy
The Space Between Seeing Cards (intervals) normally increases by a factor of 2.5
Day 1, 3, 8, 21, 54, 137
Intervals are 2, 5, 13, 33, and 83 days (see how 5 = 2*2.5, 13 = 5 *2.5)
Learning Phase → Cards on Day 1 and 3 are said to be in this phase (New Cards too)
Follows the intervals for 1 minute and 10 minute
Again → 1 minute
Good → 10 minutes
When you press Good two times in a row (aka after 10 mins) then the card Graduates
Graduation Phase
First interval after the learning phase (Day 3 → So between Day 3 and 8) is now set as 1 day and you don’t see that graduated card until 1 day after
Intervals after that increase by 2.5
If you press Good you keep seeing that card until you indicate you forgot it
Lapse Cards
If you forget a graduated card it resets
You see it after 10 minutes until it graduates (You say “good”)
This time once the card is graduated its multiplication factor is 2.3
Decrease by 0.2 every time you forget
Each card has its own multiplication factor
Edge Cases - If the user doesn’t come back to the app after the desired space it adjusts the spacing
Anki considers the actual time the card was unseen, not just the time it was scheduled for
Ex: If the card was scheduled for 5 days but you don’t study for a month
Good → the next interval goes from 10 days to something like 20 days (Factor of 2.5)
Hard → Less than last wait (Factor of 1.2)
Easy → Aggressively long interval (Factor of 3.25)
Logic: If you got it right after a long time you don’t need to see it that soon
Types of Flashcards
Basic: One directional
Reversed: Two directions (learning vocabulary)
Cloze Deletion: Fill in the blank
Overlapping Cloze: Not free
Basic Image Card: Images with one direction
Image Occlusion: diagram labelling (an add-on)
Anki-Sm2 Spaced Repetition Algorithm
Algorithm Values: https://www.remnote.com/p/help/document/8g5xHDmGSotpAR9Pc
Learning Phase:
Fixed Preset intervals
1, 2, 4 days
Graduate by either of these 2:
Pressing easy (far right option)
Or finishing the fixed preset intervals
Exponential Phase:
Computed by: Multiplier * (Last Scheduled Interval + Days Late / Hardness Divider)
Multiplier: Computed from Last Score (Hard = 1.2, Good = Ease, Easy = Ease * Set Bonus) + Ease
Ease is unique to each card
Every card starts at 2.3 and is bounded between 1.3 and 2.5
Again (-0.2)
Hard (-0.15)
Good (No Change)
Easy (+0.15)
Last Scheduled Interval: Days in the previous interval
Days Late: Number of days your card was waiting after its set study time (edge case if the user doesn’t return to study on set date)
Hardness Divider: Fixed from last score (Hard: 4, Good: 2, Easy: 1)
Relearning Phase:
Activates when you press Again while the card is in the Exponential Phase
Goes back to fixed intervals
Graduate by either of these 2:
Pressing Easy (far right option)
Or finishing the fixed preset intervals
New interval after graduation is computed by:
Multiplier * (Last Scheduled Interval + Days Late + Last Exponential Interval * Lapse Interval Multiplier[b]) / Hardness Divider)
Timings & Buttons
Every card can be in the “Learning”, “Almost-Graduated” and “Graduated” case. Based on the timing we might have an “Exam next day” schedule.
The “Learning” case has the following schedule for delays for buttons: (0%)[c]
Too early | in 2 hours |
---|---|
I forgot | in 5 mins |
Recalled with effort | in 1 hr |
Partially recalled | next day |
Easy | in 2 days |
The “Easy” brings the card in the “Almost Graduated” state
All the rest retain card in the same state (Learning)
The “Almost-Graduated” case has the following schedule for delays for buttons: (50%)
Almost-Graduated case |
---|
I forgot |
Recalled with effort |
Partially recalled |
Easy |
The “Easy” brings the card in the “Graduated” state
All the rest reset card to the learning state
The “Graduated” case has the following schedule for delays for buttons: (100%)
Graduated case |
---|
I forgot |
Recalled with effort |
Partially recalled |
Easy |
The “Easy” brings the card in the “Graduated” state
The “Partially recalled” brings the card in the “Almost-Graduated” state
All the rest reset card to the learning state
The “Next day exam” case has the following schedule for delays for buttons (Excludes cards in the graduated state):
Next day exam case |
---|
Too soon |
I forgot |
Recalled with effort |
Partially recalled |
Easy |
[a]answered
[b]Can be configured between 0 and 100
[c]Need values for 2 days
[d]Thought this was going to be based on the exam date?
[e]Also how are these calculated given the delays in the Learning state
[f]Thought this was going to be based on the exam date?
[g]Also how are these calculated given the delays in the Learning state
[h]Thought this was going to be based on the exam date?
[i]Also how are these calculated given the delays in the Learning state
[j]Thought this was going to be based on the exam date?
[k]Also how are these calculated given the delays in the Learning state
[l]Based on the exam date? Relation to previous "Almost-Graduated case?"
[m]Based on the exam date? Relation to previous "Almost-Graduated case?"
[n]Based on the exam date? Relation to previous "Almost-Graduated case?"
[o]Based on the exam date? Relation to previous "Almost-Graduated case?"
Knowt Implementation of Algorithm
Flow:
User goes to the note editor screen and clicks “Study Button”
If they haven’t made flashcards → create flashcards and show them on the note
They can then add more flashcards or continue the study process
If they have made flashcards → let user pick how they want to study
Study Flashcards
When do you want to know the material?
Ask users to place their answers into one of the boxes and use our algo to show the cards
Too Soon
I forgot
Not sure
Too easy
Keep track of terms studied
Track progress independently from quizzes
Show progress bar at the top
Learn Concepts
How well do you know the notes already?
When do you want to know the material?
Quiz:
Instant Feedback only
Create a large quiz and have the user go through the questions
Spaced out with their choices
Give “breaks” every 10 question to recap information (maybe show this in the UI)
Ask users to place their answers into one of the boxes and use our algo to show the questions
Too Soon
I forgot
Not sure
Too easy
Bad question box? (Only for quizzes)
Track progress independently from flashcards
Show progress bar at the top
You should be able to resume either from where you left off
How do we handle switching between the 2?
Show 2 buttons?
Let them switch in the settings if they’ve started
Just ask them at the end
Algorithm:
We use the same three phases: Learning, Exponential, Relearning
We adjust the values of each variable based on the following factors:
The date of their exam
Intervals should change based on the time we have
How well they know the note
Should adjust the quiz difficulty (nothing we can do for flashcards)
Quiz
Should follow showing the 4 or 5 buttons
Buttons should override the user input
Got it wrong → Clicking easy overrides (mark it as easy)
Got it right → Clicking hard overrides (mark it as hard)
Look into defining or adjusting the algo based on incorrect/correct and response to the buttons
Repeat questions as we are repeating flashcards
Notifications
Notify the user to come back (nudge) and continue studying
See how this adjusts based on the timeline that the user gives us
2 days study period → multiple notifications during the day (find a good balance)
7 day period → one notification per day
Questions
How do we handle the mix of flashcards and quiz questions
Value is assigned to each term
Proposed → Make them independent for quiz/flashcards[a]
Anki’s System
Video: https://www.youtube.com/watch?v=cUW10l4ZZjg&ab_channel=RevisingRubies
Follows the forgetting curve
Learning starts off steep so you are forgetting a lot constantly
Curve starts to get shallower as you add learn it
Spaced Repetition helps to get this curve shallower
Cards have 4 options: Again, Hard, Good, Easy
The Space Between Seeing Cards (intervals) normally increases by a factor of 2.5
Day 1, 3, 8, 21, 54, 137
Intervals are 2, 5, 13, 33, and 83 days (see how 5 = 2*2.5, 13 = 5 *2.5)
Learning Phase → Cards on Day 1 and 3 are said to be in this phase (New Cards too)
Follows the intervals for 1 minute and 10 minute
Again → 1 minute
Good → 10 minutes
When you press Good two times in a row (aka after 10 mins) then the card Graduates
Graduation Phase
First interval after the learning phase (Day 3 → So between Day 3 and 8) is now set as 1 day and you don’t see that graduated card until 1 day after
Intervals after that increase by 2.5
If you press Good you keep seeing that card until you indicate you forgot it
Lapse Cards
If you forget a graduated card it resets
You see it after 10 minutes until it graduates (You say “good”)
This time once the card is graduated its multiplication factor is 2.3
Decrease by 0.2 every time you forget
Each card has its own multiplication factor
Edge Cases - If the user doesn’t come back to the app after the desired space it adjusts the spacing
Anki considers the actual time the card was unseen, not just the time it was scheduled for
Ex: If the card was scheduled for 5 days but you don’t study for a month
Good → the next interval goes from 10 days to something like 20 days (Factor of 2.5)
Hard → Less than last wait (Factor of 1.2)
Easy → Aggressively long interval (Factor of 3.25)
Logic: If you got it right after a long time you don’t need to see it that soon
Types of Flashcards
Basic: One directional
Reversed: Two directions (learning vocabulary)
Cloze Deletion: Fill in the blank
Overlapping Cloze: Not free
Basic Image Card: Images with one direction
Image Occlusion: diagram labelling (an add-on)
Anki-Sm2 Spaced Repetition Algorithm
Algorithm Values: https://www.remnote.com/p/help/document/8g5xHDmGSotpAR9Pc
Learning Phase:
Fixed Preset intervals
1, 2, 4 days
Graduate by either of these 2:
Pressing easy (far right option)
Or finishing the fixed preset intervals
Exponential Phase:
Computed by: Multiplier * (Last Scheduled Interval + Days Late / Hardness Divider)
Multiplier: Computed from Last Score (Hard = 1.2, Good = Ease, Easy = Ease * Set Bonus) + Ease
Ease is unique to each card
Every card starts at 2.3 and is bounded between 1.3 and 2.5
Again (-0.2)
Hard (-0.15)
Good (No Change)
Easy (+0.15)
Last Scheduled Interval: Days in the previous interval
Days Late: Number of days your card was waiting after its set study time (edge case if the user doesn’t return to study on set date)
Hardness Divider: Fixed from last score (Hard: 4, Good: 2, Easy: 1)
Relearning Phase:
Activates when you press Again while the card is in the Exponential Phase
Goes back to fixed intervals
Graduate by either of these 2:
Pressing Easy (far right option)
Or finishing the fixed preset intervals
New interval after graduation is computed by:
Multiplier * (Last Scheduled Interval + Days Late + Last Exponential Interval * Lapse Interval Multiplier[b]) / Hardness Divider)
Timings & Buttons
Every card can be in the “Learning”, “Almost-Graduated” and “Graduated” case. Based on the timing we might have an “Exam next day” schedule.
The “Learning” case has the following schedule for delays for buttons: (0%)[c]
Too early | in 2 hours |
---|---|
I forgot | in 5 mins |
Recalled with effort | in 1 hr |
Partially recalled | next day |
Easy | in 2 days |
The “Easy” brings the card in the “Almost Graduated” state
All the rest retain card in the same state (Learning)
The “Almost-Graduated” case has the following schedule for delays for buttons: (50%)
Almost-Graduated case |
---|
I forgot |
Recalled with effort |
Partially recalled |
Easy |
The “Easy” brings the card in the “Graduated” state
All the rest reset card to the learning state
The “Graduated” case has the following schedule for delays for buttons: (100%)
Graduated case |
---|
I forgot |
Recalled with effort |
Partially recalled |
Easy |
The “Easy” brings the card in the “Graduated” state
The “Partially recalled” brings the card in the “Almost-Graduated” state
All the rest reset card to the learning state
The “Next day exam” case has the following schedule for delays for buttons (Excludes cards in the graduated state):
Next day exam case |
---|
Too soon |
I forgot |
Recalled with effort |
Partially recalled |
Easy |
[a]answered
[b]Can be configured between 0 and 100
[c]Need values for 2 days
[d]Thought this was going to be based on the exam date?
[e]Also how are these calculated given the delays in the Learning state
[f]Thought this was going to be based on the exam date?
[g]Also how are these calculated given the delays in the Learning state
[h]Thought this was going to be based on the exam date?
[i]Also how are these calculated given the delays in the Learning state
[j]Thought this was going to be based on the exam date?
[k]Also how are these calculated given the delays in the Learning state
[l]Based on the exam date? Relation to previous "Almost-Graduated case?"
[m]Based on the exam date? Relation to previous "Almost-Graduated case?"
[n]Based on the exam date? Relation to previous "Almost-Graduated case?"
[o]Based on the exam date? Relation to previous "Almost-Graduated case?"