In my last post Bot Generated Stories II I left off posing the question:
How then did I get my bot to generate A Halloween Tale?
The short and most direct answer I can give without getting into all the nitty-gritty design & engineering details (we’ll do that in another post) is that I built a Markov Model (a math based bot) then I “trained” my bot on some texts to “teach” it what “good” patterns of text look like.
After my bot “read” approximately 350K words, the 3 books and a few smaller texts (some of my work and a few other general text sources to fill-out the vocabulary), I gave my bot a “seed word” that it would use to start the generative process.
What’s really going on “under the hood” is a lot like the “predictive text” feature many of you use every day when you send a text on your phone.
It’s that row of words that appears above or below your text while you type, suggesting the next possible word based on what you last said, what it was trained on and your personal speech patterns it learned while it watched you send texts in the past.
Well… my writer bot is sort of a “souped up”, “home-brewed” version of that…. just built with generating a story in mind rather than predicting the next most likely word someone sending a text would need.
The thing is, we’re not going to talk about that bot today. 😛
Xavier and I have been sick and I’m just not in the mood to do a math & code heavy post today, instead we’re going to talk about rule based story generation. 😉
Rule Based Story Generation
One simple and seemingly effective way to generate a story that absolutely works (at least to some extent) is to use “rules” to select elements from groups or lists and assemble the parts into a story.
The idea is pretty simple actually, if you select or write good rules then the result is they would generate unique (enough) sentences that when combined are a story.
For example, lets say I create a generative “rule” like this:
Rule: proximity location, event.
Seems simple enough but this rule can actually generate quite a few mildly interesting sentences, like this one for example:
in a railroad station, aliens attacked.
Result: (proximity: in) (location: a railroad station), (event: aliens attacked).
Not bad huh? You’d totally read that story right? 😛
Here’s a few more results using this rule so you can see what it can do. Note that the rule pattern never changes (proximity location, event.) but due to different values being randomly selected each sentence is different and the general “tone” of the sentence changes a bit as well:
Below a dank old bar, science breakthroughs were made.
I like that one 😛
Next to a village, a robbery happened.
To the west of a cave, a bird built a nest.
On the deck of a giant spaceship, a nuclear bomb was detonated.
Eat your heart out Kubrick! 😉 😛
Beyond the mountains, a child giggled.
Notice that all three parts of the rule (proximity location, event.) can affect the tone and meaning of the generated result.
What if the rule had generated:
“On the deck of a giant spaceship, a child giggled.”
That is a vastly different result than the one in the examples above, yet perhaps it is the same story with only seconds separating both events? Maybe…
“On the deck of a giant spaceship, a child giggled. The hoards of aliens were defeated. In the distance a voice yells “Dinner’s ready!”, a toy spaceship falls to the floor as little feet scurry unseen through the house.”
What makes the determination in the readers mind about what is actually going on is the context of what was said before this sentence and what will be said after. There are those cases where not saying something is saying something too… but dammit I can’t model that! 😛
Now, lets look at how the proximity can change the meaning.
Here’s the proximity list I used with this rule:
in inside outside near on around above below next to close to far from to the north of to the south of to the east of to the west of beyond
Each ‘proximity’ by itself seems pretty self explanatory in its meaning but when combined with a location the meaning can change. For example, it seems fairly natural to discuss something being ‘beyond’ something else like “the fence is beyond the water tower” but lets say that you have an ambiguous ‘location’ like Space?
1930s & 40’s Pulp Scifi aside… what does it mean to be “Beyond Space”? 😛
Clearly we’ve run into one of the limitations of rule based story generation, of which there seems to be many… but in this case I’m referring to unintended ambiguity.
At best a rule would reduce ambiguity and at worst it could inject significant ambiguity into a sentence. Ambiguity in this case should be understood as lack of clarity or a variance in what the reader is supposed to understand is occurring and what they believe is occurring.
Limitations aside, this type of rule based generative system is surprisingly effective at crafting those direct and matter of fact type statements.
The type of problem you could write an “If This Then That” sort of rule for… hmmm.
A Few More Rules
Here are a few more rules to help you get a feel for how this whole “rule” thing works:
Rule: name is very positive_personality_trait
Rule: name is very negative_personality_trait
See if you can tell which is which in this list:
Channing Lynn is very faithful Jerome Puckett is very defeated Arturo Thomas is very nice Damon Gregory is very grumpy Calvin Weeks is very repulsive Joaquin Hicks is very gentle Amanda Calhoun is very thoughtless Matthias Welch is very polite Carter Camacho is very scary Jay Dyer is very happy Harper Buckley is very helpless Trenton Bauer is very kind Kane Owen is very lazy Lauryn Vasquez is very obedient Aleah Gilmore is very angry Ameer Cortez is very brave Kase Wolfe is very worried
This rule is static and can be improved by having fewer “hard coded” elements.
Instead of the result always containing the word “very” you might instead have a gradient of words that are selected at random (or based on some precondition) that would modify the meaning or intensity of the trait, i.e. mildly, extremely, slightly, not particularly, etc…. which could lead to interesting combinations, we could call the gradient of terms, oh I don’t know… adverbs. 😛
Technically though, adverbs in general are too broad of a category to treat as simple building blocks in a rule like this but you could build a list of adverbs that would apply in this case and replace the word ‘very’ with a selection from that list which would result in more variation in the personality trait descriptions.
Lets look at another rule.
Rule: name is adjective_condition
Annalee Sargent is shy Hugh Oconnor is helpful Tessa Rojas is gifted Cristian Castaneda is inexpensive Heavenly Patel is vast Gibson Hines is unimportant Alora Bush is alive Leona Estes is mushy
I don’t know about you but…
I’ve always found that “mushy” people are very positive_social_anecdote! 😛
Are you starting to see how the rules work? 😉
Much like the rule above that could be improved by replacing the “hard coded” adverb (very) with a gradient that is selected at random (or based on some precondition) the verb ‘is’ in this rule could be replaced with a gradient of verb tenses i.e. is, was, will be, etc…
Now, if you want to get more complicated… you could even build a system that uses or implements preconditions as I mentioned above.
An example of a precondition I gave above was verb tense to determine if something has, is or will happen… which would then modify the succeeding rules that follow and are related to it… but it’s also possible to build preconditions that modify rules directly from properties that are innate to your characters, settings, objects in the locations, the weather, the time of day etc…
For example consider the Rule: name is a gender
This rule must be able to determine the gender of the name given to it in order for the rule to work. In this case, the gendered name would act as a precondition that modifies the result of the rule.
Reyna Dunlap is a woman Nikolai Cummings is a man Emerald Lynch is a woman Lucas Woodward is a man Bailey Ramsey is a woman Matias Miller is a man Tinley Hansen is a woman Mckenzie Davidson is a woman
It’s also possible however for a name to be gender neutral, like Jamie for example, and the rule cannot simply break if the name is both male & female or neither in the case of a new or non-typical name and that level of abstraction (care and detail given to each rule so as to prevent a rule from breaking) has to extend to all rules in all cases which is why using rules to write stories is impractical.
Related to the last rule is this Rule: name is a adjectives_appearance gender
Mallory Joseph is a clean woman Talon Vazquez is a bald man Kody Maxwell is a magnificent man Meredith Strickland is a stocky woman Jaliyah Haynes is a plump woman Brian Leblanc is a ugly man Collins Warren is a scruffy woman Tenley Robbins is a chubby woman Brantley Mcpherson is a chubby man Killian Sawyer is a fit man
Here again you see the rule must identify the gender of the name given to it… but what’s more important is that I used the “present tense” ‘is’ when its just as valid grammatically to say that “Killian Sawyer was a fit man“ and in fact even if it is grammatically correct to say he “is fit” he might not even be alive any longer and ‘was’ would be logically correct being the past tense with the implication the he is no longer fit rather than dead and additional information would be required by both the reader and system to make the determination that Killian was dead but the point still stands.
Using preconditions on all aspects of the story such as the characters, places, things etc. could enable the system to properly determine if it should say something is, was, will be, maybe, never was, never will be etc… it could examine the established who, what, when, where, why & how and use that information to determine what rule to use next which would progress the story further.
It’s easy to imagine how some rules would only be applicable if certain conditions had occurred or were “scheduled” to occur later in the story. Some rules might naturally form branching chains or tree hierarchies within the “flow” of the rules.
This implies if not requires some form of higher order logic, state machines and the use of formal logic programming with special care and attention given to the semantic meaning or value of each part of each rule…
Well nobody said it was going to be easy! 😛
These Are Not The Droids You Are Looking For
Sounds too easy right? Well… you’re probably right.
I mean sure you can do this in theory and next week I will provide a crude “proof of concept” example with code demonstrating the rules I used here today, but even if you create a bunch of rules it’s not like “earth shatteringly good” and you can’t just write them and you’re done, there is a lot of trial and error to get this sort of thing just right.
Personally I’ve never even seen a fully functional implementation of this type of thing… sure i’ve seen template based stories that use placeholders but nothing as dynamic as I believe would be required to make a rule based system work as described.
Again I am not talking about simply randomly selecting rule after rule… I mean sure you could do that but you won’t get anything really useful out of it.
To do this right your system would select the rules that properly describe past, present & future events correctly based on where in the story the rule is used and it can’t simply just swap out the names and objects or descriptions in your story without concern for the context that the rule is being applied.
To do rule based story generation right means that you get different stories each time the system runs not cookie cutter templatized tales. You cant simply write a story template and fill it with placeholders and then select and assemble “story legos” and get a good story.
Though at least hypothetically it could work if you wrote enough rules and built a more robust system that keeps track of the story state, the characters and their motivations, the objects, where they are, what they are etc… of course this is tedious and ultimately still amounts to you doing an awful lot of work that looks like writing a story.
I do believe (at least in theory) a rule based story generative system as described here could work but you would be limited to the manually written rules in the system (or are you? 😉 ) and how well the state machine used the rules.
Further, its debatable that even if a rule based story generation system worked, could it actually be good enough to be the “best seller” writer bot that we’re looking for?
Seemingly the major limiting factor to me appears to be hand writing, refining and testing the rules.
Suggest A Rule
As I said I will present the code for these rules in my next post but I’d like to ask you to suggest a rule in the comments for this post and I will try to include as many of them as possible in the code and I will give the suggester credit for the rule.
Please remember to like, share & follow!
Your financial support allows me to dedicate the time & effort required to develop all the projects & posts I create and publish here on my blog.
As always, feel free to suggest a project you would like to see built or a topic you would like to hear me discuss in the comments and if it sounds interesting it might just get featured here on my blog for everyone to enjoy.