VMK dictionary analysis and link to expanded dictionary

[JeanJoe]

Thanks to ElectricLime, who originally posted how to obtain the VMK dictionary as well as the tag method VMK uses to expand it, I've been playing with ways to work with the VMK dictionary, and have written some analysis tools. For people who might be curious, these are some things that I've been messing around with.

Because it was too long to post here, you can download the current full VMK dictionary hereEdited to add: not a link that's kept current. If anyone finds it of any use / interest, let me know.

The dictionary file that your computer downloads from VMK consists of a bunch of "root" words along with tags that specify how the root words can be modified. ElectricLime proposed, and I confirmed, that VMK uses "okspell" encoding. The tags modify the root words, for example, by prefixing with "re-" "in-" "un-", or suffixing with "-ed, -ing, -th, -ings, -s, -est, -ive, -ions, -ly, -ers", etc.

I wrote a script to expand all the root words to ALL the words allowable in VMK. Because they sometimes put tags on root words they shouldn't, like the "in-" tag onto "lalala", we can actually say things like "inlalala".

I expanded the script to do the following:
- download the dictionary file from VMK
- expand the dictionary from root words only, to ALL allowed words

Then, you can ask the program to:
- list all VMK words that contain a certain text
- do fancier searches, like all VMK words that begin or end a particular way, or even WAY more convoluted (see below)
- list all VMK words of a certain length
- take a block of text, and flag all words that are not allowed

Here's an example of how it looks:

Starting the program:

Welcome to Darwin!
~ $ /tmp/501/Cleanup\ At\ Startup/dictsearchtool3-164413260.01.pl.command; exit

3134 root words loaded from dictionary.
10106 unique words expanded from dictionary.

This shows that there are a total of 10,106 words that are allowed by VMK, expanded from the tags attached to 3,134 root words.

Search regular expression, or length, TEXT, or <RET> to exit: rip

description description's descriptions trip trip's triple triply trips

This listed all words that contained 'rip' anywhere within them.

Search regular expression, or length, TEXT, or <RET> to exit: eth$

cometh giveth liveth taketh teeth

This listed all words that END with -eth (the $ tells the program that matches have to be at the end).

Search regular expression, or length, TEXT, or <RET> to exit: ^.{2}a.*[fh]$

branch coach crash dearth flash health hearth leaf search smash staff teach yeah

OK, why you want to, I don't know. But, here were all the allowed words in the VMK dictionary whose 3rd letter was 'a' AND ended with either 'f' or 'h'.

Search regular expression, or length, TEXT, or <RET> to exit: 15

adventureland's alternativeness appropriateness comfortableness congratulations entertainment's environmentally fortuneteller's imaginativeness interestingness saint patrick's understandingly

This tells the program to list all allowed words that are exactly 15 characters long.

Search regular expression, or length, TEXT, or <RET> to exit: TEXT

Text to test against dictionary, EXIT when done

When you wish upon a star
Makes no difference who you are
Anything your heart desires
Will come to you

If your heart is in your dream
No request is too extreme
When you wish upon a star
As dreamers do

Fate is kind
She brings to those who love
The sweet fulfillment of
Their secret longing

Like a bolt out of the blue
Fate steps in and sees you through
When you wish upon a star
Your dreams come true

EXIT

########################

when you wish upon a star
makes no difference who you are
anything your heart desires
will come to you

if your heart is in your dream
no request is too extreme
when you wish upon a star
as dreamers do

fate is kind
she brings to those who [##love##]
the sweet [##fulfillment##] of
their secret longing

like a [##bolt##] out of the blue
fate steps in and sees you through
when you wish upon a star
your dreams come true


########################

The program took a block of text I pasted in, and then flagged the words that are not allowed by VMK.

Search regular expression, or length, TEXT, or <RET> to exit: TEXT

Text to test against dictionary, EXIT when done

Four score and seven years ago our fathers brought forth on this continent a new nation, conceived in liberty and dedicated to the proposition that all men are created equal.

Now we are engaged in a great civil war, testing whether that nation or any nation so conceived and so dedicated can long endure. We are met on a great battlefield of that war. We have come to dedicate a portion of that field as a final resting-place for those who here gave their lives that that nation might live. It is altogether fitting and proper that we should do this. But in a larger sense, we cannot dedicate, we cannot consecrate, we cannot hallow this ground. The brave men, living and dead who struggled here have consecrated it far above our poor power to add or detract. The world will little note nor long remember what we say here, but it can never forget what they did here.

It is for us the living rather to be dedicated here to the unfinished work which they who fought here have thus far so nobly advanced. It is rather for us to be here dedicated to the great task remaining before us--that from these honored dead we take increased devotion to that cause for which they gave the last full measure of devotion--that we here highly resolve that these dead shall not have died in vain, that this nation under God shall have a new birth of freedom, and that government of the people, by the people, for the people shall not perish from the earth.

EXIT

########################

[##four##] score and [##seven##] [##years##] ago our [##fathers##] brought [##forth##] on this [##continent##] a new [##nation##] [##conceived##] in liberty and [##dedicated##] to the [##proposition##] that all [##men##] are created equal

now we are [##engaged##] in a great [##civil##] war testing whether that [##nation##] or any [##nation##] so [##conceived##] and so [##dedicated##] can long [##endure##] we are met on a great [##battlefield##] of that war we have come to [##dedicate##] a [##portion##] of that field as a final resting place for those who here gave their lives that that [##nation##] might live it is altogether [##fitting##] and proper that we should do this but in a [##larger##] sense we cannot [##dedicate##] we cannot [##consecrate##] we cannot [##hallow##] this ground the brave [##men##] living and dead who [##struggled##] here have [##consecrated##] it far above our poor power to add or [##detract##] the world will little note nor long remember what we say here but it can never forget what they did here

it is for us the living rather to be [##dedicated##] here to the [##unfinished##] work which they who fought here have thus far so [##nobly##] advanced it is rather for us to be here [##dedicated##] to the great task remaining before us that from these [##honored##] dead we take [##increased##] [##devotion##] to that cause for which they gave the last full measure of [##devotion##] that we here [##highly##] [##resolve##] that these dead shall not have [##died##] in [##vain##] that this [##nation##] under [##god##] shall have a new [##birth##] of freedom and that [##government##] of the people by the people for the people shall not [##perish##] from the earth


########################

Search regular expression, or length, TEXT, or <RET> to exit:

And this is how the VMK dictionary handles more complex text.

It'd be easy enough to have a separate list of "dictionary dances" that would substitute words not allowed by the VMK dictionary with commonly used "dances." Of course, I, as a law-abiding VMK individual who would never try to bypass the VMK dictionary, would never ever ever consider doing something like that.

Go ahead and say it... I am a geek, old, lame... AND PROUD!

 

[LisaInNc]

I love it!

Bravo!!!!

 

[Roxane]

Very cool!

 

[JeanJoe]

Hmm, I tried to post the complete VMK expanded dictionary as of today, but I guess the DIS doesn't really like 92kb posts, so it got cut-off.

In case anyone wants to look at it, you can download the expanded dictionary_85. If anyone downloads it and finds it of any use / interest at all, let me know.

There are a few things that even though they are in the list, you still can't actually say -- examples are some of the smileys as well as wuz^. I think that's because of some additional processing VMK does to prevent certain characters.

 

[kitashus]

Wow, Be, that is *way* too complex for this non-techno girl. Interesting, but way too complex. I'll just stick to our conversations in VMK where you list strange words we can say and I'll come up with strange sentences. Hence the "Giveth me your cow" (punctuated by me changing into my cowgirl outfit!)

Anyway, thanks for the info...

 

[JeanJoe]

Oh, OK... LOL, I sort of thought that's what the general reaction would be.

Then, since you are requesting it, here are some of the particularly weird things that I'm going to be enjoying saying from now:

a i u ah ai b4 bb bc bg bl dc dl gg gl gr hb hw jj jk jp jw mi np ns oc ow pm qt sb ta ty uh ul ur vn ye yo yt yw -_- 0_0 0_o 20K 2l8 >_< @_@ abu adn afk afn aka arg atm aup aww aye b4n bbl bff bfn boi bol boo bot brb btw duh dvd faq fyi g2b g2g g2j gad gbu gr8 grl gtg hah heh hmm huh iac icl idk imo irl jpk kaa l8r lol meh mim n2b n2m n2s nah ne1 nmh no1 nvm o.o o_o obv oic omg pls plz ppl rad ref rof rsn rus sk8 sld sll sok sup tho thx tmi tyt usm wat woo wrt wud wut wuz yay yea ygm yup zev ~_~ 2l2t 4get 4got asap ahoy amof amos ayer bbfn bf4l bffl bing bltn btdt bykt cept doer egad elol eyer gmta gnar gosh gtsy gurl haha heya hola holo iirc imco imho inly invu kewl kung l8er l8tr lier loti ltns ming nano nevm nite nope oops orly otoh pref rofl ruff sitd sohf srry toon ttfn ttyl tyvm vibe woot yeah yoho youd ywia zonk zoom abner afaic afaik airer amped anded avast ayers baden bayed bayly bbiab bbiaf bbiam bbiaw bbl8r bigly bling bowen bower boyer buged bused byers caner cides cmiiw comer dised doers doged egger fitly froot gizmo hided holla illin kinda l8ter mayer mhoti mored nosed oncer oogie rocka rotfl rowen sealy shoer synth thanx toper ungot uraqt vibed wazup whoah willa zooks anders anding bloted bluely buging bummed chiped clubed comest coolio dising dogest gonion grubed hurrah iykwim kungfu lalala lazied louden mayest morion snaped takion thusly verier waddup wassup yeehah yeehaw yensid chillin graving grubing supness yee-haw amusive inlalala @---->---- lalalaness extraordinariness

comest mayest openest veriest wouldest cometh giveth liveth taketh doth sired ye


I mean, who wouldn't want to say, Waddup! Cides, u so bloted and sired by a morion who smell like a gonion. I wuz chillin but whoah supness! Lets go graving and grubing. Zonk! Inlalala! U bigly baden!

What kind of outfit would go along with that, Kitashus? When you decide, can I see it?

Wow, Be, that is *way* too complex for this non-techno girl. Interesting, but way too complex. I'll just stick to our conversations in VMK where you list strange words we can say and I'll come up with strange sentences. Hence the "Giveth me your cow" (punctuated by me changing into my cowgirl outfit!)

Anyway, thanks for the info...

 

[Califgirl]

JeanJoe, you may be a geek, but I heart you! And I heart all the time and thought you put into this. Thanks for sharing. I always appreciate knowing about new words we can say in VMK. (now if we could just say "word" lol!)

And I think that last passage you wrote would probably have Kit in at least the green super hero pants, and maybe the backwards hat. She'd have to be creative on the top - Herbie maybe? At least that was how it sounded to me!

 

[Captain Brain]

It makes me wonder how many words end with -ness.

 

[JeanJoe]

(now if we could just say "word" lol!)

We *used* to be able to say "word" and "letter", but they took it away from us. Then again, we CAN say "animalness" -- perhaps we can guess the goal of the VMK dictionary is to promote animalness?

It makes me wonder how many words end with -ness.

Exactly 271 And it took me 5 seconds to find out (woot!)

Here they are:

absoluteness accurateness activeness aliveness aloneness alternativeness ambitiousness ancientness angriness animalness anxiousness apartness apparentness appropriateness artificialness attractiveness availableness averageness awareness awayness awfulness awkwardness backwardness badness baseness beautifulness betweenness bigness blackness blindness blueness boldness braveness briefness brightness broadness brokenness brownness business calmness capableness carefulness cautiousness cheapness choiceness circularness cleanness clearness closeness coldness comfortableness commercialness commonness completeness complexness concreteness confidentialness conservativeness continuousness coolness correctness craziness criticalness cruelness curiousness currentness dangerousness daringness darkness deadness dearness delicateness differentness directness distantness dizziness doubleness drawnness dueness earliness easiness effectiveness emptiness essentialness evenness everydayness evilness exactness expensiveness expertness extraordinariness extremeness faintness fairness falseness familiarness famousness fanciness fastness fellness fewness fierceness fineness fitness flatness foolishness forgiveness formalness fortunateness forwardness freeness frequentness freshness frozenness fullness funniness gameness generalness generousness gladness goldenness goodness grandness gratefulness graveness grayness greatness greenness grossness halfness happiness heaviness helpfulness hugeness hungriness imaginariness imaginativeness immediateness impossibleness interestingness jealousness justness kindness lalalaness lameness lateness laziness leanness levelness lightness likeliness likeness limitness littleness liveness livingness loneliness loneness longness lostness loudness loveliness lowness madness materialness meanness mildness minuteness miserableness moderateness modernness muchness mysteriousness narrowness naturalness nearness newness niceness nothingness oddness onliness openness oppositeness ordinariness otherness outness pastness perfectness plainness politeness poorness positiveness powerfulness practicalness presentness prettiness previousness primeness properness publicness quickness quietness rankness rapidness rareness readiness realness reasonableness recentness redness relativeness remarkableness richness rightness rottenness roughness roundness sadness safeness sameness screamness secureness selectness separateness seriousness shortness silentness silliness simpleness slightness slowness smallness smoothness softness sorriness soundness southernness spareness specialness stillness strangeness successfulness suddenness supness sureness surfaceness sweetness tallness technicalness terribleness thingness thirstiness togetherness trueness unhappiness unlikeliness usefulness usualness variableness variousness vastness warmness weakness welcomeness wellness whiteness wideness wildness wonderfulness wrongness yellowness

 

[GirlWithSpunk]

WOW! It just seems to go on and on!! Thanks for finding this out Bezoar!

 

[Captain Brain]

Thanks.

 

[Califgirl]

We *used* to be able to say "word" and "letter", but they took it away from us. Then again, we CAN say "animalness" -- perhaps we can guess the goal of the VMK dictionary is to promote animalness?

Well, it sure isn't to promote literacy!

Exactly 271 And it took me 5 seconds to find out (woot!)

Here they are:

Some of these aren't even real words! LOL

 

[Antonius]

Just Voicing my opinion...

While I completely understand the point of the VMK Dictionary, I think it's ridiculous! Let's get real. If people want to get an idea across, they will.

I was told to day by one player that he was going to hit me in the "we we." While I wanted to laugh, I was kind of angry by the attitude. So are we going to lose the word "we" now? Everytime a word is lost, bad grammar takes a step forward. Let's just get rid of the players that are abusive and keep the words. I'm tired of dictionary dancing. Kids are much smarter than people give them credit. They know what's going on in the world. They can only be protected so much. Teaching them to use poor grammar is apparently ok just like "taken" and begging. I think those two things offend more gamers than being called a "beach."

VMK! KEEP THE WORDS! LOSE THE ABUSERS!

 

[justliketink]

Blonde moment....Whoa! Complete techie stuff. I think I will reread this post after my coffee tomorrow......

 

[Chedstro]

Antoniua is my new hero Yes, what he said!

 

[kitashus]

I mean, who wouldn't want to say, Waddup! Cides, u so bloted and sired by a morion who smell like a gonion. I wuz chillin but whoah supness! Lets go graving and grubing. Zonk! Inlalala! U bigly baden!

What kind of outfit would go along with that, Kitashus? When you decide, can I see it?

OK, here you go...(thanks Califgirl for the suggestions!):

Zonk! mhoti, huh? Got some bling no?

 

[tigressjewel]

OH mygosh kit! That almost scared me!

Thanks for the great info Be! I bookmarked those links, will they stay around? Hope so, what fun!

 

[JeanJoe]

Wow, that outfit is almost enough to convince me to speak normally.

Almost, but not quite

Bigly!

OK, here you go...(thanks Califgirl for the suggestions!):

Zonk! mhoti, huh? Got some bling no?

 

[Captain Brain]

 

[bootsonton]

@_@ wow joe! that's alot of hard work and dedication!