Thursday, February 23, 2012

How-to: write a mIRC games scores file.

I dealt with a lot to get such things to work well, and I highly recommend the way I did my scores to anybody. Which is to say tokenized strings in a flatfile to be accessed with a /filter command. Unless mIRC at somepoint gets mySQL this will be the best way to do things. A well placed filter command can pretty much access anything and in a fast enough time frame for a game. And it scales well enough to use.

In Tat's Trivia you can see the var commands. Namely setvar, readvar, getvar, gttok, tvartoks. These each do specific things.

Below the fold is a rather long breakdown of everything about how this stuff works. And why you should use this method.

setvar will set a variable name it's typically called in the format:

readvar will read a variables name. It is guaranteed to be read from disk. And not cached.

getvar will get the variable names. It could be cashed or read, but it'll just get you the variable you request.

gttok will find the specific number of the token requested as plain text.

tvartoks is a period delimited list of variables being looked up.

It's usually called $getvar($nick,variable)

-- The caching scheme I used did a significant benefit when I wrote it, but presented might do nothing. I remember seeing that in the change log it said something along the lines of doing a cache scheme like that in mIRC itself. So it might be a lot of extra work for nothing. In any event, I'll break down my code for interested parties.

alias setvar {

if ($3 == $null) return
Verify that there is a value you're setting.

if ($exists($tempfil)) var %backburner = $read($tempfil, wnt, $1 $+ ;*) ;

Check if there is a backburner entry for that individual and remember it if there is. Since the scores are not cleared in any routine method everybody who uses it should be remembered forever, but since most of them won't be back there's a few hidden commands that send files to a backburner file. This file is checked in case that person has returned. And their scores are treated as if they were never out of the main file. This is more to drop non-return individuals from the scores list than for the sake of speed, since it'll check both files anyway.

if (%backburner != $null) mergeentry $tempfil $scoresfil $1
If it properly read that entry and it exists, then run the merge entry alias between the $tempfile (backburner) and the main (scoresfile) for the nick.

var %newline = $readvar($1)

Read the line we'll be modifying from the disk.

if (!%newline) { %newline = $1 }

If reading it from the disk came up with a blank line. We start our entry from scratch with just the name.

  if ($numtok(%newline,59) < $numtok($tvartoks, $asc(.))) {

If the number of tokens in this new line using the token character 59 which in asc is SEMICOLON ( ; ). is less than the number of tokens delimited by a period in the tvartoks alias. So if there are fewer tokens in the line of the score file than there are tokens in the trivia variable tokens alias, do these commands.

    %newline = %newline $+ $str(;0,$calc($numtok($tvartoks, $asc(.)) - $numtok(%newline,59)))

Add to the %newline variable enough copies of  ";0" to make up the difference in tokens between these two things.

    if ($gettok(%newline,9,59) == 0) { %newline = $puttok(%newline,trivia,9,59) }

If entry number 9 is 0, make that particular token say "trivia". Entry 9 in that list is apparently "team". I'm not sure why I do that.

    if (($gettok(%newline,8,59) == 0) && ($address($1,5))) { %newline = $puttok(%newline,$address($1,5),8,59) }

If entry 8, which is "address" is 0. Which means it was just added. Then it should be replaced with the known address for the nick $1, with appropriate wildcards.

    if ($gettok(%newline,6,59) == 0) { %newline = $puttok(%newline,$date,6,59) }

If entry 6, which is "lastwin" is 0. Then insert today's date.

    if ($gettok(%newline,3,59) == 0) { %newline = $puttok(%newline,9999,3,59) }

If entry 3, which is "time" and more properly the record time it took to answer. Replace that badboy with 9999. The point here is that if you had a zero there, then you could never best that time. 9999 is a stand-in for infinitely long.


And we're done with these we had to make some blank parts of the score line section.

  %newline = $puttok(%newline,$3-,$gttok($2),59)

And now replace the part of the %newline variable with whatever the new values are, and add that to the entry of that token translated from plain text. So "/setvar Tat score 3" would put the variable "3" into entry 2 (which is determined by looking it up in the token list). The 59 is again the ascii number of the delimiter I use namely SEMICOLON " ; ".

  write $iif($readn,-l $+ $readn) $scoresfil %newline

Write the variable %newline to the scoresfil. If my last read command was successful namely the $readvar I did above. Then overwrite that line with the flag -l.

  set -u3 %cache.nick $1

Set for three seconds a variable %cache.nick storing the name $1.

  set -u3 %cache.line %newline

Set for three seconds a variable %cache.line storing the line I spent all that time building and/or editing.


alias readvar {

Readvar will always read the entry from disk. It won't build anything. It will read it. If it doesn't read it. Tough nookies.

  if ($exists($scoresfil)) var %scoreline = $read($scoresfil, wnt, $1 $+ ;*)

If there's a scoresfil then read a through it looking for something in the format ; and make that scoreline.

  if (!%scoreline) { return }

If I don't have a scoreline that actually exists, invoke tough nookies and return.

  if (!$2) { return %scoreline }

If I didn't ask for anything specific. Return the entire scores line. "/readvar Tat" will give the entire line.

  else { return $gettok(%scoreline, $gttok($2), 59) }

If I did ask for something. Then find out which entry I asked for (converted from plain text), delimited by character 59 which is a SEMICOLON, and just return that bit.


alias getvar {

Get var will return a variable. It will not necessarily read it from disk.

  if ($1 == %cache.nick) { var %scoreline = %cache.line }

If I asked for a variable dealing with the cached nick. then just used the cached line.

  else {

If I didn't cache things...

    var %scoreline = $readvar($1)

Get the scoresline by reading the entire line from disk.

    if (!%scoreline) { return }

If I don't have anything abort..

    set -u3 %cache.nick $1
    set -u3 %cache.line %scoreline

If I have something, go ahead and set that cache myself.


Anyway it went I now have a data filled %scoreline

  if (!$2) { return %scoreline }

If I didn't ask for anything specific, return that entire line.

  else { return $gettok(%scoreline, $gttok($2), 59) }

If I did, return that specific part I asked for.


alias -l gttok { return $findtok($tvartoks, $1, 1, $asc(.)) }

Find the token in the tokenstring $tvartoks which says " $1 ". Give me the first one and use the delimit character of an ascii PERIOD. Which will always resolve to a 46. I somewhat inconsistently tell it to convert that . from ascii rather than say 46 and don't say $asc(;) for all my 59s.

alias -l tvartoks { return }

Here's the list of all the trivia variable tokens. For each entry there will be an entry in the scores file. Every line will be able to tell you any of these data points and the lookup time will be rather static. If for whatever reason the scores file has to be expanded the new information can be tagged on to the end. Since the script checks to see if the number of tokens here matches the number of tokens used in the particular score information. It will automatically make that into a zero and treat it all normal like. So if the file needs to be expanded my code will pretend it has that data and that it's a "0" which is what happened when it added all those ";0" to make up the differences in the tokens.

Okay, how about a simplified version?

alias scoresfil { return myscores.txt }
alias setvar {
  if ($3 == $null) return
 var %newline = $read($scoresfil, wnt, $1 $+ ;*)
  if (!%newline) {
    %newline = $1 $+ $str(;0, $calc($numtok($tvartoks, $asc(.)) - 1))
  %newline = $puttok(%newline,$3-,$gttok($2),59)
  write $iif($readn,-l $+ $readn) $scoresfil %newline

alias getvar {
    var %scoreline = $readvar($1)
    if (!%scoreline) { return }
    if (!$2) { return %scoreline }
    else { return $gettok(%scoreline, $gttok($2), 59) }
alias -l gttok { return $findtok($tvartoks, $1, 1, $asc(.)) }
alias -l tvartoks { return,dataiwant2.ban.blah.blah2 }
The idea is basically some variation of this. You store a tokenized string of what you're storing and a flatfile of tokenized datapoints. that you read with a $read command and use $gettok, $puttok, and $findtok to slice and dice. The end result is a pretty easy to use scores setup. -- Also, my trivia script is public domain so you can just look it up and grab it. Make your own checks in the setvar for what default settings for whichever entries, and make tvartoks have the data you want to use and you can use it out of the box for things like "$getvar(Tat,score)" and "/setvar Tat score 5" and it really should work.

There are other benefits to this scheme that come with the use of filter. It's entirely possible to just extract my /sort command too and it'll let you do high score tables based on any statistic you want pretty much out of the box.

alias -l twin { return @trivia $+ $scoresfil }

alias sort {
  if (!$exists($scoresfil)) { return }
  window -h $twin
  filter -fwcgut $+ $iif($1 != time,e) [ $$gttok($iif($2 != total,$2) $+ $1) $asc(;) ] $scoresfil $twin $iif(($2) && ($2 != total),/^ $+ $str([^;]*;,$calc($$gttok($2)-1)) $+ $eval($ $+ get. $+ $2,2) $+ ;.* $+ /,/.*/)

It makes a hidden window $twin (Trivia Window), and filters everything out of the scoresfil into that window in a sorted manner.  So /sort score week will put those bits together to find scoreweek entry. As I personally store the all time, week scores, month scores, and year scores. This is all I really bothered to sort by, but really any point of data is doable. If I was to do it over I would rewrite that alias to be much more dynamic.

The core of the sorting code is a filter command with the flags -fwcgut or from File, to Window, Clear destination, using reGex, and a nUmeric sort, sorted by [c r] ouTput.

So I'm moving the stuff from the scoresfile to the hidden window I made, clearing that window out, and sorting the entries numerically based on the column and the character I used. This column character thing is just tokenized strings again! Which is why this is so preferred. There's a lot of built in stuff with tokenized strings.

If it's asking for time, it tosses an "e" flag in there which says to invErt the sort. I want lowest times on top in that special case. The $iif command saves you a lot of if statements but destroys the readability of the code. But that $+ $iif($2 == time, e) just says append an "e" to that set of flags if my second passed variable is time.

The $gttok is the column (entry) for that particular data point, and $gttok just takes it from plain text to the number. It saves you a ton with readability and simplicity. You just ask for the score of $nick, or $getvar($nick,score) and you're done. 

If I asked for "Time Total" it will run $gttok on time (dropping the "total"). And since these bits of data are delimited by SEMICOLON (59) it'll use that as the S entry. It filters between the $scoresfil and $twin (trivia window) and builds the regular expression from there.

/filter -fwcut [ $gttok($1) 59 ] $scoresfil $twin *
More simply, that should basically work.

 Most of that code was before I used the direct tokenize code. Using the -t flag, most of that code to build a rather complex regex is obsolete. And a lot of it is spent making it compatible with the older way I did it. If I just used the correct name from the get go, it would have been much simpler.Hence why I can say with some confidence that the above code there is going to be easier. And had I done it that way any other people could have just directly used my sorting code there.

"/sort score"

Would move all the data to the window sorted by score. Then you just access the lines in the window and find out who has the best score and the second best score, etc. And since filter is an internal command it ends up going faster than any code one could otherwise script. Even if you just wanted the person with the top score, that filter to sort the entire list is going to be faster than just scanning the list once and comparing each score. Because each step would need to run the interpreted code you wrote. -- This is also the exception to the speed issue with the $tempfile or backburner.txt. Since this sorting scales at a time complexity of N*Ln(N) you can get slower speeds sorting massive lists than if you just pulled out those never really going to be used entries. If they have 4 points, and they haven't had a win in the last 4 months (my script saved the lastwin date data), they aren't worth adding to the sort you're doing here.


I could like extol the virtues of this way of doing a score file in mIRC script, for a good while longer. But, almost needless to say, this is the best way to do it. You can just pilfer my code directly for a lot of it. And use the tvartoks list to call your bits of data something else. A quick tweak or two to sort and you can use that too.

And if you do that tweak you can then use the $hof command,

alias -l hof.size { return $line($twin,0) }
alias -l hof { return $gettok($line($twin,$1),$2,59) }

Which quickly lets one ask for whatever bit they want from the sorted data. $hof(1,1) for name of the person in first place for example. Or use $gttok($2) rather than $2 for things like $hof(1,name) etc.

The $hof code predates the $gttok so I didn't bother to rewrite it all. But, if doing it yourself from scratch without all the legacy code my script has:
alias -l hof { return $gettok($line($twin,$1),$gttok($2),59) }

Is a much better command. Then it's simply $hof(,score) or whatever exact bit you want.


This is generally how you want your scores files to work. I tried other ways and this is by far the best. It's generally limitless scales really well. And mIRC has a lot of built in stuff for tokenized strings. You end up getting a lot of storage density for a small handful of commands that can be rather easily reused and accessed throughout your script.


belair577 said...

so how can i increase a users score by x number of points after they win points?

belair577 said...

also i seem to be missing the whole part that allows a user to display their score? what is the commands for that?

Tatarize said...

Those central bits depends on the way whatever game itself is programmed. The scores file will let you save whatever data you want to save to a file. It doesn't do any of that other stuff.

setvar $nick Score $calc($getvar($nick, Score) + %x)

This will for example increase a user's score by %x. It's simply saying set the score value for $nick to (get user current score) + X.

Likewise in any messages sent out to the user you can just call up their score with a $getvar($nick,Score), etc.

msg #chan $nick got that right and has $getvar($nick,Score) points!

Or whatever else. You can just access the flatfile once it's setup like a black box. You don't really need to know how the data gets there but manipulate it however you need.

-- And since everything would be through $getvar() and setvar you can actually change the entire way it works at a later date without changing the way the program works. Such are the benefits of good abstraction. Though, I'm pretty sure that this is currently the best way to do such things.

belair577 said...

so %x can be %userscore [ $+ [ $nick ] ] ?

Tatarize said...

The entire point is that you don't need to use variables at all. You're saving the score to the setvar $nick Score , so you can just call a $getvar($nick,Score) and add something.

Something like:

/setvar $nick Score $calc( $getvar($nick,Score) + $getvar($nick,Bet))

Should work fine. And then setting the bet size in some command somewhere to
/setvar $nick Bet $2
or whatnot.

I think just
on *:remote:!setbet *:#:{
/setvar $nick Bet $2

With some checks perhaps that ($2 isnum 1-999999) or whatever.

Assuming "Bet" is added to the data set you save in tvartoks or whatever you end up calling it.

Setting scores with variables is typically [[ % $+ userscore $+ nick ]] or whatnot but moreover sucks. They sometimes get lost on crashes or fail to work right, can't store more data without runaway variable declarations ect. These are among the benefits to a tokenized flatfile to store stuff.

The entire point is that the /setvar and $getvar is storing the data. More likely you'll do /setvar $nick Score $calc($getvar($nick,Score) + 1) where the %x is going to be the amount your adding to their score rather than their score from somewhere else.

The scores file is intended to completely replace things like assembled variables to store values. And let you dynamically store any amount of data about a person in a pretty easy to figure out format. So you can store score, their address, their bet, their win streak, their fastest time, etc. Whatever you want to store you can store without resorting to things like variables. And you can process the entire list with things like /filter for some highly advanced commands.

So while you could use anything for %x, it's best that it be the value of the bet or anything else.

If you wanted to use variables like %userscore you'd still be better off with something like:

alias vset {
set % $+ [ user $+ $2 $+ $1 ] $3
alias vget {
return [ [ % $+ user $+ $2 $+ $1 ] ]

Which would allow you to just use /vset $nick Score and $vget($nick,Score) and avoid the very cumbersome idea of accessing them directly by rebuilding the variables each time you use it.

Tatarize said...

Ultimately it has a lot to do with how you access things. And you want to use some abstraction. You don't need to know where or how the data is stored. You just make aliases which consistently give you the values you need and other variables that put that variable where it should go. -- It doesn't much matter if where it goes is an assembled variable, is a .ini file with /writeini, is a hashtable, is a tokenized flatfile, etc. You just want to access that data consistently. And of these methods I highly recommend the tokenized flatfile. But, ultimately the point is that however the data is actually stored, you should access the data in a consistent manner through your own aliases. In this way you can be sure to allow yourself to change the method of access with changing one variable and be sure that you access it right everytime (building your own variables with % and $+ can go wrong easily).

It breaks down to a programing question, and the answer is data abstraction. You shouldn't need to care how it stores the data just that you can put information somewhere with an alias and call it back up with another alias. How that is done, doesn't matter to the rest of your program at all. You just want your alias to tell you the right answer, how it got that answer doesn't matter.

Tatarize said...

Take for example an old video poker script I wrote. You'll see there's two aliases that do the reading and writing of data. These are pvar and pset. I also store the card data in a hash table but that's because hash tables go away when I no longer care and that data isn't data I need or care about. -- However any writing or reading of scores or bets or any information only uses pvar and pset. To convert this script over to using a tokenized flatfile (as I now strongly prefer) all I would need to do is simply change those two aliases and the rest of my code could remain exactly the same and it would work perfectly.

(and yes, this code actually works, and implements an awesome cardrank algorithm etc).

Tatarize said...

Video Poker

Brett said...

Hi Tat - thanks again for writing this amazing guide - I've been using it as the framework for my own scoreboard.

I've recently encountered a problem with the sort function though - it seems my file is so big that sort no longer works properly. After running sort and looking at the window, it was sorting properly but was missing a lot of the actual lines (did some ctrl+f's looking for what I know are the real high ranking results, finding nothing). I edited my scoreboard and pruned it back to a small amount of lines and it seemed to be working perfectly. I tried both versions of your sort alias.

Am I doing something wrong, or is there something else I'm missing?

also there's a small typo - in your revised short non-regex version of the /filter line in sort, you wrote $scorefil but I think you meant $scoresfil

Tatarize said...

The size of the file should not matter. I've applied filter commands on a hundred meg text file. The only difference is the speed it goes at.

The missing lines are likely missing because they failed to match the given criteria. Filter commands really run a search for any matching lines and transfer them from source to destination with a few optional manipulations along the way.

My native sort however, has a day/week/month/year set of added criteria. It dynamically makes regular expression that checks the timestamp. It matches *only* those those files which properly have an updated timestamp to fit that criteria. So if you have 200 points for the week but the week was last week, it doesn't match and doesn't add that to the window to be sorted.

If lines are missing. They are likely missing because they do not match the given matchtext. If your matchtext is * or /.*/ (with -g regex flag) then it should grab everything and somethings fubar. Filter works right, so best guess is you have a match text that's somehow throwing a wrench in it.

You could also get errors like that with a range -r flag in filter to take only a subset of read items or add a buffer to the window you're reading into. But, that'd require going out of your way to mess with it. Matchtext is likely the issue.

I'd need more specificity to troubleshoot it in more detail.

Brett said...

my matchtext is * - I pretty much copied your code exactly, but my tvars are different. I dont have any time stamps, I have slightly different variables:

alias -l tvartoks { return }

the stuff at the end is just gibberish placeholder. I use the scoreboard to keep track of song stats that are played.

I was using your code that included the long regex, but never encountered problems till recently, and I dont think anything has changed.

I'll try looking through my .txt document to see if there's something weird that could be messing with the filter but I'm unsure what I should be looking for.

I could throw in my scores document if you think that would help.

thanks again!

Tatarize said...

yeah, you can shoot me the relevant document at tatarize at my account. It seems really odd that some of them should be missing. If they aren't wrongly matched. You could also try some counting stuff. If all the data is moved.

//echo ... $lines($sfile($mircdir)))
//echo ... $lines()


//echo ... $line(@triviatriviascores.fil,0)
//echo ... $line(@window,0)

Should actually return the same number. If they don't you still might spot a pattern like if the lines in the window are always 1000 then something is truncating it, and oh yeah somehow you set a buffer to 1000 and boom, solved.

At the least you should be able to produce a test case where your code gives you different answers on those two data points if you are right.

Tatarize said...

Also one of the filter flags is -x, which will exclude nonmatching bits, If something somehow fails to match it would be put over if you checked it against an excluded capture. Which might help tracking down why it's not moving them all correctly.

Brett said...

Alright I'll give these a shot, then send you my list if I am still baffled.

Thanks again for the detailed help!