Singularity Watch

how can an ordinary human play if there might be somebody else at the table using a top-caliber computer program?

It depends on what resources the computer programs are using. I don't know for sure, but I'd expect computational power isn't that important. Rather, it's the development of the algorithm that predict other players' play. And it would be hard for even a very-above-average hacker to do what this team has done in his spare time.

Also, I'm not sure it's proven that poker is well-ordered. (An example of a game that isn't well-ordered is rock-paper-scissors, where a strategy of "getting paper" loses to a strategy of
"getting scissors".) A strategy that works against the top class of players may not work so well against the second-top class of players.

However, I'd expect that said hacker would be able to come up with something that can take on the average human player.

The best ever computerized poker player was done by these guys, who were able to predict every single card held by every other player, and what the flop was going to be, based on reverse-engineering the random number generator. If you want the money shot, just scroll down to the UI they wrote that shows everyone's cards. . .

http://www.cigital.com/papers/download/developer_gambling.php

ummmmm...didn't legislation already kill online poker?

If that were true than it would be irrational for the team to publicize the event (unless the team were funded by real-life casinos that were competing directly with the online ones).

However, that gets to a more interesting question of if you had a computer that could beat any human at cards (over 500 hands), how do you program the computer so that the human thinks he's playing a human and not a poker playing computer.

And what is this about a bonus system that gave the humans 12k extra. Does this mean just in winnings or does it count when they were scoring the event? Obviously this is not set up as a laboratory experiment (and would most likely be expensive to due to the high opportunity cost of top poker players), but two matches with razor thin margins is hardly statistical evidence that the machine had a negative expected return against top human poker players. Poker has as much to do with luck as it does with skill. They need to put that machine in thousands of hands against many pros with many different styles to really examine if the humans would lose to it.

However, that gets to a more interesting question of if you had a computer that could beat any human at cards (over 500 hands), how do you program the computer so that the human thinks he's playing a human and not a poker playing computer.

A poker Turing test! I can't imagine it would be that hard- the computer could just take a random number of seconds to make each move, plus perhaps not make extremely outlandish strategy decisions.

In other computer games (Counterstrike etc) cheats are a considerable problem. Aimbots are used to guarantee headshots for the cheater, wallhacks are used so that they can see through walls and speedhacks are used so they can travel incredibly fast. In persistant online games like World of Warcraft programs called bots are used to automate a character to collect gold or other things.

There is a constant battle between game makers (or cheat detection companies like Punkbuster) and cheaters. In my experience, cheats are rare, but you do run into them now and again.

With a simple game like poker though, the trick would be to have a robot punch the keys, thus making it completely undetectable, or to manually log everything that happens in the game on a second computer that then tells you what to do.

I wonder how quickly a computer program could pick up the change in style if you switched players without any kind of notification? Could you deliberately change strategies to accomplish the same thing? I cannot believe that there is not some kind of vulnerability there.

Heads Up Limit Hold 'Em

So, and as confirmed by the article, only the one-on-one version of the game. (That's what the "Heads up" means.) Online, there are generally many more people at a table, and the computers are not good enough at multihanded game yet.

Heads up is game theoretically simpler than multi player, and limit is simpler than no-limit.

The first thing that makes heads up easier is that if you play an equilibrium strategy in heads up, not only can't you lose much, but you're opponent can't win much, so you can (and should) be very conservative about exploitation if you are confident in your eq strategy. OTOH, at a table full of players of varying skill some of whom make large mistakes, a good exploitive player will do much better than a pure equilibrium player, even though they are making errors that give money to the equilibrium player.

The second problem is that multi player poker is *much* more complicated game theoretically, and nobody I'm aware of has come close to approximating an equilibrium strategy except in some very simple toy games, while people who've studied the game in depth (such as the alberta group) seem to have a reasonable grasp of what an eq strategy should look like for heads up limit, even if it's nowhere near actually solved.

For a really interesting look at poker and game theory, check out Bill Chen's and Jerrod Ankenmann's book _The Math of Poker_.

As a former (reformed?) semi-pro player I think I can offer some insight. Poker bots have been in developement in the private realm for some years now, and many serious players believe that bots are playing profitably for money online in both limit and no limit. However, the belief is that this is mostly relegated to the low limits, (say less than 1/2 in limit, and $25 max buy in for nl)and isn't particularly common. In 50k hands I don't recall ever seeing a player I thought was a bot. Yet it does exist, and as the bots get better, it figures to pose a threat to the game.

The good news is that there are plenty of factors that help to keep the status quo. As mentioned, programming a bot for ring games is quite a bit harder than heads up, as well as no limit being harder to program for than limit. Also, poker sites for certain, don't want bots and will aggressively try to remove them. Simple human verification measures, such as text transcription boxes like you see on sites such as amazon, figure to help as well.

Also, I'm not sure it's proven that poker is well-ordered. (An example of a game that isn't well-ordered is rock-paper-scissors, where a strategy of "getting paper" loses to a strategy of "getting scissors".) A strategy that works against the top class of players may not work so well against the second-top class of players.

There's actually been a huge ammount of work done on poker and game theory but people like David Sklansky, Mason Malmuth and Ed Miller. Simple games of poker are certainly 'solvable' in the way that oh, say, checkers is solvable. As an example think of a game where you are dealt only 1 card, and there is a betting round where you can bet either 1 unit, call or fold. Certainly there exists and optimal play for any position and holding. You can further exhance your expected earn by taking into account your opponent's play. If he folds too much, you would bet a little more often than optimum, etc. This extends to texas hold'em as well, although the game is vastly more complicated (and may not hold for no limit). But the theorem is that 'optimum play' does exist. Also, rock-paper-scissors is a super simple game in terms of game theory, just throw randomly and you'll aways reach equilibrium.

The first generation of poker bots were rule based and acted on this sort of reasoning. IE: If on button, then raise all hands greater than value X. I can't speak much on how the bot in question is written, but I imagine it's similiar.

In the case of chess, the maintainers of an online chess site have equal access to the programs chess players would be tempted to use to cheat. They can do random audits and check that the game play deviates from how a bot would play a move.

The online chess site I play at, redhotpawn.com, frequently runs the top players' games against the output of various engines, and has banned many cheaters.

This works fine when reputation is the gained utility from winning (and thereby, banning removes all the utility), but it doesn't work so well when you are playing for money (banning doesn't get your money back).

Arnold, to cheat this way, you'd have to track opponent plays, detect patterns, and sufficiently randomize or obfuscate your own play so patterns don't emerge.

As for ensuring human interaction... Make a Java applet and just pipe pictures across the Internet. Randomize presentation, mix in new layouts occasionally, analyze the clicks on the server side. Make it one giant captcha (in blog speak). The computers will have a fit.

Make it one giant captcha (in blog speak). The computers will have a fit.

Heh! Or you could play with captcha-style cards, where the only way to know your hand would be to recognize the skewed and skewered image on the card. How annoying would that be?

The online casinos also keep track of user's ip address to keep an eye on the number of players per IP address. This makes its harder for bot rings to parallelize their operations, which is probably necessary to capture a worthwhile ROI...

If gambling sites are shut down by cheaters, maybe people will do something productive instead of wasting time on negative sum games.

This works fine when reputation is the gained utility from winning (and thereby, banning removes all the utility), but it doesn't work so well when you are playing for money (banning doesn't get your money back).

Banning is actually a very good deterrent. First, players need to keep a cash account with a casino to play, so if you're caught cheating, all that cash and any winnings you may have pokerbotted your way to would be forfeit. Sites also generally limit transaction amount/frequency to prevent much of the old hit and run. Furthermore, poker accounts are linked to personal identities. So if I'm banned from one of the relatively few major sites, I need to commit difficult, but not impossible, identity fraud to get back online and continue cheating. Online casinos could pretty easily implement a shared blacklist as well if needed.

Legislation hasn't killed online poker just yet. It's been curtailed, but you can still find it at a few sites.

From the beginning of time man has destined to be the master of all machines, the advent of a device to win at poker or any other game of skill would be truly amazing. Great article, keep up the good work!

Pardon my ignorance, but has anyone held a tournament among the poker-playing programs? If nothing else, it would be an interesting way to check one algorithm against another.