AI MEGATHREAD
-
@Azraeel said in AI MEGATHREAD:
You didn't use Uveso-AI with Swarm-AI which is required. Its also Required for RNG-AI, so ur swarm games are completely in-accurate.
makes sense.
I was getting the impression from the youtube vids that it should be doing much better
I'll redo those.
EDIT : I've redone those and updated the sheet to match with reorder.
with uveso on swarm was indeed hardest. it's also the best non-cheating. -
@tatsu Uveso is pretty scary if he gets up. You start playing survival at that point
-
agreed. and swarm is butt-kicking also.
-
Seems like an ambitious project but seems worth it.
-
Swarm Eternal is brutal indeed, but my problem with Uveso and Swarm is that you are basically playing a survival game most of the time, they just spam T1 continously.
-
hey, idea of rating AI sounds very fun but I'm not so sure about the methodology. I mean rating them on how playing them "feels". There are some ladder players around 1300 that I feel rather comfortable playing and 900 that I wouldn't feel confident playing at all just because of their unusual playstyle and although I would still reliably win with those 900s, I probably couldn't rate them properly just after one game by "how it felt". If you really want to rate those AIs, maybe it would be optimal to also get them play lots of games against each other on multiple maps and see which ones beats the rest(similar way to how we estimate real players strength).
-
yeah, I agree.
that's the core of the method.
I'm super low on time now so that's why I'm opening this up to everyone
-
Setting up AI's to fight each other is ok to get a base number.
Good to have as side information side (another tab on your spreadsheet maybe? its easier to setup an AI game and walk away if your low on time). Small note here, recently the skirmish game mode was made to allow the human to become observer so you could run these on the offline executable without spamming replays to the replay server.But I kinda agree with tatsu in that its not representative of how those same AI's will fair against humans and there needs to be a 'feel' indicator, especially when it comes to how they play with different human play styles, map preferences, cheat multipliers and even ai game settings. One might do ok against a turtle but garbage against a pressure player and vice versa. Its all 'eye of the beholder' sort of thing, but as someone whos stared at AI games far too much its worth something.
-
Currently AI doesn't really do anything smart and barely attacks you, you actually just move up and blast them with your commander followed by your tanks and AI commander sits still in its base and dies. That's my experience.
The problem is that they don't know how to use their commander.
-
Maybe someone should contact OPENAI and see if they're interested in seeing if they can develop a competent AI for the most complex RTS game in the world
-
@Mylaur @Psions. Hold on just a hot sec.
have you guys tried enabling Uveso + Swarm + Nutcracker, then playing against a Swarm or nutcracker? it doesn't even have to be with extra eco (AIx) just the AI version.
You'll see that we have pretty darn good AI coders as is.
-
@tatsu I enabled AI Swarm and Uveso and played vs AI Swarm Terror.
https://replay.faforever.com/13437722
https://replay.faforever.com/13437811Second replay I forced map markers as I was confused to why AI didn't get out of his base much. Also second game I spawned a spy plane to see what the hell he was doing and maybe that's why he built an air factory (my bad), still he was unable to do anything with it not even make bombers. I also was a quarter trying to fix my hotkeys so sometimes I did nothing and still managed to utterly roll all over his base as he made no tanks. I'm around 1000 for team games, not really good 1v1.
-
Well I'd say small maps are often not AI's strong suit.
I dunno try with something a tiny bit more spacious like tag rock, eye of the storm, salt rock colony, Syrtis major, the Dark heart, or arcane.
all of which are 10 x 10 but less in your face then cobalt valley.
but yeah that being said AI generally deals pretty poorly with 10x10 commander bum-rush.
try it on 20x20 maps. try seton's clutch they're pretty hard to deal with in this case you may well end up dying.
if not try with AIx and 20x20
-
The replays were ok for me,
To be fair Mylaur anyone above a 300 rank playing against a non cheating AI is going to have an easy time.
For giggles you could try RNGAI (self promotion, just make sure you read the description for setup tips) with a 1.5 cheat/build multiplier on that same map and see if your experience changes.Getting the acu to be remotely in the realm of a human is damn near impossible(have spent days/weeks/months trying), an (almost) single threaded game just doesn't have enough clock cycles to do complex calculations at the speed to capture the data required to react and make decisions, you can do fake micro but like you said the acu doesn't really know 'what' to do.
Example for reference : getting aeon auroras the try and maintain weapon range on a target during combat takes approximately 7 function calls per second per unit plus supporting logic, if there are 50 T1 auroras on the field fighting that's 350 per second and 50 units is a small number.
-
That's a boatload of processing, and you're only talking about a handful of units.
-
I don't doubt the ability of our AI coders, what I doubt is the ability of any coder no matter how proficient to make an AI that is capable within the SC engine of coming even close to even a 1200 rated player.
AI is simply to bulky at the moment code wise and you either end up with an incredibly specific AI for a specific map, or an incredibly generalist AI, that is weak at everything and easily exploited.
All games face this problem. Which is why most strategy games give insane cheats to AI to offset their inefficiency/stupidity.
OpenAI - or Modern AI techniques avoid all of this, because they function on a stastitical model, which can become very accurate through reptition and therefore are uniquely situated in that they are self training, and not overly bulky when it comes to hogging processing power or requiring huge logic trees.
-
the point of this anyhow is ordering them, because right now they're unordered.
And I never suggested AIs on supcom's purpose could eventually be utilized to beat a 1200 human so I dunno why people keep turning the mic back to me on that one O.o
I do think however that once we have a good classification of AI difficulty we can utilize that to train up our roster of -600 to 400 players.
-
Yeah exactly the point is not or should not be to get an AI to >= 700 ladder rating.. It should be simple to select, not too much options. Maybe 4 difficulty levels without further options, fixed map pool that is known to work with those AIs somwhat ok, done.
And if you get people to learn to ACU-tank-spam-bum-rush the AI you have already prepared them better for PvP than they were before. Is it going to be boring fast: Yes. But thats where you switch to ladder or TMM if you want the challenge.
Now about ranking them into the 4 difficulty levels: I think this has been suggested before, but the most logical way would be to just let them play in ladder for a time. The question is how to do that automatically.
-
I like the idea of being able to have the AI's rating move up and down with it's win / loss ratio against players also.
The AI's are constantly getting better and just like a person its rating should reflect that.
Would be nice if AI's would be occasionally tested against more experienced players (no grey ratings) and
their ratings were adjusted just like the players.
There would have to be a way to "create a user name type account" for each and have designated modders play their AI's against volunteers periodically to get close to it's true rating/difficulty. -
