Introduction to the SSGM 4.0 TCP Logging System [message #434694] |
Fri, 13 August 2010 06:49 |
|
danpaul88
Messages: 5795 Registered: June 2004 Location: England
Karma: 0
|
General (5 Stars) |
|
|
This topic is intended for bot developers who are interested in supporting SSGM 4.0, which will be released as part of the scripts.dll 4.0 project.
Introduction
The file based logging system under previous versions of SSGM is no longer supported and bots which rely on the ability to read ssgmlog and gamelog files will no longer function correctly. However the renlog files will still be created to allow some limited compatibility with older bots.
Instead, SSGM 4.0 supports a new type of logging which server owners based on a TCP connection between the bot and the server. Once connected to the server bots will receive a steady stream of log messages through the connection.
In order to ease the process of updating bots to the new SSGM logging system it was decided to implement the existing ssgmlog and gamelog messages into the new system in the same format as they used in the old file based logging system. This means that bots can simply point messages to existing parsing procedures without having to significantly rewrite their log handling code.
Protocol
The logging protocol is extremely simple. The first three characters if each message sent through the protocol are digits between 0 and 9 which indicate the message type. The remainder of the message is the log entry. Each message is terminated with the NULL character (0x0).
There are 4 built in message types;
000 - SSGM log messages
001 - Gamelog messages
002 - Renlog messages (a copy of everything written to the renlog files)
003 - Console Output (a copy of everything written to the console, basically a copy of 002 without timestamps)
Here are some examples of messages a bot might receive through the TCP connection;
001[14:06:20] POS;VEHICLE;1500000126;GDI_Ceiling_Gun_AGT;-66;21;9;0;100;100
001[14:06:21] POS;SOLDIER;1500000206;CnC_GDI_MiniGunner_0;-12;27;0;179;100;100
003Total current bandwidth usage for players is 85 kilobits per second
002[14:06:22] Total current bandwidth usage for players is 85 kilobits per second
003Westwood Online mode active since 11/08/2010 - 19:06:54
002[19:18:46] Westwood Online mode active since 11/08/2010 - 19:06:54
Important Note
For reasons which are too complicated to go into at this point the first line of the player_info response (the column headers) is output with a newline character (0xD) at the front of the line. If your bot needs to parse this line and would choke on that character you should use an appropriate function to strip that character out of any messages before forwarding them for parsing.
Configuration
SSGM 4.0 uses a configuration file named ssgm.ini, located in the data folder of the FDS installation. There are two important configuration parameters in this file for the logging system.
[General]
Port=8080
EnableGamelog=true
The port parameter defines the port number on which the server should listen for incoming TCP connections to the logging system and should be different from the RenRem and game ports defined in server.ini. Bots must read the port number from this file (or have it provided to them via config settings) to allow them to connect to the server.
The EnableGamelog option turns the logging system on or off and must be set to true to enable the logging system.
Testing
Right now your probably thinking something along the lines of "yeah, this is great and all, but how can I support something I can't test?". Don't worry, I thought of that problem in advance and have prepared a test server which you may connect your bots to in order to test their handling of the protocol.
Server: 82.11.23.160
TCP Log Port: 37854
You can interact with the test server by joining it (lmsbcdan in WOL or 82.11.23.160:4848 by direct connect) and by joining the IRC channel #tt_ssgm_bottest on irc.n00bstories.com where BRenBot 1.53 is running to help you test your bots responses to the TCP logging information.
Please be aware that this server is for bot testing purposes only and should not be used for normal gameplay... apart from anything else it will probably lag due to being hosted on a fairly slow connection.
Server is now offline due to low demand, if you need the server running again to test your bot(s) please send me a PM and I will load it up again.
More Information / Help
If you need any more information or need help with anything related to this new logging system please reply here or send me a PM on these forums.
Have fun testing!
[Updated on: Sun, 26 September 2010 02:12] Report message to a moderator
|
|
|
|
|
Re: Introduction to the SSGM 4.0 TCP Logging System [message #434701 is a reply to message #434694] |
Fri, 13 August 2010 07:42 |
|
danpaul88
Messages: 5795 Registered: June 2004 Location: England
Karma: 0
|
General (5 Stars) |
|
|
It's just sent as ASCII over the network, as shown in the protocol outlined above. Crude but effective.
As for using TCP, because it allows multiple simultaneous connections it allows you to do some useful stuff like off-server rank calculations on a separate bot dedicated to the task, where it doesn't matter if it lags behind real time as it doesn't need to respond to user input. At busy times it could buffer input and process it as fast as it can, and catch up to real time during an idle period.
Lots of possibilities...
[Updated on: Fri, 13 August 2010 07:43] Report message to a moderator
|
|
|
|
Re: Introduction to the SSGM 4.0 TCP Logging System [message #434703 is a reply to message #434702] |
Fri, 13 August 2010 07:58 |
|
danpaul88
Messages: 5795 Registered: June 2004 Location: England
Karma: 0
|
General (5 Stars) |
|
|
reborn wrote on Fri, 13 August 2010 15:47 | The built in logging of renlog and console output, have you hooked these functions and stopped them being written to disk, and instead being sent via the TCP connection, or are they still being written to disk, too?
|
danpaul88 wrote on Fri, 13 August 2010 14:49 | However the renlog files will still be created to allow some limited compatibility with older bots.
|
Some still *some* disk logging, but nowhere near as much.
[Updated on: Fri, 13 August 2010 07:58] Report message to a moderator
|
|
|
|
Re: Introduction to the SSGM 4.0 TCP Logging System [message #434705 is a reply to message #434694] |
Fri, 13 August 2010 08:02 |
|
reborn
Messages: 3231 Registered: September 2004 Location: uk - london
Karma: 0
|
General (3 Stars) |
|
|
Sorry, I over-looked that. it's so rare that something awesome happens around renegade that I read it in a rush.
How does that work then? Does it read the disk file and send it off, or does it intercept that information before it's written to renlog?
It's great to see some news come out of this forum, well done Dan Paul.
|
|
|
|
Re: Introduction to the SSGM 4.0 TCP Logging System [message #434710 is a reply to message #434694] |
Fri, 13 August 2010 08:31 |
|
reborn
Messages: 3231 Registered: September 2004 Location: uk - london
Karma: 0
|
General (3 Stars) |
|
|
The servers you play in log all the events and shit that happens in-game to the hard drives, this is called disk input-output. It's a bottle neck and is CPU intensive, so a server's SFPS can drop due to this (OMG LAG!).
Not logging to disk stops that. Meaning less server lag and the ability to increase the max player limit.
|
|
|
|
Re: Introduction to the SSGM 4.0 TCP Logging System [message #434714 is a reply to message #434710] |
Fri, 13 August 2010 08:45 |
|
Starbuzzz
Messages: 1637 Registered: June 2008
Karma: 0
|
General (1 Star) |
|
|
Hitman wrote on Fri, 13 August 2010 10:22 | and what does all that mean in a language people can understand?
|
thanks for the question!
reborn wrote on Fri, 13 August 2010 10:31 | The servers you play in log all the events and shit that happens in-game to the hard drives, this is called disk input-output. It's a bottle neck and is CPU intensive, so a server's SFPS can drop due to this (OMG LAG!).
Not logging to disk stops that. Meaning less server lag and the ability to increase the max player limit.
|
and thanks for the answer!
edit:
and thanks for the update danpaul!
[Updated on: Fri, 13 August 2010 08:45] Report message to a moderator
|
|
|
Re: Introduction to the SSGM 4.0 TCP Logging System [message #434715 is a reply to message #434694] |
Fri, 13 August 2010 08:47 |
|
halo2pac
Messages: 659 Registered: December 2006 Location: Near Cleveland, Ohio
Karma: 0
|
Colonel |
|
|
Omar007 wrote on Fri, 13 August 2010 10:38 | This is pretty awesome guys
Though just wondering; why not make a bot (embedded) in scripts.dll??
|
Its called Yarr, I believe and Daniel made it.
Some people (including me) want this functionality because its just awesome to create your own bot. Especially if you know how.
I know a million ways to make a bot and I have but its private, because I barely work on since I have a life and I program for a living. That and I'm getting married soon.
Also [HB] is too complicated since it is like 6 bots in one so it has to be run a quad core. I added text messaging commands but that was a waste of time. that and TS voice commands.
Rene-Buddy | Renegade X
Join the fight against Obsessive-Compulsive Posting Disorder. Cancel is ur friend.
*Renegade X Dev Team Member*
[Updated on: Sun, 15 August 2010 15:24] Report message to a moderator
|
|
|
|
|
|
|
Re: Introduction to the SSGM 4.0 TCP Logging System [message #434796 is a reply to message #434694] |
Sat, 14 August 2010 22:44 |
|
I dont know when the actual proper SSGM 4.0 testing server (the one normal players are supposed to join and play on) will be live, we want to make sure its as bug free as we can get it first.
I believe BlackIntel will be hosting it on one of their boxes.
Jonathan Wilson aka Jonwil
Creator and Lead Coder of the Custom scripts.dll
Renegade Engine Guru
Creator and Lead Coder of TT.DLL
Official member of Tiberian Technologies
|
|
|
|
|
|
Re: Introduction to the SSGM 4.0 TCP Logging System [message #434958 is a reply to message #434942] |
Wed, 18 August 2010 00:28 |
|
reborn
Messages: 3231 Registered: September 2004 Location: uk - london
Karma: 0
|
General (3 Stars) |
|
|
Ethenal wrote on Tue, 17 August 2010 20:20 | So if I'm reading correctly, some logs will still be written to disk (renlog), but now the SSGM and gamelogs? If so, I understand why, but that'll break all the older bots and I don't foresee any up-to-date replacements appearing. However, I imagine it wouldn't take too long to add the code for logging it to disk again so I suppose that doesn't really matter.
Either way, great job! Some places have used a system like this before and it has actually made a significant difference. So hopefully we'll see some improvements in lag.
P.S. Is it just me who cringes at halo's posts?
|
I did this for the APB server and thought it would take Oxi ages to re-write the bot to manage this new method. He was done in less than an hour. I do not know which languages other bots are written in, but PERL is extremely easy to open up and start receiving TCP connections (or infact any socket).
The main bulk of functions in the renegade bots are to process the data they are reading, the functions to read the data make up very little of the bot itself.
|
|
|
Re: Introduction to the SSGM 4.0 TCP Logging System [message #434969 is a reply to message #434694] |
Wed, 18 August 2010 03:13 |
StealthEye
Messages: 2518 Registered: May 2006 Location: The Netherlands
Karma: 0
|
General (2 Stars) |
|
|
It isn't a huge difference whether you are reading from a file or from a socket. The only significant difference is that a socket may disconnect at any time, which shouldn't lead to crashes or unwanted behavior. Otherwise, it's most just find&replacing a few things and that's it; especially if the application was well-designed, it should be an easy change.
Anyway, you can always make a new bot that simply listens to the socket and echos the data to a file. It's not preferred, but it is a possibility for old bots for which the source is unavailable.
BlackIntel admin/founder/coder
Please visit http://www.blackintel.org/
[Updated on: Wed, 18 August 2010 03:14] Report message to a moderator
|
|
|
Re: Introduction to the SSGM 4.0 TCP Logging System [message #434971 is a reply to message #434694] |
Wed, 18 August 2010 03:21 |
|
danpaul88
Messages: 5795 Registered: June 2004 Location: England
Karma: 0
|
General (5 Stars) |
|
|
When I created this topic BRenBot had already been updated to fully support the new logging system, that's what is running on the test server for the SSGM 4.0 TCP logging system (see first post). I know for a fact that nightma12 has seen this topic (I linked him to it) and I believe he will be updating NR to support it to.
Due to the log messages being exactly the same, just sent via a socket instead of the disk, it should be relatively simple to update bots to connect to a socket, check the first three digits for the message type and pass it to their existing code to handle the various message types.
[Updated on: Wed, 18 August 2010 03:22] Report message to a moderator
|
|
|
|