Support for new Steam chat

[JustArchi]

Today Steam got new chat update that is trying to mimic Discord in majority of aspects. I'm wondering if we could hope for at least basic at first, SK2 support for it.

I took a quick look and the good news are that we can use good old Steam protocol for that, which makes it fall into SK2 scope, but the bad news are that it might require some extra work:

Currently NHA2 is missing relevant bits, it looks like #477 proposed a fix, we might need something like that to fully support it.

Do you have a plan to add proper support for it in near future? I even wanted to start working on it right away but since SK2 doesn't even have a specification for EMsg of 151, I'm not sure what should be done, as my knowledge of SK2 internals is rather poor. It doesn't look like it's just a body definition. If we could get some basic support for sending and receiving packets, then later on we could write proper (new) handlers for that, such as SteamFriends.

It might be even already possible to parse and send those packets from within SK2, but I'm not sure how to achieve that. Please let me know if you do.

BTW, this chat is also fully supported in web browser - I also took a quick look and web browser achieves that with websocket connection to CM servers. Nothing interesting for us specifically probably, but worth mentioning.

[yaakov-h]

Check out the steamclient-beta branch and #565.

[JustArchi]

Trying to start with ChatRoom.SendChatMessage#1 but I'm probably too stupid for this 😅

I guess I'll just wait for finished implementation instead, unless you have extra surplus of time and willings to check below and find out what is wrong 😓

internal sealed class ArchiBetaHandler : ClientMsgHandler {
	public override void HandleMsg(IPacketMsg packetMsg) { }

	internal async Task<bool> SendChatMessage(ulong chatGroupID, ulong chatID, string message) {
		CChatRoom_SendChatMessage_Request request = new CChatRoom_SendChatMessage_Request {
			chat_group_id = chatGroupID,
			chat_id = chatID,
			message = message
		};

		try {
			await SendMessage("ChatRoom.SendChatMessage#1", request);
		} catch {
			return false;
		}

		return true;
	}

	private AsyncJob<SteamUnifiedMessages.ServiceMethodResponse> SendMessage<TRequest>(string name, TRequest message, bool isNotification = false) where TRequest : IExtensible {
		if (message == null) {
			throw new ArgumentNullException(nameof(message));
		}

		ClientMsgProtobuf<CMsgClientServiceMethod> msg = new ClientMsgProtobuf<CMsgClientServiceMethod>(EMsg.ServiceMethodCallFromClient) { SourceJobID = Client.GetNextJobID() };

		using (MemoryStream ms = new MemoryStream()) {
			Serializer.Serialize(ms, message);
			msg.Body.serialized_method = ms.ToArray();
		}

		msg.Body.method_name = name;
		msg.Body.is_notification = isNotification;

		Client.Send(msg);

		return new AsyncJob<SteamUnifiedMessages.ServiceMethodResponse>(Client, msg.SourceJobID);
	}
}

[Serializable]
[ProtoContract(Name = nameof(CChatRoom_SendChatMessage_Request))]
public class CChatRoom_SendChatMessage_Request : IExtensible {
	[ProtoMember(1, IsRequired = false, Name = "chat_group_id", DataFormat = DataFormat.FixedSize)]
	public ulong chat_group_id { get; set; }

	[ProtoMember(2, IsRequired = false, Name = "chat_id", DataFormat = DataFormat.FixedSize)]
	public ulong chat_id { get; set; }

	[ProtoMember(3, IsRequired = false, Name = "message", DataFormat = DataFormat.Default)]
	public string message { get; set; }

	private IExtension extensionObject;
	IExtension IExtensible.GetExtensionObject(bool createIfMissing) => Extensible.GetExtensionObject(ref extensionObject, createIfMissing);
}

[voided]

You'd probably want to take a look at the SteamUnifiedMessages handler. That plus your re-created protos may get you the rest of the way.

I would posit that you may not need to use the ServiceMethodCallFromClient EMsg, but I might be wrong. I don't see any reason this group chat stuff is any different than existing unified service messages.

https://github.com/SteamRE/SteamKit/blob/master/Samples/8.UnifiedMessages/Program.cs for usage.

[yaakov-h]

IIRC, the body of the message is the recreated proto request.

You would need to send a ClientMsgProtobuf<CChatRoom_SendChatMessage_Request> with the ServiceMethodCallFromClient EMsg value and set the job name in the header to ChatRoom.SendChatMessage#1.

[JustArchi]

Thank you both! I managed to get first working example, sharing it for reference:

internal sealed class ArchiBetaHandler : ClientMsgHandler {
	public override void HandleMsg(IPacketMsg packetMsg) { }

	internal async Task<SteamUnifiedMessages.ServiceMethodResponse> SendChatMessage(ulong chatGroupID, ulong chatID, string message) {
		ClientMsgProtobuf<CChatRoom_SendChatMessage_Request> request = new ClientMsgProtobuf<CChatRoom_SendChatMessage_Request>(EMsg.ServiceMethodCallFromClient) {
			Body = {
				chat_group_id = chatGroupID,
				chat_id = chatID,
				message = message
			},

			SourceJobID = Client.GetNextJobID()
		};

		request.Header.Proto.target_job_name = "ChatRoom.SendChatMessage#1";

		Client.Send(request);

		try {
			return await new AsyncJob<SteamUnifiedMessages.ServiceMethodResponse>(Client, request.SourceJobID);
		} catch {
			return null;
		}
	}
}

public class CChatRoom_SendChatMessage_Request : IExtensible {
	[ProtoMember(1, IsRequired = false, Name = "chat_group_id", DataFormat = DataFormat.Default)]
	public ulong chat_group_id { get; set; }

	[ProtoMember(2, IsRequired = false, Name = "chat_id", DataFormat = DataFormat.Default)]
	public ulong chat_id { get; set; }

	[ProtoMember(3, IsRequired = false, Name = "message", DataFormat = DataFormat.Default)]
	public string message { get; set; }

	private IExtension extensionObject;
	IExtension IExtensible.GetExtensionObject(bool createIfMissing) => Extensible.GetExtensionObject(ref extensionObject, createIfMissing);
}

Now I'll try to add all missing stuff, since right now even response doesn't return, but hey, it works 🎉

[JustArchi]

@voided you were right too, this also works:

var uniMessages = SteamClient.GetHandler<SteamUnifiedMessages>();

var request = new CChatRoom_SendChatMessage_Request {
	chat_group_id = 0, // must be valid
	chat_id = 0, // must be valid
	message = "Test2"
};

uniMessages.SendMessage("ChatRoom.SendChatMessage#1", request)

Then it's just a matter of reverse-engineering protobufs and hooking it to unified messages, perfect 🎉.

[JustArchi]

Note to self and other people to not waste productivity over stupid things:

ClientMsgProtobuf<CMsgClientUIMode> request = new ClientMsgProtobuf<CMsgClientUIMode>(EMsg.ClientCurrentUIMode) { Body = { chat_mode = 2 } };
Client.Send(request);

Send this to enable beta chat mode. Otherwise you won't receive majority of callbacks and won't be able to move forward.

... Don't ask how many hours I wasted debugging only to find out about this 😅

[JustArchi]

I've successfully written very basic protobufs for sending and receiving new Steam group messages, this is a good start - steamclient-beta...JustArchi:archi-wip

I'm not sending PR with this as we'll probably want to automate generation of those (I guess?), but feel free to make use of them for time being. I'll probably go with private messaging next.

I'm wondering whether we want to make a bit more user-friendly methods to access those, and how exactly they should look like. I mean, I can totally see SteamChatRoom.IncomingChatMessageCallback and SteamChatRoom.SendChatMessage() already possible, although I'm not sure yet how those handlers should work, as under the hood everything is nicely handled by SteamUnifiedMessages, it'd basically be a wrapper over IChatRoom that would do something similar to what I'm already doing at lower level:

private async void OnServiceMethod(SteamUnifiedMessages.ServiceMethodNotification callback) {
	switch (callback.MethodName) {
		case "ChatRoomClient.NotifyIncomingChatMessage#1":
			CChatRoom_IncomingChatMessage_Notification body = (CChatRoom_IncomingChatMessage_Notification) callback.Body;

			if (body.message == "ping") {
				CChatRoom_SendChatMessage_Request request = new CChatRoom_SendChatMessage_Request {
					chat_group_id = body.chat_group_id,
					chat_id = body.chat_id,
					message = "pong"
				};

				await ChatRoomService.SendMessage(x => x.SendChatMessage(request));
			}

			break;
	}
}

On the other hand it might make sense to make SteamUnifiedMessages.ServiceMethodNotification generic firstly, so we could subscribe more easily to given notifications - currently you can see this awful body cast. Entire switch logic could be easily moved somewhere deeper inside SK2, as SteamUnifiedMessages is already smart enough to map ChatRoomClient.NotifyIncomingChatMessage#1 into NotifyIncomingChatMessage from IChatRoomClient interface. At this point I'm not even sure if what I want to see is possible to do in C#, but it sounds like it could be, as basically we just want to catch all ServiceMethod EMsg with body of CChatRoom_IncomingChatMessage_Notification.

In any case, it starts looking really good, thanks again for initial help, I'll shut up now and let you work in peace 😀. Those are just random ideas, it's your decision whether they make sense or not 🎉.

[yaakov-h]

I'll shut up now and let you work in peace 😀

Oh please don't, you've been very useful. 😉

Currently, between @DoctorMcKay and @xPaw, we've got a rough .proto file out of the Javascript frontend: https://github.com/SteamDatabase/Protobufs/blob/ff8a4dbb6a1ad54e8248bf87617f4686244a6d85/steam/WebUI/friends.proto

I think the next step would be to clean that up so that we don't have to manually maintain all the new unified services:

• Reverse-engineer the proto service definition.
• Dump each service into it's own .proto file.
• Reverse-engineer the enums.
• Generate C# code from the new .proto files.
• Wrap it with either the SteamFriends handler, or a new one entirely.