websocket 逆向(各直播平台弹幕获取)

声明：本内容仅供学习参考，本人对任何原因在使用本人中提供的代码和策略时可能对用户自己或他人造成的任何形式的损失和伤害不承担责任。如有侵权，请联系我进行删除。

什么是 websocket

WebSocket 是一种网络通信协议，它允许客户端（通常是浏览器）和服务器之间进行全双工通信。这意味着服务器可以主动发送信息给客户端，而不是像传统的网络协议（例如 HTTP）那样只能响应客户端的请求。

WebSocket 使用了一个持久的连接，这意味着只需建立一个连接就可以在客户端和服务器之间进行多次通信。这使得 WebSocket 特别适合于实时通信场景，例如聊天应用或实时数据可视化。

WebSocket 使用了一个新的协议，称为"ws"（无加密）或"wss"（加密），来代替传统的 HTTP 协议。这使得它能够使用端口 80 和 443，与 HTTP 和 HTTPS 共享端口。

获取弹幕有哪些方案

安卓+auto.js
浏览器脚本替换/浏览器插件
websocket 逆向 (推荐,稳定,数据全)

websocket 消息获取

如何通过 DOM 断点获取消息

如何本地替换脚本

浏览器替换脚本
浏览器插件替换脚本

如何用 puppeteer 或 playwright 实现脚本替换

核心代码示例

import puppeteer from "puppeteer-core";
// 没有实例的时候创建浏览器
browser = await puppeteer.launch({
  headless: true,
  slowMo: 250,
  devtools: false,
  args: [
    "--no-sandbox",
    "--disable-setuid-sandbox",
    "--use-gl=egl",
    // "--disable-gpu",
    // "--disable-dev-shm-usage",
    // "--no-first-run",
    // "--no-zygote",
    // "--single-process",
  ],
  executablePath:
    os.platform() == "linux"
      ? "/usr/bin/chromium-browser"
      : puppeteer.executablePath(),
});
// room.page = browser;
browser.on("disconnected", () => {
  log.info("浏览器关闭");
  room.ws = null;
});

const page = await browser.newPage();
const con = fs.readFileSync(path.join(__dirname, "./js.js"), "utf-8");
await page.setRequestInterception(true);
page.on("request", async (request) => {
  if (
    request.url().match(/goofy\/douyin_live\/resource\/async2\/Living\.\w+\.js/)
  ) {
    request.respond({
      status: 200,
      contentType: "application/javascript",
      body: con,
    });
    log.info("脚本替换成功");
  } else {
    request.continue();
  }
});
await page.goto(`https://live.douyin.com/${room.id}`, { timeout: 0 });

pkg 打包应用

https://github.com/vercel/pkg

浏览器插件

核心 api

const targetNode = document.querySelector(".webcast-chatroom___items div");
let config = {
  attributes: false, //目标节点的属性变化
  childList: true, //目标节点的子节点的新增和删除
  characterData: false, //如果目标节点为characterData节点(一种抽象接口,具体可以为文本节点,注释节点,以及处理指令节点)时,也要观察该节点的文本内容是否发生变化
  subtree: false, //目标节点所有后代节点的attributes、childList、characterData变化
};

// 当观察到变动时执行的回调函数
const callback = function (mutationsList, observer) {
  // Use traditional 'for loops' for IE 11
  for (let mutation of mutationsList) {
    if (mutation.type === "childList") {
      console.log("A child node has been added or removed.", mutation);
      if (mutation.addedNodes.length) {
        let current = mutation.addedNodes[0];
        console.log("msg:", current.innerText);
      }
    } else if (mutation.type === "attributes") {
      console.log("The " + mutation.attributeName + " attribute was modified.");
    }
  }
};

var observe = new MutationObserver(callback);

observe.observe(targetNode, config);

如果通过 ws 连接获取消息

常见数据格式

JSON（JavaScript Object Notation）是一种轻量级的数据交换格式，易于人阅读和编写。它基于 JavaScript 语言的一个子集，但是几乎所有编程语言都支持对 JSON 的解析和生成。JSON 具有良好的可读性，因此常用于数据交换的格式。

MessagePack（也称为 MsgPack）是一种二进制序列化格式，旨在比 JSON 更小、更快。它通过使用二进制来代替文本来节省空间，并且还支持更多的数据类型。它比 JSON 更小，因此常用于传输数据的场景，例如在网络通信中。

Protocol Buffers（也称为 Protobuf）是一种广泛使用的数据序列化格式，由 Google 开发。它和 MessagePack 类似，都是二进制格式，但是 Protobuf 有一些额外的功能，例如支持更多的数据类型和更复杂的结构。Protobuf 还有一个优点是它提供了编译器，可以将描述数据结构的.proto 文件编译成不同语言的代码。这使得在不同的系统之间传输数据变得更容易。

如何逆向 ws

protobuf

protobuf.js

message Person {
  optional string name = 1;
  optional int32 id = 2;
  optional string email = 3;
}

message PushFrame{
   string  seqid = 1
   string  login = 2
}

完整的示例

syntax = "proto3";


message CSWebHeartbeat {
  uint32 timestamp = 1;
}

message SocketMessage {
     PayloadType payloadType = 1;
     CompressionType compressionType = 2;
     bytes payload = 3;

    enum CompressionType {
        UNKNOWN = 0;
        NONE = 1;
        GZIP = 2;
        AES = 3;
    }
}

enum PayloadType {
    UNKNOWN = 0;
    CS_HEARTBEAT = 1;
    CS_ERROR = 3;
    CS_PING = 4;
    PS_HOST_INFO = 51;
    SC_HEARTBEAT_ACK = 101;
    SC_ECHO = 102;
    SC_ERROR = 103;
    SC_PING_ACK = 104;
    SC_INFO = 105;
    CS_ENTER_ROOM = 200;
    CS_USER_PAUSE = 201;
    CS_USER_EXIT = 202;
    CS_AUTHOR_PUSH_TRAFFIC_ZERO = 203;
    CS_HORSE_RACING = 204;
    CS_RACE_LOSE = 205;
    CS_VOIP_SIGNAL = 206;
    SC_ENTER_ROOM_ACK = 300;
    SC_AUTHOR_PAUSE = 301;
    SC_AUTHOR_RESUME = 302;
    SC_AUTHOR_PUSH_TRAFFIC_ZERO = 303;
    SC_AUTHOR_HEARTBEAT_MISS = 304;
    SC_PIP_STARTED = 305;
    SC_PIP_ENDED = 306;
    SC_HORSE_RACING_ACK = 307;
    SC_VOIP_SIGNAL = 308;
    SC_FEED_PUSH = 310;
    SC_ASSISTANT_STATUS = 311;
    SC_REFRESH_WALLET = 312;
    SC_LIVE_CHAT_CALL = 320;
    SC_LIVE_CHAT_CALL_ACCEPTED = 321;
    SC_LIVE_CHAT_CALL_REJECTED = 322;
    SC_LIVE_CHAT_READY = 323;
    SC_LIVE_CHAT_GUEST_END = 324;
    SC_LIVE_CHAT_ENDED = 325;
    SC_RENDERING_MAGIC_FACE_DISABLE = 326;
    SC_RENDERING_MAGIC_FACE_ENABLE = 327;
    SC_RED_PACK_FEED = 330;
    SC_LIVE_WATCHING_LIST = 340;
    SC_LIVE_QUIZ_QUESTION_ASKED = 350;
    SC_LIVE_QUIZ_QUESTION_REVIEWED = 351;
    SC_LIVE_QUIZ_SYNC = 352;
    SC_LIVE_QUIZ_ENDED = 353;
    SC_LIVE_QUIZ_WINNERS = 354;
    SC_SUSPECTED_VIOLATION = 355;
    SC_SHOP_OPENED = 360;
    SC_SHOP_CLOSED = 361;
    SC_GUESS_OPENED = 370;
    SC_GUESS_CLOSED = 371;
    SC_PK_INVITATION = 380;
    SC_PK_STATISTIC = 381;
    SC_RIDDLE_OPENED = 390;
    SC_RIDDLE_CLOESED = 391;
    SC_RIDE_CHANGED = 412;
    SC_BET_CHANGED = 441;
    SC_BET_CLOSED = 442;
    SC_LIVE_SPECIAL_ACCOUNT_CONFIG_STATE = 645;
}

message CSWebEnterRoom {

     string token = 1;
     string liveStreamId = 2;
     uint32 reconnectCount = 3;
     uint32 lastErrorCode = 4;
     string expTag = 5;
     string attach = 6;
     string pageId = 7;
}

message SCWebFeedPush {
     string displayWatchingCount = 1;
     string displayLikeCount = 2;
     uint64 pendingLikeCount = 3;
     uint64 pushInterval = 4;
    repeated WebCommentFeed commentFeeds = 5;
     string commentCursor = 6;
    repeated WebComboCommentFeed comboCommentFeed = 7;
    repeated WebLikeFeed likeFeeds = 8;
    repeated WebGiftFeed giftFeeds = 9;
    string giftCursor = 10;
    repeated WebSystemNoticeFeed systemNoticeFeeds = 11;
    repeated WebShareFeed shareFeeds = 12;

}

message WebCommentFeed {
     string id = 1;
     SimpleUserInfo user = 2;
     string content = 3;
     string deviceHash = 4;
     uint64 sortRank = 5;
     string color = 6;
     WebCommentFeedShowType showType = 7;
}



enum WebCommentFeedShowType {
    FEED_SHOW_UNKNOWN = 0;
    FEED_SHOW_NORMAL = 1;
    FEED_HIDDEN = 2;
}

message WebComboCommentFeed {
     string id = 1;
     string content = 2;
     uint32 comboCount = 3;
}

message WebLikeFeed {
     string id = 1;
     SimpleUserInfo user = 2;
     uint64 sortRank = 3;
     string deviceHash = 4;
}

message WebGiftFeed {
     string id = 1;
     SimpleUserInfo user = 2;
     uint64 time = 3;
     uint32 giftId = 4;
     uint64 sortRank = 5;
     string mergeKey = 6;
     uint32 batchSize = 7;
     uint32 comboCount = 8;
     uint32 rank = 9;
     uint64 expireDuration = 10;
     uint64 clientTimestamp = 11;
     uint64 slotDisplayDuration = 12;
     uint32 starLevel = 13;
     StyleType styleType = 14;
     WebLiveAssistantType liveAssistantType = 15;
     string deviceHash = 16;
     bool danmakuDisplay = 17;

    enum StyleType {
        UNKNOWN_STYLE = 0;
        BATCH_STAR_0 = 1;
        BATCH_STAR_1 = 2;
        BATCH_STAR_2 = 3;
        BATCH_STAR_3 = 4;
        BATCH_STAR_4 = 5;
        BATCH_STAR_5 = 6;
        BATCH_STAR_6 = 7;
    }
}

enum WebLiveAssistantType {
    UNKNOWN_ASSISTANT_TYPE = 0;
    SUPER = 1;
    JUNIOR = 2;
}

message WebSystemNoticeFeed {
     string id = 1;
     SimpleUserInfo user = 2;
     uint64 time = 3;
     string content = 4;
     uint64 displayDuration = 5;
     uint64 sortRank = 6;
     DisplayType displayType = 7;

    enum DisplayType {
        UNKNOWN_DISPLAY_TYPE = 0;
        COMMENT = 1;
        ALERT = 2;
        TOAST = 3;
    }
}

message WebShareFeed {
     string id = 1;
     SimpleUserInfo user = 2;
     uint64 time = 3;
     uint32 thirdPartyPlatform = 4;
     uint64 sortRank = 5;
     WebLiveAssistantType liveAssistantType = 6;
     string deviceHash = 7;
}

message SimpleUserInfo {
     string principalId = 1;
     string userName = 2;
     string headUrl = 3;
}
message LiveAudienceState{
     bool isFromFansTop = 1;
     bool isKoi = 2;
     AssistantType assistantType=3;
     uint32 fansGroupIntimacyLevel = 4;
     GzoneNameplate nameplate = 5;
     LiveFansGroupState LiveFansGroupState = 6;
     uint32 wealthGrade = 7;
     string badgeKey = 8;

    enum AssistantType {
        UNKNOWN_ASSISTANT_TYPE = 0;
        SUPER = 1;
        JUNIOR = 2;
    }
}
message GzoneNameplate {
         int64 fields=1;
         string name=2;
    }

message LiveFansGroupState {
         uint32 intimacyLevel=1;
         uint32 enterRoomSpecialEffect=2;
    }

message SCWebLiveWatchingUsers{
    repeated WebWatchingUserInfo watchingUser=1;
    string displayWatchingCount=2;
    uint64 pendingDuration=3;
}


message WebWatchingUserInfo{
    SimpleUserInfo user=1;
    bool offline=2;
    bool tuhao=3;
    int32 liveAssistantType=4;
    string displayKsCoin=5;

}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248

编译 protobuf

# npm install protobufjs-cli

npx pbjs -t json-module -w commonjs message.proto > message.js

npx pbjs -t json message.proto > message.json

const protobuf = require("protobufjs");
var root = protobuf.Root.fromJSON(require("./message.json"));
let PushFrame = root.lookup("PushFrame")
let Response = root.lookupType("Response")

// 反序列化
var buf = Buffer.from(params.response.payloadData, 'base64');
let o = PushFrame.decode(new Uint8Array(buf))

再结合puppeteer/playwright

核心代码

const browser = await chromium.launchPersistentContext(
      path.resolve(process.execPath, '../Resources/userData'),
      {
        channel: 'chrome',
        headless: false,
        args: ['--disable-blink-features=AutomationControlled']
      }
    )
    const page = await browser.newPage()
    await page.on('websocket', (ws) => {
      console.log(`WebSocket opened: ${ws.url()}>`)
      ws.on('framereceived', (event) => {
        DmByte({ msg: event.payload, platform: platform }, (msg) => {
          IpcMainListener.sendMessageToClient(msg)
        })
      })
      ws.on('close', () => console.log('WebSocket closed'))
    })
    await page.goto(url, { timeout: 0 })
    IpcMainListener.disposeArr.push(() => {
      browser.close()
    })

再结合浏览器插件实时抓取

chrome 扩展官方文档

// 开启 debug
 chrome.debugger.sendCommand({ tabId }, 'Network.enable', null, () => {
    if (browser.runtime.lastError)
      console.error('runtimeerr', browser.runtime.lastError.message)
    else console.log('Network enabled')
  })

// 监听 socket包含 xhr
chrome.debugger.onEvent.addListener(
  (debug: any, message: any, params: any) => {
    if (debug.tabId !== tabId)
      return

    if (message === 'Network.webSocketFrameReceived' && p !== 'wx') {
      let t = params.response.payloadData
      if (t) {
        if (typeof t == 'string') {
          try {
            t = JSON.parse(t)
          }
          catch (e) {
            // console.log(e)
          }
          const buf = base64ToBuffer(t)
          DmByte.handleMessage({ msg: buf, platform: p === 'douyin' ? 'dy' : 'tt' }, (data: any) => {
            postDmMessageToAll(data)
          })
        }
      }
    }
    else if (message === 'Network.responseReceived' && p === 'wx') {
      if (params.type === 'XHR' && params.response.url === 'https://channels.weixin.qq.com/cgi-bin/mmfinderassistant-bin/live/msg') {
        chrome.debugger.sendCommand({
          tabId: debug.tabId,
        }, 'Network.getResponseBody', {
          requestId: params.requestId,
        }, (response: { body: string }) => {
          try {
            const res = JSON.parse(response.body)
            const data = res.data
            const messageList = data.msgList
            if (messageList.length) {
              // 添加一个缓冲池，防止重复
              messageList.forEach((v: any) => {
                // 如果v存在listBuffer就不添加，通过v.seq判断,找出v在listBuffer中的位置
                const index = listBuffer.findIndex(item => item.seq === v.seq)
                if (index !== -1) {
                  listBuffer.splice(index, 1)
                  return
                }

                listBuffer.push(v)
                if (listBuffer.length > 20)
                  listBuffer.shift()

                const _data = {
                  roomId: 'wx',
                  data: {
                    type: 2,
                    typeText: 'chat',
                    message: v.content,
                    face: v.headUrl,
                    uname: v.nickname,
                    uid: v.username,
                    fans: '',
                    platform: 'wx',
                  },
                }

                dmList.value.push(_data)
                window.postMessage(_data, '*')
              })
            }
          }
          catch (error) {

          }
        })
      }
    }
  },
)

与互动直播的结合应用

可直接集成到 Unity/PyQT/H5/Electron

扩展

浏览器插件直接与原生应用如 Unity 通讯

websocket 逆向(各直播平台弹幕获取) ​

什么是 websocket ​

获取弹幕有哪些方案 ​

websocket 消息获取 ​

如何通过 DOM 断点获取消息 ​

如何本地替换脚本 ​

如何用 puppeteer 或 playwright 实现脚本替换 ​

pkg 打包应用 ​

浏览器插件 ​

如果通过 ws 连接获取消息 ​

常见数据格式 ​

如何逆向 ws ​

再结合puppeteer/playwright ​

再结合浏览器插件实时抓取 ​

与互动直播的结合应用 ​

扩展 ​