西安国内做网站的公司有哪些seo公司 杭州

日后，网络爬虫也好，数据采集也好，自动化必然是主流。因此，笔者未雨绸缪，在此研究各类自动化源码，希望能够赶上时代，做出一套实用的自动化框架。

这里先研究传统的webdriver中转来进行浏览器自动化的源码。

webdriver官方位于这里：WebDriver

用过selenium的同学应该都知道，需要有selenium这个自动化库来写脚本，需要一个webdriver.exe，还需要一个浏览器。流程基本如下：

编写脚本----->发送消息给webdriver------>webdriver发消息给浏览器

通过这样一个流程，就完成了自动化。

脚本简单，资料一大堆，浏览器也只是个执行者，所以关键在于webdriver如何接受、处理、发送消息，所以重点源码webdriver源码。

WebDriverMain.cpp：启动函数

#include "config.h"#include "LogInitialization.h"
#include "WebDriverService.h"
#include <wtf/MainThread.h>
#include <wtf/Threading.h>#if OS(ANDROID)
__attribute__((visibility("default")))
int WebKit::WebDriverProcessMain(int argc, char** argv)
#else
int main(int argc, char** argv)
#endif
{WebDriver::WebDriverService::platformInit();WTF::initializeMainThread();
#if !LOG_DISABLED || !RELEASE_LOG_DISABLEDWebDriver::logChannels().initializeLogChannelsIfNecessary(WebDriver::logLevelString());
#endifWebDriver::WebDriverService service;return service.run(argc, argv);
}

步骤 1：平台初始化：platformInit源码里空白无实现，应该是等之后更新。

步骤 2：WTF（Web Template Framework）主线程初始化

步骤 3：日志通道初始化

步骤 4：创建并运行 WebDriver 服务

WebDriverService.cpp：解析命令参数，进入监听loop循环。。

if (const char* targetEnvVar = getenv("WEBDRIVER_TARGET_ADDR"))targetString = String::fromLatin1(targetEnvVar);

先获取一个名为WEBDRIVER_TARGET_ADDR的环境变量，这个变量是用于链接已经开启的浏览器，是的，需要先把浏览器打开，然后webdriver回去链接，而不是先运行webdriver。

然后解析命令行参数，一大堆，没什么可看的：

        if (equalSpans(arg, "-h"_span) || equalSpans(arg, "--help"_span)) {printUsageStatement(argv[0]);return EXIT_SUCCESS;}if (equalSpans(arg, "-p"_span) && portString.isNull()) {if (++i == argc) {printUsageStatement(argv[0]);return EXIT_FAILURE;}portString = String::fromLatin1(argv[i]);continue;}static constexpr auto portArgument = "--port="_span;if (spanHasPrefix(arg, portArgument) && portString.isNull()) {portString = arg.subspan(portArgument.size());continue;}static constexpr auto hostArgument = "--host="_span;if (spanHasPrefix(arg, hostArgument) && !host) {host = arg.subspan(hostArgument.size());continue;}#if ENABLE(WEBDRIVER_BIDI)static constexpr auto bidiPortArgument = "--bidi-port="_span;if (spanHasPrefix(arg, bidiPortArgument) && bidiPortString.isNull()) {bidiPortString = arg.subspan(bidiPortArgument.size());continue;}
#endifif (equalSpans(arg, "-t"_span) && targetString.isNull()) {if (++i == argc) {printUsageStatement(argv[0]);return EXIT_FAILURE;}targetString = String::fromLatin1(argv[i]);continue;}static constexpr auto targetArgument = "--target="_span;if (spanHasPrefix(arg, targetArgument) && targetString.isNull()) {targetString = arg.subspan(targetArgument.size());continue;}if (equalSpans(arg, "--replace-on-new-session"_span)) {m_replaceOnNewSession = true;continue;}}if (portString.isNull()) {printUsageStatement(argv[0]);return EXIT_FAILURE;}if (!targetString.isEmpty()) {auto position = targetString.reverseFind(':');if (position != notFound) {m_targetAddress = targetString.left(position);m_targetPort = parseIntegerAllowingTrailingJunk<uint16_t>(StringView { targetString }.substring(position + 1)).value_or(0);}}auto port = parseInteger<uint16_t>(portString);if (!port) {fprintf(stderr, "Invalid port %s provided\n", portString.utf8().data());return EXIT_FAILURE;}#if ENABLE(WEBDRIVER_BIDI)auto bidiPort = parseInteger<uint16_t>(bidiPortString);if (!bidiPort) {const int16_t bidiPortIncrement = *port == std::numeric_limits<uint16_t>::max() ? -1 : 1;bidiPort = { *port + bidiPortIncrement };fprintf(stderr, "Invalid WebSocket BiDi port %s provided. Defaulting to %d.\n", bidiPortString.utf8().data(), *bidiPort);}
#endif

最后是进入主循环，这里先线程初始化，然后看是websockst还是http模式，前者双向，后者单向。之后如果监听listen成功，就进入loop的循环不断接受消息：

    WTF::initializeMainThread();const char* hostStr = host && host->utf8().data() ? host->utf8().data() : "local";
#if ENABLE(WEBDRIVER_BIDI)if (!m_bidiServer.listen(host ? *host : nullString(), *bidiPort)) {fprintf(stderr, "FATAL: Unable to listen for WebSocket BiDi server at host %s and port %d.\n", hostStr, *bidiPort);return EXIT_FAILURE;}RELEASE_LOG(WebDriverBiDi, "Started WebSocket BiDi server with host %s and port %d", hostStr, *bidiPort);
#endif // ENABLE(WEBDRIVER_BIDI)if (!m_server.listen(host, *port)) {fprintf(stderr, "FATAL: Unable to listen for HTTP server at host %s and port %d.\n", hostStr, *port);return EXIT_FAILURE;}RELEASE_LOG(WebDriverClassic, "Started HTTP server with host %s and port %d", hostStr, *port);RunLoop::run();#if ENABLE(WEBDRIVER_BIDI)m_bidiServer.disconnect();
#endifm_server.disconnect();return EXIT_SUCCESS;

其中的监听代码如下：

bool HTTPServer::listen(const std::optional<String>& host, unsigned port)
{auto& endpoint = RemoteInspectorSocketEndpoint::singleton();if (auto id = endpoint.listenInet(host ? host.value().utf8().data() : "", port, *this)) {m_server = id;return true;}return false;
}

这里先用了个设计模式：单例模式，为了可复用利用同一个监听端点。

然后是很常规的server模式，打开一个TCP端口监听。

std::optional<ConnectionID> RemoteInspectorSocketEndpoint::listenInet(const char* address, uint16_t port, Listener& listener)
{Locker locker { m_connectionsLock };auto id = generateConnectionID();auto connection = makeUnique<ListenerConnection>(id, listener, address, port);if (!connection->isListening())return std::nullopt;m_listeners.add(id, WTFMove(connection));wakeupWorkerThread();return id;
}bool RemoteInspectorSocketEndpoint::isListening(ConnectionID id)
{Locker locker { m_connectionsLock };if (m_listeners.contains(id))return true;return false;
}

主循环代码如下，典型的一个GUI的事件消息处理机制，熟悉win32的同学应该很懂，而且这里就是纯粹的win32的函数接口，获取消息，把消息转化为字符消息，分发消息：

void RunLoop::run()
{MSG message;while (BOOL result = ::GetMessage(&message, nullptr, 0, 0)) {if (result == -1)break;::TranslateMessage(&message);::DispatchMessage(&message);}
}