diff --git a/.gitignore b/.gitignore index d89faf4..f4f3fbf 100644 --- a/.gitignore +++ b/.gitignore @@ -6,6 +6,7 @@ _old/ dist/ bin/ +venv/ /bundle/fonts/extra launch launch.exe diff --git a/README.md b/README.md index 553d12c..318af92 100644 --- a/README.md +++ b/README.md @@ -344,7 +344,8 @@ Miscellaneous (WebGl spoofing, battery status, etc) - Custom implementation of Playwright for the latest Firefox - Various config patches to evade bot detection -- Removed leaking Playwright patches: +- Fixes leaking Playwright patches: + - All page agent javascript is sandboxed - Fixes frame execution context leaks - Fixes `navigator.webdriver` detection - Removed potentially leaking anti-zoom/meta viewport handling patches @@ -356,6 +357,7 @@ Miscellaneous (WebGl spoofing, battery status, etc) - Stripped out/disabled _many, many_ Mozilla services. Runs faster than the original Mozilla Firefox, and uses less memory (200mb) - Includes the debloat config from PeskyFox & LibreWolf, and others - Speed optimizations from FastFox, and other network optimizations +- Removed all CSS animations - Minimalistic theming - etc. diff --git a/pythonlib/README.md b/pythonlib/README.md new file mode 100644 index 0000000..9956d8a --- /dev/null +++ b/pythonlib/README.md @@ -0,0 +1,114 @@ +# Camoufox Python Interface + +#### This is the Python library for Camoufox. + +> [!WARNING] +> This is still experimental and in active development! + +## Installation + +```bash +git clone https://github.com/camoufox +cd camoufox/pythonlib +pip install . +``` + +
+ +## Usage + +Camoufox is fully compatible with your existing Playwright code. You only have to change your browser initialization: + +#### Sync API + +```python +from camoufox.sync_api import Camoufox + +with Camoufox() as browser: + page = browser.new_page() + page.goto("https://www.browserscan.net/") +``` + +#### Async API + +```python +from camoufox.async_api import AsyncCamoufox + +async with AsyncCamoufox() as browser: + page = await browser.new_page() + await page.goto("https://www.browserscan.net/") +``` +
+Parameters + +``` +Launches a new browser instance for Camoufox. + +Parameters: + playwright (Playwright): + The playwright instance to use. + config (Optional[Dict[str, Any]]): + The configuration to use. + addons (Optional[List[str]]): + The addons to use. + fingerprint (Optional[Fingerprint]): + The fingerprint to use. + exclude_addons (Optional[List[DefaultAddons]]): + The default addons to exclude, passed as a list of camoufox.DefaultAddons enums. + screen (Optional[browserforge.fingerprints.Screen]): + The screen constraints to use. + os (Optional[ListOrString]): + The operating system to use for the fingerprint. Either a string or a list of strings. + user_agent (Optional[ListOrString]): + The user agent to use for the fingerprint. Either a string or a list of strings. + fonts (Optional[List[str]]): + The fonts to load into Camoufox, in addition to the default fonts. + args (Optional[List[str]]): + The arguments to pass to the browser. + executable_path (Optional[str]): + The path to the Camoufox browser executable. + **launch_options (Dict[str, Any]): + Additional Firefox launch options. +``` +
+ +--- + +### Config + +Camoufox [config data](https://github.com/daijro/camoufox?tab=readme-ov-file#fingerprint-injection) can be passed as a dictionary to the `config` parameter: + +```python +from camoufox import Camoufox + +with Camoufox( + config={ + 'webrtc:ipv4': '123.45.67.89', + 'webrtc:ipv6': 'e791:d37a:88f6:48d1:2cad:2667:4582:1d6d', + } +) as browser: + page = browser.new_page() + page.goto("https://www.browserscan.net/") +``` + +
+ +### BrowserForge Integration + +Camoufox is fully compatible with BrowserForge. + +By default, Camoufox will use a random fingerprint. You can also inject your own Firefox Browserforge fingerprint into Camoufox with the following example: + +```python +from camoufox.sync_api import Camoufox +from browserforge.fingerprints import FingerprintGenerator + +fg = FingerprintGenerator(browser='firefox') + +# Launch Camoufox with a random Firefox fingerprint +with Camoufox(fingerprint=fg.generate()) as browser: + page = browser.new_page() + page.goto("https://www.browserscan.net/") +``` + +--- \ No newline at end of file diff --git a/pythonlib/camoufox/__init__.py b/pythonlib/camoufox/__init__.py new file mode 100644 index 0000000..331e7ff --- /dev/null +++ b/pythonlib/camoufox/__init__.py @@ -0,0 +1,5 @@ +from .addons import DefaultAddons +from .async_api import AsyncCamoufox, AsyncNewBrowser +from .sync_api import Camoufox, NewBrowser + +__all__ = ["Camoufox", "NewBrowser", "AsyncCamoufox", "AsyncNewBrowser", "DefaultAddons"] diff --git a/pythonlib/camoufox/__main__.py b/pythonlib/camoufox/__main__.py new file mode 100644 index 0000000..689a4fa --- /dev/null +++ b/pythonlib/camoufox/__main__.py @@ -0,0 +1,112 @@ +""" +Binary CLI manager for Camoufox. + +Adapted from https://github.com/daijro/hrequests/blob/main/hrequests/__main__.py +""" + +from importlib.metadata import PackageNotFoundError +from importlib.metadata import version as pkg_version + +import click + +from .pkgman import CamoufoxFetcher, installed_verstr, rprint + + +class CamoufoxUpdate(CamoufoxFetcher): + """ + Checks & updates Camoufox + """ + + def __init__(self) -> None: + """ + Initializes the CamoufoxUpdate class + """ + super().__init__() + try: + self.current_verstr = installed_verstr() + except FileNotFoundError: + self.current_verstr = None + + def is_updated_needed(self) -> None: + if self.current_verstr is None: + return True + # If the installed version is not the latest version + if self.current_verstr != self.verstr: + return True + return False + + def update(self) -> None: + """ + Updates the library if needed + """ + # Check if the version is the same as the latest available version + if not self.is_updated_needed(): + rprint("Camoufox binaries up to date!", fg="green") + rprint(f"Current version: v{self.current_verstr}", fg="green") + return + + # Download updated file + if self.current_verstr is not None: + # Display an updating message + rprint( + f"Updating Camoufox binaries from v{self.current_verstr} => v{self.verstr}", + fg="yellow", + ) + else: + rprint(f"Fetching Camoufox binaries v{self.verstr}...", fg="yellow") + # Install the new version + self.install() + + +@click.group() +def cli() -> None: + pass + + +@cli.command(name='fetch') +def fetch(): + """ + Fetch the latest version of Camoufox + """ + CamoufoxUpdate().update() + + +@cli.command(name='remove') +def remove() -> None: + """ + Remove all library files + """ + if not CamoufoxUpdate().cleanup(): + rprint("Camoufox binaries not found!", fg="red") + + +@cli.command(name='version') +def version() -> None: + """ + Display the current version + """ + # python package version + try: + rprint(f"Pip package:\tv{pkg_version('camoufox')}", fg="green") + except PackageNotFoundError: + rprint("Pip package:\tNot installed!", fg="red") + + updater = CamoufoxUpdate() + bin_ver = updater.current_verstr + + # If binaries are not downloaded + if not bin_ver: + rprint("Camoufox:\tNot downloaded!", fg="red") + return + # Print the base version + rprint(f"Camoufox:\tv{bin_ver} ", fg="green", nl=False) + + # Check for library updates + if updater.is_updated_needed(): + rprint(f"(Latest: v{updater.verstr})", fg="red") + else: + rprint("(Up to date!)", fg="yellow") + + +if __name__ == '__main__': + cli() diff --git a/pythonlib/camoufox/addons.py b/pythonlib/camoufox/addons.py new file mode 100644 index 0000000..35502a2 --- /dev/null +++ b/pythonlib/camoufox/addons.py @@ -0,0 +1,184 @@ +import asyncio +import os +import socket +import threading +import time +from enum import Enum +from typing import List + +import orjson + +from .exceptions import InvalidAddonPath, InvalidDebugPort, MissingDebugPort + + +class DefaultAddons(Enum): + """ + Default addons to be downloaded + """ + + uBO = "https://addons.mozilla.org/firefox/downloads/latest/ublock-origin/latest.xpi" + BPC = "https://gitflic.ru/project/magnolia1234/bpc_uploads/blob/raw?file=bypass_paywalls_clean-latest.xpi" + + +def get_debug_port(args: List[str]) -> int: + """ + Gets the debug port from the args, or creates a new one if not provided + """ + for i, arg in enumerate(args): + # Search for debugger server port + if arg == "-start-debugger-server": + # If arg is found but no port is provided, raise an error + if i + 1 >= len(args): + raise MissingDebugPort(f"No debug port provided: {args}") + debug_port = args[i + 1] + # Try to parse the debug port as an integer + try: + return int(debug_port) + except ValueError as e: + raise InvalidDebugPort( + f"Error parsing debug port. Must be an integer: {debug_port}" + ) from e + + # Create new debugger server port + debug_port_int = get_open_port() + # Add -start-debugger-server {debug_port} to args + args.extend(["-start-debugger-server", str(debug_port_int)]) + + return debug_port_int + + +def confirm_paths(paths: List[str]) -> None: + """ + Confirms that the addon paths are valid + """ + for path in paths: + if not os.path.exists(path): + raise InvalidAddonPath(path) + + +def get_open_port() -> int: + """ + Gets an open port + """ + with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: + s.bind(('localhost', 0)) + return s.getsockname()[1] + + +def threaded_try_load_addons(debug_port_int: int, addons_list: List[str]) -> None: + """ + Tries to load addons (in a separate thread) + """ + thread = threading.Thread( + target=try_load_addons, args=(debug_port_int, addons_list), daemon=True + ) + thread.start() + + +def try_load_addons(debug_port_int: int, addons_list: List[str]) -> None: + """ + Tries to load addons + """ + # Wait for the server to be open + while True: + try: + with socket.create_connection(("localhost", debug_port_int)): + break + except ConnectionRefusedError: + time.sleep(0.05) + + # Load addons + asyncio.run(load_all_addons(debug_port_int, addons_list)) + + +async def load_all_addons(debug_port_int: int, addons_list: List[str]) -> None: + """ + Loads all addons + """ + addon_loaders = [LoadFirefoxAddon(debug_port_int, addon) for addon in addons_list] + await asyncio.gather(*[addon_loader.load() for addon_loader in addon_loaders]) + + +class LoadFirefoxAddon: + ''' + Firefox addon loader + https://github.com/daijro/hrequests/blob/main/hrequests/extensions.py#L95 + ''' + + def __init__(self, port, addon_path): + self.port: int = port + self.addon_path: str = addon_path + self.success: bool = False + self.buffers: list = [] + self.remaining_bytes: int = 0 + + async def load(self): + reader, writer = await asyncio.open_connection('localhost', self.port) + writer.write(self._format_message({"to": "root", "type": "getRoot"})) + await writer.drain() + + while True: + data = await reader.read(100) # Adjust buffer size as needed + if not data: + break + await self._process_data(writer, data) + + writer.close() + await writer.wait_closed() + return self.success + + async def _process_data(self, writer, data): + while data: + if self.remaining_bytes == 0: + index = data.find(b':') + if index == -1: + self.buffers.append(data) + return + + total_data = b''.join(self.buffers) + data + size, _, remainder = total_data.partition(b':') + + try: + self.remaining_bytes = int(size) + except ValueError as e: + raise ValueError("Invalid state") from e + + data = remainder + + if len(data) < self.remaining_bytes: + self.remaining_bytes -= len(data) + self.buffers.append(data) + return + else: + self.buffers.append(data[: self.remaining_bytes]) + message = orjson.loads(b''.join(self.buffers)) + self.buffers.clear() + + await self._on_message(writer, message) + + data = data[self.remaining_bytes :] + self.remaining_bytes = 0 + + async def _on_message(self, writer, message): + if "addonsActor" in message: + writer.write( + self._format_message( + { + "to": message["addonsActor"], + "type": "installTemporaryAddon", + "addonPath": self.addon_path, + } + ) + ) + await writer.drain() + + if "addon" in message: + self.success = True + writer.write_eof() + + if "error" in message: + writer.write_eof() + + def _format_message(self, data): + raw = orjson.dumps(data) + return f"{len(raw)}:".encode() + raw diff --git a/pythonlib/camoufox/async_api.py b/pythonlib/camoufox/async_api.py new file mode 100644 index 0000000..338f1da --- /dev/null +++ b/pythonlib/camoufox/async_api.py @@ -0,0 +1,88 @@ +from typing import Any, Dict, List, Optional + +from browserforge.fingerprints import Fingerprint, Screen +from playwright.async_api import Browser, Playwright, PlaywrightContextManager + +from .addons import DefaultAddons +from .utils import ListOrString, get_launch_options + + +class AsyncCamoufox(PlaywrightContextManager): + """ + Wrapper around playwright.async_api.PlaywrightContextManager that automatically + launches a browser and closes it when the context manager is exited. + """ + + def __init__(self, **launch_options): + super().__init__() + self.launch_options = launch_options + self.browser: Optional[Browser] = None + + async def __aenter__(self) -> Browser: + await super().__aenter__() + self.browser = await AsyncNewBrowser(self._playwright, **self.launch_options) + return self.browser + + async def __aexit__(self, *args: Any): + if self.browser: + await self.browser.close() + await super().__aexit__(*args) + + +async def AsyncNewBrowser( + playwright: Playwright, + *, + config: Optional[Dict[str, Any]] = None, + addons: Optional[List[str]] = None, + fingerprint: Optional[Fingerprint] = None, + exclude_addons: Optional[List[DefaultAddons]] = None, + screen: Optional[Screen] = None, + os: Optional[ListOrString] = None, + user_agent: Optional[ListOrString] = None, + fonts: Optional[List[str]] = None, + args: Optional[List[str]] = None, + executable_path: Optional[str] = None, + **launch_options: Dict[str, Any] +) -> Browser: + """ + Launches a new browser instance for Camoufox. + + Parameters: + playwright (Playwright): + The playwright instance to use. + config (Optional[Dict[str, Any]]): + The configuration to use. + addons (Optional[List[str]]): + The addons to use. + fingerprint (Optional[Fingerprint]): + The fingerprint to use. + exclude_addons (Optional[List[DefaultAddons]]): + The default addons to exclude, passed as a list of camoufox.DefaultAddons enums. + screen (Optional[browserforge.fingerprints.Screen]): + The screen constraints to use. + os (Optional[ListOrString]): + The operating system to use for the fingerprint. Either a string or a list of strings. + user_agent (Optional[ListOrString]): + The user agent to use for the fingerprint. Either a string or a list of strings. + fonts (Optional[List[str]]): + The fonts to load into Camoufox, in addition to the default fonts. + args (Optional[List[str]]): + The arguments to pass to the browser. + executable_path (Optional[str]): + The path to the Camoufox browser executable. + **launch_options (Dict[str, Any]): + Additional Firefox launch options. + """ + opt = get_launch_options( + config=config, + addons=addons, + fingerprint=fingerprint, + exclude_addons=exclude_addons, + screen=screen, + os=os, + user_agent=user_agent, + fonts=fonts, + args=args, + executable_path=executable_path, + ) + return await playwright.firefox.launch(**opt, **launch_options) diff --git a/pythonlib/camoufox/browserforge.yaml b/pythonlib/camoufox/browserforge.yaml new file mode 100644 index 0000000..ccf0a9a --- /dev/null +++ b/pythonlib/camoufox/browserforge.yaml @@ -0,0 +1,65 @@ +# Mappings of Browserforge fingerprints to Camoufox config properties. + +navigator: + # Disabled for now, since Browserforge tends to have outdated UAs + # userAgent: navigator.userAgent + # userAgentData not in Firefox + doNotTrack: navigator.doNotTrack + appCodeName: navigator.appCodeName + appName: navigator.appName + appVersion: navigator.appVersion + oscpu: navigator.oscpu + # webdriver is always True + language: navigator.language + languages: navigator.languages + platform: navigator.platform + # deviceMemory not in Firefox + hardwareConcurrency: navigator.hardwareConcurrency + product: navigator.product + productSub: navigator.productSub + # vendor is not necessary + # vendorSub is not necessary + maxTouchPoints: navigator.maxTouchPoints + extraProperties: + # Note: Changing pdfViewerEnabled is not recommended. This will be kept to True. + globalPrivacyControl: navigator.globalPrivacyControl + +screen: + # hasHDR is not implemented in Camoufox + availHeight: screen.availHeight + availWidth: screen.availWidth + availTop: screen.availTop + availLeft: screen.availLeft + height: screen.height + width: screen.width + colorDepth: screen.colorDepth + pixelDepth: screen.pixelDepth + # devicePixelRatio is not recommended. Any value other than 1.0 is suspicious. + pageXOffset: screen.pageXOffset + pageYOffset: screen.pageYOffset + # Disable viewport hijacking temporarily. + # outerHeight: window.outerHeight + # outerWidth: window.outerWidth + # innerHeight: window.innerHeight + # innerWidth: window.innerWidth + screenX: window.screenX + # These seem to not be generating properly in Browserforge: + # clientWidth: document.body.clientWidth + # clientHeight: document.body.clientHeight + +videoCard: + renderer: webGl:renderer + vendor: webGl:vendor + +headers: + # headers.User-Agent is redundant with navigator.userAgent + Accept-Language: headers.Accept-Language + Accept-Encoding: headers.Accept-Encoding + +battery: + charging: battery:charging + chargingTime: battery:chargingTime + dischargingTime: battery:dischargingTime + +# Unsupported: videoCodecs, audioCodecs, pluginsData, multimediaDevices +# Fonts are listed through the launcher. \ No newline at end of file diff --git a/pythonlib/camoufox/exceptions.py b/pythonlib/camoufox/exceptions.py new file mode 100644 index 0000000..3f817d7 --- /dev/null +++ b/pythonlib/camoufox/exceptions.py @@ -0,0 +1,54 @@ +class UnsupportedArchitecture(Exception): + """ + Raised when the architecture is not supported. + """ + + ... + + +class UnsupportedOS(Exception): + """ + Raised when the OS is not supported. + """ + + ... + + +class UnknownProperty(Exception): + """ + Raised when the property is unknown. + """ + + ... + + +class InvalidPropertyType(Exception): + """ + Raised when the property type is invalid. + """ + + ... + + +class InvalidAddonPath(FileNotFoundError): + """ + Raised when the addon path is invalid. + """ + + ... + + +class InvalidDebugPort(ValueError): + """ + Raised when the debug port is invalid. + """ + + ... + + +class MissingDebugPort(ValueError): + """ + Raised when the debug port is missing. + """ + + ... diff --git a/pythonlib/camoufox/fingerprints.py b/pythonlib/camoufox/fingerprints.py new file mode 100644 index 0000000..03b6414 --- /dev/null +++ b/pythonlib/camoufox/fingerprints.py @@ -0,0 +1,50 @@ +import os.path +from dataclasses import asdict + +from browserforge.fingerprints import Fingerprint, FingerprintGenerator +from yaml import CLoader, load + +# Load the browserforge.yaml file +with open(os.path.join(os.path.dirname(__file__), 'browserforge.yaml'), 'r') as f: + BROWSERFORGE_DATA = load(f, Loader=CLoader) + +FP_GENERATOR = FingerprintGenerator(browser='firefox', os=('linux', 'macos', 'windows')) + + +def _cast_to_properties(camoufox_data, cast_enum: dict, bf_dict: dict): + """ + Casts Browserforge fingerprints to Camoufox config properties. + """ + for key, data in bf_dict.items(): + # Ignore non-truthy values + if not data: + continue + # Get the associated Camoufox property + type_key = cast_enum.get(key) + if not type_key: + continue + # If the value is a dictionary, recursively recall + if isinstance(data, dict): + _cast_to_properties(camoufox_data, type_key, data) + else: + camoufox_data[type_key] = data + + +def from_browserforge(fingerprint: Fingerprint) -> dict: + camoufox_data = {} + _cast_to_properties(camoufox_data, cast_enum=BROWSERFORGE_DATA, bf_dict=asdict(fingerprint)) + return camoufox_data + + +def generate(**config) -> dict: + """ + Generates a Firefox fingerprint. + """ + data = FP_GENERATOR.generate(**config) + return from_browserforge(data) + + +if __name__ == "__main__": + from pprint import pprint + + pprint(generate()) diff --git a/pythonlib/camoufox/fonts.json b/pythonlib/camoufox/fonts.json new file mode 100644 index 0000000..dc92e4e --- /dev/null +++ b/pythonlib/camoufox/fonts.json @@ -0,0 +1,11 @@ +{ + "win": [ + "Arial", "Arial Black", "Bahnschrift", "Calibri", "Calibri Light", "Cambria", "Cambria Math", "Candara", "Candara Light", "Comic Sans MS", "Consolas", "Constantia", "Corbel", "Corbel Light", "Courier New", "Ebrima", "Franklin Gothic Medium", "Gabriola", "Gadugi", "Georgia", "HoloLens MDL2 Assets", "Impact", "Ink Free", "Javanese Text", "Leelawadee UI", "Leelawadee UI Semilight", "Lucida Console", "Lucida Sans Unicode", "MS Gothic", "MS PGothic", "MS UI Gothic", "MV Boli", "Malgun Gothic", "Malgun Gothic Semilight", "Marlett", "Microsoft Himalaya", "Microsoft JhengHei", "Microsoft JhengHei Light", "Microsoft JhengHei UI", "Microsoft JhengHei UI Light", "Microsoft New Tai Lue", "Microsoft PhagsPa", "Microsoft Sans Serif", "Microsoft Tai Le", "Microsoft YaHei", "Microsoft YaHei Light", "Microsoft YaHei UI", "Microsoft YaHei UI Light", "Microsoft Yi Baiti", "MingLiU-ExtB", "MingLiU_HKSCS-ExtB", "Mongolian Baiti", "Myanmar Text", "NSimSun", "Nirmala UI", "Nirmala UI Semilight", "PMingLiU-ExtB", "Palatino Linotype", "Segoe Fluent Icons", "Segoe MDL2 Assets", "Segoe Print", "Segoe Script", "Segoe UI", "Segoe UI Black", "Segoe UI Emoji", "Segoe UI Historic", "Segoe UI Light", "Segoe UI Semibold", "Segoe UI Semilight", "Segoe UI Symbol", "Segoe UI Variable", "SimSun", "SimSun-ExtB", "Sitka", "Sitka Text", "Sylfaen", "Symbol", "Tahoma", "Times New Roman", "Trebuchet MS", "Twemoji Mozilla", "Verdana", "Webdings", "Wingdings", "Yu Gothic", "Yu Gothic Light", "Yu Gothic Medium", "Yu Gothic UI", "Yu Gothic UI Light", "Yu Gothic UI Semibold", "Yu Gothic UI Semilight", "\u5b8b\u4f53", "\u5fae\u8edf\u6b63\u9ed1\u9ad4", "\u5fae\u8edf\u6b63\u9ed1\u9ad4 Light", "\u5fae\u8f6f\u96c5\u9ed1", "\u5fae\u8f6f\u96c5\u9ed1 Light", "\u65b0\u5b8b\u4f53", "\u65b0\u7d30\u660e\u9ad4-ExtB", "\u6e38\u30b4\u30b7\u30c3\u30af", "\u6e38\u30b4\u30b7\u30c3\u30af Light", "\u6e38\u30b4\u30b7\u30c3\u30af Medium", "\u7d30\u660e\u9ad4-ExtB", "\u7d30\u660e\u9ad4_HKSCS-ExtB", "\ub9d1\uc740 \uace0\ub515", "\ub9d1\uc740 \uace0\ub515 Semilight", "\uff2d\uff33 \u30b4\u30b7\u30c3\u30af", "\uff2d\uff33 \uff30\u30b4\u30b7\u30c3\u30af" + ], + "mac": [ + ".Al Bayan PUA", ".Al Nile PUA", ".Al Tarikh PUA", ".Apple Color Emoji UI", ".Apple SD Gothic NeoI", ".Aqua Kana", ".Aqua Kana Bold", ".Aqua \u304b\u306a", ".Aqua \u304b\u306a \u30dc\u30fc\u30eb\u30c9", ".Arial Hebrew Desk Interface", ".Baghdad PUA", ".Beirut PUA", ".Damascus PUA", ".DecoType Naskh PUA", ".Diwan Kufi PUA", ".Farah PUA", ".Geeza Pro Interface", ".Geeza Pro PUA", ".Helvetica LT MM", ".Hiragino Kaku Gothic Interface", ".Hiragino Sans GB Interface", ".Keyboard", ".KufiStandardGK PUA", ".LastResort", ".Lucida Grande UI", ".Muna PUA", ".Nadeem PUA", ".New York", ".Noto Nastaliq Urdu UI", ".PingFang HK", ".PingFang SC", ".PingFang TC", ".SF Arabic", ".SF Arabic Rounded", ".SF Compact", ".SF Compact Rounded", ".SF NS", ".SF NS Mono", ".SF NS Rounded", ".Sana PUA", ".Savoye LET CC.", ".ThonburiUI", ".ThonburiUIWatch", ".\u82f9\u65b9-\u6e2f", ".\u82f9\u65b9-\u7b80", ".\u82f9\u65b9-\u7e41", ".\u860b\u65b9-\u6e2f", ".\u860b\u65b9-\u7c21", ".\u860b\u65b9-\u7e41", "Academy Engraved LET", "Al Bayan", "Al Nile", "Al Tarikh", "American Typewriter", "Andale Mono", "Apple Braille", "Apple Chancery", "Apple Color Emoji", "Apple SD Gothic Neo", "Apple SD \uc0b0\ub3cc\uace0\ub515 Neo", "Apple Symbols", "AppleGothic", "AppleMyungjo", "Arial", "Arial Black", "Arial Hebrew", "Arial Hebrew Scholar", "Arial Narrow", "Arial Rounded MT Bold", "Arial Unicode MS", "Athelas", "Avenir", "Avenir Black", "Avenir Black Oblique", "Avenir Book", "Avenir Heavy", "Avenir Light", "Avenir Medium", "Avenir Next", "Avenir Next Condensed", "Avenir Next Condensed Demi Bold", "Avenir Next Condensed Heavy", "Avenir Next Condensed Medium", "Avenir Next Condensed Ultra Light", "Avenir Next Demi Bold", "Avenir Next Heavy", "Avenir Next Medium", "Avenir Next Ultra Light", "Ayuthaya", "Baghdad", "Bangla MN", "Bangla Sangam MN", "Baskerville", "Beirut", "Big Caslon", "Bodoni 72", "Bodoni 72 Oldstyle", "Bodoni 72 Smallcaps", "Bodoni Ornaments", "Bradley Hand", "Brush Script MT", "Chalkboard", "Chalkboard SE", "Chalkduster", "Charter", "Charter Black", "Cochin", "Comic Sans MS", "Copperplate", "Corsiva Hebrew", "Courier", "Courier New", "Czcionka systemowa", "DIN Alternate", "DIN Condensed", "Damascus", "DecoType Naskh", "Devanagari MT", "Devanagari Sangam MN", "Didot", "Diwan Kufi", "Diwan Thuluth", "Euphemia UCAS", "Farah", "Farisi", "Font Sistem", "Font de sistem", "Font di sistema", "Font sustava", "Fonte do Sistema", "Futura", "GB18030 Bitmap", "Galvji", "Geeza Pro", "Geneva", "Georgia", "Gill Sans", "Grantha Sangam MN", "Gujarati MT", "Gujarati Sangam MN", "Gurmukhi MN", "Gurmukhi MT", "Gurmukhi Sangam MN", "Heiti SC", "Heiti TC", "Heiti-\uac04\uccb4", "Heiti-\ubc88\uccb4", "Helvetica", "Helvetica Neue", "Herculanum", "Hiragino Kaku Gothic Pro", "Hiragino Kaku Gothic Pro W3", "Hiragino Kaku Gothic Pro W6", "Hiragino Kaku Gothic ProN", "Hiragino Kaku Gothic ProN W3", "Hiragino Kaku Gothic ProN W6", "Hiragino Kaku Gothic Std", "Hiragino Kaku Gothic Std W8", "Hiragino Kaku Gothic StdN", "Hiragino Kaku Gothic StdN W8", "Hiragino Maru Gothic Pro", "Hiragino Maru Gothic Pro W4", "Hiragino Maru Gothic ProN", "Hiragino Maru Gothic ProN W4", "Hiragino Mincho Pro", "Hiragino Mincho Pro W3", "Hiragino Mincho Pro W6", "Hiragino Mincho ProN", "Hiragino Mincho ProN W3", "Hiragino Mincho ProN W6", "Hiragino Sans", "Hiragino Sans GB", "Hiragino Sans GB W3", "Hiragino Sans GB W6", "Hiragino Sans W0", "Hiragino Sans W1", "Hiragino Sans W2", "Hiragino Sans W3", "Hiragino Sans W4", "Hiragino Sans W5", "Hiragino Sans W6", "Hiragino Sans W7", "Hiragino Sans W8", "Hiragino Sans W9", "Hoefler Text", "Hoefler Text Ornaments", "ITF Devanagari", "ITF Devanagari Marathi", "Impact", "InaiMathi", "Iowan Old Style", "Iowan Old Style Black", "J\u00e4rjestelm\u00e4fontti", "Kailasa", "Kannada MN", "Kannada Sangam MN", "Kefa", "Khmer MN", "Khmer Sangam MN", "Kohinoor Bangla", "Kohinoor Devanagari", "Kohinoor Gujarati", "Kohinoor Telugu", "Kokonor", "Krungthep", "KufiStandardGK", "Lao MN", "Lao Sangam MN", "Lucida Grande", "Luminari", "Malayalam MN", "Malayalam Sangam MN", "Marion", "Marker Felt", "Menlo", "Microsoft Sans Serif", "Mishafi", "Mishafi Gold", "Monaco", "Mshtakan", "Mukta Mahee", "MuktaMahee Bold", "MuktaMahee ExtraBold", "MuktaMahee ExtraLight", "MuktaMahee Light", "MuktaMahee Medium", "MuktaMahee Regular", "MuktaMahee SemiBold", "Muna", "Myanmar MN", "Myanmar Sangam MN", "Nadeem", "New Peninim MT", "Noteworthy", "Noto Nastaliq Urdu", "Noto Sans Adlam", "Noto Sans Armenian", "Noto Sans Armenian Blk", "Noto Sans Armenian ExtBd", "Noto Sans Armenian ExtLt", "Noto Sans Armenian Light", "Noto Sans Armenian Med", "Noto Sans Armenian SemBd", "Noto Sans Armenian Thin", "Noto Sans Avestan", "Noto Sans Bamum", "Noto Sans Bassa Vah", "Noto Sans Batak", "Noto Sans Bhaiksuki", "Noto Sans Brahmi", "Noto Sans Buginese", "Noto Sans Buhid", "Noto Sans CanAborig", "Noto Sans Canadian Aboriginal", "Noto Sans Carian", "Noto Sans CaucAlban", "Noto Sans Caucasian Albanian", "Noto Sans Chakma", "Noto Sans Cham", "Noto Sans Coptic", "Noto Sans Cuneiform", "Noto Sans Cypriot", "Noto Sans Duployan", "Noto Sans EgyptHiero", "Noto Sans Egyptian Hieroglyphs", "Noto Sans Elbasan", "Noto Sans Glagolitic", "Noto Sans Gothic", "Noto Sans Gunjala Gondi", "Noto Sans Hanifi Rohingya", "Noto Sans HanifiRohg", "Noto Sans Hanunoo", "Noto Sans Hatran", "Noto Sans ImpAramaic", "Noto Sans Imperial Aramaic", "Noto Sans InsPahlavi", "Noto Sans InsParthi", "Noto Sans Inscriptional Pahlavi", "Noto Sans Inscriptional Parthian", "Noto Sans Javanese", "Noto Sans Kaithi", "Noto Sans Kannada", "Noto Sans Kannada Black", "Noto Sans Kannada ExtraBold", "Noto Sans Kannada ExtraLight", "Noto Sans Kannada Light", "Noto Sans Kannada Medium", "Noto Sans Kannada SemiBold", "Noto Sans Kannada Thin", "Noto Sans Kayah Li", "Noto Sans Kharoshthi", "Noto Sans Khojki", "Noto Sans Khudawadi", "Noto Sans Lepcha", "Noto Sans Limbu", "Noto Sans Linear A", "Noto Sans Linear B", "Noto Sans Lisu", "Noto Sans Lycian", "Noto Sans Lydian", "Noto Sans Mahajani", "Noto Sans Mandaic", "Noto Sans Manichaean", "Noto Sans Marchen", "Noto Sans Masaram Gondi", "Noto Sans Meetei Mayek", "Noto Sans Mende Kikakui", "Noto Sans Meroitic", "Noto Sans Miao", "Noto Sans Modi", "Noto Sans Mongolian", "Noto Sans Mro", "Noto Sans Multani", "Noto Sans Myanmar", "Noto Sans Myanmar Blk", "Noto Sans Myanmar ExtBd", "Noto Sans Myanmar ExtLt", "Noto Sans Myanmar Light", "Noto Sans Myanmar Med", "Noto Sans Myanmar SemBd", "Noto Sans Myanmar Thin", "Noto Sans NKo", "Noto Sans Nabataean", "Noto Sans New Tai Lue", "Noto Sans Newa", "Noto Sans Ol Chiki", "Noto Sans Old Hungarian", "Noto Sans Old Italic", "Noto Sans Old North Arabian", "Noto Sans Old Permic", "Noto Sans Old Persian", "Noto Sans Old South Arabian", "Noto Sans Old Turkic", "Noto Sans OldHung", "Noto Sans OldNorArab", "Noto Sans OldSouArab", "Noto Sans Oriya", "Noto Sans Osage", "Noto Sans Osmanya", "Noto Sans Pahawh Hmong", "Noto Sans Palmyrene", "Noto Sans Pau Cin Hau", "Noto Sans PhagsPa", "Noto Sans Phoenician", "Noto Sans PsaPahlavi", "Noto Sans Psalter Pahlavi", "Noto Sans Rejang", "Noto Sans Samaritan", "Noto Sans Saurashtra", "Noto Sans Sharada", "Noto Sans Siddham", "Noto Sans Sora Sompeng", "Noto Sans SoraSomp", "Noto Sans Sundanese", "Noto Sans Syloti Nagri", "Noto Sans Syriac", "Noto Sans Tagalog", "Noto Sans Tagbanwa", "Noto Sans Tai Le", "Noto Sans Tai Tham", "Noto Sans Tai Viet", "Noto Sans Takri", "Noto Sans Thaana", "Noto Sans Tifinagh", "Noto Sans Tirhuta", "Noto Sans Ugaritic", "Noto Sans Vai", "Noto Sans Wancho", "Noto Sans Warang Citi", "Noto Sans Yi", "Noto Sans Zawgyi", "Noto Sans Zawgyi Blk", "Noto Sans Zawgyi ExtBd", "Noto Sans Zawgyi ExtLt", "Noto Sans Zawgyi Light", "Noto Sans Zawgyi Med", "Noto Sans Zawgyi SemBd", "Noto Sans Zawgyi Thin", "Noto Serif Ahom", "Noto Serif Balinese", "Noto Serif Hmong Nyiakeng", "Noto Serif Myanmar", "Noto Serif Myanmar Blk", "Noto Serif Myanmar ExtBd", "Noto Serif Myanmar ExtLt", "Noto Serif Myanmar Light", "Noto Serif Myanmar Med", "Noto Serif Myanmar SemBd", "Noto Serif Myanmar Thin", "Noto Serif Yezidi", "Optima", "Oriya MN", "Oriya Sangam MN", "PT Mono", "PT Sans", "PT Sans Caption", "PT Sans Narrow", "PT Serif", "PT Serif Caption", "Palatino", "Papyrus", "Party LET", "Phosphate", "Ph\u00f4ng ch\u1eef H\u1ec7 th\u1ed1ng", "PingFang HK", "PingFang SC", "PingFang TC", "Plantagenet Cherokee", "Police syst\u00e8me", "Raanana", "Rendszerbet\u0171t\u00edpus", "Rockwell", "STIX Two Math", "STIX Two Text", "STIXGeneral", "STIXIntegralsD", "STIXIntegralsSm", "STIXIntegralsUp", "STIXIntegralsUpD", "STIXIntegralsUpSm", "STIXNonUnicode", "STIXSizeFiveSym", "STIXSizeFourSym", "STIXSizeOneSym", "STIXSizeThreeSym", "STIXSizeTwoSym", "STIXVariants", "STSong", "Sana", "Sathu", "Savoye LET", "Seravek", "Seravek ExtraLight", "Seravek Light", "Seravek Medium", "Shree Devanagari 714", "SignPainter", "SignPainter-HouseScript", "Silom", "Sinhala MN", "Sinhala Sangam MN", "Sistem Fontu", "Skia", "Snell Roundhand", "Songti SC", "Songti TC", "Sukhumvit Set", "Superclarendon", "Symbol", "Systeemlettertype", "System Font", "Systemschrift", "Systemskrift", "Systemtypsnitt", "Syst\u00e9mov\u00e9 p\u00edsmo", "Tahoma", "Tamil MN", "Tamil Sangam MN", "Telugu MN", "Telugu Sangam MN", "Thonburi", "Times", "Times New Roman", "Tipo de letra del sistema", "Tipo de letra do sistema", "Tipus de lletra del sistema", "Trattatello", "Trebuchet MS", "Verdana", "Waseem", "Webdings", "Wingdings", "Wingdings 2", "Wingdings 3", "Zapf Dingbats", "Zapfino", "\u0393\u03c1\u03b1\u03bc\u03bc\u03b1\u03c4\u03bf\u03c3\u03b5\u03b9\u03c1\u03ac \u03c3\u03c5\u03c3\u03c4\u03ae\u03bc\u03b1\u03c4\u03bf\u03c2", "\u0421\u0438\u0441\u0442\u0435\u043c\u043d\u0438\u0439 \u0448\u0440\u0438\u0444\u0442", "\u0421\u0438\u0441\u0442\u0435\u043c\u043d\u044b\u0439 \u0448\u0440\u0438\u0444\u0442", "\u05d2\u05d5\u05e4\u05df \u05de\u05e2\u05e8\u05db\u05ea", "\u0627\u0644\u0628\u064a\u0627\u0646", "\u0627\u0644\u062a\u0627\u0631\u064a\u062e", "\u0627\u0644\u0646\u064a\u0644", "\u0628\u063a\u062f\u0627\u062f", "\u0628\u064a\u0631\u0648\u062a", "\u062c\u064a\u0632\u0629", "\u062e\u0637 \u0627\u0644\u0646\u0638\u0627\u0645", "\u062f\u0645\u0634\u0642", "\u062f\u064a\u0648\u0627\u0646 \u062b\u0644\u062b", "\u062f\u064a\u0648\u0627\u0646 \u0643\u0648\u0641\u064a", "\u0635\u0646\u0639\u0627\u0621", "\u0641\u0627\u0631\u0633\u064a", "\u0641\u0631\u062d", "\u0643\u0648\u0641\u064a", "\u0645\u0646\u0649", "\u0645\u0650\u0635\u062d\u0641\u064a", "\u0645\u0650\u0635\u062d\u0641\u064a \u0630\u0647\u0628\u064a", "\u0646\u062f\u064a\u0645", "\u0646\u0633\u062e", "\u0648\u0633\u064a\u0645", "\u0906\u0908\u0970\u091f\u0940\u0970\u090f\u092b\u093c\u0970 \u0926\u0947\u0935\u0928\u093e\u0917\u0930\u0940", "\u0906\u0908\u0970\u091f\u0940\u0970\u090f\u092b\u093c\u0970 \u0926\u0947\u0935\u0928\u093e\u0917\u0930\u0940 \u092e\u0930\u093e\u0920\u0940", "\u0915\u094b\u0939\u093f\u0928\u0942\u0930 \u0926\u0947\u0935\u0928\u093e\u0917\u0930\u0940", "\u0926\u0947\u0935\u0928\u093e\u0917\u0930\u0940 \u090f\u092e\u0970\u091f\u0940\u0970", "\u0926\u0947\u0935\u0928\u093e\u0917\u0930\u0940 \u0938\u0902\u0917\u092e \u090f\u092e\u0970\u090f\u0928\u0970", "\u0936\u094d\u0930\u0940 \u0926\u0947\u0935\u0928\u093e\u0917\u0930\u0940 \u096d\u0967\u096a", "\u0e41\u0e1a\u0e1a\u0e2d\u0e31\u0e01\u0e29\u0e23\u0e23\u0e30\u0e1a\u0e1a", "\u2e41\u7175\u6120\u82a9\u82c8", "\u30b7\u30b9\u30c6\u30e0\u30d5\u30a9\u30f3\u30c8", "\u30d2\u30e9\u30ae\u30ce\u4e38\u30b4 Pro", "\u30d2\u30e9\u30ae\u30ce\u4e38\u30b4 Pro W4", "\u30d2\u30e9\u30ae\u30ce\u4e38\u30b4 ProN", "\u30d2\u30e9\u30ae\u30ce\u4e38\u30b4 ProN W4", "\u30d2\u30e9\u30ae\u30ce\u660e\u671d Pro", "\u30d2\u30e9\u30ae\u30ce\u660e\u671d Pro W3", "\u30d2\u30e9\u30ae\u30ce\u660e\u671d Pro W6", "\u30d2\u30e9\u30ae\u30ce\u660e\u671d ProN", "\u30d2\u30e9\u30ae\u30ce\u660e\u671d ProN W3", "\u30d2\u30e9\u30ae\u30ce\u660e\u671d ProN W6", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 Pro", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 Pro W3", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 Pro W6", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 ProN", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 ProN W3", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 ProN W6", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 Std", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 Std W8", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 StdN", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 StdN W8", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 \u7c21\u4f53\u4e2d\u6587", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 \u7c21\u4f53\u4e2d\u6587 W3", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4 \u7c21\u4f53\u4e2d\u6587 W6", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4\u30b7\u30c3\u30af", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4\u30b7\u30c3\u30af W0", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4\u30b7\u30c3\u30af W1", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4\u30b7\u30c3\u30af W2", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4\u30b7\u30c3\u30af W3", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4\u30b7\u30c3\u30af W4", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4\u30b7\u30c3\u30af W5", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4\u30b7\u30c3\u30af W6", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4\u30b7\u30c3\u30af W7", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4\u30b7\u30c3\u30af W8", "\u30d2\u30e9\u30ae\u30ce\u89d2\u30b4\u30b7\u30c3\u30af W9", "\u51ac\u9752\u9ed1\u4f53\u7b80\u4f53\u4e2d\u6587", "\u51ac\u9752\u9ed1\u4f53\u7b80\u4f53\u4e2d\u6587 W3", "\u51ac\u9752\u9ed1\u4f53\u7b80\u4f53\u4e2d\u6587 W6", "\u51ac\u9752\u9ed1\u9ad4\u7c21\u9ad4\u4e2d\u6587", "\u51ac\u9752\u9ed1\u9ad4\u7c21\u9ad4\u4e2d\u6587 W3", "\u51ac\u9752\u9ed1\u9ad4\u7c21\u9ad4\u4e2d\u6587 W6", "\u5b8b\u4f53-\u7b80", "\u5b8b\u4f53-\u7e41", "\u5b8b\u9ad4-\u7c21", "\u5b8b\u9ad4-\u7e41", "\u7cfb\u7d71\u5b57\u9ad4", "\u7cfb\u7edf\u5b57\u4f53", "\u82f9\u65b9-\u6e2f", "\u82f9\u65b9-\u7b80", "\u82f9\u65b9-\u7e41", "\u8371\u8389\u834d\u836d\u8a70\u8353\u2050\u726f", "\u8371\u8389\u834d\u836d\u8a70\u8353\u2053\u7464", "\u8371\u8389\u834d\u836d\u8a70\u8353\u8356\u8362\u834e", "\u8371\u8389\u834d\u836d\u8adb\u8353\u2050\u726f", "\u8371\u8389\u834d\u836d\u96be\u92a9\u2050\u726f", "\u860b\u65b9-\u6e2f", "\u860b\u65b9-\u7c21", "\u860b\u65b9-\u7e41", "\u9ed1\u4f53-\u7b80", "\u9ed1\u4f53-\u7e41", "\u9ed1\u9ad4-\u7c21", "\u9ed1\u9ad4-\u7e41", "\u9ed2\u4f53-\u7c21", "\u9ed2\u4f53-\u7e41", "\uc2dc\uc2a4\ud15c \uc11c\uccb4" + ], + "lin": [ + "Arimo", "Cousine", "Noto Naskh Arabic", "Noto Sans Adlam", "Noto Sans Armenian", "Noto Sans Balinese", "Noto Sans Bamum", "Noto Sans Bassa Vah", "Noto Sans Batak", "Noto Sans Bengali", "Noto Sans Buginese", "Noto Sans Buhid", "Noto Sans Canadian Aboriginal", "Noto Sans Chakma", "Noto Sans Cham", "Noto Sans Cherokee", "Noto Sans Coptic", "Noto Sans Deseret", "Noto Sans Devanagari", "Noto Sans Elbasan", "Noto Sans Ethiopic", "Noto Sans Georgian", "Noto Sans Grantha", "Noto Sans Gujarati", "Noto Sans Gunjala Gondi", "Noto Sans Gurmukhi", "Noto Sans Hanifi Rohingya", "Noto Sans Hanunoo", "Noto Sans Hebrew", "Noto Sans JP", "Noto Sans Javanese", "Noto Sans KR", "Noto Sans Kannada", "Noto Sans Kayah Li", "Noto Sans Khmer", "Noto Sans Khojki", "Noto Sans Khudawadi", "Noto Sans Lao", "Noto Sans Lepcha", "Noto Sans Limbu", "Noto Sans Lisu", "Noto Sans Mahajani", "Noto Sans Malayalam", "Noto Sans Mandaic", "Noto Sans Masaram Gondi", "Noto Sans Medefaidrin", "Noto Sans Meetei Mayek", "Noto Sans Mende Kikakui", "Noto Sans Miao", "Noto Sans Modi", "Noto Sans Mongolian", "Noto Sans Mro", "Noto Sans Multani", "Noto Sans Myanmar", "Noto Sans NKo", "Noto Sans New Tai Lue", "Noto Sans Newa", "Noto Sans Ol Chiki", "Noto Sans Oriya", "Noto Sans Osage", "Noto Sans Osmanya", "Noto Sans Pahawh Hmong", "Noto Sans Pau Cin Hau", "Noto Sans Rejang", "Noto Sans Runic", "Noto Sans SC", "Noto Sans Samaritan", "Noto Sans Saurashtra", "Noto Sans Sharada", "Noto Sans Shavian", "Noto Sans Sinhala", "Noto Sans Sora Sompeng", "Noto Sans Soyombo", "Noto Sans Sundanese", "Noto Sans Syloti Nagri", "Noto Sans Symbols", "Noto Sans Symbols 2", "Noto Sans Syriac", "Noto Sans TC", "Noto Sans Tagalog", "Noto Sans Tagbanwa", "Noto Sans Tai Le", "Noto Sans Tai Tham", "Noto Sans Tai Viet", "Noto Sans Takri", "Noto Sans Tamil", "Noto Sans Telugu", "Noto Sans Thaana", "Noto Sans Thai", "Noto Sans Tifinagh", "Noto Sans Tifinagh APT", "Noto Sans Tifinagh Adrar", "Noto Sans Tifinagh Agraw Imazighen", "Noto Sans Tifinagh Ahaggar", "Noto Sans Tifinagh Air", "Noto Sans Tifinagh Azawagh", "Noto Sans Tifinagh Ghat", "Noto Sans Tifinagh Hawad", "Noto Sans Tifinagh Rhissa Ixa", "Noto Sans Tifinagh SIL", "Noto Sans Tifinagh Tawellemmet", "Noto Sans Tirhuta", "Noto Sans Vai", "Noto Sans Wancho", "Noto Sans Warang Citi", "Noto Sans Yi", "Noto Sans Zanabazar Square", "Noto Serif Armenian", "Noto Serif Balinese", "Noto Serif Bengali", "Noto Serif Devanagari", "Noto Serif Dogra", "Noto Serif Ethiopic", "Noto Serif Georgian", "Noto Serif Grantha", "Noto Serif Gujarati", "Noto Serif Gurmukhi", "Noto Serif Hebrew", "Noto Serif Kannada", "Noto Serif Khmer", "Noto Serif Khojki", "Noto Serif Lao", "Noto Serif Malayalam", "Noto Serif Myanmar", "Noto Serif NP Hmong", "Noto Serif Sinhala", "Noto Serif Tamil", "Noto Serif Telugu", "Noto Serif Thai", "Noto Serif Tibetan", "Noto Serif Yezidi", "STIX Two Math", "Tinos", "Twemoji Mozilla" + ] +} diff --git a/pythonlib/camoufox/pkgman.py b/pythonlib/camoufox/pkgman.py new file mode 100644 index 0000000..b79e2ec --- /dev/null +++ b/pythonlib/camoufox/pkgman.py @@ -0,0 +1,339 @@ +import os +import platform +import re +import shutil +import sys +import tempfile +from io import BytesIO +from pathlib import Path +from typing import List, Optional, Union +from zipfile import ZipFile + +import click +import orjson +import requests +from platformdirs import user_cache_dir +from tqdm import tqdm +from typing_extensions import TypeAlias + +from .exceptions import UnsupportedArchitecture, UnsupportedOS + +DownloadBuffer: TypeAlias = Union[BytesIO, tempfile._TemporaryFileWrapper] + +# Map machine architecture to Camoufox binary name +ARCH_MAP: dict[str, str] = { + 'amd64': 'x86_64', + 'x86_64': 'x86_64', + 'x86': 'x86_64', + 'i686': 'i686', + 'i386': 'i686', + 'arm64': 'arm64', + 'aarch64': 'arm64', + 'armv5l': 'arm64', + 'armv6l': 'arm64', + 'armv7l': 'arm64', +} +OS_MAP: dict[str, str] = {'darwin': 'mac', 'linux': 'lin', 'win32': 'win'} + +if sys.platform not in OS_MAP: + raise UnsupportedOS(f"OS {sys.platform} is not supported") + +OS_NAME: str = OS_MAP[sys.platform] + +INSTALL_DIR: Path = Path(user_cache_dir("camoufox")) + +# The supported architectures for each OS +OS_ARCH_MATRIX: dict[str, List[str]] = { + 'mac': ['x86_64', 'arm64'], + 'win': ['x86_64', 'i686'], + 'lin': ['x86_64', 'arm64', 'i686'], +} + + +def rprint(*a, **k): + click.secho(*a, **k, bold=True) + + +class CamoufoxFetcher: + def __init__(self) -> None: + self.arch = self.get_platform_arch() + self._version: str | None = None + self._release: str | None = None + self.pattern: re.Pattern = re.compile(rf'camoufox-(.+)-(.+)-{OS_NAME}\.{self.arch}\.zip') + + self.fetch_latest() + + @staticmethod + def get_platform_arch() -> str: + """ + Get the current platform and architecture information. + + Returns: + + + Raises: + UnsupportedOS: If the current OS is not supported + UnsupportedArchitecture: If the current architecture is not supported + """ + + # Check if the architecture is supported for the OS + plat_arch = platform.machine().lower() + if plat_arch not in ARCH_MAP: + raise UnsupportedArchitecture(f"Architecture {plat_arch} is not supported") + + arch = ARCH_MAP[plat_arch] + + # Check if the architecture is supported for the OS + if arch not in OS_ARCH_MATRIX[OS_NAME]: + raise UnsupportedArchitecture(f"Architecture {arch} is not supported for {OS_NAME}") + + return arch + + def fetch_latest(self) -> None: + """ + Fetch the URL of the latest camoufox release for the current platform. + Sets the version, release, and url properties. + + Raises: + requests.RequestException: If there's an error fetching release data + ValueError: If no matching release is found for the current platform + """ + api_url = "https://api.github.com/repos/daijro/camoufox/releases/latest" + response = requests.get(api_url, timeout=20) + response.raise_for_status() + + release_data = response.json() + assets = release_data['assets'] + + for asset in assets: + if match := self.pattern.match(asset['name']): + # Set the version and release + self._version = match.group(1) + self._release = match.group(2) + # Return the download URL + self._url = asset['browser_download_url'] + return + + raise ValueError(f"No matching release found for {OS_NAME}-{self.arch}") + + @staticmethod + def download_file(file: DownloadBuffer, url: str) -> DownloadBuffer: + """ + Download a file from the given URL and return it as BytesIO. + + Args: + url (str): The URL to download the file from + + Returns: + BytesIO: The downloaded file content as a BytesIO object + """ + rprint(f'Downloading package: {url}') + return webdl(url, buffer=file) + + def extract_zip(self, zip_file: DownloadBuffer) -> None: + """ + Extract the contents of a zip file to the installation directory. + + Args: + zip_file (BytesIO): The zip file content as a BytesIO object + """ + rprint(f'Extracting Camoufox: {INSTALL_DIR}') + unzip(zip_file, str(INSTALL_DIR)) + + @staticmethod + def cleanup() -> bool: + """ + Clean up the old installation. + """ + if INSTALL_DIR.exists(): + rprint(f'Cleaning up cache: {INSTALL_DIR}') + shutil.rmtree(INSTALL_DIR) + return True + return False + + def set_version(self) -> None: + """ + Set the version in the INSTALL_DIR/version.json file + """ + with open(INSTALL_DIR / 'version.json', 'wb') as f: + f.write(orjson.dumps({'version': self.version, 'release': self.release})) + + def install(self) -> None: + """ + Download and install the latest version of camoufox. + + Raises: + Exception: If any error occurs during the installation process + """ + # Clean up old installation + self.cleanup() + try: + # Install to directory + INSTALL_DIR.mkdir(parents=True, exist_ok=True) + + # Fetch the latest zip + with tempfile.NamedTemporaryFile() as temp_file: + self.download_file(temp_file, self.url) + self.extract_zip(temp_file) + self.set_version() + + # Run chmod -R 755 on INSTALL_DIR + if OS_NAME != 'win': + shutil.chmod(INSTALL_DIR, 0o755) + + # Clean up old installation + rprint('\nCamoufox successfully installed.', fg="yellow") + except Exception as e: + rprint(f"Error installing Camoufox: {str(e)}") + self.cleanup() + raise + + @property + def url(self) -> str: + """ + Url of the fetched latest version of camoufox. + + Returns: + str: The version of the installed camoufox + + Raises: + ValueError: If the version is not available (fetch_latest not ran) + """ + if self._url is None: + raise ValueError("Url is not available. Make sure to run fetch_latest first.") + return self._url + + @property + def version(self) -> str: + """ + Version of the fetched latest version of camoufox. + + Returns: + str: The version of the installed camoufox + + Raises: + ValueError: If the version is not available (fetch_latest not ran) + """ + if self._version is None: + raise ValueError("Version is not available. Make sure to run the fetch_latest first.") + return self._version + + @property + def release(self) -> str: + """ + Release of the fetched latest version of camoufox. + + Returns: + str: The release of the installed camoufox + + Raises: + ValueError: If the release information is not available (fetch_latest not ran) + """ + if self._release is None: + raise ValueError( + "Release information is not available. Make sure to run the installation first." + ) + return self._release + + @property + def verstr(self) -> str: + """ + Fetches the version and release in version-release format + + Returns: + str: The version of the installed camoufox + """ + return f"{self.version}-{self.release}" + + +def installed_verstr() -> str: + """ + Get the full version string of the installed camoufox. + """ + version_path = INSTALL_DIR / 'version.json' + if not os.path.exists(version_path): + raise FileNotFoundError(f"Version information not found at {version_path}") + + with open(version_path, 'rb') as f: + version_data = orjson.loads(f.read()) + return f"{version_data['version']}-{version_data['release']}" + + +def camoufox_path(download_if_missing: bool = True) -> Path: + """ + Full path to the camoufox folder. + """ + if not os.path.exists(INSTALL_DIR): + if not download_if_missing: + raise FileNotFoundError(f"Camoufox executable not found at {INSTALL_DIR}") + + installer = CamoufoxFetcher() + installer.install() + # Rerun and ensure it's installed + return camoufox_path() + + return INSTALL_DIR + + +def get_path(file: str) -> str: + """ + Get the path to the camoufox executable. + """ + if OS_NAME == 'mac': + return str(camoufox_path() / 'Camoufox.app' / 'Contents' / 'Resources' / file) + return str(camoufox_path() / file) + + +def webdl( + url: str, + desc: Optional[str] = None, + buffer: Optional[DownloadBuffer] = None, +) -> DownloadBuffer: + """ + Download a file from the given URL and return it as BytesIO. + + Args: + url (str): The URL to download the file from + buffer (Optional[BytesIO]): A BytesIO object to store the downloaded file + + Returns: + DownloadBuffer: The downloaded file content as a BytesIO object + + Raises: + requests.RequestException: If there's an error downloading the file + """ + response = requests.get(url, stream=True, timeout=20) + response.raise_for_status() + + total_size = int(response.headers.get('content-length', 0)) + block_size = 8192 + if buffer is None: + buffer = BytesIO() + + with tqdm(total=total_size, unit='iB', unit_scale=True, desc=desc) as progress_bar: + for data in response.iter_content(block_size): + size = buffer.write(data) + progress_bar.update(size) + + buffer.seek(0) + return buffer + + +def unzip( + zip_file: DownloadBuffer, + extract_path: str, + desc: Optional[str] = None, +) -> None: + """ + Extract the contents of a zip file to the installation directory. + + Args: + zip_file (BytesIO): The zip file content as a BytesIO object + + Raises: + zipfile.BadZipFile: If the zip file is invalid or corrupted + OSError: If there's an error creating directories or writing files + """ + with ZipFile(zip_file) as zf: + for member in tqdm(zf.infolist(), desc=desc): + zf.extract(member, extract_path) diff --git a/pythonlib/camoufox/sync_api.py b/pythonlib/camoufox/sync_api.py new file mode 100644 index 0000000..3130f45 --- /dev/null +++ b/pythonlib/camoufox/sync_api.py @@ -0,0 +1,88 @@ +from typing import Any, Dict, List, Optional + +from browserforge.fingerprints import Fingerprint, Screen +from playwright.sync_api import Browser, Playwright, PlaywrightContextManager + +from .addons import DefaultAddons +from .utils import ListOrString, get_launch_options + + +class Camoufox(PlaywrightContextManager): + """ + Wrapper around playwright.sync_api.PlaywrightContextManager that automatically + launches a browser and closes it when the context manager is exited. + """ + + def __init__(self, **launch_options): + super().__init__() + self.launch_options = launch_options + self.browser: Optional[Browser] = None + + def __enter__(self) -> Browser: + super().__enter__() + self.browser = NewBrowser(self._playwright, **self.launch_options) + return self.browser + + def __exit__(self, *args: Any): + if self.browser: + self.browser.close() + super().__exit__(*args) + + +def NewBrowser( + playwright: Playwright, + *, + config: Optional[Dict[str, Any]] = None, + addons: Optional[List[str]] = None, + fingerprint: Optional[Fingerprint] = None, + exclude_addons: Optional[List[DefaultAddons]] = None, + screen: Optional[Screen] = None, + os: Optional[ListOrString] = None, + user_agent: Optional[ListOrString] = None, + fonts: Optional[List[str]] = None, + args: Optional[List[str]] = None, + executable_path: Optional[str] = None, + **launch_options: Dict[str, Any] +) -> Browser: + """ + Launches a new browser instance for Camoufox. + + Parameters: + playwright (Playwright): + The playwright instance to use. + config (Optional[Dict[str, Any]]): + The configuration to use. + addons (Optional[List[str]]): + The addons to use. + fingerprint (Optional[Fingerprint]): + The fingerprint to use. + exclude_addons (Optional[List[DefaultAddons]]): + The default addons to exclude, passed as a list of camoufox.DefaultAddons enums. + screen (Optional[browserforge.fingerprints.Screen]): + The screen constraints to use. + os (Optional[ListOrString]): + The operating system to use for the fingerprint. Either a string or a list of strings. + user_agent (Optional[ListOrString]): + The user agent to use for the fingerprint. Either a string or a list of strings. + fonts (Optional[List[str]]): + The fonts to load into Camoufox, in addition to the default fonts. + args (Optional[List[str]]): + The arguments to pass to the browser. + executable_path (Optional[str]): + The path to the Camoufox browser executable. + **launch_options (Dict[str, Any]): + Additional Firefox launch options. + """ + opt = get_launch_options( + config=config, + addons=addons, + fingerprint=fingerprint, + exclude_addons=exclude_addons, + screen=screen, + os=os, + user_agent=user_agent, + fonts=fonts, + args=args, + executable_path=executable_path, + ) + return playwright.firefox.launch(**opt, **launch_options) diff --git a/pythonlib/camoufox/utils.py b/pythonlib/camoufox/utils.py new file mode 100644 index 0000000..567f4a9 --- /dev/null +++ b/pythonlib/camoufox/utils.py @@ -0,0 +1,221 @@ +# Set to environment variables +import os +import sys +from os import environ +from random import randrange +from typing import Any, Dict, List, Optional, Tuple, TypeAlias, Union + +import numpy as np +import orjson +from browserforge.fingerprints import Fingerprint, Screen +from ua_parser import user_agent_parser + +from .addons import ( + DefaultAddons, + confirm_paths, + get_debug_port, + threaded_try_load_addons, +) +from .exceptions import InvalidPropertyType, UnknownProperty +from .fingerprints import from_browserforge, generate +from .pkgman import OS_NAME, get_path +from .xpi_dl import add_default_addons + +LAUNCH_FILE = { + 'win': 'camoufox.exe', + 'lin': 'camoufox-bin', + 'mac': 'camoufox', +} + +ListOrString: TypeAlias = Union[Tuple[str, ...], List[str], str] + + +def get_env_vars( + config_map: Dict[str, str], user_agent_os: str +) -> Dict[str, Union[str, float, bool]]: + """ + Gets a dictionary of environment variables for Camoufox. + """ + env_vars = {} + try: + updated_config_data = orjson.dumps(config_map) + except orjson.JSONEncodeError as e: + print(f"Error updating config: {e}") + sys.exit(1) + + # Split the config into chunks + chunk_size = 2047 if OS_NAME == 'windows' else 32767 + config_str = updated_config_data.decode('utf-8') + + for i in range(0, len(config_str), chunk_size): + chunk = config_str[i : i + chunk_size] + env_name = f"CAMOU_CONFIG_{(i // chunk_size) + 1}" + try: + env_vars[env_name] = chunk + except Exception as e: + print(f"Error setting {env_name}: {e}") + sys.exit(1) + + if OS_NAME == 'linux': + fontconfig_path = get_path(os.path.join("fontconfig", user_agent_os)) + env_vars['FONTCONFIG_PATH'] = fontconfig_path + + return env_vars + + +def _load_properties() -> Dict[str, str]: + """ + Loads the properties.json file. + """ + prop_file = get_path("properties.json") + with open(prop_file, "rb") as f: + prop_dict = orjson.loads(f.read()) + + return {prop['property']: prop['type'] for prop in prop_dict} + + +def validate_config(config_map: Dict[str, str]) -> None: + """ + Validates the config map. + """ + property_types = _load_properties() + + for key, value in config_map.items(): + expected_type = property_types.get(key) + if not expected_type: + raise UnknownProperty(f"Unknown property {key} in config") + + if not validate_type(value, expected_type): + raise InvalidPropertyType( + f"Invalid type for property {key}. Expected {expected_type}, got {type(value).__name__}" + ) + + +def validate_type(value: Any, expected_type: str) -> bool: + """ + Validates the type of the value. + """ + if expected_type == "str": + return isinstance(value, str) + elif expected_type == "int": + return isinstance(value, int) or (isinstance(value, float) and value.is_integer()) + elif expected_type == "uint": + return ( + isinstance(value, int) or (isinstance(value, float) and value.is_integer()) + ) and value >= 0 + elif expected_type == "double": + return isinstance(value, (float, int)) + elif expected_type == "bool": + return isinstance(value, bool) + elif expected_type == "array": + return isinstance(value, list) + else: + return False + + +def get_target_os(config: Dict[str, Any]) -> str: + """ + Gets the OS from the config if the user agent is set, + otherwise returns the OS of the current system. + """ + if config.get("navigator.userAgent"): + return determine_ua_os(config["navigator.userAgent"]) + return OS_NAME + + +def determine_ua_os(user_agent: str) -> str: + """ + Determines the OS from the user agent string. + """ + parsed_ua = user_agent_parser.ParseOS(user_agent).get('family') + if not parsed_ua: + raise ValueError("Could not determine OS from user agent") + if parsed_ua.startswith("Mac"): + return "mac" + if parsed_ua.startswith("Windows"): + return "win" + return "lin" + + +def update_fonts(config: Dict[str, Any], target_os: str) -> None: + """ + Updates the fonts for the target OS. + """ + with open(os.path.join(os.path.dirname(__file__), "fonts.json"), "rb") as f: + fonts = orjson.loads(f.read())[target_os] + + # Merge with existing fonts + if 'fonts' in config: + config['fonts'] = np.unique(fonts + config['fonts']).tolist() + else: + config['fonts'] = fonts + + +def get_launch_options( + *, + config: Optional[Dict[str, Any]] = None, + addons: Optional[List[str]] = None, + fingerprint: Optional[Fingerprint] = None, + exclude_addons: Optional[List[DefaultAddons]] = None, + screen: Optional[Screen] = None, + os: Optional[ListOrString] = None, + user_agent: Optional[ListOrString] = None, + fonts: Optional[List[str]] = None, + args: Optional[List[str]] = None, + executable_path: Optional[str] = None, +) -> Dict[str, Any]: + """ + Builds the launch options for the Camoufox browser. + """ + # Validate the config + if config is None: + config = {} + + if addons is None: + addons = [] + if args is None: + args = [] + + # Add the default addons + add_default_addons(addons, exclude_addons) + + # Confirm all addon paths are valid + if addons: + confirm_paths(addons) + + # Generate new fingerprint + if fingerprint is None: + config = { + **generate( + screen=screen, + os=os, + user_agent=user_agent, + ), + **config, + } + else: + config = { + **from_browserforge(fingerprint), + **config, + } + + # Set a random window.history.length + config['window.history.length'] = randrange(1, 6) + + if fonts: + config['fonts'] = fonts + + validate_config(config) + + # Update fonts list + target_os = get_target_os(config) + update_fonts(config, target_os) + + # Launch + threaded_try_load_addons(get_debug_port(args), addons) + env_vars = {**get_env_vars(config, target_os), **environ} + return { + "executable_path": executable_path or get_path(LAUNCH_FILE[OS_NAME]), + "args": args, + "env": env_vars, + } diff --git a/pythonlib/camoufox/xpi_dl.py b/pythonlib/camoufox/xpi_dl.py new file mode 100644 index 0000000..d00d432 --- /dev/null +++ b/pythonlib/camoufox/xpi_dl.py @@ -0,0 +1,61 @@ +import os +from typing import List, Optional + +from .addons import DefaultAddons +from .pkgman import get_path, unzip, webdl + + +def add_default_addons( + addons_list: List[str], exclude_list: Optional[List[DefaultAddons]] = None +) -> None: + """ + Adds default addons, minus any specified in exclude_list, to addons_list + """ + # Build a dictionary from DefaultAddons, excluding keys found in exclude_list + if exclude_list is None: + exclude_list = [] + + addons = [addon for addon in DefaultAddons if addon not in exclude_list] + + maybe_download_addons(addons, addons_list) + + +def download_and_extract(url: str, extract_path: str) -> None: + """ + Downloads and extracts an addon from a given URL to a specified path + """ + # Create a temporary file to store the downloaded zip + buffer = webdl(url, desc="Downloading addon") + unzip(buffer, extract_path) + + +def get_addon_path(addon_name: str) -> str: + """ + Returns a path to the addon + """ + return get_path(os.path.join("addons", addon_name)) + + +def maybe_download_addons(addons: List[DefaultAddons], addons_list: List[str]) -> None: + """ + Downloads and extracts addons from a given dictionary to a specified list + Skips downloading if the addon is already downloaded + """ + for addon in addons: + # Get the addon path + addon_path = get_addon_path(addon.name) + + # Check if the addon is already extracted + if os.path.exists(addon_path): + # Add the existing addon path to addons_list + addons_list.append(addon_path) + continue + + # Addon doesn't exist, create directory and download + try: + os.makedirs(addon_path, exist_ok=True) + download_and_extract(addon.value, addon_path) + # Add the new addon directory path to addons_list + addons_list.append(addon_path) + except Exception as e: + print(f"Failed to download and extract {addon.name}: {e}") diff --git a/pythonlib/pyproject.toml b/pythonlib/pyproject.toml new file mode 100644 index 0000000..18d70d4 --- /dev/null +++ b/pythonlib/pyproject.toml @@ -0,0 +1,43 @@ +[build-system] +requires = ["poetry-core>=1.0.0"] +build-backend = "poetry.core.masonry.api" + +[tool.poetry] +name = "camoufox" +version = "1.0.0" +description = "Wrapper around Playwright to help launch Camoufox" +authors = ["daijro "] +license = "MIT" +repository = "https://github.com/daijro/camoufox" +readme = "README.md" +keywords = [ + "client", + "fingerprint", + "browser", + "scraping", + "injector", + "firefox", + "playwright", +] +classifiers = [ + "Topic :: Internet :: WWW/HTTP", + "Topic :: Internet :: WWW/HTTP :: Browsers", + "Topic :: Software Development :: Libraries :: Python Modules", +] + +[tool.poetry.dependencies] +python = "^3.8" +click = "*" +requests = "*" +orjson = "*" +browserforge = "*" +playwright = "*" +pyyaml = "*" +platformdirs = "*" +tqdm = "*" +numpy = "*" +ua_parser = "*" +typing_extensions = "*" + +[tool.poetry.scripts] +camoufox = "camoufox.__main__:cli"