Add dependency download and CLI functionality
- Implement dependency download and extraction for Zig packages - Create new CLI commands for downloading and converting ZON files - Add support for downloading dependencies from ZON files - Update project dependencies to include httpx and tqdm - Add WTFPL license file - Enhance README with more detailed usage instructions and project motivation
This commit is contained in:
13
LICENSE
Normal file
13
LICENSE
Normal file
@ -0,0 +1,13 @@
|
||||
DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
|
||||
Version 2, December 2004
|
||||
|
||||
Copyright (C) 2024 <crosstyan@outlook.com>
|
||||
|
||||
Everyone is permitted to copy and distribute verbatim or modified
|
||||
copies of this license document, and changing it is allowed as long
|
||||
as the name is changed.
|
||||
|
||||
DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
|
||||
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
|
||||
|
||||
0. You just DO WHAT THE FUCK YOU WANT TO.
|
||||
141
README.md
141
README.md
@ -1,17 +1,27 @@
|
||||
# zig-fetch-py
|
||||
|
||||
A Python tool to parse Zig Object Notation (ZON) files and convert them to JSON.
|
||||
A Python utility for working with Zig package manager files and Zig Object Notation (ZON).
|
||||
|
||||
## Features
|
||||
|
||||
- Parse ZON files into Python dictionaries
|
||||
- Convert ZON files to JSON
|
||||
- Download and extract dependencies from ZON files
|
||||
|
||||
## Installation
|
||||
|
||||
### Using uv (recommended)
|
||||
|
||||
[uv](https://github.com/astral-sh/uv) is a fast Python package installer and resolver. To install zig-fetch-py using uv:
|
||||
[uv](https://github.com/astral-sh/uv) is a fast Python package installer and resolver.
|
||||
|
||||
```bash
|
||||
# Install uv if you don't have it
|
||||
curl -sSf https://astral.sh/uv/install.sh | bash
|
||||
|
||||
# Clone the repository
|
||||
git clone https://github.com/yourusername/zig-fetch-py.git
|
||||
cd zig-fetch-py
|
||||
|
||||
# Create and activate a virtual environment
|
||||
uv venv
|
||||
source .venv/bin/activate # On Windows: .venv\Scripts\activate
|
||||
@ -26,7 +36,11 @@ uv pip install -e ".[dev]"
|
||||
### Using pip
|
||||
|
||||
```bash
|
||||
# Create and activate a virtual environment
|
||||
# Clone the repository
|
||||
git clone https://github.com/yourusername/zig-fetch-py.git
|
||||
cd zig-fetch-py
|
||||
|
||||
# Create a virtual environment
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate # On Windows: .venv\Scripts\activate
|
||||
|
||||
@ -39,84 +53,85 @@ pip install -e ".[dev]"
|
||||
|
||||
## Usage
|
||||
|
||||
### Command Line
|
||||
### Command Line Interface
|
||||
|
||||
The package provides a command-line interface with the following commands:
|
||||
|
||||
#### Download Dependencies
|
||||
|
||||
Download and extract dependencies from a ZON file:
|
||||
|
||||
```bash
|
||||
# Basic usage
|
||||
zon2json path/to/file.zon
|
||||
|
||||
# Output to a file
|
||||
zon2json path/to/file.zon -o output.json
|
||||
|
||||
# Pretty print the JSON
|
||||
zon2json path/to/file.zon -p
|
||||
|
||||
# Enable verbose logging
|
||||
zon2json path/to/file.zon -v
|
||||
zig-fetch download examples/test.zon
|
||||
```
|
||||
|
||||
This will download all dependencies specified in the ZON file to `~/.cache/zig/p` and extract them to directories named after their hash values.
|
||||
|
||||
#### Convert ZON to JSON
|
||||
|
||||
Convert a ZON file to JSON:
|
||||
|
||||
```bash
|
||||
zig-fetch convert examples/test.zon
|
||||
```
|
||||
|
||||
Or use the dedicated command:
|
||||
|
||||
```bash
|
||||
zon2json examples/test.zon
|
||||
```
|
||||
|
||||
Options:
|
||||
- `--indent N`, `-i N`: Set the indentation level for the JSON output (default: 2)
|
||||
- `--output PATH`, `-o PATH`: Output file (default: stdout)
|
||||
- `--empty-tuple-as-dict`: Parse empty tuples (`.{}`) as empty dictionaries (`{}`) instead of empty lists (`[]`)
|
||||
- `--verbose`, `-v`: Enable verbose logging
|
||||
|
||||
### Python API
|
||||
|
||||
You can also use the package as a Python library:
|
||||
|
||||
```python
|
||||
from zig_fetch_py.parser import parse_zon_file, zon_to_json
|
||||
from zig_fetch_py.downloader import process_dependencies
|
||||
|
||||
# Parse a ZON file
|
||||
result = parse_zon_file("path/to/file.zon")
|
||||
print(result) # Python dictionary
|
||||
zon_data = parse_zon_file("examples/test.zon")
|
||||
|
||||
# Convert ZON content to JSON
|
||||
zon_content = """.{
|
||||
.name = "test",
|
||||
.version = "1.0.0",
|
||||
}"""
|
||||
json_str = zon_to_json(zon_content, indent=4)
|
||||
print(json_str)
|
||||
# Convert ZON to JSON
|
||||
json_str = zon_to_json(zon_content, indent=2)
|
||||
|
||||
# Download dependencies
|
||||
dependencies = process_dependencies("examples/test.zon")
|
||||
```
|
||||
|
||||
## Development
|
||||
## ZON Parser Options
|
||||
|
||||
### Running Tests
|
||||
The ZON parser supports the following options:
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
pytest
|
||||
- `empty_tuple_as_dict`: If True, empty tuples (`.{}`) will be parsed as empty dictionaries (`{}`) instead of empty lists (`[]`)
|
||||
|
||||
# Run tests with coverage
|
||||
pytest --cov=zig_fetch_py
|
||||
## Trivia
|
||||
|
||||
# Generate coverage report
|
||||
pytest --cov=zig_fetch_py --cov-report=html
|
||||
```
|
||||
Cursor (powered by Claude 3.7) help me do almost all of the heavy lifting. I
|
||||
can't even write a proper parser by my own.
|
||||
|
||||
## ZON Format
|
||||
|
||||
Zig Object Notation (ZON) is a data format used by the Zig programming language. It's similar to JSON but with some differences in syntax:
|
||||
|
||||
- Objects (anonymous structs) are defined with `.{ .key = value, ... }`
|
||||
- Keys are prefixed with a dot: `.key = value`
|
||||
- Tuples are defined with `.{value1, value2, ...}` and are parsed as arrays in JSON
|
||||
- Special identifiers can be quoted with `@`: `.@"special-name" = value`
|
||||
- Comments use `//` syntax
|
||||
|
||||
Note: ZON doesn't have a dedicated array syntax like JSON's `[]`. Instead, tuples (`.{value1, value2, ...}`) serve a similar purpose and are converted to arrays in JSON.
|
||||
|
||||
Example:
|
||||
|
||||
```zon
|
||||
.{
|
||||
.name = "example",
|
||||
.version = "1.0.0",
|
||||
.dependencies = .{
|
||||
.lib1 = .{
|
||||
.url = "https://example.com/lib1.tar.gz",
|
||||
.hash = "abcdef123456",
|
||||
},
|
||||
},
|
||||
.tags = .{1, 2, 3}, // Tuple (parsed as array in JSON)
|
||||
.paths = .{""}, // Single-item tuple
|
||||
}
|
||||
```
|
||||
The motivation of it is this issue ([add http/socks5 proxy support for package manager](https://github.com/ziglang/zig/issues/15048)).
|
||||
Until proper proxy support is added to zig, I'll maintain this repo. (The Zon parser might be useful for other projects though)
|
||||
|
||||
## License
|
||||
|
||||
MIT
|
||||
```
|
||||
DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
|
||||
Version 2, December 2004
|
||||
|
||||
Copyright (C) 2024 <crosstyan@outlook.com>
|
||||
|
||||
Everyone is permitted to copy and distribute verbatim or modified
|
||||
copies of this license document, and changing it is allowed as long
|
||||
as the name is changed.
|
||||
|
||||
DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
|
||||
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
|
||||
|
||||
0. You just DO WHAT THE FUCK YOU WANT TO.
|
||||
@ -4,10 +4,17 @@ version = "0.1.0"
|
||||
description = "A tool to parse Zig Object Notation (ZON) files and convert them to JSON"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
dependencies = ["click>=8.1.8", "loguru>=0.7.3"]
|
||||
dependencies = [
|
||||
"click>=8.1.8",
|
||||
"httpx>=0.28.1",
|
||||
"loguru>=0.7.3",
|
||||
"tqdm>=4.67.1",
|
||||
]
|
||||
license = "WTFPL"
|
||||
|
||||
[project.scripts]
|
||||
zon2json = "zig_fetch_py.main:main"
|
||||
zig-fetch = "zig_fetch_py.__main__:main"
|
||||
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
|
||||
@ -4,7 +4,6 @@ Unit tests for the ZON parser.
|
||||
|
||||
import json
|
||||
import pytest
|
||||
from pathlib import Path
|
||||
|
||||
from zig_fetch_py.parser import ZonParser, parse_zon_file, zon_to_json
|
||||
|
||||
|
||||
103
uv.lock
generated
103
uv.lock
generated
@ -2,6 +2,29 @@ version = 1
|
||||
revision = 1
|
||||
requires-python = ">=3.12"
|
||||
|
||||
[[package]]
|
||||
name = "anyio"
|
||||
version = "4.8.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "idna" },
|
||||
{ name = "sniffio" },
|
||||
{ name = "typing-extensions", marker = "python_full_version < '3.13'" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/a3/73/199a98fc2dae33535d6b8e8e6ec01f8c1d76c9adb096c6b7d64823038cde/anyio-4.8.0.tar.gz", hash = "sha256:1d9fe889df5212298c0c0723fa20479d1b94883a2df44bd3897aa91083316f7a", size = 181126 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/46/eb/e7f063ad1fec6b3178a3cd82d1a3c4de82cccf283fc42746168188e1cdd5/anyio-4.8.0-py3-none-any.whl", hash = "sha256:b5011f270ab5eb0abf13385f851315585cc37ef330dd88e27ec3d34d651fd47a", size = 96041 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "certifi"
|
||||
version = "2025.1.31"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/1c/ab/c9f1e32b7b1bf505bf26f0ef697775960db7932abeb7b516de930ba2705f/certifi-2025.1.31.tar.gz", hash = "sha256:3d5da6925056f6f18f119200434a4780a94263f10d1c21d032a6f6b2baa20651", size = 167577 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/38/fc/bce832fd4fd99766c04d1ee0eead6b0ec6486fb100ae5e74c1d91292b982/certifi-2025.1.31-py3-none-any.whl", hash = "sha256:ca78db4565a652026a4db2bcdf68f2fb589ea80d0be70e03929ed730746b84fe", size = 166393 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "click"
|
||||
version = "8.1.8"
|
||||
@ -62,6 +85,52 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/fb/b2/f655700e1024dec98b10ebaafd0cedbc25e40e4abe62a3c8e2ceef4f8f0a/coverage-7.6.12-py3-none-any.whl", hash = "sha256:eb8668cfbc279a536c633137deeb9435d2962caec279c3f8cf8b91fff6ff8953", size = 200552 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "h11"
|
||||
version = "0.14.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/f5/38/3af3d3633a34a3316095b39c8e8fb4853a28a536e55d347bd8d8e9a14b03/h11-0.14.0.tar.gz", hash = "sha256:8f19fbbe99e72420ff35c00b27a34cb9937e902a8b810e2c88300c6f0a3b699d", size = 100418 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/95/04/ff642e65ad6b90db43e668d70ffb6736436c7ce41fcc549f4e9472234127/h11-0.14.0-py3-none-any.whl", hash = "sha256:e3fe4ac4b851c468cc8363d500db52c2ead036020723024a109d37346efaa761", size = 58259 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "httpcore"
|
||||
version = "1.0.7"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "certifi" },
|
||||
{ name = "h11" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/6a/41/d7d0a89eb493922c37d343b607bc1b5da7f5be7e383740b4753ad8943e90/httpcore-1.0.7.tar.gz", hash = "sha256:8551cb62a169ec7162ac7be8d4817d561f60e08eaa485234898414bb5a8a0b4c", size = 85196 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/87/f5/72347bc88306acb359581ac4d52f23c0ef445b57157adedb9aee0cd689d2/httpcore-1.0.7-py3-none-any.whl", hash = "sha256:a3fff8f43dc260d5bd363d9f9cf1830fa3a458b332856f34282de498ed420edd", size = 78551 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "httpx"
|
||||
version = "0.28.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "anyio" },
|
||||
{ name = "certifi" },
|
||||
{ name = "httpcore" },
|
||||
{ name = "idna" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/b1/df/48c586a5fe32a0f01324ee087459e112ebb7224f646c0b5023f5e79e9956/httpx-0.28.1.tar.gz", hash = "sha256:75e98c5f16b0f35b567856f597f06ff2270a374470a5c2392242528e3e3e42fc", size = 141406 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/2a/39/e50c7c3a983047577ee07d2a9e53faf5a69493943ec3f6a384bdc792deb2/httpx-0.28.1-py3-none-any.whl", hash = "sha256:d909fcccc110f8c7faf814ca82a9a4d816bc5a6dbfea25d6591d6985b8ba59ad", size = 73517 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "idna"
|
||||
version = "3.10"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/f1/70/7703c29685631f5a7590aa73f1f1d3fa9a380e654b86af429e0934a32f7d/idna-3.10.tar.gz", hash = "sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9", size = 190490 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/76/c6/c88e154df9c4e1a2a66ccf0005a88dfb2650c1dffb6f5ce603dfbd452ce3/idna-3.10-py3-none-any.whl", hash = "sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3", size = 70442 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "iniconfig"
|
||||
version = "2.0.0"
|
||||
@ -130,6 +199,36 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/36/3b/48e79f2cd6a61dbbd4807b4ed46cb564b4fd50a76166b1c4ea5c1d9e2371/pytest_cov-6.0.0-py3-none-any.whl", hash = "sha256:eee6f1b9e61008bd34975a4d5bab25801eb31898b032dd55addc93e96fcaaa35", size = 22949 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "sniffio"
|
||||
version = "1.3.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/a2/87/a6771e1546d97e7e041b6ae58d80074f81b7d5121207425c964ddf5cfdbd/sniffio-1.3.1.tar.gz", hash = "sha256:f4324edc670a0f49750a81b895f35c3adb843cca46f0530f79fc1babb23789dc", size = 20372 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/e9/44/75a9c9421471a6c4805dbf2356f7c181a29c1879239abab1ea2cc8f38b40/sniffio-1.3.1-py3-none-any.whl", hash = "sha256:2f6da418d1f1e0fddd844478f41680e794e6051915791a034ff65e5f100525a2", size = 10235 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tqdm"
|
||||
version = "4.67.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "colorama", marker = "sys_platform == 'win32'" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/a8/4b/29b4ef32e036bb34e4ab51796dd745cdba7ed47ad142a9f4a1eb8e0c744d/tqdm-4.67.1.tar.gz", hash = "sha256:f8aef9c52c08c13a65f30ea34f4e5aac3fd1a34959879d7e59e63027286627f2", size = 169737 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/d0/30/dc54f88dd4a2b5dc8a0279bdd7270e735851848b762aeb1c1184ed1f6b14/tqdm-4.67.1-py3-none-any.whl", hash = "sha256:26445eca388f82e72884e0d580d5464cd801a3ea01e63e5601bdff9ba6a48de2", size = 78540 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "typing-extensions"
|
||||
version = "4.12.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/df/db/f35a00659bc03fec321ba8bce9420de607a1d37f8342eee1863174c69557/typing_extensions-4.12.2.tar.gz", hash = "sha256:1a7ead55c7e559dd4dee8856e3a88b41225abfe1ce8df57b7c13915fe121ffb8", size = 85321 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/26/9f/ad63fc0248c5379346306f8668cda6e2e2e9c95e01216d2b8ffd9ff037d0/typing_extensions-4.12.2-py3-none-any.whl", hash = "sha256:04e5ca0351e0f3f85c6853954072df659d0d13fac324d0072316b67d7794700d", size = 37438 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "win32-setctime"
|
||||
version = "1.2.0"
|
||||
@ -145,7 +244,9 @@ version = "0.1.0"
|
||||
source = { editable = "." }
|
||||
dependencies = [
|
||||
{ name = "click" },
|
||||
{ name = "httpx" },
|
||||
{ name = "loguru" },
|
||||
{ name = "tqdm" },
|
||||
]
|
||||
|
||||
[package.optional-dependencies]
|
||||
@ -157,8 +258,10 @@ dev = [
|
||||
[package.metadata]
|
||||
requires-dist = [
|
||||
{ name = "click", specifier = ">=8.1.8" },
|
||||
{ name = "httpx", specifier = ">=0.28.1" },
|
||||
{ name = "loguru", specifier = ">=0.7.3" },
|
||||
{ name = "pytest", marker = "extra == 'dev'", specifier = ">=8.3.5" },
|
||||
{ name = "pytest-cov", marker = "extra == 'dev'", specifier = ">=6.0.0" },
|
||||
{ name = "tqdm", specifier = ">=4.67.1" },
|
||||
]
|
||||
provides-extras = ["dev"]
|
||||
|
||||
121
zig_fetch_py/__main__.py
Normal file
121
zig_fetch_py/__main__.py
Normal file
@ -0,0 +1,121 @@
|
||||
"""
|
||||
Command-line interface for zig-fetch-py.
|
||||
"""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import click
|
||||
from loguru import logger
|
||||
|
||||
from zig_fetch_py.downloader import process_dependencies
|
||||
from zig_fetch_py.parser import zon_to_json
|
||||
|
||||
|
||||
def setup_logger(verbose: bool = False):
|
||||
"""
|
||||
Set up the logger.
|
||||
|
||||
Args:
|
||||
verbose: Whether to enable verbose logging
|
||||
"""
|
||||
logger.remove()
|
||||
log_level = "DEBUG" if verbose else "INFO"
|
||||
logger.add(sys.stderr, level=log_level)
|
||||
|
||||
|
||||
@click.group()
|
||||
@click.option("-v", "--verbose", is_flag=True, help="Enable verbose logging")
|
||||
@click.pass_context
|
||||
def cli(ctx: click.Context, verbose: bool):
|
||||
"""Zig package manager utilities."""
|
||||
# Set up logging
|
||||
setup_logger(verbose)
|
||||
|
||||
# Ensure we have a context object
|
||||
ctx.ensure_object(dict)
|
||||
ctx.obj["VERBOSE"] = verbose
|
||||
|
||||
|
||||
@cli.command()
|
||||
@click.argument("zon_file", type=click.Path(exists=True, readable=True, path_type=Path))
|
||||
@click.pass_context
|
||||
def download(ctx: click.Context, zon_file: Path):
|
||||
"""
|
||||
Download dependencies from a ZON file.
|
||||
|
||||
ZON_FILE: Path to the ZON file
|
||||
"""
|
||||
logger.info(f"Processing dependencies from {zon_file}")
|
||||
dependencies = process_dependencies(str(zon_file))
|
||||
|
||||
if dependencies:
|
||||
logger.info(f"Successfully processed {len(dependencies)} dependencies:")
|
||||
for name, path in dependencies.items():
|
||||
logger.info(f" - {name}: {path}")
|
||||
else:
|
||||
logger.warning("No dependencies were processed")
|
||||
|
||||
|
||||
@cli.command()
|
||||
@click.argument("zon_file", type=click.Path(exists=True, readable=True, path_type=Path))
|
||||
@click.option(
|
||||
"-o",
|
||||
"--output",
|
||||
type=click.Path(writable=True, path_type=Path),
|
||||
help="Output file (default: stdout)",
|
||||
)
|
||||
@click.option(
|
||||
"-i", "--indent", type=int, default=2, help="Indentation for the JSON output"
|
||||
)
|
||||
@click.option(
|
||||
"--empty-tuple-as-dict",
|
||||
is_flag=True,
|
||||
help="Parse empty tuples as empty dictionaries",
|
||||
)
|
||||
@click.pass_context
|
||||
def convert(
|
||||
ctx: click.Context,
|
||||
zon_file: Path,
|
||||
output: Path,
|
||||
indent: int,
|
||||
empty_tuple_as_dict: bool,
|
||||
):
|
||||
"""
|
||||
Convert a ZON file to JSON.
|
||||
|
||||
ZON_FILE: Path to the ZON file to convert
|
||||
"""
|
||||
try:
|
||||
# Read the ZON file
|
||||
with open(zon_file, "r") as f:
|
||||
zon_content = f.read()
|
||||
|
||||
# Convert to JSON
|
||||
json_content = zon_to_json(
|
||||
zon_content, indent=indent, empty_tuple_as_dict=empty_tuple_as_dict
|
||||
)
|
||||
|
||||
# Output the JSON
|
||||
if output:
|
||||
with open(output, "w") as f:
|
||||
f.write(json_content)
|
||||
logger.info(f"JSON written to {output}")
|
||||
else:
|
||||
click.echo(json_content)
|
||||
|
||||
except FileNotFoundError:
|
||||
logger.error(f"File not found: {zon_file}")
|
||||
sys.exit(1)
|
||||
except Exception as e:
|
||||
logger.error(f"Error: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def main():
|
||||
"""Entry point for the CLI."""
|
||||
cli() # pylint: disable=no-value-for-parameter
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
193
zig_fetch_py/downloader.py
Normal file
193
zig_fetch_py/downloader.py
Normal file
@ -0,0 +1,193 @@
|
||||
"""
|
||||
Dependency downloader for Zig packages.
|
||||
|
||||
This module handles downloading and extracting dependencies specified in ZON files.
|
||||
"""
|
||||
|
||||
import shutil
|
||||
import tarfile
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Dict, Any, Optional
|
||||
|
||||
import httpx
|
||||
from loguru import logger
|
||||
|
||||
from zig_fetch_py.parser import parse_zon_file
|
||||
|
||||
|
||||
def get_cache_dir() -> Path:
|
||||
"""
|
||||
Get the Zig cache directory for packages.
|
||||
|
||||
Returns:
|
||||
Path to the Zig cache directory (~/.cache/zig/p)
|
||||
"""
|
||||
cache_dir = Path.home() / ".cache" / "zig" / "p"
|
||||
cache_dir.mkdir(parents=True, exist_ok=True)
|
||||
return cache_dir
|
||||
|
||||
|
||||
def download_file(url: str, target_path: Path) -> None:
|
||||
"""
|
||||
Download a file from a URL to a target path.
|
||||
|
||||
Args:
|
||||
url: URL to download from
|
||||
target_path: Path to save the downloaded file
|
||||
"""
|
||||
logger.info(f"Downloading {url} to {target_path}")
|
||||
|
||||
# Create client with environment proxies
|
||||
with httpx.Client(follow_redirects=True) as client:
|
||||
with client.stream("GET", url) as response:
|
||||
response.raise_for_status()
|
||||
|
||||
# Create parent directories if they don't exist
|
||||
target_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Write the file
|
||||
with open(target_path, "wb") as f:
|
||||
for chunk in response.iter_bytes():
|
||||
f.write(chunk)
|
||||
|
||||
|
||||
def extract_tarball(tarball_path: Path, extract_dir: Path) -> Path:
|
||||
"""
|
||||
Extract a tarball to a directory.
|
||||
|
||||
Args:
|
||||
tarball_path: Path to the tarball
|
||||
extract_dir: Directory to extract to
|
||||
|
||||
Returns:
|
||||
Path to the extracted directory
|
||||
"""
|
||||
logger.info(f"Extracting {tarball_path} to {extract_dir}")
|
||||
|
||||
# Create extraction directory if it doesn't exist
|
||||
extract_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
with tarfile.open(tarball_path, "r:*") as tar:
|
||||
# Get the common prefix of all files in the tarball
|
||||
members = tar.getmembers()
|
||||
common_prefix = Path(Path(members[0].name).parts[0]) if members else None
|
||||
|
||||
# Extract all files
|
||||
tar.extractall(path=extract_dir)
|
||||
|
||||
# Return the path to the extracted directory
|
||||
return extract_dir / common_prefix if common_prefix else extract_dir
|
||||
|
||||
|
||||
def process_dependency(
|
||||
name: str, dep_info: Dict[str, Any], cache_dir: Path
|
||||
) -> Optional[Path]:
|
||||
"""
|
||||
Process a single dependency from a ZON file.
|
||||
|
||||
Args:
|
||||
name: Name of the dependency
|
||||
dep_info: Dependency information from the ZON file
|
||||
cache_dir: Cache directory to store the dependency
|
||||
|
||||
Returns:
|
||||
Path to the extracted dependency directory, or None if the dependency is already cached
|
||||
"""
|
||||
url = dep_info.get("url")
|
||||
hash_value = dep_info.get("hash")
|
||||
|
||||
if not url or not hash_value:
|
||||
logger.warning(f"Dependency {name} is missing url or hash, skipping")
|
||||
return None
|
||||
|
||||
# Check if the dependency is already cached
|
||||
target_dir = cache_dir / hash_value
|
||||
if target_dir.exists():
|
||||
logger.info(
|
||||
f"Dependency {name} ({hash_value}) is already cached at {target_dir}"
|
||||
)
|
||||
return target_dir
|
||||
|
||||
# Create a temporary directory for downloading and extracting
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
temp_dir_path = Path(temp_dir)
|
||||
|
||||
# Download the tarball
|
||||
tarball_path = temp_dir_path / f"{name}.tar.gz"
|
||||
download_file(url, tarball_path)
|
||||
|
||||
# Extract the tarball to a temporary directory
|
||||
extract_path = extract_tarball(tarball_path, temp_dir_path / "extract")
|
||||
|
||||
# Move the extracted directory to the cache directory with the hash as the name
|
||||
if extract_path and extract_path.exists():
|
||||
if not target_dir.parent.exists():
|
||||
target_dir.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.move(str(extract_path), str(target_dir))
|
||||
|
||||
logger.info(f"Dependency {name} ({hash_value}) cached at {target_dir}")
|
||||
return target_dir
|
||||
else:
|
||||
logger.error(f"Failed to extract {name} from {tarball_path}")
|
||||
return None
|
||||
|
||||
|
||||
def process_dependencies(zon_file_path: str) -> Dict[str, Path]:
|
||||
"""
|
||||
Process all dependencies from a ZON file.
|
||||
|
||||
Args:
|
||||
zon_file_path: Path to the ZON file
|
||||
|
||||
Returns:
|
||||
Dictionary mapping dependency names to their extracted paths
|
||||
"""
|
||||
# Parse the ZON file
|
||||
zon_data = parse_zon_file(zon_file_path)
|
||||
|
||||
# Get the dependencies section
|
||||
dependencies = zon_data.get("dependencies", {})
|
||||
if not dependencies:
|
||||
logger.warning(f"No dependencies found in {zon_file_path}")
|
||||
return {}
|
||||
|
||||
# Get the cache directory
|
||||
cache_dir = get_cache_dir()
|
||||
|
||||
# Process each dependency
|
||||
result = {}
|
||||
for name, dep_info in dependencies.items():
|
||||
path = process_dependency(name, dep_info, cache_dir)
|
||||
if path:
|
||||
result[name] = path
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def main(zon_file_path: str) -> None:
|
||||
"""
|
||||
Main entry point for the dependency downloader.
|
||||
|
||||
Args:
|
||||
zon_file_path: Path to the ZON file
|
||||
"""
|
||||
logger.info(f"Processing dependencies from {zon_file_path}")
|
||||
dependencies = process_dependencies(zon_file_path)
|
||||
|
||||
if dependencies:
|
||||
logger.info(f"Successfully processed {len(dependencies)} dependencies:")
|
||||
for name, path in dependencies.items():
|
||||
logger.info(f" - {name}: {path}")
|
||||
else:
|
||||
logger.warning("No dependencies were processed")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import sys
|
||||
|
||||
if len(sys.argv) != 2:
|
||||
print(f"Usage: {sys.argv[0]} <zon_file_path>")
|
||||
sys.exit(1)
|
||||
|
||||
main(sys.argv[1])
|
||||
@ -1,60 +1,79 @@
|
||||
"""
|
||||
Command-line interface for the ZON parser.
|
||||
Command-line interface for zon2json.
|
||||
"""
|
||||
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import click
|
||||
from loguru import logger
|
||||
|
||||
from zig_fetch_py.parser import parse_zon_file
|
||||
from zig_fetch_py.parser import zon_to_json
|
||||
|
||||
|
||||
def setup_logger(verbose: bool = False):
|
||||
"""
|
||||
Set up the logger.
|
||||
|
||||
Args:
|
||||
verbose: Whether to enable verbose logging
|
||||
"""
|
||||
logger.remove()
|
||||
log_level = "DEBUG" if verbose else "INFO"
|
||||
logger.add(sys.stderr, level=log_level)
|
||||
|
||||
|
||||
@click.command()
|
||||
@click.argument("file", type=click.Path(exists=True, readable=True))
|
||||
@click.argument("zon_file", type=click.Path(exists=True, readable=True, path_type=Path))
|
||||
@click.option(
|
||||
"-o",
|
||||
"--output",
|
||||
type=click.Path(writable=True),
|
||||
help="Output JSON file path (default: stdout)",
|
||||
type=click.Path(writable=True, path_type=Path),
|
||||
help="Output file (default: stdout)",
|
||||
)
|
||||
@click.option(
|
||||
"-i", "--indent", type=int, default=2, help="Indentation for the JSON output"
|
||||
)
|
||||
@click.option(
|
||||
"--empty-tuple-as-dict",
|
||||
is_flag=True,
|
||||
help="Parse empty tuples as empty dictionaries",
|
||||
)
|
||||
@click.option("-p", "--pretty", is_flag=True, help="Pretty print JSON output")
|
||||
@click.option("-v", "--verbose", is_flag=True, help="Enable verbose logging")
|
||||
def main(file, output, pretty, verbose):
|
||||
"""Parse ZON files and convert to JSON.
|
||||
|
||||
This tool parses Zig Object Notation (ZON) files and converts them to JSON format.
|
||||
def main(zon_file, output, indent, empty_tuple_as_dict, verbose):
|
||||
"""
|
||||
# Configure logging
|
||||
log_level = "DEBUG" if verbose else "INFO"
|
||||
logger.remove() # Remove default handler
|
||||
logger.add(sys.stderr, level=log_level)
|
||||
Convert a ZON file to JSON.
|
||||
|
||||
logger.info(f"Processing file: {file}")
|
||||
ZON_FILE: Path to the ZON file to convert
|
||||
"""
|
||||
# Set up logging
|
||||
setup_logger(verbose)
|
||||
|
||||
try:
|
||||
result = parse_zon_file(file)
|
||||
# Read the ZON file
|
||||
with open(zon_file, "r") as f:
|
||||
zon_content = f.read()
|
||||
|
||||
indent = 4 if pretty else None
|
||||
json_str = json.dumps(result, indent=indent)
|
||||
# Convert to JSON
|
||||
json_content = zon_to_json(
|
||||
zon_content, indent=indent, empty_tuple_as_dict=empty_tuple_as_dict
|
||||
)
|
||||
|
||||
# Output the JSON
|
||||
if output:
|
||||
logger.info(f"Writing output to: {output}")
|
||||
with open(output, "w") as f:
|
||||
f.write(json_str)
|
||||
f.write(json_content)
|
||||
logger.info(f"JSON written to {output}")
|
||||
else:
|
||||
logger.debug("Writing output to stdout")
|
||||
click.echo(json_str)
|
||||
click.echo(json_content)
|
||||
|
||||
except FileNotFoundError:
|
||||
logger.error(f"File not found: {zon_file}")
|
||||
sys.exit(1)
|
||||
except Exception as e:
|
||||
logger.error(f"Error: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
# This is only executed when the module is run directly
|
||||
if __name__ == "__main__":
|
||||
# When imported as a module, click will handle the function call
|
||||
# When run directly, we need to call it explicitly
|
||||
main()
|
||||
|
||||
@ -3,8 +3,7 @@ ZON parser module - Parses Zig Object Notation (ZON) files.
|
||||
"""
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Union, Tuple, Optional
|
||||
from typing import Any, Dict, List, Union, Optional
|
||||
|
||||
from loguru import logger
|
||||
|
||||
|
||||
Reference in New Issue
Block a user