It’s a potential single point of failure. Which have experienced first hand. The rest of the app could not run cuz a non-essential piece was non-operable due to the missing compiled message definitions file or message definitions file was updated but not compiled.
So protobuf carries a non-zero risk.
Could the app have been designed without an essential exploding binary blob? Most definitely yes!
A better approach
That unfortunately isn’t a better approach. The compilation step requires protobuf to be installed, by the distro package manager. To my knowledge it’s not available from pypi.
An uncompiled protobuf file is essentially worthless unless it’s compiled. But if it’s compiled then it’s a binary blob.
Not anti-protobuf. Just make the protobuf compiler available without getting a distro package manager involved.
Otherwise slower alternatives might be more viable.
strictyaml bundles strictyaml.ruamel, which used to be an external unmaintained C package.
This reduces strictyaml dependencies to:
pyproject.toml
dependencies = [
"python-dateutil>=2.6.0"
]
Just that one. So can be confident strictyaml will work.
Can the same be said for protobuf and Google (over invested in AI and is probably dying underneath a huge debt burden while spending tons of money on AI wash propaganda while not funding Python projects enough. Maintainer leave or burn out while everyone is too busy head fcking us with the AI washing to notice.)
Also there is strictyaml that validates against schemas. Don’t touch the builtin yaml module.
protobuf needs to be compiled. This introduces possibility of coder error. Just forgetting to compile and commit protobuf files after a change. This affected the electrum btc and ltc (light) wallets.
and the entire ecosystem of packages which needs to be maintained.
Unless you think that is done by magic fairy elves
Allude to Rust without saying Rust
Oh you mean MSFT is part of Alpha-Omega foundation and their money is not green.
tl;dr;
Alpha-Omega funds Seth Larson and Mike Fiedler. Alpha-Omega is big corp grants to open source projects, except yours. Never yours.
Anthropic is the AI research and development company behind Claude — the frontier model used by millions of people worldwide.
btw use mypy AND pyright. The OP writes, mypy OR pyright.
mypy – chokes on excessive number of overloads. Annoyingly runs forever.
pyright – more verbose; finds issues mypy never will
So when mypy broke had to use pyright. Then fixed mypy by ignoring a module and some configuration voodoo.
wreck can. It’s venv aware. Takes full advantage of hierarchical requirement files. Is intuitive. The learning curve is minimal. Written in Python.
[[tool.wreck.venvs]]
venv_base_path = '.venv'
reqs = [
'requirements/pip',
'requirements/pip-tools',
'requirements/prod',
'requirements/dev',
'requirements/manage',
'requirements/kit',
'requirements/mypy',
'requirements/tox',
]
[[tool.wreck.venvs]]
venv_base_path = '.doc/.venv'
reqs = [
'docs/requirements',
'docs/pip-tools',
]
[tool.setuptools.dynamic]
dependencies = { file = ['requirements/prod.unlock'] }
optional-dependencies.pip = { file = ['requirements/pip.lock'] }
optional-dependencies.pip_tools = { file = ['requirements/pip-tools.lock'] }
optional-dependencies.dev = { file = ['requirements/dev.lock'] }
optional-dependencies.manage = { file = ['requirements/manage.lock'] }
optional-dependencies.docs = { file = ['docs/requirements.lock'] }
reqs fix --venv-relpath='.venv'
reqs fix --venv-relpath='.doc/.venv'
From *.in requirements files would produce *.unlock and *.lock files for venv .venv. Package versions are sync’ed within all requirements files within that venv.
That means it can’t be maintained by the community it’s serving. For those skilled in both Rust and Python can understand why it would appear as the best thing since sliced bread.
What about for those not skilled in Rust? Should we also learn 15 other coding languages while we are at it?
wreck is written in Python and manages dependencies. via hierarchy of requirements files, not TOML. It’s venv aware. For Python package authors.
On this board, and hope everyone else is, to learn and share. Try to contribute, not just state my opinion.
Didn’t feel this thread offered anything substantial to learn, so threw out some gem that others might find useful in their projects.
Package DevOPs is a skill. Half assing it will not:
impress a published package author’s peers
encourage confidence in the authors perseverance to maintain the package
encourage confidence in the author’s willingness to politely respond to Issues and PRs
package will have issues that never get resolved cuz the author packaging skillset is woefully lacking including but not limited to: Makefile, pre-commit, tox, gha, doctest, documentation, test coverage, multiple platform support, manylinux support, setup for collaboration, and static typing.
So suggest redirecting efforts towards studying how to improve packaging rather than how to avoid packaging.
A suggestion on things to improve documentation:
move the doctest out of in-code documentation and into test suite! So the doctest are proof and contribute to coverage
python -m coverage run --parallel --data-file=.coverage-combine-tests -m pytest --doctest-glob="*.rst" --showlocals $(verbose_text) tests/safe tests/a tests/b
This is straight from a Makefile. Where the file extension for doctest was changed to .rst. Can see have A B testing setup to compare two packages.
Example how to include doctest into Sphinx document
.. literalinclude:: ../tests/a/test_presentation.rst
:language: pycon
:name: unique-id
:caption: illustrative-description
project cost = sigma(1...n)(risk likelihood of occurring * risk cost), but we aren’t discussing every possible risk. Only the one risk.The risk of having to:
setup.py. This is referred to asthe sewer, which is what is targeted by hackers e.g. xvor
Just not doing that
The only justification for going with protoc, over other methods, could only come down to data serialization speed. But in that case, wouldn’t a rust solution be: not only as fast, but also much safer.