将Kotti从Pyramid1.0升级到2.0 (失败的记录)
Kotti是非常好的一个基于Pyramid的web信息管理系统框架,但是它从2022年开始就没有更新了,只支持Pyramid1.10版本,不支持Pyramid2.0版本。
准备将Kotti的代码升级到Pyramid2.0
基本思路,就是像论文复现一样,将pyrmaid1.0的api替换成2.0代码。
在pyramid手册里,有Pyramid2.0 弃用的api,需要子kotti中将那些弃用的api都替换掉。
这次没有升级成功,但是最终升级成功了:使用auto-coder将kotti项目的pyramid依赖从1.x升级到2.x,将SQLALchemy从1.x升级到2.x-CSDN博客
转换前的工作:
将主git选择为atomgit :https://atomgit.com/,理由是速度要比github快,而且不抽风,相比github可以提高工作效率。选atomgit而不选启智平台:https://openi.pcl.ac.cn,是因为启智平台是三级域名,不如atomgit的一级域名好记好用。
在github fork Kotti主库到:https://github.com/skywalk163/Kotti
在atomgit镜像github自己的Kotti库:https://atomgit.com/skywalk/Kotti
在atomgit创建一个专门用于转换工作的repo:https://atomgit.com/skywalk/Kotti/tree/kotti_pyramid2
开始转换
第一步修改kotti支持pyramid2.0:
下载Kotti源码并切换到kotti_pyramid2 分支:
git clone https://atomgit.com/skywalk/Kotticd Kotti && git switch kotti_pyramid2
修改Kotti库setup.py文件,以便支持pyramid2.0 ,将45行的'pyramid>=1.9,<2' 修改成'pyramid>=1.9,<2.2'
安装pyramid2.0
pip install pyramid -U
用开发模式安装新版本的kotti
python setup.py develop
如果这里没有安装好kotti的依赖库的话,就需要手动安装一下:
pip install -r requirements.txt
第二步 执行代码测试
参考文档Upgrading Pyramid — The Pyramid Web Framework v2.0.2 进行代码测试
(py310) [skywalk@x250 ~/github/Kotti]$ python -Wd setup.py test -q
/usr/home/skywalk/py310/lib/python3.10/site-packages/setuptools/dist.py:487: UserWarning: Normalizing '2.0.10dev0' to '2.0.10.dev0'warnings.warn(tmpl.format(**locals()))
running test
WARNING: Testing via this command is deprecated and will be removed in a future version. Users looking for a generic test entry point independent of test runner are encouraged to use tox.
running egg_info
writing Kotti.egg-info/PKG-INFO
writing dependency_links to Kotti.egg-info/dependency_links.txt
writing entry points to Kotti.egg-info/entry_points.txt
writing requirements to Kotti.egg-info/requires.txt
writing top-level names to Kotti.egg-info/top_level.txt
reading manifest template 'MANIFEST.in'
warning: no files found matching 'Dockerfile'
warning: no previously-included files matching '*.pyc' found under directory '*'
warning: no previously-included files matching '__pycache__' found under directory '*'
warning: no previously-included files matching '._DS_Store' found under directory '*'
warning: no previously-included files matching '*' found under directory 'website'
no previously-included directories found matching 'docs/_build'
adding license file 'LICENSE.txt'
adding license file 'AUTHORS.txt'
writing manifest file 'Kotti.egg-info/SOURCES.txt'
running build_ext
E
======================================================================
ERROR: kotti (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: kotti
Traceback (most recent call last):File "/usr/local/lib/python3.10/unittest/loader.py", line 470, in _find_test_pathpackage = self._get_module_from_name(name)File "/usr/local/lib/python3.10/unittest/loader.py", line 377, in _get_module_from_name__import__(name)File "/usr/home/skywalk/github/Kotti/kotti/__init__.py", line 18, in <module>from kotti.sqla import Base as KottiBaseFile "/usr/home/skywalk/github/Kotti/kotti/sqla.py", line 7, in <module>from pyramid.compat import json
ModuleNotFoundError: No module named 'pyramid.compat'----------------------------------------------------------------------
Ran 1 test in 0.000sFAILED (errors=1)
Test failed: <unittest.runner.TextTestResult run=1 errors=1 failures=0>
error: Test failed: <unittest.runner.TextTestResult run=1 errors=1 failures=0>
根据报错修改代码,修改kotti/sqla.py文件即可。详细见调试部分
发现好几个地方调用compat.py文件。同时手册提醒升级要一级一级的升,以免错过弃用警告。
因此先升级到1.10版本吧。
第三步 执行pyramid1.0版本的Kotti测试
有很多显示:
python -Wd setup.py test -q
/usr/home/skywalk/py310/lib/python3.10/site-packages/setuptools/dist.py:487: UserWarning: Normalizing '2.0.10dev0' to '2.0.10.dev0'warnings.warn(tmpl.format(**locals()))
running test
WARNING: Testing via this command is deprecated and will be removed in a future version. Users looking for a generic test entry point independent of test runner are encouraged to use tox.
running egg_info
writing Kotti.egg-info/PKG-INFO
writing dependency_links to Kotti.egg-info/dependency_links.txt
writing entry points to Kotti.egg-info/entry_points.txt
writing requirements to Kotti.egg-info/requires.txt
writing top-level names to Kotti.egg-info/top_level.txt
reading manifest template 'MANIFEST.in'
warning: no files found matching 'Dockerfile'
warning: no previously-included files matching '*.pyc' found under directory '*'
warning: no previously-included files matching '__pycache__' found under directory '*'
warning: no previously-included files matching '._DS_Store' found under directory '*'
warning: no previously-included files matching '*' found under directory 'website'
no previously-included directories found matching 'docs/_build'
adding license file 'LICENSE.txt'
adding license file 'AUTHORS.txt'
writing manifest file 'Kotti.egg-info/SOURCES.txt'
running build_ext
/usr/home/skywalk/py310/lib/python3.10/site-packages/pyramid/path.py:4: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative usesimport imp
/usr/home/skywalk/github/Kotti/kotti/sqla.py:18: SADeprecationWarning: Baked lazy loading is now the default implementation. (deprecated since: 1.2)baked.bake_lazy_loaders()
/usr/local/lib/python3.10/unittest/loader.py:122: DeprecationWarning: ObjectAfterDelete: The ObjectAfterDelete event is deprecated and will be no longer available starting with Kotti 0.10.obj = getattr(module, name)
/usr/home/skywalk/github/Kotti/kotti/tests/__init__.py:71: PytestDeprecationWarning: @pytest.yield_fixture is deprecated.
Use @pytest.fixture instead; they are the same.def allwarnings(request):
/usr/home/skywalk/github/Kotti/kotti/tests/__init__.py:118: PytestDeprecationWarning: @pytest.yield_fixture is deprecated.
Use @pytest.fixture instead; they are the same.def config(settings):
/usr/home/skywalk/github/Kotti/kotti/tests/__init__.py:194: PytestDeprecationWarning: @pytest.yield_fixture is deprecated.
Use @pytest.fixture instead; they are the same.def db_session(config, content, connection):
/usr/home/skywalk/github/Kotti/kotti/tests/__init__.py:237: PytestDeprecationWarning: @pytest.yield_fixture is deprecated.
Use @pytest.fixture instead; they are the same.def events(config):
/usr/home/skywalk/github/Kotti/kotti/tests/__init__.py:331: PytestDeprecationWarning: @pytest.yield_fixture is deprecated.
Use @pytest.fixture instead; they are the same.def depot_tween(config, dummy_request):
/usr/home/skywalk/github/Kotti/kotti/tests/__init__.py:359: PytestDeprecationWarning: @pytest.yield_fixture is deprecated.
Use @pytest.fixture instead; they are the same.def mock_filedepot(depot_tween):
/usr/home/skywalk/github/Kotti/kotti/tests/__init__.py:376: PytestDeprecationWarning: @pytest.yield_fixture is deprecated.
Use @pytest.fixture instead; they are the same.def filedepot(db_session, depot_tween):
/usr/home/skywalk/github/Kotti/kotti/tests/__init__.py:392: PytestDeprecationWarning: @pytest.yield_fixture is deprecated.
Use @pytest.fixture instead; they are the same.def no_filedepots(db_session, depot_tween):
/usr/local/lib/python3.10/unittest/loader.py:122: DeprecationWarning: EventTestBase: Unittest-style tests are deprecated as of Kotti 0.7. Please use pytest function arguments instead.obj = getattr(module, name)
/usr/local/lib/python3.10/unittest/loader.py:122: DeprecationWarning: FunctionalTestBase: Unittest-style tests are deprecated as of Kotti 0.7. Please use pytest function arguments instead.obj = getattr(module, name)
/usr/local/lib/python3.10/unittest/loader.py:122: DeprecationWarning: UnitTestBase: Unittest-style tests are deprecated as of Kotti 0.7. Please use pytest function arguments instead.obj = getattr(module, name)
/usr/local/lib/python3.10/unittest/loader.py:122: DeprecationWarning: _init_testing_db: Unittest-style tests are deprecated as of Kotti 0.7. Please use pytest function arguments instead.obj = getattr(module, name)----------------------------------------------------------------------
Ran 0 tests in 0.000sOK
将这些显示分门别类并进行相应的处理。
/usr/home/skywalk/github/Kotti/kotti/sqla.py:18: SADeprecationWarning: Baked lazy loading is now the default implementation. (deprecated since: 1.2)
baked.bake_lazy_loaders()
也就是baked.bake_lazy_loaders()这句是默认了,不需要重复写。去源码里注释掉。
/usr/local/lib/python3.10/unittest/loader.py:122: DeprecationWarning: ObjectAfterDelete: The ObjectAfterDelete event is deprecated and will be no longer available starting with Kotti 0.10.
obj = getattr(module, name)
弃用警告从Kotti0.10开始么有ObjectAfterDelete,这个不用管。
/usr/home/skywalk/github/Kotti/kotti/tests/__init__.py:71: PytestDeprecationWarning: @pytest.yield_fixture is deprecated.
Use @pytest.fixture instead; they are the same.
def allwarnings(request):
后面也有很多,都是说这一件事情,就是@pytest.yield_fixture 废弃,使用 @pytest.fixture 代替,他们是一样的。这样修改多处的,就用vscode吧。改了8处,很奇怪,作者里面既有yield_fixture也有fixture,不明白当时为什么没有全改掉。
里面有pyramid的报错不用管,因为pyramid现在是1.10,自然有pyramid的官方来处理它的代码遗留问题。
测试还是有问题,Unittest-style tests are deprecated as of Kotti 0.7. Please use pytest function arguments instead. 这里还要改,不然整个测试就没继续下去。
第四步 处理测试0个tests的问题
一顿操作,反正现在是能跑了:
大约改了几个地方:
1 安装了
if __name__ == '__main__':
unittest.main()
1756 vi /home/linuxskywalk/github/Kotti/kotti/tests/conftest.py1757 pip install -e . 1758 pytest1759 pytest .1760 pytest kotti/tests1761 pip install pytest-flake8 pytest-cov1762 pytest
咨询文心一言:请帮我查询pytest官方文档,并给出关于如何从unittest迁移到pytest的部分的详细的指导。
/usr/local/lib/python3.10/unittest/loader.py:122: DeprecationWarning: EventTestBase: Unittest-style tests are deprecated as of Kotti 0.7. Please use pytest function arguments instead.
obj = getattr(module, name)
/usr/local/lib/python3.10/unittest/loader.py:122: DeprecationWarning: FunctionalTestBase: Unittest-style tests are deprecated as of Kotti 0.7. Please use pytest function arguments instead.
obj = getattr(module, name)
/usr/local/lib/python3.10/unittest/loader.py:122: DeprecationWarning: UnitTestBase: Unittest-style tests are deprecated as of Kotti 0.7. Please use pytest function arguments instead.
obj = getattr(module, name)
/usr/local/lib/python3.10/unittest/loader.py:122: DeprecationWarning: _init_testing_db: Unittest-style tests are deprecated as of Kotti 0.7. Please use pytest function arguments instead.
obj = getattr(module, name)
调试
报错No module named 'pyramid.compat'
发现老版本有compat.py这个文件https://atomgit.com/skywalk/pyramid/blob/1.9-branch/pyramid/compat.py,但是2.0版本已经没有了。
修改kotti/sqla.py文件
# from pyramid.compat import json
import json
在ubuntu下执行报错no attribute 'bake_lazy_loaders'
File "/home/linuxskywalk/github/Kotti/kotti/sqla.py", line 17, in <module>
baked.bake_lazy_loaders()
AttributeError: module 'sqlalchemy.ext.baked' has no attribute 'bake_lazy_loaders'
将/home/linuxskywalk/github/Kotti/kotti/sqla.py里这句注释掉:baked.bake_lazy_loaders()
报错 /home/skywalk/github/Kotti/kotti/sqla.py:9: DeprecationWarning: Allow: "pyramid.security.Allow" is deprecated in Pyramid 2.0
从测试报错来看,主要问题是缺少db_session fixture。建议检查测试文件中的fixture定义,确保正确导入了相关依赖。同时需要确认测试环境是否已正确配置数据库连接。
首先搜索测试文件中是否已定义db_session fixture,以确认问题所在。
通过搜索测试文件,确认db_session fixture已在kotti/tests/ init .py中定义,且被多个测试用例使用。测试环境中的数据库连接配置需要进一步检查。
kotti/sqla.py:9
/home/skywalk/github/Kotti/kotti/sqla.py:9: DeprecationWarning: Allow: "pyramid.security.Allow" is deprecated in Pyramid 2.0. Adjust your import to "pyramid.authorization.Allow"
from pyramid.security import Allow
也就是sqla.py这里就通不过。