diff --git a/.markdownlint.rb b/.markdownlint.rb
new file mode 100644
index 0000000..a190c0c
--- /dev/null
+++ b/.markdownlint.rb
@@ -0,0 +1,4 @@
+all
+# MD033 Inline HTML
+# TODO: Tweak elements when https://github.com/markdownlint/markdownlint/issues/118 will be done?
+exclude_rule 'MD033'
\ No newline at end of file
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 7cce82a..4250433 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -30,6 +30,14 @@ repos:
rev: "6.1.0"
hooks:
- id: flake8
+
+ - repo: https://github.com/igorshubovych/markdownlint-cli
+ rev: "v0.41.0"
+ hooks:
+ - id: markdownlint
+ args: [ --disable=MD033 ]
+
+
# - repo: https://gitlab.com/smop/pre-commit-hooks
# rev: v1.0.0
# hooks:
diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md
index 903091e..b9ec520 100644
--- a/CONTRIBUTORS.md
+++ b/CONTRIBUTORS.md
@@ -1,4 +1,6 @@
-# Developers
+# Contributors
+
+## Developers
* Sylvain LE GAL, from [GRAP](http://www.grap.coop), since March 2022
* Rémy TAYMANS, from [Coop It Easy](https://coopiteasy.be/), since June 2022
@@ -9,9 +11,11 @@
* Hugues DE KEYSER, from [Coop It Easy](https://coopiteasy.be/), since April 2024
* Gabriel PICKENHAYN, since May 2024
* Boris GALLET, since September 2024
-* Ahmet YIĞIT BUDAK, from [Altinkaya](https://www.altinkaya.com/fr), since October 2024
-* Alexandre AUBIN, from [Coopaname](https://www.coopaname.coop/), since October 2024
+* Ahmet YIĞIT BUDAK, from [Altinkaya](https://www.altinkaya.com/fr),
+ since October 2024
+* Alexandre AUBIN, from [Coopaname](https://www.coopaname.coop/),
+ since October 2024
-# Reviewers
+## Reviewers
-* Sébastien BEAU, from Akretion (https://akretion.com)
+* Sébastien BEAU, from [Akretion](https://akretion.com)
diff --git a/DEVELOP.md b/DEVELOP.md
index 866e315..f16f19f 100644
--- a/DEVELOP.md
+++ b/DEVELOP.md
@@ -1,8 +1,10 @@
-# Installation to develop
+# Develop
-## Basic installation
+## Installation to develop
-```
+### Basic installation
+
+```shell
git clone https://gitlab.com/odoo-openupgrade-wizard/odoo-openupgrade-wizard/
cd odoo-openupgrade-wizard
virtualenv env --python=python3
@@ -12,19 +14,20 @@ poetry install
``odoo-openupgrade-wizard`` commands are now available in your virutalenv.
-## Advanced installation
+### Advanced installation
If you want to use this library without installing anything in your
system, execute the following steps, otherwise, go to 'Installation' part.
-1. Run a docker container:
-
-``docker run -it ubuntu:focal``
-
-2. Execute the following commnands
+- Run a docker container:
+```shell
+docker run -it ubuntu:focal
```
+- Execute the following commnands
+
+```shell
apt-get update
apt-get install git python3 python3-pip python3-venv
@@ -37,40 +40,40 @@ pipx install virtualenv
pipx install poetry
```
-# Run tests
+## Run tests
-## Via pytest (simple)
+### Via pytest (simple)
This will run tests only for the current ``python3.X`` version.
(in your virtualenv)
-```
+
+```shell
poetry run pytest --cov odoo_openupgrade_wizard --verbosity=2 --exitfirst
```
-## Via Tox (advanced)
+### Via Tox (advanced)
This will run tests for all the python versions put in the ``tox.ini`` folder.
(in your virtualenv)
-```
+
+```shell
tox
```
Note : you should have all the python versions available in your local system.
-
-```
+```shell
sudo apt-get install python3.6 python3.6-distutils
sudo apt-get install python3.7 python3.7-distutils
sudo apt-get install python3.8 python3.8-distutils
sudo apt-get install python3.9 python3.9-distutils
```
-## Via Gitlab Runner locally
+### Via Gitlab Runner locally
-
-```
+```shell
# Install tools
pipx install gitlabci-local
@@ -78,17 +81,19 @@ pipx install gitlabci-local
gitlabci-local
```
-# Debugging
+## Debugging
Some docker command could help you when using / developping this tools.
-**Enter the postgres container**
+To Enter the postgres container, run:
+```shell
docker exec -it POSTGRES_CONTAINER_NAME /bin/bash
+```
-# Contribute
+## Contribute
-## Provide newsfragments in your merge requests
+### Provide newsfragments in your merge requests
If you propose a merge request, please add a newsfragments with it.
@@ -115,11 +120,12 @@ a `+` symbol.
This is a example of newsfragments.
`newsfragments/+sub-command-cowsay.feature`:
+
```
Adds a new subcommand `cowsay` to allow poeple to speak like a cow.
```
-## Add python dependencies
+### Add python dependencies
If you add new dependencies, you have to:
@@ -127,7 +133,7 @@ If you add new dependencies, you have to:
- run the following command in your virtualenv : ``poetry update``
-## Release on Gitlab and publish on PyPI
+### Release on Gitlab and publish on PyPI
Building, releasing and publishing a new version works with tags.
@@ -143,7 +149,7 @@ on gitlab, but they trigger a publication on PyPI.
Before creating a tag, ensure that the version of the program is
updated. To update the program version you can use the command:
-```
+```shell
poetry version {major,minor,patch}
```
@@ -163,27 +169,26 @@ To see if the publication on PyPI and the release on gitlab were done
correctly, go in Build > Pipelines. You will find a pipeline for the tag
you just created.
+## Understanding the library
-# Understanding the library
-
-## Tools to understand
+### Tools to understand
The library is using many tools. It is recommanded to understand that tools
to contribute to that project:
-* Docker (https://www.docker.com/)
-* Gitlab CI (https://docs.gitlab.com/ee/ci/quick_start/index.html)
-* openupgrade project (https://github.com/oca/openupgrade) and related openupgradelib (https://github.com/oca/openupgradelib)
-* poetry (https://python-poetry.org/)
-* odoorpc (https://github.com/OCA/odoorpc)
-* git-aggregator (https://github.com/acsone/git-aggregator)
+- Docker ()
+- Gitlab CI ()
+- openupgrade project ()
+ and related openupgradelib ()
+- poetry ()
+- odoorpc ()
+- git-aggregator ()
Also this project is inspired by the following tools:
-* click-odoo-contrib (https://github.com/acsone/click-odoo-contrib)
+- click-odoo-contrib ()
-
-# Dockerfile information
+## Dockerfile information
### From version 5 to 7
@@ -195,61 +200,68 @@ Try to create dockerfile, based on the odoo official ones fails. Any help welcom
### From version 11.0 to latest version. (Python3)
-The Dockerfile of the version 11 to the lastest version of Odoo are written this way :
+The Dockerfile of the version 11 to the lastest version of Odoo are written
+this way:
-- Copy the content of https://github.com/odoo/odoo/blob/ODOO_VERSION/setup/package.dfsrc
+- Copy the content of
- remove all the part after the big ``apt-get install``
-- install debian package ``git`` to have the possibility to pip install from git url.
+- install debian package ``git`` to have the possibility
+ to pip install from git url.
- install custom debian packages
- install python odoo requirements
-- install python ``setuptools-scm`` lib to have the possibility to pip install ``openupgradelib`` from git url.
+- install python ``setuptools-scm`` lib to have the possibility
+ to pip install ``openupgradelib`` from git url.
- install python custom requirements
- makes link between external user and docker odoo user
-## Réferences
+## References
- how to install gitlab runner locally:
-https://docs.gitlab.com/runner/install/linux-manually.html
+
- Check your CI locally. (French)
-https://blog.stephane-robert.info/post/gitlab-valider-ci-yml/
-
-https://blog.callr.tech/building-docker-images-with-gitlab-ci-best-practices/
+
+
## Python version settings depending on the debian version
This part can help you if you want to change your autogenerated Dockerfiles.
-See (https://github.com/odoo/odoo/blob/ODOO_VERSION/setup/package.dfdebian)
+See ().
### debian:wheezy (V7) (for Odoo: 8.0)
+
- Ubuntu release : 12.04 xxx, 12.10 xxx, 13.04 xxx, 14.10 xxx
- python2.7
- First release : 04/05/2013
- End LTS : May 2018
### debian:jessie (V8) (for Odoo: 9.0, 10.0)
+
- Ubuntu release : 14.04 trusty, 14.10 utopic, 15.04 vivid, 15.10 wily
- python2.7
- First release : 26/04/2015
- End LTS : June 2020
### debian:stretch (V9) (for Odoo: 11.0, 12.0)
+
- Ubuntu releases : 16.04 xenial, 16.10 yakkety, 17.04 zesty, 17.10 artful
- python2.7 and python3.5
- First release : 17/06/2017
- End LTS : June 2022
### debian:buster (13.0, 14.0)
+
- Ubuntu release : 18.04 bionic, 18.10 cosmic, 19.04 disco, 19.10 eoan
- python2.7 and python3.7
- First release : 06/07/2019
- End LTS : Undefined.
-## debian:bullseye (15.0, 16.0)
+### debian:bullseye (15.0, 16.0)
+
- Ubuntu release : 20.04 focal, 20.10 groovy, 21.04 hirsute, 21.10 impish
- python3.9
- First release : 14/07/2021
diff --git a/INTERNAL_NOTES.md b/INTERNAL_NOTES.md
deleted file mode 100644
index 20dcffc..0000000
--- a/INTERNAL_NOTES.md
+++ /dev/null
@@ -1,69 +0,0 @@
-# TODO
-
-* with coop it easy :
-- short_help of group decorator ? seems useless...
-
-* add constrains on ``--step`` option.
-
-* revert : set 777 to log and filestore to be able to write on this folder
- inside the containers. TODO, ask to coop it easy or commown for better alternative.
-
-* allow to call odoo-bin shell, via : https://github.com/d11wtq/dockerpty
- (see https://github.com/docker/docker-py/issues/247)
-
-
-# List of the series of odoo
-# python version is defined, based on the OCA CI.
-# https://github.com/OCA/oca-addons-repo-template/blob/master/src/.github/workflows/%7B%25%20if%20ci%20%3D%3D%20'GitHub'%20%25%7Dtest.yml%7B%25%20endif%20%25%7D.jinja
-
-
-# tips
-```
-# execute sql request in postgres docker
-docker exec db psql --username=odoo --dbname=test_v12 -c "update res_partner set ""email"" = 'bib@bqsdfqsdf.txt';"
-```
-
-# TODO Nice To have
-
-- Fix gitlabci-local. For the time being, it is not possible to debug
- locally. (there are extra bugs locally that doesn't occures on gitlab,
- in ``cli_B_03_run_test.py``...
-
-
-- add
-
-# Try gitlab runner
-
-curl -LJO "https://gitlab-runner-downloads.s3.amazonaws.com/latest/deb/gitlab-runner_amd64.deb"
-
-sudo dpkg -i gitlab-runner_amd64.deb
-
-(https://docs.gitlab.com/runner/install/linux-manually.html)
-
-
-# TODO:
-- check dynamic user id with
-https://github.com/camptocamp/docker-odoo-project/blob/master/bin/docker-entrypoint.sh
-
-
-in modules.csv.j2 :
-# TODO, this value are usefull for test for analyse between 13 and 14.
-# move that values in data/extra_script/modules.csv
-# and let this template with only 'base' module.
-
-
-
-## Without postgres optimization
-2022-07-13 19:42
-2022-07-13 21:20
-
-Duration : 1:37 (107)
-
-## With postgres optimization
-
-2022-07-13 21:52
-2022-07-14 23:11
-
-duration : 1:19 (79)
-
-16%
diff --git a/README.bk b/README.bk
new file mode 100644
index 0000000..53d0525
--- /dev/null
+++ b/README.bk
@@ -0,0 +1,449 @@
+
+
+
+# Usage
+
+**Note:**
+
+the term ``odoo-openupgrade-wizard`` can be replaced by ``oow``
+in all the command lines below.
+
+
+
+## Command: ``init``
+
+```shell
+odoo-openupgrade-wizard init\
+ --initial-version=10.0\
+ --final-version=12.0\
+ --project-name=my-customer-10-12\
+ --extra-repository=OCA/web,OCA/server-tools
+```
+
+Initialize a folder to make a migration from a 10.0 and a 12.0 database.
+This will generate the following structure :
+
+```
+filestore/
+log/
+ 2022_03_25__23_12_41__init.log
+ ...
+postgres_data/
+scripts/
+ step_1__update__10.0/
+ pre-migration.sql
+ post-migration.py
+ step_2__upgrade__11.0/
+ ...
+ step_3__upgrade__12.0/
+ ...
+ step_4__update__12.0/
+ ...
+src/
+ env_10.0/
+ extra_debian_requirements.txt
+ Dockerfile
+ odoo.conf
+ extra_python_requirements.txt
+ repos.yml
+ src/
+ env_11.0/
+ ...
+ env_12.0/
+ ...
+config.yml
+modules.csv
+```
+
+- ``config.yml`` is the main configuration file of your project.
+
+- ``modules.csv`` file is an optional file. You can fill it with the list
+ of your modules installed on your production. The first column of this
+ file should contain the technical name of the module.
+
+- ``log`` folder will contains all the log of the ``odoo-openupgrade-wizard``
+ and the logs of the odoo instance that will be executed.
+
+- ``filestore`` folder will contains the filestore of the odoo database(s)
+
+- ``postgres_data`` folder will be used by postgres docker image to store
+ database.
+
+- ``scripts`` folder contains a folder per migration step. In each step folder:
+ - ``pre-migration.sql`` can contains extra SQL queries you want to execute
+ before beginning the step.
+ - ``post-migration.py`` can contains extra python command to execute
+ after the execution of the step.
+ Script will be executed with ``odoo shell`` command. All the ORM is available
+ via the ``env`` variable.
+
+- ``src`` folder contains a folder per Odoo version. In each environment folder:
+ - ``repos.yml`` enumerates the list of the repositories to use to run the odoo instance.
+ The syntax should respect the ``gitaggregate`` command.
+ (See : https://pypi.org/project/git-aggregator/)
+ Repo files are pre-generated. You can update them with your custom settings.
+ (custom branches, extra PRs, git shallow options, etc...)
+ - ``extra_python_requirements.txt`` enumerates the list of extra python librairies
+ required to run the odoo instance.
+ The syntax should respect the ``pip install -r`` command.
+ (See : https://pip.pypa.io/en/stable/reference/requirements-file-format/)
+ - ``extra_debian_requirements.txt`` enumerates the list of extra system librairies
+ required to run the odoo instance.
+ - ``odoo.conf`` file. Add here extra configuration required for your custom modules.
+ the classical keys (``db_host``, ``db_port``, etc...) are automatically
+ autogenerated.
+
+At this step, you should change the autogenerated files.
+You can use default files, if you have a very simple odoo instance without custom code,
+extra repositories, or dependencies...
+
+**Note:**
+
+- In your repos.yml, preserve ``openupgrade`` and ``server-tools`` repositories
+ to have all the features of the librairies available.
+- In your repos.yml file, the odoo project should be in ``./src/odoo``
+ and the openupgrade project should be in ``./src/openupgrade/`` folder.
+
+
+
+## Command: ``pull-submodule``
+
+**Prerequites:** init + being in a git repository. (if not, you can simply run ``git init``)
+
+if you already have a repos.yml file on github / gitlab, it can be convenient to
+synchronize the repository, instead of copy past the ``repos.yml`` manually.
+
+In that case, you can add extra values, in the ``config.yml`` file in the section
+
+```yaml
+odoo_version_settings:
+ 12.0:
+ repo_url: url_of_the_repo_that_contains_a_repos_yml_file
+ repo_branch: 12.0
+ repo_file_path: repos.yml
+```
+
+then run following command :
+
+```shell
+odoo-openupgrade-wizard pull-submodule
+```
+
+
+
+## Command: ``get-code``
+
+**Prerequites:** init
+
+```shell
+odoo-openupgrade-wizard get-code
+```
+
+This command will simply get all the Odoo code required to run all the steps
+for your migration with the ``gitaggregate`` tools.
+
+The code is defined in the ``repos.yml`` of each environment folders. (or in the
+directory ``repo_submodule`` if you use ``pull-submodule`` feature.)
+
+**Note**
+
+This step could take a big while !
+
+**Optional arguments**
+
+if you want to update the code of some given versions, you can provide an extra parameter:
+
+```shell
+odoo-openupgrade-wizard get-code --versions 10.0,11.0
+```
+
+
+
+## Command: ``docker-build``
+
+**Prerequites:** init + get-code
+
+This will build local docker images that will be used in the following steps.
+
+At this end of this step executing the following command should show a docker image per version.
+
+
+```shell
+docker images --filter "reference=odoo-openupgrade-wizard-*"
+```
+```
+REPOSITORY TAG IMAGE ID CREATED SIZE
+odoo-openupgrade-wizard-image---my-customer-10-12---12.0 latest ef664c366208 2 weeks ago 1.39GB
+odoo-openupgrade-wizard-image---my-customer-10-12---11.0 latest 24e283fe4ae4 2 weeks ago 1.16GB
+odoo-openupgrade-wizard-image---my-customer-10-12---10.0 latest 9d94dce2bd4e 2 weeks ago 924MB
+```
+
+**Optional arguments**
+
+- if you want to (re)build an image for some given versions, you can provide
+ an extra parameter: ``--versions 10.0,12.0``
+
+**Note**
+
+- This step could take a big while also !
+
+
+
+
+## Command: ``run``
+
+**Prerequites:** init + get-code + build
+
+```shell
+odoo-openupgrade-wizard run\
+ --step 1\
+ --database DB_NAME
+```
+
+Run an Odoo instance with the environment defined by the step argument.
+
+The database will be created, if it doesn't exists.
+
+if ``stop-after-init`` is disabled, the odoo instance will be available
+at your host, at the following url : http://localhost:9069
+(Port depends on your ``host_odoo_xmlrpc_port`` setting of your ``config.yml`` file)
+
+**Optional arguments**
+
+- You can add ``--init-modules=purchase,sale`` to install modules.
+
+- You can add ``stop-after-init`` flag to turn off the process at the end
+ of the installation.
+
+
+
+
+## Command: ``install-from-csv``
+
+**Prerequites:** init + get-code + build
+
+```shell
+odoo-openupgrade-wizard install-from-csv\
+ --database DB_NAME
+```
+
+Install the list of the modules defined in your ``modules.csv`` files on the
+given database.
+
+The database will be created, if it doesn't exists.
+
+To get a correct ``modules.csv`` file, the following query can be used:
+```shell
+psql -c "copy (select name, shortdesc from ir_module_module where state = 'installed' order by 1) to stdout csv" coopiteasy
+```
+
+
+
+
+## Command: ``upgrade``
+
+**Prerequites:** init + get-code + build
+
+```shell
+odoo-openupgrade-wizard upgrade\
+ --database DB_NAME
+```
+
+Realize an upgrade of the database from the initial version to
+the final version, following the different steps.
+
+For each step, it will :
+
+1. Execute the ``pre-migration.sql`` of the step.
+2. Realize an "update all" (in an upgrade or update context)
+3. Execute the scripts via XML-RPC (via ``odoorpc``) defined in
+ the ``post-migration.py`` file.
+
+**Optional arguments**
+
+- You can add ``--first-step=2`` to start at the second step.
+
+- You can add ``--last-step=3`` to end at the third step.
+
+
+
+
+## Command: ``generate-module-analysis``
+
+**Prerequites:** init + get-code + build
+
+```shell
+odoo-openupgrade-wizard generate-module-analysis\
+ --database DB_NAME
+ --step 2
+ --modules MODULE_LIST
+```
+
+Realize an analyze between the target version (in parameter via the step argument)
+and the previous version. It will generate analysis_file.txt files present
+in OpenUpgrade project.
+You can also use this fonction to analyze differences for custom / OCA modules
+between several versions, in case of refactoring.
+
+
+
+## Command: ``estimate-workload``
+
+**Prerequites:** init + get-code
+
+```shell
+odoo-openupgrade-wizard estimate-workload
+```
+
+Generate an HTML file name ``analysis.html`` with all the information regarding
+the work to do for the migration.
+- checks that the modules are present in each version. (by managing the
+ renaming or merging of modules)
+- check that the analysis and migration have been done for the official
+ modules present in odoo/odoo
+
+
+
+## Command: ``psql``
+
+**Prerequites:** init
+
+```shell
+odoo-openupgrade-wizard psql
+ --database DB_NAME
+ --command "SQL_REQUEST"
+```
+
+Execute an SQL Request on the target database.
+
+**Optional arguments**
+
+- If no ``database`` is provided, default ``postgres`` database will be used. exemple:
+
+```shell
+odoo-openupgrade-wizard psql --command "\l";
+```
+Result:
+```
+ List of databases
+ Name | Owner | Encoding | Collate | Ctype | Access privileges
+------------+-------+----------+------------+------------+-------------------
+ postgres | odoo | UTF8 | en_US.utf8 | en_US.utf8 |
+ template0 | odoo | UTF8 | en_US.utf8 | en_US.utf8 | =c/odoo +
+ | | | | | odoo=CTc/odoo
+ template1 | odoo | UTF8 | en_US.utf8 | en_US.utf8 | =c/odoo +
+ | | | | | odoo=CTc/odoo
+ test_psql | odoo | UTF8 | en_US.utf8 | en_US.utf8 |
+
+```
+
+- if you execute request that return long result, you can choose to select ``pager`` or ``-no-pager``
+ option to display the result via the click function ``echo_via_pager``.
+ (see : https://click.palletsprojects.com/en/8.1.x/utils/#pager-support)
+
+Note : Pager is enabled by default.
+
+
+- you can pass extra psql arguments inline.
+
+```shell
+odoo-openupgrade-wizard psql
+ --database=test_psql
+ --command "select id, name from res_partner where name ilike '%admin%';"
+ -H
+```
+Result:
+```html
+
+
+ | id |
+ name |
+
+
+ | 3 |
+ Administrator |
+
+
+(1 row)
+
+
+```
+
+See all the options here https://www.postgresql.org/docs/current/app-psql.html
+
+
+
+## Command: ``copydb``
+
+**Prerequites:** init
+
+```shell
+odoo-openupgrade-wizard copydb
+ --source DB_NAME
+ --dest NEW_DB_NAME
+```
+
+Create an Odoo database by copying an existing one.
+
+This script copies using postgres CREATEDB WITH TEMPLATE. It also copies
+the filestore.
+
+
+
+## Command: ``dropdb``
+
+**Prerequites:** init
+
+```shell
+odoo-openupgrade-wizard dropdb
+ --database DB_NAME
+```
+
+Delete an Odoo database and its filestore.
+
+This command will always success even if DB_NAME does not exists.
+
+
+
+## Command: ``dumpdb``
+
+**Prerequites:** init
+
+```shell
+odoo-openupgrade-wizard dumpdb
+ --database DB_NAME
+ --database-path DATABASE_PATH
+ --filestore-path FILESTORE_PATH
+```
+
+Dump the database DB_NAME to DATABASE_PATH and export the filestore
+related to DB_NAME into FILESTORE_PATH. To choose the format of the
+backup files look at the `--database-format` and `--filestore-format`.
+
+*WARNING*: DATABASE_PATH should be a sub directory of the project path
+in orter to have the postgresql container able to write the dump file.
+For example, the project path is `/path/to/myproject` (where you run the
+`init` command), then DATABASE_PATH can be any of the subdirectory of
+`/path/to/myproject`.
+
+**Optional arguments**
+
+- To chose the database format use `--database-format`. Format can be
+ one of the following:
+ - `p` for plain sql text
+ - `c` for custom compressed backup of `pg_dump`
+ - `d` for directory structure
+ - `t` for a tar version of the directory structure
+ See also https://www.postgresql.org/docs/current/app-pgdump.html
+ The default database format is `c`.
+
+- To chose the filestore format use `--filestore-format`. Format can be
+ one of the following:
+ - `d` copy of the directory structure
+ - `t` tar version of the directory structure (not compressed)
+ - `tgz` tar version of the directory structure compressed with gzip.
+ The default filestore format is `tgz`.
+
+- By default, if database file or filestore file already exists, the
+ command will fail, preserving the existing dump. If you need to
+ overwrite the existing files, the `--force` option can be used.
diff --git a/README.md b/README.md
index e3cb191..9739a4d 100644
--- a/README.md
+++ b/README.md
@@ -1,3 +1,5 @@
+
+
[](https://gitlab.com/odoo-openupgrade-wizard/odoo-openupgrade-wizard/-/pipelines)
[](https://gitlab.com/odoo-openupgrade-wizard/odoo-openupgrade-wizard/)
[](https://www.gnu.org/licenses/agpl-3.0)
@@ -11,9 +13,10 @@
Odoo Openupgrade Wizard is a tool that helps developpers to make major
upgrade of Odoo Community Edition. (formely OpenERP).
-It works with Openupgrade OCA tools. (https://github.com/oca/openupgrade)
+It works with Openupgrade OCA tools. ()
this tool is useful for complex migrations:
+
- migrate several versions
- take advantage of the migration to install / uninstall modules
- execute sql requests or odoo shell scripts between each migration
@@ -22,37 +25,45 @@ this tool is useful for complex migrations:
It will create a migration environment (with all the code available)
and provides helpers to run (and replay) migrations until it works.
-* To develop and contribute to the library, refer to the ``DEVELOP.md`` file.
-* Refer to the ``ROADMAP.md`` file to see the current limitation, bugs, and task to do.
-* See authors in the ``CONTRIBUTORS.md`` file.
+- To develop and contribute to the library, refer to the
+ [DEVELOP.md](https://gitlab.com/odoo-openupgrade-wizard/odoo-openupgrade-wizard/-/blob/main/DEVELOP.md)
+ file.
+- Refer to the
+ [Gitlab Issue Page](https://gitlab.com/odoo-openupgrade-wizard/odoo-openupgrade-wizard/-/issues)
+ to see the current limitation, bugs, and task to do.
+- See authors in the
+ [CONTRIBUTORS.md](https://gitlab.com/odoo-openupgrade-wizard/odoo-openupgrade-wizard/-/blob/main/CONTRIBUTORS.md)
+ file.
+- See history in the
+ [CHANGES.rst](https://gitlab.com/odoo-openupgrade-wizard/odoo-openupgrade-wizard/-/blob/main/CHANGES.rst)
+ file.
-# Table of Contents
+## Table of Contents
-* [Installation](#installation)
-* [Usage](#usage)
- * [Command ``init``](#command-init)
- * [Command ``pull-submodule``](#command-pull-submodule)
- * [Command ``get-code``](#command-get-code)
- * [Command ``docker-build``](#command-docker-build)
- * [Command ``run``](#command-run)
- * [Command ``install-from-csv``](#command-install-from-csv)
- * [Command ``upgrade``](#command-upgrade)
- * [Command ``generate-module-analysis``](#command-generate-module-analysis)
- * [Command ``estimate-workload``](#command-estimate-workload)
- * [Command ``psql``](#command-psql)
- * [Command ``copydb``](#command-copydb)
- * [Command ``dropdb``](#command-dropdb)
- * [Command ``dumpdb``](#command-dumpdb)
+- [Installation](#installation)
+- [Usage](#usage)
+ - [Command ``init``](#command-init)
+ - [Command ``init``](#command-init)
+ - [Command ``pull-submodule``](#command-pull-submodule)
+ - [Command ``get-code``](#command-get-code)
+ - [Command ``docker-build``](#command-docker-build)
+ - [Command ``run``](#command-run)
+ - [Command ``install-from-csv``](#command-install-from-csv)
+ - [Command ``upgrade``](#command-upgrade)
+ - [Command ``generate-module-analysis``](#command-generate-module-analysis)
+ - [Command ``estimate-workload``](#command-estimate-workload)
+ - [Command ``psql``](#command-psql)
+ - [Command ``copydb``](#command-copydb)
+ - [Command ``dropdb``](#command-dropdb)
+ - [Command ``dumpdb``](#command-dumpdb)
-
-
-# Installation
+## Installation
**Prerequites:**
-* The tools run on debian system
-* You should have docker installed on your system
-* Some features require extra packages. To have all the features available run:
+- The tools run on debian system
+- You should have docker installed on your system
+- Some features require extra packages. To have all the features available run:
**Installation:**
@@ -60,459 +71,10 @@ The library is available on [PyPI](https://pypi.org/project/odoo-openupgrade-wiz
To install it simply run :
-``pipx install odoo-openupgrade-wizard``
-
-(See alternative installation in ``DEVELOP.md`` file.)
-
-
-
-# Usage
-
-**Note:**
-
-the term ``odoo-openupgrade-wizard`` can be replaced by ``oow``
-in all the command lines below.
-
-
-
-## Command: ``init``
-
```shell
-odoo-openupgrade-wizard init\
- --initial-version=10.0\
- --final-version=12.0\
- --project-name=my-customer-10-12\
- --extra-repository=OCA/web,OCA/server-tools
+pipx install odoo-openupgrade-wizard
```
-Initialize a folder to make a migration from a 10.0 and a 12.0 database.
-This will generate the following structure :
-
-```
-filestore/
-log/
- 2022_03_25__23_12_41__init.log
- ...
-postgres_data/
-scripts/
- step_1__update__10.0/
- pre-migration.sql
- post-migration.py
- step_2__upgrade__11.0/
- ...
- step_3__upgrade__12.0/
- ...
- step_4__update__12.0/
- ...
-src/
- env_10.0/
- extra_debian_requirements.txt
- Dockerfile
- odoo.conf
- extra_python_requirements.txt
- repos.yml
- src/
- env_11.0/
- ...
- env_12.0/
- ...
-config.yml
-modules.csv
-```
-
-* ``config.yml`` is the main configuration file of your project.
-
-* ``modules.csv`` file is an optional file. You can fill it with the list
- of your modules installed on your production. The first column of this
- file should contain the technical name of the module.
-
-* ``log`` folder will contains all the log of the ``odoo-openupgrade-wizard``
- and the logs of the odoo instance that will be executed.
-
-* ``filestore`` folder will contains the filestore of the odoo database(s)
-
-* ``postgres_data`` folder will be used by postgres docker image to store
- database.
-
-* ``scripts`` folder contains a folder per migration step. In each step folder:
- - ``pre-migration.sql`` can contains extra SQL queries you want to execute
- before beginning the step.
- - ``post-migration.py`` can contains extra python command to execute
- after the execution of the step.
- Script will be executed with ``odoo shell`` command. All the ORM is available
- via the ``env`` variable.
-
-* ``src`` folder contains a folder per Odoo version. In each environment folder:
-
- - ``repos.yml`` enumerates the list of the repositories to use to run the odoo instance.
- The syntax should respect the ``gitaggregate`` command.
- (See : https://pypi.org/project/git-aggregator/)
- Repo files are pre-generated. You can update them with your custom settings.
- (custom branches, extra PRs, git shallow options, etc...)
-
- - ``extra_python_requirements.txt`` enumerates the list of extra python librairies
- required to run the odoo instance.
- The syntax should respect the ``pip install -r`` command.
- (See : https://pip.pypa.io/en/stable/reference/requirements-file-format/)
-
- - ``extra_debian_requirements.txt`` enumerates the list of extra system librairies
- required to run the odoo instance.
-
- - ``odoo.conf`` file. Add here extra configuration required for your custom modules.
- the classical keys (``db_host``, ``db_port``, etc...) are automatically
- autogenerated.
-
-At this step, you should change the autogenerated files.
-You can use default files, if you have a very simple odoo instance without custom code,
-extra repositories, or dependencies...
-
-**Note:**
-
-- In your repos.yml, preserve ``openupgrade`` and ``server-tools`` repositories
- to have all the features of the librairies available.
-- In your repos.yml file, the odoo project should be in ``./src/odoo``
- and the openupgrade project should be in ``./src/openupgrade/`` folder.
-
-
-
-## Command: ``pull-submodule``
-
-**Prerequites:** init + being in a git repository. (if not, you can simply run ``git init``)
-
-if you already have a repos.yml file on github / gitlab, it can be convenient to
-synchronize the repository, instead of copy past the ``repos.yml`` manually.
-
-In that case, you can add extra values, in the ``config.yml`` file in the section
-
-```yaml
-odoo_version_settings:
- 12.0:
- repo_url: url_of_the_repo_that_contains_a_repos_yml_file
- repo_branch: 12.0
- repo_file_path: repos.yml
-```
-
-then run following command :
-
-```shell
-odoo-openupgrade-wizard pull-submodule
-```
-
-
-
-## Command: ``get-code``
-
-**Prerequites:** init
-
-```shell
-odoo-openupgrade-wizard get-code
-```
-
-This command will simply get all the Odoo code required to run all the steps
-for your migration with the ``gitaggregate`` tools.
-
-The code is defined in the ``repos.yml`` of each environment folders. (or in the
-directory ``repo_submodule`` if you use ``pull-submodule`` feature.)
-
-**Note**
-
-* This step could take a big while !
-
-**Optional arguments**
-
-if you want to update the code of some given versions, you can provide an extra parameter:
-
-```shell
-odoo-openupgrade-wizard get-code --versions 10.0,11.0
-```
-
-
-
-## Command: ``docker-build``
-
-**Prerequites:** init + get-code
-
-This will build local docker images that will be used in the following steps.
-
-At this end of this step executing the following command should show a docker image per version.
-
-
-```shell
-docker images --filter "reference=odoo-openupgrade-wizard-*"
-```
-```
-REPOSITORY TAG IMAGE ID CREATED SIZE
-odoo-openupgrade-wizard-image---my-customer-10-12---12.0 latest ef664c366208 2 weeks ago 1.39GB
-odoo-openupgrade-wizard-image---my-customer-10-12---11.0 latest 24e283fe4ae4 2 weeks ago 1.16GB
-odoo-openupgrade-wizard-image---my-customer-10-12---10.0 latest 9d94dce2bd4e 2 weeks ago 924MB
-```
-
-**Optional arguments**
-
-* if you want to (re)build an image for some given versions, you can provide
- an extra parameter: ``--versions 10.0,12.0``
-
-**Note**
-
-* This step could take a big while also !
-
-
-
-
-## Command: ``run``
-
-**Prerequites:** init + get-code + build
-
-```shell
-odoo-openupgrade-wizard run\
- --step 1\
- --database DB_NAME
-```
-
-Run an Odoo instance with the environment defined by the step argument.
-
-The database will be created, if it doesn't exists.
-
-if ``stop-after-init`` is disabled, the odoo instance will be available
-at your host, at the following url : http://localhost:9069
-(Port depends on your ``host_odoo_xmlrpc_port`` setting of your ``config.yml`` file)
-
-**Optional arguments**
-
-* You can add ``--init-modules=purchase,sale`` to install modules.
-
-* You can add ``stop-after-init`` flag to turn off the process at the end
- of the installation.
-
-
-
-
-## Command: ``install-from-csv``
-
-**Prerequites:** init + get-code + build
-
-```shell
-odoo-openupgrade-wizard install-from-csv\
- --database DB_NAME
-```
-
-Install the list of the modules defined in your ``modules.csv`` files on the
-given database.
-
-The database will be created, if it doesn't exists.
-
-To get a correct ``modules.csv`` file, the following query can be used:
-```shell
-psql -c "copy (select name, shortdesc from ir_module_module where state = 'installed' order by 1) to stdout csv" coopiteasy
-```
-
-
-
-
-## Command: ``upgrade``
-
-**Prerequites:** init + get-code + build
-
-```shell
-odoo-openupgrade-wizard upgrade\
- --database DB_NAME
-```
-
-Realize an upgrade of the database from the initial version to
-the final version, following the different steps.
-
-For each step, it will :
-
-1. Execute the ``pre-migration.sql`` of the step.
-2. Realize an "update all" (in an upgrade or update context)
-3. Execute the scripts via XML-RPC (via ``odoorpc``) defined in
- the ``post-migration.py`` file.
-
-**Optional arguments**
-
-* You can add ``--first-step=2`` to start at the second step.
-
-* You can add ``--last-step=3`` to end at the third step.
-
-
-
-
-## Command: ``generate-module-analysis``
-
-**Prerequites:** init + get-code + build
-
-```shell
-odoo-openupgrade-wizard generate-module-analysis\
- --database DB_NAME
- --step 2
- --modules MODULE_LIST
-```
-
-Realize an analyze between the target version (in parameter via the step argument)
-and the previous version. It will generate analysis_file.txt files present
-in OpenUpgrade project.
-You can also use this fonction to analyze differences for custom / OCA modules
-between several versions, in case of refactoring.
-
-
-
-## Command: ``estimate-workload``
-
-**Prerequites:** init + get-code
-
-```shell
-odoo-openupgrade-wizard estimate-workload
-```
-
-Generate an HTML file name ``analysis.html`` with all the information regarding
-the work to do for the migration.
-- checks that the modules are present in each version. (by managing the
- renaming or merging of modules)
-- check that the analysis and migration have been done for the official
- modules present in odoo/odoo
-
-
-
-## Command: ``psql``
-
-**Prerequites:** init
-
-```shell
-odoo-openupgrade-wizard psql
- --database DB_NAME
- --command "SQL_REQUEST"
-```
-
-Execute an SQL Request on the target database.
-
-**Optional arguments**
-
-* If no ``database`` is provided, default ``postgres`` database will be used. exemple:
-
-```shell
-odoo-openupgrade-wizard psql --command "\l";
-```
-Result:
-```
- List of databases
- Name | Owner | Encoding | Collate | Ctype | Access privileges
-------------+-------+----------+------------+------------+-------------------
- postgres | odoo | UTF8 | en_US.utf8 | en_US.utf8 |
- template0 | odoo | UTF8 | en_US.utf8 | en_US.utf8 | =c/odoo +
- | | | | | odoo=CTc/odoo
- template1 | odoo | UTF8 | en_US.utf8 | en_US.utf8 | =c/odoo +
- | | | | | odoo=CTc/odoo
- test_psql | odoo | UTF8 | en_US.utf8 | en_US.utf8 |
-
-```
-
-* if you execute request that return long result, you can choose to select ``pager`` or ``-no-pager``
- option to display the result via the click function ``echo_via_pager``.
- (see : https://click.palletsprojects.com/en/8.1.x/utils/#pager-support)
-
-Note : Pager is enabled by default.
-
-
-* you can pass extra psql arguments inline.
-
-```shell
-odoo-openupgrade-wizard psql
- --database=test_psql
- --command "select id, name from res_partner where name ilike '%admin%';"
- -H
-```
-Result:
-```html
-
-
- | id |
- name |
-
-
- | 3 |
- Administrator |
-
-
-(1 row)
-
-
-```
-
-See all the options here https://www.postgresql.org/docs/current/app-psql.html
-
-
-
-## Command: ``copydb``
-
-**Prerequites:** init
-
-```shell
-odoo-openupgrade-wizard copydb
- --source DB_NAME
- --dest NEW_DB_NAME
-```
-
-Create an Odoo database by copying an existing one.
-
-This script copies using postgres CREATEDB WITH TEMPLATE. It also copies
-the filestore.
-
-
-
-## Command: ``dropdb``
-
-**Prerequites:** init
-
-```shell
-odoo-openupgrade-wizard dropdb
- --database DB_NAME
-```
-
-Delete an Odoo database and its filestore.
-
-This command will always success even if DB_NAME does not exists.
-
-
-
-## Command: ``dumpdb``
-
-**Prerequites:** init
-
-```shell
-odoo-openupgrade-wizard dumpdb
- --database DB_NAME
- --database-path DATABASE_PATH
- --filestore-path FILESTORE_PATH
-```
-
-Dump the database DB_NAME to DATABASE_PATH and export the filestore
-related to DB_NAME into FILESTORE_PATH. To choose the format of the
-backup files look at the `--database-format` and `--filestore-format`.
-
-*WARNING*: DATABASE_PATH should be a sub directory of the project path
-in orter to have the postgresql container able to write the dump file.
-For example, the project path is `/path/to/myproject` (where you run the
-`init` command), then DATABASE_PATH can be any of the subdirectory of
-`/path/to/myproject`.
-
-**Optional arguments**
-
-* To chose the database format use `--database-format`. Format can be
- one of the following:
- - `p` for plain sql text
- - `c` for custom compressed backup of `pg_dump`
- - `d` for directory structure
- - `t` for a tar version of the directory structure
- See also https://www.postgresql.org/docs/current/app-pgdump.html
- The default database format is `c`.
-
-* To chose the filestore format use `--filestore-format`. Format can be
- one of the following:
- - `d` copy of the directory structure
- - `t` tar version of the directory structure (not compressed)
- - `tgz` tar version of the directory structure compressed with gzip.
- The default filestore format is `tgz`.
-
-* By default, if database file or filestore file already exists, the
- command will fail, preserving the existing dump. If you need to
- overwrite the existing files, the `--force` option can be used.
+See alternative installation in the
+[DEVELOP.md](https://gitlab.com/odoo-openupgrade-wizard/odoo-openupgrade-wizard/-/blob/main/DEVELOP.md)
+file.
diff --git a/ROADMAP.md b/ROADMAP.md
deleted file mode 100644
index 7e4bb00..0000000
--- a/ROADMAP.md
+++ /dev/null
@@ -1,45 +0,0 @@
-# Python Version
-
-* py310 is not available, due to dependencies to ``odoorpc`` that raise an error :
- ``ERROR tests/cli_A_init_test.py - AttributeError: module 'collections' has no attribute 'MutableMapping'``
- Follow bug : https://stackoverflow.com/questions/69512672/getting-attributeerror-module-collections-has-no-attribute-mutablemapping-w
-
-# openUpgradelib Versions
-
-* ``openupgradelib`` requires a new feature psycopg2.sql since
- (21 Aug 2019)
- https://github.com/OCA/openupgradelib/commit/7408580e4469ba4b0cabb923da7facd71567a2fb
- so we pin openupgradelib==2.0.0 (21 Jul 2018)
-
-The python version in the Odoo:12 docker image is : ``Python 3.5.3 (default, Apr 5 2021, 09:00:41)`` that is very old.
-
-
-- https://github.com/OCA/openupgradelib/issues/248
-- https://github.com/OCA/openupgradelib/issues/288
-- https://github.com/OCA/openupgradelib.git@ed01555b8ae20f66b3af178c8ecaf6edd110ce75#egg=openupgradelib
-
-TODO : Fix via another way (other way than pining ``openuppgradelib`` version) the problem of old odoo versions. (it makes the upgrade failing for old revision (V8, etc...))
-
-# Gitlab-CI
-
-* for the time being, Features requiring ``odoorpc`` are failing in gitlab-CI.
- Tests are working locally but there is a network problem. For that reason, tests witch names
- begins by ``cli_2`` like (``cli_20_install_from_csv_test.py``) are disabled in ``.gitlab-ci.yml``.
-
-TODO : work with Pierrick Brun, to run gitlab-runner on Akretion CI (without docker), to see if it is
-fixing the problem.
-
-# Features Work In Progress
-
-* Add a tools to analyze workload.
-
-# Possible Improvments
-
-* select ``without-demo all`` depending on if the database
- is created or not (and if current database contains demo data, checking if base.user_demo exists ?)
-
-# Other points not in the scope of GRAP work
-
-* Allow to use custom docker images.
-
-* Check if there are default values for containers, limiting ressources.