Hello!
Last week I tried out TDE building scripts: for example 99_build_TDE.sh which successfully finished its work in 38 hours. But later that week I looked into 0_logs/build_result.log and found there some "[FAILED]" messages for ten packages ("k3b-i18n", "k9copy", "kaffeine", "kasablanca", "katapult", "kbarcode", "kbfx", "kbibtex", "kbiff", "kpilot"). Later I tried to run the same script in "local mode" ("-l" flag in BUILD_DEFAULT_OPTIONS variable of _config.sh) but it seems the modules don't see the compilation results of the previously compiled modules. And I didn't find any way in the scripts for them to see. So here are some questions about building scripts:
0) Do I understand correctly that "modules" (as this word is used in command "build_module" for example in 06_applications_03.sh script) are corresponding 1-to-1 to debian source packages on the one hand and to git submodules of TDE/tde repository on the other hand?
1) What exactly is the mechanism used while building with "pbuilder" (i.e. WITHOUT "-l") for the module A to use package B built in one of the previous modules?
2) Does something similar exist while building everything in "local mode" (i.e. WITH "-l") on a clean machine? I see there only the *.deb files, but they don't seem to be installed immediately after creation and then the next step is about to start already and *.deb files are just left there not installed.
3) Is there any simple way to disable colored output which is very nice on the terminal, but quite disturbing while reading logs in a file?
4) Is there a way to stop let's say 99_build_TDE.sh script if a single module compilation failed?
5) Sometimes I see the following lines in the log file (using "show log" -sl option and piping output into a file): dpkg-checkbuilddeps: error: Unmet build dependencies: tdelibs14-trinity-dev [and many more -I.D.] dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting dpkg-buildpackage: warning: (Use -d flag to override.) Building completed SUCCESSFULLY [in green color -I.D.]
Does this combination (package can't be built, but it's kinda success nevertheless) have any sense or is it some kind of error in the scripts?
The meaning of all these questions for me is to understand how to create an environment, where I can rebuild the whole TDE in a reliable way (run a single command and be sure that its success indicates nothing went wrong and everything is built) on a clean machine.
Thanks!
-- Ilya D.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
Hi Ilya, see comments below mixed with your email. You are building R14.1.0-DEV right?
Last week I tried out TDE building scripts: for example 99_build_TDE.sh which successfully finished its work in 38 hours. But later that week I looked into 0_logs/build_result.log and found there some "[FAILED]" messages for ten packages ("k3b-i18n", "k9copy", "kaffeine", "kasablanca", "katapult", "kbarcode", "kbfx", "kbibtex", "kbiff", "kpilot").
Under TDE_SRC/2_build/debs/<module>/src you will find a detailed build log of the module, so you can see what went wrong for that specific module
Later I tried to run the same script in "local
mode" ("-l" flag in BUILD_DEFAULT_OPTIONS variable of _config.sh) but it seems the modules don't see the compilation results of the previously compiled modules. And I didn't find any way in the scripts for them to see. So here are some questions about building scripts
- -l is used to build a module locally on your live system. It is mostly useful if you are testing changes and want to do quick local builds. It is not recommended to use it to build a clean TDE since it depends on locally install modules and build may fails if you don't update step by step
- Do I understand correctly that "modules" (as this word is used in command "build_module" for example in
06_applications_03.sh script) are corresponding 1-to-1 to debian source packages on the one hand and to git submodules of TDE/tde repository on the other hand?
Yes, that is correct.
- What exactly is the mechanism used while building with "pbuilder" (i.e. WITHOUT "-l") for the module A to use
package B built in one of the previous modules?
This is done by pbuilder as long as everything has been setup correctly. A pbuilder hook script is used to scan the "debs" folder before building each module, specifically the D05deps script.
- Does something similar exist while building everything in "local mode" (i.e. WITH "-l") on a clean machine? I
see there only the *.deb files, but they don't seem to be installed immediately after creation and then the next step is about to start already and *.deb files are just left there not installed.
As said above this is not recommended and in some unlucky case you may not even be able to build TDE from within TDE. If you want to use -l you need to install the various .deb by yourself in your system before proceeding with the next package. "-l" does not create clean chrooted modules, so think of it as a development tool/commodity.
- Is there any simple way to disable colored output which is very nice on the terminal, but quite disturbing while
reading logs in a
I may be saying something wrong here going by memory, but I think I have already addressed this issue in my scripts. Am I wrong?
- Is there a way to stop let's say 99_build_TDE.sh script if a single module compilation failed?
the various build set have been thought as a way to automate building of multiple modules, so they normally continue in case of a build failure. You will need to edit the scripts and look at the result returned by build_module and stop in case of a fault. Perhaps an idea to consider for future improvements :-)
- Sometimes I see the following lines in the log file (using "show log" -sl option and piping output into a
file): dpkg-checkbuilddeps: error: Unmet build dependencies: tdelibs14-trinity-dev [and many more -I.D.] dpkg-buildpackage: warning: build dependencies/conflicts unsatisfied; aborting dpkg-buildpackage: warning: (Use -d flag to override.) Building completed SUCCESSFULLY [in green color -I.D.]
Does this combination (package can't be built, but it's kinda success nevertheless) have any sense or is it some kind of error in the scripts?
It is probably a corner case of the scripts when missing dependencies are detected. I have seen something like this from time to time as well, not often enough to bother me to the point to work on it yet :-D
The meaning of all these questions for me is to understand how to create an environment, where I can rebuild the whole TDE in a reliable way (run a single command and be sure that its success indicates nothing went wrong and everything is built) on a clean machine.
Don't use "-l" in this case. you can use the build TDE script and grep the build_result.log file for "FAILED". That will tell you if any package has failed the build process.
If you need further info, please let me know. Btw, I have not yet come around the meta package building. I am under very heavy work load and for a while have very limited time for TDE unfortunately :-(
Cheers Michele
Hello!
ср, 11 дек. 2019 г. в 20:22, deloptes deloptes@gmail.com:
just curios - are you reading logs with "vi"?
Yes, usually I do. The U+1B character (ESCAPE) looks very ugly itself but the whole colored line looks even worse: ^[[1;36m#### Processing module "dependencies/libr" ####^[[0m
There it is "syntax on" or "syntax off"
How does vim's syntax highlighting help here?
The only way I know is to read such files with "less -r". If you know how to force vim to interpret escape sequences --- please explain how to do it.
Cheers,
Ilya D.
Ilya Dogolazky wrote:
How does vim's syntax highlighting help here?
I agree this was, what in German is called "Voll daneben" :)
The only way I know is to read such files with "less -r". If you know how to force vim to interpret escape sequences --- please explain how to do it.
If you want to remove all escape codes, you could use:
:%s/<1b>[[0-9;]*m//g
https://vi.stackexchange.com/questions/485/can-vim-interpret-terminal-color-...
I personally do not use colored output, but the question is interesting. I also do not use Micheles scripts, because of pbuilder - it is too heavy for me. But I used various parts to setup my build process, which is continious work in progress :)
On 12/11/2019 01:24 AM, Ilya Dogolazky wrote:
Hello!
Last week I tried out TDE building scripts: for example 99_build_TDE.sh which successfully finished its work in 38 hours.
Ouch!!
Thank cannot be right. I used to build TDE (including koffice) in less than 7 hr. on a P4 2.8 GHz box. Now with a normal i7 on SSD you are looking at an hour. If you are running 38 hours on the build something is very, very wrong.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512
On 2019/12/12 07:09 AM, David C. Rankin wrote:
On 12/11/2019 01:24 AM, Ilya Dogolazky wrote:
Hello!
Last week I tried out TDE building scripts: for example 99_build_TDE.sh which successfully finished its work in 38 hours.
It depends on how you build. 38 hours can be quite normal for a total TDE build on debian in clean root environment. It depends on machine power and network connection when looking for packages. Keep in mind local build and clean chroot pbuilder builds are quite different!
Cheers Michele
On 12/11/2019 06:39 PM, Michele Calgaro via trinity-devel wrote:
It depends on how you build. 38 hours can be quite normal for a total TDE build on debian in clean root environment. It depends on machine power and network connection when looking for packages. Keep in mind local build and clean chroot pbuilder builds are quite different!
Cheers Michele
Thank you Michele,
I never knew there could be such a drastic difference in build time. I too would use a clean chroot build environment and start with the dependencies needed for qt3, tqtinterface, arts, dbus-tqt, dbus-1-tqt, qca-tls, etc... through koffice. I guess the Arch a deb build systems are just that different.
Question, distros like openSUSE are dropping python2 (completely). What will happen to packages like python-tqt and any others with a python2 dependency? I guess the only issue would be if a non-backwards compatible python feature was being used. I don't write in python so I don't have a feel for if it will even be a problem. Does TDE build against py3 only?
David C. Rankin wrote:
Ouch!!
Thank cannot be right. I used to build TDE (including koffice) in less than 7 hr. on a P4 2.8 GHz box. Now with a normal i7 on SSD you are looking at an hour. If you are running 38 hours on the build something is very, very wrong.
Perhaps the key here is the SSD, because for me it also takes ~15h - normal disks in RAID1 and I use a kind of light version of build scripts.