On Thursday 31 of May 2018 19:36:06 Slávek Banko wrote:

On Thursday 31 of May 2018 10:21:24 gregory guy wrote:

...

Hi there!

Sorry to step in the devs's room but I'm seeking information about git uses to clone the TDE repository.

Yesterday I applyed the method displayed in the wiki, namely:

• git clone https://mirror.git.trinitydesktop.org/cgit/tde
• cd tde
• git submodule init -- scripts
• git submodule update -- scripts
• ./scripts/switch_all_submodules_to_head_and_clean anonymous

After more than 6 hours of downloading (at an average of 70 kib/s) the connexion stopped and wouldn't restart properly. I don't think I can fetch the entire project history in those conditions, this would take days...I guess!

Is there a way to shorten the cloning in order to get the last code? I tryed with "--depth=1" but I was rather unsuccessfull on the matter.

Regards gregory

Hi Gregory,

this is strange behavior. VPS for hosting mirror.git.trinitydesktop.org is on a 300 Mbps bandwidth. Therefore, I do not suppose there should be a problem with the speed of this server.

Please, can you check what the urls for the submodules are in tde/.git/config? Are these urls correctly replaced to mirror.git.trinitydesktop.org or stay referenced to the scm.trinitydesktop.org (master server)?

Cheers

Ok, now I've tried to measure the first step with two different services on the same server:

$ time git clone https://mirror.git.trinitydesktop.org/cgit/tde Cloning into 'tde'...

real 0m59.683s user 0m27.600s sys 0m2.452s

$ time git clone https://mirror.git.trinitydesktop.org/gitea/TDE/tde Cloning into 'tde'... remote: Counting objects: 60857, done. remote: Compressing objects: 100% (22885/22885), done. remote: Total 60857 (delta 39752), reused 56119 (delta 36438) Receiving objects: 100% (60857/60857), 30.45 MiB | 23.63 MiB/s, done. Resolving deltas: 100% (39752/39752), done.

real 0m5.512s user 0m3.168s sys 0m0.720s

This was a repeated measurement - at the first measurement times were doubled. The second measurements apparently benefited from the cache on the server. However, it is clear that Gitea gives better results while providing standard output during cloning.

Subsequently, I tested cloning the entire repository from Gitea:

$ time ./scripts/switch_all_submodules_to_head_and_clean anonymous

real 22m6.332s user 10m34.756s sys 0m53.580s

Please also try using Gitea instead of the CGit interface. It looks like Gitea could be an advantage!

Cheers

Dne pá 1. června 2018 gregory guy napsal(a):

Le Thu, 31 May 2018 20:12:53 +0200,

Slávek Banko slavek.banko@axis.cz a écrit :

...

On Thursday 31 of May 2018 19:36:06 Slávek Banko wrote:

...

On Thursday 31 of May 2018 10:21:24 gregory guy wrote:

...

Hi there!

Sorry to step in the devs's room but I'm seeking information about git uses to clone the TDE repository.

Yesterday I applyed the method displayed in the wiki, namely:

• git clone https://mirror.git.trinitydesktop.org/cgit/tde
• cd tde
• git submodule init -- scripts
• git submodule update -- scripts
• ./scripts/switch_all_submodules_to_head_and_clean anonymous

After more than 6 hours of downloading (at an average of 70 kib/s) the connexion stopped and wouldn't restart properly. I don't think I can fetch the entire project history in those conditions, this would take days...I guess!

Is there a way to shorten the cloning in order to get the last code? I tryed with "--depth=1" but I was rather unsuccessfull on the matter.

Regards gregory

Hi Gregory,

this is strange behavior. VPS for hosting mirror.git.trinitydesktop.org is on a 300 Mbps bandwidth. Therefore, I do not suppose there should be a problem with the speed of this server.

Please, can you check what the urls for the submodules are in tde/.git/config? Are these urls correctly replaced to mirror.git.trinitydesktop.org or stay referenced to the scm.trinitydesktop.org (master server)?

Cheers

Ok, now I've tried to measure the first step with two different services on the same server:

$ time git clone https://mirror.git.trinitydesktop.org/cgit/tde Cloning into 'tde'...

real 0m59.683s user 0m27.600s sys 0m2.452s

$ time git clone https://mirror.git.trinitydesktop.org/gitea/TDE/tde Cloning into 'tde'... remote: Counting objects: 60857, done. remote: Compressing objects: 100% (22885/22885), done. remote: Total 60857 (delta 39752), reused 56119 (delta 36438) Receiving objects: 100% (60857/60857), 30.45 MiB | 23.63 MiB/s, done. Resolving deltas: 100% (39752/39752), done.

Please also try using Gitea instead of the CGit interface. It looks like Gitea could be an advantage!

Cheers

Oh my gosh, gitea is so much faster! Each submodule usually tops my broadband at 1.5MIB/s.

Slavek, thank you for the dodge and everyone for their input on the matter.

Regards. gregory

I thank both for performing tests and confirming the results. It seems like a good idea to set in the cgit configuration to display a cloning url referring to Gitea. What do you think?

Cheers

Dne pá 1. června 2018 Michele Calgaro napsal(a):

...

I thank both for performing tests and confirming the results. It seems like a good idea to set in the cgit configuration to display a cloning url referring to Gitea. What do you think?

Cheers

Sounds good to me. Cheers Michele

Done :)

Cheers

On Thursday 31 of May 2018 20:22:36 Mike Bird wrote:

On Thu May 31 2018 10:36:06 Slávek Banko wrote:

...

this is strange behavior. VPS for hosting mirror.git.trinitydesktop.org is on a 300 Mbps bandwidth. Therefore, I do not suppose there should be a problem with the speed of this server.

Hi Slávek,

I just ran git clone https://mirror.git.trinitydesktop.org/cgit/tde for a few minutes on three VPSs in data centers in Texas, California, and Washington State. In each case there was a high initial transfer rate around 35Mbps. After less than a minute transfer rate dropped drastically or even zeroed before bouncing around and finally settling near 200kbps.

When I tested transfers from Washington State to Texas I was able to sustain around 47Mbps. Transfers from the TDE UK mirror to Texas started around 170Mbps before bouncing around wildly but averaged 90Mbps over several minutes.

I doubt this is an artifact of disk caching. I suspect something is limiting mirror.git.trinitydesktop.org bandwidth despite the 300Mbps connection.

--Mike

Hi Mike,

I watched your activity on cgit during my test with Gitea :) You can try the same cloning using Gitea's url.

If you want to test line speed without affecting by the application server (either cgit or Gitea), you can download the iso for Slax with Trinity:

http://mirror.ppa.trinitydesktop.org/trinity-sb/cdimages/slax/

Cheers