Prepare to have more logic for cloning and updating git repositories in the script, see follow-up patches. The purpose of this patch series is to fix the rate limiting errors we are seeing from gitlab eclipse, that lead to aborts of our ttcn3 jobs:
[titan.ProtocolModules.ROSE] Updating URL to https://gitlab.eclipse.org/eclipse/titan/titan.ProtocolModules.ROSE remote: You have reached the limit of requests you can make to Eclipse GitLab. This could be caused by too many open tabs, which query the GitLab server in the background. Please close unused tabs, or put them to sleep so they don't issue requests needlessly. fatal: unable to access 'https://gitlab.eclipse.org/eclipse/titan/titan.ProtocolModules.M3UA/': The requested URL returned error: 429 make[1]: *** [Makefile:174: titan.ProtocolModules.M3UA/update] Error 128
Prepare to add a function that gets the proper git clone URL for gitlab repos in the next patch. In order to do this, we need to have the update url logic in update.sh first.
Use ".git" at the end of gitlab URLs to avoid the redirects, e.g.:
warning: redirecting to https://gitlab.eclipse.org/eclipse/titan/titan.ProtocolModules.M3UA.git/
This is not just a cosmetic improvement that gets rid of these warnings, but it actually has the effect that we make less requests to the gitlab eclipse server and are less likely to trigger the rate limiting. On my machine I do trigger it with "make deps" without this patch, and with this patch I don't.
Replace the previous logic of running the whole Makefile again on any failure, with wrapping all git clone and fetch commands in a new retry_with_backoff_time function that actually sleeps before retrying (a random amount of seconds to make less requests at once), and retries up to 5 times, each time with likely more sleep time.
With this change it is more likely to succeed and we have less confusing output as deps/Makefile will not run twice if it fails for any reason (e.g. a syntax error).
