curl


[GitLab.com] Clone all repositories in your account

GitLab.com offers a public API that allows us to get information related to our accounts. One of the API calls available is the account projects call (http://gitlab.com/api/v3/projects).

This call will return a JSON object describing the projects available to your account.

To clone all of the projects available to you, you can use the following:

TOKEN="PASTE_YOUR_PRIVATE_TOKEN_HERE"; PREFIX="ssh_url_to_repo"; curl --header "PRIVATE-TOKEN: $TOKEN" http://gitlab.com/api/v3/projects | grep -o "\"$PREFIX\":[^ ,]\+" | awk -F ':' '{printf "ssh://"; for (i=2; i<NF; i++) printf $i "/"; print $NF}' | xargs -L1 git clone

The above code will bring the JSON object, filter out everything except for the “ssh_url_to_repo” member of each project and then it will use it to clone the project by fixing up the URL to be used by git.

To get the above code working: the GitLab API requires that you use a token that is related to your account instead of using your credentials to make the call to the API.

To get your private token, visit this page http://gitlab.com/profile/account , the private token is the random sequence of characters in the white box:

[GitLab.com] Private TokenYou need to copy that value in the place of the variable TOKEN in the above script.

In case you have a lot of projects (more than 10), the default call will only produce the results for the first 10 repositories only.

To list all available repositories you have two options:

  1.  Set the per_page query parameter to a value big enough to fetch all your projects information if they are less than 100. e.g http://gitlab.com/api/v3/projects?per_page=100
  2. Follow the link headers from the initial response to make all the next calls.

[GitLab.com] Get a list with the names of all repositories in your account

GitLab.com offers a public API that allows us to get information related to our accounts. One of the API calls available is the account projects call (http://gitlab.com/api/v3/projects).

This call will return a JSON object describing the projects available to your account.

To get a list of the names of the projects available to you, you can use the following:

TOKEN="PASTE_YOUR_PRIVATE_TOKEN_HERE"; PREFIX="ssh_url_to_repo"; curl --header "PRIVATE-TOKEN: $TOKEN" http://gitlab.com/api/v3/projects | grep -o "\"$PREFIX\":[^ ,]\+" | xargs -L1 basename | awk -F '.' '{print $1}'

The above code will bring the JSON object, filter out everything except for the “ssh_url_to_repo” member of each project and then it will print it out on screen.

 

To get the above code working: the GitLab API requires that you use a token that is related to your account instead of using your credentials to make the call to the API.

To get your private token, visit this page http://gitlab.com/profile/account , the private token is the random sequence of characters in the white box:

[GitLab.com] Private TokenYou need to copy that value in the place of the variable TOKEN in the above script.

 

In case you have a lot of projects (more than 10), the default call will only produce the results for the first 10 repositories only.

To list all available repositories you have two options:

  1.  Set the per_page query parameter to a value big enough to fetch all your projects information if they are less than 100. e.g http://gitlab.com/api/v3/projects?per_page=100
  2. Follow the link headers from the initial response to make all the next calls.

[BitBucket.org] Clone all repositories of your account 2

Clone all bitbucket projects

 BBA=MyUserName; curl --user ${BBA} https://api.bitbucket.org/2.0/repositories/${BBA} | grep -o '"ssh:[^ ,]\+' | xargs -L1 git clone 

The above curl call will connect to the server using your username and return the list of repositories that are available to your account.

Please note that you need to provide you username NOT your email.
If you make these calls using the email that was used to register the account, then the call will fail.

After the call succeeds, the results will be filtered and each repository will be cloned to the current folder.
In case your ssh key is locked via a password, each time a clone operation will start, you will be asked for the password.

Example:

BBA="bytefreaks"; curl --user ${BBA} https://api.bitbucket.org/2.0/repositories/${BBA} | grep -o '"ssh:[^ ,]\+' | xargs -L1 git clone
Enter host password for user 'bytefreaks':
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3834  100  3834    0     0   4414      0 --:--:-- --:--:-- --:--:--  4411
Cloning into 'bluetoothclicker'...
Warning: Permanently added the RSA host key for IP address '104.192.143.1' to the list of known hosts.
Enter passphrase for key '/home/bytefreaks/.ssh/BitBucket/id_rsa': 
warning: You appear to have cloned an empty repository.
Checking connectivity... done.
Cloning into 'watch'...
Enter passphrase for key '/home/bytefreaks/.ssh/BitBucket/id_rsa': 
warning: You appear to have cloned an empty repository.
Checking connectivity... done.

List all bitbucket projects

In case what you want is just to list your repositories, execute the following:

 curl --user ${BBA} https://api.bitbucket.org/2.0/repositories/${BBA} | grep -o '"ssh:[^ ,]\+' | xargs -L1 echo 

Usage instructions: set your username to the BBA variable and execute.

BBA="bytefreaks"; curl --user ${BBA} https://api.bitbucket.org/2.0/repositories/${BBA} | grep -o '"ssh:[^ ,]\+' | xargs -L1 echo
Enter host password for user 'bytefreaks':
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3834  100  3834    0     0   3543      0  0:00:01  0:00:01 --:--:--  3546
ssh://[email protected]/bytefreaks/bluetoothclicker.git
ssh://[email protected]/bytefreaks/watch.git