PHP & MySQL | Tutorials | Development

David Carr

Gitlab download private repository as tar.gz with php

Gitlab is a great alternative to Github or BitBucket, they offer free private and public repositories which is great when working with a team, I’ve recently started using Gitlab for a private project, where I need to download private repositories to the project server and extract their contents.

Github offers a great API http://doc.gitlab.com/ee/api/

To get started with their API you need a private token. Your token can be found inside your account page https://gitlab.com/profile/account

All API requests require authentication. You need to pass a private_token parameter via query string or header. If passed as a header, the header name must be PRIVATE-TOKEN (uppercase and with a dash instead of an underscore). You can find or reset your private token in your account page (/profile/account).

To use the API the token must be passed in all requests, for get requests its as simple as ?private_token=yourtoken

In order to download the repository it’s id is needed. A list of repositories can be seen by going to 

https://gitlab.com/api/v3/projects?private_token=yourtoken 

Other options can be passed such as looking at private repositories only and ordering them by name: 

https://gitlab.com/api/v3/projects?private_token=yourtoken&visibility=private&order_by=name 

This will give you a list of projects:

[
  {
    "id": 4,
    "description": null,
    "default_branch": "master",
    "public": false,
    "visibility_level": 0,
    "ssh_url_to_repo": "git@example.com:diaspora/diaspora-client.git",
    "http_url_to_repo": "http://example.com/diaspora/diaspora-client.git",
    "web_url": "http://example.com/diaspora/diaspora-client",
    "tag_list": [
      "example",
      "disapora client"
    ],
    "owner": {
      "id": 3,
      "name": "Diaspora",
      "created_at": "2013-09-30T13: 46: 02Z"
    },
    "name": "Diaspora Client",
    "name_with_namespace": "Diaspora / Diaspora Client",
    "path": "diaspora-client",
    "path_with_namespace": "diaspora/diaspora-client",
    "issues_enabled": true,
    "open_issues_count": 1,
    "merge_requests_enabled": true,
    "builds_enabled": true,
    "wiki_enabled": true,
    "snippets_enabled": false,
    "created_at": "2013-09-30T13: 46: 02Z",
    "last_activity_at": "2013-09-30T13: 46: 02Z",
    "creator_id": 3,
    "namespace": {
      "created_at": "2013-09-30T13: 46: 02Z",
      "description": "",
      "id": 3,
      "name": "Diaspora",
      "owner_id": 1,
      "path": "diaspora",
      "updated_at": "2013-09-30T13: 46: 02Z"
    },
    "archived": false,
    "avatar_url": "http://example.com/uploads/project/avatar/4/uploads/avatar.png",
    "shared_runners_enabled": true,
    "forks_count": 0,
    "star_count": 0,
    "runners_token": "b8547b1dc37721d05889db52fa2f02",
    "public_builds": true
  },
  {
    "id": 6,
    "description": null,
    "default_branch": "master",
    "public": false,
    "visibility_level": 0,
    "ssh_url_to_repo": "git@example.com:brightbox/puppet.git",
    "http_url_to_repo": "http://example.com/brightbox/puppet.git",
    "web_url": "http://example.com/brightbox/puppet",
    "tag_list": [
      "example",
      "puppet"
    ],
    "owner": {
      "id": 4,
      "name": "Brightbox",
      "created_at": "2013-09-30T13:46:02Z"
    },
    "name": "Puppet",
    "name_with_namespace": "Brightbox / Puppet",
    "path": "puppet",
    "path_with_namespace": "brightbox/puppet",
    "issues_enabled": true,
    "open_issues_count": 1,
    "merge_requests_enabled": true,
    "builds_enabled": true,
    "wiki_enabled": true,
    "snippets_enabled": false,
    "created_at": "2013-09-30T13:46:02Z",
    "last_activity_at": "2013-09-30T13:46:02Z",
    "creator_id": 3,
    "namespace": {
      "created_at": "2013-09-30T13:46:02Z",
      "description": "",
      "id": 4,
      "name": "Brightbox",
      "owner_id": 1,
      "path": "brightbox",
      "updated_at": "2013-09-30T13:46:02Z"
    },
    "permissions": {
      "project_access": {
        "access_level": 10,
        "notification_level": 3
      },
      "group_access": {
        "access_level": 50,
        "notification_level": 3
      }
    },
    "archived": false,
    "avatar_url": null,
    "shared_runners_enabled": true,
    "forks_count": 0,
    "star_count": 0,
    "runners_token": "b8547b1dc37721d05889db52fa2f02",
    "public_builds": true
  }
]

The id is what’s important here so make a note of the id for the project that will be used.

To then download the repository pass the id to /projects/:id so /projects/6 followed by your token:

https://gitlab.com/api/v3/projects/6/repository/archive?private_token=yourtoken

Now to use this in a php script:

Set a name for the local tar.gz file to be used, I use temp as it will be deleted once finished.

Next use file_get_contents to download the file from the api and save it to $name notice the .tar.gz on the end.

//set file name
$name = 'temp';

//download contents file the above filename with .tar.gz appended. 
//Remember to replace your id and token
file_put_contents($name.'.tar.gz', fopen("https://gitlab.com/api/v3/projects/id/repository/archive?private_token=your_token_here", 'r'));

Next unpack the .tar.gz file this creates a .tar file so unpack that too.

// decompress from gz
$p = new \PharData($zipname.'.tar.gz');
$p->decompress(); // creates files.tar

// unarchive from the tar
$phar = new \PharData($zipname.'.tar');
$phar->extractTo('folder/to/extract/to/', null, true); // extract all files, and overwrite

The packed files are no longer needed to delete them.

//delete temp.tar.gz
if (file_exists($zipname.'.tar.gz')) {
    unlink($zipname.'.tar.gz');
}

//delete temp.tar
if (file_exists($zipname.'.tar')) {
    unlink($zipname.'.tar');
}

This leaves a folder with a name similar to this: projectname-master- followed by a long string, to have the folder only containing the fodler name ie projectname:

$dirs = glob('folder/files/extracted/to/*');
//loop through them
foreach ($dirs as $dir) {
    //if is a directory
    if (is_dir($dir)) {
        //explode the name where there is a -
        $parts = explode('-', $dir);
        //if the name does not exist ie there is no projectname only projectname-master-3r3rr4
        if (!file_exists($parts[0])) {
            //rename to lose everything after and including the -
            rename($dir, $parts[0]);
        }
    }
}

Putting it all together:

//set file name
$name = 'temp';

//download contents file the above filename with .tar.gz appended. 
//Remember to replace your id and token
file_put_contents($name.'.tar.gz', fopen("https://gitlab.com/api/v3/projects/id/repository/archive?private_token=your_token_here", 'r'));

// decompress from gz
$p = new \PharData($zipname.'.tar.gz');
$p->decompress(); // creates files.tar

// unarchive from the tar
$phar = new \PharData($zipname.'.tar');
$phar->extractTo('folder/to/extract/to/', null, true); // extract all files, and overwrite

//delete temp.tar.gz
if (file_exists($zipname.'.tar.gz')) {
    unlink($zipname.'.tar.gz');
}

//delete temp.tar
if (file_exists($zipname.'.tar')) {
    unlink($zipname.'.tar');
}

$dirs = glob('folder/files/extracted/to/*');
//loop through them
foreach ($dirs as $dir) {
    //if is a directory
    if (is_dir($dir)) {
        //explode the name where there is a -
        $parts = explode('-', $dir);
        //if the name does not exist ie there is no projectname only projectname-master-3r3rr4
        if (!file_exists($parts[0])) {
            //rename to lose everything after and including the -
            rename($dir, $parts[0]);
        }
    }
}

 

Domains are often purchased from multiple providers, keeping track of where a domain is and its DNS settings can be tricky. Domain Mapper solves this by listing all your domains in one place. View your DNS settings and receive reminders to renew your domains. Try it today.

Support my work by donating with PayPal.

Subscribe to my newsletter

Subscribe and get my books and product announcements.

© 2009 - 2021 DC Blog. All code MIT license. All rights reserved.