Currently seeking new career opportunities in web development, particularly with Laravel, Hire Me

Gitlab download private repository as tar.gz with php

David Carr

Development Tutorials PHP & MySQL

Table of Contents

Gitlab is a great alternative to Github or BitBucket, they offer free private and public repositories which is great when working with a team, I’ve recently started using Gitlab for a private project, where I need to download private repositories to the project server and extract their contents.

Github offers a great API http://doc.gitlab.com/ee/api/

To get started with their API you need a private token. Your token can be found inside your account page https://gitlab.com/profile/account

All API requests require authentication. You need to pass a private_token parameter via query string or header. If passed as a header, the header name must be PRIVATE-TOKEN (uppercase and with a dash instead of an underscore). You can find or reset your private token in your account page (/profile/account).

To use the API the token must be passed in all requests, for get requests its as simple as ?private_token=yourtoken

In order to download the repository it’s id is needed. A list of repositories can be seen by going to 

https://gitlab.com/api/v3/projects?private_token=yourtoken 

Other options can be passed such as looking at private repositories only and ordering them by name: 

https://gitlab.com/api/v3/projects?private_token=yourtoken&visibility=private&order_by=name 

This will give you a list of projects:

[
  {
    "id": 4,
    "description": null,
    "default_branch": "master",
    "public": false,
    "visibility_level": 0,
    "ssh_url_to_repo": "git@example.com:diaspora/diaspora-client.git",
    "http_url_to_repo": "http://example.com/diaspora/diaspora-client.git",
    "web_url": "http://example.com/diaspora/diaspora-client",
    "tag_list": [
      "example",
      "disapora client"
    ],
    "owner": {
      "id": 3,
      "name": "Diaspora",
      "created_at": "2013-09-30T13: 46: 02Z"
    },
    "name": "Diaspora Client",
    "name_with_namespace": "Diaspora / Diaspora Client",
    "path": "diaspora-client",
    "path_with_namespace": "diaspora/diaspora-client",
    "issues_enabled": true,
    "open_issues_count": 1,
    "merge_requests_enabled": true,
    "builds_enabled": true,
    "wiki_enabled": true,
    "snippets_enabled": false,
    "created_at": "2013-09-30T13: 46: 02Z",
    "last_activity_at": "2013-09-30T13: 46: 02Z",
    "creator_id": 3,
    "namespace": {
      "created_at": "2013-09-30T13: 46: 02Z",
      "description": "",
      "id": 3,
      "name": "Diaspora",
      "owner_id": 1,
      "path": "diaspora",
      "updated_at": "2013-09-30T13: 46: 02Z"
    },
    "archived": false,
    "avatar_url": "http://example.com/uploads/project/avatar/4/uploads/avatar.png",
    "shared_runners_enabled": true,
    "forks_count": 0,
    "star_count": 0,
    "runners_token": "b8547b1dc37721d05889db52fa2f02",
    "public_builds": true
  },
  {
    "id": 6,
    "description": null,
    "default_branch": "master",
    "public": false,
    "visibility_level": 0,
    "ssh_url_to_repo": "git@example.com:brightbox/puppet.git",
    "http_url_to_repo": "http://example.com/brightbox/puppet.git",
    "web_url": "http://example.com/brightbox/puppet",
    "tag_list": [
      "example",
      "puppet"
    ],
    "owner": {
      "id": 4,
      "name": "Brightbox",
      "created_at": "2013-09-30T13:46:02Z"
    },
    "name": "Puppet",
    "name_with_namespace": "Brightbox / Puppet",
    "path": "puppet",
    "path_with_namespace": "brightbox/puppet",
    "issues_enabled": true,
    "open_issues_count": 1,
    "merge_requests_enabled": true,
    "builds_enabled": true,
    "wiki_enabled": true,
    "snippets_enabled": false,
    "created_at": "2013-09-30T13:46:02Z",
    "last_activity_at": "2013-09-30T13:46:02Z",
    "creator_id": 3,
    "namespace": {
      "created_at": "2013-09-30T13:46:02Z",
      "description": "",
      "id": 4,
      "name": "Brightbox",
      "owner_id": 1,
      "path": "brightbox",
      "updated_at": "2013-09-30T13:46:02Z"
    },
    "permissions": {
      "project_access": {
        "access_level": 10,
        "notification_level": 3
      },
      "group_access": {
        "access_level": 50,
        "notification_level": 3
      }
    },
    "archived": false,
    "avatar_url": null,
    "shared_runners_enabled": true,
    "forks_count": 0,
    "star_count": 0,
    "runners_token": "b8547b1dc37721d05889db52fa2f02",
    "public_builds": true
  }
]

The id is what’s important here so make a note of the id for the project that will be used.

To then download the repository pass the id to /projects/:id so /projects/6 followed by your token:

https://gitlab.com/api/v3/projects/6/repository/archive?private_token=yourtoken

Now to use this in a php script:

Set a name for the local tar.gz file to be used, I use temp as it will be deleted once finished.

Next use file_get_contents to download the file from the api and save it to $name notice the .tar.gz on the end.

//set file name
$name = 'temp';

//download contents file the above filename with .tar.gz appended. 
//Remember to replace your id and token
file_put_contents($name.'.tar.gz', fopen("https://gitlab.com/api/v3/projects/id/repository/archive?private_token=your_token_here", 'r'));

Next unpack the .tar.gz file this creates a .tar file so unpack that too.

// decompress from gz
$p = new \PharData($zipname.'.tar.gz');
$p->decompress(); // creates files.tar

// unarchive from the tar
$phar = new \PharData($zipname.'.tar');
$phar->extractTo('folder/to/extract/to/', null, true); // extract all files, and overwrite

The packed files are no longer needed to delete them.

//delete temp.tar.gz
if (file_exists($zipname.'.tar.gz')) {
    unlink($zipname.'.tar.gz');
}

//delete temp.tar
if (file_exists($zipname.'.tar')) {
    unlink($zipname.'.tar');
}

This leaves a folder with a name similar to this: projectname-master- followed by a long string, to have the folder only containing the fodler name ie projectname:

$dirs = glob('folder/files/extracted/to/*');
//loop through them
foreach ($dirs as $dir) {
    //if is a directory
    if (is_dir($dir)) {
        //explode the name where there is a -
        $parts = explode('-', $dir);
        //if the name does not exist ie there is no projectname only projectname-master-3r3rr4
        if (!file_exists($parts[0])) {
            //rename to lose everything after and including the -
            rename($dir, $parts[0]);
        }
    }
}

Putting it all together:

//set file name
$name = 'temp';

//download contents file the above filename with .tar.gz appended. 
//Remember to replace your id and token
file_put_contents($name.'.tar.gz', fopen("https://gitlab.com/api/v3/projects/id/repository/archive?private_token=your_token_here", 'r'));

// decompress from gz
$p = new \PharData($zipname.'.tar.gz');
$p->decompress(); // creates files.tar

// unarchive from the tar
$phar = new \PharData($zipname.'.tar');
$phar->extractTo('folder/to/extract/to/', null, true); // extract all files, and overwrite

//delete temp.tar.gz
if (file_exists($zipname.'.tar.gz')) {
    unlink($zipname.'.tar.gz');
}

//delete temp.tar
if (file_exists($zipname.'.tar')) {
    unlink($zipname.'.tar');
}

$dirs = glob('folder/files/extracted/to/*');
//loop through them
foreach ($dirs as $dir) {
    //if is a directory
    if (is_dir($dir)) {
        //explode the name where there is a -
        $parts = explode('-', $dir);
        //if the name does not exist ie there is no projectname only projectname-master-3r3rr4
        if (!file_exists($parts[0])) {
            //rename to lose everything after and including the -
            rename($dir, $parts[0]);
        }
    }
}

 

Laravel Modules Your Logo Your Logo Your Logo

Become a sponsor

Help support the blog so that I can continue creating new content!

Sponsor

My Latest Book

Modular Laravel Book - Laravel: The Modular way

Learn how to build modular applications with Laravel Find out more

Subscribe to my newsletter

Subscribe and get my books and product announcements.

Learn Laravel with Laracasts

Faster Laravel Hosting

© 2006 - 2024 DC Blog. All code MIT license. All rights reserved.