Black Friday sale is now on! 50% off Laravel: The Modular Way. Learn more

Gitlab download private repository as tar.gz with php

David Carr

PHP & MySQL Tutorials Development

  • Putting it all together:
  • Gitlab is a great alternative to Github or BitBucket, they offer free private and public repositories which is great when working with a team, I’ve recently started using Gitlab for a private project, where I need to download private repositories to the project server and extract their contents.

    Github offers a great API http://doc.gitlab.com/ee/api/

    To get started with their API you need a private token. Your token can be found inside your account page https://gitlab.com/profile/account

    All API requests require authentication. You need to pass a private_token parameter via query string or header. If passed as a header, the header name must be PRIVATE-TOKEN (uppercase and with a dash instead of an underscore). You can find or reset your private token in your account page (/profile/account).

    To use the API the token must be passed in all requests, for get requests its as simple as ?private_token=yourtoken

    In order to download the repository it’s id is needed. A list of repositories can be seen by going to 

    https://gitlab.com/api/v3/projects?private_token=yourtoken 

    Other options can be passed such as looking at private repositories only and ordering them by name: 

    https://gitlab.com/api/v3/projects?private_token=yourtoken&visibility=private&order_by=name 

    This will give you a list of projects:

    [
      {
        "id": 4,
        "description": null,
        "default_branch": "master",
        "public": false,
        "visibility_level": 0,
        "ssh_url_to_repo": "git@example.com:diaspora/diaspora-client.git",
        "http_url_to_repo": "http://example.com/diaspora/diaspora-client.git",
        "web_url": "http://example.com/diaspora/diaspora-client",
        "tag_list": [
          "example",
          "disapora client"
        ],
        "owner": {
          "id": 3,
          "name": "Diaspora",
          "created_at": "2013-09-30T13: 46: 02Z"
        },
        "name": "Diaspora Client",
        "name_with_namespace": "Diaspora / Diaspora Client",
        "path": "diaspora-client",
        "path_with_namespace": "diaspora/diaspora-client",
        "issues_enabled": true,
        "open_issues_count": 1,
        "merge_requests_enabled": true,
        "builds_enabled": true,
        "wiki_enabled": true,
        "snippets_enabled": false,
        "created_at": "2013-09-30T13: 46: 02Z",
        "last_activity_at": "2013-09-30T13: 46: 02Z",
        "creator_id": 3,
        "namespace": {
          "created_at": "2013-09-30T13: 46: 02Z",
          "description": "",
          "id": 3,
          "name": "Diaspora",
          "owner_id": 1,
          "path": "diaspora",
          "updated_at": "2013-09-30T13: 46: 02Z"
        },
        "archived": false,
        "avatar_url": "http://example.com/uploads/project/avatar/4/uploads/avatar.png",
        "shared_runners_enabled": true,
        "forks_count": 0,
        "star_count": 0,
        "runners_token": "b8547b1dc37721d05889db52fa2f02",
        "public_builds": true
      },
      {
        "id": 6,
        "description": null,
        "default_branch": "master",
        "public": false,
        "visibility_level": 0,
        "ssh_url_to_repo": "git@example.com:brightbox/puppet.git",
        "http_url_to_repo": "http://example.com/brightbox/puppet.git",
        "web_url": "http://example.com/brightbox/puppet",
        "tag_list": [
          "example",
          "puppet"
        ],
        "owner": {
          "id": 4,
          "name": "Brightbox",
          "created_at": "2013-09-30T13:46:02Z"
        },
        "name": "Puppet",
        "name_with_namespace": "Brightbox / Puppet",
        "path": "puppet",
        "path_with_namespace": "brightbox/puppet",
        "issues_enabled": true,
        "open_issues_count": 1,
        "merge_requests_enabled": true,
        "builds_enabled": true,
        "wiki_enabled": true,
        "snippets_enabled": false,
        "created_at": "2013-09-30T13:46:02Z",
        "last_activity_at": "2013-09-30T13:46:02Z",
        "creator_id": 3,
        "namespace": {
          "created_at": "2013-09-30T13:46:02Z",
          "description": "",
          "id": 4,
          "name": "Brightbox",
          "owner_id": 1,
          "path": "brightbox",
          "updated_at": "2013-09-30T13:46:02Z"
        },
        "permissions": {
          "project_access": {
            "access_level": 10,
            "notification_level": 3
          },
          "group_access": {
            "access_level": 50,
            "notification_level": 3
          }
        },
        "archived": false,
        "avatar_url": null,
        "shared_runners_enabled": true,
        "forks_count": 0,
        "star_count": 0,
        "runners_token": "b8547b1dc37721d05889db52fa2f02",
        "public_builds": true
      }
    ]

    The id is what’s important here so make a note of the id for the project that will be used.

    To then download the repository pass the id to /projects/:id so /projects/6 followed by your token:

    https://gitlab.com/api/v3/projects/6/repository/archive?private_token=yourtoken

    Now to use this in a php script:

    Set a name for the local tar.gz file to be used, I use temp as it will be deleted once finished.

    Next use file_get_contents to download the file from the api and save it to $name notice the .tar.gz on the end.

    //set file name
    $name = 'temp';
    
    //download contents file the above filename with .tar.gz appended. 
    //Remember to replace your id and token
    file_put_contents($name.'.tar.gz', fopen("https://gitlab.com/api/v3/projects/id/repository/archive?private_token=your_token_here", 'r'));

    Next unpack the .tar.gz file this creates a .tar file so unpack that too.

    // decompress from gz
    $p = new \PharData($zipname.'.tar.gz');
    $p->decompress(); // creates files.tar
    
    // unarchive from the tar
    $phar = new \PharData($zipname.'.tar');
    $phar->extractTo('folder/to/extract/to/', null, true); // extract all files, and overwrite

    The packed files are no longer needed to delete them.

    //delete temp.tar.gz
    if (file_exists($zipname.'.tar.gz')) {
        unlink($zipname.'.tar.gz');
    }
    
    //delete temp.tar
    if (file_exists($zipname.'.tar')) {
        unlink($zipname.'.tar');
    }

    This leaves a folder with a name similar to this: projectname-master- followed by a long string, to have the folder only containing the fodler name ie projectname:

    $dirs = glob('folder/files/extracted/to/*');
    //loop through them
    foreach ($dirs as $dir) {
        //if is a directory
        if (is_dir($dir)) {
            //explode the name where there is a -
            $parts = explode('-', $dir);
            //if the name does not exist ie there is no projectname only projectname-master-3r3rr4
            if (!file_exists($parts[0])) {
                //rename to lose everything after and including the -
                rename($dir, $parts[0]);
            }
        }
    }

    Putting it all together:

    //set file name
    $name = 'temp';
    
    //download contents file the above filename with .tar.gz appended. 
    //Remember to replace your id and token
    file_put_contents($name.'.tar.gz', fopen("https://gitlab.com/api/v3/projects/id/repository/archive?private_token=your_token_here", 'r'));
    
    // decompress from gz
    $p = new \PharData($zipname.'.tar.gz');
    $p->decompress(); // creates files.tar
    
    // unarchive from the tar
    $phar = new \PharData($zipname.'.tar');
    $phar->extractTo('folder/to/extract/to/', null, true); // extract all files, and overwrite
    
    //delete temp.tar.gz
    if (file_exists($zipname.'.tar.gz')) {
        unlink($zipname.'.tar.gz');
    }
    
    //delete temp.tar
    if (file_exists($zipname.'.tar')) {
        unlink($zipname.'.tar');
    }
    
    $dirs = glob('folder/files/extracted/to/*');
    //loop through them
    foreach ($dirs as $dir) {
        //if is a directory
        if (is_dir($dir)) {
            //explode the name where there is a -
            $parts = explode('-', $dir);
            //if the name does not exist ie there is no projectname only projectname-master-3r3rr4
            if (!file_exists($parts[0])) {
                //rename to lose everything after and including the -
                rename($dir, $parts[0]);
            }
        }
    }

     

    Laravel Modules Book by David Carr

    Help support the blog so that I can continue creating new content!

    Subscribe to my newsletter

    Subscribe and get my books and product announcements.

    © 2009 - 2022 DC Blog. All code MIT license. All rights reserved.