Ansible Migrate GIT repos

Summary

I recently had to figure out how to automate a migration of a dev git repo to a qa git repo. For security reasons, the repositories had to be kept separate, using one repo with branches for environments was immediately ruled out. The challenge then is to maintain 2 seperate git repositories, build an automated sequence that would allow for content from one repo to be migrated into the other.

The Solution

Cool… I will automate this with Ansible. I began by looking into the Ansible git module, the module does half of what I am trying to acheive, it allows for the automation of clone/checkout of specific branches of a git repo.

I broke the solution down into a number of smaller steps, first of all I will use the git module to check out the source branch. Secondly the git module will be used to checkout the destination repo and create a new branch with a meaningful title and a timestamp appended to the end. This ensures that imported data doesn’t break anything on the destination repo and allows maintainers to review and decide if they want to merge the code into other branches. Lastly I will sync the files from src to destination on the local copy located on the Ansible server, the changes will be added, committed and pushed to the new branch on the destination repo.

- name: migrate repo to dest
  hosts: localhost
  connection: local
  tasks:
  - name: get repo name from URL
    set_fact:
      repo_name:  "{{ src_repo | regex_replace('.+\\/(.+)\\.git', '\\1') }}"
  - debug:
      var: repo_name
  - name: set branch name
    set_fact:
      branch_name: "dev_repo_import_{{ ansible_date_time.iso8601_basic_short }}"
  - name: generate timestamp
    debug:
      var: branch_name
  - name: get a copy of the source repo
    git:
      repo: "{{ src_repo }}"
      dest: "/tmp/{{ repo_name }}"
      clone: yes
      version: "{{ src_branch }}"
  - name: checkout dest repo
    git:
      repo: "{{ dest_repo }}"
      dest: "/tmp/dest_{{ repo_name }}"
      clone: yes
  - name: switch to new branch
    shell: "git --git-dir=/tmp/dest_{{ repo_name }}/.git checkout -b {{ branch_name }}"
  - name: check if env_vars,yml exists
    stat:
     path: "/tmp/{{ repo_name }}/vars/env_vars.yml"
    register: env_vars_file
  - name: update environment variable to QA
    replace:
      path: "/tmp/{{ repo_name }}/vars/env_vars.yml"
      regexp: '^environment: DEV$'
      replace: 'environment: QA'
    when: env_vars_file.stat.exists
  - name: copy files from source to dest
    synchronize:
      src: "/tmp/{{ repo_name }}/"
      dest: "/tmp/dest_{{ repo_name }}"
      rsync_opts:
        - "--exclude=.git*"
  - name: add files to local repo
    shell: "cd /tmp/dest_{{ repo_name }} && git add ."
  - name: commit new files
    shell: "git --git-dir=/tmp/dest_{{ repo_name }}/.git commit -m 'importing from MSE repo'"
  - name: push changes
    shell: "git --git-dir=/tmp/dest_{{ repo_name }}/.git push -u origin {{ branch_name }}"
  - name: clean up source repo in /tmp/src
    file:
      path: "/tmp/{{ repo_name }}"
      state: absent
  - name: clean up dest repo in //tmp/dest
    file:
      path: "/tmp/dest_{{ repo_name }}"
      state: absent

I had to use the shell module to complete the add, push, commit as the current version of the git module simply doesn’t provide these features.

In my example my repository contains a vars/env_vars.yml file, I use this within my different environments to set conditionals such as if environment == “DEV” then URL = http://dev.example.com. Using these types of variables can make it a bit easier when managing playbooks in multiple enviornments.